How-are-we-going-

How are we going?

What are our outcomes?

Updated outcome data for Dr Vanessa Spiller at JumpStart Psychology 2009 - 2015

JumpStart Psychology embraces a Feedback Informed Treatment (FIT) approach which means that we use data to tell us how each of our clients is progressing. However, we also keep statistics on what outcomes we are achieving with our clients overall. We now have a database of over 600 episodes of service for Dr Spiller and thought it was time for an up-date!

What data is included? What data is excluded?

Since the practice started in 2009, the Outcome Rating Scale (ORS) has been offered to all clients of Dr Spiller (except 1) who have attended the service representing 626 episodes of treatment! (as of September 2015).

In analysing the effectiveness of services, data from several groups has been excluded because these groups have different data patterns to adults coming to counselling for the first time. For example, returning clients typically come in to counselling with much lower levels of distress than those attending for the first time (meaning they come in doing better). These data exclusions are consistent with other psychologists who publish their outcome data and include:

•

children (under the age of 16 years old) (59 episodes)

•

returning clients (151 episodes)

•

those who only attend for a single session (40 episodes)

•

clients who were unable, abstained or did not complete measures due to technical issue (3 episodes)

How do these statistics compare to other psychologists/counsellors?

Unfortunately, most psychologists/counsellors don’t keep statistics about their effectiveness or client outcomes however some data does exist that may be useful to consider. Dr Jason Seidel provides a good comparison of outcome data compared to research studies looking at the effectiveness of therapy (http://www.thecolorado center.com/outcomes.html). Dr Aaron Frost from Benchmark Psychology (another Brisbane based Psychology practice) also measures and makes his effectiveness data publicly available http://www.aaronfrost.com.au/home/research/.

Want to read more?

Outcome Rating Scale (ORS):

Miller, S.D., Duncan, B.L., Brown, J., Sparks, J.A., & Claud, D.A. (2003). The Outcome Rating Scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. Journal of Brief Therapy, 2(2), 91-100.

Client Dropout Rates:

Wierzbicki, M. & Pekarik, G. (1993). A Meta-Analysis of Psychotherapy Dropout. Professional Psychology: Research and Practice, 24(2), 190-195.

Client Deterioration Rates:

Harmon, S.C., Lambert, M.J., Smart, D.M., Hawkins, E., Nielsen, S.L., Slade, K., & Lutz, W. (2007). Enhancing outcome for potential treatment failures: Therapist-client feedback and clinical support tools. Psychotherapy Research, 17(4), 379-392.

Benefits of a FIT approach:

Bertonlino, B., Bargmann, S., & Miller, S. (2011). ICCE Manuals on Feedback (FIT). Manual 1. What Works in Therapy: A Primer. International Centre for Clinical Excellence.

Bohanske, B. & Franczak, M. (2009). Transforming public behavioral health care: A case example of consumer directed services, recovery, and the common factors. In B. Duncan, S.

Miller, S.D. (2010) Psychometrics of the ORS and SRS. Results from RCT’s and meta-analyses of routine outcome monitoring and feedback: The available evidence. http://www.scottmiller.com /?q=blog/1&page=2

Miller, B. Wampold, & M. Hubble. (Eds.) (2009). The Heart and Soul of Change (2nd Ed.). Washington, D.C.: APA Press.

Research studies looking at the effectiveness of therapy:

Anker, M.G., Duncan, B.L., & Sparks, J.A. (2009). Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77(4), 693-704.

Asay, T.P., Lambert, M.J., Gregersen, A.T., & Goates, M.K. (2002). Using patient-focused research in evaluating treatment outcome in private practice. Journal of Clinical Psychology, 58(10), 1213-1225.

Barkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans, C., Benson, L., Connell, J., & Audin, K. (2001). Service profiling and outcomes benchmarking using the CORE-OM: Toward practice-based evidence in the psychological therapies. Journal of Consulting and Clinical Psychology, 69(2), 184-196.

Brown, G.S., Lambert, M.J., Jones, E.R., & Minami, T. (2005). Identifying highly effective psychotherapists in a managed care environment. American Journal of Managed Care, 11(8), 513-520.

Hansen, N.B., Lambert, M.J., & Forman, E.M. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9(3), 329-343.

Hawkins, E.J., Lambert, M.J., Vermeersch, D.A., Slade, K.L., & Tuttle, K.C. (2004). The therapeutic effects of providing patient progress information to therapists and patients. Psychotherapy Research, 14(3), 308-327.

The client’s referred to JumpStart Psychology typically arrive at counselling with a high level of distress (this is defined as an ORS score of less than 25). 88% of new adult clients referred have a score in this range.

The outcome data presented below is based on 376 episodes of service (because of the exclusions listed above). 60% of Dr Spiller’s clients are female and 40% are male. In addition, 56% of clients are bulk-billed while the remainder are fee paying.

87% of clients experience some positive change in counselling, that is, their ORS scores increase from the start of counselling to the end. However, positive change can be due to chance and when this is taken into account, 75% of the JumpStart clients experience reliable improvement (more than 5 points of positive change) when it is measured on the ORS. Individuals are considered “recovered” when they have experienced a reliable amount of change and when they have moved from being clinically distressed at the start of counselling to no longer clinically distressed at the end of counselling. 53% of clients from JumpStart Psychology would be considered “recovered” using this definition at the conclusion of therapy.

In looking at counselling outcomes it is also important to consider the number clients who don’t improve in therapy. Research would indicate that 8 – 10% of clients get reliably worse in the course of counselling. 4% of clients show reliable decrease in the course of counselling at JumpStart Psychology.

At JumpStart Psychology, 94% of Dr Spillers new clients attend more than one session and while the number of sessions attended varies considerably, the average number of sessions attended is 8. The drop-out rate as calculated by the FITOUTCOMES program is currently 10.1% for Dr Spiller’s clients.

Please note that counselling is a very individual experience and so individual outcomes vary considerably.

Based on these figures, the outcomes achieved so far at JumpStart Psychology are equal to or higher than many of those achieved in several well-recognised research studies looking at the overall effectiveness of therapy (Raw pre-post Effect Size for Dr Vanessa Spiller is 1.5. This would be considered a “strong” effect size - please contact Dr Spiller for further information regarding the calculation of this data).

What does an ES (Effect size) of 1.5 actually mean? It means that 93% of the treatment group will be above the mean of the control group (Cohen's U3) and that there is an 86% chance that a person picked at random from the treatment group will have a higher score than a person picked at random from the control group (probability of superiority) (http://rpsychologist.com/d3/cohend/)