As public and professional attention to outcome and evaluation research grows, focus commonly remains centered on the question, “Does this treatment work or not?” Consequently, much less emphasis is given to what exactly it means for a treatment to be effective. This article examines 5 issues relevant to whether an intervention is deemed successful or not: (a) Sponsorship: Who generates the empirical evidence? (b) Sensitivity: How deep does the evidence gathering go? (c) Scope: Over what time period does the evidence span? (d) Source: According to which data is a determination reached? (e) Fairness: How seriously are negative cases examined? After exploring each issue, larger implications for youth treatment are discussed. Ultimately, we propose that current trends warrant more thoughtful deliberation and education among citizens and practitioners about treatment outcomes generally.
Get full access to this article
View all access options for this article.
References
1.
AdamsK. B., MattoH. C., & LeCroyC. W. (2009). Limitations of evidence-based practice for social work education: Unpacking the complexity. Journal of Social Work Education, 45(2), 1–22.
2.
AndersenS. L., & NavaltaC. P. (2004). Altering the course of neurodevelopment: A framework for understanding the enduring effects of psychotropic drugs. International Journal of Developmental Neuroscience, 22, 423–440.
3.
AntonuccioD. O., BurnsD. D., & DantonW. G. (2002). Antidepressants: A triumph of marketing over science? Prevention & Treatment, 5(1), article 25.
4.
AntonuccioD. O., DantonW. O., & McClanahanT. M. (2003). Psychology in the prescription era: Building a firewall between marketing and science. American Psychologist, 58, 1028–1043.
5.
BehrensE. N., & SatterfieldK. M. (2007). Longitudinal family and academic outcomes in residential programs: How students function in two important areas of their lives. Journal of Therapeutic Schools & Programs, 2(1), 95–105.
6.
BerryM., BrandonM., ChaskinR., FernandezE., GrietensH., LightburnA., … ZeiraA. (2006). Identifying sensitive outcomes of interventions in community-based centres. International Journal of Child & Family Welfare, 9, 1–10.
7.
BickmanL., LambertE. W., AndradeA. R., & PenalozaR. (2000). The Fort Bragg continuum of care for children and adolescents: Mental health outcomes over 5 years. Journal of Consulting and Clinical Psychology, 68(4), 710–716.
8.
BodenheimerT. (2000). Uneasy alliance—Clinical investigators and the pharmaceutical industry. New England Journal of Medicine, 342(20), 1539–1544.
9.
BridgeJ. A., IyengarS., SalaryC. B., BarbeR. P., BirmaherB., PincusH. A., … BrentD. A. (2007). Clinical response and risk for reported suicidal ideation and suicide attempts in pediatric antidepressant treatment: A meta-analysis of randomized controlled trials. JAMA, 297(15), 1683–1696.
10.
ButlerS. M. (2006, June 30). Implementing and evaluating multi-systemic therapy with families of persistent young offenders in the UK: Initial reflections and observations from a longitudinal trial. Paper presented at Jack Tizard Memorial Lecture and Day Conference on Evidence-Based Psychotherapies in Child and Adolescent Mental Health Practice, Royal College of Surgeons, London, UK.
11.
CairnsB. (2004). Fostering attachments: Long-term outcomes in family group care. London, UK: BAAF.
12.
CampbellE. G., WeissmanJ. S., EhringhausS., RaoS. R., MoyB., FeibelmannS., & GooldS. D. (2007). Institutional academic-industry relationships. JAMA, 298, 1779–1786.
13.
CervenkaK. A., DemboR., & BrownC. H. (1996). A family empowerment intervention for families of juvenile offenders. Aggression and Violent Behavior, 1(3), 205–216.
CohenD., HughesS., & JacobsD. J. (2009). The deficiencies of drug treatment research: The case of Strattera. In TimimiS. & LeoJ. (Eds.), Rethinking ADHD: From brain to culture (pp. 313–333). New York and London, UK: Palgrave Macmillan.
16.
CrawfordM. J., WeaverT., RutterD., SenskyT., & TyrerP. (2002). Evaluating new treatments in psychiatry: The potential value of combining qualitative and quantitative research methods. International Review of Psychiatry, 14(1), 6–11.
17.
CreswellJ. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches. New York: SAGE.
18.
DantzigT. (1930). Number: The language of science. New York: Macmillan.
19.
DelateT., GelenbergA. J., SimmonsV. A., & MotheralB. (2004). Trends in the use of antidepressants in a national sample of commercially insured pediatric patients, 1998 to 2002. Journal of Psychiatric Services, 55, 387–391.
20.
DenzinN. K., & LincolnY. (2005). The SAGE handbook of qualitative research (3rd ed.). London, UK: SAGE.
21.
DriskoJ. (1997). Strengthening qualitative studies and reports: Standards to promote academic integrity. Journal of Social Work Education, 33(1), 185–197.
22.
DriskoJ. (2008). How is qualitative research taught at the master's level?Journal of Social Work Education, 44(1), 85–101.
23.
EdwardsA., & ElwynG (2009). Shared decision-making in health care: Achieving evidence-based patient choice (2nd ed.). New York: Oxford University Press.
24.
EmslieG. J., RushA. J., WeinbergW. A., KowatchR. A., HughesC. W., CarmodyT., & RintelmannJ. (1997). A double-blind, randomized, placebo-controlled trial of fluoxetine in children and adolescents with depression. Archives of General Psychiatry, 54(11), 1031–1037.
25.
FlewA. (1995). Thinking about social thinking. Amherst, NY: Prometheus.
FrankelR. (1999). Standards of qualitative research. In CrabtreeB. & MillerW. (Eds.), Doing qualitative research (2nd ed.; pp. 333–346). Los Angeles, CA: SAGE.
28.
GambrillE. (1999). Evidence-based practice: An alternative to authority-based practice. Families in Society: The Journal of Contemporary Human Services, 80(4), 341–350.
29.
GambrillE. (2001). Social work: An authority-based profession. Research on Social Work Practice, 11(2), 166–175.
30.
GambrillE. (2010). Evidence-informed practice: Antidote to propaganda in the helping professions?Research on Social Work Practice, 20, 302–320.
31.
GilgunJ. (1994). A case for case studies in social work research. Social Work, 39, 371–380.
32.
GoergeR., WulczynF., & FanshelD. (1994). A foster care research agenda for the ’90s. Child Welfare, 73(5), 525–549.
33.
GordonJ. S. (2008). Unstuck: Your guide to the seven-stage journey out of depression. New York: Penguin Press.
34.
HarrisG. (2009, October 22). Drug makers are advocacy group's biggest donors. The New York Times, A23.
35.
HarrisG., & CareyB. (2008, June 8). Researchers fail to reveal full drug pay. The New York Times, A1.
36.
HealyD. (2006). Manufacturing consensus. Culture, medicine, and psychiatry, 30(2), 135–156.
37.
HenggelerS. W., RowlandM. D., RandallJ., WardD. M., PickrelS. G., CunninghamP. B., … SantosA. B. (1999). Home-based multisystemic therapy as an alternative to the hospitalization of youths in psychiatric crisis: Clinical outcomes. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 1331–1339.
38.
HenggelerS. W., & SchoenwaldK. (1994). Boot camps for juvenile offenders: Just say no. Journal of Child and Family Studies, 3, 243–248.
39.
HessJ. Z., BjorklundE., PreeceN., & DraperS. (under review). Poison apples, big bad wolves and other ‘happy ending’ spoilers: Challenges to enduring change following youth residential treatment.
40.
Journal of Therapeutic Schools and Programs. HinshawS. (2007). Moderators and mediators of treatment outcome for youth with ADHD: Understanding for whom and how interventions work. Journal of Pediatric Psychology, 32(6), 664–675.
41.
HoagwoodK. E. (2005). Family-based services in children's mental health: A research review and synthesis. Journal of Child Psychology and Psychiatry, 46(7), 690–713.
42.
HughesS., & CohenD (2009). A systematic review of long-term studies of drug treated and non-drug treated depression. Journal of Affective Disorders, 118(1), 9–18.
43.
HughesS., & CohenD (2010). Understanding the assessment of psychotropic drug harms in clinical trials to improve social workers’ role in medication monitoring. Social Work, 55(2), 105–111.
44.
JacobsD. H. (1999). A close and critical examination of how psychopharmacotherapy research is conducted. Journal of Mind and Behavior, 20(3), 311–349.
45.
JureidiniJ. N., & JureidiniE. S. (2008, February). 226 papers/letters that cite Keller et al.'s report of GlaxoSmithkline's Study 329 of antidepressants for adolescents. Healthy Skepticism International News. Retrieved June 1, 2010, from http://www.healthyskepticism.org/global/news/extra/keller_citations_update
46.
JureidiniJ. N., McHenryL. B., & MansfieldP. R. (2008). Clinical trial and drug promotion: Selective reporting of Study 329. International Journal of Risk and Safety in Medicine, 20, 73–81.
47.
KarpD. A. (1997). Speaking of sadness: Depression, disconnection, and the meanings of illness. London: Oxford University.
48.
KarpD. A. (2006). Is it me or my meds? Living with antidepressants. Cambridge, MA: Harvard University Press.
49.
KazdinA. E., & NockM. K. (2003). Delineating mechanisms of change in child and adolescent therapy: Methodological issues and research recommendations. Journal of Child Psychology and Psychiatry 44, 1116-1129.
50.
KellerM. B., RyanN. D., StroberR. G., KleinS. P., KutcherB., BirmaherB., … McCaffertyJ. P. (2001). Efficacy of paroxetine in the treatment of adolescent major depression: A randomized, controlled trial. Journal of the American Academy of Adolescent and Child Psychiatry, 40, 762–772.
51.
KnorthE. J., HarderA. T., ZandbergT., & KendrickA. J. (2008). Under one roof: A review and selective meta-analysis on the outcomes of residential child and youth care. Children and Youth Services Review, 30(2), 123–140.
52.
KnudsenP., HansenE. H., TraulsenJ. M., & EskildsenK. (2002). Changes in selfconcept while using SSRI antidepressants. Qualitative Health Research, 12(7), 932–944.
53.
LacasseJ. R. (2005). Consumer advertising of psychiatric medications biases the public against non-pharmacological treatment. Ethical Human Psychology and Psychiatry, 7(3), 175–179.
54.
LacasseJ. R., & GomoryT. (2003). Is graduate social work education promoting a critical approach to mental health practice?Journal of Social Work Education, 39(3), 383–408.
55.
LacasseJ. R., & LeoJ. (2010). Ghostwriting at elite academic medical centers in the United States. PLoS Medicine, 7(2), e1000230. doi: 10.1371/journal.pmed.1000230
56.
LeoJ., & LacasseJ. R. (2008). The media and the chemical imbalance theory of depression. Society, 45, 35–45.
57.
LewisR. E. (2004). The effectiveness of Families First services: An experimental study. Children and Youth Services Review, 27, 499–509.
58.
LexchinJ., BeroL. A., DjulbegovicB., & ClarkO. (2003). Pharmaceutical industry sponsorship and research outcome and quality: A systematic review. BMJ, 326, 1167–1176.
59.
LietzC. A. (2009). Examining families’ perceptions of intensive in-home services: A mixed method study. Children and Youth Services Review, 31(12), 1337–1345.
60.
LynamD. R., MilichaR., ZimmermanR., NovakS. P., LoganT. K., MartinC., … ClaytonR. (1999). Project DARE: No effects at 10-year follow-up. Journal of Consulting and Clinical Psychology, 67(4), 590–593.
61.
McCardleP., & ChhabraV (2004). The voice of evidence in reading research. In KeoghB. K. (Ed.), The importance of longitudinal research for early intervention practices (pp. 81–102). Baltimore, MD: Paul H. Brookes.
62.
McDonaldT. P., AllenR. I., WesterfeltA., & PiliavinI. (1996). Assessing the longterm effects of foster care: A research synthesis. Washington, DC: CWLA Press.
63.
McKayJ. (2007). So you want to run an outcome study? The challenges to measuring adolescent residential treatment outcomes. Journal of Therapeutic Schools & Programs, 2(1), 63–80.
64.
MolloyD., WoodfieldK., & BaconJ. (2002). Longitudinal qualitative research approaches in evaluation studies (Working Paper No. 7). Department for Work and Pensions, London, UK. Retrieved October 10, 2010, from http://research.dwp.gov.uk/asd/asd5/WP7.pdf
65.
MosesT., & KirkS. A. (2005). Psychosocial side effects of drug treatment of youth. In KirkS. A. (Ed.), Mental disorders in the social environment: Critical perspectives. Foundations of social work knowledge (pp. 385–407). New York: Columbia University Press.
66.
National Registry of Evidence-Based Programs and Practices. (2009). What is evidence-based? U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration (SAMHSA). Retrieved May 5, 2010, from http://www.nrepp.samhsa.gov/about-evidence.asp
67.
PachterW. S., FoxR. E., ZimbardoP., & AntonuccioD. O. (2007). Corporate funding and conflicts of interest: A primer for psychologists. American Psychologist, 62, 1005–1015.
68.
PadgettD. (2008). Qualitative methods in social work research. Los Angeles, CA: SAGE.
69.
PoundP., BrittenN., MorganM., YardleyL., PopeC., Daker-WhiteG., & CampbellR. (2005). Resisting medicines: A synthesis of qualitative studies of medicine taking. Social Science & Medicine, 61, 133–155.
70.
RomynD. M., AllenM. N., BoschmaG., DuncanS. M., EdgecombeN., JensenL. A., … WarnockF. (2003). The notion of evidence in evidence-based practice by the nursing philosophy working group. Journal of Professional Nursing, 19(4), 184–188.
71.
RosackJ. (2005, November 18). FDA panel rejects new rule for psychiatric drug testing. Government New Psychiatric News, American Psychiatric Association, 40(22), 1. Retrieved June 23, 2009, from http://pn.psychiatryonline.org/content/40/22/1.1.full
72.
RuspiniE. (2008). Longitudinal research: An emergent method in the social sciences. In Hesse-BiberS. N. & LeavyP. (Eds.), Handbook of emergent methods (pp. 437–460). New York, NY: Guilford Press.
73.
SaferD. J. (2006). Should selective serotonin reuptake inhibitors be prescribed for children with major depressive and anxiety disorders?Pediatrics, 118(3), 1248–1251.
74.
SimonN. M., SafrenS. A., OttoM. W., SharmaS. G., LankaG. D., & PollackM. H. (2002). Longitudinal outcome with pharmacotherapy in a naturalistic study of panic disorder. Journal of Affective Disorders, 69(1/3), 201–208.
75.
SismondoS. (2008). How pharmaceutical industry funding affects trial outcomes: Causal structures and responses. Social Science and Medicine, 66(9), 1909–1914
76.
SismondoS. (2009). Ghosts in the machine: Publication planning in the medical sciences. Social Studies of Science, 39(2), 171–198.
77.
SlifeB. D., & GanttE. (1999). Methodological pluralism: A framework for psychotherapy research. Journal of Clinical Psychology, 55(12), 1–13.
78.
StoppardJ. M., & McMullenL. M. (Eds.). (2003). Situating sadness: Women and depression in social context. New York, NY: New York University Press.
79.
TanneJ. H. (2008). U.S. psychiatrist steps down after questions about drug company payments. BMJ, 337, a2088.
80.
TurnerE. H., MatthewsA. M., LinardatosE., TellR. A., & RosenthalR. (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine, 358(3), 252–260. doi:10.1056/NEJMsa065779
81.
VenardeD. F. (1999). Medication and meaning: Psychotherapy patients’ subjective experiences of taking selective serotonin reuptake inhibitors (SSRIs) (Unpublished doctoral dissertation). State University of New Jersey, Piscataway.
82.
Verbeek-HeidaP. M., & MathotE. F. (2006). Better safe than sorry—why patients prefer to stop using selective serotonin reuptake inhibitor (SSRI) antidepressants but are afraid to do so: Results of a qualitative study. Chronic Illness, 2(2), 133–142.
83.
WhitakerR. (2010). Anatomy of an epidemic: Magic bullets, psychiatric drugs, and the astonishing rise of mental illness in America. New York: Crown.
84.
WilliamsM., TeasdaleJ., ZindelS., & Kabat-ZinnJ. (2007). The mindful way through depression: Freeing yourself from chronic unhappiness. New York, NY: Guilford Press.
85.
WhitemoreE., FordM., & SackW. H. (2003). Effectiveness of day treatment with proctor care for young children: A four-year follow-up. Journal of Community Psychology, 31(5), 459–468.
