Clinical Effectiveness of Using an Integrated Model to Treat Depressive Symptoms in Veterans Affairs Primary Care Clinics and Its Impact on Health Care Utilization

Clinical Effectiveness of Using an Integrated Model to Treat Depressive Symptoms in Veterans Affairs Primary Care Clinics and Its Impact on Health Care Utilization

Anna G. Engel, MD; Loretta S. Malta, PhD; Cheryl A. Davies, PhD; and Margaret Momot Baker, PhD

Objective: To determine if veterans treated in an integrated mental health program within a Veterans Affairs (VA) primary care clinic sustained long-term improvement in depressive symptoms and changed their use of health care.

Method: In this pilot program, 72 veterans were offered short-term treatment for depressive symptoms by a colocated psychiatrist who was integrated into a VA primary care team (October 1, 1997, through September 30, 1999). Patients were assessed initially and at their final session using the Hamilton Depression Rating Scale. Veterans who completed treatment were referred back to their primary care provider or to specialty mental health services. Patients were contacted and invited to be reevaluated 3 to 5 years later using the same measure (December 1, 2001, through November 30, 2002). Health care utilization data were collected for 1 year preintervention and 2 years postintervention. Outcomes for treatment completers were compared to outcomes for those who declined or dropped out of treatment.

Results: Of 48 patients who agreed to participate in the study, 27 completed treatment and showed a significant decline in symptoms from pretreatment to follow-up (P = .008) compared to 16 noncompleters, as well as a moderate-to-large between-group effect size (d = 0.78) and trends for higher remission and response rates. Completers ranked significantly higher in the number of antidepressant prescriptions filled before (P = .002) and after treatment (P = .001) and in the number of medical visits postintervention (year 1: P = .021; year 2: P = .023), without an associated cost increase.

Conclusions: Colocated mental health care integrated into VA primary care is associated with sustained improvement of depressive symptoms in a heterogeneous patient population with a high incidence of psychiatric comorbidities. This finding compares favorably with the results of earlier controlled clinical trials and suggests a potential effect on health care utilization.

Prim Care Companion CNS Disord 2011;13(4): doi:10.4088/PCC.10m01096

© Copyright 2011 Physicians Postgraduate Press, Inc.

Submitted: October 7, 2010; accepted January 31, 2011.

Published online: August 18, 2011 (doi:10.4088/PCC.10m01096).

Corresponding author: Anna G. Engel, MD, Boston VA Healthcare System, Department of Psychiatry, 1400 VFW Pkwy, West Roxbury, MA 02132 ([email protected]).

Depressive symptoms alone, defined as having 2 of the 8 criteria required for a diagnosis of depression, are associated with impaired social functioning and increased health care utilization.1 Primary care providers manage the treatment of depression in one-third to one-half of adults under age 65 years and in almost two-thirds of older adults.2 Over the last 20 years, strategies to improve treatment of depression in primary care were developed and tested in randomized, controlled trials. These include use of depression care managers,3 telephone interventions,4 training for primary care providers,5 and models in which a mental health professional is integrated into the medical team in various ways.6 A meta-analysis of the long-term effects of controlled trials of collaborative care demonstrated that patients treated with these approaches maintained statistically significant improvements at 12 and 18 months and at 5 years thus establishing the efficacy of interventions integrated into primary care.7

Prevalence rates for depression in veteran patients of 16.7% to 31%8,9 are higher than those for the general primary care population, which range from 5% to 13%.2 Prevalence of depression among combat veterans entering Veterans Affairs (VA) care from 2002 to 2008 was 17.4%, with 42% of these diagnoses made in primary care.10

Several treatment models that integrate mental health services into the primary care setting were tested in VA and military settings. The White River Junction Vermont VA Medical Center model demonstrated faster and improved care of patients who screened positive for depression.11 A telephone service featuring nurse depression care managers resulted in increases in numbers of veterans receiving antidepressant therapy and mental health services.12 Other interventions with generally encouraging results include depression decision support in primary care,13 application of the Re-Engineering Systems for the Treatment of Depression in Primary Care strategy3 to active military primary care,14 a telemedicine program promoting antidepressant adherence,15 a comparison of a collaborative care model with a standard consult/liaison approach,16 a depression telephone management program,17 and insertion of a clinical nurse specialist into the primary care team.18 The generalizability and practical utility of these interventions may be limited by exclusion of veterans with a wide range of comorbid conditions more typical of VA primary care patients and a lack of data on outcomes beyond 1 year.

The well-established association of depression and depressive symptoms with increased health care costs in the civilian population19,20 is less certain in the veteran population.21 The impact on health care costs of collaborative interventions is mixed.22-25 Significant reductions were demonstrated when such interventions were studied over 2 to 4 years.26,27

To investigate the real-world utility and efficacy of a program providing mental health care colocated in the primary care clinics of an urban VA medical center, we reevaluated consenting patients 2 to 5 years after their initial referrals. We sought to determine late outcomes in those who completed treatment as compared to those who declined treatment or dropped out. In addition, we determined health care resource use and costs in these groups before and following referral.

METHOD

Setting

The study was conducted at the Stratton VA Medical Center, Albany, New York, and approved by the center’s Institutional Review Board. The study psychiatrist (A.G.E.), who conducted all assessments and treatments, was located in the primary care clinics as a member of the primary care team. Patients were treated on site.

Participants

Of 203 veterans referred to the psychiatrist for mental health treatment by their primary care providers from October 1, 1997, through September 30, 1999, 72 received a standardized psychological measure before being offered treatment. These individuals were contacted by phone or letter between December 1, 2001, and November 30, 2002, and invited to participate in a reassessment of their mental health status since last meeting with the team psychiatrist. A total of 48 patients agreed to participate in the study (Figure 1).

Treatment

The primary care psychiatrist provided short-term treatment during a 2-year period (October 1, 1997-September 30, 1999) in a colocated integrated model in close collaboration with the primary care physician. This collaboration consisted of personal contact to discuss patients, phone calls, and notes in the electronic medical record. In some cases, the psychiatrist met the patient in the primary care physician’s office. Treatment consisted of pharmacotherapy and psychotherapeutic intervention (relaxation training, cognitive reframing, problem-solving therapy techniques, and grief counseling).28,29 Patients were offered 12 visits. When the patient and psychiatrist were jointly satisfied with the level of improvement achieved, the veteran was referred back to his/her primary care physician for management. If the patient remained significantly symptomatic, then he/she was offered a referral for specialty mental health care.

Assessments

All assessments were conducted by the study psychiatrist. At the initial assessment, diagnoses were made using DSM-IV criteria, and 1 of the standardized measures described below was administered to determine symptom severity. This measure was repeated at the veteran’s last or next-to-last scheduled session and again at the reevaluation that occurred a mean of 36 (SD = 7.70) months later. Participants also completed a standardized interview about antidepressant adherence and follow through with any mental health referral.

Outcome Measures

The measures used included the 17-item Hamilton Depression Rating Scale (HDRS),30 Beck Depression Inventory,31 Beck Depression Inventory for Primary Care,32 and Beck Anxiety Inventory.33 Since the HDRS was completed by 43 of the 48 patients, this was the only treatment outcome measure used for analysis. Depression symptoms were designated as remitted if HDRS scores were ≤ 7. Treatment response was determined by a reduction in HDRS scores of ≥ 50%. All 48 participants were included in the analyses of adherence and health care utilization.

Adherence was coded as a categorical variable, defined as continuing to follow guidelines for medication maintenance recommended to the patient and his/her primary care provider or changing/ceasing the regimen upon the recommendation of the primary care provider or another psychiatrist. Because of a time delay between improvement in depression and economic outcomes, we collected data regarding filled medication prescriptions, medical visits, and emergency room (ER) visits for the 1-year period before treatment and the 2-year period following the last visit with the study psychiatrist.26,27

Automated pharmacy prescription data for antidepressants were compared with self-report data. Pharmacy data were also collected for benzodiazepines, opioid analgesics, and sedatives/hypnotics. Follow through with referral was based on self-report. Outpatient utilization (number and costs of ER and medical visits) was determined for the year prior and for 2 years following treatment using clinical encounter data. Total specialty medical and primary care visits were combined under the category of general medical visits, and cost was derived from the primary care stop code. ER visit costs were based on billing rates for the ER current procedural terminology codes. Median values of $392.00/visit and $129.00/visit were used for ER and general medical visits, respectively. A chronic disease score was calculated to control for the presence of chronic diseases when examining health care utilization.34,35

Statistical Analyses

Quantitative analyses were conducted in the following outcome domains: (1) change in symptoms of depression at the last visit with the mental health professional and at reevaluation, (2) variables that predicted treatment completion, (3) adherence to treatment regimen and utilization of health care during the first year prior and 2 years following the last visit with the treating psychiatrist, and (4) follow-up on referrals.

As this was an effectiveness study that did not utilize random assignment to treatment groups, in lieu of a control group, we conducted analyses of variance (ANOVAs) to compare change in HDRS scores at posttreatment and at follow-up reevaluation for veterans who completed treatment versus those who declined or did not complete treatment. Following an approach used in other open trials, we calculated treatment effect sizes using Cohen’s formula to permit comparison to those obtained in randomized clinical trials.36,37 Symptom remission, treatment response, and adherence to treatment regimen were coded as categorical variables, and χ2 and Fisher exact tests were conducted to compare these variables in treatment completers versus those who declined or dropped out. Nonparametric tests were used for analyses of health care utilization and medication prescriptions, as these variables were not normally distributed (kurtotic).

RESULTS

Demographics and Study Participation Status

Of 72 eligible patients, 48 (67%) agreed to participate. No significant differences were found in demographic characteristics for patients who participated in the study versus those who did not (P > .10).

As shown in Table 1, the majority of the patients were non-service-connected, white men. Most of the sample was diagnosed with a depressive disorder. Of these, 58% were comorbid for an anxiety disorder and 38% for alcohol/substance use disorders, with 8 patients actively using alcohol or other substances at some time during treatment.

Of the 48 participants, 43 (90%) completed the HDRS, 29 (60%) completed treatment offered in primary care, and 27 of those patients completed a posttreatment HDRS, 15 (31%) dropped out, and 4 (8%) declined treatment. Four participants who commenced treatment received a posttreatment HDRS but were unable to attend the last session. We included them in the group who received a posttreatment measure (n = 31). Patients who completed treatment attended a mean of 7.03 (SD ± 4.23) sessions compared with 3.73 (SD ± 2.65) for those who dropped out. Of the 29 participants who completed treatment in primary care, 13 (45%) were referred to their primary care provider, and the remaining 16 (55%) were referred for specialty mental health services. These patients remained symptomatic from depressive symptoms, suffered comorbid conditions, or experienced persistent life stressors that could trigger relapse.

Treatment Outcomes

Table 2 presents HDRS scores for the 43 participants assessed with this instrument. Analyses were conducted for the 31 participants with posttreatment HDRS data (27 treatment completers, 4 dropouts). For the intent-to-treat sample, we used the last available HDRS score.

Repeated-measures ANOVAs testing change in HDRS scores at posttreatment and follow-up found a significant effect of time (F2,60 = 24.377; P < .001). Univariate repeated-measures ANOVAs found a significant pretreatment to posttreatment decline in HDRS scores and moderate-to-large effect sizes (d) for the sample of 31 participants with posttreatment scores (F1,11 = 8.799; P = .013) and for the intent-to-treat sample (F1,42 = 40.719; P < .001). As shown in Table 2, effect sizes calculated from pretreatment to follow-up HDRS scores were also moderate for both samples. A significant increase in HDRS scores from posttreatment to follow-up for the sample of 31 participants with posttreatment scores (F1,30 = 5.618; P = .024) and for the intent-to-treat sample (F1,42 = 4.331; P = .044) was found using univariate repeated-measures ANOVAs.

Table 3 displays HDRS scores grouped by participation status. Time × group repeated-measures ANOVAs were conducted to compare HDRS scores between treatment completers and the declined treatment/dropped out group. For the sample of 31 participants with posttreatment measures, the ANOVA found a significant main effect of time (F1,29 = 33.088; P < .001), but the time × group interaction was not significant (P > .10). For follow-up scores, the ANOVA found a significant main effect of time (F1,41 = 7.261; P = .010) and a significant time × group interaction (F1,41 = 7.596; P = .009). Treatment completers showed a significantly greater decline in HDRS scores from pretreatment to follow-up. An ANCOVA controlling for pretreatment scores found significantly lower HDRS scores at follow-up for treatment completers (F1,40 = 7.715; P = .008). The between-group effect size for differences in HDRS scores was moderate to large (d = 0.78).

Independent variables for analysis of predictors of treatment completion included age, gender, race, chronic disease score, service-connected disability (coded dichotomously as presence/absence), type of service-connected disability (coded as 0 = none, 1 = physical health, 2 = mental health, 3 = physical and mental health), HDRS score at pretreatment, and diagnosis at pretreatment. The only significant predictor of treatment completion was pretreatment diagnosis. Patients diagnosed with a depressive disorder were significantly more likely to complete treatment (26/37, 70%) versus those who reported symptoms of depression but not the full disorder (3/11, 28%) (Fisher exact test, P = .016).

Table 3 shows treatment responders and remission status at posttreatment and follow-up. At posttreatment, the majority of the 31 participants with HDRS scores had achieved remission and/or a treatment response. A Fisher exact test did not find significant differences in remission or treatment response rates at posttreatment between treatment completers (n = 27) and those who declined/dropped out (n = 4 with data) (P > .10). Of the entire sample of completers and noncompleters, almost half (20/43, 47%) showed essentially little or no change in HDRS scores from pretreatment to follow-up 2 to 5 years later. At follow-up, symptoms of depression remitted in 30% of the sample. For treatment completers, using Fisher exact tests, we found a significantly larger proportion (10/27, 37%) were responders when compared with the decline/dropout group (1/16, 6.25%, P = 0.33) but only a trend for remission (11/27 [40%] versus 2/16 [12.5%], P = .86). Depression recurred in 30% of that group. Slightly less than half of all the participants were depressed at follow-up.

Adherence and Health Care Utilization

Table 4 displays quantitative data on health care utilization and adherence for 45 of the 48 participants. Slightly over half of participants reported they did not adhere to the medication regimen. The most frequent reason given was adverse reactions (33%), followed by feeling well enough to cease taking it (24%). Self-reported adherence was not related to remission of depression symptoms at posttreatment or follow-up.

We found a trend for the number of antidepressant medication prescriptions filled during the 2-year postintervention period to be ranked higher in patients who reported adhering to the regimen, compared to patients who reported nonadherence (Mann Whitney U = 76.00, P = .071). The number of antidepressant medication prescriptions filled by treatment completers was ranked significantly higher than the number filled by those who declined or dropped out of treatment at pretreatment and at both years posttreatment: Mann Whitney U = 146.00, P = .002 (pretreatment); Mann Whitney U = 112.00, P = .001 (posttreatment year 1); Mann Whitney U = 110.00, P = .001 (posttreatment year 2). Treatment completers were also ranked significantly higher in the number of prescriptions filled for benzodiazepine medication at pretreatment (Mann Whitney U = 199.50, P = .030), but not at posttreatment. There was no significant group difference in number of prescriptions filled for sedative/hypnotics or opioid analgesics.

Sixteen patients were referred for specialty mental health services. Of these, 13 (81%) attended at least 1 appointment, and 9 remained in treatment at the time of the follow-up interview.

Veterans who completed treatment showed greater attendance at medical visits than the comparison group. Mann Whitney U tests found that treatment completers’ number of medical visits in the 2 years following treatment was significantly higher than that attended by treatment decliners/dropouts: Mann Whitney U = 119.00, P = .021 (posttreatment year 1) and Mann Whitney U = 121.00, P = .023 (posttreatment year 2). The groups did not differ in the number of pretreatment visits. An ANCOVA found significantly more medical visits for treatment completers even when controlling for chronic disease score (F1,39 = 10.763; P = .002). There were no significant differences between groups in the number of ER visits or general medical costs.

DISCUSSION

This investigation tested the effectiveness of treatment for symptoms of depression using a colocated, integrated model in VA primary care. As anticipated, treated veterans showed improvement in depressive symptoms that was sustained for at least 3 years.

We used a benchmarking strategy in which effect sizes were calculated from symptom scores before and after treatment to compare our results with those obtained in randomized, controlled trials.36 We found large effect sizes for pretreatment to posttreatment for the group with a postintervention HDRS score and a moderate effect size for the intent-to-treat group. We also found a moderate-to-large between-group effect size on symptom reduction at follow-up that favored treatment completion. A meta-analysis of randomized, controlled trials of depression care management programs in primary care found an average effect size of 0.33.38 Another meta-analysis of the long-term effects of collaborative care trials after the year 2000 found average effect sizes of 0.25 at 12 months postintervention and 0.15 at 5-year follow-up.7 Although we cannot conclude that sustained symptom reduction was due to treatment, our outcomes are consistent with those of randomized, controlled trials within the VA system,13,15-17 despite the high rates of psychiatric and medical comorbidities.8,13,21

Contrary to our expectations, veterans completing the intervention provided in primary care made more rather than fewer medical visits. This added care was associated with no significant increase in cost, a finding similar to that for disease management programs.38 Increases in visits may reflect better attendance at regularly scheduled appointments. Moreover, the relationship with the mental health provider may facilitate engagement with the health care team. The influence of a positive relationship is suggested by the Improving Mood-Promoting Access to Collaborative Treatment and nurse telehealth studies.4,39 In our sample, the relationship with the mental health professional in primary care was experienced as a benefit.40

Treatment completers filled significantly more 30-day antidepressant prescriptions, a finding also consistent with that of other VA studies.13,15,16 They also filled significantly more benzodiazepine prescriptions before but not after completing treatment, which may reflect the effectiveness of antidepressants in treating depression and anxiety.

This retrospective reevaluation of an established clinical program is limited by lack of randomization and a small sample size. However, the sample size is comparable to those used in other clinical effectiveness studies.41,42 Our results suggest that a colocated integrated model can be effective when delivered by a working field clinician outside the structure of a randomized, controlled trial. The same provider conducted the assessments and treatment. This might have introduced bias. No other instrument was used to confirm diagnosis, but consistency with the psychological measure used provides a small degree of confidence. These findings support testing an integrated model of ready access to mental health care for acute symptoms that includes a chronic disease management component to assist primary care providers.43 Larger, prospective studies using limited exclusion criteria are needed to work toward determining the realistic potential of integrated care models.

Author affiliations: Department of Psychiatry, Boston VA Healthcare System, West Roxbury, and Department of Psychiatry, Boston University School of Medicine, Boston, Massachusetts (Dr Engel); PTSD/Veterans Readjustment Program, Behavioral Health Clinic (Dr Malta), and Behavioral Health Careline (Dr Davies), Stratton VA Medical Center, Albany, New York; and Department of Psychology, Center for Disability Services, Albany, New York (Dr Baker).

Potential conflicts of interest: None reported.

Funding/support: Supported by a grant from the Center for Integrated Healthcare, VISN 2, Syracuse VA Medical Center, Syracuse, New York. This material is based upon work supported by the Office of Research and Development, Department of Veterans Affairs.

REFERENCES

1. Johnson J, Weissman MM, Klerman GL. Service utilization and social morbidity associated with depressive symptoms in the community. JAMA. 1992;267(11):1478-1483. PubMed doi:10.1001/jama.267.11.1478

2. O’ Connor EA, Whitlock EP, Beil TL, et al. Screening for depression in adult patients in primary care settings: a systematic evidence review. Ann Intern Med. 2009;151(11):793-803. PubMed

3. Dietrich AJ, Oxman TE, Williams JW Jr, et al. Re-Engineering Systems for the Treatment of Depression in Primary Care: cluster randomised controlled trial. BMJ. 2004;329(7466):602. PubMed doi:10.1136/bmj.38219.481250.55

4. Hunkeler EM, Meresman JF, Hargreaves WA, et al. Efficacy of nurse telehealth care and peer support in augmenting treatment of depression in primary care. Arch Fam Med. 2000;9(8):700-708. PubMed doi:10.1001/archfami.9.8.700

5. Gilbody S, Whitty P, Grimshaw J, et al. Educational and organizational interventions to improve the management of depression in primary care: a systematic review. JAMA. 2003;289(23):3145-3151. PubMed doi:10.1001/jama.289.23.3145

6. Williams JW Jr, Gerrity M, Holsinger T, et al. Systematic review of multifaceted interventions to improve depression care. Gen Hosp Psychiatry. 2007;29(2):91-116. PubMed doi:10.1016/j.genhosppsych.2006.12.003

7. Gilbody S, Bower P, Fletcher J, et al. Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med. 2006;166(21):2314-2321. PubMed doi:10.1001/archinte.166.21.2314

8. Liu CF, Campbell DG, Chaney EF, et al. Depression diagnosis and antidepressant treatment among depressed VA primary care patients. Adm Policy Ment Health. 2006;33(3):331-341. PubMed doi:10.1007/s10488-006-0043-5

9. Hankin CS, Spiro A 3rd, Miller DR, et al. Mental disorders and mental health treatment among US Department of Veterans Affairs outpatients: the Veterans Health Study. Am J Psychiatry. 1999;156(12):1924-1930. PubMed

10. Seal KH, Bertenthal D, Miner CR, et al. Bringing the war back home: mental health disorders among 103,788 US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs facilities. Arch Intern Med. 2007;167(5):476-482. PubMed doi:10.1001/archinte.167.5.476

11. Watts BV, Shiner B, Pomerantz A, et al. Outcomes of a quality improvement project integrating mental health into primary care. Qual Saf Health Care. 2007;16(5):378-381. PubMed doi:10.1136/qshc.2007.022418

12. Felker BL, Chaney E, Rubenstein LV, et al. Developing effective collaboration between primary care and mental health providers. Prim Care Companion J Clin Psychiatry. 2006;8(1):12-16. PubMed doi:10.4088/PCC.v08n0102

13. Dobscha SK, Corson K, Hickam DH, et al. Depression decision support in primary care: a cluster randomized trial. Ann Intern Med. 2006;145(7):477-487. PubMed

14. Engel CC, Oxman T, Yamamoto C, et al. RESPECT-Mil: feasibility of a systems-level collaborative care approach to depression and post-traumatic stress disorder in military primary care. Mil Med. 2008;173(10):935-940. PubMed

15. Fortney JC, Pyne JM, Edlund MJ, et al. A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med. 2007;22(8):1086-1093. PubMed doi:10.1007/s11606-007-0201-9

16. Hedrick SC, Chaney EF, Felker B, et al. Effectiveness of collaborative care depression treatment in Veterans’ Affairs primary care. J Gen Intern Med. 2003;18(1):9-16. PubMed doi:10.1046/j.1525-1497.2003.11109.x

17. Oslin DW, Sayers S, Ross J, et al. Disease management for depression and at-risk drinking via telephone in an older population of veterans. Psychosom Med. 2003;65(6):931-937. PubMed doi:10.1097/01.PSY.0000097335.35776.FB

18. Swindle RW, Rao JK, Helmy A, et al. Integrating clinical nurse specialists into the treatment of primary care patients with depression. Int J Psychiatry Med. 2003;33(1):17-37. PubMed doi:10.2190/QRY5-B61V-QE4R-8141

19. Simon G, Ormel J, VonKorff M, et al. Health care costs associated with depressive and anxiety disorders in primary care. Am J Psychiatry. 1995;152(3):352-357. PubMed

20. Unützer J, Patrick DL, Simon G, et al. Depressive symptoms and the cost of health services in HMO patients aged 65 years and older: a 4-year prospective study. JAMA. 1997;277(20):1618-1623. PubMed doi:10.1001/jama.277.20.1618

21. Hankin CS, Spiro A 3rd, Mansell D, et al. Mental disorders and medical care utilization of VA ambulatory care patients: the Veterans Health Study. J Ambul Care Manage. 2006;29(1):51-60. PubMed

22. Bosmans J, de Bruijne M, van Hout H, et al. Cost-effectiveness of a disease management program for major depression in elderly primary care patients. J Gen Intern Med. 2006;21(10):1020-1026. PubMed doi:10.1111/j.1525-1497.2006.00555.x

23. Gilbody S, Bower P, Whitty P. Costs and consequences of enhanced primary care for depression: systematic review of randomised economic evaluations. Br J Psychiatry. 2006;189(4):297-308. PubMed doi:10.1192/bjp.bp.105.016006

24. Simon GE, Katon WJ, VonKorff M, et al. Cost-effectiveness of a collaborative care program for primary care patients with persistent depression. Am J Psychiatry. 2001;158(10):1638-1644. PubMed doi:10.1176/appi.ajp.158.10.1638

25. Simon GE, Manning WG, Katzelnick DJ, et al. Cost-effectiveness of systematic depression treatment for high utilizers of general medical care. Arch Gen Psychiatry. 2001;58(2):181-187. PubMed doi:10.1001/archpsyc.58.2.181

26. Simon GE, Revicki D, Heiligenstein J, et al. Recovery from depression, work productivity, and health care costs among primary care patients. Gen Hosp Psychiatry. 2000;22(3):153-162. PubMed doi:10.1016/S0163-8343(00)00072-4

27. Unutzer J, Katon WJ, Fan MY, et al. Long-term cost effects of collaborative care for late-life depression. Am J Manag Care. 2008;14(2):95-100. PubMed

28. Bell AC, D’ Zurilla TJ. Problem-solving therapy for depression: a meta-analysis. Clin Psychol Rev. 2009;29(4):348-353. PubMed doi:10.1016/j.cpr.2009.02.003

29. Cuijpers P, van Straten A, Andersson G, et al. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol. 2008;76(6):909-922. PubMed doi:10.1037/a0013075

30. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23(1):56-62. PubMed doi:10.1136/jnnp.23.1.56

31. Beck AT, Ward CH, Mendelson M, et al. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561-571. PubMed

32. Steer RA, Cavalieri TA, Leonard DM, et al. Use of the Beck Depression Inventory for primary care to screen for major depression disorders. Gen Hosp Psychiatry. 1999;21(2):106-111. PubMed doi:10.1016/S0163-8343(98)00070-X

33. Beck AT, Epstein N, Brown G, et al. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56(6):893-897. PubMed doi:10.1037/0022-006X.56.6.893

34. Malone DC, Billups SJ, Valuck RJ, et al; IMPROVE Investigators. Development of a chronic disease indicator score using a Veterans Affairs medical center medication database. J Clin Epidemiol. 1999;52(6):551-557. PubMed doi:10.1016/S0895-4356(99)00029-3

35. Von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol. 1992;45(2):197-203. PubMed doi:10.1016/0895-4356(92)90016-G

36. Wade WA, Treat TA, Stuart GL. Transporting an empirically supported treatment for panic disorder to a service clinic setting: a benchmarking strategy. J Consult Clin Psychol. 1998;66(2):231-239. PubMed doi:10.1037/0022-006X.66.2.231

37. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.

38. Badamgarav E, Weingarten SR, Henning JM, et al. Effectiveness of disease management programs in depression: a systematic review. Am J Psychiatry. 2003;160(12):2080-2090. PubMed doi:10.1176/appi.ajp.160.12.2080

39. Hunkeler EM, Katon W, Tang L, et al. Long term outcomes from the IMPACT randomised trial for depressed elderly patients in primary care. BMJ. 2006;332(7536):259-263. PubMed doi:10.1136/bmj.38683.710255.BE

40. Engel AG, Aquilino CA. Combat duty in Iraq and Afghanistan and mental health problems. N Engl J Med. 2004;351(17):1798-1800.

41. Levitt JT, Malta LS, Martin A, et al. The flexible application of a manualized treatment for PTSD symptoms and functional impairment related to the 9/11 World Trade Center attack. Behav Res Ther. 2007;45(7):1419-1433. PubMed doi:10.1016/j.brat.2007.01.004

42. Persons J, Bostrom A, Bertagnolli A. Results of randomized controlled trials of cognitive therapy for depression generalize to private practice. Cognit Ther Res. 1999;23(5):535-548. doi:10.1023/A:1018724505659

43. Pomerantz AS, Sayers SL. Primary care-mental health integration in healthcare in the Department of Veterans Affairs. Fam Syst Health. 2010;28(2):78-82. PubMed doi:10.1037/a0020341