Computer-Assisted Cognitive-Behavior Therapy for Depression in Primary Care: Systematic Review and Meta-Analysis

ABSTRACT

Adequate treatment of depression remains a major challenge for primary care clinicians. With an 8.6% annual prevalence of major depressive disorder in the general population in the United States1 and a heightened risk for depression in patients with medical illnesses such as diabetes, cardiovascular disease, and chronic obstructive pulmonary disease,2 this illness is one of the more common problems encountered in primary care. The intense personal and familial suffering associated with depression and the large economic burden of over 200 billion dollars annually in the United States3 underscore the need for development and dissemination of effective treatments.

The US Preventive Services Task Force4 recommends that primary care doctors screen for depression in adults when resources are in place to “assure accurate diagnosis, effective treatment, and follow-up”; however, there can be many problems in service delivery, especially when psychotherapy is recommended. For example, there are a number of recognized barriers to implementation of cognitive-behavior therapy (CBT), a treatment that has been shown to be effective in a large number of outcome studies5,6 in mental health populations. These obstacles include an insufficient number of trained therapists, costs of treatment, lack of transportation, and difficulty in scheduling and attending a large number of therapy sessions.7-9 Thus, most primary care patients who could benefit from CBT do not participate in this form of treatment or receive any psychotherapy.10 Integration of behavioral health into primary care can increase the frequency of psychotherapy contacts.11,12 However, collaborative care with integrated behavioral health treatment is unavailable in most primary care practices,13 and cost, transportation, and time constraints also are present in collaborative care delivery models.

Computer-assisted CBT (CCBT) has been developed as a way to reduce cost and improve access to evidence-based treatment for depression and other psychiatric disorders while providing data tracking and treatment management features not found in standard CBT.14-16 Previous reviews and meta-analyses17-20 of CCBT that focused largely on patient populations from mental health or community settings found evidence for effectiveness and acceptance of this form of treatment. Medium-to-large effect sizes have been reported for CCBT of depression in such reviews and meta-analyses.17-20 One previous meta-analysis21 of all forms of CBT for depression or anxiety disorders in primary care included 4 studies of CCBT among a total sample of 29 randomized controlled trials. Three of the 4 CCBT studies in this meta-analysis21 had effect sizes that indicated significantly better reduction in depression than control conditions. However, there have been no previous reviews and meta-analyses with a focus on studies of CCBT for depression in primary care.

METHODS

A computerized literature search was conducted using Ovid MEDLINE, PsycINFO, PubMed, and Scopus from their inception to July 18, 2016. In addition, the authors performed a manual search using other meta-analyses and published reports of CCBT.17-20 The search keywords were randomized controlled trials of computer-assisted cognitive-behavior therapy for depression and randomized controlled trials of mobile apps for cognitive-behavior therapy of depression. Criteria for inclusion of studies were (1) randomized controlled trial with control group (ie, no treatment, wait list, attention control, or treatment as usual [TAU] other than standard face-to-face CBT); (2) subjects were depressed as measured by depression rating scales; (3) inclusion criteria specified for depression (ie, clinical diagnosis of depression, diagnosis with standardized assessment, eg, DSM-IV, Structured Clinical Interview for DSM-IV Axis I Disorders,22 Mini International Neuropsychiatric Interview23 [see Supplementary Appendix 1 for listing of full names of diagnostic instruments and measures] or assessment with validated measure for depressive symptoms and appropriate cutoff score, eg, 9-item Patient Health Questionnaire [PHQ-9],24 Beck Depression Inventory [BDI],25 Center for Epidemiologic Studies Depression Scale [CES-D]26); (4) participants were 16 years of age or older; (5) involved use of a computer program or mobile app that covers core methods of CBT to deliver all or part of the treatment; (6) pre- and posttreatment mean scores with standard deviation using a standard depression rating scale (eg, PHQ-9, BDI, Hamilton Depression Rating Scale,27 CES-D); and (7) participants were drawn from a primary care setting (family medicine and internal medicine).

• Computer-assisted cognitive-behavior therapy (CCBT) works best for depression if guided and supported by a clinician.
• Although primary care physicians could provide support for CCBT, it may be more practical to have mental health practitioners or care coordinators (either on site in the primary care practice or available by phone, internet, or telemedicine) guide patients in use of CCBT.
• CCBT offers opportunities for improving the efficiency of treatment for depression while maintaining effectiveness and reducing cost.

Decisions on inclusion and exclusion were reached by consensus of 3 of the authors (D.R., M.E.T., and J.H.W.). Data were abstracted on pre- and posttreatment means and standard deviations on depression rating scales, means and standard deviations on depression rating scales for follow-up assessments (if available), numbers of subjects, type of control group, CCBT program utilized, completion rate, clinician support time, and type of clinician support. If these data were not in the published report, the corresponding author of the study was contacted to request data.

Study quality was assessed using the CLEAR NPT,28 a checklist developed to evaluate reports of nonpharmacologic, randomized clinical trials. The CLEAR NPT includes 15 questions that address study characteristics such as the adequacy of the randomization process; description of the interventions; care provider experience or skill; measurement of adherence to treatment protocols; blinding of care providers, participants undergoing treatments, and outcome evaluators; consistency of follow-up assessments across all groups; and whether the data analysis used intent-to-treat principles. Two of the study authors (T.D.E. and G.K.B.) independently evaluated each study using the CLEAR NPT. Ratings were compared, and differences were reconciled through consensus.

Data Analysis

To determine the efficacy of CCBT versus control conditions, we calculated effect sizes with Hedges g, which is the difference in means at posttreatment or follow-up divided by the pooled standard deviation of both conditions and the estimate of variance. The primary measure of depression was used for these calculations.29 Two studies30,31 did not conduct follow-up assessments after completion of treatment, while 5 studies32-36 had multiple follow-up assessments at varying times. A single study37 had only 1 follow-up assessment. For the follow-up analysis, we aggregated all of the follow-up assessments per study. For some studies,34,35 there were multiple comparisons (eg, multiple versions of CCBT vs control conditions), which were aggregated per study. We used random effects estimates to better generalize beyond the participants in these studies. The heterogeneity of the effects was examined with Q-tests and I2 statistics. Comparison of effects of studies that used therapist-supported CCBT versus self-guided CCBT was planned in advance because of previous research18 showing heterogeneity between these 2 types of studies and lower effect sizes for self-guided CCBT.

RESULTS

Of the 223 studies identified in the search, 215 were excluded, most commonly because the study was not a randomized controlled trial (60 studies). A list of reasons for exclusion is provided in Supplementary Appendix 1. We found 830-37 studies that were randomized controlled trials of CCBT for depression in primary care that met our study criteria. Study characteristics, other than means and SDs, are reported in Table 1. Three33,35,36 of the 8 studies reported no therapist support time and thus were self-guided, 4 studies30,34,36,37 utilized a blended method of a computer program plus therapist support (ranging from 60 to 194 minutes for the entire course of treatment), and 1 study32 did not report support time. All studies used multimedia computer programs that integrated text with video or other multimedia elements and were delivered on personal computers or electronic notebooks. No studies used mobile delivery. Typically, CCBT was delivered in a series of lessons (5-18) over a time period of 7 to 16 weeks.

A forest plot for the posttreatment effects and 95% confidence intervals, along with numerical effect sizes for each study, is displayed in Figure 1. The random effects weighted mean effect size for CCBT versus TAU at posttreatment was g = 0.258 (SE = 0.097; 95% CI, 0.068-0.449; P = .008). As expected, there was significant heterogeneity in the effects (Q7 = 47.397, P < .001, I2 = 85.23), most likely influenced, in part, by inclusion of studies of self-guided CCBT. Examining the funnel plot for posttreatment effects, there was good symmetry, and, consistently, the Egger test of asymmetry was not significant (intercept = 0.65, SE = 2.34, P = .78); however, some caution should be taken with this test given the limited number of studies. A Duval and Tweedie trim and fill analysis yielded 1 study adjustment that only affected the overall g by 0.02. Collectively, these bias tests revealed no meaningful indications of bias in findings.

Six studies32-37 included follow-up assessments ranging from 1 to 8 months after completion of treatment. In these studies, the random effects weighted mean effect size for CCBT versus control was g = 0.400 (SE = 0.103; 95% CI, 0.198-0.602; P < .001). Similar to posttreatment effects, there was significant heterogeneity in the effects (Q5 = 29.782, P < .001, I2 = 83.21).

For analysis of the influence of therapist support on outcome, we dichotomized the studies on the basis of whether patients received significant therapist support time (k = 4) or no (or negligible) therapist support time (k = 3). Therapist support was usually provided on a weekly basis throughout the active treatment period of 7 to 12 weeks. Because a limited number of studies reported the therapist support time, and some studies did not calculate exact time spent in support (only the time scheduled for possible delivery), we did not attempt to examine the relationship between number of minutes of support and outcome. Two35,36 of the studies placed in the no therapist support category offered minimal assistance. Gilbody et al35 provided a mean of less than 7 minutes of technical support and no therapy support to patients in their study. Montero-Marin et al36 provided a total of 17 e-mails to 13 patients out of 291 participants. The random effects weighted mean effect size for CCBT versus control at posttreatment for studies with therapist support time was g = 0.372 (SE = 0.086; 95% CI, 0.203-0.541; P < .001). In contrast, the random effects weighted mean effect size for CCBT versus control at posttreatment for studies without significant therapist support time was g = 0.038 (SE = 0.062; 95% CI, −0.083 to 0.160; P = .535). There was support for the homogeneity of these effects (Q3 = 4.50, P = .212, I2 = 33.378; Q2 = 3.08, P = .214, I2 = 35.277; CCBT versus control, respectively).

Study quality was evaluated with CLEAR NPT ratings that are shown in Figure 2. Design features such as appropriate randomization, adequate description of the interventions, reporting of participant adherence to treatment interventions, and following intent-to-treat principles in analyzing data were achieved by a majority of the studies. Some criteria that were not met, such as blinding participants and treatment providers to treatment conditions, are precluded from being realized by the nature of the interventions.

DISCUSSION

The findings of our review and meta-analysis indicate that CCBT has potential for improving delivery of effective psychotherapy in primary care settings. Studies that incorporated a modest amount of therapist support (60-194 minutes) had a mean effect size in the moderate range, indicating that CCBT was significantly better at relieving depression than usual care or a wait-list control. However, self-guided CCBT (3 of 7 studies reporting therapist support time) had a negligible effect size and thus was ineffective.

Meta-analyses17-20 of larger samples of CCBT studies that included persons recruited on the internet, patients from mental health care delivery settings, and other non-primary care populations have found somewhat higher overall effect sizes for both guided and unguided CCBT. For example, Richards and Richardson18 reported a mean effect size of d = 0.56 for all of the 19 studies in their meta-analysis. Therapist-supported studies in this meta-analysis18 had a mean effect size of d = 0.78, while unsupported studies had a mean effect size of d = 0.36. In contrast, the effect sizes in our meta-analysis of primary care patients were lower. Although available data do not provide enough information to discern why CCBT may be less effective in primary care patients, these influences could be implicated:

1. CCBT has been investigated less frequently in primary care than in other settings. Thus, less is known about how to effectively implement CCBT in primary care patients. It is possible that further research could help improve the delivery of CCBT in primary care.
2. Recruitment methods in many studies in non-primary care settings utilized the internet or advertisements, perhaps gathering a more highly motivated, better-educated, healthier, and computer-savvy group of participants than may be drawn from primary care practices. All but one38 of the studies in our meta-analysis recruited from general practice (term used in European studies), family practice, or internal medicine settings. One trial33 in the Netherlands used the internet to identify potential participants, but treatment was provided by their general practitioners.
3. It is likely that comorbid medical conditions are more common in primary care patients. These conditions could make it more difficult to participate in CCBT or benefit from it. None of the CCBT programs used to date in primary care have been tailored to persons who may have a significant physical illness.

Six32-37 of the 8 studies in our meta-analysis provided follow-up data, an important indicator of the durability of treatment. Although the mean effect size was in the moderate range for follow-up assessments, there has not been enough research to conclude that CCBT for depression is a durable treatment in primary care settings. It would be ideal to have follow-up assessments at least a year posttreatment. However, funding is usually limited for extending evaluations to a year or longer.

The primary limitation of our review and meta-analysis is the small number of studies that have been conducted to date. However, research on CCBT has been expanding,14,15,17-20 and additional studies in primary care are anticipated. Our research group is currently conducting an investigation of CCBT for depression in primary care using therapist support via telephone (NCT 02700009). An additional study (NCT 03068676) of CCBT for anxiety and depression in primary care is being conducted using graduate interns as supporters.

Quality assessments of studies also raise questions about research performed to date on CCBT for depression in primary care. Although some of the CLEAR NPT criteria (eg, care providers are blinded to treatment allocation; patients are blinded to treatment allocation; if care providers not blinded, other treatments are adequately controlled: if patients not blinded, other treatments are adequately controlled; assessments are performed by blind raters) are very difficult standards for practical studies of the effectiveness of CCBT in primary care settings, there is room for improvement in other CLEAR NPT criteria. Two31,33 of the 8 studies did not use intent-to-treat principles for data analysis. Their reliance on data from treatment completers could have inflated effect sizes compared to investigations that employed more rigorous intent-to-treat analytic methods. Additional areas for enhanced study design include using best-practices randomization sequences, insuring that therapists are experienced in the delivery model, and employing outcome measures that do not rely solely on participant self-report.

Another limitation of the current review is that the optimal amount and type of therapist support (eg, face-to-face, telephone, e-mail) and predictors of outcome, other than therapist support, could not be determined. It would be helpful for future studies to report actual therapist support time and to give more detail on the methods of support. Only one38 of the studies published a cost-benefit comparison. In this investigation, CCBT was more efficacious and cost-effective than usual treatment. Because studies39,40 of patients treated in mental health settings have found economic advantages for CCBT, and CCBT requires considerably less therapist time than standard CBT, it is likely that CCBT would offer cost savings if widely disseminated in primary care.

Engagement in CCBT among primary care patients is another concern that has not been fully addressed to date. Will patients accept and complete this form of treatment compared to other approaches to treat depression? Two33,36 of the 8 studies reported patient satisfaction, and both found high levels of acceptance of CCBT. Treatment satisfaction for CCBT typically has been high in studies in mental health settings,14,18,39,40 and completion rates usually have been good in therapist-supported CCBT.28,35,41,42 The overall completion rate in studies reviewed by Richards and Richardson18 was 72%. However, more research is needed on acceptance of CCBT in primary care patients.

None of the 8 studies reviewed here used mobile delivery for the CBT computer program. A wide variety of CBT apps have been developed.43 However, the quality, security, and efficacy of most apps have been questioned,43 and CBT apps are typically designed for specific interventions such as relaxation or breathing training instead of delivery of a full program of CBT. Only a few mobile apps have been investigated as treatments for depression in randomized controlled trials.44,45 It is possible that mobile delivery could provide greater flexibility and engagement if used as part of CCBT for depression.

Despite gaps in our current knowledge of CCBT for depression in primary care, there are indications that this method, if combined with therapist support, offers a way to engage greater numbers of patients in evidence-based psychotherapy, while improving the efficiency of treatment and reducing cost. Some of the important challenges for future research include detailing implementation strategies and improving the effectiveness of CCBT in primary care practice, delineation of the most effective ways of integrating human and computer elements of treatment, and realizing the potential for newer technologies as they become available.

Submitted: July 13, 2017; accepted October 11, 2017.

Published online: March 1, 2018.

Potential conflicts of interest: Dr Richards is an employee of SilverCloud Health, developers of computerized psychological interventions for depression, anxiety, stress and comorbid long-term conditions. Dr Thase reports the following relationships during the course of this study. He was an advisory/consultant to Acadia, Alkermes, Allergan (Forest, Naurex), AstraZeneca, Cerecor, Eli Lilly, Johnson & Johnson (Janssen, Ortho-McNeil), Lundbeck, MedAvante, Merck, Mocksha8, Nestlé (PamLab), Neuronetics, Novartis, Otsuka, Pfizer, Shire, Sunovion, and Takeda. In addition to the National Institute of Mental Health, he received grant support from the Agency for Healthcare Research and Quality, Alkermes, Assurex, Avanir, Forest, Johnson & Johnson, Otsuka, and Takeda. Dr Thase received royalties from the American Psychiatric Press, Guilford Publications, Herald House, and W.W. Norton & Company, Inc. Dr Thase’s spouse, Dr Diane Sloan, works for Peloton Advantage, which did business with Pfizer and AstraZeneca. Dr Wright is an author of the Good Days Ahead (GDA) program used in this investigation and has an equity interest in Empower Interactive and Mindstreet, developers and distributors of GDA. He receives no royalties or other payments from sales of this program. His conflict of interest is managed with an agreement with the University of Louisville. He receives book royalties from American Psychiatric Press, Inc, Guilford Press, and Simon and Schuster. Dr Wright receives grant support from the National Institutes of Health (Agency for Health Care Research and Quality). Drs Wells, Owen, McCray, Bishop, Eells, and Brown report no conflicts of interest related to the subject of this article.

Funding/support: None.

Supplementary material: See accompanying pages.

REFERENCES

1. Kessler RC, Petukhova M, Sampson NA, et al. Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. Int J Methods Psychiatr Res. 2012;21(3):169-184. PubMed CrossRef

2. Meader N, Mitchell AJ, Chew-Graham C, et al. Case identification of depression in patients with chronic physical health problems: a diagnostic accuracy meta-analysis of 113 studies. Br J G Pract. 2011;61(593):e808-e820. PubMed CrossRef

3. Greenberg PE, Fournier AA, Sisitsky T, et al. The economic burden of adults with major depressive disorder in the United States (2005 and 2010). J Clin Psychiatry. 2015;76:155-162. PubMed CrossRef

4. US Preventive Services Task Force. Final Recommendation Statement. Depression in Adults: Screening. https://www.uspreventiveservicestaskforce.org/Page/Document/
RecommendationStatementFinal/depression-in-adults-screening1. Accessed February 8, 2018.

5. Cuijpers P, Berking M, Andersson G, et al. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry. 2013;58(7):376-385. PubMed CrossRef

6. Vittengl JR, Clark LA, Dunn TW, et al. Reducing relapse and recurrence in unipolar depression: a comparative meta-analysis of cognitive-behavioral therapy’s effects. J Consult Clin Psychol. 2007;75(3):475-488. PubMed CrossRef

7. Kaltenthaler E, Sutcliffe P, Parry G, et al. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med. 2008;38(11):1521-1530. PubMed CrossRef

8. Mohr DC, Ho J, Duffecy J, et al. Perceived barriers to psychological treatments and their relationship to depression. J Clin Psychol. 2010;66(4):394-409. PubMed

9. Mohr DC, Hart SL, Howard I, et al. Barriers to psychotherapy among depressed and nondepressed primary care patients. Ann Behav Med. 2006;32(3):254-258. PubMed CrossRef

10. Mohr DC, Ho J, Duffecy J, et al. Effect of telephone-administered vs face-to-face cognitive behavioral therapy on adherence to therapy and depression outcomes among primary care patients. JAMA. 2012;307(21):2278-2285. PubMed CrossRef

11. Kessler R. Mental health care treatment initiation when mental health services are incorporated into primary care practice. J Am Board Fam Med. 2012;25(2):255-259. PubMed CrossRef

12. Auxier A, Runyan C, Mullin D, et al. Behavioral health referrals and treatment initiation rates in integrated primary care: a Collaborative Care Research Network study. Transl Behav Med. 2012;2(3):337-344. PubMed CrossRef

13. Kessler R, Miller BF, Kelly M, et al. Mental health, substance abuse, and health behavior services in patient-centered medical homes. J Am Board Fam Med. 2014;27(5):637-644. PubMed CrossRef

14. Eells TD, Barrett MS, Wright JH, et al. Computer-assisted cognitive-behavior therapy for depression. Psychotherapy. 2014;51(2):191-197. PubMed CrossRef

15. Spurgeon JA, Wright JH. Computer-assisted cognitive-behavioral therapy. Curr Psychiatry Rep. 2010;12(6):547-552. PubMed CrossRef

16. Greist JH. Computer interviews for depression management. J Clin Psychiatry. 1998;59(suppl 16):20-24. PubMed

17. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196-205. PubMed CrossRef

18. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329-342. PubMed CrossRef

19. Arnberg FK, Linton SJ, Hultcrantz M, et al. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PLoS One. 2014;9(5):e98118. PubMed CrossRef

20. So M, Yamaguchi S, Hashimoto S, et al. Is computerized CBT really helpful for adult depression? a meta-analytic re-evaluation of CCBT for adult depression in terms of clinical implementation and methodological validity. BMC Psychiatry. 2013;13(1):113. PubMed CrossRef

21. Twomey C, O’ Reilly G, Myrne M. Effectiveness of cognitive behavioural therapy for anxiety and depression in primary care: a meta-analysis. Fam Pract. 2015;32(1):3-15. PubMed CrossRef

22. First M, Spitzer R, Williams J, et al. Structured Clinical Interview for DSM-IV Axis 1 Disorders (SCID-I). Handbook of Psychiatric Measures. Washington, DC: American Psychiatric Association; 2000:49-53.

23. Sheehan DV, Lecrubier YK, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): The Development and Validation of a Structured Diagnostic Psychiatric Interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(suppl 20):22-33. PubMed

24. Spitzer RL, Kroenke K, Williams JBW. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA. 1999;282(18):1737-1744. PubMed CrossRef

25. Beck AT, Ward CH, Mendelson M, et al. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4(6):561-571. PubMed CrossRef

26. Radloff LS. The CES-D Scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1(3):385-401. CrossRef

27. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23(1):56-62. PubMed CrossRef

28. Boutron I, Moher D, Tugwell P, et al. A checklist to evaluate a report of a nonpharmacological trial (CLEAR NPT) was developed using consensus. J Clin Epidemiol. 2005;58(12):1233-1240. PubMed CrossRef

29. Hedges LV, Olkin I. Statistical Methods for Meta-Analysis. Orlando, FL: Academic Press; 1985.

30. Kivi M, Eriksson M, Hange D, et al. Internet-based therapy for mild to moderate depression in Swedish primary care: short term results from the PRIM-NET randomized controlled trial. Cogn Behav Ther. 2014;43(4):289-298. PubMed CrossRef

31. Hallgren M, Kraepelien M, Öjehagen A, et al. Physical exercise and internet-based cognitive-behavioural therapy in the treatment of depression: randomized controlled trial. Br J Psychiatry. 2015;207(3):227-234. PubMed CrossRef

32. Proudfoot J, Ryden C, Everitt B, et al. Clinical efficacy of computerized cognitive-behavioral therapy for anxiety and depression in primary care: randomized controlled trial. Br J Psychiatry. 2004;185(1):46-54. PubMed CrossRef

33. de Graaf LE, Gerhards SAH, Arntz A, et al. Clinical effectiveness of online computerized cognitive-behavioral therapy without support for depression in primary care: randomized trial. Br J Psychiatry. 2009;195(1):73-80. PubMed CrossRef

34. Mohr DC, Duffecy J, Ho J, et al. A randomized controlled trial evaluating a manualized telecoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One. 2013;8(8):e70086. PubMed CrossRef

35. Gilbody S, Littlewood E, Hewitt C, et al. Computerized cognitive behavioral therapy (CCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomized controlled trial. BMJ. 2015;351:h5627. PubMed CrossRef

36. Montero-Marin J, Araya R, Perez-Yus MC, et al. An internet-based intervention for depression in primary care in Spain: a randomized controlled trial. J Med Internet Res. 2016;18(8):e231. PubMed CrossRef

37. Hoifodt R, Lillevoll K, Griffiths K, et al. The clinical effectiveness of web-based cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled trial. J Med Internet Res. 2013;15(8):e153. PubMed CrossRef

38. McCrone P, Knapp M, Proudfoot J, et al. Cost-effectiveness of computerized cognitive-behavioral therapy for anxiety and depression in primary care: randomized controlled trial. Br J Psychiatry. 2004;185(1):55-62. PubMed CrossRef

39. Hollinghurst S, Peters T, Kaur S, et al. Cost-effectiveness of therapist-delivered online cognitive-behavioral therapy for depression: randomized controlled trial. Br J Psychiatry. 2010;197(4):297-304. PubMed CrossRef

40. Gerhards S, de Graaf LE, Jacobs L, et al. Economic evaluation of online computerized cognitive-behavioral therapy without support for depression in primary care: randomized trial. Br J Psychiatry. 2010;196(4):310-318. PubMed CrossRef

41. Wright JH, Wright AS, Salmon P, et al. Development and initial testing of a multimedia program for computer-assisted cognitive therapy. Am J Psychother. 2002;56(1):76-86. PubMed

42. Wright JH, Wright AS, Albano AM, et al. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry. 2005;162(6):1158-1164. PubMed CrossRef

43. Van Singer M, Chatton A, Khazaal Y. Quality of smartphone apps related to panic disorder. Front Psychiatry. 2015;6(96). 10.3389/fpsyt.2015.00096 PubMed

44. Birney AJ, Gunn R, Russell JK, et al. MoodHacker mobile web app with email for adults to self-manage mild-to-moderate depression: randomized controlled trial. JMIR Mhealth Uhealth. 2016;4(1):e8. PubMed CrossRef

45. Roepke AM, Jaffee SR, Riffle OM, et al. Randomized controlled trial of SuperBetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J. 2015;4(3):235-246. PubMed CrossRef