Clinical Guidance on the Identification and Management of Treatment-Resistant Schizophrenia

This work may not be copied, distributed, displayed, published, reproduced, transmitted, modified, posted, sold, licensed, or used for commercial purposes. By downloading this file, you are agreeing to the publisher’s Terms & Conditions.

Schizophrenia is a complex, progressive, and severe mental disorder characterized by distortions in thinking, perception, emotions, language, sense of self, and behavior.1 It is estimated that over 21 million people worldwide have schizophrenia.1 The majority of people with this illness exhibit a prodromal period characterized by subtle changes in thoughts and perceptions, followed by the onset of psychotic symptoms.2 Between 75% and 87% of patients with first-episode psychosis responded to the first treatment with an antipsychotic medication by 4 weeks to 1 year.3-7 However, for patients whose illness has not clinically improved with the first antipsychotic, response rates to subsequent nonclozapine antipsychotic treatment are much lower. Studies in patients with chronic schizophrenia show that up to 30% of patients diagnosed with schizophrenia meet criteria for treatment-resistant schizophrenia (TRS), that is, failure to respond to ≥ 2 different nonclozapine antipsychotic medications.2,8,9 Longer duration of untreated psychosis and having received multiple different antipsychotic treatments are indicators of poor prognosis.6,10 Data suggest that response rates fall dramatically with additional psychotic relapses and serial administration of different antipsychotic treatment trials, with only 9% of patients responding to a third nonclozapine antipsychotic treatment trial in one study.11 Furthermore, high-quality evidence for effective antipsychotic augmentation and/or combination treatments for schizophrenia is currently insufficient.12,13

To optimize the treatment of TRS, early identification is critical. Through early identification, the duration of inadequately controlled illness may be reduced, thereby improving long-term outcomes.14 However, clinicians in daily practice may not readily recognize TRS as a distinct clinical entity. As a result, appropriate treatment may be delayed or not offered at all. The identification of TRS is further complicated by difficulties distinguishing it from antipsychotic nonadherence, as well as by the heterogeneous and multidimensional nature of TRS (eg, varying onset of treatment nonresponse5,15,16 and involvement of nonresponsive positive, negative, and cognitive symptoms17) as well as the lack of consensus on clinically relevant criteria for defining and treating TRS.2,9,17,18

This report summarizes the consensus findings of a roundtable meeting on TRS (June 23, 2017; New York, NY; sponsored by Lundbeck) that included 9 clinicians with expertise in the areas of schizophrenia and TRS. The report provides an overview of TRS and the consensus derived regarding its key domains. The goal is to increase awareness of this common phenomenon among general psychiatrists to facilitate early identification and more effective treatment of patients with TRS.

DEFINITION AND IDENTIFICATION OF TREATMENT-RESISTANT SCHIZOPHRENIA

Current treatment guidelines for schizophrenia are broadly aligned in terms of their definition of TRS (Table 1).2,9,17,18 TRS is characterized by persistence of positive symptoms (eg, delusions, hallucinations, disorganized speech and behavior) despite adequate treatment trials with antipsychotic medications.19 Key criteria for the definition of TRS include no significant improvement in positive symptoms after treatment with ≥ 2 different nonclozapine antipsychotic medications at adequate dose, duration, and documented adherence. Thus, the term TRS in this context refers to patients not adequately responding to first-line medication approaches. Patients not responding to clozapine or a nonpharmacologic treatment option, such as electroconvulsive therapy (ECT), represent distinct subgroups of TRS. These 2 groups can descriptively be referred to as clozapine-resistant and ECT-resistant schizophrenia.

In contrast to the relative consistency in major aspects of the definition of TRS, recommendations for what constitutes a lack of treatment response and the clinical outcomes used to assess the level of treatment response vary among the guidelines.2,9,17,18 This issue is further complicated in patients with TRS because they are a heterogeneous patient population, likely reflecting the complexity of the clinical and neurobiological pathways leading to TRS.

Another potentially complicating issue when determining the level of treatment response is the concept of pseudoresistance, which posits that certain factors can make it appear as if a patient is not responsive to treatment when in fact the treatment response can be modified, such as through improved and/or verified antipsychotic adherence. Factors that need to be evaluated as underlying causes of pseudoresistance include medication nonadherence, pharmacokinetic issues (eg, poor drug absorption or brain penetration, drug-drug interactions, changes in metabolism or body volume) that lead to inadequate therapeutic drug levels, and specific patient characteristics (eg, substance use disorders, other medical conditions) that may exacerbate symptoms or modify treatment response.

• Treatment-resistant schizophrenia (TRS) occurs in approximately 30% of individuals diagnosed with schizophrenia, but identification and management of TRS in clinical practice are inconsistent and not evidence based.
• To establish TRS, a patient must demonstrate inadequate response to 2 different antipsychotics, each taken with adequate dose, duration, and confirmed adherence.
• Upon recognition of TRS, the treatment plan should be reevaluated and alternative treatments considered. Recognition of TRS earlier in the course of disease will allow for more informed treatment decisions to be made by clinicians, patients, and caregivers.

PATHWAYS TO TREATMENT-RESISTANT SCHIZOPHRENIA

Current data suggest that there are probably multiple clinical pathways and different potential neurobiological mechanisms involved in the pathophysiology of TRS (Figure 1).5,15,16,20-29 For example, heterogeneity has been observed in the onset of treatment nonresponse. Approximately 30% of individuals diagnosed with schizophrenia exhibit a partial response or no response to initial treatment,8 whereas 10%-60% of individuals exhibiting an initial response to antipsychotic treatment develop treatment resistance over time5,15,16 and after multiple relapses.30

There are also multiple hypotheses concerning the underlying neurobiological mechanisms of TRS that account for the 2 main clinical paths to TRS (initial resistance vs resistance that emerges over time). One hypothesis posits that supersensitivity of dopamine D2 (DRD2) receptors to dopamine contributes to the pathophysiology and development of TRS in some individuals.20-24 This supersensitivity may be due to an increase in the number of receptors or an increase in receptor affinity for dopamine.29 Antipsychotic medications mitigate psychosis by blocking DRD2 receptors. However, prolonged blockade of postsynaptic dopamine receptors may lead to dopamine supersensitivity, resulting in increased psychotic symptoms (breakthrough psychosis) and motor side effects such as abnormal involuntary movements (eg, tardive dyskinesia) in treated patients.20-22 Supersensitivity may also lead to increasingly higher antipsychotic doses being used in an attempt to control breakthrough psychosis. Supporting the supersensitivity hypothesis, increases in DRD2 receptor levels have been observed in individuals with schizophrenia after prolonged antipsychotic treatment.23,24 However, clinical support for a causal role of dopamine supersensitivity psychosis in TRS is currently lacking.29,31

An alternative theory for TRS, the “normodopaminergic” hypothesis, is based on evidence indicating that not all individuals with schizophrenia have characteristics of a hyperdopaminergic system.25,26 In one positron emission tomography study, patients with TRS had lower dopamine synthesis capacity than treatment-responsive patients but a similar capacity to healthy volunteers.26 In these patients with TRS and normal dopaminergic function, other neurotransmitter systems are proposed to contribute to the symptoms of schizophrenia. For example, changes in the glutamatergic system have been postulated to play a role in the etiology and development of TRS.24-28 Supporting this hypothesis, anterior cingulate glutamate metabolite concentrations were found to be elevated in patients with TRS relative to healthy volunteers but not in treatment responders, suggesting a role for changes in the glutamatergic system in patients with TRS.28 Another potential pathway to TRS is the emerging notion of N-methyl-d-aspartate (NMDA) antibody psychosis,32 which can be diagnosed with a positive antibody titer to the NR1 subunit of the NMDA receptor.

BURDEN OF TREATMENT-RESISTANT SCHIZOPHRENIA

As summarized in Table 2, TRS is associated with a substantial burden for patients, treatment teams, and families/caregivers.33-37 Patients with TRS have more severe positive and negative symptoms, greater functional limitations, and higher health care costs compared with antipsychotic treatment responders.33 In one study, total health care costs were twice as high for patients with TRS compared with antipsychotic treatment responders after 8 weeks of treatment.35 The persistent negative or cognitive symptoms and poor social functioning characteristics of TRS have been associated with self-stigma.38,39 Additionally, the perception of psychosis-related violence and dangerousness can result in stigma40 and potentially discrimination, self-harm, long-term hospitalization, and incarceration. These patient-centered aspects of TRS also require attention.

With respect to the burden of TRS on treatment teams, treatment for patients with TRS is too often associated with pessimism on the part of all concerned. Furthermore, specialized treatment units for TRS are generally lacking, which often results in patients with TRS being treated in a less sophisticated environment by health care providers with little or no training or education in the appropriate treatment of patients with TRS.41,42 The burden of TRS on families/caregivers grows as disease chronicity increases over time, as measured by decreases in family cohesion and flexibility (ie, quality and expression of leadership and organization) and by increases in indirect costs (eg, lost work time, stress-related illness).37

Direct health care costs tend to be higher in patients with TRS than in the general population of patients with schizophrenia, with estimates of excess annual costs per patient (adjusted to 2016 US $) ranging from $9,550 to $35,281 across 3 studies.43-45 One systematic review of studies of the burden of TRS reported that total health resource utilization cost estimates were 3 to 11 times greater than the costs for non-TRS.46 The wide range of estimated excess costs for TRS across different analyses is likely related to differences in study populations, treatment durations, definitions of TRS used, and variations in the completeness of the cost data.

Another analysis found that if 20% of the veterans with TRS initiated clozapine treatment, the mean 1-year cost savings to the Veterans Health Administration (VHA) would be $22,444 per veteran with TRS.47 Furthermore, 95% of model simulations from this analysis estimated a savings of at least $290 million if all VHA patients with TRS not on clozapine were to initiate clozapine treatment.47

There are no published studies from the United States describing productivity losses in patients with TRS. However, unemployment rates may serve as a proxy for lost productivity. Two studies of individuals with schizophrenia in the United States reported high rates of unemployment (> 70%).48,49 Given that schizophrenia symptoms and cognitive function are predictors of work outcomes50 and antipsychotic treatment has been associated with cognitive improvement,51 it seems likely that patients with TRS may have higher unemployment rates than patients who respond to antipsychotic treatment. In a cohort of patients with TRS from Italy, the unemployment rate was 64.7%.52 Although unemployment data across countries are likely to be region specific, these overall data reflect the potentially high level of lost productivity due to TRS.

The economic burden of schizophrenia on society can also be observed with data from Social Security Disability Insurance (SSDI) and Supplemental Security Income (SSI) beneficiaries. Persons diagnosed with schizophrenia account for approximately 5% and 7% of beneficiaries, respectively.53,54 The high proportion of beneficiaries with schizophrenia is markedly disproportionate to the relative prevalence of schizophrenia in the overall population (< 1%) and further reflects the low rates of employment among this population. In 2015, total annual costs associated with schizophrenia and other psychotic disorders were approximately $443 million and $248 million for SSDI and SSI, respectively.53,54 Despite the substantial economic burden of TRS, an online review of federal funding by the National Institutes of Health indicated that funding for TRS research is approximately 7 times lower than funding for treatment-resistant major depressive disorder and approximately 563 times lower than funding for other dimensions of schizophrenia. More effective and earlier treatment of patients with TRS could potentially lessen the economic burden associated with this condition (eg, lower health care costs and increased rates of employment).

CURRENT TREATMENT AND UNMET NEEDS IN TREATMENT-RESISTANT SCHIZOPHRENIA

Current Treatment Strategies

Several strategies are widely used in clinical practice to manage patients with TRS (eg, increasing the antipsychotic trial duration, using higher doses of nonclozapine antipsychotics, switching to other nonclozapine antipsychotics, employing polypharmacy [which includes augmentation with nonclozapine antipsychotics or mood stabilizers]).12 Across 4 studies,55-58 the use of antipsychotic doses above recommended maximums was reported in 10.1%-36.2% of patients before they received clozapine. Furthermore, across several studies, the percentage of patients receiving antipsychotic polypharmacy ranged from 15.9% to 60.5% before they received clozapine.56-59

However, there is limited clinical benefit to using nonclozapine management strategies to address inadequate treatment response to antipsychotics. Agid and colleagues60 reported that 77% (23/30) of individuals treated with at least 2 different antipsychotics (olanzapine, risperidone, or quetiapine) did not respond after 25 to 28 weeks of treatment. Additionally, only 15.5% (11/71) of individuals treated with higher dose ranges of antipsychotic therapy (olanzapine 22.5-30 mg or risperidone 6.5-10 mg) and 16.7% (10/60) of individuals switched to a different antipsychotic therapy responded to treatment.61 Furthermore, another study3 demonstrated that switching to another nonclozapine treatment after nonresponse to an initial nonclozapine antipsychotic treatment resulted in a treatment response in only 16.7% of individuals experiencing a first psychotic episode. These data are consistent with those of another study4 that reported that treatment response and remission rates progressively decreased when switching between nonclozapine antipsychotics. Recently published meta-analyses12,13 have shown that there is limited high-quality evidence to support antipsychotic augmentation strategies.

Taken together, these data suggest that increased trial duration,60 using high-dose nonclozapine antipsychotics,55-58,61 switching to other nonclozapine antipsychotics,3 or employing polypharmacy (including augmentation with nonclozapine antipsychotics12,13)56-59 may unnecessarily delay the initiation of clozapine treatment or future effective treatments for TRS and hinder clinical response in patients who do not respond to antipsychotic therapy. Retrospective studies conducted outside the United States similarly reveal considerable hesitancy and delay in initiating patients on clozapine.58,62-64 In a Canadian study62 of outpatients (n = 467), approximately two-thirds (68%) were treated with ≥ 3 different antipsychotics before clozapine initiation; the median length of therapy before starting clozapine was 8.9 years for male patients and 7.7 years for female patients. Reports from the United Kingdom and Australia revealed that, on average, 3.5 to 5.5 antipsychotics were used before initiating clozapine treatment and an average of 4.0 to 9.7 years passed from diagnosis to start of clozapine.58,63,64 Delays in the initiation of clozapine treatment have been associated with reductions in the percentage of symptomatic improvement in individuals with TRS, as measured by Brief Psychiatric Rating Scale total and psychosis scores.14

In patients with suboptimal response to previous antipsychotic treatment, clozapine and olanzapine were found to be superior to haloperidol for improvement in Positive and Negative Syndrome Scale (PANSS) total and negative symptom scores.65 In an observational study of Medicaid beneficiaries with schizophrenia, treatment with clozapine instead of antipsychotic polypharmacy was associated with reduced disease-specific emergency department use and reduced disease-specific and all-cause health care costs.66

Since clozapine is currently the only indicated and evidence-based treatment for TRS, but also has a number of potentially severe adverse effects,67 the decision to use clozapine is influenced by weighing the burden of psychosis and suicidality versus the risk of weight gain, diabetes, and other adverse effects. Furthermore, the decision to initiate clozapine should be undertaken on an individual basis, in close collaboration with patients, and closely associated with early recognition of TRS because there is evidence that earlier use of clozapine is associated with better response.68

Efficacy of Current Treatments

There is evidence suggesting that up to 75% of individuals with a first episode of psychosis3,60,61 and 40% of individuals with TRS69 who did not respond to nonclozapine antipsychotics might respond to clozapine. A recent 3-phase switching study70 found that some first-episode schizophrenia patients who do achieve cross-sectional remission after a 4-week open-label trial with an initial nonclozapine antipsychotic (amisulpride; phase 1) continued to remit after a 6-week double-blind trial of either continued treatment with amisulpride or a switch to olanzapine (phase 2). However, those patients who had not experienced cross-sectional remission after a total of 10 weeks of nonclozapine treatment had a subsequent cross-sectional remission rate after another 12 weeks of open treatment with clozapine (phase 3) of 18% among patients in the intention-to-treat group and 28% among patients who completed the clozapine trial.70 Furthermore, the results from a network meta-analysis of randomized controlled trials comparing the efficacy of clozapine with that of other second-generation antipsychotics are inconsistent.71 The mixed results from meta-analyses regarding the superiority of clozapine compared with other second-generation antipsychotics are likely due to meta-analytic-, patient-, and treatment-related factors, as well as problems in some cases with trial design and implementation.72 In particular, some studies included treatment-intolerant patients as well as treatment-resistant patients, and there is considerable variation in the way treatment resistance is defined and evaluated among studies.17 In a systematic review,73 short-term and long-term clozapine use was reported to be superior to other antipsychotics in reducing positive psychotic symptoms, but only short-term clozapine use (not long-term use) was superior to other antipsychotics for negative symptoms in patients with treatment-refractory schizophrenia. The lack of superiority of clozapine with longer-term treatment could be due to insufficient statistical power.72 Supporting a benefit for clozapine in the long term, naturalistic studies show reduced mortality over 7 to 11 years of treatment and reduced hospital readmission rates over an average of 5.7 years of treatment in patients receiving clozapine.74,75 More recently, a systematic review of meta-analyses that included a quality assessment of previously published meta-analyses found that no pharmacologic combination treatment (including 5 combinations with clozapine) had sufficiently consistent efficacy to recommend the combination therapy over antipsychotic monotherapy.12 There are those who would argue that there are insufficient data from blinded, well-controlled studies to determine which antipsychotic is most efficacious for the treatment of patients with TRS.

Nonpharmacologic adjunctive treatment (eg, cognitive behavioral therapy for psychosis [CBTp], hallucination-focused integrative therapy [HIT], repetitive transcranial magnetic stimulation [rTMS], ECT) to antipsychotic treatment may be effective for managing symptoms in individuals with TRS.76-81 A meta-analysis of studies using CBTp in individuals with schizophrenia reported a modest beneficial effect on positive symptoms (mean weight effect size, 0.37).76 A meta-analysis of the use of CBTp in individuals with medication-resistant psychosis reported effect sizes (Hedges g) of 0.47 and 0.41 for positive symptoms and 0.52 and 0.40 for general symptoms at posttreatment and follow-up, respectively.77 In patients with persistent (> 10 years), drug-refractory auditory hallucinations, HIT resulted in significant improvements versus control patients (provided routine care) on distress scores and total burden measured with the Psychotic Symptom Rating Scales—Auditory Hallucination Rating Scale and on positive symptoms measured with the PANSS.81 In a review78 of studies that used rTMS as a means of neuromodulation, moderate reductions in the severity of auditory-verbal hallucinations were reported in individuals whose auditory-verbal hallucinations were medication resistant. In a meta-analysis79 of randomized controlled trials, ECT as an adjunct to nonclozapine antipsychotic monotherapy was reported to be superior to antipsychotic monotherapy. Furthermore, in a prospective study80 of individuals with TRS who were also nonresponders to clozapine, ECT augmentation resulted in 50% of study participants (10/20) achieving a response (defined as ≥ 40% improvement in psychotic symptoms, a Clinical Global Impressions-Severity rating < 3, and a Clinical Global Impressions-Improvement rating ≤ 2) compared with 0% of participants (n = 19) receiving clozapine without ECT augmentation. Taken together, these data suggest adjunctive nonpharmacologic therapy may be beneficial in individuals with TRS. There is evidence from meta-analyses supporting the addition of CBTp to an antipsychotic for various degrees of treatment-resistant symptoms.76,77 However, the relative sparsity of data available for other therapies (in part due to their underutilization) limits the certainty with which conclusions can be drawn regarding their overall clinical utility.

Unmet Treatment Needs

It is important to recognize that while many individuals with TRS exhibit a positive response to clozapine, there is a subset of individuals who are clozapine resistant.3,60,61 It has been proposed that individuals with TRS should be categorized according to this dichotomy in treatment response (clozapine responders vs clozapine nonresponders).19,82 A wide range of values has been reported for the prevalence of clozapine-resistant schizophrenia, or “ultra TRS,” but a recent meta-analysis69 found that as many as 60% of patients failed to respond to clozapine across short- and long-term studies. Moreover, mixed results have been obtained for the use of pharmacologic12 adjunctive treatments, while results have been generally positive for nonpharmacologic adjunctive treatments including ECT79 in non-TRS and TRS83 patients as well as for 9 months of CBT in patients who are resistant to clozapine, although results were not sustained at 21 months.84 As a clinically defined subgroup that may have unique biological characteristics, further studies of patients not responding to ECT are warranted.

A percentage of individuals with TRS refuse clozapine treatment. In one study, 9 of 23 patients (39%) who failed to respond to 2 antipsychotic trials refused to receive clozapine in a subsequent trial.60 In another study, 35% (20/57) of patients (no previous trials with clozapine) admitted to an acute ward for schizophrenia or schizoaffective disorder stated that they would refuse clozapine.85 The defining features of these populations of individuals with TRS (nonclozapine antipsychotic treatment resistant vs clozapine treatment resistant) need to be further elucidated. In addition, it will be important to further understand how to optimize clozapine treatment, as there is limited evidence regarding adequate clozapine doses and adequate plasma clozapine levels required for a clozapine trial. It is possible that in some of the cases of clozapine resistance, treatment was not optimized.

CONSENSUS SUMMARY

Definition and Identification of Treatment-Resistant Schizophrenia

The early recognition of TRS and management of its associated psychotic symptoms are critical because a prolonged duration of persistent and ineffectively treated psychosis is associated with negative outcomes.14 It is important to overcome the therapeutic nihilism that sometimes leads to sustained, ineffective treatment.86 Rather, it should be considered that the appropriate treatment has not yet been provided to the patient, including currently available long-acting injectable medications in the case of nonadherence. Identifying the underlying reason that patients are treatment resistant may be difficult because of the heterogeneous and multidimensional nature of the factors that may contribute to its etiology. Additionally, there may be other contributing factors, such as access and continuity of comprehensive care. There is clearly a need for more training of clinicians to facilitate the earlier identification and appropriate management of TRS. In some regions and settings, the establishment of specialty units or clinics could help to improve the care of such patients. In a study examining a comprehensive, multidisciplinary, team-based treatment approach for first-episode psychosis in the United States, it was reported that comprehensive care for first-episode psychosis improved both functional and clinical outcomes.87 Clozapine clinics have been established at a number of psychiatric facilities around the world, and although no formal studies of outcomes in comparison to usual care have been reported, it is likely that treatment by clinicians with special expertise has a variety of advantages. It has been suggested that clozapine clinics expand clozapine accessibility, enhance physician competency with clozapine, and provide better training for residents and other trainees.88

The consensus definition of TRS is the failure of ≥ 2 different antipsychotic medication trials at adequate dose and duration of ≥ 6 weeks, with objective assessments of adherence (Tables 1 and 3). This recommendation is consistent with the recently published guidelines of the Treatment Response and Resistance in Psychosis Working Group.17 Before and after antipsychotic trials, objective symptom measurement and medication adherence tools (eg, blood or urine levels, pill counts, supervised medication intake, use of long-acting injectable formulations) should be used to verify the presence of TRS.

Pathways to Treatment Resistance in Treatment-Resistant Schizophrenia

Evidence suggests that there is no single pathway, biological phenotype, or clinical phenotype that accounts for all cases of TRS. Multiple neurobiological mechanisms are likely to occur across patients with TRS. Although gaps in understanding of the causes and evolution of TRS exist, current evidence suggests both dopaminergic and nondopaminergic mechanisms may account for the pathophysiology of TRS.20-29 There is also a need to clarify the role of pseudoresistance in TRS. The development of algorithms/flow diagrams may help to provide direction to clinicians on the potential factors that may contribute to nonresponse, including the role of nonadherence.

Areas of future research that will help to further clarify the pathophysiology of TRS and reliably identify it include the use of genotyping and other biomarkers (eg, neuroimaging) and further examination of the role of oxidative stress and inflammation. The development of an algorithm based on a simple assessment tool, which evaluates treatment response, focuses on positive symptoms, and accounts for sources of pseudoresistance, would also be useful in accelerating the identification of patients with TRS. However, a detailed description of such an assessment tool is beyond the scope of the current report. The authors plan on presenting a proposal in a future publication.

Burden of Treatment-Resistant Schizophrenia

There are substantial personal, societal, and economic burdens associated with TRS (Tables 2 and 3). Based on an online review of federal funding, TRS research funding is substantially lower than that provided for other psychiatric disorders (ie, bipolar disorder, major depressive disorder, and other domains in schizophrenia). The disease burden associated with TRS may be substantially reduced by early identification and effective treatment.

Current Treatment and Unmet Needs in Treatment-Resistant Schizophrenia

Consensus treatment guidelines for TRS recommend that patients be reevaluated at 12 weeks after a first episode of psychosis (after 2 antipsychotic trials of ≥ 6 weeks in duration at an adequate dose [ie, ≥ 600 mg chlorpromazine equivalents/d]; Table 3). However, in some circumstances (eg, high risk of suicide), a treatment duration of 2 weeks may be sufficient before considering if additional clinical intervention is needed. The use of alternative pharmacotherapies, including clozapine, or nonpharmacologic adjunctive treatments (eg, CBTp, ECT, rTMS, HIT) should be considered if treatment response is not optimal. It is also important to emphasize that treatment choices should be a shared decision-making process among the treatment team, the patient, and the patient’s family/caregivers.

CONCLUSIONS

The early identification of patients with TRS is critical for optimizing treatment because early recognition may help reduce the duration of persistent psychosis and improve long-term treatment outcomes. By increasing awareness and disseminating criteria for identifying TRS, those involved in the care of patients with schizophrenia will be able to identify TRS earlier in its course and make more informed treatment decisions. Considering the available evidence, it is anticipated that more timely and effective intervention for patients with TRS will result in improved long-term outcomes, reduced disease burden, and lower overall health care costs.

Published online: March 5, 2019.

Potential conflicts of interest: All authors have received financial support from Lundbeck. Dr Kane has been a consultant and/or advisor to or has received honoraria from Alkermes, Allergan, IntraCellular Therapies, Janssen/J&J, LB Pharma, Lundbeck, Medscape, Neurocrine, Otsuka, Pfizer, Pierre Fabre, Roche, Sunovion, Takeda, and Teva; has received grant support from Janssen, Lundbeck, and Otsuka; and is a shareholder of The Vanguard Research Group and LB Pharma. Dr Agid has been a consultant and/or advisor to or has received honoraria from Janssen-Ortho (Johnson & Johnson), Otsuka, Lundbeck, Sumitomo Dainippon Pharma (DSP), Mylan Pharmaceuticals, and HLS Therapeutics and has received grant support from Janssen-Ortho (Johnson & Johnson), Otsuka, Boehringer Ingelheim, and Neurocrine Biosciences. Dr Baldwin served as a consultant to Lundbeck. Dr Howes has received investigator-initiated research funding from and/or participated in advisory/speaker meetings organized by AstraZeneca, Autifony, Biogen, BMS, Eli Lilly, Heptares, Janssen, Lundbeck, Leyden Delta, Otsuka, Servier, Sunovion, Rand, and Roche. Neither Dr Howes nor his family have been employed by or have holdings/a financial stake in any biomedical company. Dr Lindenmayer has been a consultant for Alkermes, Janssen/J&J, Lundbeck, Neurocrine, Otsuka, and Newron and has received research grant support from Janssen, Neurocrine, Roche, Intracellular, Avanir, Takeda, and Alkermes. Dr Marder has been a consultant or advisory board member for Allergan, Boehringer Ingelheim, Lundbeck, Neurocrine, Newron, Otsuka, Roche, and Teva and has received research support from Boehringer Ingelheim, Neurocrine, and Takeda. Dr Olfson has worked on a grant from Janssen Scientific Affairs to Columbia University and served as a consultant to Lundbeck. Dr Potkin has been a consultant and/or advisor to or has received honoraria from Alkermes, Allergan, Gerson Lehrman Group, Lundbeck, Medscape, Merck, Otsuka, Pfizer, Sunovion, Takeda, Teva, Toyama Chemical Co, and Vanda and has received grant support from Allergan, Otsuka, Lundbeck, Takeda, and the University of California San Diego and University of Southern California. Dr Correll has been a consultant and/or advisor to or has received honoraria from Alkermes, Allergan, Gerson Lehrman Group, IntraCellular Therapies, Janssen/J&J, LB Pharma, Lundbeck, MedAvante, Medscape, Merck, Neurocrine, Otsuka, Pfizer, ROVI, Servier, Sunovion, Takeda, and Teva; has provided expert testimony for Bristol-Myers Squibb, Janssen, and Otsuka; served on a Data Safety Monitoring Board for Lundbeck, Pfizer, Roche, and ROVI; received royalties from UpToDate and grant support from Janssen and Takeda; and is a shareholder of LB Pharma.

Funding/support: The information reported in this article was derived from a roundtable sponsored by Lundbeck. Medical writing assistance was provided by the CHC Group (North Wales, PA) and was supported by Lundbeck.

Role of the sponsor: Lundbeck provided financial support for the conduct of the roundtable and for editorial support in the development of this article. Lundbeck is currently developing a potential treatment option for patients with treatment-resistant schizophrenia. The final content and consensus recommendations were the decisions of the authors.

Previous presentation: Poster presented at the 30th annual Psychiatric and Mental Health Congress; September 16-19, 2017; New Orleans, LA.

REFERENCES

1. World Health Organization. Schizophrenia. WHO website. http://www.who.int/mediacentre/factsheets/fs397/en. Accessed July 13, 2017.

2. National Collaborating Centre for Mental Health, NICE. Psychosis and Schizophrenia in Adults: Treatment and Management. National clinical guideline number 178. https://www.nice.org.uk/guidance/cg178. London, UK: National Institute for Health and Care Excellence; 2014.

3. Agid O, Arenovich T, Sajeev G, et al. An algorithm-based approach to first-episode schizophrenia: response rates over 3 prospective antipsychotic trials with a retrospective data analysis. J Clin Psychiatry. 2011;72(11):1439-1444. PubMed CrossRef

4. Yoshimura B, Sato K, Takaki M, et al. Algorithm-based pharmacotherapy for first-episode schizophrenia involuntarily hospitalized: a retrospective analysis of real-world practice. Early Interv Psychiatry. 2019;13(1):39-46. PubMed CrossRef

5. Robinson DG, Woerner MG, Alvir JMJ, et al. Predictors of treatment response from a first episode of schizophrenia or schizoaffective disorder. Am J Psychiatry. 1999;156(4):544-549. PubMed

6. Carbon M, Correll CU. Clinical predictors of therapeutic response to antipsychotics in schizophrenia. Dialogues Clin Neurosci. 2014;16(4):505-524. PubMed

7. Lieberman JA, Phillips M, Gu H, et al. Atypical and conventional antipsychotic drugs in treatment-naive first-episode schizophrenia: a 52-week randomized trial of clozapine vs chlorpromazine. Neuropsychopharmacology. 2003;28(5):995-1003. PubMed CrossRef

8. Meltzer HY. Treatment-resistant schizophrenia—the role of clozapine. Curr Med Res Opin. 1997;14(1):1-20. PubMed CrossRef

9. Lehman AF, Lieberman JA, Dixon LB, et al; American Psychiatric Association; Steering Committee on Practice Guidelines. Practice guideline for the treatment of patients with schizophrenia, second edition. Am J Psychiatry. 2004;161(2 suppl):1-56. PubMed

10. Agid O, Zipursky R, Siu C, et al. Antipsychotic re-challenge in previous responders. In: 53rd Annual Meeting of the American College of Neuropsychopharmacology, ACNP 2014; December 7-11, 2014; Phoenix, Arizona.

11. Kinon BJ, Kane JM, Chakos M, et al. Possible predictors of neuroleptic-resistant schizophrenic relapse: influence of negative symptoms and acute extrapyramidal side effects. Psychopharmacol Bull. 1993;29(3):365-369. PubMed

12. Correll CU, Rubio JM, Inczedy-Farkas G, et al. Efficacy of 42 pharmacologic cotreatment strategies added to antipsychotic monotherapy in schizophrenia: systematic overview and quality appraisal of the meta-analytic evidence. JAMA Psychiatry. 2017;74(7):675-684. PubMed CrossRef

13. Galling B, Roldán A, Hagi K, et al. Antipsychotic augmentation vs monotherapy in schizophrenia: systematic review, meta-analysis and meta-regression analysis. World Psychiatry. 2017;16(1):77-89. PubMed CrossRef

14. Yada Y, Yoshimura B, Kishi Y. Correlation between delay in initiating clozapine and symptomatic improvement. Schizophr Res. 2015;168(1-2):585-586. PubMed CrossRef

15. Lally J, Ajnakina O, Di Forti M, et al. Two distinct patterns of treatment resistance: clinical predictors of treatment resistance in first-episode schizophrenia spectrum psychoses. Psychol Med. 2016;46(15):3231-3240. PubMed CrossRef

16. Lieberman JA. Pathophysiologic mechanisms in the pathogenesis and clinical course of schizophrenia. J Clin Psychiatry. 1999;60(suppl 12):9-12. PubMed

17. Howes OD, McCutcheon R, Agid O, et al. Treatment-resistant schizophrenia: Treatment Response and Resistance in Psychosis (TRRIP) Working Group consensus guidelines on diagnosis and terminology. Am J Psychiatry. 2017;174(3):216-229. PubMed CrossRef

18. Hasan A, Falkai P, Wobrock T, et al; World Federation of Societies of Biological Psychiatry (WFSBP) Task Force on Treatment Guidelines for Schizophrenia. World Federation of Societies of Biological Psychiatry (WFSBP) Guidelines for Biological Treatment of Schizophrenia, part 1: update 2012 on the acute treatment of schizophrenia and the management of treatment resistance. World J Biol Psychiatry. 2012;13(5):318-378. PubMed CrossRef

19. Farooq S, Agid O, Foussias G, et al. Using treatment response to subtype schizophrenia: proposal for a new paradigm in classification. Schizophr Bull. 2013;39(6):1169-1172. PubMed CrossRef

20. Oda Y, Kanahara N, Iyo M. Alterations of dopamine D2 receptors and related receptor-interacting proteins in schizophrenia: the pivotal position of dopamine supersensitivity psychosis in treatment-resistant schizophrenia. Int J Mol Sci. 2015;16(12):30144-30163. PubMed CrossRef

21. Suzuki T, Kanahara N, Yamanaka H, et al. Dopamine supersensitivity psychosis as a pivotal factor in treatment-resistant schizophrenia. Psychiatry Res. 2015;227(2-3):278-282. PubMed CrossRef

22. Kimura H, Kanahara N, Sasaki T, et al. Risperidone long-acting injectable in the treatment of treatment-resistant schizophrenia with dopamine supersensitivity psychosis: results of a 2-year prospective study, including an additional 1-year follow-up. J Psychopharmacol. 2016;30(8):795-802. PubMed CrossRef

23. Silvestri S, Seeman MV, Negrete JC, et al. Increased dopamine D2 receptor binding after long-term treatment with antipsychotics in humans: a clinical PET study. Psychopharmacology (Berl). 2000;152(2):174-180. PubMed CrossRef

24. Abi-Dargham A, Rodenhiser J, Printz D, et al. Increased baseline occupancy of D2 receptors by dopamine in schizophrenia. Proc Natl Acad Sci U S A. 2000;97(14):8104-8109. PubMed CrossRef

25. Howes OD, Kapur S. A neurobiological hypothesis for the classification of schizophrenia: type A (hyperdopaminergic) and type B (normodopaminergic). Br J Psychiatry. 2014;205(1):1-3. PubMed CrossRef

26. Demjaha A, Murray RM, McGuire PK, et al. Dopamine synthesis capacity in patients with treatment-resistant schizophrenia. Am J Psychiatry. 2012;169(11):1203-1210. PubMed CrossRef

27. Mouchlianitis E, McCutcheon R, Howes OD. Brain-imaging studies of treatment-resistant schizophrenia: a systematic review. Lancet Psychiatry. 2016;3(5):451-463. PubMed CrossRef

28. Demjaha A, Egerton A, Murray RM, et al. Antipsychotic treatment resistance in schizophrenia associated with elevated glutamate levels but normal dopamine function. Biol Psychiatry. 2014;75(5):e11-e13. PubMed CrossRef

29. Yin J, Barr AM, Ramos-Miguel A, et al. Antipsychotic induced dopamine supersensitivity psychosis: a comprehensive review. Curr Neuropharmacol. 2017;15(1):174-183. PubMed CrossRef

30. Kane JM. Improving patient outcomes in schizophrenia: achieving remission, preventing relapse, and measuring success. J Clin Psychiatry. 2013;74(9):e18. PubMed CrossRef

31. Correll CU, Rubio JM, Kane JM. What is the risk-benefit ratio of long-term antipsychotic treatment in people with schizophrenia? World Psychiatry. 2018;17(2):149-160. PubMed

32. Pollak TA, McCormack R, Peakman M, et al. Prevalence of anti-N-methyl-D-aspartate (NMDA) receptor [corrected] antibodies in patients with schizophrenia and related psychoses: a systematic review and meta-analysis. Psychol Med. 2014;44(12):2475-2487. PubMed CrossRef

33. Iasevoli F, Giordano S, Balletta R, et al. Treatment resistant schizophrenia is associated with the worst community functioning among severely-ill highly-disabling psychiatric conditions and is the most relevant predictor of poorer achievements in functional milestones. Prog Neuropsychopharmacol Biol Psychiatry. 2016;65:34-48. PubMed CrossRef

34. Frydecka D, BeszŠ‚ej JA, Gościmski P, et al. Profiling cognitive impairment in treatment-resistant schizophrenia patients. Psychiatry Res. 2016;235:133-138. PubMed CrossRef

35. Ascher-Svanum H, Nyhuis AW, Faries DE, et al. Clinical, functional, and economic ramifications of early nonresponse to antipsychotics in the naturalistic treatment of schizophrenia. Schizophr Bull. 2008;34(6):1163-1171. PubMed CrossRef

36. Flyckt L, Löthman A, Jörgensen L, et al. Burden of informal care giving to patients with psychoses: a descriptive and methodological study. Int J Soc Psychiatry. 2013;59(2):137-146. PubMed CrossRef

37. Koutra K, Triliva S, Roumeliotaki T, et al. Family functioning in families of first-episode psychosis patients as compared to chronic mentally ill patients and healthy controls. Psychiatry Res. 2014;219(3):486-496. PubMed CrossRef

38. Chan SKW, Kao SYS, Leung SL, et al. Relationship between neurocognitive function and clinical symptoms with self-stigma in patients with schizophrenia-spectrum disorders [published online ahead of print June 23, 2017]. J Ment Health. PubMed CrossRef

39. Hill K, Startup M. The relationship between internalized stigma, negative symptoms and social functioning in schizophrenia: the mediating role of self-efficacy. Psychiatry Res. 2013;206(2-3):151-157. PubMed CrossRef

40. Torrey EF. Stigma and violence: isn’ t it time to connect the dots? Schizophr Bull. 2011;37(5):892-896. PubMed CrossRef

41. Torrey EF, Wolfe SM, Erdman K, et al. Care of the Seriously Mentally Ill: A Rating of State Programs. Washington, DC: Public Citizen Health Research Group: National Health Alliance for the Mentally Ill; 1990.

42. Nielsen J, Dahm M, Lublin H, et al. Psychiatrists’ attitude towards and knowledge of clozapine treatment. J Psychopharmacol. 2010;24(7):965-971. PubMed CrossRef

43. Meltzer HY, Cola P, Way L, et al. Cost effectiveness of clozapine in neuroleptic-resistant schizophrenia. Am J Psychiatry. 1993;150(11):1630-1638. PubMed CrossRef

44. Blieden N, Flinders S, Hawkins K, et al. Health status and health care costs for publicly funded patients with schizophrenia started on clozapine. Psychiatr Serv. 1998;49(12):1590-1593. PubMed CrossRef

45. Peng X, Ascher-Svanum H, Faries DE, et al. Cost-effectiveness of early responders versus early nonresponders to atypical antipsychotic therapy. Clinicoecon Outcomes Res. 2011;3:79-87. PubMed CrossRef

46. Kennedy JL, Altar CA, Taylor DL, et al. The social and economic burden of treatment-resistant schizophrenia: a systematic literature review. Int Clin Psychopharmacol. 2014;29(2):63-76. PubMed CrossRef

47. Gören JL, Rose AJ, Smith EG, et al. The business case for expanded clozapine utilization. Psychiatr Serv. 2016;67(11):1197-1205. PubMed CrossRef

48. Rosenheck R, Leslie D, Keefe R, et al; CATIE Study Investigators Group. Barriers to employment for people with schizophrenia. Am J Psychiatry. 2006;163(3):411-417. PubMed CrossRef

49. Salkever DS, Karakus MC, Slade EP, et al. Measures and predictors of community-based employment and earnings of persons with schizophrenia in a multisite study. Psychiatr Serv. 2007;58(3):315-324. PubMed CrossRef

50. McGurk SR, Mueser KT, Harvey PD, et al. Cognitive and symptom predictors of work outcomes for clients with schizophrenia in supported employment. Psychiatr Serv. 2003;54(8):1129-1135. PubMed CrossRef

51. Bilder RM, Goldman RS, Volavka J, et al. Neurocognitive effects of clozapine, olanzapine, risperidone, and haloperidol in patients with chronic schizophrenia or schizoaffective disorder. Am J Psychiatry. 2002;159(6):1018-1028. PubMed CrossRef

52. Ciapparelli A, Dell’ Osso L, Bandettini di Poggio A, et al. Clozapine in treatment-resistant patients with schizophrenia, schizoaffective disorder, or psychotic bipolar disorder: a naturalistic 48-month follow-up study. J Clin Psychiatry. 2003;64(4):451-458. PubMed CrossRef

53. Social Security Administration. Annual statistical report on the Social Security Disability Insurance Program, 2015. Social Security Administration website. https://www.ssa.gov/policy/docs/statcomps/di_asr/2015/di_asr15.pdf. Accessed August 16, 2017.

54. Social Security Administration. SSI annual statistical report, 2015. Social Security Administration website. https://www.ssa.gov/policy/docs/statcomps/ssi_asr/2015/ssi_asr15.pdf. Accessed September 25, 2017.

55. Walkup JT, McAlpine DD, Olfson M, et al. Patients with schizophrenia at risk for excessive antipsychotic dosing. J Clin Psychiatry. 2000;61(5):344-348. PubMed CrossRef

56. Patel MX, Bishara D, Jayakumar S, et al. Quality of prescribing for schizophrenia: evidence from a national audit in England and Wales. Eur Neuropsychopharmacol. 2014;24(4):499-509. PubMed CrossRef

57. Paton C, Barnes TR, Cavanagh MR, et al; POMH-UK project team. High-dose and combination antipsychotic prescribing in acute adult wards in the UK: the challenges posed by p.r.n. prescribing. Br J Psychiatry. 2008;192(6):435-439. PubMed CrossRef

58. Howes OD, Vergunst F, Gee S, et al. Adherence to treatment guidelines in clinical practice: study of antipsychotic treatment prior to clozapine initiation. Br J Psychiatry. 2012;201(6):481-485. PubMed CrossRef

59. Üçok A, Çikrikçili U, Karabulut S, et al. Delayed initiation of clozapine may be related to poor response in treatment-resistant schizophrenia. Int Clin Psychopharmacol. 2015;30(5):290-295. PubMed

60. Agid O, Remington G, Kapur S, et al. Early use of clozapine for poorly responding first-episode psychosis. J Clin Psychopharmacol. 2007;27(4):369-373. PubMed CrossRef

61. Agid O, Schulze L, Arenovich T, et al. Antipsychotic response in first-episode schizophrenia: efficacy of high doses and switching. Eur Neuropsychopharmacol. 2013;23(9):1017-1022. PubMed CrossRef

62. Alessi-Severini S, Le Dorze JA, Nguyen D, et al. Clozapine prescribing in a Canadian outpatient population. PLoS One. 2013;8(12):e83539. PubMed CrossRef

63. Taylor DM, Young C, Paton C. Prior antipsychotic prescribing in patients currently receiving clozapine: a case note review. J Clin Psychiatry. 2003;64(1):30-34. PubMed CrossRef

64. Wheeler AJ. Treatment pathway and patterns of clozapine prescribing for schizophrenia in New Zealand. Ann Pharmacother. 2008;42(6):852-860. PubMed CrossRef

65. Volavka J, Czobor P, Sheitman B, et al. Clozapine, olanzapine, risperidone, and haloperidol in the treatment of patients with chronic schizophrenia and schizoaffective disorder. Am J Psychiatry. 2002;159(2):255-262. PubMed CrossRef

66. Velligan DI, Carroll C, Lage MJ, et al. Outcomes of Medicaid beneficiaries with schizophrenia receiving clozapine only or antipsychotic combinations. Psychiatr Serv. 2015;66(2):127-133. PubMed CrossRef

67. Nielsen J, Correll CU, Manu P, et al. Termination of clozapine treatment due to medical reasons: when is it warranted and how can it be avoided? J Clin Psychiatry. 2013;74(6):603-613, quiz 613. PubMed CrossRef

68. Yoshimura B, Yada Y, So R, et al. The critical treatment window of clozapine in treatment-resistant schizophrenia: secondary analysis of an observational study. Psychiatry Res. 2017;250:65-70. PubMed CrossRef

69. Siskind D, Siskind V, Kisely S. Clozapine response rates among people with treatment-resistant schizophrenia: data from a systematic review and meta-analysis. Can J Psychiatry. 2017;62(11):772-777. PubMed CrossRef

70. Kahn RS, Winter van Rossum I, Leucht S, et al; OPTiMiSE study group. Amisulpride and olanzapine followed by open-label treatment with clozapine in first-episode schizophrenia and schizophreniform disorder (OPTiMiSE): a three-phase switching study. Lancet Psychiatry. 2018;5(10):797-807. PubMed CrossRef

71. Samara MT, Dold M, Gianatsi M, et al. Efficacy, acceptability, and tolerability of antipsychotics in treatment-resistant schizophrenia: a network meta-analysis. JAMA Psychiatry. 2016;73(3):199-210. PubMed CrossRef

72. Kane JM, Correll CU. The role of clozapine in treatment-resistant schizophrenia. JAMA Psychiatry. 2016;73(3):187-188. PubMed CrossRef

73. Siskind D, McCartney L, Goldschlager R, et al. Clozapine v. first- and second-generation antipsychotics in treatment-refractory schizophrenia: systematic review and meta-analysis. Br J Psychiatry. 2016;209(5):385-392. PubMed CrossRef

74. Tiihonen J, Lönnqvist J, Wahlbeck K, et al. 11-year follow-up of mortality in patients with schizophrenia: a population-based cohort study (FIN11 study). Lancet. 2009;374(9690):620-627. PubMed CrossRef

75. Tiihonen J, Mittendorfer-Rutz E, Majak M, et al. Real-world effectiveness of antipsychotic treatments in a nationwide cohort of 29823 patients with schizophrenia. JAMA Psychiatry. 2017;74(7):686-693. PubMed CrossRef

76. Wykes T, Steel C, Everitt B, et al. Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophr Bull. 2008;34(3):523-537. PubMed CrossRef

77. Burns AMN, Erickson DH, Brenner CA. Cognitive-behavioral therapy for medication-resistant psychosis: a meta-analytic review. Psychiatr Serv. 2014;65(7):874-880. PubMed CrossRef

78. Slotema CW, Blom JD, van Lutterveld R, et al. Review of the efficacy of transcranial magnetic stimulation for auditory verbal hallucinations. Biol Psychiatry. 2014;76(2):101-110. PubMed CrossRef

79. Zheng W, Cao X-L, Ungvari GS, et al. Electroconvulsive therapy added to non-clozapine antipsychotic medication for treatment resistant schizophrenia: meta-analysis of randomized controlled trials. PLoS One. 2016;11(6):e0156510. PubMed CrossRef

80. Petrides G, Malur C, Braga RJ, et al. Electroconvulsive therapy augmentation in clozapine-resistant schizophrenia: a prospective, randomized study. Am J Psychiatry. 2015;172(1):52-58. PubMed CrossRef

81. Jenner JA, Nienhuis FJ, Wiersma D, et al. Hallucination focused integrative treatment: a randomized controlled trial. Schizophr Bull. 2004;30(1):133-145. PubMed CrossRef

82. Lee J, Takeuchi H, Fervaha G, et al. Subtyping schizophrenia by treatment response: antipsychotic development and the central role of positive symptoms. Can J Psychiatry. 2015;60(11):515-522. PubMed CrossRef

83. Wang G, Zheng W, Li XB, et al. ECT augmentation of clozapine for clozapine-resistant schizophrenia: a meta-analysis of randomized controlled trials. J Psychiatr Res. 2018;105:23-32. PubMed CrossRef

84. Morrison AP, Pyle M, Gumley A, et al; FOCUS trial group. Cognitive behavioural therapy in clozapine-resistant schizophrenia (FOCUS): an assessor-blinded, randomised controlled trial. Lancet Psychiatry. 2018;5(8):633-643. PubMed CrossRef

85. Gee SH, Shergill SS, Taylor DM. Patient attitudes to clozapine initiation. Int Clin Psychopharmacol. 2017;32(6):337-342. PubMed CrossRef

86. Marcus SC, Zummo J, Pettit AR, et al. Antipsychotic adherence and rehospitalization in schizophrenia patients receiving oral versus long-acting injectable antipsychotics following hospital discharge. J Manag Care Spec Pharm. 2015;21(9):754-768. PubMed

87. Kane JM, Robinson DG, Schooler NR, et al. Comprehensive versus usual community care for first-episode psychosis: 2-year outcomes from the NIMH RAISE Early Treatment Program. Am J Psychiatry. 2016;173(4):362-372. PubMed CrossRef

88. Hughes A, Singh B. Clozapine clinic: the need of the hour. Am J Psychiatry. 2016;2016(11):3. CrossRef