From the Editor
Here’s a quick statistical summary of the Readings for the past 12 months.
Number discussing the prevention of mental illness: One.
Like all of medicine, psychiatry tends to emphasize the treatment of illness, not its prevention. This isn’t the result of a vast medical-industrial conspiracy, of course, but the reality that our field is young and the causes of mental illness aren’t well understood.
But preventing illness is our ultimate goal. Consider the suffering and cost that could be avoided if a person at risk of psychosis didn’t convert, as an example.
Can we prevent psychotic illness?
Prevention is built on two things: we need to identify at risk individuals, and then we need to use appropriate measures to prevent the illness.
Over the next two weeks, we look at a few papers that seek to identify at risk individuals and prevent psychosis in them.
This week. The psychosis risk calculator.
Next week. Cost-effective prevention.
In this week’s paper from The American Journal of Psychiatry, Cannon et al. develop a risk calculator to predict psychotic disorder. The tool they develop has an accuracy rate of 71% – comparable to calculators used for determining cancer recurrence.
Predicting Risk for Psychotic Illness
“An Individualized Risk Calculator for Research in Prodromal Psychosis”
Tyrone D. Cannon et al., The American Journal of Psychiatry
Online First, 1 July 2016
Given limitations of available treatments for schizophrenia, with most patients showing substantial deficits in social and occupational functioning throughout life, there is considerable interest in developing preventive approaches to psychotic disorders. Ascertainment of individuals at greatest risk is crucial to these efforts. For the majority of patients, onset of fully psychotic symptoms is preceded by the emergence of subtler changes in belief, thought, and perception that appear to represent attenuated forms of delusions, formal thought disorder, and hallucinations, respectively. Among individuals 12–35 years old with a recent onset of such symptoms (termed clinical high-risk cases), approximately 20%−35% develop fully psychotic symptoms over a 2-year period, an incidence rate that is more than 100 times larger than in the same age band in the general population. Furthermore, it appears that the clinical high-risk criteria are sensitive to an imminent risk for onset, as most of the conversions occur during the first year after ascertainment, with a decelerating conversion rate thereafter.
Although clinical high-risk criteria have been validated in epidemiological studies as sensitive to conversion risk, their utility in individual decision making is currently limited, given that 65%–80% of cases ascertained by these methods do not convert to psychosis within a 2-year time frame. About a dozen studies have examined combinations of clinical and demographic variables to determine whether prediction of psychosis can be enhanced beyond the 20%−35% risk associated with clinical high-risk status. Multivariate algorithms requiring particular combinations of symptoms and demographic factors achieve relatively high positive predictive values and specificity (e.g., in the 50%−70% range) but low sensitivity (e.g., in the 10%−30% range). There is consistency among studies in showing (unsurprisingly) that greater severity of the psychosis-risk symptoms at baseline is the best predictor of conversion; nevertheless, the most predictive multivariate profiles vary across studies. Although it should be noted that few studies have attempted direct replication of one another’s risk algorithms, this pattern suggests heterogeneity among profiles of clinical and demographic risk indicators among patients who convert to psychosis.
To maximize clinical utility, we require an approach that can be applied to scale the risk in an individual patient at the initial clinical contact.
So opens a new paper just published in The American Journal of Psychiatry (Online First).
This paper is relatively straight-forward, but has far-reaching implications. After all, if we can clearly identify people at risk of psychotic illness, it would make it possible to prevent illness in these individuals.
Here’s what the author did:
· Essentially, drawing data from the second phase of the North American Prodrome Longitudinal Study (NAPLS-2), the authors studied 596 high-risk participants to the point of conversion or the last follow up (at two years).
· Conversion to psychosis was determined by SIPS (Structured Interview for Prodromal Syndromes) criteria.
· Clinical evaluations were done 6 months after people entered the study and through to 2 years.
· Exclusion criteria included low IQ, substance dependence, neurological disorder, and a history of a psychotic disorder. Also, those lost to follow up were dropped.
· In terms of the selection of measures: “our focus was on demographic, clinical, neurocognitive, and functioning measures that are easily administered in general clinical settings.” The authors reviewed the literature, and settled on several measures, including age, family history, the Global Functioning: Social scale.
· Complicated statistical analysis was done, with the authors building a multivariate proportional hazards model to predict conversion.
Here’s what they found:
· “Of the 596 participants for whom follow-up data were available, 84 converted to psychosis within the 2-year study period.” The mean age: 18.5 years. The time to conversion was 7.3 months.
· The 2-year probability of conversion to psychosis was 0.16.
· Which measures were useful? “Prodromal symptom severity (SIPS items P1 and P2, modified and summed), decline in social functioning, and verbal learning and memory (Hopkins Verbal Learning Test–Revised scores) were significant predictors…”
· In contrast: “Stressful life events, traumas, and family history of schizophrenia were not significant predictors in univariate or multivariate analyses.”
· “Based on the bootstrap internal validation, the multivariate model achieved a C-index of 0.71. As shown in Figure 2, the calibration plot revealed a high degree of consistency between observed probabilities and model-predicted probabilities of conversion to psychosis within the range of 0.0–0.4…” See below.
The goal of this study was to develop a practical tool for the individualized prediction of psychosis in clinical high-risk patients. A well-performing risk calculator was generated from the NAPLS-2 cohort data using a small number of demographic (age, family history of psychosis), clinical (unusual thought content and suspiciousness), neurocognitive (verbal learning and memory, speed of processing), and psychosocial (traumas, stressful life events, decline in social functioning) predictor variables.
The overall model achieved a C-index of 0.71,which is in the range of values for established calculators currently in use for cardiovascular disease and cancer recurrence risk, which range from 0.58 to 0.81.
In an accompanying paper, Ricardo E. Carrión et al. use the risk calculator for a sample of high-risk individuals for “external validation.”
This prediction tool represents a potential breakthrough for early intervention in psychiatry. However, as with any predictive analytic model, its performance must be validated in samples of clinical high-risk patients collected independently of NAPLS-2.
His team set out to do just that using data from the Early Detection, Intervention, and Prevention of Psychosis Program (EDIPPP). The overall model accuracy rate was 79% in the EDIPPP sample.
The full paper is here:
A few thoughts:
1. The Cannon et al. is an important paper. For decades, researchers have worked to identify risk factors; the risk calculator is a big step forward.
2. The risk calculator withstands – to use banking terminology – a stress test: it works well with an independent data set, as Carrión et al. show.
3. Of course, the risk calculator is just the start. Remember: this isn’t intended for your office. Participants had two structured interviews (the Structured Interview for Prodromal Syndromes (SIPS) and the Structured Clinical Interview for DSM-IV Axis I Disorders). The risk calculator is a research tool.
4. 71%. Now imagine if we could add in biomarkers to create an even better risk calculator.
5. We also need to be very careful here. Identifying risk is potentially important – a step towards prevention. It also opens up ethical questions. I’m not criticizing the authors in any way, but am noting that risk calculators in any field of medicine carry, well, risk.
Reading of the Week. Every week I pick articles and papers from the world of Psychiatry.