From the Editor
Does decriminalizing the possession of small amounts of street drugs reduce overdoses? Proponents argue yes because those who use substances can seek care – including in emergency situations – without fear of police involvement and charges. Opponents counter that decriminalization means fewer penalties for drug use, resulting in more misuse and thus more overdoses. The debate can be shrill – but lacking in data.
Spruha Joshi (of New York University) and co-authors bring numbers to the policy discussion with a new JAMA Psychiatry paper. They analyze the impact of decriminalization in two states, Oregon and Washington, contrasting overdoses there and in other US states that didn’t decriminalize. “This study found no evidence of an association between legal changes that removed or substantially reduced criminal penalties for drug possession in Oregon and Washington and fatal drug overdose rates.” We consider the paper and its implications.
In the second selection, Dr. Ashwin Nayak (of Stanford University) and his co-authors look at AI for the writing of patient histories. In a new research letter for JAMA Internal Medicine, they do a head-to-head (head-to-CPU?) comparison with ChatGPT and residents both writing patient histories (specifically, the history of present illness, or HPI). “HPIs generated by a chatbot or written by senior internal medicine residents were graded similarly by internal medicine attending physicians.”
And in the third selection, medical student Howard A. Chang (of Johns Hopkins University) wonders about “good” psychiatry in a paper for Academic Psychiatry. He reflects on the comments of surgeons, pediatricians, and obstetricians, and then mulls the role of our specialty. “I have gleaned that a good psychiatrist fundamentally sees and cares about patients with mental illness as dignified human beings, not broken brains. The good psychiatrist knows and treats the person in order to treat the disease.”
Selection 1: “One-Year Association of Drug Possession Law Change With Fatal Drug Overdose in Oregon and Washington”
Spruha Joshi, Bianca D. Rivera, Magdalena Cerdá, et al.
JAMA Psychiatry, 27 September 2023
The overdose crisis in the US continues to be a leading factor in potentially preventable morbidity and mortality. Nationwide, there were 106 699 drug overdose deaths in 2021, a 16% increase over 2020… [I]t has been hypothesized that removing criminal penalties for possession of currently criminalized drugs may reduce drug overdose fatality and other drug-related harm, particularly in the context of an illicit drug market highly contaminated with fentanyl and fentanyl analogues. Data from numerous states including Oregon show that persons who use drugs (PWUD) are often reluctant to call 911 during an overdose due to fear of arrest and other negative criminal-legal outcomes. Removing these penalties may increase calls for help and reduce arrest rates and related incarceration leading to lower fatal overdose rates. Alternatively, removing criminal penalties for drug possession might increase overdose deaths by removing some disincentives for drug use…
On February 1, 2021, a citizen initiative termed Measure 110 went into effect in Oregon. This law, approved by over 58% of Oregon voters, changed the possession of small amounts of all previously criminalized drugs from a crime to a noncriminal Class E violation… Shortly thereafter, Washington arrived at a similar policy change through a combination of a court decision and ensuing action by the state legislature.”
So begins a paper by Joshi et al.
Here’s what they did:
- They conducted a cohort study using “a synthetic control method approach to examine whether there were changes in drug possession laws and fatal drug overdose rates in Oregon and Washington in the postpolicy period…”
- “A counterfactual comparison group (synthetic controls) was created for Oregon and Washington, using 48 states and the District of Columbia, that did not implement similar policies during the study period (January 1, 2018, to March 31, 2022). For 2018-2021, final multiple cause-of-death data from the National Vital Statistics System (NVSS) were used. For 2022, provisional NVSS data were used.”
- Main outcome: Monthly fatal drug overdose rates.
Here’s what they found:
- Oregon. “Following the implementation of Measure 110, absolute monthly rate differences between Oregon and its synthetic control were not statistically significant (probability = 0.26). The average rate difference post Measure 110 was 0.268 fatal drug overdoses per 100 000 state population.”
- Washington. “Following the implementation of the policy change in Washington, the absolute monthly rate differences between Washington and synthetic Washington were not statistically significant (probability = 0.06). The average rate difference post Blake was 0.112 fatal drug overdoses per 100 000 state population.”
A few thoughts:
1. This is a good paper.
2. The main finding in a sentence: “we found no evidence that either Measure 110 in Oregon or the Washington Blake decision and subsequent legislative amendments were associated with changes in fatal drug overdose rates in either state.”
4. How to explain small changes? The authors conducted permutation-based tests for both the Oregon and Washington data, finding that “chance not policy change” explains differences.
5. This study analyzed early data – 12 months worth. What will things look like in two or three years? Further research will be important.
6. And it will be interesting to study other experiments with decriminalization, including British Columbia’s (now in its tenth month).
The full JAMA Psych paper can be found here:
Selection 3: “Comparison of History of Present Illness Summaries Generated by a Chatbot and Senior Internal Medicine Residents”
Ashwin Nayak, Matthew S. Alkaitis, Kristen Nayak, et al.
JAMA Internal Medicine, 17 July 2023
Large language model (LLM) chatbots have received widespread attention for their ability to produce complex human-like conversational output. These models represent a substantial advancement in generative artificial intelligence (AI) with potential applications in many industries. Medical documentation is a health care use case worth examining given its notable burden on clinicians. In this prognostic study, we evaluated the ability of a chatbot to generate a history of present illness (HPI) compared with senior internal medicine residents.
So begins an research letter by Nayak et al.
Here’s what they did:
- The HPIs were generated by ChatGPT and four residents based on three patient interview scripts for different types of chest pains.
- “The HPIs produced by the chatbot were generated using an iterative process known as prompt engineering. Ten HPIs per script were first generated using a basic prompt and analyzed for errors. The HPIs were considered acceptable if they contained no errors. The prompt was then modified, and this process was repeated twice.”
- One AI-generated HPI was then selected from the final round of prompt engineering and compared to the residents’ scripts.
- 30 attending physicians then graded them “on level of detail, succinctness, and organization. They also stated whether they believed HPIs were produced by a resident or chatbot.”
Here’s what they found:
- Grades. “Grades of resident and chatbot-generated HPIs differed by less than 1 point on the 15-point composite scale (resident mean [SD], 12.18 [2.40] vs chatbot mean [SD], 11.23 [2.84]…), although resident HPIs scored higher on level of detail scale (resident mean [SD], 4.13 [0.86] vs chatbot mean [SD], 3.57 [1.04]…).”
- Errors. “The most common error was addition of patient age and gender, which none of the scripts specified.” (!)
- Acceptance. “The acceptance rate for chatbot-generated HPIs improved from 10.0% to 43.3% by the final round of prompt engineering.”
- Classification. “Attending physicians correctly classified HPIs as written by residents or the chatbot with an accuracy of 61%…”
A few thoughts:
1. This is an interesting research letter.
2. The finding in a sentence: ChatGPT could generate comparable HPIs – grades of resident and chatbot-generated HPIs differed by less than 1 point on a 15-point scale – but with AI needing significant help.
3. Hallucinations continue to be a problem. When details weren’t available (like the patient’s age), ChatGPT often made up the information. It’s not ideal when ChatGPT hallucinates a citation for your cousin’s grade 10 English essay; it’s completely unacceptable when it “creates” information in patient histories.
4. There were several limitations, including: “only the best chatbot outputs were compared to resident performance.”
5. No need to retrain just yet – AI needed significant help. Still, perspective: the drafting of an HPI is more than just repeating information, it needs to synthesize that information. As the lead author notes in an interview with another publication: “It takes medical students and residents years to learn.”
6. Past Readings have considered AI and clinical work. In a JAMA Internal Medicine paper, Ayers et al. compared ChatGPT answers to those of physicians in terms of quality and empathy for basic, Reddit-style medical questions. “In this cross-sectional study, a chatbot generated quality and empathetic responses to patient questions posed in an online forum.” That Reading can be found here:
The full JAMA Internal Medicine research letter can be found here:
Selection 3: “More (Good) Psychiatry”
Howard A. Chang
Academic Psychiatry, June 2023
During my clerkships as a medical student, I discovered an unusual similarity among internists, surgeons, pediatricians, obstetricians, and gynecologists: their reply when I informed them that I was interested in psychiatry.
‘We need more good psychiatrists.’
I heard this statement often, and initially it baffled me. Don’t we need more good doctors in all areas of medicine?
So begins an essay by Chang.
He writes about the comments of physicians in other specialties. “Frequently, they offered some personal or professional insight into psychiatry. Sometimes they shared how their patients’ psychiatric comorbidities impeded treatment of their medical problems – for instance, the forgetfulness in depression that reduced medication adherence, the substance use that damaged patients’ health, or the personality disorder that garnered physician bias.” A conversation is particularly memorable. “One bariatric surgeon remarked that the operation is usually unsuccessful in patients who are unwilling to optimize their diet and exercise. Depression, common in the postoperative setting, makes it especially challenging to perform healthy behaviors and thus predicts worse weight loss outcomes following bariatric surgery. Broadly speaking, serious mental illness increases the odds of postoperative complications and longer hospital stays.”
He notes the challenges that patients have in accessing mental health services. “Over 50% of Americans want access to mental healthcare but many face barriers such as high costs, inadequate insurance coverage, and limited options.” He sees several solutions, including more residency spots for psychiatry.
But he argues that while more psychiatrists are needed, other things must be done. Psychotherapy is critical: “the practice settings of many psychiatrists may not permit them to offer dedicated psychotherapy sessions.” He continues: “What all psychiatrists should offer patients is supportive psychotherapy, which involves building therapeutic alliance to help patients deal with psychosocial and emotional problems through empathic listening, encouragement, validation, comfort, reframing, advice, and other interpersonal techniques.”
How can he be a good psychiatrist? “I can strive to practice good psychiatry by exploring my patients’ interests and hobbies, not simply their signs and symptoms. I can ask them what their illness feels like and means to them, rather than assuming that I already know from referencing the Diagnostic and Statistical Manual of Mental Disorders. I can read about the lived experiences of those with mental illness, practice greater empathy, and engage with families and caregivers. I can emulate excellent psychiatrists in how they practice supportive psychotherapy. I can also meditate daily on my values and care for my own mental health to improve my ability to be fully present with patients.”
A few thoughts:
1. This is a good essay.
2. Chang has a great career ahead of him.
3. He makes a particularly nice point about the criticism that psychiatry often faces. “I could not recall ever hearing a non-cardiologist argue that we need better medications in cardiology or a non-pulmonologist remark that we need greater diagnostic acumen in pulmonology.” He argues that psychiatry is “more vulnerable” to judgment. Is he right?
The full Acad Psychiatry paper can be found here:
Reading of the Week. Every week I pick articles and papers from the world of Psychiatry.