what is the Smallest Worthwhile Difference?

Antidepressants do work (Cipriani et al., 2018; Tomlin, 2018) but they can also be problematic. Whilst they are very helpful for some, others find little difference in their mood or even feel worse. Antidepressants have a troubling range of side effects and can be difficult to stop without experiencing symptoms of withdrawal (Bowers, 2024; Read et al., 2023). Subsequently, how worthwhile they are, weighing up both the burdens and benefits, remains controversial. Reported effect size doesn’t help us to answer this question as this fails to consider the patient’s perspective on the significance of benefits, and neither does calculating the Minimum Important Change (MIC; the smallest change after treatment in an important health outcome) as this doesn’t account for burdens and benefits in comparison to trying different treatment options.

This current study by Sahker and colleagues (2024) reports an attempt to estimate the Smallest Worthwhile Difference (SWD) of antidepressants, which is defined as the “smallest beneficial effect of an intervention that justifies the costs, risks and inconveniences of that intervention” (Ferreira et al., 2012).  It is “patient-derived, intervention-specific and expressed as an absolute difference between treatment options” and has never been previously estimated for antidepressants.

Sahker and colleagues attempted to estimate the Smallest Worthwhile Difference (SWD) of antidepressants, which refers to the smallest beneficial effect that justifies the costs, risks and inconveniences of antidepressants.

Methods

The study design was an online cross-sectional survey, completed by participants in the general population who were experiencing symptoms of depression but were not currently in treatment.

Participants were recruited via Prolific, MQ Mental Health and Amazon Mechanical Turk research participants crowdsourcing services in both the USA and UK between October 2022 and January 2023, and had the following characteristics:

• N = 935
• Mean age: 44.1 years (SD = 13.9)
• Proportion of women: 66% (n = 617)
• Caucasian: 84% (n = 785)

The primary outcome was the SWD. The authors presented a summary of major depressive episode symptoms and explained the benefits and burdens of antidepressants treatment and the natural course alternatives (based on DSM-5 and Food and Drug Administration sources). The authors provided an estimate that response (defined as 50% or greater reduction in symptom severity) occurs in approximately 30% of people after 2 months without treatment. They asked participants about their beliefs and attitudes towards antidepressants, to weigh up the benefits and burdens and decide if they would accept the drug.  The SWD for each participant was calculated as the difference between 30% and the minimum antidepressant response the participant would consider.

Results

Of the 935 participants recruited to the study:

• 124 had moderate-to-severe symptoms of depression (PHQ -9 ≥ 10) but were not in treatment.
• 390 were in treatment (including medication and psychotherapy).
• 495 had absent-to-mild symptoms (PHQ -9<10) with or without treatment experiences.
• Of the whole sample, 10.2% (n = 95) would not consider antidepressants even if they achieved 100% response. These participants were removed from the main analyses, leaving 840 participants.

If the estimated response without receiving treatment is 30% and the likely response rate for medication is 45% (also estimated by the authors from a review of the literature), the most realistic response rate difference between the most efficacious treatment currently available and no treatment is 15%. The SWD reported by participants according to depression symptoms and treatment experience was as follows:

Note: IQR = interquartile range.

Across groups, all median SWDs were larger than the 15% realistic response rate difference calculated by the authors. This means that, on average across the entire sample, all participants would require antidepressants to have a higher SWD than currently estimated.

However, there was wide variability in individual SWDs. People currently in treatment showed the largest variability, and those with absent-to-mild depressive symptoms without treatment experiences showed the smallest variability.

• Approximately 1/3 of both those with moderate-to-severe symptoms not in treatment and the entire sample would be willing to take antidepressants for depression at the current expected response rate (15%), with the expected burdens.
• Another 1/3 would need double the current antidepressant effect (30%) before they started treatment.
• 1/3 would need even greater benefits or fewer.

The only important predictor for the SWD was the participant’s preference for antidepressants or psychotherapy, with a median of 20% (IQR = 10-35%) vs 25% (IQR = 15 to 35%) respectively. Clinical need and lived experience did not predict SWD.

The study found that one in three people with moderate-to-severe symptoms of depression who are not already in treatment would consider antidepressants worth taking despite the associated burdens.

Conclusions

The authors conclude that:

Our observed SWDs mean that the current 15% antidepressant benefit over no treatment was sufficient for one in three people to accept antidepressants given the burdens, but two in three expected greater treatment benefits.

They suggest the SWD could provide a useful benchmark for future antidepressant drugs in comparison to placebo and that we need more research estimating the SWD for other treatments, disorders and diseases.

Two in three individuals expected greater benefits or fewer burdens from antidepressants in order to consider them as a treatment option.

Strengths and limitations

The authors acknowledge the limitations of participants from crowdsourcing services who tend to be younger, more educated and report more symptoms. They recruited a large sample from different countries with differing healthcare systems, using recognised ways of improving data quality, and have provided readers with the complete script, including the information about benefits and burdens, used in the survey. Their method is much more generalisable than convenience samples (advertising a survey online for instance) and the MQ participant pool particularly includes people with lived experience in mental health and healthcare professionals who volunteer to improve representation in research.

The exclusion of those who replied that they would never take antidepressants also seemed very reasonable.

However, PHQ-9 score of moderate-to-severe symptom severity is only a proxy for a potential diagnosis of depression. No doubt, these people might be “personally invested because of their current depressive symptoms and potential treatment needs”. However, they would not necessarily be people who would ever be offered antidepressants under current NICE guidance. The cut-off score of 10 for the PHQ-9 score used in this study is also less likely to detect major depression than a higher cut-off of 12 (Gilbody et al. 2007).

I would also like to see this survey repeated in other populations, particularly in primary care where most antidepressant medication is prescribed. In this setting, those who are screened for depression but not currently taking antidepressants would not only be more likely to be older and report more concurrent physical health problems (Kendrick et al., 2009) but would also vary considerably in terms of ethnicity and socioeconomic status, especially in comparison to a population recruited entirely online.

The authors discuss how “systematic differences in depression treatment burdens must be considered” and how these differ not only between countries but between individuals too. They searched for, and found no association between the SWD and variables such as age, sex, race or insurance status – but how much was this related to the limitations of their sample, as stated above?

Due to limitations with the recruited sample, it would be appropriate and informative to repeat this study in a primary care population where most antidepressant medication is prescribed.

Implications for practice

This is a novel study which examines a conceptually different approach to measuring what patients consider to be the smallest beneficial effect that is acceptable, given the burdens of a treatment when compared with another. Although the authors focus their conclusions on the future potential use of the SWD in a research setting, this paper also encourages us to focus more on how the potential benefits and burdens of antidepressants are explained and discussed in clinical settings. It prompts us to question the quality of decision-making, not only in the prescription of antidepressants, but in other treatments too.

This study found that the SWD didn’t substantially differ between those with and without clinical need, which is interesting and unexpected. I’ve taken antidepressants now for 30 years. When I began them, I had already had the opportunity to try psychological therapy and knew a great deal about medication when I raised the question of antidepressants with my doctor. I now know even more about the benefits and burdens of antidepressants, but I would still make the same decision because I was very low and desperate to try something different.

So, what is going on in the complex world of antidepressant prescribing? The authors speculate that the difference between the high SWD for antidepressants and increasing prescription rates suggests,

a lack of understanding from patients and a lack of communication from doctors (understanding burdens or overstating efficacy) that factor into hasty decision making in prescription acceptance.

Much of the increase in prescription is related to people staying on antidepressants for longer (Kendrick, 2021), but the latest NICE guidance is clear about what should be tried before starting them in the first place. However, the problem may be difficulty in getting access to some of these alternatives, such as brief psychological therapies, and insufficient time to fully discuss all of the issues, especially the benefits and burdens of different treatment options.

Given the pressure that primary care and mental health services are currently under in the UK, it is hardly surprising that prescribers (not all of them are doctors) and patients might decide on antidepressants – but that doesn’t mean we shouldn’t examine in greater depth how and why that decision is arrived at. We do need to know more about what happens during real-world conversations between patients and prescribers, and professionals need to reflect on the quality of our ‘shared decision making.’

Finally, as someone who takes them, I’d add that we still need better antidepressant medication.

We need to know more about what happens in real-world conversations between patients and prescribers to better understand how and why antidepressants are chosen as treatment options.

Statement of interests

During my career I received funding from various pharmaceutical companies for speaking about treatment of depression at meetings and employing researchers. I have never been involved in any clinical trials of medication, but I take antidepressants for recurrent depression.

Links

Primary paper

Sahker, E., Furukawa, T. A., Luo, Y., Ferreira, M. L., Okazaki, K., Chevance, A., … & Salanti, G. (2024). Estimating the smallest worthwhile difference of antidepressants: a cross-sectional survey. BMJ Mental Health, 27(1).

Other references

Bowers, H. (2024). Antidepressant withdrawal: recommendations for support from people with lived experience. The Mental Elf.

Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., … & Geddes, J. R. (2018). Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. The Lancet, 391(10128), 1357-1366.

Ferreira, M. L., Herbert, R. D., Ferreira, P. H., Latimer, J., Ostelo, R. W., Nascimento, D. P., & Smeets, R. J. (2012). A critical review of methods used to determine the smallest worthwhile effect of interventions for low back pain. Journal of Clinical Epidemiology, 65(3), 253-261.

Gilbody, S., Richards, D., & Barkham, M. (2007). Diagnosing depression in primary care using self-completed instruments: UK validation of PHQ–9 and CORE–OM. British Journal of General Practice, 57(541), 650-652.

Kendrick, T. (2021). Strategies to reduce use of antidepressants. British Journal of Clinical Pharmacology, 87(1), 23-33.

Kendrick, T., Dowrick, C., McBride, A., Howe, A., Clarke, P., Maisey, S., … & Smith, P. W. (2009). Management of depression in UK general practice in relation to scores on depression severity questionnaires: analysis of medical record data. BMJ, 338.

Read, J., Lewis, S., Horowitz, M., & Moncrieff, J. (2023). The need for antidepressant withdrawal support services: recommendations from 708 patients. Psychiatry Research, 326, 115303.

Tomlin, A. (2018). Antidepressants can help adults with major depression. The Mental Elf.

Photo credits