08.30.2020

Correction of a Key Study: No Evidence of “Gender-Affirming” Surgeries Improving Mental Health

Allowing scientific debate in transgender medicine improves evidence basis

In October 2019, the American Journal of Psychiatry (AJP) published a study from the Karolinska Institute in Sweden, and the Yale School of Public Health which reported that “gender-affirming" surgeries for gender dysphoric patients are associated with improved mental health outcomes (1). Looking at mental health utilization in the year 2015, a retrospective analysis showed that the more time passed since surgery, the fewer mental services were utilized by patients, with an average 8% reduction in mental health utilization for each year following surgery. From this, the study concluded that surgery has a beneficial effect on mental health, and that benefits continue to accrue over time. However, following a reanalysis of the data, this conclusion has now been officially corrected to indicate that there is “no advantage of surgery.”

Original Study by Bränström & Pachankis (2019)

The study (1) analyzed health records of 2,679 Swedes diagnosed with gender dysphoria between 2005 and 2015 to determine whether hormonal or surgical treatments improved their mental health over time. To approximate mental health outcomes, the authors relied on the count of mental health visits, psychiatric medication prescriptions, and hospitalizations following suicide attempts. It was presumed that the fewer “mental health events” a person experienced, the better their mental health.

While the authors found no evidence of benefits of hormonal treatments (adjusted odds ratio=1.01, 95% CI=0.98 - 1.03), they noted a statistically significant relationship between time since surgery and mental health status (adjusted odds ratio=0.92, 95% CI=0.87 - 0.98). Specifically, the researchers observed that as of 2015, patients who had surgeries further in the past had better mental health than patients whose surgeries were more recent. This “longitudinal association between gender-affirming surgery and reduced likelihood of mental health treatment” was interpreted by the authors as the evidence of a positive, time-release-like effect of “gender-affirming” surgery. The authors opined that this finding should “lend support to the decision to provider gender-affirming surgeries to transgender individuals who seek them.” The conclusions of the study were widely publicized by mass media outlets. The study also made a rapid and significant impact on clinical and public health education.

In a media interview, Bränström, the primary author, suggested that the study results lent “strong support” for providing “gender-affirming” care, while Pachankis, the study-co-author, issued a statement saying, “no longer can we say that we lack high-quality evidence of the benefits of providing gender-affirming surgeries to transgender individuals who seek them.” The study and its conclusions were disseminated by Medscape, relied on by millions of clinicians and researchers worldwide, and has become the basis for Medscape's Continuing Medical Education class (CME). The US-based Association of Schools and Programs of Public Health, which describes itself as the “voice of academic public health,” also disseminated the study and its findings to its member public health programs. To date, the study has been referenced by at least 9 other studies as evidence of benefit of “gender-affirming” surgeries (Table 1).

TABLE 1. Publications Found Citing Bränström & Pachankis (2019).

Citation

Quote, with reference to Bränström & Pachankis (2019) indicated with bold

Sevelius J.M., Gutierrez-Mock L., Zamudio-Haas S., et al. (2020). Research with Marginalized Communities: Challenges to Continuity During the COVID‑19 Pandemic, AIDS and Behavior, Volume 24, 2009–2012. https://doi.org/10.1007/s10461-020-02920-3

“Among our transgender and gender diverse staff and participants, delays in gender-affirming treatment, such as hormone injections and surgeries, due to the shutdown of clinics and delays in “elective” surgeries, can cause serious mental health issues, including depression, anxiety, and suicidality [16].”

Wang Y., Pan B., Liu Y., et al. (2020). Health care and mental health challenges for transgender individuals during the COVID-19 pandemic, Lancet Diabetes Endocrinology, Volume 8, Issue 7, 565-565. https://doi.org/10.1016/S2213-8587(20)30182-0

“Besides access to health care, it is also important to highlight mental health issues of transgender individuals. Previous studies showed that gender-affirming surgery was associated with reduced mental health problems.4

Roque R.A. (2020). Transgender pediatric surgical patients—Important perioperative considerations, Pediatric Anesthesia, Volume 30, Issue 5, 520-528. https://doi.org/10.1111/pan.13845

"Importantly, surgeries for transgender individuals are medically necessary procedures, proven to improve dysphoria, mental health, and quality of life,26, 29-33 and should not be viewed as elective.14, 15"

Konrad, M., & Kostev, K. (2020). Increased prevalence of depression, anxiety, and adjustment and somatoform disorders in transsexual individuals, Journal of Affective Disorders, Volume 274, 482-485, https://doi.org/10.1016/j.jad.2020.05.074

“Bränström and Pachankis (2019) published a large study with approximately 3000 TSI, which were about six times as likely to have had a mood and anxiety disorder health care visit, or to have been hospitalized after a suicide attempt, as compared with the general population”

Aldridge Z., Patel S., Guo B., et al. (2020). Long term effect of gender affirming hormone treatment on depression and anxiety symptoms in transgender people: A prospective cohort study, Andrology, published online 10 August. https://doi.org/10.1111/andr.12884

"In contrast, Bränström and Pachankis [40] using the Swedish population register showed no significant association between the likelihood of accessing mental health treatment and time since initiation of GAHT."

Flaherty, A.J., Sharma, A., Crosby, D. (2020). Should Gender-Affirming Surgery Be Prioritized During the COVID-19 Pandemic?,  Otolaryngology–Head and Neck Surgery, Published June 30, https://doi.org/10.1177/0194599820939072

“A recent study found significantly reduced risk of mental health treatment in TGNC patients after receiving GAS but not HRT alone.13

Schvey, N.A., Klein, D.A., Pearlman, A.T., Riggs, D.S. (2020), A Descriptive Study of Transgender Active Duty Service Members in the U.S. Military, Transgender Health, first published 19 May 2020, https://doi.org/10.1089/trgh.2019.0044

“The finding that transmales, who were significantly more likely to have taken steps toward gender affirmation, presented with better mental health may provide further evidence supporting the effectiveness of gender-affirming treatments in ameliorating distress and improving quality of life.69,70

Surmaitis, R.M., Greenberg, M.R., Ebeling-Koning N.E., et al. (2020). Characteristics of Transgender Patient Cases Managed by a Toxicologist: an Analysis of the Toxicology Investigator’s Consortium (ToxIC) Registry: January 2017–June 2019, J Med Tox, Published: 17 June 2020, 

“It would be helpful for further study to gather more information on whether patients have undergone surgical procedures or hormonal therapy because it has been shown that rates of suicidality and utilization of mental health services may differ based on transition-related medical interventions [20, 21].”

Giraldi A. (2020). Mental health and gender dysphoria – why does it matter? Acta Psychiatrica Scandinavica, Volume 141, Issue 6, 483-485, https://doi.org/10.1111/acps.13182

“Though, some studies indicate that some of the increased risk of mental health problems persist, which may in part be related to internal and external factors (17-19).”

Vigorous Debate Leads to Correction of Key Finding

After the study was published, many researchers and scientists (including some SEGM advisors) alerted the AJP to multiple serious methodological problems that challenged the study’s conclusion. In response, the AJP editor requested an independent statistical review of the data, which led to a reanalysis of the data and an official correction (2,3). When gender dysphoric patients who received surgeries were compared to those who did not have surgeries, there was no statistically significant difference in their mental health utilization (Figure 1).

Nine months after the study’s original publication, the AJP stated, “the results [of the reanalysis] demonstrated no advantage of surgery in relation to subsequent mood or anxiety disorder-related health care visits or prescriptions or hospitalizations following suicide attempts” (2).

Figure 1. 2015 Mental Health Services Utilization Among Patients Diagnosed with Gender Dysphoria in Sweden Between 2005 and 2015 by "Gender-Affirming" Surgery Status

From Table 1,  Bränström, R., & Pachankis, J. (2020). Toward Rigorous Methodologies for Strengthening Causal Inference in the Association Between Gender-Affirming Care and Transgender Individuals’ Mental Health: Response to Letters. American Journal of Psychiatry, 177(8), 769-772.

The methodology used by the authors was deemed inadequate to reach the conclusions supporting surgical treatment for gender dysphoria (4-10). The multiple methodological issues identified by researchers from Sweden, Norway, US, and UK, included:

  • Single year short-term outcomes. While the study was interpreted by many as a 10-year follow-up, it is effectively a single-year, short-term outcome study. Of the individuals who underwent surgery (n=1,018), fewer than 2% had surgery 10 or more years ago (n=19), while over 75% had surgeries within the past three years (n=772). Analyzing single-year outcomes from different initial years is fraught with methodological problems and is not equivalent to a longitudinal study.
  • Failure to account for the recent rise in co-occuring mental health problems. In recent years, there has been a general increase in mental health morbidity in the Swedish population, as well as a noted shift in patients presenting for gender dysphoria treatments, with most patients now adolescent females with significant mental health comorbidities (11,12). Thus, the better mental health of the older cohorts could explain why those who transitioned further in the past utilized fewer mental health services in 2015, as compared to the more recent transitioners.
  • No comparison to pre-surgery mental health status. Rather than comparing the patients' mental health services utilization in 2015 to utilization pre-surgery in order to estimate the impact of surgery, the authors instead compared it to the utilization of mental health services during the year of the (last) surgery. Choosing the year of the surgery as the baseline is highly problematic, because surgery in itself may have been associated with a temporary increase in mental health services utilization (such as additional routine evaluations or a result of surgery-related stress). Thus, it is impossible to determine whether the detected reduction in mental health utilization in 2015 compared to the year of surgery is a true reduction in one’s baseline mental health utilization, or if it is merely reflecting a return to the baseline, following a potential short-term spike in mental health needs brought about by the surgery itself.
  • Unaccounted for high loss to follow-up. It’s unclear why the researchers only identified a total of 19 patients who underwent surgeries before 2005, since another study using the same national registry found 324 patients in the years prior to 2004 (13). It’s highly problematic if over 90% of postoperative gender dysphoria subjects were lost to follow-up. It's surprising that the reasons for such a large discrepancy were not addressed by the study authors, especially since both studies were conducted by the Karolinska Institute and they studied the same population. The reasons for this large discrepancy must be clarified. Prior studies demonstrated that post-surgery transgender patients suffer from elevated mortality due to cardiovascular disease and suicide (13). To the extent that even some of this loss of postoperative transgender patients is due to premature death, this large loss of study subjects may have masked adverse outcomes.
  • Questionable proxy for mental health. The authors used outpatient mental health visits, prescriptions for mental health conditions, and hospitalizations following suicide attempts as surrogate measures of mental health. However, the use of these measures is problematic. Outpatient visits recorded as related to mental health may or may not signal the presence of mental health issues. For example, mental health assessments were required prior to surgery; conversely, visits not recorded as related to mental health could signal mental health problems, especially those related to sleeping disorders or substance use. The inclusion or exclusion of different drug prescriptions was arbitrary and inconsistent; histamine blockers were judged related to mental health issues, whereas antipsychotic medications were not. Also, only suicide-related mental health admissions were considered, ignoring all other mental health-related admissions. The suicidality measure is particularly questionable, since only suicide attempts requiring hospitalization were analyzed, with no accounting for patients who actually died by suicide or those whose suicide attempt did not result in a hospitalization.  
  • Failure to correct for multiple hypotheses testing. The observed reduction in mental health treatments utilization associated with surgeries was small - an average 8% reduction per year - and was statistically significant by only a thin margin (adjusted odds ratio=0.92; 95% CI= 0.87-0.98). Further, this was the only statistically significant finding among several that the authors report to have tested. However, the authors did not appear to have performed a standard statistical adjustment for multiple hypotheses testing. This is a critical concern, since this necessary correction would likely have rendered this already weak result not statistically significant. The increasing frequency of “fishing expeditions,” made possible by large datasets and sophisticated software, has led to a well-recognized crisis of replication in healthcare research, with many findings of statistical significance that cannot be reproduced by other researchers. Reducing false positive findings is crucial when these findings are used to guide decisions about medical interventions that carry substantial health risks and that are irreversible.
  • Failure to include a control group. The authors failed to include a control group - comparing gender dysphoric patients who underwent surgery with those who did not. The Swedish database contains comprehensive information for both cohorts, and such a comparison should be performed before any conclusions about the benefits of gender affirming surgeries are made.

After these and a number of other limitations had been pointed out by several researchers, the AJP editor commissioned an independent statistical evaluation. The independent expert concurred with the concerns raised (3), and recommended that the answer to the key study question - whether surgeries improve mental health outcomes - should be clarified by creating a control group and reanalyzing the data. Complying with the independent statistical expert’s recommendations, the study authors created two equally-sized groups of patients diagnosed with gender dysphoria, the "surgery" and the "no-surgery" groups, which were matched on key demographic characteristics. When the two groups were compared, no significant differences were found in any of the mental health utilization measures used as a proxy for overall mental health (14). These findings are illustrated in Figure 1 above.

As a result, the AJP issued a correction, which states, “the results [of the reanalysis] demonstrated no advantage of surgery in relation to subsequent mood or anxiety disorder-related healthcare visits or prescriptions or hospitalizations following suicide attempts,” and “given that the study used neither a prospective cohort design nor a randomized controlled trial design, the conclusion that “the longitudinal association between gender-affirming surgery and lower use of mental health treatment lends support to the decision to provide gender-affirming surgeries to transgender individuals who seek them” is too strong (2).

The authors summarized the key findings in Table 1 of their new paper, “Toward Rigorous Methodologies for Strengthening Causal Inference in the Association Between Gender-Affirming Care and Transgender Individuals’ Mental Health: Response to Letters” (14).

Range of Interpretations

The results of the Bränström & Pachankis original analysis (2019) and their re-analysis (2020) may be interpreted in a number of ways. They include the possibility that "gender-affirming" surgeries improve mental health, worsen mental health, or that the data are insufficient to draw conclusions about the impact of hormonal and surgical interventions on mental health.

  1. "Gender-affirming" surgeries may improve mental health
    It is possible that despite the fact that the “surgery” group, on average, did not show a reduction in utilization of mental health services compared to the “no-surgery” group, some patients may have benefited from surgery. The study authors correctly point out that the comparator group may include gender dysphoric patients who did not seek surgery and therefore it cannot be concluded that the lack of difference between the two groups signals the lack of benefit of surgeries for those who seek them.
  2. "Gender-affirming" surgeries may worsen mental health
    It may be argued that not only has the re-analysis invalidated the original conclusion of proven benefits of “gender-affirming” surgery, but it has identified a possible worsening of mental health following surgery. The control group represents a strong comparator: patients in both the control and the intervention groups had confirmed DSM diagnoses of gender dysphoria, which takes at least a year to obtain; further, the two groups were matched using age, legal gender, education, and country of birth. Worse mental health outcomes during the peri-operative period (the year of surgery) noted in the original study, and the nearly doubling of serious suicide attempts in the “surgery” cohort compared with the “no surgery” cohort, which emerged during the re-analysis of the data, are consistent with this interpretation.
  3. The current study is insufficient to draw conclusions about the impact of "gender-affirming" interventions on mental health
    Given the challenges of conducting quality research in the absence of randomized control groups, and based on the study’s chosen methodology, it can be argued that the data presented in the study can provide no reliable information whether “gender-affirming” surgeries improve, worsen or have no effect on mental health.
SEGM Position

It is SEGM’s view that the data presented in the original study and the subsequent re-analysis do not support the claim of an expected "reduction in mental health treatment as a function of time since completing such treatment” (14).  After the reanalysis of the data, we conclude the following:

  • The mental health needs of people suffering from gender dysphoria are significantly greater than those of the general population, which confirms previous research
  • No mental health benefit of hormonal interventions was demonstrated
  • No mental health benefit of "gender-affirming" surgery was demonstrated
  • Specific to the question of longitudinal association between time from surgery and mental health outcomes, due to unaddressed study design limitations, no improvement of mental health with time after surgery was demonstrated
  • Despite the higher rate of suicide attempts requiring hospitalization in the "surgery" group, the study design precludes the assertion that that "gender-affirming" surgery is harmful

SEGM is concerned by the renewed claim that despite the reanalysis, which showed no difference between the control and intervention group, the original study’s conclusion of continued improvements in mental health after surgery stands (14). SEGM does not believe that the data presented by the authors support this claim, because as yet unaddressed methodological limitations continue to confound the original study’s findings. 

It is SEGM’s view that the longitudinal association between time from surgery and improved mental health outcomes, cited in the original analysis, is confounded by serious methodological limitations, and cannot be considered valid until these limitations are addressed. Notably, contrary to the authors’ claim, the analysis failed to control for differences in the baseline mental health status of the cohorts of patients seeking surgery. The well-documented increase in mental health burden of the more recent patient cohorts (11,12) alone can explain why those who had surgeries a decade ago have better mental health status in 2015, than those with more recent surgeries. It is critical that the analysis corrects for the underlying mental health of each patient cohort before any claims of “a reduction in mental health treatment as a function of time” are posited, let alone accepted. Other issues raised in the letters also must be addressed, especially in the context of the fact that a reanalysis using a control group failed to lend support to the original finding.

SEGM is also concerned that the surgery group experienced nearly twice as many suicides compared to the non-surgery group, based on 2015 numbers (13 and 7 suicide attempts, respectively). While not statistically significant, this finding deserves additional analysis. Since suicides are somewhat rare events, the reanalysis should include a longer-term, rather than a single-year, follow-up, in order to properly power the study. 

Closing Thoughts

We applaud the AJP and the study authors for addressing some of the concerns expressed by researchers and scientists regarding the flawed methodology of the study and the problematic conclusion, and for issuing a correction to the study’s key finding. We are heartened that the Karolinska Institute, home of the study’s primary author, promptly acknowledged the correction. More recently, Yale University's School of Public Health, the home of the study’s other author, has also replaced its news story lauding the original study findings with an acknowledgement of the correction.

Unfortunately, the original study with its misleading title and incorrect conclusion continues to be available on the AJP's website. The original, uncorrected study is also part of Continuing Medical Education courses by Medscape, which is relied on by clinicians worldwide for accurate, evidence-based information. The Association of Schools and Programs of Public Health, disseminating information to 180 schools and programs in public health, has yet to correct its publication. 

Gender dysphoric patients and the clinicians who care for them need quality, accurate information to make informed decisions. This is especially true for adolescents and young adults, who are currently the vast majority of patients presenting to gender clinics, and whose decisions will have profound, life-long implications. It’s critical that the AJP update the original study’s title and conclusion in order to reflect this critical correction. It is also urgent that all organizations that had disseminated the incorrect conclusions publicize the fact that the conclusions have now been corrected, and that any treatment guidelines or recommendations based on the original finding are promptly updated to reflect this new information.

The number of critical letters to the editor and the need to correct a core conclusion of a published paper highlights the importance of stringent methodology and rigor in medicine. Many of the original study’s shortcomings should have been obvious and might have been avoided by better protocol design. Nevertheless, this episode shows the self-correcting mechanism of "the many eyes of science." It underscores the need for researchers to engage in scientific debate in order to further the quality of scientific inquiry in the rapidly changing field of gender medicine. 

The need for quality information in gender medicine is particularly crucial, given that the majority of patients seeking hormonal and surgical interventions for gender dysphoria are now young people, most of whom are females with adolescent-onset of gender dysphoria and co-occurring mental health difficulties (15). Today, irreversible hormonal and surgical interventions are routinely offered as first line treatment to this novel and poorly-understood cohort of patients. 

It is significant that the largest population study conducted to date, which assessed the mental health benefits of hormonal and surgical treatments to alleviate gender dysphoria, did not provide evidence that these interventions were accompanied by a reduction in subsequent mental health treatments or serious suicide attempts. Substantial research is needed in this vitally important area of medicine to establish the type and timing of interventions for the increasingly varied presentation of gender dysphoria worldwide. 

References

  1. Bränström, R., & Pachankis, J. (2019). Reduction in Mental Health Treatment Utilization Among Transgender Individuals After Gender-Affirming Surgeries: A Total Population Study. American Journal Of Psychiatry, 177(8), 727-734. https://doi.org/10.1176/appi.ajp.2019.19010080
  2. Correction to Bränström and Pachankis. (2020), 177(8), 734-734. https://doi.org/10.1176/appi.ajp.2020.1778correction
  3. Kalin, N. (2020). Reassessing Mental Health Treatment Utilization Reduction in Transgender Individuals After Gender-Affirming Surgeries: A Comment by the Editor on the Process. American Journal Of Psychiatry, 177(8), 764-764. https://doi.org/10.1176/appi.ajp.2020.20060803
  4. Anckarsäter, H., & Gillberg, C. (2020). Methodological Shortcomings Undercut Statement in Support of Gender-Affirming Surgery. American Journal Of Psychiatry, 177(8), 764-765. https://doi.org/10.1176/appi.ajp.2020.19111117
  5. Curtis, D. (2020). Study of Transgender Patients: Conclusions Are Not Supported by Findings. American Journal Of Psychiatry, 177(8), 766-766. https://doi.org/10.1176/appi.ajp.2020.19111131
  6. Landén, M. (2020). The Effect of Gender-Affirming Treatment on Psychiatric Morbidity Is Still Undecided. American Journal Of Psychiatry, 177(8), 767-768. https://doi.org/10.1176/appi.ajp.2020.19111165
  7. Malone, W., & Roman, S. (2020). Calling Into Question Whether Gender-Affirming Surgery Relieves Psychological Distress. American Journal Of Psychiatry, 177(8), 766-767. https://doi.org/10.1176/appi.ajp.2020.19111149
  8. Ring, A., & Malone, W. (2020). Confounding Effects on Mental Health Observations After Sex Reassignment Surgery. American Journal Of Psychiatry, 177(8), 768-769. https://doi.org/10.1176/appi.ajp.2020.19111169
  9. Wold, A. (2020). Gender-Corrective Surgery Promoting Mental Health in Persons With Gender Dysphoria Not Supported by Data Presented in Article. American Journal Of Psychiatry, 177(8), 768-768. https://doi.org/10.1176/appi.ajp.2020.19111170
  10. Van Mol, A., Laidlaw, M., Grossman, M., & McHugh, P. (2020). Gender-Affirmation Surgery Conclusion Lacks Evidence. American Journal Of Psychiatry, 177(8), 765-766. https://doi.org/10.1176/appi.ajp.2020.19111130
  11. “Why has mental illness increased among children and young people in Sweden?” Swedish (”Varför har psykiska ohälsan ökat hos barn och unga ?”) National Health Agency, 2018. Manuscript number: 18023-2. www.folkhalsomyndigheten.se https://www.folkhalsomyndigheten.se/contentassets/628f1bfc932b474f9503cc6f8e29fd45/varfor-psykiska-ohalsan-okat-barn-unga-18023-2-webb-rapport.pdf 
  12. Bremberg, S. & Dalman, C.. (2015). “Mental illness and psychiatric conditions in children and adolescents”. Swedish (”Psykisk ohälsa och psykiatriska tillstånd hos barn och unga”). Ministry of Health and Social Affairs, Swedish Research Council for Health, Working Life and Welfare. www.forte.se. https://forte.se/app/uploads/2014/12/kunskapsoversikt-begrepp.pdf
  13. Dhejne, C., Lichtenstein, P., Boman, M., Johansson, A., Långström, N., & Landén, M. (2011). Long-Term Follow-Up of Transsexual Persons Undergoing Sex Reassignment Surgery: Cohort Study in Sweden. Plos ONE, 6(2), e16885. https://doi.org/10.1371/journal.pone.0016885
  14. Bränström, R., & Pachankis, J. (2020). Toward Rigorous Methodologies for Strengthening Causal Inference in the Association Between Gender-Affirming Care and Transgender Individuals’ Mental Health: Response to Letters. American Journal Of Psychiatry, 177(8), 769-772. https://doi.org/10.1176/appi.ajp.2020.20050599
  15. Kaltiala-Heino, R., Bergman, H., Työläjärvi, M., & Frisen, L. (2018). Gender dysphoria in adolescence: current perspectives. Adolescent Health, Medicine And Therapeutics, Volume 9, 31-41. https://doi.org/10.2147/ahmt.s135432