Close printable page
Recommendation

The Causal Effects of Hormonal Contraceptives on Psychological Outcomes

ORCID_LOGO based on reviews by Summer Mengelkoch and 2 anonymous reviewers
A recommendation of:

Hormonal Contraceptive Use and Women’s Sexuality and Well-Being: Estimating Treatment Effects and Their Heterogeneity Based on Longitudinal Data

Abstract

EN
AR
ES
FR
HI
JA
PT
RU
ZH-CN
Submission: posted 30 November 2022
Recommendation: posted 27 September 2023, validated 28 September 2023
Cite this recommendation as:
Evans, T. (2023) The Causal Effects of Hormonal Contraceptives on Psychological Outcomes. Peer Community in Registered Reports, . https://rr.peercommunityin.org/articles/rec?id=355

Recommendation

Ensuring universal access to sexual and reproductive health and reproductive rights is a global concern, exemplified by goal 5.6 of the Sustainable Development Goals (UN General Assembly, 2015). Whilst the range of contraceptive options have increased, our understanding of the impacts of use for women are inadequate and represent a key barrier to positive change in policies and practices. In particular, we have few consensuses on the expected impacts of hormonal contraceptive use on women's sexuality and wellbeing.
 
In the current programmatic submission, Botzet et al. (2023) argue that this inconclusive evidence base could be due to the wide heterogeneity in responses, the impacts of this heterogeneity upon attrition, differences in contraceptive methods and dosage effects, confounders, and the potential for reverse causality. Tackling some of these potential factors, Botzet (2023) explore whether hormonal contraceptive use influences sexuality and well-being outcomes, and whether (and to what extent) the effects vary between women. To achieve this they have proposed analysis of longitudinal data from the German Family Panel (PAIRFAM) which includes annual waves of data collection from >6500 women, with separate Stage 2 submissions planned to report findings based on sexuality and well-being. The proposed work will progress our understanding of the impact of hormonal contraceptives by overcoming limitations of more common research approaches in this field, and has the potential to contribute to a more contextualised view of the impact of their impacts in real-world practice.
 
The Stage 1 manuscript was evaluated over three rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/kj3h2
 
Level of bias control achieved: Level 3: At least some of the data/evidence that will be used to answer the research question already exists AND is accessible in principle to the authors BUT the authors certify that they have not yet accessed any part of that data/evidence.
 
List of eligible PCI-RR-friendly journals:
 
References
 
Botzet, L. J., Rohrer, J. M., Penke, L. & Arslan, R. C. (2023). Hormonal Contraceptive Use and Women's Sexuality and Well-Being: Estimating Treatment Effects and Their Heterogeneity Based on Longitudinal Data. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/kj3h2
 
UN General Assembly (2015). Transforming our world : the 2030 Agenda for Sustainable Development, 21 October 2015, A/RES/70/1. Available at: https://www.refworld.org/docid/57b6e3e44.html [accessed 27 September 2023]
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Reviews

Evaluation round #3

DOI or URL of the report: https://osf.io/3zcty?view_only=da9c96e72dec45ad81936c991245a236

Version of the report: 2

Author's Reply, 20 Sep 2023

Download author's reply Download tracked changes file

Dear Prof. Dr. Evans, dear reviewers,

Thank you very much for the positive evaluation of our revised programmatic registered report stage 1 on heterogeneous effects of contraception and for the opportunity to revise and resubmit the manuscript to Peer Community In - Registered Reports.

Please find our revision letter and a revised version of the manuscript with tracked changes attached. You can also access the revised version with tracked changes on the OSF: https://osf.io/ejmk6?view_only=9e9a21d41c9a47f998669fec70ea722e. In addition, we also uploaded a revised version without tracked changes to the OSF: https://osf.io/hws6m?view_only=9e9a21d41c9a47f998669fec70ea722e in case you prefer that for reading the manuscript again.

We look forward to the IPA for our manuscript and the start of the second phase of our registered report.

Best regards,

Laura Botzet, on behalf of all co-authors

Decision by ORCID_LOGO, posted 14 Sep 2023, validated 14 Sep 2023

Thanks for resubmitting your RR and thank-you for your patience in awaiting a response. Two reviewers returned to consider the manuscript and as you can see below, they are content with the management of their previous feedback. One reviewer has a few additional comments to consider so before I 'accept' and grant IPA, I'd like to give you the opportunity to reflect upon their suggestion and action it as you see fit. Following your resubmission, I will not look to secure further reviewer feedback and will be aiming to provide IPA as swiftly as possible. I imagine this won't take long to action so I will be vigilant in awaiting and actioning your final revised Stage 1 submission.

Thanks and take care,

Dr Thomas Rhys Evans

Reviewed by , 28 Jul 2023

The authors did an excellent job of responding to comments and updating their registered report. I was really impressed with the revised report, and approve of the current version. I had just had one actual note:

1. HCs are often prescribed to both homosexual women and women who are not sexually active. The assumption that women who have never had sex will not be using HCs seems a bit flawed - to me, this is just missing data. It is more likely that they are not using HCs, but not certain. The authors should consider treating this as missing data in the main analysis as opposed to just in a robustness analysis. In a related vein, perhaps a robustness analysis could also include homosexual women? I trust the authors to consider this feedback and adjust things if appropriate, and do not need a response to this comment. 

 

Minor things I noticed in my read through:

1. Pg 1: missing commas here and there (e.g., between exsisted and which; after But); consider changing like to such as "...such as how HCs influence sexuality and well-being." I thought I saw a missing oxford comma too, but maybe I imagined that. 

2. Pg 19: missing word of "...inform their decision OF which..."

3. Pg 21: missing a space after ;

4. Table 3: check the header formmatting, new column seems one character too small!

Reviewed by anonymous reviewer 2, 27 Aug 2023

I have gone through the authors' reply to reviewer comments in Round 1 and the new manuscript file with tracked changes. I believe the authors have sufficiently addressed the previous concerns and I did not see any additional concerns (ones that I have not raised before).

Evaluation round #2

DOI or URL of the report: https://osf.io/xmgh5?view_only=6d5b0a56a41541249cab38c51847157c

Version of the report: 1

Author's Reply, 13 Jul 2023

Download author's reply Download tracked changes file

Dear Prof. Dr. Evans, dear reviewers,

Thank you very much for the helpful and constructive feedback on our programmatic registered report stage 1 on heterogeneous effects of contraception and for the opportunity to revise and resubmit the manuscript to Peer Community In - Registered Reports.

Please find our revision letter and a revised version of the manuscript with tracked changes attached. You can also access the revised version with tracked changes on the OSF: https://osf.io/3zcty?view_only=da9c96e72dec45ad81936c991245a236. In addition, we also uploaded a revised version without tracked changes to the OSF: https://osf.io/tf7r4?view_only=da9c96e72dec45ad81936c991245a236 in case you prefer that for reading the manuscript again.

Best regards,

Laura Botzet, on behalf of all co-authors

Decision by ORCID_LOGO, posted 13 Jul 2023, validated 13 Jul 2023

Upon request of the authors, I am bouncing back this submission to the authors for revision.

Evaluation round #1

DOI or URL of the report: https://osf.io/xmgh5?view_only=6d5b0a56a41541249cab38c51847157c

Version of the report: 1

Author's Reply, 12 Jul 2023

Download author's reply Download tracked changes file

Dear Prof. Dr. Evans, dear reviewers,

please find our revision letter as well as the manuscript with tracked changes attached.

 

Best regards,

Laura Botzet, on behalf of all co-authors

Decision by ORCID_LOGO, posted 23 Jan 2023, validated 23 Jan 2023

Thank-you again for submitting your Stage 1 to PCI:RR. I thoroughly enjoyed reading through your work and the prepared materials shared via the OSF page. I wish to make it immediately explicit in framing my evaluation of the manuscript that I do not have experience researching contraceptive use. However, I sincerely hope that my methodological/open scholarship expertise and contributions to publishing 11 (and counting!) of my own RRs provides a rigorous basis for providing you with fruitful feedback. My goal is to help you make this work the best it can be and I look forward to supporting you on that basis.

Just a few hours ago the second review was submitted so I am pleased to confirm that I now have two appropriate reviews and I would like to encourage you to make revisions considering their supportive and constructive feedback as seen below. Their comments and suggestions cover a range of themes from the need to justify your analysis approach further to consideration of other factors and models. The crux of their feedback, and of this paper as a whole, is the modelling (both conceptual and statistical) where there is opportunity to go in many directions and subsequently where you may wish to further justify or reconsider decisions previously made in context of the available options the data allows. I have no interest in dictating to you which of these need full and comprehensive implementation and which do not, but I encourage you to be systematic and open in considering all their feedback, and providing a clear response to each comment regardless of whether they are actioned. I particularly appreciate use of ‘tracked changes’ or similar approaches to log changes, but please ensure your response to the feedback is as comprehensive and accessible as possible to facilitate the next stage of review.

From my own reading, your work is a really interesting analysis of the substantive PAIRFAM dataset and considers two sets of highly important outcomes. I particularly appreciated the critical tone adopted when considering the conclusions and limitations of the extant literature. Your introduction provides a clear overview of the limitations to our current approach to studying these effects and whilst you identified a range of dimensions which problematise our current understanding well, both reviewers suggested further justification for some of your core decisions which would help to build a more convincing and rigorous basis for your study. You could provide a little more detail on the simulation and access to the data provided by Tita Gonzalez Avilés too, to make it clearer as to how you used this information. More broadly, throughout there were occasionally some sentences which were a little wordy or heavy to follow and I encourage you to proof-read through the manuscript with this in mind to ensure it is as accessible and clear as possible, particularly as the reviewers highlighted some specific areas where this obfuscated the meaning of the content. Whilst I would have liked a little more annotation alongside the code, and I encourage you to revisit this as you work through the comments above, all the materials were well-structured and easily accessed.

In sum, I would like to encourage you to reflect upon the feedback provided, and to resubmit when you have systematically reconsidered the core decisions made behind the models proposed. I sincerely hope you find this feedback and process to be fruitful in supporting the further clarity and impact of the work, and I look forward to hearing from you in due course,

Stay safe and take care,

Dr Thomas Rhys Evans (Tom/He/Him)

 

Reviewed by anonymous reviewer 1, 21 Dec 2022

The authors propose a series of analyses to be conducted on a large, longitudinal dataset containing data from 6,565 women collected over the course of 13 waves (waves collected annually, average of 6 waves per participant). The authors seek to estimate the effect of hormonal contraceptive (HC) use on both sexual outcomes and well-being. Further, they also seek to investigate the heterogeneity in women’s responses to HC treatment and if women’s treatment responses predict their future HC use. Generally, I found the research questions to be well supported and the relevant literature to be well described. The research questions the authors propose are of keen scientific interest, and the authors’ choice of study design is appropriate to answer these questions. That is, naturalistic, longitudinal studies are certainly one vital piece of the puzzle needed to understand how HC use impacts both women’s sexuality and well-being. While I am not as statistically literate as the authors, their statistical approach appears well conceived and appropriate, although given my limited knowledge of Bayesian models in practice, I will ask a few clarifying questions below to ensure that these approaches are indeed appropriate and would not benefit from supplementation with additional exploratory analyses or alternate approaches. While of acceptable standards, there are a few points in the methodology and data analysis plan which I believe could use additional clarification before I recommend this Stage 1 registered report, which I detail below. My only major concerns with the data analytic strategy lies in the authors’ abilities to answer the questions they seek to answer given the limited specificity of the data available and if this is being appropriately modeled, given the lack of a statistical model included in the report. Finally, I would like to see the authors provide what their interpretation will be for potential results, to get an idea of what the data analysis will reveal. Overall, I found this to be an impressively well written registered report and an interesting research study.

Specific points:

1.      The variable options available in this dataset are somewhat limited. For example, there is an option for IUDs in type of contraceptive used, but no specification of if this is a hormonal or nonhormonal IUD – is this information provided in a follow up question? How will the authors determine if a woman is using HCs if she reports an IUD? I saw that IUD was a potential exclusion criterion, but that does prevent analysis of changing methods to/from HCs in the event that one of the options is an IUD (hormonal or copper). How do the authors plan to handle this confound? This seems important, given the popularity of IUDs as both a hormonal and non-hormonal contraceptive option.

a.      Is there any more specificity in the data about what actual type of HC is being used? I worry that given the likelihood that different methods have different effects (and perhaps more meaningfully, different effect sizes) that lumping methods into such broad categories will limit the impact of the results and the reliability of effect size estimates.

b.      Likewise, I am surprised that the authors did not plan to investigate how oral vs non-oral HC methods differed in their effects. I would like to see this question addressed within the current protocol, given that it seems like a decently simple difference to investigate here and would be of interest to other HC researchers. (Perhaps the authors are planning to do this and I misinterpreted this plan)

2.      Instead of excluding women who report homosexual relationships, I would prefer to see models run with and without these exclusions. Additionally, I am a bit confused as to why women using no contraceptive method are being excluded from analysis – it seems to me that they are an important comparison group for HC users, given the goals of the research design. Perhaps the authors had considered this and have another reason for this exclusion, however, I would like to see this explained or an additional exploratory analysis conducted.

a.      As a follow-up: excluding women who are sterilized, use a nonhormonal IUD, who are trying to become pregnant, or are just not using a listed contraceptive method is excluding a large amount of women who are free from HC use. Comparing the effects of using HCs to only women using certain other methods but not the previously listed circumstances seems a limitation here. Maybe that is the better way to approach this (no study is without limitations) however I would like the authors to consider this potential limitation in the design.

3.      I have some concerns about grouping all effects within each DV (sexual desire, sexual behavior, satisfaction; depression, life satisfaction, self-esteem). I would like to see an exploration of each of these individually and one of them comprised into a latent factor/composite of sexuality and well-being. This may be what the authors were intending however this is not specified (and my apologies I am not great at reading other people’s code, so I would prefer to see the statistical model included in the report).

4.      Is there a plan to control for how having never had sex might impact results on sexuality? Perhaps a follow-up analysis with and without these women for sexuality would be appropriate.

5.      Figure 1 – while this is an adequate conceptual model, I would like to see more information regarding the statistical model included so that I understand how variables are being treated in analyses. What does relationship mean in this figure? (Relationship status/length?) I would like to see the names of the constructs the same in figures as in tables.

6.      Table 1 – please add a column which breaks down how results will be interpreted (especially when multiple modeling approaches will be used to answer the same questions) per the PCI RR recommendations.

7.      For the heterogeneity question – I would love to see this question addressed with more statistical nuance than comparing the percentage of women with positive effects (guessing effects here is meant to refer to the totality of sexuality effects and well-being effects separately, but the authors do not adequately explain how they plan to conceptualize these comparison variables) to the percentage with negative effects. Here, again, I would be interested to see if there is heterogeneity in responses to treatment in each DV and the latent/composite DVs. I would also be curious if there was a way to profile those with different effects (not just positive or negative) based on the groupings of their responses to treatment. (e.g., is there a group of women with high self-esteem and sex drive, but low sexual behavior? Another group with high sexuality and high depression? Is there something that predicts membership to these groups?)

8.      For the question of investigating if adverse experiences on HCs influence future use – how does investigating relationships between how long a woman has been using HCs and her future use answer this question? Are there additional variables being included in the model to answer this question that I am missing? How are positive or negative experiences being conceptualized?

9.      For data simulations, how was -.45 selected as the estimate of the true causal effect?

10.   Why were the following variables not considered here: age at first HC use, duration of HC use prior to the study beginning, childhood trauma, pre-existing mental health Dx. If data not available, this is absolutely appropriate, however, given that these factors are more supported as influencing HC treatment outcomes than are personality traits, I would expect these to be included.

11.   Given the multilevel nested nature of this data (and my own training), I might have approached analyses using a RI-CLPM. I would love if the authors could describe why they selected their analysis plan in comparison to this type of analysis plan (or other appropriate models)! That is, the authors explained why their approach was a good approach, but not why it might be a better way to answer this question compared to other approaches! I am very ready to be convinced.

Reviewed by anonymous reviewer 2, 23 Jan 2023

This manuscript presents the Stage 1 proposal to examine relating to effects of hormonal contraceptive use and two outcome measures (sexual desire and well-being, which will be presented in 2 separate Stage 2 manuscripts). The research question is important and I commend the authors for finding a dataset that can be used to test their hypotheses (though I understand that the research is not actually testing hypotheses as much as describing the strength of associations). This will be an important contribution to the literature. I did find some aspects of the Stage 1 proposal unclear and/or difficult to follow. I list specific concerns below, ordered by section.

Introduction

I found the logic transition from talking about the need to conduct experiments to examining treatment effects (in the first part of the introduction) to be muddy. I think the authors can create a stronger framing for their use of the longitudinal data. I would de-emphasize experimental approaches and just focus on the strength of the present approach, which is strong in its own right. I am also then unsure about the focus on randomized control trials right after listing the five possible explanations for the mixed findings in the literature. Again, I would reframe to focus on the current method. If a comparison to randomized controlled studies is needed, I would add it after.

I had to read this sentence over multiple times before I understood what exactly was being said because it seems to start a completely different point than the beginning of the paragraph (the sentence that starts with "To apply this perspective to the current research question, assume that for example an ineffable..." on page 11). I think the authors can make this point in a more straight-forward way. Perhaps by introducing the time-varying versus time-invarying terms first.

Skovlund et al., (2016) found age to be particularly important for whether HC use was associated with greater risk for depression. I think this point should be more thoroughly discussed. Additionally, are any of the findings in the sexuality section similarly moderated by age? The Skovlund reference may also need to be mentioned after the "Older women might be more likely to experience beneficial side effects.." on page 14. The given reason for this association focuses on finding a method that fits them best, however, is there any evidence that sensitivity to steroid hormones decreases with age as well?

Conceptual Design and Underlying Assumptions

In both the time-lagged regression and the IPTW approach, the model includes the interaction of the outcome and contraceptive method at the previous assessment (in addition to the main effects). It is not clear to me conceptually why the interaction term is needed. Can the authors clarify their hypothesis? If the interaction term is not significant, will it remain in the model?

For variables related to relationship status, would it not be best to include relationship status at previous assessment to examine change in relationship status? This seems like a better test of the theoretical rationale that was laid out (about starting a relationship being a potential confounder).

Would the data allow for some sort of latent class analysis to classify women as having positive, negative, or neutral effects?

Method and Analysis Plan

Do the authors mean that women will be excluded if they ever hit 50 or menopause during the data collection period or only their timepoints after 50/menopause (i.e., earlier timepoints from those women would be included)? That sentence was unclear to me.

What is the logic for excluding women in homosexual relationships? Research on cycle effects documents similar shifts in women who are in homosexual relationships as women who are in heterosexual relationships.

I am not sure why women who indicated using no contraceptive method will be excluded in robustness analyses. That sentence does not make sense to me. Specifically, after reading this sentence, I became confused about the "control group" - will it just be users of NON-HC METHODS or anyone not currently on a HC? NON-OC METHODS? The authors should specify exactly how hormonal contreceptive use will be operationalized from the questions in the PARFAIM dataset (if this is reported, it is not prominent enough).

Are there questions on whether women had sex in the last 3 months? Or whether women are sexually active?

Is there any information on age of menarche?

Why is relationship duration separated into quantiles?

For the test of whether women guide their contraceptive method choices, will the authors use HC use at the last wave as a dichotomous DV? I found that paragraph a bit difficult to follow.