The Causal Effects of Hormonal Contraceptives on Psychological Outcomes

The authors did an excellent job of responding to comments and updating their registered report. I was really impressed with the revised report, and approve of the current version. I had just had one actual note:

1. HCs are often prescribed to both homosexual women and women who are not sexually active. The assumption that women who have never had sex will not be using HCs seems a bit flawed - to me, this is just missing data. It is more likely that they are not using HCs, but not certain. The authors should consider treating this as missing data in the main analysis as opposed to just in a robustness analysis. In a related vein, perhaps a robustness analysis could also include homosexual women? I trust the authors to consider this feedback and adjust things if appropriate, and do not need a response to this comment. 

Minor things I noticed in my read through:

1. Pg 1: missing commas here and there (e.g., between exsisted and which; after But); consider changing like to such as "...such as how HCs influence sexuality and well-being." I thought I saw a missing oxford comma too, but maybe I imagined that. 

2. Pg 19: missing word of "...inform their decision OF which..."

3. Pg 21: missing a space after ;

4. Table 3: check the header formmatting, new column seems one character too small!

I have gone through the authors' reply to reviewer comments in Round 1 and the new manuscript file with tracked changes. I believe the authors have sufficiently addressed the previous concerns and I did not see any additional concerns (ones that I have not raised before).

The authors propose a series of analyses to be conducted on a large, longitudinal dataset containing data from 6,565 women collected over the course of 13 waves (waves collected annually, average of 6 waves per participant). The authors seek to estimate the effect of hormonal contraceptive (HC) use on both sexual outcomes and well-being. Further, they also seek to investigate the heterogeneity in women’s responses to HC treatment and if women’s treatment responses predict their future HC use. Generally, I found the research questions to be well supported and the relevant literature to be well described. The research questions the authors propose are of keen scientific interest, and the authors’ choice of study design is appropriate to answer these questions. That is, naturalistic, longitudinal studies are certainly one vital piece of the puzzle needed to understand how HC use impacts both women’s sexuality and well-being. While I am not as statistically literate as the authors, their statistical approach appears well conceived and appropriate, although given my limited knowledge of Bayesian models in practice, I will ask a few clarifying questions below to ensure that these approaches are indeed appropriate and would not benefit from supplementation with additional exploratory analyses or alternate approaches. While of acceptable standards, there are a few points in the methodology and data analysis plan which I believe could use additional clarification before I recommend this Stage 1 registered report, which I detail below. My only major concerns with the data analytic strategy lies in the authors’ abilities to answer the questions they seek to answer given the limited specificity of the data available and if this is being appropriately modeled, given the lack of a statistical model included in the report. Finally, I would like to see the authors provide what their interpretation will be for potential results, to get an idea of what the data analysis will reveal. Overall, I found this to be an impressively well written registered report and an interesting research study.

Specific points:

1.      The variable options available in this dataset are somewhat limited. For example, there is an option for IUDs in type of contraceptive used, but no specification of if this is a hormonal or nonhormonal IUD – is this information provided in a follow up question? How will the authors determine if a woman is using HCs if she reports an IUD? I saw that IUD was a potential exclusion criterion, but that does prevent analysis of changing methods to/from HCs in the event that one of the options is an IUD (hormonal or copper). How do the authors plan to handle this confound? This seems important, given the popularity of IUDs as both a hormonal and non-hormonal contraceptive option.

a.      Is there any more specificity in the data about what actual type of HC is being used? I worry that given the likelihood that different methods have different effects (and perhaps more meaningfully, different effect sizes) that lumping methods into such broad categories will limit the impact of the results and the reliability of effect size estimates.

b.      Likewise, I am surprised that the authors did not plan to investigate how oral vs non-oral HC methods differed in their effects. I would like to see this question addressed within the current protocol, given that it seems like a decently simple difference to investigate here and would be of interest to other HC researchers. (Perhaps the authors are planning to do this and I misinterpreted this plan)

2.      Instead of excluding women who report homosexual relationships, I would prefer to see models run with and without these exclusions. Additionally, I am a bit confused as to why women using no contraceptive method are being excluded from analysis – it seems to me that they are an important comparison group for HC users, given the goals of the research design. Perhaps the authors had considered this and have another reason for this exclusion, however, I would like to see this explained or an additional exploratory analysis conducted.

a.      As a follow-up: excluding women who are sterilized, use a nonhormonal IUD, who are trying to become pregnant, or are just not using a listed contraceptive method is excluding a large amount of women who are free from HC use. Comparing the effects of using HCs to only women using certain other methods but not the previously listed circumstances seems a limitation here. Maybe that is the better way to approach this (no study is without limitations) however I would like the authors to consider this potential limitation in the design.

3.      I have some concerns about grouping all effects within each DV (sexual desire, sexual behavior, satisfaction; depression, life satisfaction, self-esteem). I would like to see an exploration of each of these individually and one of them comprised into a latent factor/composite of sexuality and well-being. This may be what the authors were intending however this is not specified (and my apologies I am not great at reading other people’s code, so I would prefer to see the statistical model included in the report).

4.      Is there a plan to control for how having never had sex might impact results on sexuality? Perhaps a follow-up analysis with and without these women for sexuality would be appropriate.

5.      Figure 1 – while this is an adequate conceptual model, I would like to see more information regarding the statistical model included so that I understand how variables are being treated in analyses. What does relationship mean in this figure? (Relationship status/length?) I would like to see the names of the constructs the same in figures as in tables.

6.      Table 1 – please add a column which breaks down how results will be interpreted (especially when multiple modeling approaches will be used to answer the same questions) per the PCI RR recommendations.

7.      For the heterogeneity question – I would love to see this question addressed with more statistical nuance than comparing the percentage of women with positive effects (guessing effects here is meant to refer to the totality of sexuality effects and well-being effects separately, but the authors do not adequately explain how they plan to conceptualize these comparison variables) to the percentage with negative effects. Here, again, I would be interested to see if there is heterogeneity in responses to treatment in each DV and the latent/composite DVs. I would also be curious if there was a way to profile those with different effects (not just positive or negative) based on the groupings of their responses to treatment. (e.g., is there a group of women with high self-esteem and sex drive, but low sexual behavior? Another group with high sexuality and high depression? Is there something that predicts membership to these groups?)

8.      For the question of investigating if adverse experiences on HCs influence future use – how does investigating relationships between how long a woman has been using HCs and her future use answer this question? Are there additional variables being included in the model to answer this question that I am missing? How are positive or negative experiences being conceptualized?

9.      For data simulations, how was -.45 selected as the estimate of the true causal effect?

10.   Why were the following variables not considered here: age at first HC use, duration of HC use prior to the study beginning, childhood trauma, pre-existing mental health Dx. If data not available, this is absolutely appropriate, however, given that these factors are more supported as influencing HC treatment outcomes than are personality traits, I would expect these to be included.

11.   Given the multilevel nested nature of this data (and my own training), I might have approached analyses using a RI-CLPM. I would love if the authors could describe why they selected their analysis plan in comparison to this type of analysis plan (or other appropriate models)! That is, the authors explained why their approach was a good approach, but not why it might be a better way to answer this question compared to other approaches! I am very ready to be convinced.

This manuscript presents the Stage 1 proposal to examine relating to effects of hormonal contraceptive use and two outcome measures (sexual desire and well-being, which will be presented in 2 separate Stage 2 manuscripts). The research question is important and I commend the authors for finding a dataset that can be used to test their hypotheses (though I understand that the research is not actually testing hypotheses as much as describing the strength of associations). This will be an important contribution to the literature. I did find some aspects of the Stage 1 proposal unclear and/or difficult to follow. I list specific concerns below, ordered by section.

Introduction

I found the logic transition from talking about the need to conduct experiments to examining treatment effects (in the first part of the introduction) to be muddy. I think the authors can create a stronger framing for their use of the longitudinal data. I would de-emphasize experimental approaches and just focus on the strength of the present approach, which is strong in its own right. I am also then unsure about the focus on randomized control trials right after listing the five possible explanations for the mixed findings in the literature. Again, I would reframe to focus on the current method. If a comparison to randomized controlled studies is needed, I would add it after.

I had to read this sentence over multiple times before I understood what exactly was being said because it seems to start a completely different point than the beginning of the paragraph (the sentence that starts with "To apply this perspective to the current research question, assume that for example an ineffable..." on page 11). I think the authors can make this point in a more straight-forward way. Perhaps by introducing the time-varying versus time-invarying terms first.

Skovlund et al., (2016) found age to be particularly important for whether HC use was associated with greater risk for depression. I think this point should be more thoroughly discussed. Additionally, are any of the findings in the sexuality section similarly moderated by age? The Skovlund reference may also need to be mentioned after the "Older women might be more likely to experience beneficial side effects.." on page 14. The given reason for this association focuses on finding a method that fits them best, however, is there any evidence that sensitivity to steroid hormones decreases with age as well?

Conceptual Design and Underlying Assumptions

In both the time-lagged regression and the IPTW approach, the model includes the interaction of the outcome and contraceptive method at the previous assessment (in addition to the main effects). It is not clear to me conceptually why the interaction term is needed. Can the authors clarify their hypothesis? If the interaction term is not significant, will it remain in the model?

For variables related to relationship status, would it not be best to include relationship status at previous assessment to examine change in relationship status? This seems like a better test of the theoretical rationale that was laid out (about starting a relationship being a potential confounder).

Would the data allow for some sort of latent class analysis to classify women as having positive, negative, or neutral effects?

Method and Analysis Plan

Do the authors mean that women will be excluded if they ever hit 50 or menopause during the data collection period or only their timepoints after 50/menopause (i.e., earlier timepoints from those women would be included)? That sentence was unclear to me.

What is the logic for excluding women in homosexual relationships? Research on cycle effects documents similar shifts in women who are in homosexual relationships as women who are in heterosexual relationships.

I am not sure why women who indicated using no contraceptive method will be excluded in robustness analyses. That sentence does not make sense to me. Specifically, after reading this sentence, I became confused about the "control group" - will it just be users of NON-HC METHODS or anyone not currently on a HC? NON-OC METHODS? The authors should specify exactly how hormonal contreceptive use will be operationalized from the questions in the PARFAIM dataset (if this is reported, it is not prominent enough).

Are there questions on whether women had sex in the last 3 months? Or whether women are sexually active?

Is there any information on age of menarche?

Why is relationship duration separated into quantiles?

For the test of whether women guide their contraceptive method choices, will the authors use HC use at the last wave as a dichotomous DV? I found that paragraph a bit difficult to follow.