Brooding increases conspiracy beliefs but with practical significance to be determined

I highly appreciate having been part of this review process (my first RR via PCI RR), and in particular, of this thoughtful and rigorous research project. I learned a lot through its findings and comprehensive review of the literature, but also through the high standards the authors used in terms of transparency, openness, and pre-registration.

I will follow the PCI RR criteria for the review of this Stage 2 manuscript, with the hope that my comments are helpful in improving an already solid piece of research.   

2A. Whether the data are able to test the authors’ proposed hypotheses (or answer the proposed research question) by passing the approved outcome-neutral criteria, such as absence of floor and ceiling effects or success of positive controls or other quality checks.

The control group included in the experiment was successfully distinguished from the experimental groups (brooding and reflection conditions) based on the manipulation checks (i.e., measuring self-reported thinking styles used during the task). A second important difference between the control and experimental conditions was the frequency of early terminations of the survey, higher in the brooding and the reflection conditions (as confirmed in the exploratory analyses). This can signal an effect of irritation, boredom, or disengagement that some participants could have experienced during the brooding and reflection tasks (as discussed in Stage 1), and is in line with the authors’ finding that both the brooming AND reflection (but MORE brooming THAN reflection) induced negative affect and decreased positive affect, relative to the control condition. I appreciate that you discuss this issue as a limitation in the General Discussion, as it could introduce an important confound regarding an additional “irritation/boredom” effect of the experimental conditions. In the end, it is plausible that the predicted increase (or the observed lower decrease) of conspiracy beliefs could partly be due to the irritation/boredom experienced during the task being satisfied by the entertaining value of conspiracy beliefs (van Prooijen et al., 2022), which may have led participants to endorse these beliefs more, relative to the control condition.

A small comment about the sample demographics. It is possible that highly educated people, like the majority of your sample, show higher dispositional levels of reflection/brooding due to their training and educational background. Is there any reference to this? If not, perhaps it should also be mentioned in the Limitation section, as a more equally distributed sample may attenuate the effect of your manipulation.

2B. Whether the introduction, rationale, and stated hypotheses (where applicable) are the same as the approved Stage 1 submission. This can be readily assessed by referring to the tracked-changes manuscript supplied by the authors.

The introduction, rationale, and tested hypotheses were exactly the same as the ones approved in the Stage 1 submission.

2C. Whether the authors adhered precisely to the registered study procedures.

Based on their report and their research materials, the authors adhered to the registered study procedures, analysis plan, and their sequential approach to data collection. 

2D. Where applicable, whether any unregistered exploratory analyses are justified, methodologically sound, and informative.

Most exploratory analyses reported by the authors were registered in Stage 1 and are relevant insofar as they offer insight regarding:

a)      the second, exploratory experimental condition (i.e., reflection condition), which had the potential to inform the mixed results of the prior Pilot Studies 1-3,

b)      dependent measures related to theoretically relevant mechanisms of the effect under study (i.e., negative and positive affect), 

c)      the role of conceptually relevant dispositional moderators (i.e., participants’ tendency to brood and their baseline conspiracy beliefs), and

d)      robustness checks to rule out potential random effects of the experimental stimuli (e.g., worry topic of each participant).

The authors further included dropout analyses, unregistered, yet important to clarify the difference in the early terminations between the control and the experimental conditions. 

I assume that the open and transparent report of every exploratory analysis will eventually clash with the word count limitations of some journals. Thus, if necessary, I would make an even shorter mention of the exploratory results in the main paper, and share the full Exploratory Analyses section in the Supplement. 

2E. Whether the authors’ conclusions are justified given the evidence.

I think you did a very good job in summarizing your results and, consistently with how you did throughout the rest of the paper, in acknowledging when evidence or your experimental design was limited. I just have very few minor comments:

p. 42 - “Building on a series of correlational and experimental pilot studies, this Registered Report disentangled the causal effects of two subtypes of rumination on conspiracy beliefs: brooding and reflection.”

Although it is true that your results quantitively distinguished the effect of brooding from the effect of reflection (i.e., a smaller decrease of conspiracy beliefs, and a bigger increase of negative affect due to brooding vs. reflection), there are still some question marks about what underlies this distinction (e.g., level of irritation/boredom, task duration) that would prevent me from claiming that this RR clearly “disentangled the causal effects” of brooding and reflection. Yes, we observe some differences between brooding and reflection, but we do not know exactly why. Thus, I would focus this first paragraph more on the main effect of interest –i.e., the effect of brooding, as you do in the 2nd paragraph of the General Discussion–, and only introduce reflection as an exploratory comparison, as you originally proposed in Stage 1.

P. 44 “…participants aware that a conspiracy is not such an unplausible explanation after all.” Typo, it should be “implausible”.

I really like the section regarding opening a discussion about SESOIs in the subfield of research on conspiracy theories. I totally agree with you that is critical to think about the size of the effects we study, and also about how these effect sizes are related to the temporal features of the sociopsychological phenomenon under study (using your example, the frequency in which people brood over worrying societal issues may determine whether the small effect is cumulative over time, and therefore, its ultimate practical implications). In the case of conspiracy beliefs, they are a phenomenon that unpacks over time, in a process of internalization that goes from their mere entertainment to their integration within a broader system of attitudes and the individual’s social identity (for references on this, see Franks et al., 2017; Sutton & Douglas, 2022). And this is unlikely to be captured by experiments that offer single snapshots, as you describe. Thus, I think that the discussion about SESOIs could be accompanied by a discussion about which experimental designs enable us to predict and capture the temporal characteristics of the effect of interest (e.g., longitudinal designs). This reminded me of your PSPB paper on the longitudinal effects of existential motives on conspiracy beliefs, and how different intervals between waves led to different results. To me, this is another paradigmatic example justifying the importance of this type of conversation, as it showed how some effects might only have short-term (or long-term) consequences, and therefore, might only be observed in longitudinal designs with shorter (longer) intervals between waves.

I wish you the best of luck with the publication of this RR!