Recommendation

Brooding increases conspiracy beliefs but with practical significance to be determined

ORCID_LOGO based on reviews by Matt Williams and Daniel Toribio-Flórez
A recommendation of:
toto

The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report

Abstract

EN
AR
ES
FR
HI
JA
PT
RU
ZH-CN
Submission: posted 19 October 2023
Recommendation: posted 04 December 2023, validated 04 December 2023
Cite this recommendation as:
Chambers, C. (2023) Brooding increases conspiracy beliefs but with practical significance to be determined. Peer Community in Registered Reports, 100580. 10.24072/pci.rr.100580

Recommendation

The world is seemingly awash with conspiracy theories – from well-trodden examples such as fake Moon landings, the 9/11 truth movement, and Holocaust denial, to relative newcomers including COVID as a bioweapon, QAnon, and the belief that the science of climate change has been invented or falsified. While there is a public perception that conspiracy theories are becoming more prevalent, recent evidence suggests that the rate of conspiracism is relatively stable over time (Uscinski et al., 2022). At any point in history, it seems that a certain proportion of people find themselves vulnerable to conspiracy beliefs, but what distinguishes those who do from those who don’t, and what are the causal factors?
 
In the current study, Liekefett et al. (2023) investigated the critical role of rumination – a perseverative and repetitive focus on negative content leading to emotional distress. In particular, the authors asked whether one component of rumination referred to as brooding (dwelling on one’s worries and distressing emotions) has a specific causal role in the formation of conspiracy beliefs. In a series of preliminary experiments, the authors first established a procedure for successfully inducing rumination, identifying various boundary conditions and requirements for a successful design. In the main study (N=1,638 to 2,007 depending on the analysis), they asked whether the induction of brooding causes a significant increase in conspiracy beliefs. Manipulation checks were also included to confirm intervention fidelity (independently of this hypothesis), and exploratory analyses tested the effect of various moderators, as well as the causal role of a complementary manipulation of reflection – a component of rumination in which attention is focused on the issue at hand rather than one’s emotions.
 
As expected by the authors' preliminary work, manipulation checks independently confirmed the effectiveness of the brooding intervention. In answer to the main research question, participants who brooded over the worries and negative emotions associated with an issue were more susceptible to conspiracy beliefs compared to a control group. However, while this effect of brooding was statistically significant, the confidence interval of the effect size estimate overlapped with the authors' proposed smallest effect size of interest (d = 0.20), suggesting that the practical value of the effect remains to be determined.
 
Overall the findings are consistent with a range of psychological theories suggesting that rumination induces negative affect and/or narrows attention to negative information, which in turn may make conspiracy theories seem more probable and render individuals more vulnerable to cognitive bias. The authors note the importance of future work to define the smallest effect of practical significance, analagous to the criteria used to determine the 'minimal clinically important difference’ in medical research.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/y82bs
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Uscinski, J., Enders, A., Klofstad, C., Seelig, M., Drochon, H., Premaratne, K. & Murthi, M. (2022) Have beliefs in conspiracy theories increased over time? PLOS ONE 17: e0270429. https://doi.org/10.1371/journal.pone.0270429

2. Liekefett, L. Sebben, S. & Becker, J. C. (2023). The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report. Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/3e8wc
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Evaluation round #1

DOI or URL of the report: https://osf.io/spw3c?view_only=5286ad5b89584a0ba7d1f238db9aa0b4

Version of the report: 1

Author's Reply, 27 Nov 2023

Decision by ORCID_LOGO, posted 23 Nov 2023, validated 23 Nov 2023

I have now received two reviews of your Stage 2 submission and I'm happy to say that both are very positive. I completely agree with the reviewers that this is a rigorous and well-written RR, and is already very close to meeting the Stage 2 criteria. Within the reviews you will find some helpful points, mostly concerning the interpretration of the results, consideration of limitations (including limits on causal inference) and general conclusions. Provided you are able to address these comments in a revision and response, I anticipate being able to accept your next submission without further review.

Please note: to accommodate reviewer and recommender holiday schedules, PCI RR will be closed to all submissions (including revised submissions) from 1st December 2023 until 10th January 2024. During this time, reviewers will remain able to submit reviews, and recommenders can issue decision letters for ongoing submissions, but no new or revised submissions can be made by authors. Therefore, if you wish to submit a revision before 10th January, please be sure to do so no later than 30th November.

Reviewed by , 22 Nov 2023

I highly appreciate having been part of this review process (my first RR via PCI RR), and in particular, of this thoughtful and rigorous research project. I learned a lot through its findings and comprehensive review of the literature, but also through the high standards the authors used in terms of transparency, openness, and pre-registration.

I will follow the PCI RR criteria for the review of this Stage 2 manuscript, with the hope that my comments are helpful in improving an already solid piece of research.   

2A. Whether the data are able to test the authors’ proposed hypotheses (or answer the proposed research question) by passing the approved outcome-neutral criteria, such as absence of floor and ceiling effects or success of positive controls or other quality checks.

The control group included in the experiment was successfully distinguished from the experimental groups (brooding and reflection conditions) based on the manipulation checks (i.e., measuring self-reported thinking styles used during the task). A second important difference between the control and experimental conditions was the frequency of early terminations of the survey, higher in the brooding and the reflection conditions (as confirmed in the exploratory analyses). This can signal an effect of irritation, boredom, or disengagement that some participants could have experienced during the brooding and reflection tasks (as discussed in Stage 1), and is in line with the authors’ finding that both the brooming AND reflection (but MORE brooming THAN reflection) induced negative affect and decreased positive affect, relative to the control condition. I appreciate that you discuss this issue as a limitation in the General Discussion, as it could introduce an important confound regarding an additional “irritation/boredom” effect of the experimental conditions. In the end, it is plausible that the predicted increase (or the observed lower decrease) of conspiracy beliefs could partly be due to the irritation/boredom experienced during the task being satisfied by the entertaining value of conspiracy beliefs (van Prooijen et al., 2022), which may have led participants to endorse these beliefs more, relative to the control condition.

A small comment about the sample demographics. It is possible that highly educated people, like the majority of your sample, show higher dispositional levels of reflection/brooding due to their training and educational background. Is there any reference to this? If not, perhaps it should also be mentioned in the Limitation section, as a more equally distributed sample may attenuate the effect of your manipulation.

2B. Whether the introduction, rationale, and stated hypotheses (where applicable) are the same as the approved Stage 1 submission. This can be readily assessed by referring to the tracked-changes manuscript supplied by the authors.

The introduction, rationale, and tested hypotheses were exactly the same as the ones approved in the Stage 1 submission.

2C. Whether the authors adhered precisely to the registered study procedures.

Based on their report and their research materials, the authors adhered to the registered study procedures, analysis plan, and their sequential approach to data collection. 

2D. Where applicable, whether any unregistered exploratory analyses are justified, methodologically sound, and informative.

Most exploratory analyses reported by the authors were registered in Stage 1 and are relevant insofar as they offer insight regarding:

a)      the second, exploratory experimental condition (i.e., reflection condition), which had the potential to inform the mixed results of the prior Pilot Studies 1-3,

b)      dependent measures related to theoretically relevant mechanisms of the effect under study (i.e., negative and positive affect), 

c)      the role of conceptually relevant dispositional moderators (i.e., participants’ tendency to brood and their baseline conspiracy beliefs), and

d)      robustness checks to rule out potential random effects of the experimental stimuli (e.g., worry topic of each participant).

The authors further included dropout analyses, unregistered, yet important to clarify the difference in the early terminations between the control and the experimental conditions. 

I assume that the open and transparent report of every exploratory analysis will eventually clash with the word count limitations of some journals. Thus, if necessary, I would make an even shorter mention of the exploratory results in the main paper, and share the full Exploratory Analyses section in the Supplement. 

2E. Whether the authors’ conclusions are justified given the evidence.

I think you did a very good job in summarizing your results and, consistently with how you did throughout the rest of the paper, in acknowledging when evidence or your experimental design was limited. I just have very few minor comments:

p. 42 - “Building on a series of correlational and experimental pilot studies, this Registered Report disentangled the causal effects of two subtypes of rumination on conspiracy beliefs: brooding and reflection.”

Although it is true that your results quantitively distinguished the effect of brooding from the effect of reflection (i.e., a smaller decrease of conspiracy beliefs, and a bigger increase of negative affect due to brooding vs. reflection), there are still some question marks about what underlies this distinction (e.g., level of irritation/boredom, task duration) that would prevent me from claiming that this RR clearly “disentangled the causal effects” of brooding and reflection. Yes, we observe some differences between brooding and reflection, but we do not know exactly why. Thus, I would focus this first paragraph more on the main effect of interest –i.e., the effect of brooding, as you do in the 2nd paragraph of the General Discussion–, and only introduce reflection as an exploratory comparison, as you originally proposed in Stage 1.

P. 44 “…participants aware that a conspiracy is not such an unplausible explanation after all.” Typo, it should be “implausible”.

I really like the section regarding opening a discussion about SESOIs in the subfield of research on conspiracy theories. I totally agree with you that is critical to think about the size of the effects we study, and also about how these effect sizes are related to the temporal features of the sociopsychological phenomenon under study (using your example, the frequency in which people brood over worrying societal issues may determine whether the small effect is cumulative over time, and therefore, its ultimate practical implications). In the case of conspiracy beliefs, they are a phenomenon that unpacks over time, in a process of internalization that goes from their mere entertainment to their integration within a broader system of attitudes and the individual’s social identity (for references on this, see Franks et al., 2017; Sutton & Douglas, 2022). And this is unlikely to be captured by experiments that offer single snapshots, as you describe. Thus, I think that the discussion about SESOIs could be accompanied by a discussion about which experimental designs enable us to predict and capture the temporal characteristics of the effect of interest (e.g., longitudinal designs). This reminded me of your PSPB paper on the longitudinal effects of existential motives on conspiracy beliefs, and how different intervals between waves led to different results. To me, this is another paradigmatic example justifying the importance of this type of conversation, as it showed how some effects might only have short-term (or long-term) consequences, and therefore, might only be observed in longitudinal designs with shorter (longer) intervals between waves.

I wish you the best of luck with the publication of this RR!

Reviewed by ORCID_LOGO, 12 Nov 2023

User comments

No user comments yet