RHEMTULLA Mijke
Recommendations: 0
Review: 1
Review: 1
31 Dec 2024
STAGE 1
How Does Model (Mis)Specification Impact Statistical Power, Type I Error Rate, and Parameter Bias in Moderated Mediation?
Dependence of power and type I error on model misspecification for mediated moderation
Recommended by Zoltan Dienes based on reviews by Mijke Rhemtulla, Pier-Olivier Caron and Reny BaykovaResearchers are often interested in moderated mediation. A predictor variable, such as number of counselling sessions, may predict an outcome, such as approach to a feared object, by way of a mediator, for example number of times the object was described in counselling. The strength of mediation in turn may depend on a moderator, such as vividness of imagery: Counselling reduces fear by way of imaginative exposure, particularly in those with vivid imagery. There may be a number of mediators ("indirect" paths), and any or all of these mediators may be moderated. In testing moderated mediation, a statistical model is specified which may or may not match the data generating process; in particular, there may or may not be moderators in the model corresponding to moderators that may or may not exist in the real data generating process, resulting in overspecification (more moderators of the indirect paths in the model than reality), underspecification (less moderators of indirect paths in the model than reality) or complete misspecification (where the moderated indirect paths in the model are not moderated in reality, and vice versa).
Researchers rely on the validity of tests (correct type I error rates), if they use frequentist statistics. Model misspecification may impact the validity of inferential tests for moderated mediation. Similarly, researchers need to be able to assess power for any analysis. In simulating power for mediated moderation, it may be important to know the possible extent to which the model is misspecified, and take this into account in planning numbers of participants. Fossum et al. (2024) will address this important problem with a series of simulations to determine if power is reduced meaningfully with over or under specification, or type I error and parameter estimates are biased for complete misspecification.
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to reviewers’ and the recommender’s comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.
URL to the preregistered Stage 1 protocol: https://osf.io/8gwfu
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI-RR-friendly journals:
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI-RR-friendly journals:
- Advances in Methods and Practices in Psychological Science *pending editorial consideration of disciplinary fit
- Collabra: Psychology
- Peer Community Journal
- PeerJ
- Royal Society Open Science
- Studia Psychologica
- Swiss Psychology Open
References
Fossum, J. L., Montoya, A. K., & Anderson, S. F. (2024). How Does Model (Mis)Specification Impact Statistical Power, Type I Error Rate, and Parameter Bias in Moderated Mediation? A Registered Report. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/8gwfu