Submit a report

Announcements

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.


 

Latest recommendationsrssmastodon

IdTitleAuthorsAbstractPictureThematic fieldsRecommenderReviewersSubmission date
11 Apr 2023
STAGE 2
(Go to stage 1)
toto

Stage 2 Registered Report: Stress regulation via being in nature and social support in adults, a meta-analysis

Does emotional support and being in nature influence stress?

Recommended by ORCID_LOGO based on reviews by Felix Schönbrodt and Siu Kit Yeung
Stress is a familiar presence in modern life and may be rising in severity (Almeida et al., 2020). As a key driver of many health problems, controlling stress and its impacts is a central goal in clinical and health psychology, yet the effectiveness of existing interventions to regulate stress remains unclear. 
 
In the current study, Sparacio et al tackled this question from a meta-analytic perspective, focusing on a corpus of existing research that has addressed the efficacy of two specific stress regulation interventions: being in nature and emotional social support. As well as evaluating the evidential content of the relevant literatures, the authors also examined signs of publication bias and the moderating role of personality traits.
 
After correcting for publication bias, the results reveal evidence that being in nature is effective at reducing stress while emotional social support is not. The moderating role of personality for both interventions was inconclusive due to lack of evidence. In addition, the quality of the surveyed literature was found to be low overall, suffering from a high risk of bias and high rate of statistical reporting errors. The authors offer several recommendations to improve the rigour and quality of studies in this field, including open data, open materials, code review, preregistration and the use of Registered Reports.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/c25qw
 
Level of bias control achieved: Level 3. At least some data/evidence that was used to the answer the research question had been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they did not observe ANY part of the data/evidence prior to Stage 1 IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Almeida, D. M., Charles, S. T., Mogle, J., Drewelies, J., Aldwin, C. M., Spiro, A. III, & Gerstorf, D. (2020). Charting adult development through (historically changing) daily stress processes. American Psychologist, 75(4), 511–524. https://doi.org/10.1037/amp0000597
 
2. Sparacio, A., Ropovik, I., Jiga-Boy, G. M., Lağap, A. C. & IJzerman, H. (2023). Stage 2 Registered Report: Stress regulation via being in nature and social support in adults, a meta-analysis. Acceptance of version 2 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/a4zmj
Stage 2 Registered Report: Stress regulation via being in nature and social support in adults, a meta-analysisAlessandro Sparacio, Ivan Ropovik, Gabriela M. Jiga-Boy, Adar Cem Lağap, Hans IJzerman<p>In this meta-analysis, the authors investigated whether being in nature and emotional social support are reliable strategies to downregulate stress. We retrieved all the relevant articles that investigated a connection between one of these two ...Social sciencesChris Chambers2023-01-09 09:32:27 View
11 Apr 2023
STAGE 2
(Go to stage 1)
toto

Does childhood adversity alter opioid drug reward? A conceptual replication in outpatients before surgery

Is childhood adversity associated with a heightened response to opioids?

Recommended by ORCID_LOGO based on reviews by Zoltan Dienes, Yuki Yamada and 1 anonymous reviewer
A convergence of evidence suggests that early life adversity may cause dysfunction in opioid-sensitive reward systems. Childhood adversity is associated with opioid use, potentially by altering reward and motivation networks, and experimental models in animals have found that early life adversity increases and consolidates opioid seeking behaviours. Further, in a recent controlled experiment, Carlyle et al. (2021) found that opioid administration produced stronger positive responses, and weaker negative responses, in adults with a history of childhood abuse and neglect.
 
In the current study, Carlyle et al. (2023) tested the generalisability of these previous findings in a pre-operative clinical setting. Using partially observed data from an existing cohort study (N=155), the authors asked whether patients with greater experience of childhood trauma in turn exhibit a larger mood boost and express greater subjective pleasure following opioid administration.
 
In contrast to previous findings, the results did not support the hypotheses that more experiences of childhood adversity would heighten ratings of drug liking and feeling good following opioid administration. Regression analyses instead revealed a statistically significant negative association between childhood adversity and post-opioid liking and no significant relationship with feeling good. The authors suggest that the discrepancy between the current and previous results may be due to stress related to the pre-surgical setting, the brief duration of drug exposure, and the relatively limited levels of high childhood adversity in the study sample. Nevertheless, these findings cast some doubt on the theory that adversity elevates risk of opioid addiction by altering sensitivity to subjectively pleasurable effects.
 
Following one round of in-depth review, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/7ymts
 
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to IPA, but the authors certify that they had not yet observed the key variables within the data that were used to answer the research question AND they took additional steps to maximise bias control and rigour. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Carlyle M., Broomby R., Simpson G., Hannon R., Fawaz L., Mollaahmetoglu O.M., Drain, J., Mostazir, M., & Morgan C. (2021). A randomised, double‐blind study investigating the relationship between early childhood trauma and the rewarding effects of morphine. Addiction Biology, 26(6):e13047. https://doi.org/10.1111/adb.13047
 
2. Carlyle, M., Kvande, M., Meier, I. M., Trøstheim, M., Buen, K., Jensen, E. N., Ernst, G. & Leknes, S. & Eikemo, M. (2023). Does childhood adversity alter opioid drug reward? A conceptual replication in outpatients before surgery, acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/9kt3a?view_only=4238d2ee3d654c4f908a94efea82a027
Does childhood adversity alter opioid drug reward? A conceptual replication in outpatients before surgeryMolly Carlyle1*, Malin Kvande*, Isabell M. Meier, Martin Trøstheim, Kaja Buen, Eira Nordeng Jensen, Gernot Ernst, Siri Leknes, Marie Eikemo (*denotes equal contribution)<p><strong>Introduction</strong>: Opioid analgesic treatment during surgery entails risk of persistent use. Experiences of childhood adversity have been shown to increase opioid reward in preclinical models, a finding recently extended to healthy ...Medical Sciences, Social sciencesChris Chambers2023-02-02 08:29:18 View
11 Apr 2023
STAGE 1
toto

The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1

Is empathy important for forgiveness?

Recommended by ORCID_LOGO based on reviews by Wenrui Cao, James Bartlett and Saleh Shuqair
Forgiveness is a core feature of human psychology in which a person makes a deliberate decision to cease negative emotions or attitudes toward an offender who has done them harm. The concept of interpersonal forgiveness is deeply embedded across societies, but much remains to be understood about how it actually works. What are its key ingredients and why does it occur in the first place? Research in social psychology has demonstrated a range of personal and social benefits of forgiveness, giving rise to two dominant mechanistic accounts – one that positions empathy as the driving factor and another that centres motivated reasoning (Donovan & Priester, 2017).
 
In the current study, Chan and Feldman (2023) seek to replicate a formative study by McCullough et al (1997) that led to the so-called Empathy Model of forgiveness. According to this theory, forgiving is a motivational change facilitated (crucially) by empathy, promoting constructive over destructive behaviour toward the offender. Chan and Feldman will replicate Study 1 from McCullough et al., measuring the correlational relationship between apology, forgiving, and empathy for offenders, and exploring whether forgiving is associated with increased conciliation and decreased avoidance motivation. As well as closely replicating the original study, the authors will extend it to test the more severe hypothesis that empathy causally influences forgiveness. To achieve this, they will experimentally manipulate empathy by adding two groups to the design: one in which participants are asked to recall hurtful past experiences in which they were not empathetic to the offender, and another in which they were highly empathetic.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/q78fs
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Donovan, L. A. N., & Priester, J. R. (2017). Exploring the psychological processes underlying interpersonal forgiveness: The superiority of motivated reasoning over empathy. Journal of Experimental Social Psychology, 71, 16-30. https://doi.org/10.1016/j.jesp.2017.02.005
 
2. Chan, C. F. & Feldman, G. (2023). The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/q78fs
The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1Chan Chi Fung, Gilad Feldman <p>[IMPORTANT: Abstract, method, and results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection. For the purpose o...Social sciencesChris Chambers Wenrui Cao, James Bartlett2023-01-25 11:15:13 View
07 Apr 2023
STAGE 1
toto

Psychological predictors of long-term esports success: A Registered Report

What psychological factors predict long-term success in esports?

Recommended by and ORCID_LOGO based on reviews by Justin Bonny and Maciej Behnke
Electronic sports (esports), the competitive play of video games, has seen a large surge in popularity over the past few decades. Millions of people nowadays participate in esports as a hobby, and many consider becoming professional esports athletes as a potential career path. However, psychological factors that may predict one's long-term success in esports have remained unclear.

In the current study, Martončik and colleagues (2023) propose to examine potential predictors of long-term esports success, in three currently most impactful PC esports games, namely League of Legends, Counter Strike: Global Offensive, and Fortnite. Based on an extensive review of the literature and four pilot studies, the authors will examine to what extent naive practice and deliberate practice, as well as other psychological factors such as attention, speed of decision-making, reaction time, teamwork, intelligence and persistence, can predictor player's highest rank in the past 12 months, as an indicator of long-term success. Deliberate practice has been proposed to play an essential role in the development of expertise in other domains, and the current study offers a test of the role of both naive and deliberate practice in long-term esports success. The novel measurement on naive and deliberate practice, developed as part of the current investigation, will also be a valuable contribution to future research on esports. Lastly, from an applied perspective, the results of the current study will be of great interest to individuals who are considering pursuing a professional career in esports, as well as professional and semi-professional esports teams and coaches.

This Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on the comprehensive responses to the reviewers' feedback, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/84zbv
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals: 
 
References
 
Martončik, M., Karhulahti, V.-M., Jin, Y. & Adamkovič, M. (2023). Psychological predictors of long-term esports success: A Registered Report, in principle acceptance of Version 1.4 by Peer Community in Registered Reports. https://osf.io/84zbv
Psychological predictors of long-term esports success: A Registered ReportMarcel Martončik, Veli-Matti Karhulahti, Yaewon Jin, Matúš Adamkovič<p>The competitive play of digital games, esports, has attracted worldwide attention of hundreds of millions of young people. Although esports players are known to practice in similar ways to other athletes, it remains largely unknown what factors...Social sciencesZhang Chen2022-08-17 12:12:51 View
07 Apr 2023
STAGE 1
toto

The WEIRD problem in a “non-WEIRD” context: A meta-research on the representativeness of human subjects in Chinese psychological research

How well do "non-WEIRD" participants in multi-lab studies represent their local population?

Recommended by ORCID_LOGO based on reviews by Zoltan Dienes, Patrick Forscher and Kai Hiraishi
In this protocol, Yue et al. (2023) aim to clarify whether the sample of non-WEIRD countries included in multi-lab studies is actually representative of those countries and cultures. Focusing on China, this study will compare Chinese samples in several multi-lab studies with participants in studies published in leading national Chinese journals on various aspects, including demographic data and geographic information. This work will provide useful information on the extent to which multi-lab studies are able to deal with generalizability, especially as they intend to address the generalizability problem.
 
The Stage 1 manuscript was reviewed by three experts, including two with an interest in the WEIRD problem and a wealth of experience in open science and multi-lab research, plus an expert in Bayesian statistics, which this manuscript uses. Following multilpe rounds of peer review, and based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/ehw54
 
Level of bias control achieved: Level 4. At least some of the data/evidence that will be used to answer the research question already exists AND is accessible in principle to the authors (e.g. residing in a public database or with a colleague) BUT the authors certify that they have not yet accessed any part of that data/evidence.
 
List of eligible PCI RR-friendly journals: 
 
 
References
 
Yue, L., Zuo, X.-N., & Hu, C.-P. (2023) The WEIRD problem in a “non-WEIRD” context: A meta-research on the representativeness of human subjects in Chinese psychological research, in principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/ehw54
The WEIRD problem in a “non-WEIRD” context: A meta-research on the representativeness of human subjects in Chinese psychological researchYUE Lei, ZUO Xi-Nian, HU Chuan-Peng<p><strong>​​​​</strong><strong>​Psychological science aims at understanding human mind and behavior, but it primarily relies on subjects from Western, Educated, Industrialized, Rich, and Democratic regions, i.e., the WEIRD problem. This lack of d...Social sciencesYuki Yamada Zoltan Dienes2021-09-07 11:25:52 View
29 Mar 2023
STAGE 2
(Go to stage 1)
toto

Exploring How Members of Illicit Networks Navigate Investigative Interviews

What to say to help one's partners in crime

Recommended by ORCID_LOGO based on reviews by Lorraine Hope
When interviewing members of a criminal network, what determines the information a given interviewee chooses to disclose, as guided by the network's collective planning? What principles could help inform a detective preparing for such interviews? In the current study, Neequaye et al. recruited groups of people known to each other to assume the role of networks that run an illegal sports betting business, fronting as a chain of tanning salons. Although each network launders money, they have to come up with a strategy to convince investigators they are legit. The groups are motivated to disclose some information when individuals are interviewed, but only enough to appear cooperative. Members disclosed information they perceived would yield benefical outcomes, but the extent to which members disclosed varied substantially according to the groups they were in.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/n7ugr
 
Level of bias control achieved: Level 6. Data collection began during the final round of Stage 1 peer review. Since no further revisions were made after this review round, the risk of bias due to prior data observation remained zero, and the manuscript therefore qualified for Level 6.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Neequaye, D. A., Granhag, P. A. & Luke, T. J. (2023). Exploring How Members of Illicit Networks Navigate Investigative Interviews. Acceptance of Version 3 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/f3ct4
Exploring How Members of Illicit Networks Navigate Investigative InterviewsDavid A. Neequaye, Pär Anders Granhag, Timothy Luke<p>This study explored how members of an illicit network navigate investigative interviews probing their crimes. We examined how perceived disclosure outcomes, namely, the projected costs and benefits, affect what members choose to reveal. We recr...Humanities, Social sciencesZoltan Dienes2022-07-11 15:21:09 View
27 Mar 2023
STAGE 1
article picture

Optimizing Esports Performance Using a Synergistic Mindsets Intervention

Synergistic Mindset Intervention for Competitive Situations

Recommended by ORCID_LOGO based on reviews by Lee Moore, Ivan Ropovik , Ivana Piterová and Jacob Keech
Mindset theories suggest that the mere belief in the malleability of human abilities can already help one to develop related performance. On the other hand, one and the same performance situation can also be experienced in various affective ways, which differently contribute to performance outcomes. Arguably, appraising a performance situation as a “threat” instead of “challenge” is associated with maladaptive responses, such as impaired cardiovascular mobilization. If people could experience performance situations as positive challenges, this might also improve performance outcomes. Drawing from these connected theoretical premises, the synergistic mindset intervention was developed and tentatively found to help adolescents in stressful situations (Yeager et al., 2022).
 
In the present registered report, Behnke et al. (2023) build on the above and test whether the synergistic mindset intervention can help individuals in competitive gaming situations. The authors utilize one of the leading esport games, Counter-Strike: Global Offensive​, and recruit its active players into randomized control and intervention groups for two weeks. Ultimately, the participants compete in a cash-prize tournament involving measures of affective experience and cardiovascular responses. Behnke et al. (2023) hypothesize that the synergistic mindset group will show greater challenge affective responses and superior performance outcomes. As such, the study design has significant potential to generate valuable evidence for various theoretical models and the synergistic mindset model in particular.
 
The Stage 1 manuscript was evaluated over two rounds by four experts with experimental psychology specializations in mindsets, stress, and statistics. Based on the comprehensive responses to the reviewers' feedback, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/z3adb
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals: 
 
 
References
 
Behnke M., Lakens D., Petrova K., Chwiłkowska P., Kaczmarek L. D., Jamieson J. P., & Gross J. J. (2023) Optimizing Esports Performance Using a Synergistic Mindsets Intervention. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/z3adb

Yeager D.S., Bryan C.J., Gross J.J., Murray J., Krettek D., Santos P., ... & Jamieson J.P. (2022) A synergistic mindsets intervention protects adolescents from stress. Nature 607, 512–520. https://doi.org/10.1038/s41586-022-04907-7
Optimizing Esports Performance Using a Synergistic Mindsets InterventionMaciej Behnke, Daniël Lakens, Kate Petrova, Patrycja Chwiłkowska, Lukasz D. Kaczmarek, Jeremy P. Jamieson, James J. Gross<p>Affective responses during stressful high-stakes situations can play an important role in shaping performance outcomes. For example, feeling shaky and nervous at a job interview can undermine performance, whereas feeling pumped and excited duri...Social sciencesVeli-Matti Karhulahti2023-01-04 10:12:55 View
23 Mar 2023
STAGE 2
(Go to stage 1)
toto

Revisiting the role of public exposure and moral beliefs on feelings of shame and guilt: Replication Registered Report of Smith et al. (2002)’s Study 1

The effect of public exposure and moral beliefs on feelings of shame and guilt

Recommended by ORCID_LOGO based on reviews by Uriel Haran
Shame and guilt are powerful negative emotions that are notable for their external vs. internal focus: while shame is generally experienced in response to public scrutiny, guilt arises from a self-directed, private evaluation. In a formative study, Smith et al. (2002) asked whether the level of public exposure influenced levels of shame and guilt arising from one's transgressions, and found that, compared to private situations, public exposure was more strongly associated with shame than with guilt. Since then, these findings have had significant implications for theories and applications of moral psychology.
 
In the current study, Zhang et al. (2023) directly replicated Smith et al. (2002) in a large online sample, revisiting two critical questions from Study 1: (a) whether public exposure affects the magnitude of shame and guilt over one’s misconduct, and (b) whether stronger moral belief increases guilt and shame over one’s misconduct. The results fail to confirm the original conclusions: both public exposure and manipulation of moral beliefs were found to influence shame and guilt, with no reliable evidence that shame was influenced more strongly than guilt. These findings thus constitute a non-replication and offer a challenge to theoretical models that hinge on the separability of shame and guilt as separate constructs.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewer's comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/j7kt2
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Smith, R. H., Webster, J. M., Parrott, W. G., & Eyre, H. L. (2002). The role of public exposure in moral and nonmoral shame and guilt. Journal of Personality and Social Psychology, 83, 138-159. https://doi.apa.org/doi/10.1037/0022-3514.83.1.138
 
2. Zhang, Y., Cheung, F. C., Wong, H.T., Yuen, L. Y., Sin, H. C., Chow, H. T. & Feldman, G. (2023). Revisiting the role of public exposure and moral beliefs on feelings of shame and guilt: Replication Registered Report of Smith et al. (2002)’s Study 1. Acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/jpx87
Revisiting the role of public exposure and moral beliefs on feelings of shame and guilt: Replication Registered Report of Smith et al. (2002)’s Study 1 Yikang Zhang, Fung Chit (Jack) Cheung, Hei Tung (Patrina) Wong, Lok Yee (Noel) Yuen, Hui Ching (Rachel) Sin, Hiu Tung Kristy Chow, Gilad Feldman<p>Shame and guilt are unpleasant self-conscious emotions associated with negative evaluations of oneself or one’s behavior. Smith et al. (2002) demonstrated that shame and guilt are distinct and are impacted differently by public exposure, that i...Social sciencesChris Chambers2023-02-03 10:58:20 View
23 Mar 2023
STAGE 2
(Go to stage 1)
toto

Revisiting the links between numeracy and decision making: Replication Registered Report of Peters et al. (2006) with an extension examining confidence

Assessing the replicability of specific links between numeracy and decision-making

Recommended by ORCID_LOGO based on reviews by Elena Rusconi
Numeracy – the ability to understand and work with numbers – is associated with a wide range of social and health-related outcomes, including socioeconomic status, employment, literacy, reasoning, and life satisfaction. A substantial body of evidence has also shown links between numeracy and decision-making, prompting the question of how it relates to finer-grained measures of reasoning, judgment and affect/emotion.
 
In the current study, Zhu and Feldman repeated four influential experiments from a study by Peters et al. (2006), which reported links between numeracy and performance on a variety of decision-making tasks, including attribute framing, frequency-percentage framing, susceptibility to affective influences, and various cognitive biases. The authors also explored several extended questions, including refinements of the original hypotheses and an examination of the relationship between numeracy and confidence in numeric judgments (subjective numeracy).
 
The results broadly constitute a successful replication, with higher numeracy associated with weaker attribute framing and susceptibility to bias. The relationship between numeracy and the frequency-percentage framing effect – that is, the change in decision-making when numbers are presented as frequencies (e.g. 5 out of 100) rather than percentages (e.g. 5%) – was inconclusive for the main analysis that treated numeracy as a categorical variable (low vs. high); however the link emerged reliably in exploratory analyses that considered numeracy as a continuous variable. The outcomes of the extended analyses were mixed, revealing evidence for a potentially weak relationship between numeracy and confidence.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewer's comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/r73fb
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Zhu, M. & Feldman, G. (2022). Revisiting the links between numeracy and decision making: Replication Registered Report of Peters et al. (2006) with an extension examining confidence. Acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/62wqb
 
2. Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17, 407-413. https://doi.org/10.1111%2Fj.1467-9280.2006.01720.x
Revisiting the links between numeracy and decision making: Replication Registered Report of Peters et al. (2006) with an extension examining confidenceMinrui Zhu, Gilad Feldman<p>Numeracy is individuals’ capacity to understand and process basic probability and numerical information required to make decisions. We conducted a Replication Registered Report of Peters et al. (2006) examining numeracy as a predictor of positi...Social sciencesChris Chambers2023-01-16 10:34:09 View
21 Mar 2023
STAGE 1
toto

Convenience Samples and Measurement Equivalence in Replication Research

Does data from students and crowdsourced online platforms measure the same thing? Determining the external validity of combining data from these two types of subjects

Recommended by ORCID_LOGO based on reviews by Benjamin Farrar and Shinichi Nakagawa
Comparative research is how evidence is generated to support or refute broad hypotheses (e.g., Pagel 1999). However, the foundations of such research must be solid if one is to arrive at the correct conclusions. Determining the external validity (the generalizability across situations/individuals/populations) of the building blocks of comparative data sets allows one to place appropriate caveats around the robustness of their conclusions (Steckler & McLeroy 2008).
 
In this registered report, Alley and colleagues plan to tackle the external validity of comparative research that relies on subjects who are either university students or participating in experiments via an online platform (Alley et al. 2023). They will determine whether data from these two types of subjects have measurement equivalence - whether the same trait is measured in the same way across groups. Although they use data from studies involved in the Many Labs replication project to evaluate this question, their results will be of crucial importance to other comparative researchers whose data are generated from these two sources (students and online crowdsourcing). If Alley and colleagues show that these two types of subjects have measurement equivalence, then this indicates that it is more likely that equivalence could hold for other studies relying on these type of subjects as well. If measurement equivalence is not found, then it is a warning to others to evaluate their experimental design to improve validity. In either case, it gives researchers a way to test measurement equivalence for themselves because the code is well annotated and openly available for others to use.

The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).

URL to the preregistered Stage 1 protocol: https://osf.io/7gtvf
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour (e.g. conservative statistical threshold; recruitment of a blinded analyst; robustness testing, multiverse/specification analysis, or other approach) 
 
List of eligible PCI RR-friendly journals:
 
 
References
 
Alley L. J., Axt, J., & Flake J. K. (2023). Convenience Samples and Measurement Equivalence in Replication Research, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/7gtvf
 
Steckler, A. & McLeroy, K. R. (2008). The importance of external validity. American Journal of Public Health 98, 9-10. https://doi.org/10.2105/AJPH.2007.126847
 
Pagel, M. (1999). Inferring the historical patterns of biological evolution. Nature, 401, 877-884. https://doi.org/10.1038/44766
Convenience Samples and Measurement Equivalence in Replication ResearchLindsay J. Alley, Jordan Axt, Jessica Kay Flake<p>A great deal of research in psychology employs either university student or online crowdsourced convenience samples (Chandler &amp; Shapiro, 2016; Strickland &amp; Stoops, 2019) and there is evidence that these groups differ in meaningful ways ...Social sciencesCorina Logan2022-11-29 18:37:54 View