Submit a report

Announcements

Please note that we will be CLOSED to ALL SUBMISSIONS from 1 December 2024 through 12 January 2025 to give our recommenders and reviewers a holiday break.

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.

Latest recommendationsrssmastodon

IdTitle * Authors * Abstract * PictureThematic fields * RecommenderReviewersSubmission date
04 Dec 2023
STAGE 2
(Go to stage 1)

The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report

Brooding increases conspiracy beliefs but with practical significance to be determined

Recommended by ORCID_LOGO based on reviews by Matt Williams and Daniel Toribio-Flórez
The world is seemingly awash with conspiracy theories – from well-trodden examples such as fake Moon landings, the 9/11 truth movement, and Holocaust denial, to relative newcomers including COVID as a bioweapon, QAnon, and the belief that the science of climate change has been invented or falsified. While there is a public perception that conspiracy theories are becoming more prevalent, recent evidence suggests that the rate of conspiracism is relatively stable over time (Uscinski et al., 2022). At any point in history, it seems that a certain proportion of people find themselves vulnerable to conspiracy beliefs, but what distinguishes those who do from those who don’t, and what are the causal factors?
 
In the current study, Liekefett et al. (2023) investigated the critical role of rumination – a perseverative and repetitive focus on negative content leading to emotional distress. In particular, the authors asked whether one component of rumination referred to as brooding (dwelling on one’s worries and distressing emotions) has a specific causal role in the formation of conspiracy beliefs. In a series of preliminary experiments, the authors first established a procedure for successfully inducing rumination, identifying various boundary conditions and requirements for a successful design. In the main study (N=1,638 to 2,007 depending on the analysis), they asked whether the induction of brooding causes a significant increase in conspiracy beliefs. Manipulation checks were also included to confirm intervention fidelity (independently of this hypothesis), and exploratory analyses tested the effect of various moderators, as well as the causal role of a complementary manipulation of reflection – a component of rumination in which attention is focused on the issue at hand rather than one’s emotions.
 
As expected by the authors' preliminary work, manipulation checks independently confirmed the effectiveness of the brooding intervention. In answer to the main research question, participants who brooded over the worries and negative emotions associated with an issue were more susceptible to conspiracy beliefs compared to a control group. However, while this effect of brooding was statistically significant, the confidence interval of the effect size estimate overlapped with the authors' proposed smallest effect size of interest (d = 0.20), suggesting that the practical value of the effect remains to be determined.
 
Overall the findings are consistent with a range of psychological theories suggesting that rumination induces negative affect and/or narrows attention to negative information, which in turn may make conspiracy theories seem more probable and render individuals more vulnerable to cognitive bias. The authors note the importance of future work to define the smallest effect of practical significance, analagous to the criteria used to determine the 'minimal clinically important difference’ in medical research.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/y82bs
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Uscinski, J., Enders, A., Klofstad, C., Seelig, M., Drochon, H., Premaratne, K. & Murthi, M. (2022) Have beliefs in conspiracy theories increased over time? PLOS ONE 17: e0270429. https://doi.org/10.1371/journal.pone.0270429

2. Liekefett, L. Sebben, S. & Becker, J. C. (2023). The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report. Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/3e8wc
The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered ReportLuisa Liekefett, Simone Sebben, Julia C. Becker<p>This Stage 2 Registered Report concerns the relationship between rumination, a repetitive style of negative thinking, and conspiracy beliefs (Stage 1 protocol: https://osf.io/y82bs, date of in-principle-acceptance: 23/05/2023). Based on four pi...Humanities, Social sciencesChris Chambers2023-10-19 17:46:59 View
04 Dec 2023
STAGE 1

Self-Control beyond inhibition. German Translation and Quality Assessment of the Self-Control Strategy Scale (SCSS)

Strategies for self control: German translation and evaluation of the Self Control Strategy Scale

Recommended by ORCID_LOGO based on reviews by Eleanor Miles, Kaitlyn Werner and Sebastian Bürgler
Self-control has shown to be a trait related to beneficial outcomes, including health, academic achievement and relationship quality. It is mostly understood as the ability to surpress immediate urges in order to achieve long-term goals, such as not watching another episode and therefore reaching a healthy amount of sleep. An emerging perspective on self-control shows that there is broader variety in applied strategies, such as removing oneself from a tempting situation, or reminding oneself of one's long-term goal, or reinterpreting the temptation.
 
Katzir et al. (2021) developed a novel instrument, the Self-Control Strategy Scale, that measured the tendency to engage in eight such strategies. In the current study, Roth et al. (2023) propose to translate the scale into German and assess its psychometric properties. Further, they will determine which strategies are related to particular outcomes that may be beneficial; for example, amount of physical activity engaged in, how healthy the diet is, exam performance and life satisfaction.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review by the recommender and at least two expert reviewers, before issuing in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/s7qwk
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

References
 
1. Katzir, M., Baldwin, M., Werner, K. M., & Hofmann, W. (2021). Moving beyond inhibition: Capturing a broader scope of the self-control construct with the Self-Control Strategy Scale (SCSS). Journal of Personality Assessment, 103, 762-776. https://doi.org/10.1080/00223891.2021.1883627
 
2. Roth, L. H. O., Jankowski, J., Clay, G., Meindl, D., Vogt, L.-M., Wagner, V., Nordmann, A., Stenzel, L., Freiman, O., Mlynski, C., & Job, V. (2023). Self-Control beyond inhibition. German Translation and Quality Assessment of the Self-Control Strategy Scale (SCSS). In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/s7qwk

Self-Control beyond inhibition. German Translation and Quality Assessment of the Self-Control Strategy Scale (SCSS)Leopold H. O. Roth1, Julia Jankowski1, Georgia Clay1, Dominik Meindl1, Lisa-Marie Vogt1, Victoria Wagner1, Artemis Nordmann1, Loana Stenzel1, Olga Freiman1, Christopher Mlynski1, Veronika Job1; 1University of Vienna, Vienna, Austria<p>Self-control is crucial for goal attainment and related to several beneficial outcomes, such as health and education. For a long time, it was predominantly understood in terms of inhibition, namely the ability to suppress immediate urges for th...Social sciencesZoltan Dienes2023-07-13 13:46:30 View
04 Dec 2023
STAGE 1

Cerebral laterality as assessed by functional transcranial Doppler ultrasound in left-and right-handers: A comparison between handwriting and writing using a smartphone

Does typing on a smartphone involve the same neural mechanisms as writing by hand?

Recommended by ORCID_LOGO based on reviews by Todd Richards and Dorothy Bishop
Language production is associated with a distinct lateralised pattern of brain activation biased toward the left cerebral hemisphere. This also applies to writing. It has also been shown to be modulated by handedness, with less pronounced lateralisation in left-handers. However, in recent decades the use of handwriting has declined significantly while the use of smartphones has exploded. To date, no study has explored whether the same neural correlates of written language production found for handwriting also hold for typing on a smartphone.
 
In the current study, Samsouris et al. (2023) will use functional transcranial Doppler ultrasound (fTCD) to measure blood flow velocity within cerebral hemispheres to investigate this question. This technique is particularly suited for this purpose because it provides better control for the movement confounds associated with a writing task and the technical challenges of using a smart device than other neuroimaging techniques like fMRI or M/EEG. The authors hypothesise that there will be no difference in left cerebral lateralisation for handwriting and typing on a smartphone. They also expect to replicate previous findings of weaker lateralisation in left-handers in written language production when typing on a smartphone. To isolate the effect of written language production, both these conditions will be corrected for their corresponding motor component using control conditions without a linguistic component.
 
The Stage 1 manuscript was evaluated over 6 rounds of in-depth review by the recommender and two expert reviewers, before issuing in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/j7egz
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Samsouris, C., Badcock, N. A., Vlachos, F., & Papadatou-Pastou, M. (2023). Cerebral laterality as assessed by functional transcranial Doppler ultrasound in left-and right-handers: A comparison between handwriting and writing using a smartphone. In principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/j7egz
Cerebral laterality as assessed by functional transcranial Doppler ultrasound in left-and right-handers: A comparison between handwriting and writing using a smartphoneChristos Samsouris, Nicholas A. Badcock, Filippos Vlachos, Marietta Papadatou-Pastou<p>Neuroscientific studies of traditional handwriting have revealed a left cerebral lateralization pattern for written language production, with distinct patterns between left- and right-handers. However, no study to date has investigated the cere...Life Sciences, Social sciencesD. Samuel Schwarzkopf2022-11-01 10:27:39 View
28 Nov 2023
STAGE 1

One and only SNARC? A Registered Report on the SNARC Effect’s Range Dependency

Is the SNARC effect modulated by absolute number magnitude?

Recommended by ORCID_LOGO based on reviews by Melinda Mende and 1 anonymous reviewer
The Spatial-Numerical Association of Response Codes (SNARC) effect refers to the fact that smaller numbers receive faster responses with the left hand, and larger numbers with the right hand (Dehaene et al., 1993). This robust finding implies that numbers are associated with space, being represented on a mental number line that progresses from left to right. The SNARC effect is held to depend on relative number magnitude, with the mental number line dynamically adjusting to the numerical range used in a given context. This characterisation is based on significant effects of relative number magnitude, with no significant influence of absolute number magnitude. However, a failure to reject the null hypothesis, within the standard frequentist statistical framework, is not firm evidence for the absence of an effect. In this Stage 1 Registered Report, Roth and colleagues (2023) propose two experiments adapted from Dahaene’s (1993) original methods, with a Bayesian statistical approach to confirm—or rule out—a small effect (d = 0.15) of absolute number magnitude in modulating the classic SNARC effect.
 
The study plan was refined across two rounds of review, with input from two external reviewers and the recommender, after which it was judged to satisfy the Stage 1 criteria for in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/ae2c8
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
Dehaene, S., Bossini, S., & Giraux, P. (1993). The mental representation of parity and number magnitude. Journal of Experimental Psychology: General, 122(3), 371–396. https://doi.org/10.1037/0096-3445.122.3.371
 
Roth, L., Caffier, J., Reips, U.-D., Nuerk, H.-C., & Cipora, K. (2023). One and only SNARC? A Registered Report on the SNARC Effect’s Range Dependency. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/ae2c8
One and only SNARC? A Registered Report on the SNARC Effect’s Range DependencyLilly Roth, John Caffier, Ulf-Dietrich Reips, Hans-Christoph Nuerk, Krzysztof Cipora<p>Numbers are associated with space, but it is unclear how flexible these associations are. In this study, we will investigate whether the SNARC effect (Spatial-Numerical Association of Response Codes; Dehaene et al., 1993), which describes faste...Social sciencesRobert McIntosh2022-11-30 12:36:08 View
27 Nov 2023
STAGE 1

Cortical voice processing in Autism Spectrum Disorder

Is voice processing impacted in Autism Spectrum Disorder?

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by 2 anonymous reviewers
Vocal sounds, including both speech and non-speech sounds, have been found to activate the Superior Temporal Sulci and Gyri in comparison to non-vocal sounds. These regions, termed Temporal Voice Areas (TVAs), are considered to be involved in early voice processing and therefore critical for social interaction. TVA activation has been examined in Autism Spectrum Disorder (ASD) to determine if the characteristic difficulties in social communication and interaction are linked to an impaired early voice processing. Using functional magnetic resonance imaging (fMRI), one study found typical brain activation in TVAs for 15 out of 16 autistic participants (Schelinski et al., 2016), whereas another found atypical activation in 4 out of 5 autistic participants (Gervais et al., 2004).
 
Here, the inconsistencies in the previous literature propel Gautier et al. (2023) to examine brain activation of TVAs with a larger sample size (26 ASD and 26 non-ASD participants). Gautier et al. (2023) will present vocal sounds and non-vocal sounds to both groups of participants during fMRI and predict that fewer participants in the ASD group will show a preferential response to voices in TVAs compared to the non-ASD group. These results would suggest that symptoms of ASD interfere with early stages of social interaction, at the level of voice processing.
 
This Stage 1 manuscript was evaluated in an initial round by the co-recommenders and another two rounds of in-depth review by two expert reviewers. With these revisions, the recommenders judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/538m4
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Gautier, R., Houy-Durand, E., Barantin, L., Briend, F. & Latinus, M. (2023). Cortical voice processing in Autism Spectrum Disorder. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/538m4
Cortical voice processing in Autism Spectrum DisorderRaphaël Gautier, Emmanuelle Houy-Durand, Laurent Barantin, Frederic Briend, Marianne Latinus<p>Voice processing is central to social functioning. A specific brain response to vocal sounds has been described and extensively characterized in the general population but remains critically unexplored in Autism Spectrum Disorder (ASD), a condi...Life SciencesGrace Edwards2023-03-28 09:51:48 View
23 Nov 2023
STAGE 2
(Go to stage 1)

The Medusa effect: A registered replication report of Will, Merritt, Jenkins, and Kingstone (2021)

Looking (again) at Medusa: Evidence that pictorial abstraction influences mind perception

Recommended by ORCID_LOGO based on reviews by Alan Kingstone and 1 anonymous reviewer
The Medusa effect is a recently described phenomenon in which people judge a person to be more mindful when they appear as a picture (termed L1) than as a picture within a picture (L2). Across a series of experiments, Will et al. (2021) reported that at higher levels of abstraction, images of people were judged lower in realness (how real the person seemed), experience (the ability to feel) and agency (the ability to plan and act), and also benefited less from prosocial behaviour. The findings provide an intriguing window into mind perception – the extent to which we attribute minds and mental capacities to others.
 
In the current study, Han et al. (2023) undertook a close replication of two experiments from the original report by Will et al. (2021), asking first, whether the level of pictorial abstraction influences ratings of realness, agency and experience, and second, whether it also influences prosocial behaviour as measured in the dictator game (with participants predicted to allocate more money to recipients presented as pictures than as pictures within pictures). In the event of a non-replication using the original materials, the authors planned to further repeat the experiments using newly generated stimuli that are better matched for cultural context and more tightly controlled along various dimensions.
 
Results supported all pre-registered hypotheses. Participants rated and perceived L1 stimuli as having significantly higher levels of realness, agency, and experience than L2, and they also allocated significantly more money to L1 recipients than L2 recipients in a dictator game. Furthermore, participants who judged L1 as higher than L2 on all three dimensions also differentiated significantly between L1 and L2 in the dictator game, indicating a relationship between mind perception and prosociality. Overall, the findings confirm that pictures with lower levels of abstraction are perceived as more mindful and are associated with higher levels of prosocial behavior. Consequently, the results suggest that the Medusa effect is a reproducible phenomenon.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/xj46z
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Will, P., Merritt, E., Jenkins, R., & Kingstone, A. (2021). The Medusa effect reveals levels of mind perception in pictures. Proceedings of the National Academy of Sciences, 118(32), e2106640118. https://doi.org/10.1073/pnas.2106640118
 
2. Han, J., Zhang, M., Liu, J., Song, Y. & Yamada, Y. (2023).The Medusa effect: A registered replication report of Will, Merritt, Jenkins, and Kingstone (2021). Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/yqnu8
The Medusa effect: A registered replication report of Will, Merritt, Jenkins, and Kingstone (2021)Jing Han, Minjun Zhang, Jiaxin Liu, Yu Song, Yuki Yamada<p>Will et al.'s (2021) found the Medusa effect, which refers to the tendency that people evaluate a “person in picture” more mindful than a “person in picture of a picture”. The present study tried to directly replicate the Experiments 2 and 5 of...Social sciencesChris Chambers2023-10-24 03:42:14 View
16 Nov 2023
STAGE 1

The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered Report

Are there oscillatory markers of pain intensity?

Recommended by ORCID_LOGO based on reviews by Markus Ploner and Björn Horing
Rhythmic changes in pain can lead to corresponding modulations of EEG amplitudes in theta, alpha, and beta bands. But the question remains open as to whether these modulations are actually tracking pain, or maybe rather saliency or stimulus intensity. The question is of some importance because a marker of pain per se could be useful for tracking felt pain without a verbal response, and could be useful in investigating interventions for treating pain (such as suggestion).  Here, Leu et al. (2023) will address the question of whether modulations reflect saliency or else the intensity of pain, by using an oddball paradigm in which most trials are a pain stimulus of a certain intensity, and oddball trials will sometimes occur, at either a higher intensity or a lower intensity than the baseline ones. If the modulations reflect salience, the modulation at the frequency of the oddball will be similar for high and low intensity oddballs. However, if the modulations reflect pain intensity, the modulations for the low rather than high oddball condition will be lower.
 
The Stage 1 manuscript was evaluated over three rounds of in-depth peer review, the first two consisting of substantial comments from two scholars with relevant expertise, and the third consisting of a close review by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/qbrf2
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Leu, C., Forest, S., Legrain, V., & Liberati, G. (2023). The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered Report. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/qbrf2
The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered ReportChiara Leu, Sébastien Forest, Valéry Legrain, Giulia Liberati<p>Ongoing oscillations have been shown to be modulated in different frequency bands following phasic, tonic as well as periodic thermonociceptive stimulation. Yet, it remains unclear whether these modulations are related to pain perception, salie...Life Sciences, Medical SciencesZoltan Dienes2023-09-06 15:15:19 View
15 Nov 2023
STAGE 1

Somatosensory Response Changes During Illusory Finger Stretching

Neural responses to a finger-stretching illusion in human somatosensory cortex

Recommended by ORCID_LOGO based on reviews by Harry Farmer, Alexandra Mitchell and Susanne Stoll
Chronic pain is a major cause of disability that can often poorly managed with pharmacological treatments. This has prompted the exploration of other interventions like resizing illusions of body parts in augmented reality. These illusions have shown promise in conditions like osteoarthritis and complex regional pain syndrome, but it remains unclear how they alter the neural representation of body parts in the brain. The study by Hansford and colleagues aims to investigate these mechanisms in healthy participants, using somatosensory steady state evoked potentials (SSEP) and self-report questionnaires.
 
The study will involve finger stretching in an augmented reality setup that allows the researchers to independently manipulate visual and tactical stimulation. Assuming that multisensory stimulation indeed produces a robust illusion, the researchers will quantify the somatosensory evoked potentials in multisensory, unisensory, and two non-illusion control conditions. The study will provide inights into the neural mechanisms of these illusions and lay the ground for future investigations of these processes as a potential treatment for chronic pain.
 
The manuscript was evaluated over seven rounds of in-depth review by the recommender and three expert reviewers. After substantial revisions, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/u6gsb
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

References
 
1. Hansford, K. J., Baker, D. H., McKenzie, K. J., & Preston, C. E. J. (2023). Somatosensory Response Changes During Illusory Finger Stretching. In principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/u6gsb
Somatosensory Response Changes During Illusory Finger Stretching Kirralise J. Hansford, Daniel H. Baker, Kirsten J. McKenzie & Catherine E. J. Preston<p>Resizing illusions, delivered using augmented reality, resize a body part through either stretching or shrinking manipulations. These resizing illusions have been investigated in visuotactile, visual-only and visuo-auditory presentations. Howev...Life Sciences, Medical SciencesD. Samuel Schwarzkopf2022-11-15 17:17:18 View
14 Nov 2023
STAGE 1

Scrolling to wisdom: the impact of social media news exposure on knowledge perception

Might we know less about current events than we think we do?

Recommended by ORCID_LOGO based on reviews by Adrien Fillon, Erik Løhre and Moritz Ingendahl
​​We are bombarded with news about current events from multiple sources: print media, digital media, friends, family, and more. At the same time, there is an imperative to “stay informed” and be knowledgeable of happenings both local and global. But how much knowledge do we actually gain from this bombardment of information? How informed are we really? It turns out that our perceptions of our knowledge tends to overstate our actual knowledge of a topic. This “illusion of knowledge” effect has been studied across a wide variety of contexts, but is especially relevant for understanding how people learn about and interact with politicized topics.
 
In the current study, Ruzzante et al. (2023) propose to further our understanding of the illusion of knowledge effect in the context of news exposure on social media. They will use an online pre-post experimental design that assesses participants’ perceived knowledge of a number of topics prior to the manipulation, which involves exposure to different social media news feeds, coming two weeks later. Central to the study, participants will be randomized to news stories that differ in their degree of self-involvement, that is how emotionally involved the topics are. Ruzzante et al. will test the hypothesis that more highly self-involved topics (e.g., abortion) will lead to a greater illusion of knowledge effect than less self-involved topics (e.g., feline immunodeficiency).
 
The Stage 1 manuscript was evaluated over two rounds of in-depth peer review, the first consisting of substantial comments from three scholars with relevant expertise, and the second consisting of a close review by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and was therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/qa7tb
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Ruzzante, F., Cevolani, G., & Panizza, F. (2023). Scrolling to wisdom: The impact of social media news exposure on knowledge perception. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/qa7tb
Scrolling to wisdom: the impact of social media news exposure on knowledge perceptionFederica Ruzzante, Gustavo Cevolani, Folco Panizza<p>The present study aims to test the effect of exposure to news in a social media environment on people’s perceived knowledge of selected topics and on the “illusion of knowledge” effect, i.e., the overestimation of one’s perceived knowledge rela...Social sciencesMoin Syed2022-10-12 21:16:51 View
13 Nov 2023
STAGE 2
(Go to stage 1)

Convenience Samples and Measurement Equivalence in Replication Research

Data from students and crowdsourced online platforms do not often measure the same thing

Recommended by ORCID_LOGO based on reviews by Benjamin Farrar and Shinichi Nakagawa

Comparative research is how evidence is generated to support or refute broad hypotheses (e.g., Pagel 1999). However, the foundations of such research must be solid if one is to arrive at the correct conclusions. Determining the external validity (the generalizability across situations/individuals/populations) of the building blocks of comparative data sets allows one to place appropriate caveats around the robustness of their conclusions (Steckler & McLeroy 2008).

In the current study, Alley and colleagues (2023) tackled the external validity of comparative research that relies on subjects who are either university students or participating in experiments via an online platform. They determined whether data from these two types of subjects have measurement equivalence - whether the same trait is measured in the same way across groups.

Although they use data from studies involved in the Many Labs replication project to evaluate this question, their results are of crucial importance to other comparative researchers whose data are generated from these two sources (students and online crowdsourcing). The authors show that these two types of subjects do not often have measurement equivalence, which is a warning to others to evaluate their experimental design to improve validity. They provide useful recommendations for researchers on how to to implement equivalence testing in their studies, and they facilitate the process by providing well annotated code that is openly available for others to use.

After one round of review and revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.

URL to the preregistered Stage 1 protocol: https://osf.io/7gtvf
 
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to Stage 1 IPA, but the authors certify that they had not yet observed the key variables within the data that were used to answer the research question AND they took additional steps to maximise bias control and rigour.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Pagel, M. (1999). Inferring the historical patterns of biological evolution. Nature, 401, 877-884. https://doi.org/10.1038/44766
 
2. Steckler, A. & McLeroy, K. R. (2008). The importance of external validity. American Journal of Public Health 98, 9-10. https://doi.org/10.2105/AJPH.2007.126847
 
3. Alley L. J., Axt, J., & Flake J. K. (2023). Convenience Samples and Measurement Equivalence in Replication Research [Stage 2 Registered Report] Acceptance of Version 2 by Peer Community in Registered Reports​. https://osf.io/s5t3v
Convenience Samples and Measurement Equivalence in Replication ResearchLindsay J. Alley, Jordan Axt, Jessica Kay Flake<p>A great deal of research in psychology employs either university student or online crowdsourced convenience samples (Chandler &amp; Shapiro, 2016; Strickland &amp; Stoops, 2019) and there is evidence that these groups differ in meaningful ways ...Social sciencesCorina Logan Alison Young Reusser2023-08-31 20:26:43 View