Submit a report

Announcements

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.


 

Latest recommendationsrssmastodon

Id▲TitleAuthorsAbstractPictureThematic fieldsRecommenderReviewersSubmission date
14 Nov 2023
STAGE 1
toto

Scrolling to wisdom: the impact of social media news exposure on knowledge perception

Might we know less about current events than we think we do?

Recommended by ORCID_LOGO based on reviews by Adrien Fillon, Erik Løhre and Moritz Ingendahl
​​We are bombarded with news about current events from multiple sources: print media, digital media, friends, family, and more. At the same time, there is an imperative to “stay informed” and be knowledgeable of happenings both local and global. But how much knowledge do we actually gain from this bombardment of information? How informed are we really? It turns out that our perceptions of our knowledge tends to overstate our actual knowledge of a topic. This “illusion of knowledge” effect has been studied across a wide variety of contexts, but is especially relevant for understanding how people learn about and interact with politicized topics.
 
In the current study, Ruzzante et al. (2023) propose to further our understanding of the illusion of knowledge effect in the context of news exposure on social media. They will use an online pre-post experimental design that assesses participants’ perceived knowledge of a number of topics prior to the manipulation, which involves exposure to different social media news feeds, coming two weeks later. Central to the study, participants will be randomized to news stories that differ in their degree of self-involvement, that is how emotionally involved the topics are. Ruzzante et al. will test the hypothesis that more highly self-involved topics (e.g., abortion) will lead to a greater illusion of knowledge effect than less self-involved topics (e.g., feline immunodeficiency).
 
The Stage 1 manuscript was evaluated over two rounds of in-depth peer review, the first consisting of substantial comments from three scholars with relevant expertise, and the second consisting of a close review by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and was therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/qa7tb
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Ruzzante, F., Cevolani, G., & Panizza, F. (2023). Scrolling to wisdom: The impact of social media news exposure on knowledge perception. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/qa7tb
Scrolling to wisdom: the impact of social media news exposure on knowledge perceptionFederica Ruzzante, Gustavo Cevolani, Folco Panizza<p>The present study aims to test the effect of exposure to news in a social media environment on people’s perceived knowledge of selected topics and on the “illusion of knowledge” effect, i.e., the overestimation of one’s perceived knowledge rela...Social sciencesMoin Syed2022-10-12 21:16:51 View
13 Jul 2023
STAGE 1
toto

Modulatory effects of instructions on extinction efficacy in appetitive and aversive learning: A registered report

Neurocognitive insights on instructed extinction in the context of pain

Recommended by ORCID_LOGO based on reviews by Tom Beckers, Gaëtan Mertens and Karita Ojala
Rapid learning in response to pain is a crucial survival mechanism, relying on forming associations between cues in the environment and subsequent pain or injury. Existing evidence suggests that associations between conditioned stimuli (cues) and unconditioned aversive stimuli (such as pain) are learned faster than for appetitive stimuli that signal pain relief. In addition, when the link between a conditioned and unconditioned stimulus is broken (by unpairing them), the extinction of this learning effect is slower for aversive that appetitive stimuli, resulting in a flatter extinction slope. Understanding why extinction slopes are reduced for aversive stimuli is important for advancing theoretical models of learning, and for devising ways of increasing the slope (and thus facilitating extinction learning) could help develop more effective methods of pain relief, particularly in the treatment of chronic pain.
 
In the current programmatic submission, Busch et al. (2023) will undertake two Registered Reports to test whether a verbal instruction intervention that explicitly informs participants about contingency changes between conditioned and unconditioned stimuli facilitates extinction learning, especially for aversive (painful) stimuli, and how changes in extinction learning relate to neural biomarkers of functional connectivity. In the first Registered Report, they will initially seek to replicate previous findings including faster acquisition of aversive than appetitive conditioned stimuli as well as incomplete extinction of aversive conditioned stimuli without verbal instruction. They will then test how the instruction intervention alters extinction slopes and the completeness of extinction for appetitive and aversive stimuli, using a range of behavioral measures (expectancy and valence ratings) and physiological measures (pupillometry, skin conductance responses). To shed light on the neural correlates of these processes, in the second Registered Report the authors will use functional magnetic resonance imaging (fMRI) to ask firstly how acquisition and extinction of aversive and appetitive conditioned responses are related to resting state brain connectivity within a network that includes ventromedial prefrontal cortex, amygdala, and striatum, and secondly, whether the effectiveness of instruction on extinction learning is associated with differences in resting state connectivity across this network.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/cj75p (under temporary private embargo)
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Busch, L., Wiech, K., Gamer, M., Knicses, B., Spisak, T., Schmidt, K., & Bingel, U. (2023). Modulatory effects of instructions on extinction efficacy in appetitive and aversive learning: A registered report. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/cj75p
Modulatory effects of instructions on extinction efficacy in appetitive and aversive learning: A registered reportLea Busch, Katja Wiech, Matthias Gamer, Balint Kincses, Tamas Spisak, Katharina Schmidt, Ulrike Bingel<p>In the context of pain, extinction learning has been shown to be slower or incomplete for aversive compared to appetitive cues (i.e., cues signaling pain exacerbation and pain relief, respectively), potentially due to their higher biological re...Medical SciencesChris Chambers2022-10-15 19:45:48 View
10 Feb 2023
STAGE 2
(Go to stage 1)
toto

The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered Report

Different ways of describing problematic substance use and its treatment influence public stigma

Recommended by ORCID_LOGO based on reviews by Nicholas Sinclair-House
People experiencing problematic substance use are often stigmatised by the general public. This public stigma may impair such people obtaining help and the quality of help that they receive. For this reason, previous research has investigated the factors that may exacerbate or lessen stigma by focusing on the terminology used to describe problematic substance use. However, the evidence is not clear cut, with some studies suggesting that labelling the condition as a "chronically relapsing brain disease" vs a "problem" reduces certain elements of stigma and other studies finding absence of evidence. A closer look at these studies points to methodological differences that may explain their results, such as whether problematic substance use is compared with another health condition, whether the individual is described as seeking treatment or not, and whether general or discrete elements of stigma are measured.
 
In this Stage 2 Registered Report, Pennington et al. (2023) isolated these methodological differences to investigate if any of them influenced two different measures of stigma used in previous work. They found that greater social distance, danger and public stigma but lower blame were ascribed to drug use relative to a health concern, supporting previous research to suggest that problematic substance use is a highly stigmatised health condition. Furthermore, greater (genetic) blame was reported when drug use was labelled as a ‘chronically relapsing brain disease’ relative to a ‘problem’. The results for attributional judgement were either inconclusive or statistically equivalent. In summary, these findings suggest that the labels and models used to describe problematic substance use may impact upon public stigma in distinct ways. The authors suggest that future research should justify which measures are being used in line with theory. They also put forward the notion that addiction is a functional attribution, which may explain the mixed literature on the brain disease model of addiction to date.
 
The Stage 2 manuscript was evaluated over one round of specialist review and several rounds of discussion with the recommender. Based on comprehensive responses, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/4vscg
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
Pennington, C. R., Monk, R. L., Heim, D., Rose, A. K., Gough, T., Clarke, R., Knibb, G., Patel, R., Rai, P., Ravat, H., Ali, R., Anastasiou, G., Asgari, F., Bate, E., Bourke, T., Boyles, J., Campbell, A., Fowler, N., Hester, S., Neil, C., McIntrye, B., Ogilvy, E., Renouf, A., Stafford, J., Toothill, K., Wong, H. K., &  Jones, A. (2023). The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered Report. Stage 2 acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/z9bnf
The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered ReportCharlotte R. Pennington, Rebecca L. Monk, Derek Heim, Abi K. Rose, Thomas Gough, Ross Clarke, Graham Knibb, Roshni Patel, Priya Rai, Halimah Ravat, Ramsha Ali, Georgiana Anastasiou, Fatemeh Asgari, Eve Bate, Tara Bourke, Jayme Boyles, Alix Campbel...<p>Objectives: Problematic substance use is one of the most stigmatised health conditions leading research to examine how the labels and models used to describe it influence public stigma. Two recent studies examine whether beliefs in a disease mo...Social sciencesZoltan Dienes2022-10-21 16:13:49 View
04 Dec 2023
STAGE 1
toto

Cerebral laterality as assessed by functional transcranial Doppler ultrasound in left-and right-handers: A comparison between handwriting and writing using a smartphone

Does typing on a smartphone involve the same neural mechanisms as writing by hand?

Recommended by ORCID_LOGO based on reviews by Todd Richards and Dorothy Bishop
Language production is associated with a distinct lateralised pattern of brain activation biased toward the left cerebral hemisphere. This also applies to writing. It has also been shown to be modulated by handedness, with less pronounced lateralisation in left-handers. However, in recent decades the use of handwriting has declined significantly while the use of smartphones has exploded. To date, no study has explored whether the same neural correlates of written language production found for handwriting also hold for typing on a smartphone.
 
In the current study, Samsouris et al. (2023) will use functional transcranial Doppler ultrasound (fTCD) to measure blood flow velocity within cerebral hemispheres to investigate this question. This technique is particularly suited for this purpose because it provides better control for the movement confounds associated with a writing task and the technical challenges of using a smart device than other neuroimaging techniques like fMRI or M/EEG. The authors hypothesise that there will be no difference in left cerebral lateralisation for handwriting and typing on a smartphone. They also expect to replicate previous findings of weaker lateralisation in left-handers in written language production when typing on a smartphone. To isolate the effect of written language production, both these conditions will be corrected for their corresponding motor component using control conditions without a linguistic component.
 
The Stage 1 manuscript was evaluated over 6 rounds of in-depth review by the recommender and two expert reviewers, before issuing in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/j7egz
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Samsouris, C., Badcock, N. A., Vlachos, F., & Papadatou-Pastou, M. (2023). Cerebral laterality as assessed by functional transcranial Doppler ultrasound in left-and right-handers: A comparison between handwriting and writing using a smartphone. In principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/j7egz
Cerebral laterality as assessed by functional transcranial Doppler ultrasound in left-and right-handers: A comparison between handwriting and writing using a smartphoneChristos Samsouris, Nicholas A. Badcock, Filippos Vlachos, Marietta Papadatou-Pastou<p>Neuroscientific studies of traditional handwriting have revealed a left cerebral lateralization pattern for written language production, with distinct patterns between left- and right-handers. However, no study to date has investigated the cere...Life Sciences, Social sciencesD. Samuel Schwarzkopf2022-11-01 10:27:39 View
15 Nov 2023
STAGE 1
toto

Somatosensory Response Changes During Illusory Finger Stretching

Neural responses to a finger-stretching illusion in human somatosensory cortex

Recommended by ORCID_LOGO based on reviews by Harry Farmer, Alexandra Mitchell and Susanne Stoll
Chronic pain is a major cause of disability that can often poorly managed with pharmacological treatments. This has prompted the exploration of other interventions like resizing illusions of body parts in augmented reality. These illusions have shown promise in conditions like osteoarthritis and complex regional pain syndrome, but it remains unclear how they alter the neural representation of body parts in the brain. The study by Hansford and colleagues aims to investigate these mechanisms in healthy participants, using somatosensory steady state evoked potentials (SSEP) and self-report questionnaires.
 
The study will involve finger stretching in an augmented reality setup that allows the researchers to independently manipulate visual and tactical stimulation. Assuming that multisensory stimulation indeed produces a robust illusion, the researchers will quantify the somatosensory evoked potentials in multisensory, unisensory, and two non-illusion control conditions. The study will provide inights into the neural mechanisms of these illusions and lay the ground for future investigations of these processes as a potential treatment for chronic pain.
 
The manuscript was evaluated over seven rounds of in-depth review by the recommender and three expert reviewers. After substantial revisions, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/u6gsb
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

References
 
1. Hansford, K. J., Baker, D. H., McKenzie, K. J., & Preston, C. E. J. (2023). Somatosensory Response Changes During Illusory Finger Stretching. In principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/u6gsb
Somatosensory Response Changes During Illusory Finger Stretching Kirralise J. Hansford, Daniel H. Baker, Kirsten J. McKenzie & Catherine E. J. Preston<p>Resizing illusions, delivered using augmented reality, resize a body part through either stretching or shrinking manipulations. These resizing illusions have been investigated in visuotactile, visual-only and visuo-auditory presentations. Howev...Life Sciences, Medical SciencesD. Samuel Schwarzkopf2022-11-15 17:17:18 View
27 Jun 2023
STAGE 1
toto

Functional MRI brain state occupancy in the presence of cerebral small vessel disease -- pre-registration for a replication analysis of the Hamburg City Health Study

Testing the replicability of dynamic functional connectivity correlates of cerebral small vessel disease in the Hamburg City Health Study

Recommended by ORCID_LOGO based on reviews by Olivia Hamilton and 1 anonymous reviewer
A recent study has reported that the extent of cerebral small vessel disease (cSVD) shows associations with dynamic functional connectivity measures obtained from resting state functional MRI scans (Schlemm et al, 2022). Specifically, when the functional scan was parsed into time spent in discrete brain states, the proportion of time spent in the two most-occupied states was negatively related to a structural indicator of cSVD (volume of white matter hyperintensities of presumed vascular origin). This measure of 'fractional occupancy' was also associated with cognitive impairment as indicated by longer time to complete part B of the Trail Making Test. These findings were based on the analysis of data from 988 participants in the Hamburg City Health Study (HCHS).
 
In the present Registered Report, Schlemm (2023) will test whether these associations can be replicated in an independent sample of participants from the HCHS, not included in the earlier analysis (projected N for new analysis ~1500). In addition to the two main hypothesis tests, an exploratory multiverse analysis will be reported, systematically varying some key parameters of the MRI processing pipeline to provide further information about the robustness of the outcome of the primary hypothesis test. 
 
The Stage 1 plan was refined over two rounds of review by two relevant experts, with additional input from the recommender on the specification of the registered plan. Both reviewers are satisfied that the plan constitutes an appropriate approach to this question, and on the basis of their comments and his own evaluation, the recommender judged that the Stage 1 report meets the criteria for in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/9yhzc
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Schlemm, E., Frey, B. M., Mayer, C., Petersen, M., Fiehler, J., Hanning, U., Kühn, S., Twerenbold, R., Gallinat, J., Gerloff, C., Thomalla, G. & Cheng, B. (2022). Equalization of brain state occupancy accompanies cognitive impairment in cerebral small vessel disease. Biological Psychiatry, 92, 592-602. https://doi.org/10.1016/j.biopsych.2022.03.019
 
2. Schlemm, E. (2023). Functional MRI brain state occupancy in the presence of cerebral small vessel disease – pre-registration for a replication analysis of the Hamburg City Health Study. In principle acceptance of Version 1.5 by Peer Community in Registered Reports. https://osf.io/9yhzc
Functional MRI brain state occupancy in the presence of cerebral small vessel disease -- pre-registration for a replication analysis of the Hamburg City Health StudyEckhard Schlemm<p>Objective: To replicate recent findings about the association between the extent of cerebral small vessel disease (cSVD), functional brain network dedifferentiation and cognitive impairment.</p> <p>Methods: We will analyze demographic, imaging...Life Sciences, Medical SciencesRobert McIntosh2022-11-19 14:21:28 View
20 Jan 2023
STAGE 2
(Go to stage 1)
toto

No reliable effect of task-irrelevant cross-modal statistical regularities on distractor suppression

Failure to learn cross-modally to suppress distractors

Recommended by ORCID_LOGO based on reviews by Miguel Vadillo and 1 anonymous reviewer
There are two fundamental processes that the brain engages in: statistical learning and selection. Indeed, past work has shown these processes often come together: People can use a task-irrelevant stimulus to predict a target stimulus even in different modalities (crossmodal statistical learning), thereby enhancing the processing of the target stimulus (selection). Further, people can learn where a distractor will be in order to efficiently suppress it (selecting out), using task irrelevant stimuli in the same modality (within-modality statistical learning).
 
In two experiments Jagini and Sunny found that people did not learn to use a task-irrelevant stimulus from a different modality (cross modal statistical learning) to suppress a distractor (selecting out). They also found that people had little awareness of the relation between the predictor task-irrelevant stimulus and the location of the distractor. The results may reflect limits on what can be achieved unconsciously.
 
Following peer review, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/qjbmg
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Jagini, K. K. & Sunny, M. M. (2023). No reliable effect of task-irrelevant cross-modal statistical regularities on distractor suppression. Stage 2 Registered Report, acceptance of Version 4 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/d8wes
No reliable effect of task-irrelevant cross-modal statistical regularities on distractor suppressionKishore Kumar Jagini, Meera Mary Sunny<p>Our sensory systems are known to extract and utilize statistical regularities in sensory inputs across space and time for efficient perceptual processing. Past research has shown that participants can utilize statistical regularities of target ...Humanities, Life Sciences, Social sciencesZoltan Dienes2022-11-21 15:30:30 View
20 Apr 2023
STAGE 1
toto

Oxytocin, individual differences, and trust game behavior: a registered large-scale replication

Does oxytocin enhance pro-social behaviors?

Recommended by ORCID_LOGO based on reviews by Thibaut Arpinon and 1 anonymous reviewer
Trust is a fundamental element for the well-functioning of our society. By achieving large levels of trust, societies can reduce monitoring costs (e.g., contract supervision, conflict resolution) and increase investments in more profitable but socially riskier projects. Trust serves as a way to foster cooperation among individuals and can benefit the entire social group.
 
As a result, numerous researchers have sought to explore the biological foundations of trust. In particular, several research teams have tried to explore the role of oxytocin in modulating prosocial behaviors. However, previous research findings obtained mixed findings, and several methodological issues, such as low statistical power, have hindered the development of a definite view on the matter.
 
In the current study, Kroll et al. (2023) seek to replicate the seminal study of Kosfeld et al. (2005). Kosfeld and co-authors originally found that the intranasal administration of oxytocin can significantly increase trust among individuals. However, recent replication evidence shows no general effect of oxytocin on pro-social behaviors or suggests that the effect might be heterogeneous among social groups (Declerck et al., 2020). The objective of the current study of Kroll et al. (2023) is twofold. First, the authors propose to replicate the original study of Kosfeld et al. (2005) and to investigate whether the lack of replication can be due to an overall effect that is smaller than the original findings suggest. Second, the authors propose to replicate recent but exploratory results on the heterogeneous effect of oxytocin on pro-social choices. 
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/h5gdp
 
Level of bias control achieved: Level 4. At least some of the data/evidence that will be used to answer the research question already exists AND is accessible in principle to the authors (e.g. residing in a public database or with a colleague) BUT the authors certify that they have not yet accessed any part of that data/evidence.
 
List of eligible PCI RR-friendly journals:
 
 
References:
 
1. Declerck, C. H., Boone, C., Pauwels, L., Vogt, B., & Fehr, E. (2020). A registered replication study on oxytocin and trust. Nature Human Behaviour, 4(6), 646–655. https://doi.org/10.1038/s41562-020-0878-x
 
2. Kosfeld, M., Heinrichs, M., Zak, P. J., Fischbacher, U., & Fehr, E. (2005). Oxytocin increases trust in humans. Nature, 435(7042), 673–676. https://doi.org/10.1038/nature03701
 
3. Kroll, C. F., Schruers, K., Viechtbauer, W., Vingerhoets, C., Seidel, L., Riedl, A. & Hernaus, D. (2023). Oxytocin, individual differences, and trust game behavior: a registered large-scale replication, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/h5gdp
Oxytocin, individual differences, and trust game behavior: a registered large-scale replicationCharlotte F. Kroll, Koen Schruers, Wolfgang Viechtbauer, Claudia Vingerhoets, Leonie Seidel, Arno Riedl, and Dennis Hernaus<p>The neuropeptide oxytocin (OXT) is thought to modulate important aspects of prosocial behavior. In a seminal paper, Kosfeld et al. (2005) reported that intranasally administered OXT modulated trusting behavior in an economic trust game. Several...Life Sciences, Social sciencesRomain Espinosa2022-11-25 18:07:23 View
31 May 2023
STAGE 1
toto

Defacing biases in manual and automated quality assessments of structural MRI with MRIQC

The impact of removing facial features on quality measures of structural MRI scans

Recommended by ORCID_LOGO based on reviews by Catherine Morgan and Cassandra Gould van Praag
Data sharing is perhaps the most fundamental step for increasing the transparency and reproducibility of scientific research. However, the goals of open science must be tempered by ethical considerations, protecting the privacy and safety of research participants. Bridging this gap causes challenges for many fields, such as human neuroimaging. Brain images, as measured with magnetic resonance imaging (MRI), are unique to the participant and therefore contain identifying information by definition. One way to mitigate the risk to participants arising from public data sharing has been "defacing" the MRI scans, i.e., literally removing the part of the image that contains the face and surrounding tissue, while preserving the brain structure. This procedure however also removes information that is not (or at least minimally) identifiable. It also remains unclear whether defacing the images affects image quality and thus the information necessary for addressing many research questions.
 
The current study by Provins et al. (2023) seeks to address this question. Leveraging a publicly available "IXI dataset" comprising hundreds of T1-weighted structural MRI scans, they will assess the effect of defacing on manual and automatic estimates of image quality. Specifically, the researchers will compare image quality ratings by experts for a subset of 185 images. They hypothesise that images in which facial features have been removed are typically assigned higher quality ratings. Moreover, using a full data set of 580 images, which have been obtained across three scanning sites, they will also test the impact defacing MRI scans has on automated quality measures obtained with MRIQC software. The results of this study should have important implications for open science policy and for designing the optimal procedures for sharing structural MRI data in an ethical way. For example, if the authors' hypothesis is confirmed, studies relying on MRI quality measures might be better served by a custodianship model where identifiable data is shared under strict conditions, rather than relying on publishing defaced data. More generally, the outcome of this study may have significant legal implications in many jurisdictions.
 
The Stage 1 manuscript was evaluated at the inital triage stage by the Recommender and PCI:RR team, and another round of in-depth review by two experts. After a detailed response and substantial revisions, the recommender judged the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/qcket (under temporary private embargo)
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour.
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Provins, C., Savary, E., Alemán-Gómez, Y., Richiardi, J., Poldrack, R. A., Hagmann, P. & Esteban, O. (2023). Defacing biases in manual and automated quality assessments of structural MRI with MRIQC, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/qcket
Defacing biases in manual and automated quality assessments of structural MRI with MRIQCCéline Provins, Yasser Alemán-Gómez, Jonas Richiardi, Russell A. Poldrack, Patric Hagmann, Oscar Esteban<p>A critical requirement before data-sharing of human neuroimaging is removing facial features to protect individuals’ privacy. However, not only does this process redact identifiable information about individuals, but it also removes non-identif...Medical SciencesD. Samuel Schwarzkopf Cassandra Gould van Praag, Catherine Morgan, Abiola Akinnubi 2022-11-28 10:59:32 View
21 Mar 2023
STAGE 1
toto

Convenience Samples and Measurement Equivalence in Replication Research

Does data from students and crowdsourced online platforms measure the same thing? Determining the external validity of combining data from these two types of subjects

Recommended by ORCID_LOGO based on reviews by Benjamin Farrar and Shinichi Nakagawa
Comparative research is how evidence is generated to support or refute broad hypotheses (e.g., Pagel 1999). However, the foundations of such research must be solid if one is to arrive at the correct conclusions. Determining the external validity (the generalizability across situations/individuals/populations) of the building blocks of comparative data sets allows one to place appropriate caveats around the robustness of their conclusions (Steckler & McLeroy 2008).
 
In this registered report, Alley and colleagues plan to tackle the external validity of comparative research that relies on subjects who are either university students or participating in experiments via an online platform (Alley et al. 2023). They will determine whether data from these two types of subjects have measurement equivalence - whether the same trait is measured in the same way across groups. Although they use data from studies involved in the Many Labs replication project to evaluate this question, their results will be of crucial importance to other comparative researchers whose data are generated from these two sources (students and online crowdsourcing). If Alley and colleagues show that these two types of subjects have measurement equivalence, then this indicates that it is more likely that equivalence could hold for other studies relying on these type of subjects as well. If measurement equivalence is not found, then it is a warning to others to evaluate their experimental design to improve validity. In either case, it gives researchers a way to test measurement equivalence for themselves because the code is well annotated and openly available for others to use.

The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).

URL to the preregistered Stage 1 protocol: https://osf.io/7gtvf
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour (e.g. conservative statistical threshold; recruitment of a blinded analyst; robustness testing, multiverse/specification analysis, or other approach) 
 
List of eligible PCI RR-friendly journals:
 
 
References
 
Alley L. J., Axt, J., & Flake J. K. (2023). Convenience Samples and Measurement Equivalence in Replication Research, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/7gtvf
 
Steckler, A. & McLeroy, K. R. (2008). The importance of external validity. American Journal of Public Health 98, 9-10. https://doi.org/10.2105/AJPH.2007.126847
 
Pagel, M. (1999). Inferring the historical patterns of biological evolution. Nature, 401, 877-884. https://doi.org/10.1038/44766
Convenience Samples and Measurement Equivalence in Replication ResearchLindsay J. Alley, Jordan Axt, Jessica Kay Flake<p>A great deal of research in psychology employs either university student or online crowdsourced convenience samples (Chandler &amp; Shapiro, 2016; Strickland &amp; Stoops, 2019) and there is evidence that these groups differ in meaningful ways ...Social sciencesCorina Logan2022-11-29 18:37:54 View