Submit a report

Announcements

Please note: To accommodate reviewer and recommender holiday schedules, we will be closed to submissions from 1st July — 1st September. During this time, reviewers will be able to submit reviews and recommenders will issue decisions, but no new or revised submissions can be made by authors. The one exception to this rule is that authors using the scheduled track who submit their initial Stage 1 snapshot prior to 1st July can choose a date within the shutdown period to submit their full Stage 1 manuscript.

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.


 

Latest recommendationsrssmastodon

IdTitle * Authors * Abstract * PictureThematic fields * RecommenderReviewersSubmission date
27 Jun 2024
STAGE 1

Learning from comics versus non-comics material in education: Systematic review and meta-analysis

Comics in Education

Recommended by ORCID_LOGO based on reviews by Adrien Fillon, Benjamin Brummernhenrich, Solip Park and Pavol Kačmár
Especially after the impactful experiments in modern comics (e.g. McCloud 1993), research interest in the medium increased with new practical developments (Kukkonen 2013). Some of these developments now manifest in educational settings where comics are used for various pedagogical purposes in diverse cultural contexts. To what degree comics are able to reach educational outcomes in comparison to other pedagogical tools remains largely unknown, however.
 
In the present registered report, Pagkratidou and colleagues (2024) respond to the research gap by investigating the effectiveness of educational comics materials. By means of systematic review and meta-analysis, the authors assess all empirical studies on educational comics to map out what their claimed benefits are, how the reported effectiveness differs between STEM and non-STEM groups, and what moderating effects complicate the phenomenon. With the help of large language models, all publication languages are included in analysis. 
 
The research plan was reviewed over three rounds by four reviewers with diverse sets of expertise ranging from education and meta-analytic methodology to comics culture and design. After comprehensive revisions by the authors, the recommender considered the plan to meet high Stage 1 criteria and provided in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/vdr8c
 
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Kukkonen, K. (2013). Studying comics and graphic novels. John Wiley & Sons.
 
2. McCloud, S. (1993). Understanding comics: The invisible art. Tundra.
 
3. Pagkratidou, M., Cohn, N., Phylactou, P., Papadatou-Pastou, M., & Duffy, G. (2024). Learning from comics versus non-comics material in education: Systematic review and meta-analysis. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/vdr8c
Learning from comics versus non-comics material in education: Systematic review and meta-analysisMarianna Pagkratidou, Neil Cohn, Phivos Phylactou, Marietta Papadatou-Pastou, Gavin Duffy<p>The past decades have seen a growing use of comics (i.e., sequential presentation of images and/or text) educational material. However, there are inconsistent reports regarding their effectiveness. In this study, we aim to systematically review...Social sciencesVeli-Matti Karhulahti2023-10-16 22:55:09 View
11 Apr 2024
STAGE 1

Does retrieval practice protect memory against stress? A meta-analysis [Stage 1 Registered Report]

Can retrieval practice prevent the negative impact of acute stress on memory performance?

Recommended by ORCID_LOGO based on reviews by Chris Hartgerink (they/them) and Adrien Fillon
There are a number of broad assumptions about memory which have penetrated societal understanding and mostly reflect supporting academic evidence e.g., that acute stress can compromise memory performance (Shields et al., 2017) and that practicing recalling critical information can help retain that knowledge (Moriera et al., 2019). The evidence base is less consistent when evaluating whether retrieval practice can protect against the negative effects of acute stress on memory, despite it being highly important for educators as to whether this specific strategy for supporting memorisation can be evidenced as especially effective under stressful conditions. A rigorous review of this mixed evidence base could provide the basis for developments in memory theory and research practice, with potential for direct educational applications.
 
Meta-analyses can play a critical role in furthering our understanding of complex cognitive mechanisms where the evidence base includes a wide range of methods, factors and effect size estimates. Furthermore, there is a lack of rigorous meta-analyses that prioritise open and reproducible processes (Topor et al., 2022) which help role-model good practice. In the current Registered Report, Mihaylova et al. (2024) have proposed a rigorous meta-analysis to systematically review and synthesise the evidence on the effects of retrieval practice for memory performance under acute stress. The work looks to be especially valuable for a) informing future research directions through a structured risk of bias evaluation, and b) generating theoretical developments through a range of confirmatory moderators (including stressor types, memory strategies, time of delay and task type). The findings of the planned analyses are expected to be of immediate interest to educational and occupational domains where memory recall is a priority.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/pkrzb
 
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Mihaylova, M., Kliegel, M, & Rothen, N. (2024). Does retrieval practice protect memory against stress? A meta-analysis. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/pkrzb
 
2. Moreira, B. F. T., Pinto, T. S. S., Starling, D. S. V., & Jaeger, A. (2019). Retrieval practice in classroom settings: A review of applied research. In Frontiers in Education (Vol. 4, p. 5). Frontiers Media SA. https://doi.org/10.3389/feduc.2019.00005 
 
3. Shields, G. S., Sazma, M. A., McCullough, A. M., & Yonelinas, A. P. (2017). The effects of acute stress on episodic memory: A meta-analysis and integrative review. Psychological Bulletin, 143, 636–675. https://doi.org/10.1037/bul0000100 
 
4. Topor, M. K., Pickering, J. S., Mendes, A. B., Bishop, D., Büttner, F., Elsherif, M. M., ... & Westwood, S. (2022). An integrative framework for planning and conducting Non-Intervention, Reproducible, and Open Systematic Reviews (NIRO-SR). Meta-Psychology. https://osf.io/preprints/metaarxiv/8gu5z
Does retrieval practice protect memory against stress? A meta-analysis [Stage 1 Registered Report]Mariela Mihaylova, Matthias Kliegel, Nicolas Rothen<p>[Note: This is a Stage 1 Registered Report. All placeholders will be replaced with actual results by Stage 2.]</p> <p><br>Stressors such as test anxiety (TA) are known to decrease memory retrieval, whereas retrieval practice (RP) is the phenom...Humanities, Social sciencesThomas Evans2023-02-16 14:39:06 View
05 Aug 2022
STAGE 1

Through the lens of Developmental Coordination Disorder (DCD): experiences of a late diagnosis

Developmental Coordination Disorder Diagnosis as Part of Evolving Self-Concepts

Recommended by ORCID_LOGO based on reviews by Moin Syed, Gill Waters and Catherine Purcell
Although developmental coordination disorder (DCD) is a common neurodevelopmental disorder with an estimated prevalence of up to 6% in children (APA, 2013), many DCD diagnoses are not made before late adulthood. Receiving a neurodevelopmental disorder diagnosis has been found adding to people’s self-concepts, for instance, with autism spectrum disorder (Tan, 2018), but it is not well known if and how such events unfold in late DCD diagnoses. In this Stage 1 Registered Report, Topor et al. (2022) present a careful plan to qualitatively investigate the lived experiences of individuals with a late DCD diagnosis in order to map out the variety of emotional responses to diagnoses and their effects on self-concepts.

Topor et al. (2022) will carry out 10–15 semi-structured interviews with participants who received a DCD diagnosis at the age of 30 or after. They commit to realist epistemology when utilizing thematic analysis; namely, instructions have been preregistered for two separate analysts who will code the transcript data independently. At the same time, the methodology involves reflexive components. The authors have prepared strong positionality statements through which their analyses will be carried out with post-analysis reflections to be written at Stage 2. The coding process will explicitly involve a data analysis log that pursues interpretive transparency. The data and materials will be shared, which adds to the work's value in the context of open qualitative psychology in general.   
 
The study will help us better understand the process of receiving (late) DCD diagnoses and, specifically, how the emotional aftermath is potentially related to one’s evolving self-concept. In addition to making a clear contribution to cumulative scientific knowledge, the findings can be useful for professionals working with DCD-diagnosed individuals as well as for the development of related support services. The Registered Report format allowed the research design to be reviewed in three rounds before data collection. Initially, three experts representing developmental psychology and DCD reviewed the Stage 1 manuscript, after which the recommender carried out two iterations with further requested revisions. This was followed by in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/67h3f
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. APA (2013) Diagnostic and statistical manual of mental disorders, 5th ed, American Psychiatric Association, Washington, DC.

2. Tan, C. D. (2018). “I'm a normal autistic person, not an abnormal neurotypical”: Autism Spectrum Disorder diagnosis as biographical illumination.” Social Science & Medicine, 197, 161-167.
 
3. Topor, M., Armstrong, G., Gentle, J. (2022). Through the lens of Developmental Coordination Disorder (DCD): experiences of a late diagnosis, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/67h3f
Through the lens of Developmental Coordination Disorder (DCD): experiences of a late diagnosisMarta Topor, Gemma Armstrong, Judith Gentle<p>A late diagnosis of a lifelong neurodevelopmental condition has been shown to be an important life event with strong emotional consequences and deep personal reflections. &nbsp; Developmental Coordination Disorder (DCD/Dyspraxia) is a common, y...Medical Sciences, Social sciencesVeli-Matti Karhulahti2022-01-11 21:48:18 View
18 May 2023
STAGE 1

A multilab investigation into the N2pc as an indicator of attentional selectivity: Direct replication of Eimer (1996)

Is the N2pc a correlate of attentional selection? An #EEGManyLabs multi-lab registered replication of Eimer (1996)

Recommended by ORCID_LOGO based on reviews by Reny Baykova and Clayton Hickey
The N2pc is a lateralised ERP component that is often interpreted as a marker of attentional allocation, so much so that it is frequently used a tool in the attention literature for inferring that a stimulus was attentionally processed. This interpretation of N2pc can be traced back to the seminal work of Eimer (1996), which has been conceptually replicated many times but has never been replicated directly.
 
This registered direct replication by Constant et al. (2023) forms part of a larger series of large-scale, multi-lab replications of highly influential EEG papers by the #EEGManyLabs project (Pavlov et al., 2021). Seven labs (with the potential for more to sign up later), will conduct high-powered replications of the critical Experiment 2 of Eimer (1996), where in the crucial conditions, participants discriminate a target letter (M vs W) or colour (blue vs green) in the presence of a distractor. Using four preprocessing pipelines, including the original, the authors will test whether the N2pc is observed over parieto-occipital electrodes contralateral to target presentation.  
 
The Stage 1 manuscript was evaluated over one round of in-depth review and one additional round of minor corrections. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/dw68r
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA (so-called “primary RR”)
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Constant, M., Mandal, A., Asanowicz, D., Yamaguchi, M., Gillmeister, H., Kerzel, D., Luque, D., Pesciarelli, F., Fehr, T., Mushtaq, F., Pavlov, Y. G. & Liesefeld, H. R. (2023). A multilab investigation into the N2pc as an indicator of attentional selectivity: Direct replication of Eimer (1996), in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/dw68r
 
2. Eimer, M. (1996). The N2pc component as an indicator of attentional selectivity. Electroencephalography and Clinical Neurophysiology, 99, 225-234. https://doi.org/10.1016/0013-4694(96)95711-9
 
3. Pavlov, Y. G., Adamian, N., Appelhoff, S., Arvaneh, M., Benwell, C. S., Beste, C., ... & Mushtaq, F. (2021). #EEGManyLabs: Investigating the replicability of influential EEG experiments. Cortex, 144, 213-229. https://doi.org/10.1016/j.cortex.2021.03.013
A multilab investigation into the N2pc as an indicator of attentional selectivity: Direct replication of Eimer (1996)Martin Constant, Ananya Mandal, Dariusz Asanowicz, Motonori Yamaguchi, Helge Gillmeister, Dirk Kerzel, David Luque, Francesca Pesciarelli, Thorsten Fehr, Faisal Mushtaq, Yuri G. Pavlov, Heinrich R. Liesefeld<p>The N2pc is widely employed as an electrophysiological marker of an attention allocation. This interpretation was in no small part driven by the observation of an N2pc elicited by an isolated relevant target object, which was reported as Experi...Life Sciences, Social sciencesMaxine Sherman2023-02-24 11:41:52 View
21 Apr 2024
STAGE 1

Mechanisms of secularization: Testing between three causal pathways

Understanding links between secularization, rationalisation and insecurity

Recommended by ORCID_LOGO based on reviews by Chris Chambers and 1 anonymous reviewer
What relationship can be expected between secularization, rationalisation and insecurity? While some authors argue that rationalisation reduces the willingness to belong to religious groups, others have suggested that insecurity increases this need to belong to religious groups.
 
In the current study, Lang and Chvaja (2024) will adjudicate between these two possibilities using an economics game in participants from two countries: US and Poland. The central question posed by the authors is whether cooperative insecurity increases the probability of joining a religious normative group. They will test the relationship between an environment (secure and insecure) and institution (which related to the norm context: religious and secular) on the probability of choosing the normative group in an experimental setting. Therefore, the study will be a quantitative analysis.
 
The authors included an adequate power analysis, alternatives for non-supported hypotheses, and filtering to ensure a high quality of data collection. They also undertook a pilot study to ensure the quality of the procedure and sensitivity of the analyses.
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on ​detailed responses to reviewers’ and the recommender’s comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.​​​
 
URL to the preregistered Stage 1 protocol: https://osf.io/yzgek
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1.Lang, M. & Chvaja, R. (2024). Mechanisms of secularization: Testing between the rationalization and existential insecurity theories. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/yzgek
Mechanisms of secularization: Testing between three causal pathwaysMartin Lang, Radim Chvaja<p>The study tests two competing explanations of the secularization process related to rationalizing worldviews and decreasing existential insecurity. While the former explanation argues that people are unwilling to join religious groups because o...Social sciencesAdrien Fillon2023-11-22 11:17:30 View
14 Feb 2022
STAGE 1

Minimal mindfulness of the world as an active control for a full mindfulness of mental states intervention: A Registered Report and Pilot study

Testing the metacognitive basis and benefits of mindfulness training

Recommended by ORCID_LOGO based on reviews by Chris Noone and Julieta Galante

Mindfulness is inherently metacognitive in that it requires monitoring of one’s own thoughts and attention in order to remain on task. Mindfulness practice is especially metacognitive when focused on internal mental states, rather than on the external world. This Registered Report will compare remote training in mindfulness of mental states with remote training in mindfulness of the world, and a wait list control, to test the idea that mindfulness of mental states has an additional metacognitive component, and that this has benefits for (self-reported) mental health. A comparison of participant expectancies between mindfulness conditions will be used to establish whether mindfulness of the world can be considered a true ‘active control’ condition. Additional comparisons will test whether this mindfulness control has benefits over the (inactive) waiting list condition. The study plan is informed by prior research, including pilot data presented in the Stage 1 report, and a sample of up to 300 participants will be tested.

The Stage 1 plan has been evaluated through two round of signed external review, and a further two rounds of minor revisions, with the recommender obtaining specialist advice on key points from a relevant external expert. The recommender has judged that the manuscript now meets all Stage 1 criteria, and has awarded In Principle Acceptance (IPA).

URL to the preregistered Stage 1 protocol: https://osf.io/tx54k

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.

List of eligible PCI RR-friendly journals:

References

Lovell, M., & Dienes, Z. (2022). Minimal mindfulness of the world as an active control for a full mindfulness of mental states intervention: A Registered Report and Pilot study, in principle acceptance of version 4 by Peer Community in Registered Reports. https://osf.io/tx54k

Minimal mindfulness of the world as an active control for a full mindfulness of mental states intervention: A Registered Report and Pilot studyMax Lovell, Zoltan Dienes<p>Mindfulness is a continual renewal of non-elaborative attention on an object of focus, without clinging or aversion, and with equanimity. As this requires the capacity to monitor and control the extent to which one is on task, it is a metacogni...Social sciencesRobert McIntosh2021-06-21 16:49:39 View
09 Jun 2022
STAGE 1

Revisiting diversification bias and partition dependence: Replication and extensions of Fox, Ratner, and Lieb (2005) Studies 1, 2, and 5

Testing the replicability of diversification bias and partition dependence

Recommended by ORCID_LOGO based on reviews by Craig Fox and Leo Cohen
When offered a range of options and asked to make multiple selections, how do people choose? Over the last 30 years, a key finding to emerge from behavioural economics is that people distribute their choices more evenly than would be considered optimal – a phenomenon termed “diversification bias” or the “diversification heuristic” (Read & Loewenstein, 1995). For example, when filling a plate from a buffet, you might be inclined to choose a relatively even amount of everything on offer, even when you prefer some foods over others. Similarly, when allocating savings among different investment options, people are prone to spreading their money more evenly than would maximise utility.
 
In an influential study, Fox et al. (2005) found that diversification bias can be shaped by so-called “partition dependence” – the tendency to allocate resources differently across options depending on how they are subjectively grouped. Such groupings could be arbitrary; so, for example, to return to the buffet example, people might diversify across high-level categories such as cooked and uncooked, savoury and sweet, or surf and turf, and then diversify across the options within those categories. The nature of level of these subjective (and potentially arbitrary) categorisations can strongly influence the final allocation of resources. Diversification bias and partition dependence have important implications for basic theory in judgment and decision-making as well as applications in behavioral economics and finance.
 
In the current study, Li and Feldman (2022) propose to replicate Studies 1, 2 and 5 from Fox et al. (2002) in a large online sample. In particular, they plan to ask how partitioning influences the allocation of choices between options, and the extent to which partition dependence is reduced in people with greater relevant expertise. The authors also propose extending the original study to explore individual differences in the desire for choice diversity as predictors of partition dependence.
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/bx8vq
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Read, D., & Loewenstein, G. (1995). Diversification bias: Explaining the discrepancy in variety seeking between combined and separated choices. Journal of Experimental Psychology: Applied, 1, 34-49. https://doi.org/10.1037/1076-898X.1.1.34
 
2. Fox, C. R., Ratner, R. K., & Lieb, D. S. (2005). How subjective grouping of options influences choice and allocation: Diversification bias and the phenomenon of partition dependence. Journal of Experimental Psychology: General, 134, 538-551. https://doi.org/10.1037/0096-3445.134.4.538
  
3. Li, M. Y. & Feldman, G. (2022). Revisiting diversification bias and partition dependence: 
Replication and extension of Fox, Ratner, and Lieb (2005) Studies 1, 2, and 5, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/bx8vq
Revisiting diversification bias and partition dependence: Replication and extensions of Fox, Ratner, and Lieb (2005) Studies 1, 2, and 5Mei Yee (Alice) Li, Gilad Feldman<p>This is a scheduled PCI-RR snap shot for a planned project: "Revisiting diversification bias and partition dependence: Replication and extensions of Fox, Ratner, and Lieb (2005) Studies 1, 2, and 5 ​"</p>Social sciencesChris Chambers Craig Fox, Leo Cohen2022-02-15 09:57:51 View
17 May 2022
STAGE 1

Revisiting mental accounting classic paradigms: Replication of Thaler (1999) and an extension examining impulsivity

Mental accounting under the microscope

Recommended by ORCID_LOGO based on reviews by Barnabas Szaszi and Féidhlim McGowan
In recent years, the study of mental accounting – the thought processes by which people informally record, categorise, and evaluate the costs and benefits of their financial transactions – has been an active research area, drawing attention to a range of biases and distortions that deviate from optimal economic decision-making (Zhang & Sussman, 2018). Although the term “mental accounting” is a relatively recent construction (Thaler, 1999), it stems from a longer history of behavioural economic research on value functions, decision frames, risk-taking, and related concepts.
 
In the current study, Li and Feldman propose to replicate 17 influential mental accounting problems (or tasks) reviewed by Thaler (1999) in a large online sample. The authors also propose several extensions examining the effects of sunk costs and expenses framing. 
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/d6cjk
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Zhang, C. Y., & Sussman, A. B. (2018). Perspectives on mental accounting: An exploration of budgeting and investing. Financial Planning Review, 1, e1011. https://doi.org/10.1002/cfp2.1011
 
2. Thaler, R. H. (1999). Mental accounting matters. Journal of Behavioral Decision Making, 12, 183-206. https://doi.org/10.1002/(SICI)1099-0771(199909)12:3%3C183::AID-BDM318%3E3.0.CO;2-F
 
3. Li, M. & Feldman, G. (2022). Revisiting mental accounting classic paradigms: Replication of the problems reviewed in Thaler (1999), in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/d6cjk
Revisiting mental accounting classic paradigms: Replication of Thaler (1999) and an extension examining impulsivityMengfei Li; Gilad Feldman <p>This is a scheduled PCI-RR snap shot for a planned project: "Revisiting mental accounting classic paradigms: Replication of Thaler (1999) and an extension examining impulsivity​"</p>Social sciencesChris Chambers2022-01-31 10:18:38 View
21 Oct 2022
STAGE 1

Replicating the facilitatory effects of transcranial random noise stimulation on motion processing: A registered report

Testing the facilitatory effect of high-frequency transcranial random noise stimulation through enhancement of global motion processing

Recommended by ORCID_LOGO based on reviews by Sam Westwood and Filippo Ghin
High frequency transcranial random noise stimulation (hf-tRNS) is a relatively novel form of non-invasive brain stimulation, thought to enhance neural excitability and facilitate processing in targeted brain areas. The evidence for the efficacy of hf-tRNS is mixed, so a high-powered test of the proposed facilitatory effects would be of value to the field. This Registered Report will target the human middle temporal complex (hMT+), an area with a well-established critical role in global motion processing. The protocol is adapted from a study by Ghin and colleagues (2018) but focusing on a sub-set of the original experimental conditions and using a fully within-subjects design (n=42). Global motion processing will be operationalised in terms of the coherence threshold for identification of the dominant direction of random-dot motion. The experiment will test the predicted facilitation of contralateral motion processing (reduced coherence threshold) during hf-tRNS to the left hMT+. The specificity of this effect will be tested by comparison to a sham stimulation control condition and an active stimulation control condition (left forehead). By targeting a brain area with a well-established critical role in behaviour, this study will provide important information about the replicability and specificity of the facilitatory effects of hf-tRNS.
 
Following two rounds of in-depth review, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).  
 
URL to the preregistered Stage 1 protocol: https://osf.io/bce7u
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Ghin, F., Pavan, A., Contillo, A., & Mather, G. (2018). The effects of high-frequency transcranial random noise stimulation (hf-tRNS) on global motion processing: an equivalent noise approach. Brain Stimulation, 11, 1263–75.
 
2. Caroll, M. B., Edwards, G. & Baker, C. I. (2022). Replicating the facilitatory effects of transcranial random noise stimulation on motion processing: A registered report, in principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/bce7u
Replicating the facilitatory effects of transcranial random noise stimulation on motion processing: A registered reportMica B. Carroll*, Grace Edwards*, Chris I. Baker<p>Non-invasive brain stimulation (NIBS) techniques have the potential to demonstrate the causal impact of targeted brain regions on specific behaviors, and to regulate or facilitate behavior in clinical applications. Transcranial random noise sti...Life SciencesRobert McIntosh2022-06-02 21:25:02 View
09 Jul 2024
STAGE 1

Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task

Exploring determinants of test-retest reliabilty in fMRI

Recommended by ORCID_LOGO based on reviews by Xiangzhen Kong and 2 anonymous reviewers
Functional magnetic resonance imaging (fMRI) has been used to explore brain-behaviour relationships for many years, with proliferation of a wide range of sophisticated analytic procedures. However, rather scant attention has been paid to the reliability of findings. Concerns have been growing following failures to replicate findings in some fields, but it is hard to know how far this is a consequence of underpowered studies, flexible analytic pipelines, or variability within and between participants. 
 
Demidenko et al. (2023) plan a study that will be a major step forward in addressing these issues. They take advantage of the availability of three existing datasets, the Adolescent Brain Cognitive Development (ABCD) study, the Michigan Longitudinal Study, and the Adolescent Risk Behavior Study, which all included a version of the Monetary Incentive Delay task measured in two sessions. This gives ample data for a multiverse analysis, which will consider how within-subject and between-subject variance varies according to four analytic factors: smoothing (5 levels), motion correction (6 levels), task modelling (3 levels) and task contrasts (4 levels).  They will also consider how sample size affects estimates of reliability. This will involve a substantial amount of analysis.
 
The study is essentially focused on estimation, although specific predictions are presented regarding the combinations of factors expected to give optimal reliability.  The outcome will be a multiverse of results which will allow us to see how different pipeline decisions for this task affect reliability. In many ways, null results – finding that at least some factors have little effect on reliability – would be a positive finding for the field, as it would mean that we could be more relaxed when selecting an analytic pathway. A more likely outcome, however, is that analytic decisions will affect reliability, and this information can then guide future studies and help develop best practice guidelines. As one reviewer noted, we can’t assume that results from this analysis will generalise to other tasks, but this analysis with a widely-used task is an important step towards better and more consistent methods in fMRI.
 
The researchers present a fully worked out plan of action, with supporting scripts that have been developed in pilot testing. The Stage 1 manuscript received in-depth evaluation from three expert reviewers and the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/nqgeh
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet sufficiently observed the key variables within the data to be able to answer the research questions AND they have taken additional steps to maximise bias control and rigour.
 
List of eligible PCI RR-friendly journals:
 

References

1. Demidenko, M. I., Mumford, J. A., & Poldrack, R. A. (2023). Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/nqgeh
Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay TaskMichael I. Demidenko, Jeanette A. Mumford, Russell A. Poldrack<p>Empirical studies reporting low test-retest reliability of individual neural estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods that may improve reliability in fMR...Life Sciences, Social sciencesDorothy Bishop2023-04-17 22:27:54 View