Submit a report

Announcements

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.

Latest recommendations

Id▲Title * Authors * Abstract * PictureThematic fields * RecommenderReviewersSubmission date
18 Aug 2023
STAGE 2
(Go to stage 1)

Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation

Incorporating open research practices into the undergraduate curriculum increases understanding of such practices

Recommended by ORCID_LOGO based on reviews by Kelsey McCune, Neil Lewis, Jr., Lisa Spitzer and 1 anonymous reviewer
In a time when open research practices are becoming more widely used to combat questionable research practices (QRPs) in academia, this Registered Report by Pownall and colleagues (2023) empirically investigated the practice of preregistering study plans, which allows us to better understand to what degree such practices increase awareness of QRPs and whether experience with preregistration helps reduce engagement in QRPs. This investigation is timely because results from these kinds of studies are only recently becoming available and the conclusions are providing evidence that open research practices can improve research quality and reliability (e.g., Soderberg et al. 2021, Chambers & Tzavella 2022). The authors crucially focused on the effect of preregistering the undergraduate senior thesis (of psychology students in the UK), which is a key stage in the development of an academic.
 
Pownall and colleagues found that preregistration did not affect attitudes toward QRPs, but it did improve student understanding of open research practices. Using exploratory analyses, they additionally found that those who preregistered were those students who reported that they had more opportunity, motivation, and greater capability. This shows how important it is to incorporate the teaching of open research practices such that students can increase their capability, motivation, and opportunity to pursue such practices, whether it is preregistration or other practices that are better known to reduce QRPs (such as registered reports; Krypotos et al. 2022). 
 
After four rounds of review and revisions, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/9hjbw
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Chambers C. D. & Tzavella, L. (2022). The past, present, and future of Registered Reports. Nature Human Behaviour, 6, 29-42. https://doi.org/10.1038/s41562-021-01193-7
 
2. Krypotos, A. M., Mertens, G., Klugkist, I., & Engelhard, I. M. (2022). Preregistration: Definition, advantages, disadvantages, and how it can help against questionable research practices. In Avoiding Questionable Research Practices in Applied Psychology (pp. 343-357). Cham: Springer International Publishing.
 
3. Pownall, M., Pennington, C. R., Norris, E., Juanchich, M., Smaile, D., Russell, S., Gooch, D., Rhys Evans, T., Persson, S., Mak, M. H. C., Tzavella, L., Monk, R., Gough, T., Benwell, C. S. Y., Elsherif, M., Farran, E., Gallagher-Mitchell, T., Kendrick, L. T., Bahnmueller, J., Nordmann, E., Zaneva, M., Gilligan-Lee, K., Bazhydai, M., Jones, A., Sedgmond, J., Holzleitner, I., Reynolds, J., Moss, J., Farrelly, D., Parker, A. J. & Clark, K. (2023). Evaluating the pedagogical effectiveness of study preregistration in the undergraduate dissertation [Stage 2 Registered Report], acceptance of Version 4 by Peer Community in Registered Reports. https://psyarxiv.com/xg2ah
 
4. Soderberg C. K., Errington T. M., Schiavone S. R., Bottesini J., Thorn F. S., Vazire S., Esterling K. M. & Nosek B. A. (2021) Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour, 5, 990–997. https://doi.org/10.1038/s41562-021-01142-4
Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate DissertationMadeleine Pownall, Charlotte R. Pennington, Emma Norris, Marie Juanchich, David Smaile, Sophie Russell, Debbie Gooch, Thomas Rhys Evans, Sofia Persson, Matthew HC Mak, Loukia Tzavella, Rebecca Monk, Thomas Gough, Christopher SY Benwell, Mahmoud El...<p>Research shows that questionable research practices (QRPs) are present in undergraduate final-year dissertation projects. One entry-level Open Science practice proposed to mitigate QRPs is ‘study preregistration’, through which researchers outl...Life Sciences, Social sciencesCorina Logan2023-03-25 11:38:54 View
27 Nov 2023
STAGE 1

Cortical voice processing in Autism Spectrum Disorder

Is voice processing impacted in Autism Spectrum Disorder?

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by 2 anonymous reviewers
Vocal sounds, including both speech and non-speech sounds, have been found to activate the Superior Temporal Sulci and Gyri in comparison to non-vocal sounds. These regions, termed Temporal Voice Areas (TVAs), are considered to be involved in early voice processing and therefore critical for social interaction. TVA activation has been examined in Autism Spectrum Disorder (ASD) to determine if the characteristic difficulties in social communication and interaction are linked to an impaired early voice processing. Using functional magnetic resonance imaging (fMRI), one study found typical brain activation in TVAs for 15 out of 16 autistic participants (Schelinski et al., 2016), whereas another found atypical activation in 4 out of 5 autistic participants (Gervais et al., 2004).
 
Here, the inconsistencies in the previous literature propel Gautier et al. (2023) to examine brain activation of TVAs with a larger sample size (26 ASD and 26 non-ASD participants). Gautier et al. (2023) will present vocal sounds and non-vocal sounds to both groups of participants during fMRI and predict that fewer participants in the ASD group will show a preferential response to voices in TVAs compared to the non-ASD group. These results would suggest that symptoms of ASD interfere with early stages of social interaction, at the level of voice processing.
 
This Stage 1 manuscript was evaluated in an initial round by the co-recommenders and another two rounds of in-depth review by two expert reviewers. With these revisions, the recommenders judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/538m4
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Gautier, R., Houy-Durand, E., Barantin, L., Briend, F. & Latinus, M. (2023). Cortical voice processing in Autism Spectrum Disorder. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/538m4
Cortical voice processing in Autism Spectrum DisorderRaphaël Gautier, Emmanuelle Houy-Durand, Laurent Barantin, Frederic Briend, Marianne Latinus<p>Voice processing is central to social functioning. A specific brain response to vocal sounds has been described and extensively characterized in the general population but remains critically unexplored in Autism Spectrum Disorder (ASD), a condi...Life SciencesGrace Edwards2023-03-28 09:51:48 View
08 Sep 2023
STAGE 1

Evaluation of spatial learning and wayfinding in a complex maze using immersive virtual reality. A registered report

Evaluation of an immersive virtual reality wayfinding task

Recommended by ORCID_LOGO based on reviews by Conor Thornberry, Gavin Buckingham and 1 anonymous reviewer
The Virtual Maze Task (VMT) is a digital desktop 2D spatial learning task that has been used for research into the effect of sleep and dreaming on memory consolidation (e.g. Wamsley et al, 2010). One limitation of this task has been low rates of reported dream incorporation. Eudave and colleagues (2023) have created an immersive virtual reality (iVR) version of the VMT, which they believe might be more likely to be incorporated into dreams. As an initial step in validating this task for research, they propose a within-subjects study to compare three measures of spatial learning between the 2D desktop and iVR versions. Based on a review of relevant literature, the prediction is that performance will be similar between the two task versions. The planned sample size (n = 62) is sufficient for a .9 power test of equivalence within effect size bounds of d = -.47 to .47. Additional independent variables (gender, perspective-taking ability) and dependent measures (self-reported cybersickness and sense of presence) will be recorded for exploratory analyses.
 
The study plan was refined across four rounds of review, with input from two external reviewers and the recommender, after which it was judged to satisfy the Stage 1 criteria for in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/wba2v
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
Eudave, L., Martínez, M., Valencia, M., & Roth D. (2023). Evaluation of spatial learning and wayfinding in a complex maze using immersive virtual reality. A registered report. In principle acceptance of Version 5 by Peer Community in Registered Reports.
 
Wamsley, E. J., Tucker, M., Payne, J. D., Benavides, J. A., & Stickgold, R. (2010). Dreaming of a learning task is associated with enhanced sleep-dependent memory consolidation. Current Biology, 20, 850–855. https://doi.org/10.1016/j.cub.2010.03.027
 
† There is one minor change that the authors should make to the Methods section, which is sufficiently small that it can be incorporated at Stage 2: "if both tests reject the null hypothesis (observed data is less/greater than the lower/upper equivalence bounds), conditions are considered statistically equivalent" >> suggest changing "less/greater" to "greater/lesser" for correct correspondence with "lower/upper".
Evaluation of spatial learning and wayfinding in a complex maze using immersive virtual reality. A registered reportEudave L., Martínez M., Valencia M., Roth D.<p style="text-align: justify;"><strong>Objectives</strong>: Mazes have traditionally been used as tools for evaluating spatial learning and navigational abilities in humans. They have been also utilized in sleep and dream research, as wayfinding ...Life SciencesRobert McIntosh2023-03-31 17:21:20 View
22 Oct 2023
STAGE 1

Implicit Ideologies: Do Right-Wing Authoritarianism and Social Dominance Orientation Predict Implicit Attitudes?

Do social dominance orientation and right wing authoritarianism similarly predict both explicit and implicit attitudes?

Recommended by ORCID_LOGO based on reviews by Luisa Liekefett, Oluwaseyi Adeliyi, Abiola Akinnubi ? and 1 anonymous reviewer
Measurement is a vital activity for all research areas, but we so often fail to provide sufficient clarity, rigor and transparency about it, undermining the validity of our studies' conclusions (Flake & Fried, 2020). This concern is of wide societal interest when applied to the domains of ideology and attitudes where measurements of both implicit and explicit attitudes are assumed to reflect the same underlying concept. The extent to which this can be accepted is undermined by mixed evidence demonstrating a lack of consensus on the extent to which relevant psychological factors similarly predict both implicit and explicit attitudes.
 
In the proposed study, Reid & Inbar (2023) question these assumptions through use of the Project Implicit dataset, exploring the extent to which social dominance orientation (SDO) and right wing authoritarianism (RWA) similarly predict implicit and explicit attitudes. This work is ideally suited for publication through the Registered Reports format because whilst we may expect that relationships between SDO/RWA are similar in effect size across measures of both implicit and explicit attitude (because they tap into the same underlying attitude), there is great scope to acknowledge a more complex set of findings which may not be immediately interpretable or coherent. The proposed work will help us unpack further the assumptions surrounding measurement of attitudes and can help us better understand the extent to which SDO and RWA predict atittudes. 
 
The Stage 1 manuscript was evaluated over three rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/zv4jw
 
Level of bias control achieved: Level 5. All of the data or evidence that will be used to answer the research question already exist, but are currently inaccessible to the authors and thus unobservable prior to IPA (e.g. held by a gatekeeper) 
 
List of eligible PCI-RR-friendly journals:
 
 
References:
 
1. Flake, J. K. & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3, 456-465. https://doi.org/10.1177/2515245920952393
 
2. Reid, J. & Inbar, Y. (2023). Implicit Ideologies: Do Right-Wing Authoritarianism and Social Dominance Orientation Predict Implicit Attitudes? In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/zv4jw
Implicit Ideologies: Do Right-Wing Authoritarianism and Social Dominance Orientation Predict Implicit Attitudes? Jesse S Reid, Yoel Inbar<p>A wide variety of attitudes, beliefs and behaviours can be predicted by political ideology – and more specifically, by the two ideological scales of Right-Wing Authoritarianism (RWA; a preference for authority and tradition) and Social Dominanc...Social sciencesThomas Evans2023-04-08 01:53:30 View
25 Mar 2024
STAGE 1

The role of positive and negative emotions on multiple components of episodic memory (“what”, “when”, “in which context”) in older compared to younger adults: a pre-registered study

The role of emotion and age on different facets of episodic memory (“what”, “when”, “in which context”) 

Recommended by ORCID_LOGO based on reviews by Mara Mather and 1 anonymous reviewer
How does emotion influence item memory (what?) temporal memory (when?) and associative memory (in which context?), and does this differ for younger and older adults? Previous research has found inconsistent results, possibly due to small sample sizes. In this study, Laulan and Rimmele (2024) and will build on the paradigm in Palombo et al. (2021) in which participants see images embedded in videos and are asked to remember the images (what?), their temporal position within the videos (where?), and the association between the images and the videos (in which context?). Image valence (positive vs negative vs neutral) and participant age-group (18-30 vs 60-80 yr olds) are manipulated. Pre-registered analyses will first look at the two age groups separately to test for an effect of valence for each of the memory components, and second test for modulating effects of age-group. To be cost-effective, a sequential analysis approach with statistical analyses conducted at three time points and a maximum sample size of 150 younger and 150 older adults is planned.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/z4e8j
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

References
 
1. Laulan, P. & Rimmele, U. (2024). The role of positive and negative emotions on multiple components of episodic memory (“what”, “when”, “in which context”) in older compared to young adults: a pre-registered study. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/z4e8j
 
2. Palombo, D. J., Te, A. A., Checknita, K. J. & Madan, C. R. (2021). Exploring the Facets of Emotional Episodic Memory: Remembering “What,” “When,” and “Which”. Psychological Science, 32, 1104–1114. https://doi.org/10.1177/0956797621991548
The role of positive and negative emotions on multiple components of episodic memory (“what”, “when”, “in which context”) in older compared to younger adults: a pre-registered studyPierrick Laulan, Ulrike Rimmele<p>Emotion and age modulate episodic memory. In both young and older adults, emotion has a beneficial effect on item memory, with an advantage for positive vs. negative stimuli in older adults. In young adults, emotion has also been shown to enhan...Social sciencesElizabeth Wonnacott2023-04-12 16:58:21 View
07 Aug 2023
STAGE 2
(Go to stage 1)

The link between Empathy and Forgiveness: Replication and extensions Registered Report of McCullough et al. (1997)'s Study 1

Strong evidence that empathy is important for forgiveness

Recommended by ORCID_LOGO based on reviews by James Bartlett and Saleh Shuqair
Forgiveness is a core feature of human psychology in which a person makes a deliberate decision to cease negative emotions or attitudes toward an offender who has done them harm. The concept of interpersonal forgiveness is deeply embedded across societies, but much remains to be understood about how it actually works. What are its key ingredients and why does it occur in the first place? Research in social psychology has demonstrated a range of personal and social benefits of forgiveness, giving rise to two dominant mechanistic accounts – one that positions empathy as the driving factor and another that centres motivated reasoning (Donovan & Priester, 2017).
 
In the current study, Chan and Feldman (2023) sought to replicate a formative study by McCullough et al (1997) that led to the so-called Empathy Model of forgiveness. According to this theory, forgiving is a motivational change facilitated (crucially) by empathy, promoting constructive over destructive behaviour toward the offender. Chan and Feldman replicated Study 1 from McCullough et al., measuring the correlational relationship between apology, forgiving, and empathy for offenders, and exploring whether forgiving is associated with increased conciliation and decreased avoidance motivation. As well as closely replicating the original study, the authors extended it to test the more severe hypothesis that empathy causally influences forgiveness. To achieve this, they experimentally manipulated empathy by adding two groups to the design: one in which participants were asked to recall hurtful past experiences in which they were not empathetic to the offender, and another in which they were highly empathetic.
 
The outcomes constitute a successful replication. Affective empathy was positively associated with perceived apology and forgiveness, and forgiveness was positively associated with conciliation motivation and negatively associated with both avoidance motivation and revenge motivation. In addition, the results of the experimental extension revealed a reliable causal effect of empathy on forgiveness and perceived apology. Overall, the findings provide robust support for the Empathy Model of forgiveness.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on responses to the comments, the recommender judged that the manuscript met the Stage 2 criteria and therefore awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/q78fs
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Donovan, L. A. N., & Priester, J. R. (2017). Exploring the psychological processes underlying interpersonal forgiveness: The superiority of motivated reasoning over empathy. Journal of Experimental Social Psychology, 71, 16-30. https://doi.org/10.1016/j.jesp.2017.02.005
 
2. McCullough, M. E., Worthington, E. L., & Rachal, K. C. (1997). Interpersonal Forgiving in Close Relationships. Journal of Personality and Social Psychology, 73, 321–336. https://doi.org/10.1037/0022-3514.73.2.321 
 
3. Chan, C. F. & Feldman, G. (2023). The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1, acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/956fa
The link between Empathy and Forgiveness: Replication and extensions Registered Report of McCullough et al. (1997)'s Study 1Chan Chi Fung, Gilad Feldman <p>The empathy model of forgiveness conceptualized forgiving as an empathy-facilitated motivational change that leads to reductions in the motivation to behave in relationship -destructive ways and increases in the motivation to behave in relation...Social sciencesChris Chambers2023-04-13 14:28:05 View
25 Mar 2024
STAGE 1

Reading and vocabulary knowledge in English-Meetei Mayek biliterates

Diversifying our understanding of children’s word learning

Recommended by ORCID_LOGO based on reviews by Maxine Schaefer and 1 anonymous reviewer
The positive relation between word reading and children’s vocabulary development has been extensively documented. That said, like most research in psychology and the behavioral sciences, the available evidence comes predominantly from majority populations. In the context of language learning, that means monolingual speakers or multilingual speakers where there is close alignment between home and school language learning. But what does the relation between word learning and vocabulary knowledge look like when the learning contexts are discordant?
 
In the current study, Pamei et al. (2024) propose to examine this question by investigating word learning and vocabulary development in two languages, English and Meetei Mayek, among a sample of Grade 3 (approximate age 10), students in Manipur, India. In this context, formal literacy education begins in English rather than in students’ regional home language of Meetei Mayek. This fact provides an innovative context in which to understand how a) whether the relation between word reading and vocabulary looks different in the two languages, and b) whether there is linguistic interdependence between learning in the two languages. This study is poised to bring important underrepresented data that goes beyond the dominant contexts from which our knowledge of language learning has been generated, and thus has the potential to contribute to new lines of empirical and theoretical work that is inclusive of global variations. 
 
The Stage 1 manuscript was evaluated over three rounds of in-depth peer review, the first two consisting of substantial comments from two scholars with relevant expertise, and the third consisting of a close review by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/7htv2
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Pamei, G., McBride, C. & Inoue, T. (2024). Reading and vocabulary knowledge in English-Meetei Mayek biliterates. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/7htv2
Reading and vocabulary knowledge in English-Meetei Mayek biliteratesGairan Pamei, Catherine McBride, Tomohiro Inoue<p>The objective of the current study is to examine the association between word reading and vocabulary knowledge in English and Meetei Mayek. The target sample (N = 354) is children in Grades 3 and 4 attending schools in Manipur, the average age ...Social sciencesMoin Syed2023-04-15 14:16:14 View
18 Oct 2023
STAGE 1

Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation?

Understanding how visual cues influence extraretinal representation of planar symmetry

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by Tadamasa Sawada, Guillaume Rousselet, Benoit Cottereau and Deborah Apthorp
Visual symmetry is critical to our interaction with our environment so that when detected, symmetry automatically produces a neural marker in the form of an Event Related Potential (ERP) called Sustained Posterior Negativity (SPN). However, when symmetry is presented to the visual system slanted away from the viewer, there is a reduction in SPN, termed a perspective cost. 
 
Considering ​objects are rarely presented front-on (or frontoparallel) in our natural environment, Karakashevska et al., (2023) plan to examine the extent of the perspective cost with the addition of visual cues to facilitate extraretinal representation of the visual symmetry. The authors will record electroencephalography (EEG) from 120 participants while they perform a luminance task on symmetrical and asymmetrical stimuli. The authors hypothesize perspective cost will be reduced by three perspective cues: 1) monocular viewing, when cue conflict caused by binocular viewing is eliminated, 2) a static frame surrounding the symmetrical stimulus, adding a depth cue, and 3) a moving frame, assisting 3D perception prior to the symmetry onset. If the SPN is equivalent during frontoparallel and slanted presentation in a cue condition, the authors will conclude extraretinal representation can be automatic when sufficient visual cues are available. The proposed experiment is powered to detect a relatively small difference between perspective cue conditions. This will solidify fundamental research on visual symmetry processing and will further our understanding of object perception and recognition. 
 
The Stage 1 manuscript was evaluated over three rounds by four expert reviewers. Following in-depth review and responses from the authors, the recommenders have determined that Stage 1 criteria was met and have awarded in-principle acceptance (IPA). 
​​​​
URL to the preregistered Stage 1 protocol: https://osf.io/yzsq5
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI-RR-friendly journals:
 
 
References
 
1. Karakashevska, E., Bertamini, M. & Makin, A. D. J. (2023). Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation? [Stage 1 Registered Report]. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/yzsq5
Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation?Elena Karakashevska, Marco Bertamini and Alexis D.J. Makin <p>A challenge for the visual brain is to identify objects from a range of different viewpoints. This study will determine the conditions under which the brain spends computational resources to achieve view-invariance. We focus on view-invariant r...Life SciencesGrace Edwards2023-04-17 21:52:26 View
09 Jul 2024
STAGE 1

Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task

Exploring determinants of test-retest reliabilty in fMRI

Recommended by ORCID_LOGO based on reviews by Xiangzhen Kong and 2 anonymous reviewers
Functional magnetic resonance imaging (fMRI) has been used to explore brain-behaviour relationships for many years, with proliferation of a wide range of sophisticated analytic procedures. However, rather scant attention has been paid to the reliability of findings. Concerns have been growing following failures to replicate findings in some fields, but it is hard to know how far this is a consequence of underpowered studies, flexible analytic pipelines, or variability within and between participants. 
 
Demidenko et al. (2023) plan a study that will be a major step forward in addressing these issues. They take advantage of the availability of three existing datasets, the Adolescent Brain Cognitive Development (ABCD) study, the Michigan Longitudinal Study, and the Adolescent Risk Behavior Study, which all included a version of the Monetary Incentive Delay task measured in two sessions. This gives ample data for a multiverse analysis, which will consider how within-subject and between-subject variance varies according to four analytic factors: smoothing (5 levels), motion correction (6 levels), task modelling (3 levels) and task contrasts (4 levels).  They will also consider how sample size affects estimates of reliability. This will involve a substantial amount of analysis.
 
The study is essentially focused on estimation, although specific predictions are presented regarding the combinations of factors expected to give optimal reliability.  The outcome will be a multiverse of results which will allow us to see how different pipeline decisions for this task affect reliability. In many ways, null results – finding that at least some factors have little effect on reliability – would be a positive finding for the field, as it would mean that we could be more relaxed when selecting an analytic pathway. A more likely outcome, however, is that analytic decisions will affect reliability, and this information can then guide future studies and help develop best practice guidelines. As one reviewer noted, we can’t assume that results from this analysis will generalise to other tasks, but this analysis with a widely-used task is an important step towards better and more consistent methods in fMRI.
 
The researchers present a fully worked out plan of action, with supporting scripts that have been developed in pilot testing. The Stage 1 manuscript received in-depth evaluation from three expert reviewers and the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/nqgeh
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet sufficiently observed the key variables within the data to be able to answer the research questions AND they have taken additional steps to maximise bias control and rigour.
 
List of eligible PCI RR-friendly journals:
 

References

1. Demidenko, M. I., Mumford, J. A., & Poldrack, R. A. (2023). Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/nqgeh
Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay TaskMichael I. Demidenko, Jeanette A. Mumford, Russell A. Poldrack<p>Empirical studies reporting low test-retest reliability of individual neural estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods that may improve reliability in fMR...Life Sciences, Social sciencesDorothy Bishop2023-04-17 22:27:54 View
21 May 2023
STAGE 2
(Go to stage 1)

Sunk cost effects for time versus money: Replication and extensions Registered Report of Soman (2001)

Mixed evidence for the hypothesis that sunk cost effects are weaker for time than money

Recommended by ORCID_LOGO based on reviews by Christopher Olivola and Dilip Soman
The sunk cost fallacy is a cognitive bias in which people persist with a decision that is no longer optimal because of previous resources they have invested (now considered to be spent or “sunk”). Most of us will have heard sunk costs reflected in the saying “throwing good money after bad”, but sunk costs can, in theory, occur more broadly, whether for money, time or any other resource-limited investment. The sunk cost effect for money has been widely studied and appears robust; in contrast, the sunk cost effect for time is more uncertain, and is potentially moderated by the age of respondents (and likely resource availability), the fact that time is irreplaceable, and the tendency for people to account for time less easily than they do for money. In an impactful study, Soman (2001) found that the sunk cost effect for time was indeed weaker than for money, although this finding has not been widely replicated.
 
In the current study, Petrov et al. (2023) replicated three studies (1, 2 and 5) from Soman (2001), asking whether sunk costs are weaker for time than for money, and then testing whether the relative absence of a sunk time cost arises from the inability of participants to account for time or due to more rational beliefs in the evaluation of past time investments.
 
Results provided mixed support for the original findings. Consistent with Soman (2001), the sunk cost effect for money was reliably stronger than for time in Study 1; however, sunk costs for both money and time were comparable in Study 2 (and if anything, slightly stronger for time). The indirect replication of Study 5 from Soman (2001) found that the sunk cost effect for time was not significantly influenced by accounting for time, either using education or by highlighting opportunity costs. Robustness checks confirmed the main preregistered outcomes while also ruling out a range of potential alternative explanations. Overall, the results suggest that that sunk cost effects for time are more context-dependent and empirically volatile than sunk costs for money.
 
URL to the preregistered Stage 1 protocol: https://osf.io/65htv
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Soman, D. (2001). The mental accounting of sunk time costs: Why time is not like money. Journal of Behavioral Decision Making,14, 169-185. https://doi.org/10.1002/bdm.370
 
2. Petrov, N. B., Chan, Y. K., Lau, C. N., Kwok, T. H., Chow, L. C., Lo, W. Y. V, Song W., & Feldman, G. (2023). Sunk cost effects for time versus money: Replication and extensions Registered Report of Soman (2001), acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/q9s6p
Sunk cost effects for time versus money: Replication and extensions Registered Report of Soman (2001)Nikolay PETROV, Yin Kan (Megan) CHAN, Cheuk Nam (Chris) LAU, Tin Ho (Donald) KWOK, Lok Ching (Estelle) CHOW, Wai Yan (Vivian) LO, Wenkai SONG, Gilad FELDMAN<p>The sunk cost effect is the tendency for an individual's decision-making to be impacted by unrecoverable previous investments of resources. Soman (2001) found that sunk cost effect is weaker for time than for money (Studies 1 and 2) and that th...Social sciencesChris Chambers2023-04-18 20:57:24 View