=============================================================================
IMPORTANT ANNOUNCEMENT: To accommodate reviewer and recommender holiday schedules, we will be closed to ALL submissions from 1st Jul - 1st Sep. During this time, reviewers can submit reviews and recommenders can issue decisions, but no new or revised submissions can be made by authors.
The one exception to this rule is that authors using the scheduled track who submit their initial Stage 1 snapshot prior to 1st Jul can choose a date within the shutdown period to submit their full Stage 1 manuscript.
We recommend that authors submit at least 1-2 weeks prior to commencement of the shutdown period to enable time to make any required revisions prior to in-depth review.
=============================================================================
We are recruiting recommenders (editors) from all research fields!
Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.
Latest recommendations
| Id | Title * | Authors * | Abstract * | Picture | Thematic fields * | Recommender▼ | Reviewers | Submission date | |
|---|---|---|---|---|---|---|---|---|---|
22 Jul 2024
STAGE 1
Probing the dual-task structure of a metacontrast-masked priming paradigm with subjective visibility judgmentsDo trial-wise visibility reports - and how these reports are made - alter unconscious priming effects?Recommended by D. Samuel Schwarzkopf based on reviews by Markus Kiefer, Thomas Schmidt and 3 anonymous reviewers
Many studies of unconscious processing measure priming effects. Such experiments test whether a prime stimulus can exert an effect on speeded responses to a subsequently presented target stimulus even when participants are unaware of the prime. In some studies, participants are required to report their awareness of the prime in each trial - a dual-task design. Other studies conduct such visibility tests in separate experiments, so that the priming effect is measured via a single task. Both these approaches have pros and cons; however, it remains unclear to what extent they can affect the process of interest. Can the choice of experimental design and its parameters interfere with the priming effect? This could have implications for interpreting such effects, including in previous literature.
In the current study, Wendt and Hesselmann (2024) will investigate the effects of using a dual-task design in a masked priming paradigm, focusing on subjective visibility judgments. Based on power analysis, the study will test 34 participants performing both single-task and several dual-task conditions to measure reaction times and priming effects. Priming is tested via a speeded forced-choice identification of a target. The key manipulation is the non-speeded visibility rating of the prime using a Perceptual Awareness Scale, either with a graded (complex) rating or a dichotomous response. Moreover, participants will either provide their awareness judgement via a keyboard or vocally. Finally, participants will also complete a control condition to test prime visibility by testing the objective identification of the prime. These conditions will be presented in separate blocks, with the order randomised across participants. The authors hypothesise that using a dual-task slows down response times and boosts priming effects. However, they further posit that keyboard responses and graded visibility ratings, respectively, in the dual task reduce priming effects (but also slow response times) compared to vocal responses and dichotomous visibility judgements. In addition to the preregistered hypotheses, the study will also collect EEG data to explore the neural underpinnings of these processes.
The Stage 1 manuscript went through three rounds of review by the recommender and five expert reviewers. While the recommender would have preferred to see targeted, directional hypotheses explicitly specified in the design instead of non-directional main effects/interactions, he nevertheless considers this experimental design ready for commencing data collection, and therefore granted in-principle acceptance.
URL to the preregistered Stage 1 protocol: https://osf.io/ds2w5
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly Journals:
References Wendt, C. & Hesselmann, G. (2024). Probing the dual-task structure of a metacontrast-masked priming paradigm with subjective visibility judgments. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/ds2w5 | Probing the dual-task structure of a metacontrast-masked priming paradigm with subjective visibility judgments | Charlott Wendt, Guido Hesselmann | <p>Experiments contrasting conscious and masked stimulus processing have shaped, and continue to shape, cognitive and neurobiological theories of consciousness. However, as shown by Aru et al. (2012) the contrastive approach builds on the untenabl... | Social sciences | D. Samuel Schwarzkopf | 2024-03-02 18:20:03 | |||
17 Oct 2024
STAGE 1
Is subjective perceptual similarity metacognitive?The relationship between perceptual discriminability and subject similarityRecommended by D. Samuel Schwarzkopf based on reviews by Haiyang Jin and 1 anonymous reviewer
Much of how we use our perception involves judgements on how similar things are: You might vaguely recognise an actor's face in a movie but not immediately recognise where you have seen it before. Some people claim to be "bad with faces". Is that in fact based on objectively poorer ability to recognise faces? Psychophysical lab studies of perception typically use forced-choice discrimination tasks in which participants must make explicit, and usually binary, decisions. Such designs can yield parametric information about underlying perceptual processes, but it is very different from how we use perception in daily life.
Here, Moharramipour and colleagues (2024) seek to better understand the link between such subjective similarity jugdements and objective discrimination ability. They argue that subjective similarity can be considered a metacognitive process, reflecting the person's awareness of their perceptual capacity. Participants will be asked to discriminate face pairs across a spectrum of morphing steps in a classical forced-choice paradigm to estimate perceptual threshold performance, as well as provide subjective ratings of similarity of the face pairs. The researchers hypothesise a correlation between objective perceptual ability and subjective similarity judgements. They will evaluate this at the group level, and also use a resampling approach to determine the specificity of this relationship in individual participants. Confirming this hypothesis would advance our knowledge of how perceptual ability links with our metacognitive introspection of it. Are you really "bad with faces" or do you only think you are?
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/pzugy
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
Moharramipour, A., Zhou, W., Rahnev, D., & Lau, H. (2024). Is subjective perceptual similarity metacognitive? In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/pzugy
| Is subjective perceptual similarity metacognitive? | Ali Moharramipour, William Zhou, Dobromir Rahnev, Hakwan Lau | <p>Perceptual similarity is a cornerstone for human learning and generalization. However, in assessing the similarity between two stimuli differing in multiple dimensions, it is not well-defined which feature(s) one should focus on. The problem ha... | Life Sciences, Social sciences | D. Samuel Schwarzkopf | 2024-06-15 15:27:08 | |||
The effect of covert visual attention on pupil size during perceptual fadingDoes pupil size track high-level attention?Recommended by D. Samuel Schwarzkopf based on reviews by Sander Nieuwenhuis and Martin Rolfs
Higher-level cognitive processes like attention, memory, or mental imagery can produce pupil responses, without any actual difference in luminance of the visual stimulus. Interestingly, the inverse scenario remained untested to date: when a physical luminance difference is perceptually eliminated from awareness, does pupil size still track attention to the stimulus? In this Registered Report, Vilotijević and Mathôt (2024) sought to test this experimentally using a perceptual fading phenomenon where two Gaussian patches with different luminances fade from consciousness and are thus perceived as mid-level uniform grey (or at least the subjective difference is much reduced). This fading manipulation, as well as a control condition without perceptual fading, were presented in separate blocks. Participants were instructed to covertly attend one of the patches.
The authors hypothesised that if pupil size reflects attentional selection, these pupil responses in the fading condition should be eliminated or at least reduced, and this should evolve with time as the stimuli are perceptually fading. Their results show that pupil responses during covert attention are indeed reduced during perceptual fading - but they are not eliminated. Interestingly, this reduction did not depend on time or self-reports of the strength of perceptual fading. The findings therefore suggest that pupil dilation tracks subjective brightness differences. One inherent issue with experiments like these is that the experimental and control conditions necessarily involve a physical difference in the stimulus. Here, the fading condition had the same spatial configuration of light and dark stimuli throughout a block while in the control (non-fading) condition the light and dark stimuli alternated sides between trials. It is therefore impossible to completely rule out that the physical difference affects the results. However, the only alternative to this would be an experimental design in which the stimuli never change, but only the subjective perceptual state varies. Such a design is completely at the mercy of the participant's subjective state and therefore loses experimental control and statistical sensitivity. The present results confirmed the authors' prediction that there are indeed differences in overall pupil responses during the fading and control conditions, irrespective of covert attention.
Critically, the fact that the attention effect did not vary with time or subjective self-reports of the illusion supports the authors' interpretation that this reflects higher-level cognition: the mere act of attending to the dark side - even if the actual appearance has faded - could cause a sustained pupil dilation. This would be consistent with the type of pupil effects for memory and mental imagery that motivated the present study. However, a simpler alternative is that the experience of perceptual fading was incomplete (as possibly suggested by Figures 2D and 2F) but that self-reports fail to capture this subjective experience accurately. Perhaps a future study could compare the magnitude of the attentional pupil effects when the initial stimulus is completely removed. If similar differences in pupil response persist this would suggest that the present results are due to high-level modulation or the residual low-level luminance difference.
The Stage 2 manuscript was evaluated by two reviewers and the recommender over two rounds of review. One reviewer again advised additional robustness checks to rule out eye movement confounds, an issue they had already raised during Stage 1 review. The researchers provide clear evidence that this is unlikely to have confounded their findings. This has been added to the supplementary data repository. Following this review and revision, the recommender judged that the Stage 2 criteria were met and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/bmtp6
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Vilotijević, A. & Mathôt, S. (2024). The effect of covert visual attention on pupil size during perceptual fading [Stage 2]. Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/ku8qc?view_only=f331df53b50f431386fabba9e386b387
| The effect of covert visual attention on pupil size during perceptual fading | Ana Vilotijević, Sebastiaan Mathôt | <p>Pupil size is modulated by various cognitive factors such as attention, working memory, mental imagery, and subjective perception. Previous studies examining cognitive effects on pupil size mainly focused on inducing or enhancing a subjective e... | Social sciences | D. Samuel Schwarzkopf | 2024-08-17 12:58:58 | |||
07 May 2024
STAGE 1
Revisiting the Psychology of Waste: Replication and extensions Registered Report of Arkes (1996)When do perceptions of wastefulness affect how people make choices?Recommended by Douglas Markant based on reviews by Travis Carter and Quentin Andre
How do perceptions of wastefulness affect how people make choices? In an influential set of studies examining different conceptions of wasteful behavior (overspending, underutilization, and sunk costs), Arkes (1996) found a systematic aversion to wastefulness in decision making, even when choosing to avoid wastefulness has no economic value or works against personal interest. While these findings have been influential in basic and applied research, there have been no attempts to directly replicate the results. Moreover, the original study has several methodological limitations, including the use of relatively small samples and gaps in statistical analysis and reporting.
In this Stage 1 manuscript, Zhu and Feldman (2024) propose to conduct a high-powered replication of Arkes (1996) using an online sample of participants. The authors will incorporate several extensions to improve methodological rigor relative to the original article, including added comprehension checks, checks of the wastefulness manipulations, a within-subjects design, and a quantitative analysis of participants’ self-reported motivations for their choices. The results of the study will provide insight into the robustness of the original findings, while also better distinguishing wastefulness aversion from other potential reasons behind participants' decisions.
The Stage 1 submission was evaluated by the recommender and two expert reviewers. After two rounds of revision, the recommender determined that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/r7tsw
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Arkes, H. R. (1996). The psychology of waste. Journal of Behavioral Decision Making, 9,
213-224. https://doi.org/10.1002/(SICI)1099-0771(199609)9:3%3C213::AID-BDM230%3E3.0.CO;2-1 2. Zhu, Z. & Feldman, G. (2024). Revisiting the Psychology of Waste: Replication and extensions Registered Report of Arkes (1996). In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/r7tsw
| Revisiting the Psychology of Waste: Replication and extensions Registered Report of Arkes (1996) | Zijin Zhu, Gilad Feldman | <p>[IMPORTANT: Abstract, method, and results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection. For the purpose o... | Social sciences | Douglas Markant | 2024-01-11 06:55:16 | |||
Revisiting the Psychology of Waste: Replication and extensions Registered Report of Arkes (1996)When do perceptions of wastefulness affect how people make choices?Recommended by Douglas Markant based on reviews by Travis Carter and Quentin Andre
When does the perceived wastefulness of different actions affect people's choices? In an influential set of studies examining different conceptions of wastefulness (overspending, underutilization, and sunk costs), Arkes (1996) found a systematic aversion to wastefulness in decision making, even when choosing to avoid wastefulness has no economic value or works against personal interest. While these findings have been influential in basic and applied research, there have been no attempts to directly replicate the results. Moreover, the original study had several methodological limitations, including the use of relatively small samples and critical gaps in statistical reporting and analyses.
In the current study, Zhu and Feldman (2024) conducted a high-powered replication of Arkes (1996) using an online sample of participants (N=659). The authors incorporated several extentions to improve the methodological rigor relative to the original article, including comprehension checks, manipulation checks, a within-subjects design, and a novel quantitative analysis of participants' self-reported motivations for their choices. The authors successfully replicated the effect of perceived wastefulness on two of the three scenarios used in the original article, but participants' self-reported reasons for their choices only provided partial support for the role of perceived wastefulness in decisions, with behavioral consistency and maximizing economic value also playing a role. The original effect was not observed in a third scenario, with a failed manipulation check that may indicate changes in the perceptions of wastefulness in the domain (tax preparation). Overall, the results provide some support for the role of wastefulness aversion in decision making, while also showing that perceived wastefulness might be outweighted by other considerations depending on how people interpret or reason about a situation.
The Stage 2 manuscript was evaluated over one round of in-depth review by the recommender and two expert reviewers. Following revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/r7tsw
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Arkes, H. R. (1996). The psychology of waste. Journal of Behavioral Decision Making, 9,
213-224. https://doi.org/10.1002/(SICI)1099-0771(199609)9:3%3C213::AID-BDM230%3E3.0.CO;2-1 2. Zhu, Z. and Feldman, G. (2024). Revisiting the Psychology of Waste: Replication and extensions Registered Report of Arkes (1996) [Stage 2]. Acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/2jnc8
| Revisiting the Psychology of Waste: Replication and extensions Registered Report of Arkes (1996) | Zijin Zhu, Gilad Feldman | <p>Arkes (1996) demonstrated a phenomenon of wastefulness avoidance, showing that people’s decisions are impacted by wastefulness, making decisions that avoid appearing wasteful. In a Registered Report with a Prolific sample (N = 659), we conducte... | Social sciences | Douglas Markant | 2024-06-04 19:00:58 | |||
06 Feb 2025
STAGE 1
Disentangling the Influences of Curiosity and Active Exploration on Cognitive Map FormationInvestigating influences of curiosity and active exploration on spatial memoryRecommended by Douglas Markant based on reviews by Alexandr Ten and 2 anonymous reviewers
A growing body of research suggests that feelings of curiosity are associated with better memory for new information. However, while curiosity has been broadly defined as an instrinsic drive to obtain information, much of this existing work has focused on learning of trivia facts or factual information, settings where people have relatively little freedom to actively explore aspects of the environment that generate those feelings of curiosity. In this Stage 1 manuscript, O'Donoghue, Cen, and Gruber (2025) propose to conduct an experiment to examine whether feelings of curiosity promote the formation of cognitive maps when exploring novel spatial environments. Participants will encounter a series of virtual rooms for which they report their level of curiosity before exploring each room. The experiment will compare groups of participants who can actively explore for themselves vs. passive observers of the same exploration patterns. The proposed analyses will examine whether anticipatory curiosity leads to better memory for the spatial environments for both active or passive participants, or if benefits to memory are mediated by active participants' ability to act of their feelings of curiosity through their exploration of the environment. The findings of the study will provide new insight into the mechanisms through which anticipatory curiosity enhances memory.
The Stage 1 submission was evaluated by three expert reviewers. After three rounds of revision, the recommender determined that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/86r3x
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI RR-friendly journals:
References
1. O'Donoghue, E. M., Cen, D., & Gruber, M. (2025). Disentangling the Influences of Curiosity and Active Exploration on Cognitive Map Formation. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/86r3x
| Disentangling the Influences of Curiosity and Active Exploration on Cognitive Map Formation | Ellen M. O'Donoghue, Danlu Cen, Matthias J. Gruber | <p>Curiosity has long been assumed to promote exploration, and in turn, to support cognitive map formation. However, little research has directly investigated these claims. Recently, Cen et al. (2024) demonstrated that when participants feel more ... | Social sciences | Douglas Markant | 2024-06-28 10:30:38 | |||
09 Jul 2024
STAGE 1
Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay TaskExploring determinants of test-retest reliabilty in fMRIRecommended by Dorothy Bishop based on reviews by Xiangzhen Kong and 2 anonymous reviewers
Functional magnetic resonance imaging (fMRI) has been used to explore brain-behaviour relationships for many years, with proliferation of a wide range of sophisticated analytic procedures. However, rather scant attention has been paid to the reliability of findings. Concerns have been growing following failures to replicate findings in some fields, but it is hard to know how far this is a consequence of underpowered studies, flexible analytic pipelines, or variability within and between participants.
Demidenko et al. (2023) plan a study that will be a major step forward in addressing these issues. They take advantage of the availability of three existing datasets, the Adolescent Brain Cognitive Development (ABCD) study, the Michigan Longitudinal Study, and the Adolescent Risk Behavior Study, which all included a version of the Monetary Incentive Delay task measured in two sessions. This gives ample data for a multiverse analysis, which will consider how within-subject and between-subject variance varies according to four analytic factors: smoothing (5 levels), motion correction (6 levels), task modelling (3 levels) and task contrasts (4 levels). They will also consider how sample size affects estimates of reliability. This will involve a substantial amount of analysis.
The study is essentially focused on estimation, although specific predictions are presented regarding the combinations of factors expected to give optimal reliability. The outcome will be a multiverse of results which will allow us to see how different pipeline decisions for this task affect reliability. In many ways, null results – finding that at least some factors have little effect on reliability – would be a positive finding for the field, as it would mean that we could be more relaxed when selecting an analytic pathway. A more likely outcome, however, is that analytic decisions will affect reliability, and this information can then guide future studies and help develop best practice guidelines. As one reviewer noted, we can’t assume that results from this analysis will generalise to other tasks, but this analysis with a widely-used task is an important step towards better and more consistent methods in fMRI.
The researchers present a fully worked out plan of action, with supporting scripts that have been developed in pilot testing. The Stage 1 manuscript received in-depth evaluation from three expert reviewers and the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/nqgeh
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet sufficiently observed the key variables within the data to be able to answer the research questions AND they have taken additional steps to maximise bias control and rigour.
List of eligible PCI RR-friendly journals:
References 1. Demidenko, M. I., Mumford, J. A., & Poldrack, R. A. (2023). Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/nqgeh | Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task | Michael I. Demidenko, Jeanette A. Mumford, Russell A. Poldrack | <p>Empirical studies reporting low test-retest reliability of individual neural estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods that may improve reliability in fMR... | Life Sciences, Social sciences | Dorothy Bishop | 2023-04-17 22:27:54 | |||
19 Jan 2024
STAGE 1
A systematic review of social connection inventoriesImproving the measurement of social connectionRecommended by Dorothy Bishop based on reviews by Jacek Buczny, Richard James and Alexander Wilson
This is an ambitious systematic review that uses a combination of quantitative and qualitative methods to make the measurement of the construct of social connection more rigorous. Social connection is a heterogeneous construct that includes aspects of structure, function and quality. Here, Paris et al. (2024) will use predefined methods to create a database of social connection measures, and will assess heterogeneity of items using human coders and ChatGPT. This database will form the basis of a second systematic review which will look at evidence for validity and measurement properties. This study will also look at the population groups and country of origin for which different measures were designed, making it possible to see how far culturally specific issues affect the content of measures in this domain.
The questions asked by this study are exploratory and descriptive and so the importance of pre-registration is in achieving clear criteria for how each question is addressed, rather than evidential criteria for hypothesis-testing.
The authors responded comprehensively to three reviewer reports. This study will provide a wealth of useful information for those studying social connection, and should serve to make the literature in this field more psychometrically robust and less fragmented.
URL to the preregistered Stage 1 protocol: https://osf.io/796uv
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence. List of eligible PCI RR-friendly journals:
References
1. Paris, B., Brickau, D., Stoianova, T., Luhmann, M., Mikton, C., Holt-Lunstad, J., Maes, A., & IJzerman, H. (2024). A systematic review of social connection inventories. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/796uv | A systematic review of social connection inventories | Bastien Paris, Debora Brickau, Tetiana Stoianova, Maike Luhmann, Christopher Mikton, Julianne Holt-Lunstad, Marlies Maes, Hans IJzerman | <p>Social connection is vital to health and longevity. To date, a plethora of instruments exists to measure social connection, assessing a variety of aspects of social connection like loneliness, social isolation, or social support. For comparabil... | Social sciences | Dorothy Bishop | Jacek Buczny, Alexander Wilson, Richard James | 2023-07-09 21:33:01 | ||
Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay taskExploring determinants of test-retest reliability in fMRI: a study with the Monetary Incentive Delay TaskRecommended by Dorothy Bishop based on reviews by Xiangzhen Kong and 1 anonymous reviewer
Functional magnetic resonance imaging has been used to explore brain-behaviour relationships for many years, with proliferation of a wide range of sophisticated analytic procedures. However, rather scant attention has been paid to the reliability of findings. Concerns have been growing failures to replicate findings in some fields, but it is hard to know how far this is a consequence of underpowered studies, flexible analytic pipelines, or variability within and between participants. Demidenko et al. (2024) took advantage of the availability of three existing datasets, including the Adolescent Brain Cognitive Development (ABCD) study, the Michigan Longitudinal Study, and the Adolescent Risk Behavior Study, which all included a version of the Monetary Incentive Delay task measured in two sessions. These were entered into a multiverse analysis, which considered how within-subject and between-subject variance varies according to four analytic factors: smoothing (5 levels), motion correction (6 levels), task modelling (3 levels) and task contrasts (4 levels). They also considered how sample size affects estimates of reliability. The results have important implications for the those using fMRI with the Monetary Incentive Delay Task, and also raise questions more broadly about use of fMRI indices to study individual differences. Motion correction had relatively little impact on the ICC, and the effect size of the smoothing kernel was modest. Larger impacts on reliability were associated with choice of contrast (implicit baseline giving larger effects) and task parameterization. But perhaps the most sobering message from this analysis is that although activation maps from group data were reasonably reliable, the ICC, used as an index of reliability for individual levels of activation, was consistently low. This raises questions about the suitability of the Monetary Incentive Delay Task for studying individual differences. Another point is that reliability estimates become more stable as sample size increases; researchers may want to consider whether the trade-off between cost and gain in precision is justified for sample sizes above 250. I did a quick literature search on Web of Science: at the time of writing the search term ("Monetary Delay Task" AND fMRI) yielded 410 returns, indicating that this is a popular method in cognitive neuroscience. The detailed analyses reported here will repay study for those who are planning further research using this task. The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewer's and recommender's comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation. URL to the preregistered Stage 1 protocol: https://osf.io/nqgeh
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to IPA, but the authors certify that they had not yet sufficiently observed the key variables within the data to be able to answer the research questions and they took additional steps to maximise bias control and rigour.
List of eligible PCI RR-friendly journals:
References 1. Demidenko, M. I., Mumford, J. A., & Poldrack, R. A. (2024). Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay task [Stage 2]. Acceptance of Version 5 by Peer Community in Registered Reports. https://www.biorxiv.org/content/10.1101/2024.03.19.585755v4 | Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay task | Michael I. Demidenko, Jeanette A. Mumford, Russell A. Poldrack | <p>Empirical studies reporting low test-retest reliability of individual blood oxygen-level dependent (BOLD) signal estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods... |  | Life Sciences, Social sciences | Dorothy Bishop | 2024-03-21 02:23:30 | ||
20 Mar 2023
STAGE 1
Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird SpeciesWhat is the role of sensory perception in cognitive task performance? An improved replication of detour performance in four different bird speciesRecommended by Dieter Lukas based on reviews by 2 anonymous reviewers
The detour task, where an individual has to navigate around a see-through barrier in order to reach a goal, is one of the oldest paradigms used in animal cognition research (Kabadayi et al. 2018). While these previous tests have documented variation in the ability of animals to inhibit going straight for the visible but blocked reward, the cognitive underpinnings of this behaviour are as yet not fully understood. In the current study, Dewulf et al. (2023) propose to assess one of the specific cognitive processes that might be involved in this behaviour. Through experimental procedures, they will compare the role of signal detection in inhibitory response performance in a detour task. To assess whether variation in detection of the barrier might be linked to the ecological niche of a species, they will compare four bird species who live in different environments. Individuals from these four species were previously tested in a detour task (Regolin et al. 1994, Zucca et al. 2005), and the proposed research plan therefore also involves a partial replication of previous studies on the same issue, but improving some critical aspects.
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
URL to the preregistered Stage 1 protocol: https://osf.io/qvxgh
List of eligible PCI RR-friendly journals:
References
Dewulf, A., Garcia-Co, C., Müller, W., Madden, J.R., Martel, A., Lens, L. & Verbruggen, F. (2023). Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird Species. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/qvxgh
Kabadayi, C., Bobrowicz, K., & Osvath, M. (2018). The detour paradigm in animal cognition. Animal Cognition, 21, 21-35. https://doi.org/10.1007/s10071-017-1152-0
Regolin L, G Vallortigara, and M Zanforlin (1994). Perceptual and motivational aspects of detour behaviour in young chicks. Animal Behaviour 47, 123–131. https://doi.org/https://doi.org/10.1006/anbe.1994.1014
Zucca P, F Antonelli, and G Vallortigara (2005). Detour behaviour in three species of birds: quails (Coturnix sp.), herring gulls (Larus cachinnans) and canaries (Serinus canaria). Animal Cognition, 8, 122–128. https://doi.org/10.1007/s10071-004-0243-x
| Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird Species | Anneleen Dewulf, Clara Garcia-Co, Wendt Müller, Joah R. Madden, An Martel, Luc Lens, & Frederick Verbruggen | <p>Response inhibition, or the stopping of actions, is considered a key component of flexible and adaptive behavior. Across fields,response inhibition is often treated as a unitary cognitive mechanism. However, we propose that response inhibition ... | Social sciences | Dieter Lukas | 2022-11-30 19:25:43 |
