Please note that we will be CLOSED to ALL SUBMISSIONS from 1 December 2024 through 12 January 2025 to give our recommenders and reviewers a holiday break.
We are recruiting recommenders (editors) from all research fields!
Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.
Latest recommendations
| Id | Title * | Authors * | Abstract * | Picture | Thematic fields * | Recommender▲ | Reviewers | Submission date | |
|---|---|---|---|---|---|---|---|---|---|
15 Jan 2024
STAGE 1
Neurophysiological correlates of plasticity induced by paired associative stimulation (PAS) targeting the motor cortex: a TMS-EEG registered reportCan TMS-evoked potentials act as biomarkers of long-term potentiation or long-term depression induced by paired associative stimulation?Recommended by Grace Edwards based on reviews by Domenica Veniero, Lindsay Oberman and 1 anonymous reviewer
What are the neurophysiological correlates of paired associative stimulation (PAS) in inducing plastic changes in human motor cortex (M1)? Here, Arrigoni and colleagues (2024) will apply transcranial magnetic stimulation (TMS) to left M1 paired with electrical stimulation of the right median nerve at an ISI of 25 ms or 10 ms to induce long-term potentiation (LTP) or long-term depression (LTD), respectively. Arrigoni and colleagues (2024) will determine if these stimulation pairings effect cortical excitability using motor-evoked potentials (MEPs) and TMS-evoked potentials (TEPs). Specifically, they hypothesize PASLTP will increase the peak-to-peak amplitude of MEPs, whereas PASLTD will decrease the amplitude, replicating previous work. They will then extend these previous findings by examining TEPs. The authors anticipate modulation of the P30 and P60, which are TEPs thought to reflect local cortical excitability. They plan to account for the MEP reafference which may also mediate the P60 amplitude by stimulating at sub- and supra- motor threshold. Further, they hypothesize an increase of the N100, a marker of inhibitory processing mediated by GABA, by PASLTD. Finally, the authors will also examine the impact of cortical excitability over time to determine the duration of the PAS effects.
This detailed examination of TEPs following PAS stimulation will determine which TEPs could be used as biomarkers with the induction of LTP and LTD through stimulation. The authors have built in an MEP replication for the PAS stimulation, supporting previous literature and acting as a positive control.
The Stage 1 manuscript was evaluated by three expert reviewers across two rounds. Following in-depth review and responses from the authors, the recommender determined that Stage 1 criteria was met and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/detjc (under temporary private embargo)
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References 1. Arrigoni, E., Bolognini, N., Pisoni, A. & Guidali, G. (2024). Neurophysiological correlates of plasticity induced by paired associative stimulation (PAS) targeting the motor cortex: a TMS-EEG registered report. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/detjc | Neurophysiological correlates of plasticity induced by paired associative stimulation (PAS) targeting the motor cortex: a TMS-EEG registered report | Eleonora Arrigoni, Nadia Bolognini, Alberto Pisoni, Giacomo Guidali | <p>Paired associative stimulation (PAS) can induce long-term potentiation (LTP) and long-term depression (LTD) in the human motor system by applying transcranial magnetic stimulation (TMS) pulses on the primary motor cortex (M1) paired with electr... | Life Sciences | Grace Edwards | 2023-07-22 10:25:21 | |||
25 Oct 2024
STAGE 1
They look virtually the same: extraretinal representation of symmetry in virtual realityHow does virtual reality impact the processing of extraretinal symmetry?Recommended by Grace Edwards based on reviews by Daniel Baker, Felix Klotzsche and 1 anonymous reviewer
Karakashevska and colleagues (2024) aim to examine the extraretinal representation of visual symmetry presented in a virtual reality environment. Previous research had demonstrated that individuals can detect symmetry when the symmetry is represented on a perspective plane, slanted away from the viewer. In electroencephalography (EEG), perceived symmetry is marked by an Event Related Potential (ERP) called a Sustained Posterior Negativity (SPN). When symmetry is presented on a perspective plane in comparison to front-on (frontoparallel), the SPN is reduced, termed the perspective cost. Here, Karakashevska et al., (2024) will determine if presenting symmetry on a perspective plane in a virtual reality (VR) environment will reduce the perspective cost with the addition of 3D depth cues. Specifically, participants will be requested to detect symmetry or luminance of a stimulus presented in a VR environment whilst wearing an EEG. The authors hypothesize that no perspective cost will be identified between symmetry presented on a frontoparallel plane versus symmetry on a perspective plane. Furthermore, the authors will examine the impact of task within the virtual environment on symmetry processing. They hypothesize that a task focused on the regularity of the stimuli will result in a larger amplitude of the SPN than a luminance task. This design enables the authors to pinpoint immersive environments as providing cues critical in overcoming perspective cost.
The Stage 1 manuscript was evaluated by two expert reviewers across three rounds. Following in-depth review and responses from the authors, the recommender has determined that Stage 1 criteria was met and has awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/7pnxu
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
Karakashevska, E., Batterley, M. & Makin, A. D. J. (2024). They look virtually the same: extraretinal representation of symmetry in virtual reality. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/7pnxu
| They look virtually the same: extraretinal representation of symmetry in virtual reality | Elena Karakashevska, Michael Batterley and Alexis D.J. Makin | <p>The brain must identify objects from different viewpoints that change the retinal image. This study will determine the conditions under which the brain spends computational resources to construct view-invariant, extraretinal representations in ... | Life Sciences | Grace Edwards | Felix Klotzsche | 2024-05-23 20:13:42 | ||
Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation?Understanding how visual cues influence extraretinal representation of planar symmetryRecommended by Grace Edwards and Zoltan Dienes based on reviews by Guillaume Rousselet and 2 anonymous reviewers
Visual symmetry is critical to our interaction with our environment so that when detected, symmetry automatically produces a neural marker in the form of an Event Related Potential (ERP) called Sustained Posterior Negativity (SPN). However, when symmetry is presented to the visual system slanted away from the viewer, there is a reduction in SPN, termed a perspective cost.
Considering objects are rarely presented front-on (or frontoparallel) in our natural environment, Karakashevska et al., (2023) examined the extent of the perspective cost with the addition of visual cues to facilitate extraretinal representation of the visual symmetry. The authors recorded electroencephalography (EEG) from 120 participants while they performed a luminance task on symmetrical and asymmetrical stimuli. The authors hypothesized that perspective cost would be reduced by three perspective cues: 1) monocular viewing, eliminating binocular cue conflict, 2) a static frame surrounding the symmetrical stimulus, adding a depth cue, and 3) a moving frame, providing a structure-from-motion 3D cue, prior to the symmetry onset. If the SPN was equivalent during frontoparallel and slanted presentation in a cue condition, the authors would have concluded extraretinal representation can be automatic when sufficient visual cues are available. The experiment was powered to detect a relatively small difference between perspective cue conditions.
The authors found that there was no impact of different visual cues on the perspective cost, as measured using the SPN. Perspective cost was consistent across all conditions, contrary to the pre-registered hypotheses. Karakashevska and colleagues conclude that the three perceptual cues tested in their design do not reduce perspective cost. The study prompts future research into the nature of the extraretinal representations of planar symmetry.
The Stage 2 manuscript was evaluated over four rounds by three expert reviewers. Two of our reviewers reviewed the Stage 1 manuscript, and one new reviewer. Following in-depth review and responses from the authors, the recommenders determined that the Stage 2 criteria were met and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/yzsq5
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. List of eligible PCI-RR-friendly journals:
References 1. Karakashevska, E., Bertamini, M. & Makin, A. D. J. (2024). Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation? [Stage 2]. Acceptance of Version 5 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/z9c28 | Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation? | Elena Karakashevska, Marco Bertamini and Alexis D.J. Makin | <p>Abstract</p> <p>Introduction: Objects often project different images when viewed from different locations. Our visual system can correct for perspective distortion and identify objects from different viewpoints that change the retinal image. T... | Life Sciences | Grace Edwards | 2024-06-03 21:00:08 | |||
17 Jun 2024
STAGE 1
Loneliness in the Brain: Distinguishing Between Hypersensitivity and HyperalertnessA new look at loneliness by testing hyperalternessRecommended by Hedwig Eisenbarth and D. Samuel Schwarzkopf based on reviews by Marta Andreatta and 1 anonymous reviewer
Do people who are more alert towards social stimuli vary in loneliness? This report addresses the question how loneliness relates to hypersensitivity to social stimuli using an oddball paradigm. Based on preliminary results, the study plans to compare high and low lonely individuals for how they react to happy and angry facial expressions using neurophysiological correlates. Findings from the study will provide further insights in how loneliness might be related to processing of social information.
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' and recommender's comments, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/fxngv Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals: References
1. Bathelt, J., Dijk, C., & Otten, M. (2024). Loneliness in the Brain: Distinguishing Between Hypersensitivity and Hyperalertness. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/fxngv | Loneliness in the Brain: Distinguishing Between Hypersensitivity and Hyperalertness | Joe Bathelt, Corine Dijk, Marte Otten | <p>Introduction: Loneliness has emerged as a pressing public health issue, necessitating greater understanding of its mechanisms to devise effective treatments. While the link between loneliness and biased social cognition is a commonly proposed, ... | Social sciences | Hedwig Eisenbarth | Anonymous, Marta Andreatta | 2023-11-20 16:34:04 | ||
19 Jun 2024
STAGE 1
Culture-Driven Neural Plasticity and Imprints of Body-Movement Pace on Musical Rhythm ProcessingThe interplay of music, movement, and culture on rhythm processingRecommended by Juan David Leongómez based on reviews by Anne Keitel and 1 anonymous reviewer
The interplay of music, movement, and cultural experience shapes rhythm perception. From bouncing babies and children at play, to tapping, clapping, and dancing, music often triggers synchronous body movements that can influence how we process rhythm. And, at the same time, long-term exposure to specific musical traditions shapes how we perceive and interpret rhythms.
However, direct behavioural and neuroscientific evidence on how these processes occur remains scarce. In this programmatic submission, comprising two complementary Stage 2 reports, Guérin et al. (2024) will investigate how body movements shape the processing of auditory information, and how previous short-term motor practice and long-term cultural experience interact to shape neural and behavioural responses to rhythmic stimuli. The authors will record in separate sessions both electroencephalography (EEG) and hand clapping, in response to rhythms from West/Central Africa. These recordings will be conducted before and after a session in which participants will clap/step to either a three or a four-beat metre that is expected to influence how they interpret a rhythm.
The first Stage 2 report, which will be conducted on African-enculturated participants, aims to demonstrate how body-movement pace flexibly imprints on human sensory processing. The authors build on various theoretical models that emphasise the role of motor production in metre perception, and predict that both neural and behavioural entrainment will improve following movement, matching the rhythmic pattern set by the previous body movements.
The second Stage 2 report aims to uncover how short-term motor practice and long-term cultural experience interact to shape responses to rhythmic stimuli, by integrating short-term motor practice and long-term cultural experience. For this, the authors will test separate groups of participants from distinct cultural backgrounds (African vs. Western-enculturated), which are predicted to show neural and behavioural differences in their preferred metric mapping before body movement, and expect that neural and behavioural entrainment will improve after movement, especially for the metre set by prior movements, and more significantly for the metre common in the participant's culture.
Together, the findings from the planned Stage 2 reports are expected to clarify how long-term cultural background and short-term motor practice imprint onto rhythm processing in humans. This research will enhance our understanding of how cultural experience, body movement, and neural plasticity interact in music processing.
The Stage 1 manuscript was evaluated over three rounds of in-depth review. Based on detailed responses to the reviewers' and recommender's comments, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/skuyc
Level of bias control achieved: Level 6. Data collection commenced during the later part of Stage 1 peer review; however, since no changes to the design were made after this point, the risk of bias due to prior data observation remains zero and the manuscript therefore qualifies for Level 6. List of eligible PCI RR-friendly journals: References
Guérin, S. M. R., Coulon, E., Lenc, T., Polak, R., Keller, P. E., & Nozaradan, S. (2024). Culture-Driven Neural Plasticity and Imprints of Body-Movement Pace on Musical Rhythm Processing. In principle acceptance of Version 2.1 by Peer Community in Registered Reports. https://osf.io/skuyc
| Culture-Driven Neural Plasticity and Imprints of Body-Movement Pace on Musical Rhythm Processing | Ségolène M. R. Guérin, Emmanuel Coulon, Tomas Lenc, Rainer Polak, Peter E. Keller, Sylvie Nozaradan | <p>The proposed programmatic registered report aims at capturing direct neuroscientific evidence for the rhythmic, movement-related shaping of auditory information with a cross-cultural perspective. Specifically, West/Central African- and Western-... | | Life Sciences | Juan David Leongómez | Anonymous | 2023-11-30 11:36:06 | |
13 May 2023
STAGE 1
Personal factors and group creativity characteristics: A correlational meta-analysisWhich personal factors are associated with group creativity?Recommended by Julia M. Rohrer and Malte Elson based on reviews by Evan Carter and Greg Feist
What determines whether groups of people can come up with ideas that are both original and useful? Since the 1960s, this question has been intensively studied with the help of more or less structured group creativity activities such as brainstorming or creative problem solving, with subsequent rating of the generated ideas.
In this line of research, personal factors—such as personality traits, and other interindividual differences in emotion and cognition—have received substantial attention as potential correlates of creative outcomes of group activities. This has spawned a sprawling literature that, to date, has not yet been synthesized. Thus, empirical findings in this literature, which are also sometimes contradictory, have not yet been well-integrated.
In the present study, Fillon et al. (2022) will conduct the first meta-analysis of correlations between personal factors and group creativity outcomes. The authors will search and synthesize the existing (published and unpublished) literature according to predetermined criteria to (1) assess the overall relationship between a broad list of personal factors and creativity outcomes in group settings and (2) explore potential moderators of these relationships. The latter research question includes substantive moderators, such as familiarity between group members, group size, and type of task, but also publication status.
The Stage 1 manuscript was evaluated over five rounds of in depth-review. Based on detailed responses to reviewers’ and the recommenders’ comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.
URL to the preregistered Stage 1 protocol: https://osf.io/nybg6
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence. List of eligible PCI RR-friendly journals:
References 1. Fillon, A. A., Girandola, F., Bonnardel, N., Kenworth, J. B., Wiernik. B. M. & Souchet, L. (2022). Personal factors and group creativity characteristics: A correlational meta-analysis, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/nybg6 | Personal factors and group creativity characteristics: A correlational meta-analysis | Adrien A. Fillon, Fabien Girandola, Nathalie Bonnardel, Jared B. Kenworthy, Brenton M. Wiernik & Lionel Souchet | <p>Previous research has investigated the relationship between personal factors and group creativity outcomes. On one side, they showed that individual differences play a role in group creativity. On the other side, they showed that group creativi... | Social sciences | Julia M. Rohrer | 2021-08-03 16:32:23 | |||
Personal factors and group creativity characteristics: A correlational meta-analysisWhich personal factors are associated with group creativity?Recommended by Julia M. Rohrer
What determines whether groups of people can come up with ideas that are both original and useful? Since the 1960s, this question has been intensively studied with the help of more or less structured group creativity activities such as brainstorming or creative problem solving, with subsequent rating of the generated ideas. In this line of research, personal factors—such as personality traits, and other interindividual differences in emotion and cognition—have received substantial attention as potential correlates of creative outcomes of group activities. This has spawned a sprawling literature that, to date, has not yet been synthesized. Thus, empirical findings in this literature, which are also sometimes contradictory, have not yet been well-integrated. In the present study, Fillon et al. (2022) conducted the first meta-analysis of correlations between personal factors and group creativity outcomes. The authors searched and synthesized the existing published literature according to predetermined criteria to (1) assess the overall relationship between a broad list of personal factors and creativity outcomes in group settings and (2) explore potential moderators of these relationships. In total, 11 studies could be included in the meta-analysis. They provided weak support for a positive correlation between self-efficacy and the three investigated group creative outcomes, number of ideas, originality of ideas, and usefulness of ideas. With respect to moderators, many of the planned analyses could not be conducted due to the low number of studies. The only finding that arose was weak evidence for the idea that time constraint moderates associations: relationships between personal factors and group creativity outcomes were slightly stronger for tasks limited to 20 minutes rather than 10 minutes. Statistical power overall was low across studies. The authors conclude their meta-analysis with the observation that the available data on the topic are very limited. They suggest that to improve our knowledge of the topic, future studies should adhere to standardized creativity methods and protocols and implement expert ratings of creativity. They also call for increasing the availability of raw data in this field of study to improve the accumulation of knowledge about links between personal factors and the creative performance of groups. The Stage 2 manuscript was evaluated over one round of in-depth review provided by the recommender and Chris Chambers, as the original reviewers were no longer available. Based on additional changes to the manuscript, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation. URL to the preregistered Stage 1 protocol: https://osf.io/nybg6
Level of bias control achieved: Level 3. At least some data/evidence that was used to answer the research question had been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certified that had not yet observed ANY part of the data/evidence prior to IPA. List of eligible PCI RR-friendly journals:
References 1. Fillon, A. A., Girandola, F., Bonnardel, N., Kenworth, J. B. & Souchet, L. (2023). Personal factors and group creativity characteristics: A correlational meta-analysis, acceptance of Version 2 by Peer Community in Registered Reports. https://psyarxiv.com/4br6a/ | Personal factors and group creativity characteristics: A correlational meta-analysis | Adrien Alejandro Fillon; Fabien Girandola, Nathalie Bonnardel, Jared Kenworthy, Lionel Souchet | <p>Previous research has investigated the relationship between personal factors and group creativity outcomes. On one side, they indicated that individual differences play a role in group creativity. On the other side, they showed that group creat... | Social sciences | Julia M. Rohrer | 2023-02-21 12:41:02 | |||
19 Apr 2022
STAGE 1
Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of durationCould asymmetrical perceptions about the frequency of past and future events explain the Temporal Doppler Effect?Recommended by Ljerka Ostojic based on reviews by Chris Chambers and 2 anonymous reviewers
The Temporal Doppler Effect is a phenomenon where people subjectively perceive the past to be further away than the future even when both temporal distances are objectively the same from the present moment (Caruso et al., 2013). A common explanation for this phenomenon assumes that our perception of the past and future is based on spatial and temporal analogies (Matlock, Ramscar, & Boroditsky, 2005; Casanto & Boroditsky, 2008) and that the subjective discrepancy is due to people feeling that they are moving towards the future and away from the fast, thus underestimating the temporal distance of the former and overestimating the temporal distance of the latter (Caruso et al., 2013).
In the current study, Zhang et al. propose to replicate the Temporal Doppler Effect as tested by Caruso et al. (2013) in study 1 and to test an alternative explanation for the effect in study 2 based on the filled-duration illusion (Thomas & Brown, 1974). This alternative explanation assumes that the subjective discrepancy is based on the difference that the past and the future are filled with events that we can remember or imagine. Because the past has already happened, it is comprised of more events (those that were planned and those that were not), while the future still exists only of events that are currently planned.
The Stage 1 manuscript was evaluated over three rounds of in-depth review. The main changes during the review process involved clarifications and adaptations of the way that the authors will measure the participants’ perception of how full of events the past and the future are, as the originally proposed way measuring this did not have sufficient theoretical or empirical justifications. The authors decided to address this by firstly, clarifying this issue in the stage 1 report so that the reader is aware of the potential shortcomings of this measure, and secondly, by testing a second group of participants with an alternative measure. Based on detailed responses to the reviewers’ comments and edits to the stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/d9ec3
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI RR-friendly journals:
References
1. Caruso, E. M., Van Boven, L., Chin, M., & Ward, A. (2013). The temporal doppler effect: When the future feels closer than the past. Psychological Science, 24, 530-536. https://doi.org/10.1177/0956797612458804
2. Casanto, D., & Boroditsky, L. (2008). Time in the mind: Using space to think about time. Cognition, 106, 579-593. https://doi.org/10.1016/j.cognition.2007.03.004
3. Matlock, T., Ramscar, M., & Boroditsky, L. (2005). One the experiential link between spatial and temporal language. Cognitive Science, 29, 655-664. https://doi.org/10.1207/s15516709cog0000_17
4. Thomas, E. C., & Brown, I. (1974). Time perception and the filled-duration illusion. Perception & Psychophysics, 16, 449-458. https://doi.org/10.3758/BF03198571
5. Zhang, Q., Masuda, Y., Ueda, K.,Toda, K., & Yamada, Y. (2022). Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of duration. Stage 1 Registered Report, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/d9ec3
| Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of duration | Qinjing Zhang, Yoshitaka Masuda, Kohei Ueda, Kodai Toda, Yuki Yamada | <p>People sometimes feel events to be psychologically closer while farther at other times. Caruso et al. (2013) reported the Temporal Doppler Effect (TDE) in which people feel that the past is farther than the future, despite an equivalent objecti... | Social sciences | Ljerka Ostojic | 2021-06-14 16:04:02 | |||
16 Sep 2022
STAGE 1
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportUnderstanding the key ingredients of the Bayesian Truth SerumRecommended by Ljerka Ostojic based on reviews by 2 anonymous reviewers
The Bayesian Truth Serum, first introduced by Prelec (2004) rewards participants based on how surprisingly common their own answers are in relation to the actual distribution of answers. As such, it has been suggested as a possible incentive-compatible design for survey studies in different disciplines that rely on participants’ self-reports about their true preferences (Schoenegger, 2021).
In this study, Schoenegger and Verheyen propose to replicate the results reported by Schoenegger (2021) and to directly investigate whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts.
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers’ comments and edits to the stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI RR-friendly journals:
References
1. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
2. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4
3. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 1 Registered Report, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/dkvms
| Taking A Closer Look At The Bayesian Truth Serum: A Registered Report | Philipp Schoenegger & Steven Verheyen | <p>Over the past decades, psychology and its cognate disciplines have undergone substantial reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has received comp... | Social sciences | Ljerka Ostojic | 2021-12-06 17:36:15 | |||
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportReassessing the use of the Bayesian Truth Serum as an incentive-compatible design for self-reportsRecommended by Ljerka Ostojic based on reviews by Joël van der Weele
Different disciplines and research areas that rely on participants’ self-reports to accrue data on participants’ true preferences are faced with the question to what extent the former can be equated with the latter. Using monetary incentivisation for study participation may influence this relationship, and researchers, especially in economics, have been discussing how to develop and implement incentive-compatible research designs, i.e., those in which the incentivisation yields the best payoff for the participant if they report their true preferences (Hertwig & Ortmann, 2011; Baillon, 2017). The Bayesian Truth Serum, first introduced by Prelec (2004), according to which participants are rewarded based on how surprisingly common their own answers are relative to the actual distribution of answers, has been proposed as a possible incentive-compatible design for survey studies that rely on participants’ self-reports about their true preferences (Schoenegger, 2021).
In this study, Schoenegger and Verheyen (2022) ran a replication of the study by Schoenegger (2021) and assessed whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts. The authors report that the manipulation did not yield a significant difference compared to control conditions, which they interpret as a failure to replicate the original results. At the same time, the authors are careful in drawing conclusions as to the usefulness of the Bayesian Truth Serum for self-report studies using Likert-scale items in general, as they emphasise that smaller effect sizes may be of interest and that the results may differ when different items are used.
The Stage 2 manuscript was evaluated by two reviewers, one of whom reviewed the first Stage 1 submission, and the other one of whom reviewed the manuscript specifically to assess statistical questions.
Following a careful revision by the authors, the recommender judged that the manuscript meets the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance.
List of eligible PCI RR-friendly journals:
References
1. Baillon, A. (2017). Bayesian markets to elicit private information. Proceedings of the National Academy of Science, 114(30), 7985-7962. https://doi.org/10.1073/pnas.1703486114
2. Hertwig, R. & Ortmann (2001). Experimental practices in economics: a methodlogical challenge for psychologists? Behavioral and Brain Sciences, 24(3), 383-403. https://doi.org/10.1037/e683322011-032
3. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
4. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4
5. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 2 Registered Report, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://doi.org/10.31219/osf.io/9zvqj
| Taking A Closer Look At The Bayesian Truth Serum: A Registered Report | Philipp Schoenegger & Steven Verheyen | <p>Over the past decade, psychology and its cognate disciplines have undergone substantial scientific reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has rec... | Social sciences | Ljerka Ostojic | 2022-06-11 14:39:38 |
