Submit a report

Announcements

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.


 

Latest recommendationsrssmastodon

IdTitleAuthorsAbstractPictureThematic fieldsRecommender▲ReviewersSubmission date
25 Oct 2023
STAGE 1
toto

Does pupillometry provide a valid measure of spatial attentional bias (pseudoneglect)?

Assessing visuospatial biases (pseudoneglect) using pupillometry: A replication and extension of Strauch et al. (2022)

Recommended by ORCID_LOGO based on reviews by Christoph Strauch and Bianca Hatin
‘Pseudoneglect’ is a small, lateralised bias of visuospatial attention towards the left side of space, and is typically observed in healthy adults. Recently, Strauch et al. (2022) reported that bright stimuli presented in the left visual field induced a greater constriction of the pupil (the pupillary light reflex) compared to the same bright stimuli presented in the right visual field. Further, the pupillary restriction bias was positively correlated with a behavioural measure of pseudoneglect (the greyscales task). This is potentially an important development for attention research, because the passive nature of the task, in addition to the ability to track the time course of the bias measures, could provide a new, and highly sensitive, method of studying spatial attention.
 
In this report, Burns and McIntosh (2023) aim to replicate and extend the study of Strauch et al. (2022). The extension centres around investigating whether the pupillary biases are influenced by recording pupillary responses from the right or left eye. In their pilot replication data, Burns & McIntosh identified a larger constriction in response to stimuli on the right side when recording from the right eye. They hypothesise that pupillary biases may be stronger to stimuli presented in the ipsilateral, rather than contralateral, side of space.
 
The Stage 1 manuscript was reviewed over 2 rounds by 2 reviewers, including the authors of the study being replicated. Based on detailed responses to the reviewers’ comments and edits to the Stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/ua9jn

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI-RR-friendly journals:
 
References
 
1. Strauch, C., Romein, C., Naber, M., Van der Stigchel, S. & Ten Brink, A. F. (2022). The orienting response drives pseudoneglect—Evidence from an objective pupillometric method. Cortex, 151, 259-271.
 
2. Burns, N. E. & McIntosh, R. D. (2023). Does pupillometry provide a valid measure of spatial attentional bias (pseudoneglect)? In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/ua9jn
Does pupillometry provide a valid measure of spatial attentional bias (pseudoneglect)?Nicola E. Burns, and Robert D. McIntosh<p>Strauch et al. (2022) introduced a novel approach to assess biases of visual attention, by measuring pupillary constriction in response to split-field stimuli, in which a bright patch is presented to one visual field and a dark patch to the oth...Social sciencesGemma Learmonth Christoph Strauch2023-07-12 18:27:28 View
10 Apr 2024
STAGE 2
(Go to stage 1)
toto

Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? A Registered Report

Understanding oscillatory correlates of pain expectation

Recommended by ORCID_LOGO based on reviews by Zoltan Dienes, Chris Chambers and Markus Ploner
Recent studies using an EEG frequency tagging approach have reported modulations of alpha, beta and theta bands at the stimulation frequency during nociceptive/painful thermal stimulation compared to non-nociceptive/non-painful vibrotactile stimulation. Prior expectations of the intensity of upcoming painful stimuli are known to strongly modulate the subjective experience of those stimuli. Thus, modulating the expectation of pain should result in a change in the modulation of oscillations if these factors are indeed linked.
 
In this study, Leu, Glineur and Liberati modulated expectations of pain (low or high intensity) in 40 participants prior to delivering thermal cutaneous stimulation (low, medium or high intensity). They recorded how intense participants expected the pain to be, and how intense they felt it to be, as well as EEG to assess oscillatory differences across the expectation and intensity conditions.
 
The results confirmed that there was a strong effect of expectation on the perceived stimulus intensity. However, contrary to the hypotheses, this was not reflected in the cortical oscillations. Overall this indicates a possible dissociation between perceived pain and modulation of ongoing oscillations in the theta, alpha and beta bands. 
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/y6fb8
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Leu, C., Glineur, E. & Liberati, G. (2023). Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? [Stage 2] Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/awrge
Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? A Registered ReportChiara Leu, Esther Glineur, Giulia Liberati<p style="text-align: justify;">A promising stream of investigations is targeting ongoing neural oscillations and whether their modulation could be related to the perception of pain. Using an electroencephalography (EEG) frequency tagging approach...Life Sciences, Medical SciencesGemma Learmonth 2024-01-23 19:35:39 View
27 Nov 2023
STAGE 1
toto

Cortical voice processing in Autism Spectrum Disorder

Is voice processing impacted in Autism Spectrum Disorder?

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by 2 anonymous reviewers
Vocal sounds, including both speech and non-speech sounds, have been found to activate the Superior Temporal Sulci and Gyri in comparison to non-vocal sounds. These regions, termed Temporal Voice Areas (TVAs), are considered to be involved in early voice processing and therefore critical for social interaction. TVA activation has been examined in Autism Spectrum Disorder (ASD) to determine if the characteristic difficulties in social communication and interaction are linked to an impaired early voice processing. Using functional magnetic resonance imaging (fMRI), one study found typical brain activation in TVAs for 15 out of 16 autistic participants (Schelinski et al., 2016), whereas another found atypical activation in 4 out of 5 autistic participants (Gervais et al., 2004).
 
Here, the inconsistencies in the previous literature propel Gautier et al. (2023) to examine brain activation of TVAs with a larger sample size (26 ASD and 26 non-ASD participants). Gautier et al. (2023) will present vocal sounds and non-vocal sounds to both groups of participants during fMRI and predict that fewer participants in the ASD group will show a preferential response to voices in TVAs compared to the non-ASD group. These results would suggest that symptoms of ASD interfere with early stages of social interaction, at the level of voice processing.
 
This Stage 1 manuscript was evaluated in an initial round by the co-recommenders and another two rounds of in-depth review by two expert reviewers. With these revisions, the recommenders judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/538m4
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Gautier, R., Houy-Durand, E., Barantin, L., Briend, F. & Latinus, M. (2023). Cortical voice processing in Autism Spectrum Disorder. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/538m4
Cortical voice processing in Autism Spectrum DisorderRaphaël Gautier, Emmanuelle Houy-Durand, Laurent Barantin, Frederic Briend, Marianne Latinus<p>Voice processing is central to social functioning. A specific brain response to vocal sounds has been described and extensively characterized in the general population but remains critically unexplored in Autism Spectrum Disorder (ASD), a condi...Life SciencesGrace Edwards2023-03-28 09:51:48 View
18 Oct 2023
STAGE 1
toto

Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation?

Understanding how visual cues influence extraretinal representation of planar symmetry

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by Tadamasa Sawada, Guillaume Rousselet, Benoit Cottereau and Deborah Apthorp
Visual symmetry is critical to our interaction with our environment so that when detected, symmetry automatically produces a neural marker in the form of an Event Related Potential (ERP) called Sustained Posterior Negativity (SPN). However, when symmetry is presented to the visual system slanted away from the viewer, there is a reduction in SPN, termed a perspective cost. 
 
Considering ​objects are rarely presented front-on (or frontoparallel) in our natural environment, Karakashevska et al., (2023) plan to examine the extent of the perspective cost with the addition of visual cues to facilitate extraretinal representation of the visual symmetry. The authors will record electroencephalography (EEG) from 120 participants while they perform a luminance task on symmetrical and asymmetrical stimuli. The authors hypothesize perspective cost will be reduced by three perspective cues: 1) monocular viewing, when cue conflict caused by binocular viewing is eliminated, 2) a static frame surrounding the symmetrical stimulus, adding a depth cue, and 3) a moving frame, assisting 3D perception prior to the symmetry onset. If the SPN is equivalent during frontoparallel and slanted presentation in a cue condition, the authors will conclude extraretinal representation can be automatic when sufficient visual cues are available. The proposed experiment is powered to detect a relatively small difference between perspective cue conditions. This will solidify fundamental research on visual symmetry processing and will further our understanding of object perception and recognition. 
 
The Stage 1 manuscript was evaluated over three rounds by four expert reviewers. Following in-depth review and responses from the authors, the recommenders have determined that Stage 1 criteria was met and have awarded in-principle acceptance (IPA). 
​​​​
URL to the preregistered Stage 1 protocol: https://osf.io/yzsq5
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI-RR-friendly journals:
 
 
References
 
1. Karakashevska, E., Bertamini, M. & Makin, A. D. J. (2023). Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation? [Stage 1 Registered Report]. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/yzsq5
Putting things into perspective: Which visual cues facilitate automatic extraretinal symmetry representation?Elena Karakashevska, Marco Bertamini and Alexis D.J. Makin <p>A challenge for the visual brain is to identify objects from a range of different viewpoints. This study will determine the conditions under which the brain spends computational resources to achieve view-invariance. We focus on view-invariant r...Life SciencesGrace Edwards2023-04-17 21:52:26 View
15 Jan 2024
STAGE 1
toto

Neurophysiological correlates of plasticity induced by paired associative stimulation (PAS) targeting the motor cortex: a TMS-EEG registered report

Can TMS-evoked potentials act as biomarkers of long-term potentiation or long-term depression induced by paired associative stimulation?

Recommended by ORCID_LOGO based on reviews by Domenica Veniero, Lindsay Oberman and 1 anonymous reviewer
What are the neurophysiological correlates of paired associative stimulation (PAS) in inducing plastic changes in human motor cortex (M1)? Here, Arrigoni and colleagues (2024) will apply transcranial magnetic stimulation (TMS) to left M1 paired with electrical stimulation of the right median nerve at an ISI of 25 ms or 10 ms to induce long-term potentiation (LTP) or long-term depression (LTD), respectively. Arrigoni and colleagues (2024) will determine if these stimulation pairings effect cortical excitability using motor-evoked potentials (MEPs) and TMS-evoked potentials (TEPs). Specifically, they hypothesize PASLTP will increase the peak-to-peak amplitude of MEPs, whereas PASLTD will decrease the amplitude, replicating previous work. They will then extend these previous findings by examining TEPs. The authors anticipate modulation of the P30 and P60, which are TEPs thought to reflect local cortical excitability. They plan to account for the MEP reafference which may also mediate the P60 amplitude by stimulating at sub- and supra- motor threshold. Further, they hypothesize an increase of the N100, a marker of inhibitory processing mediated by GABA, by PASLTD. Finally, the authors will also examine the impact of cortical excitability over time to determine the duration of the PAS effects.   
 
This detailed examination of TEPs following PAS stimulation will determine which TEPs could be used as biomarkers with the induction of LTP and LTD through stimulation. The authors have built in an MEP replication for the PAS stimulation, supporting previous literature and acting as a positive control.  
 
The Stage 1 manuscript was evaluated by three expert reviewers across two rounds. Following in-depth review and responses from the authors, the recommender determined that Stage 1 criteria was met and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/detjc (under temporary private embargo)
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 

References
 
1. Arrigoni, E., Bolognini, N., Pisoni, A. & Guidali, G. (2024). Neurophysiological correlates of plasticity induced by paired associative stimulation (PAS) targeting the motor cortex: a TMS-EEG registered report. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/detjc
Neurophysiological correlates of plasticity induced by paired associative stimulation (PAS) targeting the motor cortex: a TMS-EEG registered reportEleonora Arrigoni, Nadia Bolognini, Alberto Pisoni, Giacomo Guidali<p>Paired associative stimulation (PAS) can induce long-term potentiation (LTP) and long-term depression (LTD) in the human motor system by applying transcranial magnetic stimulation (TMS) pulses on the primary motor cortex (M1) paired with electr...Life SciencesGrace Edwards2023-07-22 10:25:21 View
13 May 2023
STAGE 1
toto

Personal factors and group creativity characteristics: A correlational meta-analysis

Which personal factors are associated with group creativity?

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by Evan Carter and Greg Feist
What determines whether groups of people can come up with ideas that are both original and useful? Since the 1960s, this question has been intensively studied with the help of more or less structured group creativity activities such as brainstorming or creative problem solving, with subsequent rating of the generated ideas.
 
In this line of research, personal factors—such as personality traits, and other interindividual differences in emotion and cognition—have received substantial attention as potential correlates of creative outcomes of group activities. This has spawned a sprawling literature that, to date, has not yet been synthesized. Thus, empirical findings in this literature, which are also sometimes contradictory, have not yet been well-integrated.
 
In the present study, Fillon et al. (2022) will conduct the first meta-analysis of correlations between personal factors and group creativity outcomes. The authors will search and synthesize the existing (published and unpublished) literature according to predetermined criteria to (1) assess the overall relationship between a broad list of personal factors and creativity outcomes in group settings and (2) explore potential moderators of these relationships. The latter research question includes substantive moderators, such as familiarity between group members, group size, and type of task, but also publication status.
 
The Stage 1 manuscript was evaluated over five rounds of in depth-review. Based on detailed responses to reviewers’ and the recommenders’ comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/nybg6
 
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence. 
 
List of eligible PCI RR-friendly journals:
 

 

References

1. Fillon, A. A., Girandola, F., Bonnardel, N., Kenworth, J. B., Wiernik. B. M. & Souchet, L. (2022). Personal factors and group creativity characteristics: A correlational meta-analysis, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/nybg6

Personal factors and group creativity characteristics: A correlational meta-analysisAdrien A. Fillon, Fabien Girandola, Nathalie Bonnardel, Jared B. Kenworthy, Brenton M. Wiernik & Lionel Souchet<p>Previous research has investigated the relationship between personal factors and group creativity outcomes. On one side, they showed that individual differences play a role in group creativity. On the other side, they showed that group creativi...Social sciencesJulia M. Rohrer2021-08-03 16:32:23 View
13 May 2023
STAGE 2
(Go to stage 1)
toto

Personal factors and group creativity characteristics: A correlational meta-analysis

Which personal factors are associated with group creativity?

Recommended by ORCID_LOGO

What determines whether groups of people can come up with ideas that are both original and useful? Since the 1960s, this question has been intensively studied with the help of more or less structured group creativity activities such as brainstorming or creative problem solving, with subsequent rating of the generated ideas.

In this line of research, personal factors—such as personality traits, and other interindividual differences in emotion and cognition—have received substantial attention as potential correlates of creative outcomes of group activities. This has spawned a sprawling literature that, to date, has not yet been synthesized. Thus, empirical findings in this literature, which are also sometimes contradictory, have not yet been well-integrated.

In the present study, Fillon et al. (2022) conducted the first meta-analysis of correlations between personal factors and group creativity outcomes. The authors searched and synthesized the existing published literature according to predetermined criteria to (1) assess the overall relationship between a broad list of personal factors and creativity outcomes in group settings and (2) explore potential moderators of these relationships. 

In total, 11 studies could be included in the meta-analysis. They provided weak support for a positive correlation between self-efficacy and the three investigated group creative outcomes, number of ideas, originality of ideas, and usefulness of ideas. With respect to moderators, many of the planned analyses could not be conducted due to the low number of studies. The only finding that arose was weak evidence for the idea that time constraint moderates associations: relationships between personal factors and group creativity outcomes were slightly stronger for tasks limited to 20 minutes rather than 10 minutes. Statistical power overall was low across studies.

The authors conclude their meta-analysis with the observation that the available data on the topic are very limited. They suggest that to improve our knowledge of the topic, future studies should adhere to standardized creativity methods and protocols and implement expert ratings of creativity. They also call for increasing the availability of raw data in this field of study to improve the accumulation of knowledge about links between personal factors and the creative performance of groups.

The Stage 2 manuscript was evaluated over one round of in-depth review provided by the recommender and Chris Chambers, as the original reviewers were no longer available. Based on additional changes to the manuscript, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.

 
URL to the preregistered Stage 1 protocol: https://osf.io/nybg6
 
Level of bias control achieved: Level 3. At least some data/evidence that was used to answer the research question had been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certified that had not yet observed ANY part of the data/evidence prior to IPA.
 
List of eligible PCI RR-friendly journals:
 

 

References

1. Fillon, A. A., Girandola, F., Bonnardel, N., Kenworth, J. B. & Souchet, L. (2023). Personal factors and group creativity characteristics: A correlational meta-analysis, acceptance of Version 2 by Peer Community in Registered Reports. https://psyarxiv.com/4br6a/

Personal factors and group creativity characteristics: A correlational meta-analysisAdrien Alejandro Fillon; Fabien Girandola, Nathalie Bonnardel, Jared Kenworthy, Lionel Souchet<p>Previous research has investigated the relationship between personal factors and group creativity outcomes. On one side, they indicated that individual differences play a role in group creativity. On the other side, they showed that group creat...Social sciencesJulia M. Rohrer2023-02-21 12:41:02 View
19 Apr 2022
STAGE 1
toto

Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of duration

Could asymmetrical perceptions about the frequency of past and future events explain the Temporal Doppler Effect?

Recommended by ORCID_LOGO based on reviews by Chris Chambers and 2 anonymous reviewers
The Temporal Doppler Effect is a phenomenon where people subjectively perceive the past to be further away than the future even when both temporal distances are objectively the same from the present moment (Caruso et al., 2013). A common explanation for this phenomenon assumes that our perception of the past and future is based on spatial and temporal analogies (Matlock, Ramscar, & Boroditsky, 2005; Casanto & Boroditsky, 2008) and that the subjective discrepancy is due to people feeling that they are moving towards the future and away from the fast, thus underestimating the temporal distance of the former and overestimating the temporal distance of the latter (Caruso et al., 2013).
 
In the current study, Zhang et al. propose to replicate the Temporal Doppler Effect as tested by Caruso et al. (2013) in study 1 and to test an alternative explanation for the effect in study 2 based on the filled-duration illusion (Thomas & Brown, 1974). This alternative explanation assumes that the subjective discrepancy is based on the difference that the past and the future are filled with events that we can remember or imagine. Because the past has already happened, it is comprised of more events (those that were planned and those that were not), while the future still exists only of events that are currently planned. 
 
The Stage 1 manuscript was evaluated over three rounds of in-depth review. The main changes during the review process involved clarifications and adaptations of the way that the authors will measure the participants’ perception of how full of events the past and the future are, as the originally proposed way measuring this did not have sufficient theoretical or empirical justifications. The authors decided to address this by firstly, clarifying this issue in the stage 1 report so that the reader is aware of the potential shortcomings of this measure, and secondly, by testing a second group of participants with an alternative measure. Based on detailed responses to the reviewers’ comments and edits to the stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA). 
 
URL to the preregistered Stage 1 protocol: https://osf.io/d9ec3
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Caruso, E. M., Van Boven, L., Chin, M., & Ward, A. (2013). The temporal doppler effect: When the future feels closer than the past. Psychological Science, 24, 530-536. https://doi.org/10.1177/0956797612458804 
 
2. Casanto, D., & Boroditsky, L. (2008). Time in the mind: Using space to think about time. Cognition, 106, 579-593. https://doi.org/10.1016/j.cognition.2007.03.004
 
3. Matlock, T., Ramscar, M., & Boroditsky, L. (2005). One the experiential link between spatial and temporal language. Cognitive Science, 29, 655-664. https://doi.org/10.1207/s15516709cog0000_17 
 
4. Thomas, E. C., & Brown, I. (1974). Time perception and the filled-duration illusion. Perception & Psychophysics, 16, 449-458. https://doi.org/10.3758/BF03198571 
 
5. Zhang, Q., Masuda, Y., Ueda, K.,Toda, K., & Yamada, Y. (2022). Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of duration. Stage 1 Registered Report, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/d9ec3
Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of durationQinjing Zhang, Yoshitaka Masuda, Kohei Ueda, Kodai Toda, Yuki Yamada<p>People sometimes feel events to be psychologically closer while farther at other times. Caruso et al. (2013) reported the Temporal Doppler Effect (TDE) in which people feel that the past is farther than the future, despite an equivalent objecti...Social sciencesLjerka Ostojic2021-06-14 16:04:02 View
16 Sep 2022
STAGE 1
toto

Taking A Closer Look At The Bayesian Truth Serum: A Registered Report

Understanding the key ingredients of the Bayesian Truth Serum

Recommended by ORCID_LOGO based on reviews by 2 anonymous reviewers
The Bayesian Truth Serum, first introduced by Prelec (2004) rewards participants based on how surprisingly common their own answers are in relation to the actual distribution of answers. As such, it has been suggested as a possible incentive-compatible design for survey studies in different disciplines that rely on participants’ self-reports about their true preferences (Schoenegger, 2021). 
 
In this study, Schoenegger and Verheyen propose to replicate the results reported by Schoenegger (2021) and to directly investigate whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts. 
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers’ comments and edits to the stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
 
2. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4 
 
3. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 1 Registered Report, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/dkvms 
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportPhilipp Schoenegger & Steven Verheyen<p>Over the past decades, psychology and its cognate disciplines have undergone substantial reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has received comp...Social sciencesLjerka Ostojic2021-12-06 17:36:15 View
16 Sep 2022
STAGE 2
(Go to stage 1)
toto

Taking A Closer Look At The Bayesian Truth Serum: A Registered Report

Reassessing the use of the Bayesian Truth Serum as an incentive-compatible design for self-reports

Recommended by ORCID_LOGO based on reviews by Joël van der Weele
Different disciplines and research areas that rely on participants’ self-reports to accrue data on participants’ true preferences are faced with the question to what extent the former can be equated with the latter. Using monetary incentivisation for study participation may influence this relationship, and researchers, especially in economics, have been discussing how to develop and implement incentive-compatible research designs, i.e., those in which the incentivisation yields the best payoff for the participant if they report their true preferences (Hertwig & Ortmann, 2011; Baillon, 2017). The Bayesian Truth Serum, first introduced by Prelec (2004), according to which participants are rewarded based on how surprisingly common their own answers are relative to the actual distribution of answers, has been proposed as a possible incentive-compatible design for survey studies that rely on participants’ self-reports about their true preferences (Schoenegger, 2021).
 
In this study, Schoenegger and Verheyen (2022) ran a replication of the study by Schoenegger (2021) and assessed whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts. The authors report that the manipulation did not yield a significant difference compared to control conditions, which they interpret as a failure to replicate the original results. At the same time, the authors are careful in drawing conclusions as to the usefulness of the Bayesian Truth Serum for self-report studies using Likert-scale items in general, as they emphasise that smaller effect sizes may be of interest and that the results may differ when different items are used. 
 
The Stage 2 manuscript was evaluated by two reviewers, one of whom reviewed the first Stage 1 submission, and the other one of whom reviewed the manuscript specifically to assess statistical questions.
 
Following a careful revision by the authors, the recommender judged that the manuscript meets the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Baillon, A. (2017). Bayesian markets to elicit private information. Proceedings of the National Academy of Science, 114(30), 7985-7962. https://doi.org/10.1073/pnas.1703486114
 
2. Hertwig, R. & Ortmann (2001). Experimental practices in economics: a methodlogical challenge for psychologists? Behavioral and Brain Sciences, 24(3), 383-403. https://doi.org/10.1037/e683322011-032
 
3. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
 
4. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4 
 
5. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 2 Registered Report, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://doi.org/10.31219/osf.io/9zvqj
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportPhilipp Schoenegger & Steven Verheyen<p>Over the past decade, psychology and its cognate disciplines have undergone substantial scientific reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has rec...Social sciencesLjerka Ostojic2022-06-11 14:39:38 View