Submit a report

Announcements

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.


 

Latest recommendationsrssmastodon

IdTitle▼AuthorsAbstractPictureThematic fieldsRecommenderReviewersSubmission date
23 Nov 2023
STAGE 2
(Go to stage 1)
toto

The Medusa effect: A registered replication report of Will, Merritt, Jenkins, and Kingstone (2021)

Looking (again) at Medusa: Evidence that pictorial abstraction influences mind perception

Recommended by ORCID_LOGO based on reviews by Alan Kingstone and 1 anonymous reviewer
The Medusa effect is a recently described phenomenon in which people judge a person to be more mindful when they appear as a picture (termed L1) than as a picture within a picture (L2). Across a series of experiments, Will et al. (2021) reported that at higher levels of abstraction, images of people were judged lower in realness (how real the person seemed), experience (the ability to feel) and agency (the ability to plan and act), and also benefited less from prosocial behaviour. The findings provide an intriguing window into mind perception – the extent to which we attribute minds and mental capacities to others.
 
In the current study, Han et al. (2023) undertook a close replication of two experiments from the original report by Will et al. (2021), asking first, whether the level of pictorial abstraction influences ratings of realness, agency and experience, and second, whether it also influences prosocial behaviour as measured in the dictator game (with participants predicted to allocate more money to recipients presented as pictures than as pictures within pictures). In the event of a non-replication using the original materials, the authors planned to further repeat the experiments using newly generated stimuli that are better matched for cultural context and more tightly controlled along various dimensions.
 
Results supported all pre-registered hypotheses. Participants rated and perceived L1 stimuli as having significantly higher levels of realness, agency, and experience than L2, and they also allocated significantly more money to L1 recipients than L2 recipients in a dictator game. Furthermore, participants who judged L1 as higher than L2 on all three dimensions also differentiated significantly between L1 and L2 in the dictator game, indicating a relationship between mind perception and prosociality. Overall, the findings confirm that pictures with lower levels of abstraction are perceived as more mindful and are associated with higher levels of prosocial behavior. Consequently, the results suggest that the Medusa effect is a reproducible phenomenon.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/xj46z
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Will, P., Merritt, E., Jenkins, R., & Kingstone, A. (2021). The Medusa effect reveals levels of mind perception in pictures. Proceedings of the National Academy of Sciences, 118(32), e2106640118. https://doi.org/10.1073/pnas.2106640118
 
2. Han, J., Zhang, M., Liu, J., Song, Y. & Yamada, Y. (2023).The Medusa effect: A registered replication report of Will, Merritt, Jenkins, and Kingstone (2021). Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/yqnu8
The Medusa effect: A registered replication report of Will, Merritt, Jenkins, and Kingstone (2021)Jing Han, Minjun Zhang, Jiaxin Liu, Yu Song, Yuki Yamada<p>Will et al.'s (2021) found the Medusa effect, which refers to the tendency that people evaluate a “person in picture” more mindful than a “person in picture of a picture”. The present study tried to directly replicate the Experiments 2 and 5 of...Social sciencesChris Chambers2023-10-24 03:42:14 View
07 Aug 2023
STAGE 2
(Go to stage 1)
toto

The link between Empathy and Forgiveness: Replication and extensions Registered Report of McCullough et al. (1997)'s Study 1

Strong evidence that empathy is important for forgiveness

Recommended by ORCID_LOGO based on reviews by James Bartlett and Saleh Shuqair
Forgiveness is a core feature of human psychology in which a person makes a deliberate decision to cease negative emotions or attitudes toward an offender who has done them harm. The concept of interpersonal forgiveness is deeply embedded across societies, but much remains to be understood about how it actually works. What are its key ingredients and why does it occur in the first place? Research in social psychology has demonstrated a range of personal and social benefits of forgiveness, giving rise to two dominant mechanistic accounts – one that positions empathy as the driving factor and another that centres motivated reasoning (Donovan & Priester, 2017).
 
In the current study, Chan and Feldman (2023) sought to replicate a formative study by McCullough et al (1997) that led to the so-called Empathy Model of forgiveness. According to this theory, forgiving is a motivational change facilitated (crucially) by empathy, promoting constructive over destructive behaviour toward the offender. Chan and Feldman replicated Study 1 from McCullough et al., measuring the correlational relationship between apology, forgiving, and empathy for offenders, and exploring whether forgiving is associated with increased conciliation and decreased avoidance motivation. As well as closely replicating the original study, the authors extended it to test the more severe hypothesis that empathy causally influences forgiveness. To achieve this, they experimentally manipulated empathy by adding two groups to the design: one in which participants were asked to recall hurtful past experiences in which they were not empathetic to the offender, and another in which they were highly empathetic.
 
The outcomes constitute a successful replication. Affective empathy was positively associated with perceived apology and forgiveness, and forgiveness was positively associated with conciliation motivation and negatively associated with both avoidance motivation and revenge motivation. In addition, the results of the experimental extension revealed a reliable causal effect of empathy on forgiveness and perceived apology. Overall, the findings provide robust support for the Empathy Model of forgiveness.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on responses to the comments, the recommender judged that the manuscript met the Stage 2 criteria and therefore awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/q78fs
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Donovan, L. A. N., & Priester, J. R. (2017). Exploring the psychological processes underlying interpersonal forgiveness: The superiority of motivated reasoning over empathy. Journal of Experimental Social Psychology, 71, 16-30. https://doi.org/10.1016/j.jesp.2017.02.005
 
2. McCullough, M. E., Worthington, E. L., & Rachal, K. C. (1997). Interpersonal Forgiving in Close Relationships. Journal of Personality and Social Psychology, 73, 321–336. https://doi.org/10.1037/0022-3514.73.2.321 
 
3. Chan, C. F. & Feldman, G. (2023). The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1, acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/956fa
The link between Empathy and Forgiveness: Replication and extensions Registered Report of McCullough et al. (1997)'s Study 1Chan Chi Fung, Gilad Feldman <p>The empathy model of forgiveness conceptualized forgiving as an empathy-facilitated motivational change that leads to reductions in the motivation to behave in relationship -destructive ways and increases in the motivation to behave in relation...Social sciencesChris Chambers2023-04-13 14:28:05 View
11 Apr 2023
STAGE 1
toto

The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1

Is empathy important for forgiveness?

Recommended by ORCID_LOGO based on reviews by Wenrui Cao, James Bartlett and Saleh Shuqair
Forgiveness is a core feature of human psychology in which a person makes a deliberate decision to cease negative emotions or attitudes toward an offender who has done them harm. The concept of interpersonal forgiveness is deeply embedded across societies, but much remains to be understood about how it actually works. What are its key ingredients and why does it occur in the first place? Research in social psychology has demonstrated a range of personal and social benefits of forgiveness, giving rise to two dominant mechanistic accounts – one that positions empathy as the driving factor and another that centres motivated reasoning (Donovan & Priester, 2017).
 
In the current study, Chan and Feldman (2023) seek to replicate a formative study by McCullough et al (1997) that led to the so-called Empathy Model of forgiveness. According to this theory, forgiving is a motivational change facilitated (crucially) by empathy, promoting constructive over destructive behaviour toward the offender. Chan and Feldman will replicate Study 1 from McCullough et al., measuring the correlational relationship between apology, forgiving, and empathy for offenders, and exploring whether forgiving is associated with increased conciliation and decreased avoidance motivation. As well as closely replicating the original study, the authors will extend it to test the more severe hypothesis that empathy causally influences forgiveness. To achieve this, they will experimentally manipulate empathy by adding two groups to the design: one in which participants are asked to recall hurtful past experiences in which they were not empathetic to the offender, and another in which they were highly empathetic.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/q78fs
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Donovan, L. A. N., & Priester, J. R. (2017). Exploring the psychological processes underlying interpersonal forgiveness: The superiority of motivated reasoning over empathy. Journal of Experimental Social Psychology, 71, 16-30. https://doi.org/10.1016/j.jesp.2017.02.005
 
2. Chan, C. F. & Feldman, G. (2023). The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/q78fs
The link between Empathy and Forgiveness: Replication and extensions of McCullough et al. (1997)'s Study 1Chan Chi Fung, Gilad Feldman <p>[IMPORTANT: Abstract, method, and results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection. For the purpose o...Social sciencesChris Chambers Wenrui Cao, James Bartlett2023-01-25 11:15:13 View
10 Feb 2023
STAGE 2
(Go to stage 1)
toto

The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered Report

Different ways of describing problematic substance use and its treatment influence public stigma

Recommended by ORCID_LOGO based on reviews by Nicholas Sinclair-House
People experiencing problematic substance use are often stigmatised by the general public. This public stigma may impair such people obtaining help and the quality of help that they receive. For this reason, previous research has investigated the factors that may exacerbate or lessen stigma by focusing on the terminology used to describe problematic substance use. However, the evidence is not clear cut, with some studies suggesting that labelling the condition as a "chronically relapsing brain disease" vs a "problem" reduces certain elements of stigma and other studies finding absence of evidence. A closer look at these studies points to methodological differences that may explain their results, such as whether problematic substance use is compared with another health condition, whether the individual is described as seeking treatment or not, and whether general or discrete elements of stigma are measured.
 
In this Stage 2 Registered Report, Pennington et al. (2023) isolated these methodological differences to investigate if any of them influenced two different measures of stigma used in previous work. They found that greater social distance, danger and public stigma but lower blame were ascribed to drug use relative to a health concern, supporting previous research to suggest that problematic substance use is a highly stigmatised health condition. Furthermore, greater (genetic) blame was reported when drug use was labelled as a ‘chronically relapsing brain disease’ relative to a ‘problem’. The results for attributional judgement were either inconclusive or statistically equivalent. In summary, these findings suggest that the labels and models used to describe problematic substance use may impact upon public stigma in distinct ways. The authors suggest that future research should justify which measures are being used in line with theory. They also put forward the notion that addiction is a functional attribution, which may explain the mixed literature on the brain disease model of addiction to date.
 
The Stage 2 manuscript was evaluated over one round of specialist review and several rounds of discussion with the recommender. Based on comprehensive responses, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/4vscg
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
Pennington, C. R., Monk, R. L., Heim, D., Rose, A. K., Gough, T., Clarke, R., Knibb, G., Patel, R., Rai, P., Ravat, H., Ali, R., Anastasiou, G., Asgari, F., Bate, E., Bourke, T., Boyles, J., Campbell, A., Fowler, N., Hester, S., Neil, C., McIntrye, B., Ogilvy, E., Renouf, A., Stafford, J., Toothill, K., Wong, H. K., &  Jones, A. (2023). The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered Report. Stage 2 acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/z9bnf
The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered ReportCharlotte R. Pennington, Rebecca L. Monk, Derek Heim, Abi K. Rose, Thomas Gough, Ross Clarke, Graham Knibb, Roshni Patel, Priya Rai, Halimah Ravat, Ramsha Ali, Georgiana Anastasiou, Fatemeh Asgari, Eve Bate, Tara Bourke, Jayme Boyles, Alix Campbel...<p>Objectives: Problematic substance use is one of the most stigmatised health conditions leading research to examine how the labels and models used to describe it influence public stigma. Two recent studies examine whether beliefs in a disease mo...Social sciencesZoltan Dienes2022-10-21 16:13:49 View
17 Jan 2024
STAGE 1
toto

The Efficacy of Attentional Bias Modification for Anxiety: A Registered Replication

Examining attentional retraining of threat as an intervention in pathological worry

Recommended by ORCID_LOGO based on reviews by Thomas Gladwin, Jakob Fink-Lamotte and 1 anonymous reviewer
Cognitive models ascribe a pivotal role to cognitive biases in the development and maintenance of mental disorders. For instance, attentional biases that prioritize the processing of threat-related stimuli have been suggested to be causally involved in the development and maintenance of anxiety disorders, including generalized anxiety disorder (GAD), which is marked by pathological worry. Therefore, these biases have garnered significant interest as potential diagnostic indicator and as targets for modification.
 
The idea that attention bias modification (ABM) can serve as a therapeutic intervention for GAD and other disorders was fueled by a seminal study by Hazen et al. (2009). In this study, 23 individuals experiencing high levels of worry underwent a computerized attentional retraining of threat stimuli (ARTS) or placebo control training during five training sessions. Relative to control, attention retraining was found to reduce preferential attention to threat, as well as depression and anxiety symptoms. However, as Pond et al. (2024) highlight in their review of the literature, the evidence endorsing the efficacy of ABM in alleviating anxiety disorders is still inconclusive. Moreover, some researchers contend that early positive findings might have been inflated due to demand effects.
 
Based on these considerations, Pond et al. (2024) propose a direct replication of Hazen et al. (2009) by subjecting a high-worry sample to five sessions of ARTS or placebo control. Departing from the frequentist analyses used in the original study, the authors will employ Bayesian analyses that allow more nuanced interpretation of the results, allowing consideration of evidence in support of the null hypothesis. The sampling plan will adhere to a Bayesian stopping rule, whereby the maximal sample size will be set at n=200. Furthermore, the authors extend the original study by addressing potential demand effects. For this purpose, they include a measure of phenomenological control (i.e., the ability to generate experiences align with the expectancies of a given situation) and evaluate its potential moderating impact on the attention bias training.
 
The Stage 1 manuscript was evaluated by three expert reviewers in two rounds of in-depth review. Following responses from the authors, the recommender determined that Stage 1 criteria were met and awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/5f7u9
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Hazen, R. A., Vasey, M. W., & Schmidt, N. B. (2009). Attentional retraining: A randomized clinical trial for pathological worry. Journal of Psychiatric Research, 43, 627-633. https://doi.org/10.1016/j.jpsychires.2008.07.004 
 
2. Pond, N., Meeten, F., Clarke, P., Notebaert, L., & Scott, R. B. (2024). The efficacy of attentional bias modification for anxiety: A registered replication. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/5f7u9
 
The Efficacy of Attentional Bias Modification for Anxiety: A Registered ReplicationNathan Pond, Frances Meeten, Patrick Clarke, Lies Notebaert, Ryan Scott<p>Generalised anxiety disorder (GAD) is a prevalent condition that has been linked to the presence of certain cognitive biases, including attention bias. Attention bias is the tendency to attend preferentially to threat-related stimuli and has be...Social sciencesThomas Meyer2023-09-15 19:25:47 View
16 Nov 2023
STAGE 1
toto

The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered Report

Are there oscillatory markers of pain intensity?

Recommended by ORCID_LOGO based on reviews by Markus Ploner and Björn Horing
Rhythmic changes in pain can lead to corresponding modulations of EEG amplitudes in theta, alpha, and beta bands. But the question remains open as to whether these modulations are actually tracking pain, or maybe rather saliency or stimulus intensity. The question is of some importance because a marker of pain per se could be useful for tracking felt pain without a verbal response, and could be useful in investigating interventions for treating pain (such as suggestion).  Here, Leu et al. (2023) will address the question of whether modulations reflect saliency or else the intensity of pain, by using an oddball paradigm in which most trials are a pain stimulus of a certain intensity, and oddball trials will sometimes occur, at either a higher intensity or a lower intensity than the baseline ones. If the modulations reflect salience, the modulation at the frequency of the oddball will be similar for high and low intensity oddballs. However, if the modulations reflect pain intensity, the modulations for the low rather than high oddball condition will be lower.
 
The Stage 1 manuscript was evaluated over three rounds of in-depth peer review, the first two consisting of substantial comments from two scholars with relevant expertise, and the third consisting of a close review by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/qbrf2
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Leu, C., Forest, S., Legrain, V., & Liberati, G. (2023). The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered Report. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/qbrf2
The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered ReportChiara Leu, Sébastien Forest, Valéry Legrain, Giulia Liberati<p>Ongoing oscillations have been shown to be modulated in different frequency bands following phasic, tonic as well as periodic thermonociceptive stimulation. Yet, it remains unclear whether these modulations are related to pain perception, salie...Life Sciences, Medical SciencesZoltan Dienes2023-09-06 15:15:19 View
04 Dec 2023
STAGE 2
(Go to stage 1)
toto

The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report

Brooding increases conspiracy beliefs but with practical significance to be determined

Recommended by ORCID_LOGO based on reviews by Matt Williams and Daniel Toribio-Flórez
The world is seemingly awash with conspiracy theories – from well-trodden examples such as fake Moon landings, the 9/11 truth movement, and Holocaust denial, to relative newcomers including COVID as a bioweapon, QAnon, and the belief that the science of climate change has been invented or falsified. While there is a public perception that conspiracy theories are becoming more prevalent, recent evidence suggests that the rate of conspiracism is relatively stable over time (Uscinski et al., 2022). At any point in history, it seems that a certain proportion of people find themselves vulnerable to conspiracy beliefs, but what distinguishes those who do from those who don’t, and what are the causal factors?
 
In the current study, Liekefett et al. (2023) investigated the critical role of rumination – a perseverative and repetitive focus on negative content leading to emotional distress. In particular, the authors asked whether one component of rumination referred to as brooding (dwelling on one’s worries and distressing emotions) has a specific causal role in the formation of conspiracy beliefs. In a series of preliminary experiments, the authors first established a procedure for successfully inducing rumination, identifying various boundary conditions and requirements for a successful design. In the main study (N=1,638 to 2,007 depending on the analysis), they asked whether the induction of brooding causes a significant increase in conspiracy beliefs. Manipulation checks were also included to confirm intervention fidelity (independently of this hypothesis), and exploratory analyses tested the effect of various moderators, as well as the causal role of a complementary manipulation of reflection – a component of rumination in which attention is focused on the issue at hand rather than one’s emotions.
 
As expected by the authors' preliminary work, manipulation checks independently confirmed the effectiveness of the brooding intervention. In answer to the main research question, participants who brooded over the worries and negative emotions associated with an issue were more susceptible to conspiracy beliefs compared to a control group. However, while this effect of brooding was statistically significant, the confidence interval of the effect size estimate overlapped with the authors' proposed smallest effect size of interest (d = 0.20), suggesting that the practical value of the effect remains to be determined.
 
Overall the findings are consistent with a range of psychological theories suggesting that rumination induces negative affect and/or narrows attention to negative information, which in turn may make conspiracy theories seem more probable and render individuals more vulnerable to cognitive bias. The authors note the importance of future work to define the smallest effect of practical significance, analagous to the criteria used to determine the 'minimal clinically important difference’ in medical research.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/y82bs
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Uscinski, J., Enders, A., Klofstad, C., Seelig, M., Drochon, H., Premaratne, K. & Murthi, M. (2022) Have beliefs in conspiracy theories increased over time? PLOS ONE 17: e0270429. https://doi.org/10.1371/journal.pone.0270429

2. Liekefett, L. Sebben, S. & Becker, J. C. (2023). The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report. Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/3e8wc
The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered ReportLuisa Liekefett, Simone Sebben, Julia C. Becker<p>This Stage 2 Registered Report concerns the relationship between rumination, a repetitive style of negative thinking, and conspiracy beliefs (Stage 1 protocol: https://osf.io/y82bs, date of in-principle-acceptance: 23/05/2023). Based on four pi...Humanities, Social sciencesChris Chambers2023-10-19 17:46:59 View
18 Jul 2023
STAGE 1
toto

Test-Retest Reliability of the STRAQ-1: A Registered Report

A reliable measure of physical closeness in interpersonal relationships?

Recommended by ORCID_LOGO based on reviews by Jacek Buczny and Ian Hussey
Attachment and interpersonal relationships are a major subject of research and clinical work in psychology. There are, accordingly, a proliferation of measurement instruments to tap into these broad constructs. The emphasis in these measures tends to be on the emotional dimensions of the relationships—how people feel about their partners and the support that they receive. However, that is not all there is to relationship quality. Increasing attention has been paid to the physical and physiological aspects of relationships, but there are few psychometrically sound measures available to assess these dimensions.
 
In the current study, Dujols et al. (2023) seek to assess the psychometric properties of the Social Thermoregulation and Risk Avoidance Questionnaire (STRAQ-1), a measure of physical relationships that targets social thermoregulation, or how physical proximity is used to promote warmth and closeness. The proposed project will be a thorough assessment of the measure’s reliability over time—that is, the degree to which the measure assesses the construct similarly across administrations. The authors will assess the test-retest reliability and longitudinal measurement invariance of the STRAQ-1, providing much-needed psychometric data that can build confidence in the utility of the measure.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review, the first round consisting of detailed comments from two reviewers and the second round consisting of a close read by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and was therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/pmnk2
 
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Dujols, O., Klein, R. A., Lindenberg, S., Van Lissa, C. J., & IJzerman, H. (2023). Test-Retest Reliability of the STRAQ-1: A Registered Report. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/pmnk2
Test-Retest Reliability of the STRAQ-1: A Registered ReportOlivier Dujols; Richard A. Klein; Siegwart Lindenberg; Hans IJzerman<p>This Registered Report provides the first test of measurement invariance across time points and estimates of test-retest reliability for the Social Thermoregulation, Risk Avoidance Questionnaire (STRAQ-1, Vergara et al., 2019). The scale was de...Social sciencesMoin Syed2023-03-01 16:07:17 View
16 Sep 2022
STAGE 1
toto

Taking A Closer Look At The Bayesian Truth Serum: A Registered Report

Understanding the key ingredients of the Bayesian Truth Serum

Recommended by ORCID_LOGO based on reviews by 2 anonymous reviewers
The Bayesian Truth Serum, first introduced by Prelec (2004) rewards participants based on how surprisingly common their own answers are in relation to the actual distribution of answers. As such, it has been suggested as a possible incentive-compatible design for survey studies in different disciplines that rely on participants’ self-reports about their true preferences (Schoenegger, 2021). 
 
In this study, Schoenegger and Verheyen propose to replicate the results reported by Schoenegger (2021) and to directly investigate whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts. 
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers’ comments and edits to the stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
 
2. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4 
 
3. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 1 Registered Report, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/dkvms 
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportPhilipp Schoenegger & Steven Verheyen<p>Over the past decades, psychology and its cognate disciplines have undergone substantial reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has received comp...Social sciencesLjerka Ostojic2021-12-06 17:36:15 View
16 Sep 2022
STAGE 2
(Go to stage 1)
toto

Taking A Closer Look At The Bayesian Truth Serum: A Registered Report

Reassessing the use of the Bayesian Truth Serum as an incentive-compatible design for self-reports

Recommended by ORCID_LOGO based on reviews by Joël van der Weele
Different disciplines and research areas that rely on participants’ self-reports to accrue data on participants’ true preferences are faced with the question to what extent the former can be equated with the latter. Using monetary incentivisation for study participation may influence this relationship, and researchers, especially in economics, have been discussing how to develop and implement incentive-compatible research designs, i.e., those in which the incentivisation yields the best payoff for the participant if they report their true preferences (Hertwig & Ortmann, 2011; Baillon, 2017). The Bayesian Truth Serum, first introduced by Prelec (2004), according to which participants are rewarded based on how surprisingly common their own answers are relative to the actual distribution of answers, has been proposed as a possible incentive-compatible design for survey studies that rely on participants’ self-reports about their true preferences (Schoenegger, 2021).
 
In this study, Schoenegger and Verheyen (2022) ran a replication of the study by Schoenegger (2021) and assessed whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts. The authors report that the manipulation did not yield a significant difference compared to control conditions, which they interpret as a failure to replicate the original results. At the same time, the authors are careful in drawing conclusions as to the usefulness of the Bayesian Truth Serum for self-report studies using Likert-scale items in general, as they emphasise that smaller effect sizes may be of interest and that the results may differ when different items are used. 
 
The Stage 2 manuscript was evaluated by two reviewers, one of whom reviewed the first Stage 1 submission, and the other one of whom reviewed the manuscript specifically to assess statistical questions.
 
Following a careful revision by the authors, the recommender judged that the manuscript meets the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Baillon, A. (2017). Bayesian markets to elicit private information. Proceedings of the National Academy of Science, 114(30), 7985-7962. https://doi.org/10.1073/pnas.1703486114
 
2. Hertwig, R. & Ortmann (2001). Experimental practices in economics: a methodlogical challenge for psychologists? Behavioral and Brain Sciences, 24(3), 383-403. https://doi.org/10.1037/e683322011-032
 
3. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
 
4. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4 
 
5. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 2 Registered Report, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://doi.org/10.31219/osf.io/9zvqj
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportPhilipp Schoenegger & Steven Verheyen<p>Over the past decade, psychology and its cognate disciplines have undergone substantial scientific reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has rec...Social sciencesLjerka Ostojic2022-06-11 14:39:38 View