Please note that we will be CLOSED to ALL SUBMISSIONS from 1 December 2024 through 12 January 2025 to give our recommenders and reviewers a holiday break.
We are recruiting recommenders (editors) from all research fields!
Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.
Latest recommendations
| Id | Title * | Authors * | Abstract * | Picture | Thematic fields * | Recommender | Reviewers | Submission date | |
|---|---|---|---|---|---|---|---|---|---|
Breaking Ban: Belgium’s ineffective gambling law regulation of video game loot boxesLoot boxes remain prevalent in Belgium despite their “ban”Recommended by Veli-Matti Karhulahti based on reviews by Andrew Moshirnia, Joseph Macey and Jason Chin
Several countries currently struggle to legally interpret and deal with loot boxes, i.e. gambling-like mechanisms that have become common especially in contemporary videogame design. One of the few countries to take a clear regulation stance is Belgium, which officially announced that they interpret paid loot boxes as games of chance that violate their Gaming and Betting Act and can thus be criminally prosecuted (Naessens 2018). This announcement four years ago laid the basis for a unique social experiment where companies offering loot boxes in Belgium had to decide whether to modify their games, exit the market, or continue to monetize with loot boxes. In the present study, Xiao (2022) investigated the outcomes of this experiment both via hypotheses testing and exploratory analysis.
Using the 100 highest-grossing iPhone games in Belgium as data and applying comprehensive qualitative mechanical analysis to each title, Xiao (2022) tested three preregistered hypotheses regarding loot box prevalence. None of the hypotheses were confirmed: the prevalence rate of loot boxes in the Belgian App Store was not null but extremely high (82.0%), also among mobile games designed for minors (54.2–80.2%), and significantly more compared to global standards when assessed by a binomial test (p<.001). Corroborating a fourth hypothesis, Xiao was also able to access various UK loot boxes in Belgium. In exploratory research, Xiao received a confirmation from the Belgian Gaming Commission that even “simulated gambling games” (that do not yield monetary wins) also legally constitute gambling in Belgium.
The results are the first to describe outcomes of ban-driven loot box regulation globally. Despite Belgium’s clear statement that “paid loot boxes must be removed from the video games in order to comply with the Belgian Gaming and Betting Act” (Naessens 2018, p. 16), almost all highest-grossing iPhone games in the Belgian App Store keep offering them. The finding is important especially for the legal authorities around the world who are currently assessing their own positions in potential regulation: simple statements are unlikely to have immediate effects on international companies in local markets. On the other hand, it remains unknown how these companies have interpreted the statement and what are their distinct reasons for continuing to offer loot boxes in the Belgian App Store. It would be important in future research to investigate the companies’ perspectives.
This study (Xiao 2022) is important also from a meta-scientific perspective. It paves the way for Registered Reports in new disciplines and methods, and involves an exceptional field experiment that was carefully documented with open data and materials. Three external experts reviewed the Stage 2 manuscript twice, based on which the recommender awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/5mxp6 Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance. List of eligible PCI RR-friendly journals:
References
1. Naessens, P. (2018) Research Report on Loot Boxes [English Translation]. Belgian Gaming Commission. URL: https://www.gamingcommission.be/sites/default/files/2021-08/onderzoeksrapport-loot-boxen-Engels-publicatie.pdf
2. Xiao, L.Y. (2022) Breaking Ban: Belgium’s ineffective gambling law regulation of video game loot boxes. Stage 2 Registered Report, acceptance of Version 2 by Peer Community in Registered Reports. https://doi.org/10.31219/osf.io/hnd7w
| Breaking Ban: Belgium’s ineffective gambling law regulation of video game loot boxes | Leon Y. Xiao | <p>Loot boxes in video games are gambling-like mechanics that players buy to obtain randomised rewards of varying value. Loot boxes are conceptually and psychologically similar to gambling, and loot box expenditure is positively correlated with se... |  | Humanities, Social sciences | Veli-Matti Karhulahti | 2022-07-28 12:29:57 | ||
Does alleviating poverty increase cognitive performance? Short- and long-term evidence from a randomized controlled trialNo strong effect of unconditional cash transfers on cognitionRecommended by Chris Chambers based on reviews by Charlotte Pennington and Matúš Adamkovič
Recent studies have revealed potential benefits of unconditional cash transfers (UCTs) on a variety of health and social outcomes, including self-reported happiness and life satisfaction (Haushofer & Shapiro, 2016), economic and financial well-being (Blattman et al., 2013; Baird et al., 2018) and educational attainment (Baird et al., 2016). Although the effects of UCTs do not always out-perform rigorous control conditions (Whillans & West, 2022), these findings prompt the question of whether the alleviation of poverty via UCTs can also influence cognitive processing and performance.
In the current study, Szaszi et al. analysed the results of a previous randomised trial of UCTs by Blattman et al. (2017) to test whether a $200 lump sum – equivalent to three months of income – administered to a sample of young men in Liberia carries both short- and long-term benefits for a range of executive functions, including attention, response inhibition, and working memory capacity. Overall, the results suggest minimal if any consequences of the intervention – the observed effects of UCTs on cognition were several times smaller than suggested by previous research, and the evidence for a positive effect was inconclusive. Extensive multiverse analyses showed that these findings were robust to a range of alternative analytical specifications, and the authors estimate that a sample size of nearly 5000 would be required to provide strong evidence.
In their Discussion, the authors explore a range of reasons for the negative findings compared with previous research, including the more rigorous and severe causal test enabled by the randomised trial design, the demographic homogeneity of the sample demographic, the use of pen-and-paper tests (cf. computerised tests in previous studies), and the delivery of a lump-sum cash transfer compared with a regular monthly installment. In addition, although the results were negative or inconclusive, there were hints that a positive effect of UCTs may be more evident in some cognitive domains than in others – in this case, potentially benefiting working memory more than inhibitory control. Further research would be required to confirm this hypothesis.
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on the responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/k56yv
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to Stage 1 acceptance, but the authors certified that they had not yet observed the key variables within the data that would be used to answer the research question AND they took additional steps to maximise bias control and rigour.
List of eligible PCI RR-friendly journals:
References
1. Haushofer, J. & Shapiro, J. (2016). The short-term impact of unconditional cash transfers to the poor: Experimental evidence from Kenya. Quarterly Journal of Economics, 13, 1973–2042. https://doi.org/10.1093/qje/qjw025
2. Blattman, C., Fiala, N. & Martinez, S. (2013) Generating skilled self-employment in developing countries: Experimental evidence from Uganda. Quarterly Journal of Economics, 129, 697–752. https://doi.org/10.1093/qje/qjt057
3. Baird, S., McKenzie, D., & Özler, B. (2018). The effects of cash transfers on adult labor market outcomes. IZA Journal of Development and Migration, 8, 1-20. https://doi.org/10.1186/s40176-018-0131-9
4. Baird, S., Chirwa, E., De Hoop, J., & Özler, B. (2016). Girl power: cash transfers and adolescent welfare: evidence from a cluster-randomized experiment in Malawi. In African Successes, Volume II: Human Capital (pp. 139-164). University of Chicago Press. https://www.nber.org/system/files/chapters/c13380/c13380.pdf
5. Whillans, A., & West, C. (2022). Alleviating time poverty among the working poor: A pre-registered longitudinal field experiment. Scientific Reports, 12(1), 1-17. https://doi.org/10.1038/s41598-021-04352-y
6. Blattman, C., Jamison, J. C. & Sheridan, M. (2017). Reducing crime and violence: Experimental evidence from cognitive behavioral therapy in Liberia. American Economic Review, 107, 1165–1206. http://doi.org/10.1257/aer.20150503
7. Szaszi, B., Palfi, B., Neszveda, G., Taka, A., Szecsi, P., Blattman, C., Jamison, J. C., & Sheridan, M. (2022). Does alleviating poverty increase cognitive performance? Short- and long-term evidence from a randomized controlled trial. Stage 2 Registered Report, acceptance of Version 2 by Peer Community in Registered Reports. https://psyarxiv.com/4gyzh
| Does alleviating poverty increase cognitive performance? Short- and long-term evidence from a randomized controlled trial | Barnabas Szaszi, Bence Palfi, Gabor Neszveda, Aikaterini Taka, Péter Szécsi, Christopher Blattman, Julian C. Jamison, Margaret Sheridan | <p>In this Registered Report, we investigated the impact of a poverty alleviation program on cognitive performance. We analyzed data from a randomized controlled trial conducted on low-income, high-risk individuals in Liberia where a random half o... | Social sciences | Chris Chambers | 2022-07-13 12:10:49 | |||
24 Oct 2022
STAGE 1
Does childhood adversity alter opioid drug reward? A conceptual replication in outpatients before surgeryIs childhood adversity associated with a heightened response to opioids?Recommended by Chris Chambers based on reviews by Zoltan Dienes, Yuki Yamada and 1 anonymous reviewer
A convergence of evidence suggests that early life adversity may cause dysfunction in opioid-sensitive reward systems. Childhood adversity is associated with opioid use, potentially by altering reward and motivation networks, and experimental models in animals have found that early life adversity increases and consolidates opioid seeking behaviours. Further, in a recent controlled experiment, Carlyle et al. (2021) found that opioid administration produced stronger positive responses, and weaker negative responses, in adults with a history of childhood abuse and neglect.
In the current study, Carlyle et al. seek to test the generalisability of these previous findings in a pre-operative clinical setting. Using partially observed data from an existing cohort study (N=155), the authors will test whether patients with greater experience of childhood trauma in turn exhibit a larger mood boost and express greater subjective pleasure following opioid administration. Although not a randomised experimental design, this study provides the opportunity to examine the relationship between opioid response and history of childhood adversity in a naturalistic setting, and thus has the potential to either support or cast doubt on the theory that adversity elevates risk of opioid addiction by altering sensitivity to subjectively pleasurable effects.
Following three rounds of in-depth review, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/7ymts Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour. List of eligible PCI RR-friendly journals:
References
1. Carlyle M., Broomby R., Simpson G., Hannon R., Fawaz L., Mollaahmetoglu O.M., Drain, J., Mostazir, M., & Morgan C. (2021). A randomised, double‐blind study investigating the relationship between early childhood trauma and the rewarding effects of morphine. Addiction Biology, 26(6):e13047.
2. Carlyle, M., Kvande, M., Leknes, S., Meier, I., Buen, K., Jensen, E. N., Ernst, G. & Eikemo, M. (2022). Does childhood adversity alter opioid drug reward? A conceptual replication in outpatients before surgery, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/7ymts
| Does childhood adversity alter opioid drug reward? A conceptual replication in outpatients before surgery | Molly Carlyle*, Malin Kvande*, Siri Leknes, Isabell Meier, Kaja Buen, Eira Nordeng Jensen, Gernot Ernst, Marie Eikemo. *denotes equal contribution. | <p>PCI-RR Snapshot only</p> | Life Sciences, Medical Sciences, Social sciences | Chris Chambers | 2022-03-15 10:07:29 | |||
21 Oct 2022
STAGE 1
Replicating the facilitatory effects of transcranial random noise stimulation on motion processing: A registered reportTesting the facilitatory effect of high-frequency transcranial random noise stimulation through enhancement of global motion processingRecommended by Robert McIntosh based on reviews by Sam Westwood and Filippo Ghin
High frequency transcranial random noise stimulation (hf-tRNS) is a relatively novel form of non-invasive brain stimulation, thought to enhance neural excitability and facilitate processing in targeted brain areas. The evidence for the efficacy of hf-tRNS is mixed, so a high-powered test of the proposed facilitatory effects would be of value to the field. This Registered Report will target the human middle temporal complex (hMT+), an area with a well-established critical role in global motion processing. The protocol is adapted from a study by Ghin and colleagues (2018) but focusing on a sub-set of the original experimental conditions and using a fully within-subjects design (n=42). Global motion processing will be operationalised in terms of the coherence threshold for identification of the dominant direction of random-dot motion. The experiment will test the predicted facilitation of contralateral motion processing (reduced coherence threshold) during hf-tRNS to the left hMT+. The specificity of this effect will be tested by comparison to a sham stimulation control condition and an active stimulation control condition (left forehead). By targeting a brain area with a well-established critical role in behaviour, this study will provide important information about the replicability and specificity of the facilitatory effects of hf-tRNS.
Following two rounds of in-depth review, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/bce7u Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Ghin, F., Pavan, A., Contillo, A., & Mather, G. (2018). The effects of high-frequency transcranial random noise stimulation (hf-tRNS) on global motion processing: an equivalent noise approach. Brain Stimulation, 11, 1263–75.
2. Caroll, M. B., Edwards, G. & Baker, C. I. (2022). Replicating the facilitatory effects of transcranial random noise stimulation on motion processing: A registered report, in principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/bce7u
| Replicating the facilitatory effects of transcranial random noise stimulation on motion processing: A registered report | Mica B. Carroll*, Grace Edwards*, Chris I. Baker | <p>Non-invasive brain stimulation (NIBS) techniques have the potential to demonstrate the causal impact of targeted brain regions on specific behaviors, and to regulate or facilitate behavior in clinical applications. Transcranial random noise sti... | Life Sciences | Robert McIntosh | 2022-06-02 21:25:02 | |||
17 Oct 2022
STAGE 1
Relationship between creativity and depression: the role of reappraisal and ruminationUnderstanding the relationship between creativity and depressive traitsRecommended by Chris Chambers based on reviews by Kate Button and 1 anonymous reviewer
For centuries, the relationship between creativity and mental health has been a subject of fascination, propelled by the impression that many of the most famous artists in history likely suffered from mood disorders or other mental illnesses. However, with the advent of psychological science – including more precise and diagnostic clinical measures – the empirical evidence for an association between creativity and depressive symptoms has been mixed, with some studies suggesting a positive relationship and others showing either no effect or indicating that the link, if there is one, may be driven by other personality characteristics (Verhaeghen et al., 2005).
In the current study, Lam and Saunders will use an online design in 200 participants to ask whether creativity is associated with higher depressive traits, and further, whether that relationship depends on two additional variables that could explain an observed positive correlation: self-reflective rumination (repetitive thoughts that maintain a negative mood state) and the frequency with which individuals engage in reappraisal (a regulation strategy that involves reinterpreting an event or situation to diminish its negative impact). If justified by the main confirmatory findings, the authors will also explore the moderating role of gender and how any observed associations are reflected in more fine-grained measures of creativity. The results promise to shed light on not only the basic question of whether creativity is related to depressive traits, but the extent to which that association depends on related determinants of mental health.
Following two rounds of in-depth review, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/yub7n
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Verhaeghen, P., Joormann, J., & Khan, R. (2005). Why we sing the blues: The relation between self-reflective rumination, mood, and creativity. Emotion, 5(2), 226-232. https://doi.org/10.1037/1528-3542.5.2.226
2. Lam, C. Y. & Saunders, J. A. (2022). Relationship between creativity and depression: the role of reappraisal and rumination, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/yub7n
| Relationship between creativity and depression: the role of reappraisal and rumination | Chin Yui Lam and Jeffrey Allen Saunders | <p>Previous research has found mixed evidence about whether increased creativity is associated with higher depression. We investigated the relationship between creativity and depression, and the role of two emotion regulation strategies: ruminatio... | Social sciences | Chris Chambers | 2022-01-27 10:53:10 | |||
Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of durationThe Temporal Doppler Effect may not be a robust and culturally universal phenomenonRecommended by Ljerka Ostojic based on reviews by Chris Chambers and 1 anonymous reviewer
The Temporal Doppler Effect refers to the subjective perception that the past is further away than the future even when both temporal distances are objectively the same from the present moment (Caruso et al., 2013). In the current study, Zhang et al. ran a replication of this phenomenon and tested one possible explanation for it, namely that people overestimate the temporal distance of the past because the past is filled with more events than the future. This is because we can access information only about planned events for the future, but have access to both planned and unplanned events that happened in the past (filled-duration illusion; Thomas & Brown, 1974).
Over two studies, the authors found that the sampled participants reported feeling that the past was psychologically closer than the future, which is the opposite of what has previously been reported and termed the Temporal Doppler Effect (Caruso et al., 2013). In addition, the authors reported inconsistent results regarding the correlations between the psychological distance and different variables associated with the filling rate of duration. The authors discuss the differences between their own results and those by Caruso et al. (2013) in terms of methodological and contextual differences and highlight cultural aspects that may be critical to consider in future replications and overall testing of this phenomenon. As such, they highlight that, at the moment, the Temporal Doppler Effect should not be considered a robust and culturally universal phenomenon.
The Stage 2 manuscript was evaluated by two reviewers who had also reviewed the stage 1 report. Following a revision by the authors, which consisted of adding the Data Availability statement, as well as a more precise summary of the results in various sections of the report, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/d9ec3/
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance. List of eligible PCI RR-friendly journals:
References
1. Caruso, E. M., Van Boven, L., Chin, M., & Ward, A. (2013). The temporal doppler effect: When the future feels closer than the past. Psychological Science, 24, 530-536. https://doi.org/10.1177/0956797612458804
2. Thomas, E. C., & Brown, I. (1974). Time perception and the filled-duration illusion. Perception & Psychophysics, 16, 449-458. https://doi.org/10.3758/BF03198571
3. Zhang, Q., Masuda, Y., Ueda, K.,Toda, K., & Yamada, Y. (2022). Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of duration. Stage 2 Registered Report, acceptance of Version 2 by Peer Community in Registered Reports. https://psyarxiv.com/pb47n/
| Is the past farther than the future? A registered replication and test of the time-expansion hypothesis based on the filling rate of duration | Qinjing Zhang, Yoshitaka Masuda, Kodai Toda, Kohei Ueda, Yuki Yamada | <p>People feel some events to be psychologically closer, while others to be farther away. Caruso et al. (2013) reported the Temporal Doppler Effect (TDE), in which people feel that the past is farther than the future, despite an equivalent objecti... |  | Social sciences | Ljerka Ostojic | 2022-08-20 09:59:09 | ||
Phenomenological Strands for Gaming Disorder and Esports Play: A Qualitative Registered ReportThe lived experience of gamers: a comparative qualitative investigation of treatment-seekers and esports playersRecommended by Chris Chambers based on reviews by Malte Elson and Peter Branney
Since 2018, the inclusion of “gaming disorder” in the ICD-11 has been met with a mixture of interest, confusion and controversy (Van Rooij et al, 2018), mirroring broader debates about the effects of gaming on mental health. One of the major gaps in understanding the validity of gaming disorder as an identifiable mental illness is the absence of qualitative studies comparing the lived experience of gamers who seek treatment with esports players who do not report health problems.
Here, Karhulahti et al. (2022) tackle this question in the first of two Stage 2 Registered Reports associated with their previous programmatic Stage 1 submission. Using interpretative phenomenological analysis in gamers and medical experts, they find that treatment-seekers and esports players differ in how gaming is associated with the sense of self, either interfering with the self for treatment-seekers or successfully integrating into the self for esports players. These findings help to identify the key characteristics of problematic and non-problematic gaming and call for more intensive and wide-reaching qualitative research in this area.
Following one round of in-depth review and revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/a2rwg
Level of bias control achieved: Level 4. At least some of the data/evidence that was used to answer the research question existed prior to in-principle acceptance(IPA) but the authors certify that they did not access any part of that data/evidence prior to IPA.
List of eligible PCI RR-friendly journals:
References
1. van Rooij AJ, Ferguson CJ, Carras MC, Kardefelt-Winther D, Shi J, Aarseth E, Bean AM, Bergmark KH, Brus A, Coulson M, Deleuze J, Dullur P, Dunkels E, Edman J, Elson M, Etchells PJ, Fiskaali A, Granic I, Jansz J, Karlsen F, Kaye LK, Kirsh B, Lieberoth A, Markey P, Mills KL, Nielsen RKL, Orben A, Poulsen A, Prause N, Prax P, Quandt T, Schimmenti A, Starcevic V, Stutman G, Turner NE, Looy J van, Przybylski AK (2018) A weak scientific basis for gaming disorder: Let us err on the side of caution. Journal of Behavioral Addictions, 7, 1–9. https://doi.org/10.1556/2006.7.2018.19
2. Karhulahti V-M, Siutila M, Vahlo J, Koskimaa R (2022) Phenomenological Strands for Gaming Disorder and Esports Play: A Qualitative Registered Report. Stage 2 Registered Report, acceptance of Version 2 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/q53jz
| Phenomenological Strands for Gaming Disorder and Esports Play: A Qualitative Registered Report | Veli-Matti Karhulahti, Miia Siutila, Jukka Vahlo, Raine Koskimaa | <p>The recent inclusion of gaming disorder in the ICD-11 as a mental disorder has further increased the importance of researching the health spectrum related to gaming. A critical area in this regard is the lack of clarity concerning the differenc... |  | Medical Sciences, Social sciences | Chris Chambers | 2022-06-11 23:49:03 | ||
16 Sep 2022
STAGE 1
Action-Inaction Asymmetries in Emotions and Counterfactual Thoughts: Meta-Analysis of the Action EffectCharting meta-analytic evidence for the action-effectRecommended by Chris Chambers based on reviews by Dan Quintana, Emiel Cracco and priyali rajagopal
Winston Churchill once famously quipped, “I never worry about action, but only inaction.” Churchill, however, may have been an exception to the rule, with psychological research suggesting that people are more concerned about the consequences of actions than inactions. During the so-called “action-effect”, first reported by Kahneman and Tversky (1982), people regret an action leading to a bad outcome more than they do an inaction leading to the same bad outcome In the current study, Yeung and Feldman (2022) propose a wide-ranging meta-analysis to characterise evidence for the action-effect, focusing in particular on emotions and counterfactual thoughts – that is, mental representations of alternative decisions (or “what if” thoughts). Consistent with the expected consequences of the action-effect on emotion, they predict that action will be associated with stronger negative emotions than inaction (when outcomes are negative), and with stronger positive emotions than inaction (when outcomes are positive). The authors also expect action to be associated with a greater abundance of counterfactual thought compared to inaction. In addition to examining the overall reliability of the action-effect (plus a range of exploratory questions), the study will also examine the extent to which the action-effect is moderated by temporal distance (with more recent events or behaviours predicted to associated with a stronger action effect), the type of study design, prior outcomes and social norms, the specificity (vs. generality) of the prior event, and whether the study employed a hypothetical scenario or a real-life event. The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA). URL to the preregistered Stage 1 protocol: https://osf.io/4pvs6 Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question References | Action-Inaction Asymmetries in Emotions and Counterfactual Thoughts: Meta-Analysis of the Action Effect | Siu Kit Yeung, Gilad Feldman | <p>Action-effect refers to the phenomenon in which people experience, associate, or attribute stronger emotions for action compared to inaction. In this registered report, we conducted a meta-analysis of the action effect literature (k = [enter nu... | Social sciences | Chris Chambers | 2021-07-16 14:02:56 | |||
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportReassessing the use of the Bayesian Truth Serum as an incentive-compatible design for self-reportsRecommended by Ljerka Ostojic based on reviews by Joël van der Weele
Different disciplines and research areas that rely on participants’ self-reports to accrue data on participants’ true preferences are faced with the question to what extent the former can be equated with the latter. Using monetary incentivisation for study participation may influence this relationship, and researchers, especially in economics, have been discussing how to develop and implement incentive-compatible research designs, i.e., those in which the incentivisation yields the best payoff for the participant if they report their true preferences (Hertwig & Ortmann, 2011; Baillon, 2017). The Bayesian Truth Serum, first introduced by Prelec (2004), according to which participants are rewarded based on how surprisingly common their own answers are relative to the actual distribution of answers, has been proposed as a possible incentive-compatible design for survey studies that rely on participants’ self-reports about their true preferences (Schoenegger, 2021).
In this study, Schoenegger and Verheyen (2022) ran a replication of the study by Schoenegger (2021) and assessed whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts. The authors report that the manipulation did not yield a significant difference compared to control conditions, which they interpret as a failure to replicate the original results. At the same time, the authors are careful in drawing conclusions as to the usefulness of the Bayesian Truth Serum for self-report studies using Likert-scale items in general, as they emphasise that smaller effect sizes may be of interest and that the results may differ when different items are used.
The Stage 2 manuscript was evaluated by two reviewers, one of whom reviewed the first Stage 1 submission, and the other one of whom reviewed the manuscript specifically to assess statistical questions.
Following a careful revision by the authors, the recommender judged that the manuscript meets the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance.
List of eligible PCI RR-friendly journals:
References
1. Baillon, A. (2017). Bayesian markets to elicit private information. Proceedings of the National Academy of Science, 114(30), 7985-7962. https://doi.org/10.1073/pnas.1703486114
2. Hertwig, R. & Ortmann (2001). Experimental practices in economics: a methodlogical challenge for psychologists? Behavioral and Brain Sciences, 24(3), 383-403. https://doi.org/10.1037/e683322011-032
3. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
4. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4
5. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 2 Registered Report, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://doi.org/10.31219/osf.io/9zvqj
| Taking A Closer Look At The Bayesian Truth Serum: A Registered Report | Philipp Schoenegger & Steven Verheyen | <p>Over the past decade, psychology and its cognate disciplines have undergone substantial scientific reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has rec... | Social sciences | Ljerka Ostojic | 2022-06-11 14:39:38 | |||
16 Sep 2022
STAGE 1
Taking A Closer Look At The Bayesian Truth Serum: A Registered ReportUnderstanding the key ingredients of the Bayesian Truth SerumRecommended by Ljerka Ostojic based on reviews by 2 anonymous reviewers
The Bayesian Truth Serum, first introduced by Prelec (2004) rewards participants based on how surprisingly common their own answers are in relation to the actual distribution of answers. As such, it has been suggested as a possible incentive-compatible design for survey studies in different disciplines that rely on participants’ self-reports about their true preferences (Schoenegger, 2021).
In this study, Schoenegger and Verheyen propose to replicate the results reported by Schoenegger (2021) and to directly investigate whether the effect elicited by the manipulations known as the Bayesian Truth Serum is distinct from its separate constituent parts.
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers’ comments and edits to the stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/dkvms
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI RR-friendly journals:
References
1. Prelec, D. (2004). A Bayesian Truth Serum for Subjective Data. Science, 306(5695), 462-466. https://doi.org/10.1126/science.1102081
2. Schoenegger, P. (2021). Experimental Philosophy and the Incentivisation Challenge: a Proposed Application of the Bayesian Truth Serum. Review of Philosophy and Psychology https://doi.org/10.1007/s13164-021-00571-4
3. Schoenegger, P., & Verheyen, S. (2022). Taking A Closer Look At The Bayesian Truth Serum: A Registered Report. Stage 1 Registered Report, in principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/dkvms
| Taking A Closer Look At The Bayesian Truth Serum: A Registered Report | Philipp Schoenegger & Steven Verheyen | <p>Over the past decades, psychology and its cognate disciplines have undergone substantial reform, ranging from advances in statistical methodology to significant changes in academic norms. One aspect of experimental design that has received comp... | Social sciences | Ljerka Ostojic | 2021-12-06 17:36:15 |
