Bug detection is central to software engineering, but what motivates programmers to perform as optimally as possible? Despite a long history of economic experiments on incentivisation, there is surprisingly little research on how different incentives shape software engineering performance.

 

In the current study, Bershadskyy et al. (2024) undertook an experiment to evaluate how the pay-off functions associated with different financial incentives influence the performance of participants in identifying bugs during code review. The authors hypothesised that performance-based incentivisation would result in higher average performance, as defined using the F1-score, and that different incentivisation schemes may also differ in their effectiveness.

 

The results did not support the preregistered predictions, with no statistically significant differences in F1-score observed between groups that received performance-based incentives compared to a control group that received no incentive. Exploratory analyses suggested some potential trends of interest, but the main implication of this work is methodological: that experiments in this field require substantially larger sample sizes to provide definitive tests. The current work is valuable in providing a novel unbiased insight on the magnitude of this challenge, which is now primed for further investigation.

 

The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the recommender and reviewer's comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/s36c2

 

Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:

 

 

 

Bershadskyy, D., Krüger, J., Çalıklı, G., Siegmar, O., Zabel, S., Greif, J. and Heyer, R. (2024). A Laboratory Experiment on Using Different Financial-Incentivization Schemes in Software-Engineering Experimentation. Acceptance of Version 8 by Peer Community in Registered Reports. https://arxiv.org/pdf/2202.10985

Measuring people’s experiences, thoughts, and mental processes has always been a core challenge of psychological science (e.g. Nisbett & Wilson 1977). When such measurement further relates to rapidly changing and conceptually diverse human-technology interactions, the task becomes even more difficult due to protean, multidimensional constructs. A good understanding of a construct is a basic step in its measurement (Borsboom 2005).  
 
In the present registered report, Hickman Dunne et al. (2023) carry out a focus group study with adolescents (n=32) aged 11 to 15 in Northwest England to improve the understanding of constructs related to social media and mental health experiences. The work is carried out as part of a long-term measure development project. The authors apply reflexive thematic analysis to explore adolescents’ social media use experiences and related motivations in the light of mental health, in addition to which the adolescents’ own views of benefits and risks are mapped out.
 
A particular strength of the design is the engagement of three Young Researchers who will co-facilitate the focus groups and be involved in the analysis. The research plan also meets high reflexivity and transparency criteria, and as such, can significantly contribute to future scale development as well as our general understanding of adolescents’ social media experiences.
 
The Stage 1 manuscript was reviewed over two rounds by five unique reviewers, one of which participated in both rounds. The reviewers’ expertise ranged from social media and technology use research to health psychology and qualitative methods. Based on careful revisions and detailed responses to the reviewers’ comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.

 

URL to the preregistered Stage 1 protocol: https://osf.io/w24ec
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question.
 
List of eligible PCI RR-friendly journals:

 

• Collabra: Psychology
• F1000Research
• Peer Community Journal
• PeerJ
• Studia Psychologica
• Swiss Psychology Open
• WiderScreen 

 

References
 
1. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge University Press.

 

3. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological review, 84, 231–259. https://doi.org/10.1037/0033-295X.84.3.231

2. Hickman Dunne J., Black L., Banwell E., Nanda P., Anderton M, Butters L.C., Demkowicz O., Davidson B., Qualter P., Humphrey N., Jay C., and Panayiotou M. (2023). Identifying relevant dimensions to the measurement of adolescent social media experience via focus groups with young people: A registered report. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/w24ec

Large language models (LLMs) are being increasingly integrated into institutional settings in multiple different roles, arguably contributing to an ongoing transformation of cultural evolution (Brinkmann et al. 2023). Considering this change in academic institutions across disciplines and countries too, it is important to advance knowledge regarding their usage in such settings. Here, Bergmann and colleagues (2025) tackle this research gap with upcoming user data from psychology students (N=1450) who communicate with a statistics chatbot by OpenAI’s GPT-4o, StatsBot, as part of their studies.
 
The study design is built on a three-step mixed-method process, starting from deductive and inductive content analysis, and concluding with descriptive statistical analysis. The plan was shaped by a detailed pilot with similar StatsBot data (n=85) that enabled crafting a transparent plan for follow-up content analysis. Additional statistical explorations, such as those via network analyses, are promising and can yield valuable insights regarding the functions and utility of LLMs in learning processes. Comparative testing of LLMs and human analysis adds an informative meta-layer to the design.
 
The Stage 1 manuscript was reviewed by four experts, including two from psychology, one from computer science, and one specialized in the technical aspects of LLMs. Two experts returned to provide feedback on the revisions over a second review round. The third revision of the protocol was considered to satisfy Stage 1 criteria and was therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/v6bj2

 

Level of bias control achieved: Level 5. All of the data or evidence that will be used to answer the research question already exist, but are currently inaccessible to the authors and thus unobservable prior to IPA.
 
List of eligible PCI-RR-friendly journals:

 

Studying eye-gaze has long been employed as a central method for understanding attentional dynamics and cognitive processes in a variety of domains. The development of affordable, mobile eye-tracking devices, such as the Pupil Neon, promises new opportunities to extend this research beyond the contexts in which traditional eye-trackers have been available. But how good are such novel devices at detecting variables relevant for the study of eye movements and pupil dilations?

 

Foucher, Krug and Sauter (2024) propose an independent evaluation of the Pupil Neon eye-tracker using the Ehinger et al. (2019) test battery, comparing its performance with a traditional EyeLink 1000 Plus device. In an empirical study, participants will be asked to perform a wide variety of tasks while eye movements and pupil dilations are tracked using both devices. Results on the strengths and weaknesses, as well as potential use cases of the Pupil Neon will be informative for subsequent eye-tracking research. 

The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the recommender and reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).

 

 

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 

 

 

1. Ehinger, B. V., Groß, K., Ibs, I., and König, P. (2019). A New Comprehensive Eye-Tracking Test Battery Concurrently Evaluating the Pupil Labs Glasses and the Eyelink 1000. PeerJ, 7, e7086. https://doi.org/10.7717/peerj.7086

2. Foucher, V., Krug, A., and Sauter, M. (2024). Independent Comparative Evaluation of the Pupil Neon - A New Mobile Eye-tracker. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/3kc5t

 

Borderline personality disorder may be associated with an impaired tendency to discriminate self and other. When people cause tactile sensations to themselves, these sensations are attenuated relative to the same tactile stimulus caused by others. This sensory attenuation is a reflection of the ability to discriminate self and other. Sensory attenuation can be indexed either by reduced ratings of perceived intensity for self versus other-generated stimuli; or by reduced somatosensory evoked potentials for self versus other-generated stimuli. One theory of self-injury in borderline personality disorder is that people with borderline personality disorder who injure themselves (non-suicidally) may regain appropriate self-other discrimination.

Romeo et al. (2025) will test the theory that people with borderline personality disorder without self-injury have an impaired tendency to discriminate self and others by having three groups of people receive small electric shocks delivered either by their own actions or the actions of the experimenter. Perceived intensity and somatosensory evoked potentials will be used to measure sensory attenuation for self versus other-generated actions. The theory predicts that sensory attenuation will be impaired for people with borderline personality disorder without self-injury relative to both a) healthy controls, and b) people with personality disorder with self-injury. This will be an important study for assessing self-other representation in borderline personality disorder.

 

The Stage 1 manuscript was evaluated over six rounds of peer review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and was therefore awarded in-principle acceptance (IPA).

 

URL to the preregistered Stage 1 protocol: https://osf.io/sztek
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals: 

• Advances in Cognitive Psychology
• Brain and Neuroscience Advances
• Cortex
• Imaging Neuroscience
• In&Vertebrates
• NeuroImage: Reports
• Peer Community Journal
• PeerJ
• Personality Science *pending editorial consideration of disciplinary fit
• Psychology of Consciousness: Theory, Research, and Practice
• Royal Society Open Science
• Studia Psychologica

 

Romeo, M., Bortoletto, M., Zazio, A., Colle, L., Zampese, C., Fracassi, C., Rossi, R., Ricciardi, E., Garbarini, F., & Fossataro, C. (2025). Unveiling the neurophysiological blueprint of Sense of Agency in Borderline Personality Disorder: implications for self-other differentiation. In principle acceptance of Version 7 by Peer Community in Registered Reports. https://osf.io/sztek