SPITZER Lisa
- Study Planning, Data Collection, and Data Analysis Services, Leibniz Institute for Psychology (ZPID), Trier, Germany
- Social sciences
Recommendations: 0
Reviews: 3
Areas of expertise
psychology, meta-research, open science, preregistration, eye-tracking
Reviews: 3
24 Sep 2024
STAGE 1
Independent Comparative Evaluation of the Pupil Neon - A New Mobile Eye-tracker
Assessing the Promise of Affordable, Mobile Eye-Tracking Devices: Evaluation of the Pupil Neon
Recommended by Rima-Maria Rahal based on reviews by Lisa Spitzer and Benedikt EhingerStudying eye-gaze has long been employed as a central method for understanding attentional dynamics and cognitive processes in a variety of domains. The development of affordable, mobile eye-tracking devices, such as the Pupil Neon, promises new opportunities to extend this research beyond the contexts in which traditional eye-trackers have been available. But how good are such novel devices at detecting variables relevant for the study of eye movements and pupil dilations?
Foucher, Krug and Sauter (2024) propose an independent evaluation of the Pupil Neon eye-tracker using the Ehinger et al. (2019) test battery, comparing its performance with a traditional EyeLink 1000 Plus device. In an empirical study, participants will be asked to perform a wide variety of tasks while eye movements and pupil dilations are tracked using both devices. Results on the strengths and weaknesses, as well as potential use cases of the Pupil Neon will be informative for subsequent eye-tracking research.
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the recommender and reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/3kc5t
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI RR-friendly journals:
- Collabra: Psychology
- Communications in Kinesiology
- Peer Community Journal
- PeerJ
- Royal Society Open Science
- Studia Psychologica
- Swiss Psychology Open
References:
1. Ehinger, B. V., Groß, K., Ibs, I., and König, P. (2019). A New Comprehensive Eye-Tracking Test Battery Concurrently Evaluating the Pupil Labs Glasses and the Eyelink 1000. PeerJ, 7, e7086. https://doi.org/10.7717/peerj.7086
2. Foucher, V., Krug, A., and Sauter, M. (2024). Independent Comparative Evaluation of the Pupil Neon - A New Mobile Eye-tracker. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/3kc5t
Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation
Incorporating open research practices into the undergraduate curriculum increases understanding of such practices
Recommended by Corina Logan based on reviews by Kelsey McCune, Neil Lewis, Jr., Lisa Spitzer and 1 anonymous reviewerIn a time when open research practices are becoming more widely used to combat questionable research practices (QRPs) in academia, this Registered Report by Pownall and colleagues (2023) empirically investigated the practice of preregistering study plans, which allows us to better understand to what degree such practices increase awareness of QRPs and whether experience with preregistration helps reduce engagement in QRPs. This investigation is timely because results from these kinds of studies are only recently becoming available and the conclusions are providing evidence that open research practices can improve research quality and reliability (e.g., Soderberg et al. 2021, Chambers & Tzavella 2022). The authors crucially focused on the effect of preregistering the undergraduate senior thesis (of psychology students in the UK), which is a key stage in the development of an academic.
Pownall and colleagues found that preregistration did not affect attitudes toward QRPs, but it did improve student understanding of open research practices. Using exploratory analyses, they additionally found that those who preregistered were those students who reported that they had more opportunity, motivation, and greater capability. This shows how important it is to incorporate the teaching of open research practices such that students can increase their capability, motivation, and opportunity to pursue such practices, whether it is preregistration or other practices that are better known to reduce QRPs (such as registered reports; Krypotos et al. 2022).
After four rounds of review and revisions, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/9hjbw
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
List of eligible PCI RR-friendly journals:
- Advances in Cognitive Psychology
- Advances in Methods and Practices in Psychological Science
- Cambridge Educational Research e-Journal
- Collabra: Psychology
- Experimental Psychology
- F1000Research
- Meta-Psychology
- Peer Community Journal
- PeerJ
- Royal Society Open Science
- Studia Psychologica
- Swiss Psychology Open
References
1. Chambers C. D. & Tzavella, L. (2022). The past, present, and future of Registered Reports. Nature Human Behaviour, 6, 29-42. https://doi.org/10.1038/s41562-021-01193-7
2. Krypotos, A. M., Mertens, G., Klugkist, I., & Engelhard, I. M. (2022). Preregistration: Definition, advantages, disadvantages, and how it can help against questionable research practices. In Avoiding Questionable Research Practices in Applied Psychology (pp. 343-357). Cham: Springer International Publishing.
3. Pownall, M., Pennington, C. R., Norris, E., Juanchich, M., Smaile, D., Russell, S., Gooch, D., Rhys Evans, T., Persson, S., Mak, M. H. C., Tzavella, L., Monk, R., Gough, T., Benwell, C. S. Y., Elsherif, M., Farran, E., Gallagher-Mitchell, T., Kendrick, L. T., Bahnmueller, J., Nordmann, E., Zaneva, M., Gilligan-Lee, K., Bazhydai, M., Jones, A., Sedgmond, J., Holzleitner, I., Reynolds, J., Moss, J., Farrelly, D., Parker, A. J. & Clark, K. (2023). Evaluating the pedagogical effectiveness of study preregistration in the undergraduate dissertation [Stage 2 Registered Report], acceptance of Version 4 by Peer Community in Registered Reports. https://psyarxiv.com/xg2ah
4. Soderberg C. K., Errington T. M., Schiavone S. R., Bottesini J., Thorn F. S., Vazire S., Esterling K. M. & Nosek B. A. (2021) Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour, 5, 990–997. https://doi.org/10.1038/s41562-021-01142-4
29 Sep 2021
STAGE 1
Evaluating the pedagogical effectiveness of study preregistration in the undergraduate dissertation: A Registered Report
Does incorporating open research practices into the undergraduate curriculum decrease questionable research practices?
Recommended by Corina Logan and Chris Chambers based on reviews by Kelsey McCune, Neil Lewis, Jr., Lisa Spitzer and 1 anonymous reviewerIn a time when open research practices are becoming more widely used to combat questionable research practices (QRPs) in academia, this Stage 1 Registered Report by Pownall and colleagues (2021) will empirically investigate the practice of preregistering study plans, which will allow us to better understand to what degree such practices increase awareness of QRPs and whether experience with preregistration helps reduce engagement in QRPs. This investigation is timely because results from these kinds of studies are only recently becoming available and the conclusions are providing evidence that open research practices can improve research quality and reliability (e.g., Soderberg et al. 2020, Chambers & Tzavella 2021). The authors crucially focus on the effect of preregistering the undergraduate senior thesis (of psychology students in the UK), which is a key stage in the development of an academic. This data will help shape the future of how we should teach open research practices and what effect we as teachers can have on budding research careers. The five expert peer reviews were of an extremely high quality and were very thorough. The authors did an excellent job of addressing all of the comments in their responses and revised manuscript versions, which resulted in only one round of peer review, plus a second revision based on Recommender feedback. As such, this registered report meets the Stage 1 criteria and is therefore awarded in-principle acceptance (IPA). We wish the authors the best of luck with the study and we look forward to seeing the results.
URL to the preregistered Stage 1 protocol: https://osf.io/9hjbw
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI RR-friendly journals:
- Advances in Methods and Practices in Psychological Science
- Cambridge Educational Research e-Journal
- F1000Research
- Meta-Psychology
- PeerJ
- Royal Society Open Science
- Swiss Psychology Open
References
- Pownall M, Pennington CR, Norris E, Clark K. 2021. Evaluating the pedagogical effectiveness of study preregistration in the undergraduate dissertation: A Registered Report. OSF, stage 1 preregistration, in principle acceptance of version 1 by Peer Community in Registered Reports. https://doi.org/10.17605/OSF.IO/9HJBW
- Chambers C, Tzavella L (2021). The past, present, and future of Registered Reports. https://doi.org/10.31222/osf.io/43298
- Soderberg CK, Errington TM, Schiavone SR, Bottesini J, Thorn FS, Vazire S, Esterling KM, Nosek BA (2021) Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour, 5, 990–997. https://doi.org/10.1038/s41562-021-01142-4