We are recruiting recommenders (editors) from all research fields!
Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.
Latest recommendations
| Id | Title * | Authors * | Abstract * | Picture | Thematic fields * | Recommender▼ | Reviewers | Submission date | |
|---|---|---|---|---|---|---|---|---|---|
19 Jan 2024
STAGE 1
A systematic review of social connection inventoriesImproving the measurement of social connectionRecommended by Dorothy Bishop based on reviews by Jacek Buczny, Richard James and Alexander Wilson
This is an ambitious systematic review that uses a combination of quantitative and qualitative methods to make the measurement of the construct of social connection more rigorous. Social connection is a heterogeneous construct that includes aspects of structure, function and quality. Here, Paris et al. (2024) will use predefined methods to create a database of social connection measures, and will assess heterogeneity of items using human coders and ChatGPT. This database will form the basis of a second systematic review which will look at evidence for validity and measurement properties. This study will also look at the population groups and country of origin for which different measures were designed, making it possible to see how far culturally specific issues affect the content of measures in this domain.
The questions asked by this study are exploratory and descriptive and so the importance of pre-registration is in achieving clear criteria for how each question is addressed, rather than evidential criteria for hypothesis-testing.
The authors responded comprehensively to three reviewer reports. This study will provide a wealth of useful information for those studying social connection, and should serve to make the literature in this field more psychometrically robust and less fragmented.
URL to the preregistered Stage 1 protocol: https://osf.io/796uv
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence. List of eligible PCI RR-friendly journals:
References
1. Paris, B., Brickau, D., Stoianova, T., Luhmann, M., Mikton, C., Holt-Lunstad, J., Maes, A., & IJzerman, H. (2024). A systematic review of social connection inventories. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/796uv | A systematic review of social connection inventories | Bastien Paris, Debora Brickau, Tetiana Stoianova, Maike Luhmann, Christopher Mikton, Julianne Holt-Lunstad, Marlies Maes, Hans IJzerman | <p>Social connection is vital to health and longevity. To date, a plethora of instruments exists to measure social connection, assessing a variety of aspects of social connection like loneliness, social isolation, or social support. For comparabil... | Social sciences | Dorothy Bishop | Jacek Buczny, Richard James, Alexander Wilson | 2023-07-09 21:33:01 | ||
Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay taskExploring determinants of test-retest reliability in fMRI: a study with the Monetary Incentive Delay TaskRecommended by Dorothy Bishop based on reviews by Xiangzhen Kong and 1 anonymous reviewer
Functional magnetic resonance imaging has been used to explore brain-behaviour relationships for many years, with proliferation of a wide range of sophisticated analytic procedures. However, rather scant attention has been paid to the reliability of findings. Concerns have been growing failures to replicate findings in some fields, but it is hard to know how far this is a consequence of underpowered studies, flexible analytic pipelines, or variability within and between participants. Demidenko et al. (2024) took advantage of the availability of three existing datasets, including the Adolescent Brain Cognitive Development (ABCD) study, the Michigan Longitudinal Study, and the Adolescent Risk Behavior Study, which all included a version of the Monetary Incentive Delay task measured in two sessions. These were entered into a multiverse analysis, which considered how within-subject and between-subject variance varies according to four analytic factors: smoothing (5 levels), motion correction (6 levels), task modelling (3 levels) and task contrasts (4 levels). They also considered how sample size affects estimates of reliability. The results have important implications for the those using fMRI with the Monetary Incentive Delay Task, and also raise questions more broadly about use of fMRI indices to study individual differences. Motion correction had relatively little impact on the ICC, and the effect size of the smoothing kernel was modest. Larger impacts on reliability were associated with choice of contrast (implicit baseline giving larger effects) and task parameterization. But perhaps the most sobering message from this analysis is that although activation maps from group data were reasonably reliable, the ICC, used as an index of reliability for individual levels of activation, was consistently low. This raises questions about the suitability of the Monetary Incentive Delay Task for studying individual differences. Another point is that reliability estimates become more stable as sample size increases; researchers may want to consider whether the trade-off between cost and gain in precision is justified for sample sizes above 250. I did a quick literature search on Web of Science: at the time of writing the search term ("Monetary Delay Task" AND fMRI) yielded 410 returns, indicating that this is a popular method in cognitive neuroscience. The detailed analyses reported here will repay study for those who are planning further research using this task. The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewer's and recommender's comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation. URL to the preregistered Stage 1 protocol: https://osf.io/nqgeh
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to IPA, but the authors certify that they had not yet sufficiently observed the key variables within the data to be able to answer the research questions and they took additional steps to maximise bias control and rigour.
List of eligible PCI RR-friendly journals:
References 1. Demidenko, M. I., Mumford, J. A., & Poldrack, R. A. (2024). Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay task [Stage 2]. Acceptance of Version 5 by Peer Community in Registered Reports. https://www.biorxiv.org/content/10.1101/2024.03.19.585755v4 | Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay task | Michael I. Demidenko, Jeanette A. Mumford, Russell A. Poldrack | <p>Empirical studies reporting low test-retest reliability of individual blood oxygen-level dependent (BOLD) signal estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods... |  | Life Sciences, Social sciences | Dorothy Bishop | 2024-03-21 02:23:30 | ||
20 Mar 2023
STAGE 1
Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird SpeciesWhat is the role of sensory perception in cognitive task performance? An improved replication of detour performance in four different bird speciesRecommended by Dieter Lukas based on reviews by 2 anonymous reviewers
The detour task, where an individual has to navigate around a see-through barrier in order to reach a goal, is one of the oldest paradigms used in animal cognition research (Kabadayi et al. 2018). While these previous tests have documented variation in the ability of animals to inhibit going straight for the visible but blocked reward, the cognitive underpinnings of this behaviour are as yet not fully understood. In the current study, Dewulf et al. (2023) propose to assess one of the specific cognitive processes that might be involved in this behaviour. Through experimental procedures, they will compare the role of signal detection in inhibitory response performance in a detour task. To assess whether variation in detection of the barrier might be linked to the ecological niche of a species, they will compare four bird species who live in different environments. Individuals from these four species were previously tested in a detour task (Regolin et al. 1994, Zucca et al. 2005), and the proposed research plan therefore also involves a partial replication of previous studies on the same issue, but improving some critical aspects.
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
URL to the preregistered Stage 1 protocol: https://osf.io/qvxgh
List of eligible PCI RR-friendly journals:
References
Dewulf, A., Garcia-Co, C., Müller, W., Madden, J.R., Martel, A., Lens, L. & Verbruggen, F. (2023). Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird Species. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/qvxgh
Kabadayi, C., Bobrowicz, K., & Osvath, M. (2018). The detour paradigm in animal cognition. Animal Cognition, 21, 21-35. https://doi.org/10.1007/s10071-017-1152-0
Regolin L, G Vallortigara, and M Zanforlin (1994). Perceptual and motivational aspects of detour behaviour in young chicks. Animal Behaviour 47, 123–131. https://doi.org/https://doi.org/10.1006/anbe.1994.1014
Zucca P, F Antonelli, and G Vallortigara (2005). Detour behaviour in three species of birds: quails (Coturnix sp.), herring gulls (Larus cachinnans) and canaries (Serinus canaria). Animal Cognition, 8, 122–128. https://doi.org/10.1007/s10071-004-0243-x
| Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird Species | Anneleen Dewulf, Clara Garcia-Co, Wendt Müller, Joah R. Madden, An Martel, Luc Lens, & Frederick Verbruggen | <p>Response inhibition, or the stopping of actions, is considered a key component of flexible and adaptive behavior. Across fields,response inhibition is often treated as a unitary cognitive mechanism. However, we propose that response inhibition ... | Social sciences | Dieter Lukas | 2022-11-30 19:25:43 | |||
Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird SpeciesWhat is the role of sensory perception in cognitive task performance? An improved replication of detour performance in four different bird speciesRecommended by Dieter Lukas based on reviews by Christian Nawroth and 1 anonymous reviewer
The detour task, where an individual has to go around a see-through barrier in order to reach a goal, is one of the oldest paradigms used in animal cognition research (Kabadayi et al. 2018). While these previous tests have documented variation in the ability of animals to inhibit going straight for the visible reward, the cognitive underpinnings of this behaviour are as yet not fully understood. In the current study, Dewulf et al. (2025) assessed one of the specific cognitive processes that might be involved in this behaviour, the ability to identify the transparent object as a barrier. Through experimental procedures relying on large samples of individuals from four bird species, they compared the role of signal detection in inhibitory response performance in a detour task. The authors found that, unlike suggested in previous work with these four species (Regolin et al. 1994, Zucca et al. 2005), changing the markings on the barriers to potentially better match those experienced by individuals in their natural environments did not improve their performance. Nevertheless, the detailed further explorations suggest that in order to understand variation in how quickly individuals and species solve the detour task, it is important to consider that different cognitive processes are involved. Their work therefore provides a basis to better understand and further investigate why species might differ in their performance in the detour task.
The Stage 2 manuscript was evaluated over two rounds of in-depth review, the first round consisting of detailed comments from two reviewers and the second round consisting of a close read by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
URL to the preregistered Stage 1 protocol: https://osf.io/qvxgh
List of eligible PCI RR-friendly journals:
References
1. Dewulf, A., Garcia-Co, C., Müller, W., Madden, J.R., Martel, A., Lens, L. & Verbruggen, F. (2025). Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird Species [Stage 2]. Acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/j2k9h
2. Kabadayi, C., Bobrowicz, K., & Osvath, M. (2018). The detour paradigm in animal cognition. Animal Cognition, 21, 21-35. https://doi.org/10.1007/s10071-017-1152-0
3. Regolin, L., Vallortigara, G., & Zanforlin, M. (1995). Object and spatial representations in detour problems by chicks. Animal Behaviour, 49, 195-199. https://doi.org/10.1016/0003-3472(95)80167-7
4. Zucca, P., Antonelli, F., & Vallortigara, G. (2005). Detour behaviour in three species of birds: quails (Coturnix sp.), herring gulls (Larus cachinnans) and canaries (Serinus canaria). Animal Cognition, 8, 122-128. https://doi.org/10.1007/s10071-004-0243-x
| Do Ecological Valid Stop Signals Aid Detour Performance? A Comparison of Four Bird Species | Anneleen Dewulf, Clara Garcia-Co, Wendt Müller, Joah R. Madden, An Martel, Luc Lens, Frederick Verbruggen | <p>Response inhibition, or the stopping of actions, is considered a key component of flexible and adaptive behaviour. Across fields, response inhibition is often treated as a unitary cognitive mechanism. However, we propose that response inhibitio... | Life Sciences, Social sciences | Dieter Lukas | 2024-10-22 14:00:30 | |||
14 Feb 2024
STAGE 1
Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) TemplateExamining the restrictiveness of the PRP-QUANT TemplateRecommended by Daniel Lakens based on reviews by Marjan Bakker and 1 anonymous reviewer
The Psychological Research Preregistration-Quantitative Template has been created in 2022 to provide more structure and detail to preregistrations. The goal of the current study is to test if the PRP-QUANT template indeed provides greater restriction of the flexibility in a study for preregistered hypotheses than other existing templates. This question is important because one concern that has been raised about the practice of preregistration is that the quality of preregistrations is often low. Metascientific research has shown that preregistrations are often of low quality (Bakker et al., 2020), and hypothesis tests from preregistrations are still selectively reported (van den Akker, van Assen, Enting, et al., 2023). It is important to improve the quality of preregistrations, and if a better template can help, it is a cost-effective approach to improve quality if the wider adoption of the better template can be promoted.
In the current study, Spitzer and Mueller (2024) will follow the procedure of a previous meta-scientific study by Heirene et al. (2021). 74 existing preregistrations with the PRP-QUANT template are available, and will be compared with an existing dataset coded by Bakker and colleagues (2020). The sample size is limited, but allows detecting some differences that would be considered large enough to matter, even though there might be smaller differences that would not be detectable based on the currently available sample size. Nevertheless, given that there is a need for improvement, even preliminary data might already be useful to provide tentative recommendations. Restrictiveness will be coded in 23 items, and adherence to or deviations from the preregistration are coded as well. As such deviations are common, the question whether this template reduced the likelihood of deviations is important. Two coders will code all studies.
The study should provide a useful initial evaluation of the PRP-QUANT template, and has the potential to have practical implications if the PRP-QUANT template shows clear benefits. Both authors have declared COI's related to the PRP-QUANT template, making the Registered Report format a fitting approach to prevent confirmation bias from influencing the reported results.
This Stage 1 manuscript was evaluated over two rounds of in-depth review by two expert reviewers and the recommender. After the revisions, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/vhezj
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence. List of eligible PCI RR-friendly journals:
References
1. van den Akker, O. R., van Assen, M. A. L. M., Bakker, M., Elsherif, M., Wong, T. K., & Wicherts, J. M. (2023). Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02277-0
2. Bakker, M., Veldkamp, C. L. S., Assen, M. A. L. M. van, Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020). Ensuring the quality and specificity of preregistrations. PLOS Biology, 18(12), e3000937. https://doi.org/10.1371/journal.pbio.3000937
3. Spitzer, L. & Mueller, S. (2024). Stage 1 Registered Report: Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/vhezj
4. Heirene, R., LaPlante, D., Louderback, E. R., Keen, B., Bakker, M., Serafimovska, A., & Gainsbury, S. M. (2021). Preregistration specificity & adherence: A review of preregistered gambling studies & cross-disciplinary comparison. PsyArXiv. https://doi.org/10.31234/osf.io/nj4es
| Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template | Lisa Spitzer & Stefanie Mueller | <p>Preregistration can help to restrict researcher degrees of freedom and thereby ensure the integrity of research findings. However, its ability to restrict such flexibility depends on whether researchers specify their study plan in sufficient de... | | Social sciences | Daniel Lakens | 2023-06-01 10:39:20 | ||
29 Sep 2021
STAGE 1
Evaluating the pedagogical effectiveness of study preregistration in the undergraduate dissertation: A Registered ReportDoes incorporating open research practices into the undergraduate curriculum decrease questionable research practices?Recommended by Corina Logan and Chris Chambers based on reviews by Kelsey McCune, Neil Lewis, Jr., Lisa Spitzer and 1 anonymous reviewer
In a time when open research practices are becoming more widely used to combat questionable research practices (QRPs) in academia, this Stage 1 Registered Report by Pownall and colleagues (2021) will empirically investigate the practice of preregistering study plans, which will allow us to better understand to what degree such practices increase awareness of QRPs and whether experience with preregistration helps reduce engagement in QRPs. This investigation is timely because results from these kinds of studies are only recently becoming available and the conclusions are providing evidence that open research practices can improve research quality and reliability (e.g., Soderberg et al. 2020, Chambers & Tzavella 2021). The authors crucially focus on the effect of preregistering the undergraduate senior thesis (of psychology students in the UK), which is a key stage in the development of an academic. This data will help shape the future of how we should teach open research practices and what effect we as teachers can have on budding research careers. The five expert peer reviews were of an extremely high quality and were very thorough. The authors did an excellent job of addressing all of the comments in their responses and revised manuscript versions, which resulted in only one round of peer review, plus a second revision based on Recommender feedback. As such, this registered report meets the Stage 1 criteria and is therefore awarded in-principle acceptance (IPA). We wish the authors the best of luck with the study and we look forward to seeing the results. URL to the preregistered Stage 1 protocol: https://osf.io/9hjbw Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
| Evaluating the pedagogical effectiveness of study preregistration in the undergraduate dissertation: A Registered Report | Madeleine Pownall; Charlotte R. Pennington; Emma Norris; Kait Clark | <p style="text-align: justify;">Research shows that questionable research practices (QRPs) are present in undergraduate final-year dissertation projects. One entry-level Open Science practice proposed to mitigate QRPs is ‘study preregistration’, t... | Life Sciences, Social sciences | Corina Logan | 2021-07-08 15:27:24 | |||
21 Mar 2023
STAGE 1
Convenience Samples and Measurement Equivalence in Replication ResearchDoes data from students and crowdsourced online platforms measure the same thing? Determining the external validity of combining data from these two types of subjectsRecommended by Corina Logan based on reviews by Benjamin Farrar and Shinichi Nakagawa
Comparative research is how evidence is generated to support or refute broad hypotheses (e.g., Pagel 1999). However, the foundations of such research must be solid if one is to arrive at the correct conclusions. Determining the external validity (the generalizability across situations/individuals/populations) of the building blocks of comparative data sets allows one to place appropriate caveats around the robustness of their conclusions (Steckler & McLeroy 2008).
In this registered report, Alley and colleagues plan to tackle the external validity of comparative research that relies on subjects who are either university students or participating in experiments via an online platform (Alley et al. 2023). They will determine whether data from these two types of subjects have measurement equivalence - whether the same trait is measured in the same way across groups. Although they use data from studies involved in the Many Labs replication project to evaluate this question, their results will be of crucial importance to other comparative researchers whose data are generated from these two sources (students and online crowdsourcing). If Alley and colleagues show that these two types of subjects have measurement equivalence, then this indicates that it is more likely that equivalence could hold for other studies relying on these type of subjects as well. If measurement equivalence is not found, then it is a warning to others to evaluate their experimental design to improve validity. In either case, it gives researchers a way to test measurement equivalence for themselves because the code is well annotated and openly available for others to use.
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA). Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour (e.g. conservative statistical threshold; recruitment of a blinded analyst; robustness testing, multiverse/specification analysis, or other approach)
List of eligible PCI RR-friendly journals:
References
Alley L. J., Axt, J., & Flake J. K. (2023). Convenience Samples and Measurement Equivalence in Replication Research, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/7gtvf
Steckler, A. & McLeroy, K. R. (2008). The importance of external validity. American Journal of Public Health 98, 9-10. https://doi.org/10.2105/AJPH.2007.126847
Pagel, M. (1999). Inferring the historical patterns of biological evolution. Nature, 401, 877-884. https://doi.org/10.1038/44766
| Convenience Samples and Measurement Equivalence in Replication Research | Lindsay J. Alley, Jordan Axt, Jessica Kay Flake | <p>A great deal of research in psychology employs either university student or online crowdsourced convenience samples (Chandler & Shapiro, 2016; Strickland & Stoops, 2019) and there is evidence that these groups differ in meaningful ways ... | Social sciences | Corina Logan | 2022-11-29 18:37:54 | |||
Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate DissertationIncorporating open research practices into the undergraduate curriculum increases understanding of such practicesRecommended by Corina Logan based on reviews by Kelsey McCune, Neil Lewis, Jr., Lisa Spitzer and 1 anonymous reviewer
In a time when open research practices are becoming more widely used to combat questionable research practices (QRPs) in academia, this Registered Report by Pownall and colleagues (2023) empirically investigated the practice of preregistering study plans, which allows us to better understand to what degree such practices increase awareness of QRPs and whether experience with preregistration helps reduce engagement in QRPs. This investigation is timely because results from these kinds of studies are only recently becoming available and the conclusions are providing evidence that open research practices can improve research quality and reliability (e.g., Soderberg et al. 2021, Chambers & Tzavella 2022). The authors crucially focused on the effect of preregistering the undergraduate senior thesis (of psychology students in the UK), which is a key stage in the development of an academic.
Pownall and colleagues found that preregistration did not affect attitudes toward QRPs, but it did improve student understanding of open research practices. Using exploratory analyses, they additionally found that those who preregistered were those students who reported that they had more opportunity, motivation, and greater capability. This shows how important it is to incorporate the teaching of open research practices such that students can increase their capability, motivation, and opportunity to pursue such practices, whether it is preregistration or other practices that are better known to reduce QRPs (such as registered reports; Krypotos et al. 2022).
After four rounds of review and revisions, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/9hjbw
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
List of eligible PCI RR-friendly journals:
References
1. Chambers C. D. & Tzavella, L. (2022). The past, present, and future of Registered Reports. Nature Human Behaviour, 6, 29-42. https://doi.org/10.1038/s41562-021-01193-7
2. Krypotos, A. M., Mertens, G., Klugkist, I., & Engelhard, I. M. (2022). Preregistration: Definition, advantages, disadvantages, and how it can help against questionable research practices. In Avoiding Questionable Research Practices in Applied Psychology (pp. 343-357). Cham: Springer International Publishing.
3. Pownall, M., Pennington, C. R., Norris, E., Juanchich, M., Smaile, D., Russell, S., Gooch, D., Rhys Evans, T., Persson, S., Mak, M. H. C., Tzavella, L., Monk, R., Gough, T., Benwell, C. S. Y., Elsherif, M., Farran, E., Gallagher-Mitchell, T., Kendrick, L. T., Bahnmueller, J., Nordmann, E., Zaneva, M., Gilligan-Lee, K., Bazhydai, M., Jones, A., Sedgmond, J., Holzleitner, I., Reynolds, J., Moss, J., Farrelly, D., Parker, A. J. & Clark, K. (2023). Evaluating the pedagogical effectiveness of study preregistration in the undergraduate dissertation [Stage 2 Registered Report], acceptance of Version 4 by Peer Community in Registered Reports. https://psyarxiv.com/xg2ah
4. Soderberg C. K., Errington T. M., Schiavone S. R., Bottesini J., Thorn F. S., Vazire S., Esterling K. M. & Nosek B. A. (2021) Initial evidence of research quality of registered reports compared with the standard publishing model. Nature Human Behaviour, 5, 990–997. https://doi.org/10.1038/s41562-021-01142-4
| Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation | Madeleine Pownall, Charlotte R. Pennington, Emma Norris, Marie Juanchich, David Smaile, Sophie Russell, Debbie Gooch, Thomas Rhys Evans, Sofia Persson, Matthew HC Mak, Loukia Tzavella, Rebecca Monk, Thomas Gough, Christopher SY Benwell, Mahmoud El... | <p>Research shows that questionable research practices (QRPs) are present in undergraduate final-year dissertation projects. One entry-level Open Science practice proposed to mitigate QRPs is ‘study preregistration’, through which researchers outl... | Life Sciences, Social sciences | Corina Logan | 2023-03-25 11:38:54 | |||
Convenience Samples and Measurement Equivalence in Replication ResearchData from students and crowdsourced online platforms do not often measure the same thingRecommended by Corina Logan based on reviews by Benjamin Farrar and Shinichi Nakagawa
Comparative research is how evidence is generated to support or refute broad hypotheses (e.g., Pagel 1999). However, the foundations of such research must be solid if one is to arrive at the correct conclusions. Determining the external validity (the generalizability across situations/individuals/populations) of the building blocks of comparative data sets allows one to place appropriate caveats around the robustness of their conclusions (Steckler & McLeroy 2008). In the current study, Alley and colleagues (2023) tackled the external validity of comparative research that relies on subjects who are either university students or participating in experiments via an online platform. They determined whether data from these two types of subjects have measurement equivalence - whether the same trait is measured in the same way across groups. Although they use data from studies involved in the Many Labs replication project to evaluate this question, their results are of crucial importance to other comparative researchers whose data are generated from these two sources (students and online crowdsourcing). The authors show that these two types of subjects do not often have measurement equivalence, which is a warning to others to evaluate their experimental design to improve validity. They provide useful recommendations for researchers on how to to implement equivalence testing in their studies, and they facilitate the process by providing well annotated code that is openly available for others to use. After one round of review and revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation. URL to the preregistered Stage 1 protocol: https://osf.io/7gtvf
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to Stage 1 IPA, but the authors certify that they had not yet observed the key variables within the data that were used to answer the research question AND they took additional steps to maximise bias control and rigour.
List of eligible PCI RR-friendly journals:
References
1. Pagel, M. (1999). Inferring the historical patterns of biological evolution. Nature, 401, 877-884. https://doi.org/10.1038/44766
2. Steckler, A. & McLeroy, K. R. (2008). The importance of external validity. American Journal of Public Health 98, 9-10. https://doi.org/10.2105/AJPH.2007.126847
3. Alley L. J., Axt, J., & Flake J. K. (2023). Convenience Samples and Measurement Equivalence in Replication Research [Stage 2 Registered Report] Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/s5t3v
| Convenience Samples and Measurement Equivalence in Replication Research | Lindsay J. Alley, Jordan Axt, Jessica Kay Flake | <p>A great deal of research in psychology employs either university student or online crowdsourced convenience samples (Chandler & Shapiro, 2016; Strickland & Stoops, 2019) and there is evidence that these groups differ in meaningful ways ... | Social sciences | Corina Logan | Alison Young Reusser | 2023-08-31 20:26:43 | ||
30 May 2024
STAGE 1
The role of resource dynamics in the distribution of life cycles within a female human populationAn agent-based model of the role of resource dynamics and the environment in human female life cyclesRecommended by Claudio Tennie based on reviews by Cecilia Padilla-Iglesias and 1 anonymous reviewer
Among primates, the human female life cycle appears special. Aspects of these life cycles have been linked to the acquisition and distribution of resources and to environmental factors, as well as to individual differences across human females. Many questions remain regarding the causal roles that these (or also other) factors might have played in the evolution of human female life cycles – and also whether generalizing statements about these life cycles can adequately capture the wide range of the observed phenomena.
In the current study, Varas Enriquez et al. (2024) outline a plan for an agent-based model approach to study the factors that guide and channel variability in female life cycles in humans (within biological constraints), via the effects that their model will capture. The authors’ model has a particular eye towards the effects of resource dynamics (resource production and resource transfers) and environmental conditions – and their interplay. The results of this agent based model will be thoroughly analysed to better understand the evolution of the specific female human life cycle range.
The study plan was refined after one round of review, which led to input from two external reviewers and the recommender. The revised (second) version was judged to satisfy the Stage 1 criteria for in-principle acceptance.
URL to the preregistered Stage 1 protocol: https://osf.io/24c7z
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour (e.g. conservative statistical threshold; recruitment of a blinded analyst; robustness testing, multiverse/specification analysis, or other approach) List of eligible PCI RR-friendly journals: References
Varas Enríquez, P. J., Lukas, D., Colleran, H, Mulder, M. B., & Redhead, D. (2024) The role of resource dynamics in the distribution of life cycles within a female human population. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/24c7z
| The role of resource dynamics in the distribution of life cycles within a female human population | Pablo J. Varas Enríquez, Daniel Redhead, Monique Borgerhoff Mulder, Heidi Colleran, Dieter Lukas | <p>The evolution of the female human life cycle, which is characterised by having a reproductive career nested within juvenile and post-reproductive periods, has been linked to the surplus of adult resource production and downwards inter-generatio... | | Life Sciences, Medical Sciences, Social sciences | Claudio Tennie | 2023-11-13 15:45:52 |
