Announcements
Please note that we will be CLOSED to ALL SUBMISSIONS from 1 December 2024 through 12 January 2025 to give our recommenders and reviewers a holiday break.
We are recruiting recommenders (editors) from all research fields!
Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.
221 records found
Latest recommendations
Id | Title * | Authors * | Abstract * | Picture | Thematic fields * | Recommender | Reviewers | Submission date▲ | |
---|---|---|---|---|---|---|---|---|---|
28 Sep 2023
STAGE 1
Investigating the barriers and enablers to data sharing behaviours: A qualitative Registered ReportEmma L Henderson, Afrodita Marcu, Lou Atkins, Emily K Farran https://osf.io/2gm5sCapability, Opportunity, and Motivation in Data Sharing BehaviourRecommended by Veli-Matti Karhulahti based on reviews by Moin Syed, Peter Branney and Libby BishopIn the past two decades, most academic fields have witnessed an open science revolution that has led to significant increases in open access publishing, reproducibility efforts, and scientific transparency in general (e.g., Spellman et al. 2018). One of the key areas in this ongoing change is data sharing. Although some evidence already points at progress in data sharing practices, many new datasets remain unshared (see Tedersoo et al. 2021).
In the present registered report, Henderson et al. (2023) empirically explore the factors that either hinder or facilitate data sharing in the UK. By means of semi-structured interviews, the team will chart researchers’ experiences of sharing and non-sharing. Thematic template analysis will be applied to organise the data into a hierarchical map of capabilities, opportunities, and motivations in a theoretical domains framework (COM-B-TDF). The research plan itself meets the highest open science standards and reflects on the authors own positions, from which the current qualitative interview data sharing efforts will be made. The Stage 1 manuscript was reviewed over three rounds by three experts with familiary of the UK cultural context and specializations in open science practices, qualitative research, and data infrastructures. Based on careful revisions and detailed responses to the reviewers’ comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance. URL to the preregistered Stage 1 protocol: https://osf.io/2gm5s (under temporary private embargo) Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Henderson, E., Marcu, A., Atkins, L. & Farran, E.K. (2023). Investigating the barriers and enablers to data sharing behaviours: A qualitative Registered Report. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/2gm5s 2. Spellman, B. A., Gilbert, E. A. & Corker, K. S. (2018). Open Science. Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience, 5, 1-47. https://doi.org/10.1002/9781119170174.epcn519
3. Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä., ... & Sepp, T. (2021). Data sharing practices and data availability upon request differ across scientific disciplines. Scientific data, 8, 192. https://doi.org/10.1038/s41597-021-00981-0
| Investigating the barriers and enablers to data sharing behaviours: A qualitative Registered Report | Emma L Henderson, Afrodita Marcu, Lou Atkins, Emily K Farran | <p>Data sharing describes the process of making research data available for reuse. The availability of research data is the basis of transparent, effective research systems that democratise access to knowledge and advance discovery. Despite a broa... | Social sciences | Veli-Matti Karhulahti | 2023-05-11 19:18:48 | View | ||
Life Thinning and Gaming Disorder: A Longitudinal Qualitative Registered ReportVeli-Matti Karhulahti, Miia Siutila, Jukka Vahlo, Raine Koskimaa https://doi.org/10.31234/osf.io/rfbcuHow do intensive gaming experiences evolve over time in clinical and non-clinical contexts?Recommended by Chris Chambers based on reviews by Peter Branney and Michelle CarrasOver the last 5 years the inclusion of “gaming disorder” in the ICD-11 been controversial (Van Rooij et al, 2018), mirroring wider public debate about the effects of gaming on mental health. One of the major gaps in understanding the validity of gaming disorder as an identifiable mental illness is the absence of qualitative studies comparing the lived experience of gamers who seek treatment with esports players who do not report health problems.
Here, Karhulahti et al. (2023) tackle this question in the second of two Stage 2 Registered Reports associated with their previous programmatic Stage 1 submission. Using interpretative phenomenological analysis, the authors undertook in-depth interviews over a 1-year period with treatment-seeking participants (N=5) and esports-playing participants (N=4) who did not experience gaming-related health problems. The authors sought to answer the folllwing primary question: How do the experiences and meanings of playing videogames—shaped by the individuals’ diverse sociocultural contexts—evolve in those with related health problems (as defined by treatment-seeking) and those who play esports games several hours per day while self-reporting no related health problems?
Both groups exhibited intense relationships with gaming that were cyclical over time across various dimensions, with fluctuations occurring in response to changes in health, occupation, and social networks. The observed variation over time was substantial, with individuals attaching and detaching from games involving hundreds or thousands of hours. The authors report treatment-seeking being followed by a search of new gaming and life meanings, while intensive gaming without related problems continued as an integrated part of the self, with resilience adapting and evolving in the face of unexpected life events. Taking into account their findings, the authors propose life thinning and resilience integration processes to help describe and explain how some individuals end up seeking treatment for their gaming, while for others gaming supports them and becomes integrated into their identity.
Following one round of in-depth review and revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/a2rwg
Level of bias control achieved: Level 4. At least some of the data/evidence that was used to answer the research question existed prior to in-principle acceptance(IPA) but the authors certify that they did not access any part of that data/evidence prior to IPA.
List of eligible PCI RR-friendly journals:
References
1. van Rooij AJ, Ferguson CJ, Carras MC, Kardefelt-Winther D, Shi J, Aarseth E, Bean AM, Bergmark KH, Brus A, Coulson M, Deleuze J, Dullur P, Dunkels E, Edman J, Elson M, Etchells PJ, Fiskaali A, Granic I, Jansz J, Karlsen F, Kaye LK, Kirsh B, Lieberoth A, Markey P, Mills KL, Nielsen RKL, Orben A, Poulsen A, Prause N, Prax P, Quandt T, Schimmenti A, Starcevic V, Stutman G, Turner NE, Looy J van, Przybylski AK (2018) A weak scientific basis for gaming disorder: Let us err on the side of caution. Journal of Behavioral Addictions, 7, 1–9. https://doi.org/10.1556/2006.7.2018.19
2. Karhulahti V-M, Siutila M, Vahlo J, Koskimaa R (2023). Life Thinning and Gaming Disorder: A Longitudinal Qualitative Registered Report [Stage 2 Registered Report], acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/hmcqz
| Life Thinning and Gaming Disorder: A Longitudinal Qualitative Registered Report | Veli-Matti Karhulahti, Miia Siutila, Jukka Vahlo, Raine Koskimaa | <p>The academic debates regarding the psychiatric relevance of gaming disorder continue largely because the lived experiences of treatment-seekers remain mostly unstudied. This registered report addresses the above research gap with a longitudinal... | Humanities, Medical Sciences, Social sciences | Chris Chambers | 2023-05-15 11:01:37 | View | ||
Globally, songs and instrumental melodies are slower, higher, and use more stable pitches than speech [Stage 2 Registered Report]Yuto Ozaki, Adam Tierney, Peter Pfordresher, John Mcbride, Emmanouil Benetos, Polina Proutskova, Gakuto Chiba, Fang Liu, Nori Jacoby, Suzanne Purdy, Patricia Opondo, Tecumseh Fitch, Shantala Hegde, Martín Rocamora, Rob Thorne, Florence Ewomazino Nweke, Dhwani Sadaphal, Parimal Sadaphal, Shafagh Hadavi, Shinya Fujii, Sangbuem Choo, Marin Naruse, Utae Ehara, Latyr Sy, Mark Lenini Parselelo, Manuel Anglada-Tort, Niels Chr. Hansen, Felix Haiduk, Ulvhild Færøvik, Violeta Magalhães, Wojciech Krzyżanow... https://doi.org/10.31234/osf.io/jr9x7Strong evidence for cross-cultural regularities in music and speechRecommended by Chris Chambers based on reviews by Bob Slevc and Nai DingFor centuries, the ubiquity of language and music across human societies has prompted scholars to speculate about their cross-cultural origins as well as their shared and unique characteristics. Depending on the extent to which contemporary theories emphasise the role of biology vs. culture, a range of hypotheses have been proposed concerning expected similarities and differences in song and speech. One class of hypotheses stemming from cultural relativism assumes a lack of universal regularities in song and speech, and therefore predicts no systematic cross-cultural relationships. On the other hand, more recent evolutionary hypotheses such as the social bonding hypothesis, motor constraint hypothesis, and sexual selection hypothesis all predict differences or similarities in specific characteristic of vocalisations, such as pitch regularity, pitch interval size, and melodic contour. Existing results are mixed in their support of these predictions.
In the current study, Ozaki et al. (2023) elucidated cross-cultural similarities and differences between speech and song in 75 different linguistic varieties spanning 21 language families. Understanding precisely how song and speech are related is methodologically challenging due to the multitude of confounds that can arise in comparing natural recordings. Here the authors overcame these difficulties with four types of carefully controlled recordings: singing, recitation of sung lyrics, spoken description of the song, and instrumental version of the sung melody. The authors then examined six features that are amenable to reliable comparison, including pitch height, temporal rate, pitch stability, timbral brightness, pitch interval size, and pitch declination. With this data in hand, the authors asked which acoustic features differ reliably between song and speech across cultures, with the expectation that song would exhibit higher pitch, slower rate and more stable pitch than speech. At the same time, the authors expected song and speech to be reliably similar in the characteristics of timbral brightness, pitch intervals and pitch contours
The findings provided strong support for the preregistered hypotheses. Relative to speech, songs exhibited higher pitch, slower temporal rate, and more stable pitches, while both songs and speech had similar pitch interval size and timbral brightness. Only one hypothesis was unsupported, with the comparison of pitch declination between song and speech turning out inconclusive. To overcome potential sources of analytic bias, the authors undertook additional robustness checks, including reanalysis of a previously published dataset of over 400 song/speech recordings; this exploratory analysis corroborated the conclusions from the confirmatory analysis. Overall this study offers a unique insight into the shared global characteristics of langage and music, with implications for understanding their cultural and biological (co)evolution.
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and therefore awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/jdhtz
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors, but the authors certify that they had not yet observed the key variables within the data that were be used to answer the research question AND they took additional steps to maximise bias control and rigour.
List of eligible PCI RR-friendly journals: References
1. Ozaki, Y. et al. (2023). Globally, songs and instrumental melodies are slower, higher, and use more stable pitches than speech [Stage 2 Registered Report]. Acceptance of Version 11 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/jr9x7
| Globally, songs and instrumental melodies are slower, higher, and use more stable pitches than speech [Stage 2 Registered Report] | Yuto Ozaki, Adam Tierney, Peter Pfordresher, John Mcbride, Emmanouil Benetos, Polina Proutskova, Gakuto Chiba, Fang Liu, Nori Jacoby, Suzanne Purdy, Patricia Opondo, Tecumseh Fitch, Shantala Hegde, Martín Rocamora, Rob Thorne, Florence Ewomazino N... | <p>What, if any, similarities and differences between music and speech are consistent across cultures? Both music and language are found in all known human societies and are argued to share evolutionary roots and cognitive resources, yet no studie... | Humanities, Social sciences | Chris Chambers | 2023-05-16 17:22:17 | View | ||
02 Jun 2024
STAGE 1
Mapping Cross-Disciplinary Perspectives on Responsible Conduct of Research: A Delphi StudySarahanne M. Field, Jackie Thompson, Tom van Drimmelen, Jennifer Ferrar, Bart Penders, Sarah de Rijcke, and Marcus R. Munafò https://osf.io/6hqysCapturing Perspectives on Responsible Research Practice: A Delphi StudyRecommended by Charlotte Pennington and Maanasa Raghavan based on reviews by Moin Syed, Veli-Matti Karhulahti, Thomas Evans, Priya Silverstein and Sean GrantThe responsible conduct of research (RCR) is crucial for the health of the research ecosystem: high quality research should lead to more credible findings and increase public trust. However, the dimensions and responsibilities that make up RCR differ across disciplines, who together can learn from one another to ensure rigorous, transparent, and reliable research and foster healthier research culture.
Bridging this gap, in their Stage 1 Registered Report, Field and colleagues (2024) outline their plans for a large-scale Delphi study to evaluate academics' perceived levels of importance of the most crucial elements of RCR and how these align and differ across disciplines. First, they plan to assemble a Delphi panel of RCR experts across multiple disciplines who will evaluate a list of RCR dimensions to suggest any additions. Then, these same panellists will judge each RCR dimension on its importance within their discipline of expertise, with iterative rounds of ratings until stability is reached. In this latter phase, the goal is to probe which items are more broadly appreciated by the sample (i.e., those that are perceived as a universally valuable RCR practice), versus which might be more discipline specific. The findings will present the median importance ratings and categories of response agreement across the entire panel and between different disciplines. Finally, to contextualise these findings, the team will analyse qualitative findings from open-ended text responses with a simple form of thematic analysis. From this, the team will develop a framework, using the identified RCR dimensions, that reflects the needs of the academic community.
By mapping a broader multidisciplinary perspective on RCR, this research will fill the gap between the two extremes that existing conceptualisations of RCR tend to fall under: high-level frameworks designed to be universally applicable across all disciplines (e.g., the Singapore Statement on Research Integrity) and prescriptive guides tailored to the practical instruction of researchers within a specific discipline or field (e.g., RCR training designed for members of a university department). The hope is that this will stimulate a more nuanced understanding and discussion of cross-disciplinary conceptions of RCR.
Five expert reviewers with field expertise assessed the Stage 1 manuscript over two rounds of in-depth review. Based on detailed and informed responses to the reviewer’s comments, the recommenders judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/xmnu5 Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
Field, S. M., Thompson, J., van Drimmelen, T., Ferrar, J., Penders, B., de Rijcke, S., & Munafò, M. R. (2024). Mapping Cross-Disciplinary Perspectives on Responsible Conduct of Research: A Delphi Study. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/xmnu5
| Mapping Cross-Disciplinary Perspectives on Responsible Conduct of Research: A Delphi Study | Sarahanne M. Field, Jackie Thompson, Tom van Drimmelen, Jennifer Ferrar, Bart Penders, Sarah de Rijcke, and Marcus R. Munafò | <p>Responsible conduct of research (RCR) is generally agreed to be a laudable goal. It promotes high quality research practices, which should lead to more credible findings, and instill confidence in the research community. However, it is as yet u... | Social sciences | Charlotte Pennington | 2023-05-19 15:27:54 | View | ||
How Intelligence Interviewees Mentally Identify Relevant InformationDavid A. Neequaye & Alexandra Lorson https://doi.org/10.31234/osf.io/bpdn2How an interviewee knows what information is key to disclose or withholdRecommended by Zoltan Dienes based on reviews by 1 anonymous reviewerResearch on interviewing has often focused on topics (such as aiding memory of witnesses) which presume the interviewee has already correctly identified the precise information that the interviewer is really after. But how does an informant know what sort of information is asked for, a precondition for an informant to then choose to provide the information or withhold it (depending on their own interests)?
In this study, Neequaye and Lorson (2023) asked subjects to take the role of an informant about a criminal gang, with the further instructions to be cooperative or resistant in helping the interviewer obtain the information they want. In one study, the participants were asked merely to identify what information the interviewer wants. In the second study, the participants answered the interviewer's questions, disclosing whatever information they felt best suited their interest. Crucially, the level of detail of the questions was manipulated, such that the question specified a clear objective or not. Contrary to the theory, mental designation preferences indicated that interviewees generally assume interviewers wanted to know complete details, irrespective of question specificity. The Stage 2 manuscript was evaluated over one round of in-depth review. Based on responses to the comments, the recommender judged that the manuscript met the Stage 2 criteria and therefore awarded a positive recommendation. URL to the preregistered Stage 1 protocol: https://osf.io/82qtn Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. List of eligible PCI RR-friendly journals: References
1. Neequaye, D. A., & Lorson, A. (2023). How Intelligence Interviewees Mentally Identify Relevant Information [Stage 2]. Acceptance of of Version 10 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/bpdn2
| How Intelligence Interviewees Mentally Identify Relevant Information | David A. Neequaye & Alexandra Lorson | <p>This research explored how intelligence interviewees mentally identify the relevant information at their disposal. We theorized that interviewees estimate the interviewer’s objectives based on how they frame any attempt to solicit information. ... | Social sciences | Zoltan Dienes | 2023-05-24 06:57:09 | View | ||
14 Feb 2024
STAGE 1
Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) TemplateLisa Spitzer & Stefanie Mueller https://doi.org/10.23668/psycharchives.14119Examining the restrictiveness of the PRP-QUANT TemplateRecommended by Daniel Lakens based on reviews by Marjan Bakker and 1 anonymous reviewerThe Psychological Research Preregistration-Quantitative Template has been created in 2022 to provide more structure and detail to preregistrations. The goal of the current study is to test if the PRP-QUANT template indeed provides greater restriction of the flexibility in a study for preregistered hypotheses than other existing templates. This question is important because one concern that has been raised about the practice of preregistration is that the quality of preregistrations is often low. Metascientific research has shown that preregistrations are often of low quality (Bakker et al., 2020), and hypothesis tests from preregistrations are still selectively reported (van den Akker, van Assen, Enting, et al., 2023). It is important to improve the quality of preregistrations, and if a better template can help, it is a cost-effective approach to improve quality if the wider adoption of the better template can be promoted.
In the current study, Spitzer and Mueller (2024) will follow the procedure of a previous meta-scientific study by Heirene et al. (2021). 74 existing preregistrations with the PRP-QUANT template are available, and will be compared with an existing dataset coded by Bakker and colleagues (2020). The sample size is limited, but allows detecting some differences that would be considered large enough to matter, even though there might be smaller differences that would not be detectable based on the currently available sample size. Nevertheless, given that there is a need for improvement, even preliminary data might already be useful to provide tentative recommendations. Restrictiveness will be coded in 23 items, and adherence to or deviations from the preregistration are coded as well. As such deviations are common, the question whether this template reduced the likelihood of deviations is important. Two coders will code all studies.
The study should provide a useful initial evaluation of the PRP-QUANT template, and has the potential to have practical implications if the PRP-QUANT template shows clear benefits. Both authors have declared COI's related to the PRP-QUANT template, making the Registered Report format a fitting approach to prevent confirmation bias from influencing the reported results.
This Stage 1 manuscript was evaluated over two rounds of in-depth review by two expert reviewers and the recommender. After the revisions, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/vhezj
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence. List of eligible PCI RR-friendly journals:
References
1. van den Akker, O. R., van Assen, M. A. L. M., Bakker, M., Elsherif, M., Wong, T. K., & Wicherts, J. M. (2023). Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02277-0
2. Bakker, M., Veldkamp, C. L. S., Assen, M. A. L. M. van, Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020). Ensuring the quality and specificity of preregistrations. PLOS Biology, 18(12), e3000937. https://doi.org/10.1371/journal.pbio.3000937
3. Spitzer, L. & Mueller, S. (2024). Stage 1 Registered Report: Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/vhezj
4. Heirene, R., LaPlante, D., Louderback, E. R., Keen, B., Bakker, M., Serafimovska, A., & Gainsbury, S. M. (2021). Preregistration specificity & adherence: A review of preregistered gambling studies & cross-disciplinary comparison. PsyArXiv. https://doi.org/10.31234/osf.io/nj4es
| Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template | Lisa Spitzer & Stefanie Mueller | <p>Preregistration can help to restrict researcher degrees of freedom and thereby ensure the integrity of research findings. However, its ability to restrict such flexibility depends on whether researchers specify their study plan in sufficient de... | Social sciences | Daniel Lakens | 2023-06-01 10:39:20 | View | ||
16 Aug 2023
STAGE 1
Identifying relevant experiences to the measurement of social media experience via focus groups with young people: A registered reportJo Hickman Dunne, Louise Black, Emily Banwell, Pratyasha Nanda, Molly Anderton, Lily Corke Butters, Ola Demkowicz, Brittany Davidson, Pamela Qualter, Neil Humphrey, Caroline Jay, and Margarita Panayiotou https://psyarxiv.com/erjvz/A mental health perspective to adolescents’ social media experiencesRecommended by Veli-Matti Karhulahti based on reviews by Amy Orben, Jana Papcunova, Lisa Orchard, Elena Gordon-Petrovskaya and Gaurav SaxenaMeasuring people’s experiences, thoughts, and mental processes has always been a core challenge of psychological science (e.g. Nisbett & Wilson 1977). When such measurement further relates to rapidly changing and conceptually diverse human-technology interactions, the task becomes even more difficult due to protean, multidimensional constructs. A good understanding of a construct is a basic step in its measurement (Borsboom 2005).
In the present registered report, Hickman Dunne et al. (2023) carry out a focus group study with adolescents (n=32) aged 11 to 15 in Northwest England to improve the understanding of constructs related to social media and mental health experiences. The work is carried out as part of a long-term measure development project. The authors apply reflexive thematic analysis to explore adolescents’ social media use experiences and related motivations in the light of mental health, in addition to which the adolescents’ own views of benefits and risks are mapped out. A particular strength of the design is the engagement of three Young Researchers who will co-facilitate the focus groups and be involved in the analysis. The research plan also meets high reflexivity and transparency criteria, and as such, can significantly contribute to future scale development as well as our general understanding of adolescents’ social media experiences. The Stage 1 manuscript was reviewed over two rounds by five unique reviewers, one of which participated in both rounds. The reviewers’ expertise ranged from social media and technology use research to health psychology and qualitative methods. Based on careful revisions and detailed responses to the reviewers’ comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance. URL to the preregistered Stage 1 protocol: https://osf.io/w24ec
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question. List of eligible PCI RR-friendly journals: References
1. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge University Press. 3. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological review, 84, 231–259. https://doi.org/10.1037/0033-295X.84.3.231
2. Hickman Dunne J., Black L., Banwell E., Nanda P., Anderton M, Butters L.C., Demkowicz O., Davidson B., Qualter P., Humphrey N., Jay C., and Panayiotou M. (2023). Identifying relevant dimensions to the measurement of adolescent social media experience via focus groups with young people: A registered report. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/w24ec | Identifying relevant experiences to the measurement of social media experience via focus groups with young people: A registered report | Jo Hickman Dunne, Louise Black, Emily Banwell, Pratyasha Nanda, Molly Anderton, Lily Corke Butters, Ola Demkowicz, Brittany Davidson, Pamela Qualter, Neil Humphrey, Caroline Jay, and Margarita Panayiotou | <p>Background: While work on the relationship between social media use and adolescent mental health has allowed for some progress, research in this area is still relatively new and shows mixed evidence. This is partly the consequence of a rapidly ... | Computer science, Social sciences | Veli-Matti Karhulahti | 2023-06-14 21:10:43 | View | ||
11 Sep 2023
STAGE 1
Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabularyTom E. Hardwicke, Sarah Schiavone, Beth Clarke, Simine Vazire https://osf.io/e5pkz?view_only=3f65527bb5dc428382f4b9154bfc58e0Understanding the validity of standardised language in research evaluationRecommended by Sarahanne Miranda Field and Chris Chambers based on reviews by Chris Hartgerink (they/them), Veli-Matti Karhulahti, Štěpán Bahník and Ross MounceIn 2023, the journal eLife ended the practice of making binary accept/reject decisions following peer review, instead sharing peer review reports (for manuscripts that are peer-reviewed) and brief “eLife assessments” representing the consensus opinions of editors and peer reviewers. As part of these assessments, the journal draws language from a "common vocabulary" to linguistically rank the significance of findings and strength of empirical support for the article's conclusions. In particular, the significance of findings is described using an ordinal scale of terms from "landmark" → "fundamental" → "important" → "valuable" → "useful", while the strength of support is ranked across six descending levels from "exceptional" down to "inadequate".
In the current study, Hardwicke et al. (2023) question the validity of this taxonomy, noting a range of linguistic ambiguities and counterintuitive characteristics that may undermine the communication of research evaluations to readers. Given the centrality of this common vocabulary to the journal's policy, the authors propose a study to explore whether the language used in the eLife assessments will be interpreted as intended by readers. Using a repeated-measures experimental design, they will tackle three aims: first, to understand the extent to which people share similar interpretations of phrases used to describe scientific research; second, to reveal the extent to which people’s implicit ranking of phrases used to describe scientific research aligns with each other and with the intended ranking; and third, to test whether phrases used to describe scientific research have overlapping interpretations. The proposed study has the potential to make a useful contribution to metascience, as well as being a valuable source of information for other journals potentially interested in following the novel path made by eLife.
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/mkbtp
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Hardwicke, T. E., Schiavone, S., Clarke, B. & Vazire, S. (2023). Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/mkbtp | Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary | Tom E. Hardwicke, Sarah Schiavone, Beth Clarke, Simine Vazire | <p>The journal eLife recently announced that it would abandon binary ‘accept/reject’ decisions and instead focus on sharing both peer review reports and short “eLife assessments” representing the consensus opinions of editors and peer reviewers. F... | Life Sciences, Social sciences | Sarahanne Miranda Field | 2023-06-16 12:11:14 | View | ||
10 Feb 2024
STAGE 1
Using Shakespeare to Answer Psychological Questions: Complexity and Mental Representability of Character NetworksChristian M. Thurn; Simone Sebben; Zoran Kovacevic https://osf.io/3zsvnComplexity of Shakespeare’s Social NetworksRecommended by Veli-Matti Karhulahti based on reviews by Matúš Adamkovič, James Stiller, Tomáš Lintner and Matus AdamkovicThe rapid methodological development in digital humanities keeps opening new possibilities to better understand our cultural artifacts and, in the process, also ourselves. Some of the historically most influential works of literary human culture are the plays of Shakespeare, which continue to be read and treasured around the world. Although the social networks of Shakespeare’s plays have attracted scientific attention already more than two decades (Stiller et al. 2003), the understanding of their complexity in terms of character networks remains limited and not fully contextualized in the larger landscape of European drama.
In the present registered report, Thurn and colleagues (2024) apply Kolmogorov complexity analysis to investigate the social networks in 37 existing plays of Shakespeare. The authors replicate the original work by Stiller et al. (2003) and situate the findings in a larger regional context by further analyzing over 3,000 plays available in the European Drama Corpus. Ultimately, the authors explore the relationships between (Kolmogorov) complexity and the size of character networks as well as the robustness of their results in relation to possible researcher decisions in the analytic process.
This Stage 1 manuscript was evaluated over three rounds of in-depth review by four expert reviewers from the research fields of literature, networks, and social analysis. Based on the authors’ careful revisions and responses to the reviewers’ feedback, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/6uw27
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence. List of eligible PCI RR-friendly journals: References
1. Stiller, J., Nettle, D. & Dunbar, R. I. M. (2003). The small world of shakespeare’s plays. Human Nature, 14, 397-408. https://doi.org/10.1007/s12110-003-1013-1
2. Thurn, C., Sebben, S. & Kovacevic, Z. (2024) Using Shakespeare to Answer Psychological Questions: Complexity and Mental Representability of Character Networks. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/6uw27 | Using Shakespeare to Answer Psychological Questions: Complexity and Mental Representability of Character Networks | Christian M. Thurn; Simone Sebben; Zoran Kovacevic | <p>Theater plays are a cultural product that can be used to learn about the capacity of human cognition. We argue that Kolmogorov complexity may be suited to operationalize the demand that is put onto a<br>recipient's cognitive system to represent... | Humanities, Social sciences | Veli-Matti Karhulahti | 2023-06-16 12:40:14 | View | ||
01 Jul 2024
STAGE 1
The Influence of Bilingualism on Statistical Word Learning: A Registered ReportSimonetti, M. E., Lorenz, M. G., Koch, I., & Roembke, T. https://osf.io/2ch9yComparing statistical word learning in bilinguals and monolingualsRecommended by Elizabeth Wonnacott based on reviews by 2 anonymous reviewersMany studies have investigated the extent to which word learning is underpinned by statistical learning, i.e. tracking probabilistic relationships between forms and referents. Previous literature has investigated whether these processes differ in bilingual learners – who have to track two such sets of mappings in their linguistic environment. However, the evidence is mixed: some say bilinguals have a learning advantage and some find no evidence of differences.
The current study by Simonetti et al. (2024) aims to further explore this in an experiment using the cross-situational word learning paradigm. In this paradigm participants hear words and view arrays of object across a series of trials. Taking each trial in isolation the word is ambiguous, but there are consistent co-occurrences of words with referents across the trials. Two groups of participants will be compared: monolingual English speaker and English-German bilinguals. Using this paradigm, the study can track learning over time as well as looking at individual trial by trial analyses. The researchers predict specifically that bilingual learners will have a specific advantage in learning 1:2 mappings, where one-word maps to two objects. The study will use Bayes Factors as the method of inference when analysing the data, allowing them to differentiate evidence for "no difference" from ambiguous evidence from which no conclusion can be drawn.
The Stage 1 manuscript was evaluated over four rounds of review. Based on detailed responses to the reviewers' and recommender's comments, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/8n5gh
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals: References Simonetti, M. E., Lorenz, M. G., Koch, I., & Roembke, T. C. (2024). The Influence of Bilingualism on Statistical Word Learning: A Registered Report. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/8n5gh
| The Influence of Bilingualism on Statistical Word Learning: A Registered Report | Simonetti, M. E., Lorenz, M. G., Koch, I., & Roembke, T. | <p>While statistical word learning has been the focus of many studies on monolinguals, it has<br>received little attention in bilinguals. The results of existing studies on statistical word learning<br>in bilinguals are inconsistent, with some res... | Social sciences | Elizabeth Wonnacott | 2023-06-28 15:37:58 | View |
MANAGING BOARD
Chris Chambers
Zoltan Dienes
Corina Logan
Benoit Pujol
Maanasa Raghavan
Emily S Sena
Yuki Yamada