Submit a report

Announcements

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.


 

Latest recommendationsrssmastodon

IdTitleAuthorsAbstractPictureThematic fieldsRecommenderReviewers▼Submission date
07 Dec 2023
STAGE 1
toto

Investigating individual differences in linguistic statistical learning and their relation to rhythmic and cognitive abilities: A speech segmentation experiment with online neural tracking

Individual differences in linguistic statistical learning and the relationship to rhythm perception

Recommended by ORCID_LOGO based on reviews by 2 anonymous reviewers
Previous research has demonstrated that experimental participants – both adults and infants – can track syllable co-occurrences in an artificial speech stream. This statistical learning (SL) is thought to support word segmentation and to be a core component of language acquisition. There is also some evidence that SL, as measured through artificial language tasks, is related to individual differences in language learning. What is not well understood is the underpinnings of individual differences in SL. In the current study, van der Wulp et al. (2023) will investigate the relationship between auditory SL and musical – specifically rhythmic – abilities. This work takes advantage of recent methodological innovations which allow online assessment of SL via electroencephalography (EEG) measures of neural entrainment. Up to N=105 participants will be recruited (using a well specified optional stopping procedure) and will be exposed to an artificial language using this EEG method to measure their response to the language through the exposure phase. In addition, post-tests will assess participants’ learning of the artificial language using behavioural tasks, as well as their abilities in a battery of tasks measuring rhythmic, musical, and cognitive abilities, and their vocabulary size. Core analyses will investigate whether the neural signature of statistical learning is found at a group level (i.e. replicating previous work) and whether at the individual level this is related to rhythm perception. Bayes factors will be used to assess the extent of evidence for these hypothesized relationships over the null. Exploratory analyses will explore other relationships, including with working memory and vocabulary size. 
 
The topic of individual differences in language learning and is important, but work in this area is often underpowered, and the correlational nature of the research makes it vulnerable to HARKing. In this context, the current RR looks set to make an important contribution. The Stage 1 manuscript was evaluated over two rounds of in-depth review by the recommender and two expert reviewers, and one additional round of review by the recommender, before issuing in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/2y6sx

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. van der Wulp, I. M., Struiksma, M. E., Batterink, L. J., & Wijnen, F. N. K. (2023). Investigating individual differences in linguistic statistical learning and their relation to rhythmic and cognitive abilities: A speech segmentation experiment with online neural tracking. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/2y6sx
 
Investigating individual differences in linguistic statistical learning and their relation to rhythmic and cognitive abilities: A speech segmentation experiment with online neural trackingIris van der Wulp, Marijn Struiksma, Laura Batterink, Frank Wijnen<p><strong>Objective:</strong> Statistical Learning (SL) is an essential mechanism for speech segmentation. Importantly, individual differences in SL ability are associated with language acquisition. For instance, better SL correlated with a large...Humanities, Social sciencesElizabeth Wonnacott2023-05-03 10:53:51 View
22 Oct 2023
STAGE 1
toto

Michotte's research on perceptual impressions of causality: a pre-registered replication study

Gaining confidence in Michotte’s classic studies on the perception of causality

Recommended by ORCID_LOGO based on reviews by Maxine Sherman and 1 anonymous reviewer
​​​Making causal judgements are part of everyday life, whether seeking to understand the action of complex humans or the relations between inanimate objects in our environments. Albert Michotte’s (1963) classic book, The perception of causality, contained an extensive report of experiments demonstrating not only that observers perceive causality of inanimate shapes, but do so in manifold ways, creating different “causal impressions.” This work has been highly influential across psychology and neuroscience.
 
In the current study, White (2023) proposes a series of experiments to replicate and extend Michotte’s work. Despite the fact that this research is foundational to current work on perception and understanding of causal relations, it has never been subjected to rigorous replication. Moreover, like many research studies from that era, Michotte was sparse on details about methodology and did not rely on statistical analysis. White has proposed an ambitious set of 14 experiments that directly replicate and, in some cases, extend Michotte’s experiments. 
 
The Stage 1 manuscript was evaluated over three rounds of in-depth review, the first two rounds consisting of detailed comments from two reviewers and the third round consisting of a close read by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and was therefore awarded in-principle acceptance (IPA). 
 
URL to the preregistered Stage 1 protocol: https://osf.io/5jx8f (under temporary private embargo)
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI-RR-friendly journals:
 
 
References
 
1. Michotte, A. (1963). The perception of causality (T. R. Miles & E. Miles, trans.). London: Methuen. (English translation of Michotte, 1954).
 
2. White, P. A. (2023). Michotte's research on perceptual impressions of causality: A registered replication study. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/5jx8f 
Michotte's research on perceptual impressions of causality: a pre-registered replication studyPeter A. White<p>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Michotte (1946/1954/1963) showed that visual impressions of causality can occur in perception of simple animations of moving geometrical objects. In the launching effect, one object is perceived as maki...Life Sciences, Social sciencesMoin Syed2023-05-03 13:04:00 View
28 Sep 2023
STAGE 1
toto

Investigating the barriers and enablers to data sharing behaviours: A qualitative Registered Report

Capability, Opportunity, and Motivation in Data Sharing Behaviour

Recommended by ORCID_LOGO based on reviews by Moin Syed, Peter Branney and Libby Bishop
In the past two decades, most academic fields have witnessed an open science revolution that has led to significant increases in open access publishing, reproducibility efforts, and scientific transparency in general (e.g., Spellman et al. 2018). One of the key areas in this ongoing change is data sharing. Although some evidence already points at progress in data sharing practices, many new datasets remain unshared (see Tedersoo et al. 2021).
 
In the present registered report, Henderson et al. (2023) empirically explore the factors that either hinder or facilitate data sharing in the UK. By means of semi-structured interviews, the team will chart researchers’ experiences of sharing and non-sharing. Thematic template analysis will be applied to organise the data into a hierarchical map of capabilities, opportunities, and motivations in a theoretical domains framework (COM-B-TDF). The research plan itself meets the highest open science standards and reflects on the authors own positions, from which the current qualitative interview data sharing efforts will be made.
 
The Stage 1 manuscript was reviewed over three rounds by three experts with familiary of the UK cultural context and specializations in open science practices, qualitative research, and data infrastructures. Based on careful revisions and detailed responses to the reviewers’ comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/2gm5s (under temporary private embargo)
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.  
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Henderson, E., Marcu, A., Atkins, L. & Farran, E.K. (2023). Investigating the barriers and enablers to data sharing behaviours: A qualitative Registered Report. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/2gm5s
 
2. Spellman, B. A., Gilbert, E. A. & Corker, K. S. (2018). Open Science. Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience, 5, 1-47. https://doi.org/10.1002/9781119170174.epcn519
 
3. Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä., ... & Sepp, T. (2021). Data sharing practices and data availability upon request differ across scientific disciplines. Scientific data, 8, 192. https://doi.org/10.1038/s41597-021-00981-0
Investigating the barriers and enablers to data sharing behaviours: A qualitative Registered ReportEmma L Henderson, Afrodita Marcu, Lou Atkins, Emily K Farran<p>Data sharing describes the process of making research data available for reuse. The availability of research data is the basis of transparent, effective research systems that democratise access to knowledge and advance discovery. Despite a broa...Social sciencesVeli-Matti Karhulahti2023-05-11 19:18:48 View
14 Aug 2023
STAGE 2
(Go to stage 1)
article picture

Life Thinning and Gaming Disorder: A Longitudinal Qualitative Registered Report

How do intensive gaming experiences evolve over time in clinical and non-clinical contexts?

Recommended by ORCID_LOGO based on reviews by Peter Branney and Michelle Carras
Over the last 5 years the inclusion of “gaming disorder” in the ICD-11 been controversial (Van Rooij et al, 2018), mirroring wider public debate about the effects of gaming on mental health. One of the major gaps in understanding the validity of gaming disorder as an identifiable mental illness is the absence of qualitative studies comparing the lived experience of gamers who seek treatment with esports players who do not report health problems.
 
Here, Karhulahti et al. (2023) tackle this question in the second of two Stage 2 Registered Reports associated with their previous programmatic Stage 1 submission. Using interpretative phenomenological analysis, the authors undertook in-depth interviews over a 1-year period with treatment-seeking participants (N=5) and esports-playing participants (N=4) who did not experience gaming-related health problems. The authors sought to answer the folllwing primary question: How do the experiences and meanings of playing videogames—shaped by the individuals’ diverse sociocultural contexts—evolve in those with related health problems (as defined by treatment-seeking) and those who play esports games several hours per day while self-reporting no related health problems?
 
Both groups exhibited intense relationships with gaming that were cyclical over time across various dimensions, with fluctuations occurring in response to changes in health, occupation, and social networks. The observed variation over time was substantial, with individuals attaching and detaching from games involving hundreds or thousands of hours. The authors report treatment-seeking being followed by a search of new gaming and life meanings, while intensive gaming without related problems continued as an integrated part of the self, with resilience adapting and evolving in the face of unexpected life events. Taking into account their findings, the authors propose life thinning and resilience integration processes to help describe and explain how some individuals end up seeking treatment for their gaming, while for others gaming supports them and becomes integrated into their identity.
 
Following one round of in-depth review and revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/a2rwg
 
Level of bias control achieved: Level 4.  At least some of the data/evidence that was used to answer the research question existed prior to in-principle acceptance(IPA) but the authors certify that they did not access any part of that data/evidence prior to IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. van Rooij AJ, Ferguson CJ, Carras MC, Kardefelt-Winther D, Shi J, Aarseth E, Bean AM, Bergmark KH, Brus A, Coulson M, Deleuze J, Dullur P, Dunkels E, Edman J, Elson M, Etchells PJ, Fiskaali A, Granic I, Jansz J, Karlsen F, Kaye LK, Kirsh B, Lieberoth A, Markey P, Mills KL, Nielsen RKL, Orben A, Poulsen A, Prause N, Prax P, Quandt T, Schimmenti A, Starcevic V, Stutman G, Turner NE, Looy J van, Przybylski AK (2018) A weak scientific basis for gaming disorder: Let us err on the side of caution. Journal of Behavioral Addictions, 7, 1–9. https://doi.org/10.1556/2006.7.2018.19
 
2. Karhulahti V-M, Siutila M, Vahlo J, Koskimaa R (2023). Life Thinning and Gaming Disorder: A Longitudinal Qualitative Registered Report [Stage 2 Registered Report], acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/hmcqz
Life Thinning and Gaming Disorder: A Longitudinal Qualitative Registered ReportVeli-Matti Karhulahti, Miia Siutila, Jukka Vahlo, Raine Koskimaa<p>The academic debates regarding the psychiatric relevance of gaming disorder continue largely because the lived experiences of treatment-seekers remain mostly unstudied. This registered report addresses the above research gap with a longitudinal...Humanities, Medical Sciences, Social sciencesChris Chambers2023-05-15 11:01:37 View
03 Jul 2023
STAGE 2
(Go to stage 1)
toto

Globally, songs and instrumental melodies are slower, higher, and use more stable pitches than speech [Stage 2 Registered Report]

Strong evidence for cross-cultural regularities in music and speech

Recommended by ORCID_LOGO based on reviews by Bob Slevc and Nai Ding
For centuries, the ubiquity of language and music across human societies has prompted scholars to speculate about their cross-cultural origins as well as their shared and unique characteristics. Depending on the extent to which contemporary theories emphasise the role of biology vs. culture, a range of hypotheses have been proposed concerning expected similarities and differences in song and speech. One class of hypotheses stemming from cultural relativism assumes a lack of universal regularities in song and speech, and therefore predicts no systematic cross-cultural relationships. On the other hand, more recent evolutionary hypotheses such as the social bonding hypothesis, motor constraint hypothesis, and sexual selection hypothesis all predict differences or similarities in specific characteristic of vocalisations, such as pitch regularity, pitch interval size, and melodic contour. Existing results are mixed in their support of these predictions.
 
In the current study, Ozaki et al. (2023) elucidated cross-cultural similarities and differences between speech and song in 75 different linguistic varieties spanning 21 language families. Understanding precisely how song and speech are related is methodologically challenging due to the multitude of confounds that can arise in comparing natural recordings. Here the authors overcame these difficulties with four types of carefully controlled recordings: singing, recitation of sung lyrics, spoken description of the song, and instrumental version of the sung melody. The authors then examined six features that are amenable to reliable comparison, including pitch height, temporal rate, pitch stability, timbral brightness, pitch interval size, and pitch declination. With this data in hand, the authors asked which acoustic features differ reliably between song and speech across cultures, with the expectation that song would exhibit higher pitch, slower rate and more stable pitch than speech. At the same time, the authors expected song and speech to be reliably similar in the characteristics of timbral brightness, pitch intervals and pitch contours
 
The findings provided strong support for the preregistered hypotheses. Relative to speech, songs exhibited higher pitch, slower temporal rate, and more stable pitches, while both songs and speech had similar pitch interval size and timbral brightness. Only one hypothesis was unsupported, with the comparison of pitch declination between song and speech turning out inconclusive. To overcome potential sources of analytic bias, the authors undertook additional robustness checks, including reanalysis of a previously published dataset of over 400 song/speech recordings; this exploratory analysis corroborated the conclusions from the confirmatory analysis. Overall this study offers a unique insight into the shared global characteristics of langage and music, with implications for understanding their cultural and biological (co)evolution.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and therefore awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/jdhtz
 
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors, but the authors certify that they had not yet observed the key variables within the data that were be used to answer the research question AND they took additional steps to maximise bias control and rigour.

List of eligible PCI RR-friendly journals:
 
 
References
 
1. Ozaki, Y. et al. (2023). Globally, songs and instrumental melodies are slower, higher, and use more stable pitches than speech [Stage 2 Registered Report]. Acceptance of Version 11 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/jr9x7
Globally, songs and instrumental melodies are slower, higher, and use more stable pitches than speech [Stage 2 Registered Report]Yuto Ozaki, Adam Tierney, Peter Pfordresher, John Mcbride, Emmanouil Benetos, Polina Proutskova, Gakuto Chiba, Fang Liu, Nori Jacoby, Suzanne Purdy, Patricia Opondo, Tecumseh Fitch, Shantala Hegde, Martín Rocamora, Rob Thorne, Florence Ewomazino N...<p>What, if any, similarities and differences between music and speech are consistent across cultures? Both music and language are found in all known human societies and are argued to share evolutionary roots and cognitive resources, yet no studie...Humanities, Social sciencesChris Chambers2023-05-16 17:22:17 View
09 Jul 2023
STAGE 2
(Go to stage 1)
toto

How Intelligence Interviewees Mentally Identify Relevant Information

How an interviewee knows what information is key to disclose or withhold

Recommended by ORCID_LOGO based on reviews by 1 anonymous reviewer
Research on interviewing has often focused on topics (such as aiding memory of witnesses) which presume the interviewee has already correctly identified the precise information that the interviewer is really after. But how does an informant know what sort of information is asked for, a precondition for an informant to then choose to provide the information or withhold it (depending on their own interests)?
 
In this study, Neequaye and Lorson (2023) asked subjects to take the role of an informant about a criminal gang, with the further instructions to be cooperative or resistant in helping the interviewer obtain the information they want. In one study, the participants were asked merely to identify what information the interviewer wants. In the second study, the participants answered the interviewer's questions, disclosing whatever information they felt best suited their interest. Crucially, the level of detail of the questions was manipulated, such that the question specified a clear objective or not. Contrary to the theory, mental designation preferences indicated that interviewees generally assume interviewers wanted to know complete details, irrespective of question specificity.

The Stage 2 manuscript was evaluated over one round of in-depth review. Based on responses to the comments, the recommender judged that the manuscript met the Stage 2 criteria and therefore awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/82qtn
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Neequaye, D. A., & Lorson, A. (2023). How Intelligence Interviewees Mentally Identify Relevant Information [Stage 2]. Acceptance of of Version 10 by Peer Community in Registered Reports. https://doi.org/10.31234/osf.io/bpdn2
How Intelligence Interviewees Mentally Identify Relevant InformationDavid A. Neequaye & Alexandra Lorson<p>This research explored how intelligence interviewees mentally identify the relevant information at their disposal. We theorized that interviewees estimate the interviewer’s objectives based on how they frame any attempt to solicit information. ...Social sciencesZoltan Dienes2023-05-24 06:57:09 View
14 Feb 2024
STAGE 1
article picture

Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template

Examining the restrictiveness of the PRP-QUANT Template

Recommended by ORCID_LOGO based on reviews by Marjan Bakker and 1 anonymous reviewer
The Psychological Research Preregistration-Quantitative Template has been created in 2022 to provide more structure and detail to preregistrations. The goal of the current study is to test if the PRP-QUANT template indeed provides greater restriction of the flexibility in a study for preregistered hypotheses than other existing templates. This question is important because one concern that has been raised about the practice of preregistration is that the quality of preregistrations is often low. Metascientific research has shown that preregistrations are often of low quality (Bakker et al., 2020), and hypothesis tests from preregistrations are still selectively reported (van den Akker, van Assen, Enting, et al., 2023). It is important to improve the quality of preregistrations, and if a better template can help, it is a cost-effective approach to improve quality if the wider adoption of the better template can be promoted. 
 
In the current study, Spitzer and Mueller (2024) will follow the procedure of a previous meta-scientific study by Heirene et al. (2021). 74 existing preregistrations with the PRP-QUANT template are available, and will be compared with an existing dataset coded by Bakker and colleagues (2020). The sample size is limited, but allows detecting some differences that would be considered large enough to matter, even though there might be smaller differences that would not be detectable based on the currently available sample size. Nevertheless, given that there is a need for improvement, even preliminary data might already be useful to provide tentative recommendations. Restrictiveness will be coded in 23 items, and adherence to or deviations from the preregistration are coded as well. As such deviations are common, the question whether this template reduced the likelihood of deviations is important. Two coders will code all studies. 
 
The study should provide a useful initial evaluation of the PRP-QUANT template, and has the potential to have practical implications if the PRP-QUANT template shows clear benefits. Both authors have declared COI's related to the PRP-QUANT template, making the Registered Report format a fitting approach to prevent confirmation bias from influencing the reported results. 
 
This Stage 1 manuscript was evaluated over two rounds of in-depth review by two expert reviewers and the recommender. After the revisions, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/vhezj
 
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. van den Akker, O. R., van Assen, M. A. L. M., Bakker, M., Elsherif, M., Wong, T. K., & Wicherts, J. M. (2023). Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02277-0
 
2. Bakker, M., Veldkamp, C. L. S., Assen, M. A. L. M. van, Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020). Ensuring the quality and specificity of preregistrations. PLOS Biology, 18(12), e3000937. https://doi.org/10.1371/journal.pbio.3000937
 
3. Spitzer, L. & Mueller, S. (2024). Stage 1 Registered Report: Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) Template. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/vhezj
 
4. Heirene, R., LaPlante, D., Louderback, E. R., Keen, B., Bakker, M., Serafimovska, A., & Gainsbury, S. M. (2021). Preregistration specificity & adherence: A review of preregistered gambling studies & cross-disciplinary comparison. PsyArXiv. https://doi.org/10.31234/osf.io/nj4es
Restriction of researcher degrees of freedom through the Psychological Research Preregistration-Quantitative (PRP-QUANT) TemplateLisa Spitzer & Stefanie Mueller<p>Preregistration can help to restrict researcher degrees of freedom and thereby ensure the integrity of research findings. However, its ability to restrict such flexibility depends on whether researchers specify their study plan in sufficient de...Social sciencesDaniel Lakens2023-06-01 10:39:20 View
16 Aug 2023
STAGE 1
toto

Identifying relevant experiences to the measurement of social media experience via focus groups with young people: A registered report

A mental health perspective to adolescents’ social media experiences

Recommended by ORCID_LOGO based on reviews by Amy Orben, Jana Papcunova, Lisa Orchard, Elena Gordon-Petrovskaya and Gaurav Saxena
Measuring people’s experiences, thoughts, and mental processes has always been a core challenge of psychological science (e.g. Nisbett & Wilson 1977). When such measurement further relates to rapidly changing and conceptually diverse human-technology interactions, the task becomes even more difficult due to protean, multidimensional constructs. A good understanding of a construct is a basic step in its measurement (Borsboom 2005).  
 
In the present registered report, Hickman Dunne et al. (2023) carry out a focus group study with adolescents (n=32) aged 11 to 15 in Northwest England to improve the understanding of constructs related to social media and mental health experiences. The work is carried out as part of a long-term measure development project. The authors apply reflexive thematic analysis to explore adolescents’ social media use experiences and related motivations in the light of mental health, in addition to which the adolescents’ own views of benefits and risks are mapped out.
 
A particular strength of the design is the engagement of three Young Researchers who will co-facilitate the focus groups and be involved in the analysis. The research plan also meets high reflexivity and transparency criteria, and as such, can significantly contribute to future scale development as well as our general understanding of adolescents’ social media experiences.
 
The Stage 1 manuscript was reviewed over two rounds by five unique reviewers, one of which participated in both rounds. The reviewers’ expertise ranged from social media and technology use research to health psychology and qualitative methods. Based on careful revisions and detailed responses to the reviewers’ comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.
 
URL to the preregistered Stage 1 protocol: https://osf.io/w24ec
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge University Press.
 
3. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological review, 84, 231–259. https://doi.org/10.1037/0033-295X.84.3.231

2. Hickman Dunne J., Black L., Banwell E., Nanda P., Anderton M, Butters L.C., Demkowicz O., Davidson B., Qualter P., Humphrey N., Jay C., and Panayiotou M. (2023). Identifying relevant dimensions to the measurement of adolescent social media experience via focus groups with young people: A registered report. In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/w24ec
Identifying relevant experiences to the measurement of social media experience via focus groups with young people: A registered reportJo Hickman Dunne, Louise Black, Emily Banwell, Pratyasha Nanda, Molly Anderton, Lily Corke Butters, Ola Demkowicz, Brittany Davidson, Pamela Qualter, Neil Humphrey, Caroline Jay, and Margarita Panayiotou<p>Background: While work on the relationship between social media use and adolescent mental health has allowed for some progress, research in this area is still relatively new and shows mixed evidence. This is partly the consequence of a rapidly ...Computer science, Social sciencesVeli-Matti Karhulahti2023-06-14 21:10:43 View
11 Sep 2023
STAGE 1
toto

Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary

Understanding the validity of standardised language in research evaluation

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by Chris Hartgerink (they/them), Veli-Matti Karhulahti, Štěpán Bahník and Ross Mounce
In 2023, the journal eLife ended the practice of making binary accept/reject decisions following peer review, instead sharing peer review reports (for manuscripts that are peer-reviewed) and brief “eLife assessments” representing the consensus opinions of editors and peer reviewers. As part of these assessments, the journal draws language from a "common vocabulary" to linguistically rank the significance of findings and strength of empirical support for the article's conclusions. In particular, the significance of findings is described using an ordinal scale of terms from "landmark" → "fundamental" → "important" → "valuable" → "useful", while the strength of support is ranked across six descending levels from "exceptional" down to "inadequate".
 
In the current study, Hardwicke et al. (2023) question the validity of this taxonomy, noting a range of linguistic ambiguities and counterintuitive characteristics that may undermine the communication of research evaluations to readers. Given the centrality of this common vocabulary to the journal's policy, the authors propose a study to explore whether the language used in the eLife assessments will be interpreted as intended by readers. Using a repeated-measures experimental design, they will tackle three aims: first, to understand the extent to which people share similar interpretations of phrases used to describe scientific research; second, to reveal the extent to which people’s implicit ranking of phrases used to describe scientific research aligns with each other and with the intended ranking; and third, to test whether phrases used to describe scientific research have overlapping interpretations. The proposed study has the potential to make a useful contribution to metascience, as well as being a valuable source of information for other journals potentially interested in following the novel path made by eLife.
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/mkbtp
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Hardwicke, T. E., Schiavone, S., Clarke, B. & Vazire, S. (2023). Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/mkbtp
Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabularyTom E. Hardwicke, Sarah Schiavone, Beth Clarke, Simine Vazire<p>The journal eLife recently announced that it would abandon binary ‘accept/reject’ decisions and instead focus on sharing both peer review reports and short “eLife assessments” representing the consensus opinions of editors and peer reviewers. F...Life Sciences, Social sciencesSarahanne Miranda Field2023-06-16 12:11:14 View
10 Feb 2024
STAGE 1
toto

Using Shakespeare to Answer Psychological Questions: Complexity and Mental Representability of Character Networks

Complexity of Shakespeare’s Social Networks

Recommended by ORCID_LOGO based on reviews by Matúš Adamkovič, James Stiller, Tomáš Lintner and Matus Adamkovic
The rapid methodological development in digital humanities keeps opening new possibilities to better understand our cultural artifacts and, in the process, also ourselves. Some of the historically most influential works of literary human culture are the plays of Shakespeare, which continue to be read and treasured around the world. Although the social networks of Shakespeare’s plays have attracted scientific attention already more than two decades (Stiller et al. 2003), the understanding of their complexity in terms of character networks remains limited and not fully contextualized in the larger landscape of European drama.
 
In the present registered report, Thurn and colleagues (2024) apply Kolmogorov complexity analysis to investigate the social networks in 37 existing plays of Shakespeare. The authors replicate the original work by Stiller et al. (2003) and situate the findings in a larger regional context by further analyzing over 3,000 plays available in the European Drama Corpus. Ultimately, the authors explore the relationships between (Kolmogorov) complexity and the size of character networks as well as the robustness of their results in relation to possible researcher decisions in the analytic process.
 
This Stage 1 manuscript was evaluated over three rounds of in-depth review by four expert reviewers from the research fields of literature, networks, and social analysis. Based on the authors’ careful revisions and responses to the reviewers’ feedback, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/6uw27
 
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Stiller, J., Nettle, D. & Dunbar, R. I. M. (2003). The small world of shakespeare’s plays. Human Nature, 14, 397-408. https://doi.org/10.1007/s12110-003-1013-1

2. Thurn, C., Sebben, S. & Kovacevic, Z. (2024) Using Shakespeare to Answer Psychological Questions: Complexity and Mental Representability of Character Networks. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/6uw27
Using Shakespeare to Answer Psychological Questions: Complexity and Mental Representability of Character NetworksChristian M. Thurn; Simone Sebben; Zoran Kovacevic<p>Theater plays are a cultural product that can be used to learn about the capacity of human cognition. We argue that Kolmogorov complexity may be suited to operationalize the demand that is put onto a<br>recipient's cognitive system to represent...Humanities, Social sciencesVeli-Matti Karhulahti2023-06-16 12:40:14 View