FIELD Sarahanne Miranda's profile
avatar

FIELD Sarahanne Miranda

  • CWTS, Leiden University, Leiden, Netherlands
  • Social sciences
  • recommender

Recommendation:  1

Reviews:  0

Website https://www.sarahannemfield.eu
Areas of expertise
**Education** -Bachelor of Psychology (honours I; 2014; University of Newcastle, Australia) -Research Master in Psychometrics and Statistics (2017; University of Groningen, the Netherlands) -PhD in Behavioural and Social Sciences (topic: science reform movement; 2022; University of Groningen, The Netherlands) **Research Interests, Modus Operandi and Philosophy** Broadly, I am a metascientist. I research science (the scientific community, practices etc) using the scientific method. More specifically, I am a science reform scholar. I am interested in science reform as a social movement, the communities within the science reform movement, reform practices and critique of the reform movement. I don't just study reform objectively, however. I am an activist as well, and advocate most reform/open research initiatives (such as registered reports!)... as long as they're critically engaged with by users before adoption. I am currently a postdoctoral fellow working at the centre for science and technology studies (CWTS) at Leiden University in the Netherlands. I am on a project with Sarah de Rijcke (CWTS), Bart Penders (University of Maastricht), Jackie Thompson (Bristol University) and Marcus Munafò (Bristol University), working on establishing a new, updated responsible research and innovation (RRI) framework, and embedding it in local contexts in the UK and parts of Europe. I am a mixed methodologist, with training and expertise in both qualitative and quantitative methods. I value reflexivity and good scientific mentorship. I am passionate about considering alternatives to traditional publishing systems, and about dismantling barriers to participation in reform (relating to e.g., power imbalances, hierarchy in research, accessibility and inclusivity). I would argue that the registered report format, though not a panacea, might be one of the most impactful and valuable research reform initiatives that will come of my 'generation'. I have conducted studies involving preregistration and registered reports, replication and selecting replication targets, reflexivity (especially for use in quantitative research contexts), and Bayesian analyses and reanalyses. I am (to varying degrees) familiar with digital and virtual ethnography, social network analysis, the Bayesian approach, and thematic analysis (both done 'by hand' and done using QDAS).

Recommendation:  1

11 Sep 2023
STAGE 1
toto

Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary

Tom E. Hardwicke, Sarah Schiavone, Beth Clarke, Simine Vazire https://osf.io/e5pkz?view_only=3f65527bb5dc428382f4b9154bfc58e0

Understanding the validity of standardised language in research evaluation

Recommended by and based on reviews by Chris Hartgerink, Veli-Matti Karhulahti, Štěpán Bahník and Ross Mounce
In 2023, the journal eLife ended the practice of making binary accept/reject decisions following peer review, instead sharing peer review reports (for manuscripts that are peer-reviewed) and brief “eLife assessments” representing the consensus opinions of editors and peer reviewers. As part of these assessments, the journal draws language from a "common vocabulary" to linguistically rank the significance of findings and strength of empirical support for the article's conclusions. In particular, the significance of findings is described using an ordinal scale of terms from "landmark" → "fundamental" → "important" → "valuable" → "useful", while the strength of support is ranked across six descending levels from "exceptional" down to "inadequate".
 
In the current study, Hardwicke et al. (2023) question the validity of this taxonomy, noting a range of linguistic ambiguities and counterintuitive characteristics that may undermine the communication of research evaluations to readers. Given the centrality of this common vocabulary to the journal's policy, the authors propose a study to explore whether the language used in the eLife assessments will be interpreted as intended by readers. Using a repeated-measures experimental design, they will tackle three aims: first, to understand the extent to which people share similar interpretations of phrases used to describe scientific research; second, to reveal the extent to which people’s implicit ranking of phrases used to describe scientific research aligns with each other and with the intended ranking; and third, to test whether phrases used to describe scientific research have overlapping interpretations. The proposed study has the potential to make a useful contribution to metascience, as well as being a valuable source of information for other journals potentially interested in following the novel path made by eLife.
 
The Stage 1 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/mkbtp
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Hardwicke, T. E., Schiavone, S., Clarke, B. & Vazire, S. (2023). Finding the right words to evaluate research: An empirical appraisal of eLife’s assessment vocabulary. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/mkbtp
avatar

FIELD Sarahanne Miranda

  • CWTS, Leiden University, Leiden, Netherlands
  • Social sciences
  • recommender

Recommendation:  1

Reviews:  0

Website https://www.sarahannemfield.eu
Areas of expertise
**Education** -Bachelor of Psychology (honours I; 2014; University of Newcastle, Australia) -Research Master in Psychometrics and Statistics (2017; University of Groningen, the Netherlands) -PhD in Behavioural and Social Sciences (topic: science reform movement; 2022; University of Groningen, The Netherlands) **Research Interests, Modus Operandi and Philosophy** Broadly, I am a metascientist. I research science (the scientific community, practices etc) using the scientific method. More specifically, I am a science reform scholar. I am interested in science reform as a social movement, the communities within the science reform movement, reform practices and critique of the reform movement. I don't just study reform objectively, however. I am an activist as well, and advocate most reform/open research initiatives (such as registered reports!)... as long as they're critically engaged with by users before adoption. I am currently a postdoctoral fellow working at the centre for science and technology studies (CWTS) at Leiden University in the Netherlands. I am on a project with Sarah de Rijcke (CWTS), Bart Penders (University of Maastricht), Jackie Thompson (Bristol University) and Marcus Munafò (Bristol University), working on establishing a new, updated responsible research and innovation (RRI) framework, and embedding it in local contexts in the UK and parts of Europe. I am a mixed methodologist, with training and expertise in both qualitative and quantitative methods. I value reflexivity and good scientific mentorship. I am passionate about considering alternatives to traditional publishing systems, and about dismantling barriers to participation in reform (relating to e.g., power imbalances, hierarchy in research, accessibility and inclusivity). I would argue that the registered report format, though not a panacea, might be one of the most impactful and valuable research reform initiatives that will come of my 'generation'. I have conducted studies involving preregistration and registered reports, replication and selecting replication targets, reflexivity (especially for use in quantitative research contexts), and Bayesian analyses and reanalyses. I am (to varying degrees) familiar with digital and virtual ethnography, social network analysis, the Bayesian approach, and thematic analysis (both done 'by hand' and done using QDAS).