This recommendation concerns the plan of two studies that are intended to be conducted simultaneously, using the same data collection approach, and to result in two manuscripts that will be submitted for assessment at Stage 2. The Stage 1 manuscript containing these protocols was submitted via the programmatic track.

 

Protocol 1 concerns “Open Scholarship and Feedback in Applied Research: A Qualitative Registered Report”. With this study, the authors aim to explore how applied researchers integrate feedback processes into their work, in relation to transparency and rigor in particular. They will investigate whether their sample are aware of and use feedback mechanisms from the open science movement, such as registered reports, which makes this study nicely metascientific. Through interviews with 50 applied researchers from various fields, the study will examine current feedback practices. The authors intend to use the findings of this first study to inform recommendations on how open science practices can be incorporated into research workflows.

 

Protocol 2 concerns “Understanding the Role of Climate Change in Applied Research: A Qualitative Registered Report”. This study aims to explore how applied researchers address climate change in their work, including the ways their practices are influenced by and respond to climate challenges. It addresses how their approaches may evolve, and they plan to look into the barriers and opportunities climate change presents in practice. Interviews with 50 applied researchers will be analysed to help understand these dynamics. The authors aim to provide recommendations to help applied researchers and their employers adjust their priorities to align with the urgency of climate action. One reviewer did not comment on this second protocol, as the content was outside of their own research area. Although I would have found a reviewer who specializes in this area directly ideally, I found I could still rely on the other two reviewers and my own knowledge to assess this protocol.
 
General comments: As I mentioned in my initial assessment text, these studies were well planned from the get-go and the protocol nicely articulated those plans. The use of different colour highlighting clearly helped the reviewers target different elements of the protocol and give direct feedback on specific parts. It also helped prevent me from getting lost in all the details! Well done, once again, to the authors for making the distinction between the two studies so clear. I was also pleased at how well they balanced the information between the two protocols – this made it easier to see if there were deficiencies in either one somehow. Finally, I loved that reflexivity was considered by the authors. One suggestion by me is that the authors might consider providing a collective positionality statement to go with the trainees’ reflexivity statements (if this is already in the plan and I missed it, please forgive the oversight!) in the final studies, even if as part of an appendix. This is because the open science movement and climate change can both be controversial, and because of the nature of the qualitative approach I would like to understand a little of the stance the group takes towards these issues collectively if the authors think it’s appropriate. I understand that with a big group that might be difficult or impossible, but if it is possible I would like to see it. I would also like to see initials used in the manuscripts to indicate who was responsible for what analysis elements where possible. This allows for accountability and to attribute interpretation to specific individuals involved in the data analysis. Alternatively, individuals can be attributed in a statement at the end of each manuscript to serve the same purpose and be less awkward in the text. If this won't work for some reason, please motivate this decision. 

 

The three reviewers that took the time to go through the reports nevertheless had useful comments, most of which would have contributed to strengthening the plan and minimizing problematic bias later on. The authors took these comments seriously, and thoughtfully (cheerfully even) responded to each. In my estimation, each of the suggestions of the reviewers were satisfied by the authors’ response to reviews letter. Other than my earlier comment about the positionality statement, I have no further comments for the Stage 1 protocol, and I wish the authors all the best with running the studies and writing up Stage 2 for each.

 

URL to the preregistered Stage 1 protocol: https://osf.io/jdh32
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

 

 

Evans, T. R. et al. (2024). Open Scholarship and Feedback in Applied Research/Understanding the Role of Climate Change in Applied Research: A Qualitative Registered Report. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/jdh32