This recommendation concerns the plan of two studies that are intended to be conducted simultaneously, using the same data collection approach, and to result in two manuscripts that will be submitted for assessment at Stage 2. The Stage 1 manuscript containing these protocols was submitted via the programmatic track.

 

Protocol 1 concerns “Open Scholarship and Feedback in Applied Research: A Qualitative Registered Report”. With this study, the authors aim to explore how applied researchers integrate feedback processes into their work, in relation to transparency and rigor in particular. They will investigate whether their sample are aware of and use feedback mechanisms from the open science movement, such as registered reports, which makes this study nicely metascientific. Through interviews with 50 applied researchers from various fields, the study will examine current feedback practices. The authors intend to use the findings of this first study to inform recommendations on how open science practices can be incorporated into research workflows.

 

Protocol 2 concerns “Understanding the Role of Climate Change in Applied Research: A Qualitative Registered Report”. This study aims to explore how applied researchers address climate change in their work, including the ways their practices are influenced by and respond to climate challenges. It addresses how their approaches may evolve, and they plan to look into the barriers and opportunities climate change presents in practice. Interviews with 50 applied researchers will be analysed to help understand these dynamics. The authors aim to provide recommendations to help applied researchers and their employers adjust their priorities to align with the urgency of climate action. One reviewer did not comment on this second protocol, as the content was outside of their own research area. Although I would have found a reviewer who specializes in this area directly ideally, I found I could still rely on the other two reviewers and my own knowledge to assess this protocol.
 
General comments: As I mentioned in my initial assessment text, these studies were well planned from the get-go and the protocol nicely articulated those plans. The use of different colour highlighting clearly helped the reviewers target different elements of the protocol and give direct feedback on specific parts. It also helped prevent me from getting lost in all the details! Well done, once again, to the authors for making the distinction between the two studies so clear. I was also pleased at how well they balanced the information between the two protocols – this made it easier to see if there were deficiencies in either one somehow. Finally, I loved that reflexivity was considered by the authors. One suggestion by me is that the authors might consider providing a collective positionality statement to go with the trainees’ reflexivity statements (if this is already in the plan and I missed it, please forgive the oversight!) in the final studies, even if as part of an appendix. This is because the open science movement and climate change can both be controversial, and because of the nature of the qualitative approach I would like to understand a little of the stance the group takes towards these issues collectively if the authors think it’s appropriate. I understand that with a big group that might be difficult or impossible, but if it is possible I would like to see it. I would also like to see initials used in the manuscripts to indicate who was responsible for what analysis elements where possible. This allows for accountability and to attribute interpretation to specific individuals involved in the data analysis. Alternatively, individuals can be attributed in a statement at the end of each manuscript to serve the same purpose and be less awkward in the text. If this won't work for some reason, please motivate this decision. 

 

The three reviewers that took the time to go through the reports nevertheless had useful comments, most of which would have contributed to strengthening the plan and minimizing problematic bias later on. The authors took these comments seriously, and thoughtfully (cheerfully even) responded to each. In my estimation, each of the suggestions of the reviewers were satisfied by the authors’ response to reviews letter. Other than my earlier comment about the positionality statement, I have no further comments for the Stage 1 protocol, and I wish the authors all the best with running the studies and writing up Stage 2 for each.

 

URL to the preregistered Stage 1 protocol: https://osf.io/jdh32
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

 

 

Evans, T. R. et al. (2024). Open Scholarship and Feedback in Applied Research/Understanding the Role of Climate Change in Applied Research: A Qualitative Registered Report. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/jdh32

The enhanced processing account suggests that domain-specific expertise enhances the processing of information in that domain, such as enhanced color perception in visual artists and grapheme-color synaesthetes. A previous study (Ovalle-Fresa et al., 2021) found support for this account in both synaesthetes and non-synaesthete color experts; however, to fully understand the domain specificity of enhanced processing, other domains of expertise should be investigated and compared against each other in a double dissociation.

 

In this Stage 1 manuscript, Sachdeva et al. (2024) will investigate whether an enhanced processing account can explain domain-specific expertise in groups of color experts (i.e., visual artists) / grapheme-color synaesthetes and spatial experts (i.e., architects) / sequence-space synaesthetes. The spatial domain was chosen to compare to color since color and spatial processing recruit two distinctive cortical visual pathways: the ventral and dorsal streams, respectively. If enhanced processing is domain specific, then color experts / grapheme-color synaesthetes should show selective perceptual and memory performance enhancements for color tasks but not spatial tasks; and vice versa for spatial experts and sequence-space synaesthetes. The authors of this planned study further propose that perceptual performance should predict memory performance in the domain of expertise only; and that synaesthetes and non-synaesthetes with the same domain of expertise should perform similarly to each other.

 

To address these hypotheses, four independent groups will be recruited: grapheme-color synaesthetes, visual artists, sequence-space synaesthetes, and architects. Individuals with overlapping expertise and synaesthesia (e.g., visual artists who also have grapheme-color synaesthesia) will be excluded. Perception, short-term memory, and long-term memory for color (3 tasks) and space (3 tasks) will be assessed in all groups, and the authors will compare groups (synaesthetes, non-synaesthetes), domains of expertise (color, space), and task feature (color, space) in 2x2x2 linear mixed models. For all models, the authors predict that a significant interaction between domain of expertise and task feature will provide evidence for the domain specificity of the enhanced processing account. Additional analyses concerning working memory load (one, three, or five items) in the short-term memory tasks, and testing day (one, two, or three days post-training) in the long-term memory tasks, will be conducted to more deeply explore potential performance enhancements related to domain-specific expertise. Although these analyses may potentially provide additional evidence in favor of the hypothesized direction of effects, any deviation from predicted may pose a challenge for the interpretation of results. Nevertheless, this planned study is methodologically rigorous, and comprehensive in its aims.

 

The Stage 1 submission was evaluated by the recommender and two expert reviewers. Following revisions, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).

 

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

References
 

1. Ovalle-Fresa, R., Ankner, S., & Rothen, N. (2021). Enhanced perception and memory: Insights from synesthesia and expertise. Cortex, 140, 14-25. https://doi.org/10.1016/j.cortex.2021.01.024

2. Sachdeva, D. & Whelan, E., Ovalle-Fresa, R., Rey-Mermet, A., Ward, J., & Rothen, N.. (2024). How perceptual ability shapes memory: An investigation in healthy special populations. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/6wn4m

 

Higher-level cognitive processes like attention, memory, or mental imagery can produce pupil responses, without any actual difference in luminance of the visual stimulus. Interestingly, the inverse scenario remained untested to date: when a physical luminance difference is perceptually eliminated from awareness, does pupil size still track attention to the stimulus? In this Registered Report, Vilotijević and Mathôt (2024) sought to test this experimentally using a perceptual fading phenomenon where two Gaussian patches with different luminances fade from consciousness and are thus perceived as mid-level uniform grey (or at least the subjective difference is much reduced). This fading manipulation, as well as a control condition without perceptual fading, were presented in separate blocks. Participants were instructed to covertly attend one of the patches.

The authors hypothesised that if pupil size reflects attentional selection, these pupil responses in the fading condition​ should be eliminated or at least reduced, and this should evolve with time as the stimuli are perceptually fading. Their results show that pupil responses during covert attention are indeed reduced during perceptual fading - but they are not eliminated. Interestingly, this reduction did not depend on time or self-reports of the strength of perceptual fading. The findings therefore suggest that pupil dilation tracks subjective brightness differences.

 

One inherent issue with experiments like these is that the experimental and control conditions necessarily involve a physical difference in the stimulus. Here, the fading condition had the same spatial configuration of light and dark stimuli throughout a block while in the control (non-fading) condition the light and dark stimuli alternated sides between trials. It is therefore impossible to completely rule out that the physical difference affects the results. However, the only alternative to this would be an experimental design in which the stimuli never change, but only the subjective perceptual state varies. Such a design is completely at the mercy of the participant's subjective state and therefore loses experimental control and statistical sensitivity. The present results confirmed the authors' prediction that there are indeed differences in overall pupil responses during the fading and control conditions, irrespective of covert attention.

 

Critically, the fact that the attention effect did not vary with time or subjective self-reports of the illusion supports the authors' interpretation that this reflects higher-level cognition: the mere act of attending to the dark side - even if the actual appearance has faded - could cause a sustained pupil dilation. This would be consistent with the type of pupil effects for memory and mental imagery that motivated the present study. However, a simpler alternative is that the experience of perceptual fading was incomplete (as possibly suggested by Figures 2D and 2F) but that self-reports fail to capture this subjective experience accurately. Perhaps a future study could compare the magnitude of the attentional pupil effects when the initial stimulus is completely removed. If similar differences in pupil response persist this would suggest that the present results are due to high-level modulation or the residual low-level luminance difference.

 

The Stage 2 manuscript was evaluated by two reviewers and the recommender over two rounds of review. One reviewer again advised additional robustness checks to rule out eye movement confounds, an issue they had already raised during Stage 1 review. The researchers provide clear evidence that this is unlikely to have confounded their findings. This has been added to the supplementary data repository. Following this review and revision, the recommender judged that the Stage 2 criteria were met and awarded a positive recommendation.

 

URL to the preregistered Stage 1 protocol: https://osf.io/bmtp6
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:

 

 

 

1. Vilotijević, A. & Mathôt, S. (2024). The effect of covert visual attention on pupil size during perceptual fading [Stage 2]. Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/ku8qc?view_only=f331df53b50f431386fabba9e386b387

 

Experimental research demonstrates that executing or inhibiting motor responses (or approaching / avoiding) towards a stimulus can alter the valuation of the stimulus (Yang et al., 2022). There are competing theories as to the proposed mechanisms of value change, such as increased response conflict or prediction errors (Houben & Aulbach, 2023). However, research has mostly examined response execution/inhibition and approach/avoidance in isolation and the few studies that have examined these together have focused on stimulus evaluation as an outcome.
 
In the present study, Chen et al. (2024) will use a novel version of a combined go/no-go / approach avoidance paradigm to test the effects on choice of consumable candy. In this task, participants are randomly assigned to make a response framed as a go / no-go action or an approach / avoidance action to control a shopping cart (Chen & Van Dessel, 2024). Following this they will complete a food choice task in which participants make a series of binary choices for different candies. Their performance on this task will lead to the receipt of real-world candy. The authors aim to test whether the same responses will lead to different effects on food choice, depending on how the response was interpreted (e.g. participants in the approach/avoidance instruction group will select Approach items more often than those in the go/no-go instruction group). The study is well powered to detect the proposed effect size of interest, and data will be analysed using Bayesian mixed-effect models.
 
This study will shed light onto theoretical predictions of action interpretation on stimulus value and choice, which may improve the efficacy of behaviour change tools such as approach bias training in future.  
 

The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the recommender and reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).

 

URL to the preregistered Stage 1 protocol: https://osf.io/bn5xa (under temporary private embargo)

 

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 

 

 
1. Chen, Z. and Van Dessel, P. (2024). Action interpretation determines the effects of go/no-go and approach/avoidance actions on stimulus evaluation. Open Mind, 8, 898-923.  https://doi.org/10.1162/opmi_a_00151

 

2. Chen, Z., Van Dessel, P., and Figner, B. (2024). Examining the role of action interpretation in changes in choice induced by go/no-go and approach/avoidance responses. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/bn5xa
 
3. Houben, K. and Aulbach, M. (2023). Is there a difference between stopping and avoiding? A review of the mechanisms underlying Go/No-Go and Approach-Avoidance training for food choice. Current Opinion in Behavioral Sciences, 49, 101245. https://doi.org/10.1016/j.cobeha.2022.101245
 
4. Yang, Y., Qi, L., Morys, F., Wu, Q. and Chen, H. (2022). Food-Specific Inhibition Training for Food Devaluation: A Meta-Analysis. Nutrients, 14, 1363. https://doi.org/10.3390/nu14071363
 

Studying eye-gaze has long been employed as a central method for understanding attentional dynamics and cognitive processes in a variety of domains. The development of affordable, mobile eye-tracking devices, such as the Pupil Neon, promises new opportunities to extend this research beyond the contexts in which traditional eye-trackers have been available. But how good are such novel devices at detecting variables relevant for the study of eye movements and pupil dilations?

 

Foucher, Krug and Sauter (2024) propose an independent evaluation of the Pupil Neon eye-tracker using the Ehinger et al. (2019) test battery, comparing its performance with a traditional EyeLink 1000 Plus device. In an empirical study, participants will be asked to perform a wide variety of tasks while eye movements and pupil dilations are tracked using both devices. Results on the strengths and weaknesses, as well as potential use cases of the Pupil Neon will be informative for subsequent eye-tracking research. 

The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the recommender and reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).

 

 

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 

 

 

1. Ehinger, B. V., Groß, K., Ibs, I., and König, P. (2019). A New Comprehensive Eye-Tracking Test Battery Concurrently Evaluating the Pupil Labs Glasses and the Eyelink 1000. PeerJ, 7, e7086. https://doi.org/10.7717/peerj.7086

2. Foucher, V., Krug, A., and Sauter, M. (2024). Independent Comparative Evaluation of the Pupil Neon - A New Mobile Eye-tracker. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/3kc5t