Recommendation
DOI or URL of the report: https://osf.io/m7g4a?view_only=8bc552d2d4c34017adf87cf396b3ea9c
Version of the report: 1
, posted 22 Sep 2024, validated 23 Sep 2024Dear authors
Your Stage 2 manuscript has now been reviewed by two of the original reviewers.
Before I get to their comments, I point out a bigger procedural issue: you integrated parts of your Stage 1 Methods ("Data analysis and preprocessing") into your Stage 2 Results and this has led to significant changes to the text. Unfortunately, this is not really acceptable. I appreciate why you did this: You are describing the methodology as you present results but unfortunately this confounds comparison with the preregistered version. Ideally you should revert back the methods as they were in the Stage 1 version that received In-Principle Acceptance. Arguably, it could be acceptable to start the Results section at the point where you describe testing the individual hypotheses (page 20 in the IPA version). Note that your approach seems to be above board to me; it is largely a matter of facilitating comparison between the preregistration and the final article.
The two reviewers are generally very positive. One of them has some suggestions for improving the clarity and delving deeper into what the data might mean. Further exploration of the data are theoretically optional but the reviewer's suggestion is sensible, so while it is optional to go beyond analyses preregistered in Stage 1 I would encourage you to add this.
Given the positive comments by reviewers, ideally you can make these changes without necessitating a further round of external review. Please contact me directly with any questions about the approach to take here and we can perhaps arrive at a reasonable compromise between having a good narrative flow and a robust separation of the preregistered methods.
Sam Schwarzkopf
This is a rigorous and very well written stage 2 report. I have no suggestions on how to further improve this manuscript. I also confirm that:
- The data are able to test the authors’ proposed hypotheses.
- The introduction, rationale and stated hypotheses are the same as the approved Stage 1 submission.
- The authors adhered precisely to the registered study procedures.
- The authors’ conclusions are justified given the evidence.
, 22 Sep 2024Declaration regarding possible conflict of interest: The manuscript now includes an additional author, Arne Stein, who was a research assistant and thesis student in my lab. I am currently co-supervising his Master’s thesis. I was not aware of this new collaboration when I accepted the Stage 2 review. I will provide my assessment nevertheless (and I believe it is impartial), but wanted to make this potential conflict of interest transparent to everyone involved.
This is a very neat manuscript. The introduction is clearly written and shows good scholarship. The experiments are very convincing and the hypotheses cover a range of interesting questions. Small deviations from the analysis plan have been appropriately highlighted and explained.
The results are very clear, and all sanity checks were successful: The initial pupil response is largely absent in the fading trials compared to a strong response in the non-fading trials (resulting in overall larger pupil size, in line with Hypothesis 2). Attending a specific location significantly modulated the pupil response (smaller pupil when the brighter stimulus was attended; Hypothesis 1) and accuracy was higher at the attended compared to the unattended location (Hypothesis 6). The rest of the results were also decisive and quite exciting — the impact of covert attention was smaller but still quite substantial in the condition in which the stimulus faded compared to the non-fading condition and this effect was independent of perceptual experience of fading strength. Overall, this paper will make an interesting contribution to the literature on cognitive influences on pupil size and on the relation between perception and oculomotor
I have some suggestions:
1. It is my understanding that the data in Figure 2D include only trials from fading blocks. If that is correct, then it should be highlighted in Figure 2D’s caption. If this understanding is not correct, it would be good to separate them for the fading vs non-fading blocks. In any case, it would be useful for readers to see a report/figure of the distributions of fading self-reports in those two types of blocks.
2. The authors conclude that the impact of covert attention on pupil size is modulated by stimulus luminance even when the subjective experience of brightness is substantially reduced. Did pupil size have the same chance of being altered in the two conditions? There is a limit to how strongly pupil size can increase or decrease. Could it be that these limits were closer to the observed pupil size in the conditions in which subjects experienced fading? If so, it could explain the smaller effect of covert attention in that condition. What are the arguments against this possibility?
3. The last line on page 33 of the submitted document reads “for example when making an eye movement to the dark or bright spots”. I am not sure what the authors want to say here and why they bring up eye movements in this context.
4. As I have pointed out every time: Given that saccades and blinks strongly contribute to counteracting fading, it would be great to see an analysis of eye movements (number of blinks and saccades and saccade amplitude distribution) as a function of reported fading in both blocks.
