I am willing to accept the authors' reply to my concern. I still think it will be important to also analyse the impact of eye movements in their paradigm once the data is collected.

I am happy with the way the authors have addressed my comments, but I advise the authors to look once more at the beginning of the abstract when writing the stage-2 report:

"Pupil size is modulated by various cognitive factors such as attention, working memory, mental imagery, and subjective perception. Previous studies on this topic mainly focused on inducing or enhancing a subjective experience of brightness or darkness (for example by asking participants to attend to a bright or dark stimulus), and then showing that this affects pupil size."

My point is that readers, when reading the second sentence, won't know what 'this topic' refers to, because the topic of their paper is not mentioned in the first sentence. 

Review of PCI Registered Report 

The effect of covert visual attention on pupil size during perceptual fading

by Ana Vilotijević, Sebastiaan Mathôt

The authors have addressed all comments raised by the three reviewers. They have done an excellent job at responding to my concerns, perhaps with one exception: the role of eye movements. Given that all three reviewers and the editor brought up the potential impact of eye movements, I would like to come back to this once more. The authors present two main arguments. First, they argue that they use stimuli with fuzzy edges for which microsaccades should not play a role. Martinez-Conde et al. (Neuron, 2006) and follow-up studies used stimuli with fuzzy edges, too, yet they showed that microsaccades are most efficient in preventing fading and restoring faded images (McCamy et al., J Neurosci, 2012; McCamy et al., J Physiol, 2014). While these authors used much smaller targets than the submitted study, the saccades they investigated were an order of magnitude smaller than the exclusion criteria proposed here. Leading to the second point: The exclusion criterion of looking more than 7.42 degrees away from the fixation point is far beyond microsaccades. A saccade of 5.77 degrees would displace the stimuli by 1 standard deviations of the stimulus’ Gaussian envelope. Eye movements of up to 7.42 degrees are almost certainly going to eliminate fading. I understand that subjective reports of fading are expected to be a valid source of information, but if any such trials contained large eye movements, the authors should investigate the role of these eye movements more closely. At the very least, they should test if eye movements indeed had no influence on fading and pupil size.

Side note that is relevant to the authors conclusion criterion: It is unclear what the stimulus inner edge is given the fuzzy nature of the stimulus edge. Please report the eccentricity of the stimulus center in the methods.

I thank the authors for their thorough response to our comments. I am satisfied with the revised manuscript and look forward to the results of this promising research.  

This is a very clear stage 1 RR. The research question makes sense and is clearly defined; the hypotheses are capable of answering the research question; the protocol is sufficiently detailed to enable replication by an expert in the field; the links between hypotheses, statistical tests and possible interpretations are clear; the sample size is sufficient; and the authors have built in several manipulation data quality checks. 

I have only a few minor comments:

- The text does not contain any information about eye movements right now: no instructions for subjects and no plans about how to deal with (unwanted) eye movements. These should be added.

- Two of the hypotheses rely on conventional null hypothesis significance testing to conclude evidence of absence from null results. For example: "Hypothesis 3.1 will be confirmed by the absence of a main
effect of Covertly attended brightness." The authors should use a method that allows one to draw such a conclusion, such as Bayesian hypothesis testing.

- About the writing, something to keep in mind for the stage 2 RR: The second sentence of the abstract does not follow from the first sentence.

The proposed study pursues the question how the change in perceived luminance for a perceptually fading, but physically unchanged stimulus affects pupil size. I applaud the authors for this intriguing idea and for submitting a Registered Report before starting the study. I list a number of thoughts and suggestions below.

1A. The scientific validity of the research question(s).

The literature review is highly informative with respect to the study, resulting in a scientifically valid and intriguing set of research questions.

1B. The logic, rationale, and plausibility of the proposed hypotheses, as applicable.

The authors specify five clearly spelled out hypotheses, including replications of previous studies (Hypothesis 1), sanity checks (Hypothesis 2), novel predictions (Hypotheses 3, 4, 5). The main hypothesis is that the attentional effect (attending to the location of a black vs white luminance patch) is reduced in the fading conditions. The rationale of the study makes sense, the hypothesis is derived from previous research and plausible based on pilot data included in the RR.

1C. The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis or alternative sampling plans where applicable).

The analysis plan is straight-forward and clearly described, including pre-processing steps that the authors have used in previous work. The authors provide a power analysis (90% power, alpha level = 2%) yielding an estimate of N = 24 given the effect size of a previously published study with a similar design, and round up to a planned N = 30. They specify the inferential analysis (LMM) they intend to use for each hypothesis in a way that leaves few researcher degrees of freedom.

There are a few points regarding the experimental procedure that need to be addressed, however. 

A report of perceptual fading follows each trial, but is not continuous in time. The authors should consider specifying to the participants for which moment in time they are supposed to report their perceptual state (e.g., at the time of target offset). Alternatively, they could track the degree of fading over time using a continuous psychophysical task (e.g., using a fader or a volume knob). This continuous report, however, would require a different set of analyses than planned so far. Another alternative would be to have participants press one of two buttons when the stimulus disappears or reappears, respectively, throughout each trial. This procedure is well established in the literature on rivalry and other perceptual alternations, including Troxler fading (e.g., Martinez-Conde et al., Neuron, 2006).

Speaking of which, it is not clear how the authors will deal with trials that contain microsaccades, saccades or blinks, which are known to counteract fading in this stimulus (Martinez-Conde et al., Neuron, 2006). Will these trials be excluded? Keeping track of perceptual state through continuous reports would provide an opportunity to analyse these occasions as a function of the perceptual state they elicit.

The authors use a detection task, but if I understand correctly, the staircase they use appears to focus on hits only (aiming for 70% correct), which means that performance (as defined by the authors) depends on participants’ criterion. It is inherently difficult to staircase d’, which would be solution to this dilemma, but that requires monitoring hits and false alarms/correct rejections. The authors should consider using the single-interval adjustment-matrix (SIAM) procedure  (Kaernbach, 1990), which has been successfully applied in detection tasks relating stimulus visibility to eye movement responses (White & Rolfs, 2016; White et al., 2022).

1D. Whether the clarity and degree of methodological detail is sufficient to closely replicate the proposed study procedures and analysis pipeline and to prevent undisclosed flexibility in the procedures and analyses.

The methods are described in sufficient detail to implement the study, with a few small exceptions.

The hexadecimal codes for colors are not very useful, as they depend on the specific display device being used (and its settings), and do not translate into wavelength spectra that would specify the stimulus. 

As stated above, it is unclear how exactly participants are instructed to report the perceptual state. What do they report, for instance, if fading set in during the trial? This should be specified to reduce differences between participants.

1E. Whether the authors have considered sufficient outcome-neutral conditions (e.g. absence of floor or ceiling effects; positive controls; other quality checks) for ensuring that the obtained results are able to test the stated hypotheses or answer the stated research question(s).

The authors propose contrasting two conditions, one of which aims at a replication of previous work (Non-Fading condition) and one of which is new. There are a couple of manipulation checks that would be advisable to make this data set as convincing as it should be.

First, we need to know that the attentional manipulation worked. An analysis of performance is not reported yet, but should be included in the Registered Report. Yet, this analysis alone will not speak to the successful manipulation of spatial attention, as there is no neutral or invalid condition to compare performance to. Introducing such conditions would be desirable. Note that given previous work showing effects of attention on pupil size, the authors could use a difference in pupil size as an indicator that attention was successfully deployed. But if Hypothesis 3.1 is true, this would only work in the Non-Fading condition.

Second, and more crucially, we need to know if attention is allocated to the same degree in both conditions. It could well be that the absence of a stimulus makes it easier to process the targets at the cued location. As a consequence, the attention shift might not be required (to the same degree) in the Fading as compared to the Non-Fading condition. The authors will thus need to show that performance in the detection task is comparable between the Fading and the Non-Fading condition. Given that they intend to equate performance across conditions (70% hits, but see my comment on using hit rates above), it would be necessary to show that the circles’ opacity was the same (on average).

Third, if the putatively reduced attentional effect on pupil size in Fading compared to the Non-Fading condition is indeed due to Fading, it would be important to show that this reduction is absent when no fading occurs, but the protocol is otherwise identical. The authors use alternating luminance blocks to prevent the buildup of fading over time. But it would be important to know if the alternation is also affecting the pupil response per sé, that is, as a consequence of alternating, not because on reduced Troxler fading. A negative control of this kind would be to use sharp-edged circles in a different set of blocks (either alternating or not alternating). This point is the least important of the three points so far, as it is also addressed to some extent by Hypothesis 5.

In this study, the authors propose to investigate the relationship between the attentional modulation on pupil size and the subjective experience of brightness using perceptual fading. I find the scientific question extremely interesting and definitely worth studying. I believe the experimental strategy is overall valid and well designed. The procedure is clearly explained. 

My main concern is about the non-fading block. I believe that it is not an ideal control, because of the different experimental procedure employed (no swapping vs. swapping), possibly introducing confounding factors.  I think that a more rigorous and efficient solution would be to design white and black stimuli so as to have a sufficiently high number of trials where participants report fading (3-4) and to use the remaining trials, where participants report 1 or 2 on the fading scale, as a control.This way the two conditions would differ only because of the degree of fading, leaving the experimental procedure unaffected. 

In any case, it would be useful if the authors would report the rate of fading of the selected stimuli on 2-3 pilot datasets. 

Finally, two minor comments: 

- Considering their role in adaptation, I believe that reporting eye movements data and their statistics across conditions would be useful.

-  The authors might be a bit more specific on how to quantify the target sample size for studies with linear-mixed effects models.