Constant et al. have replicated the second EEG experiment reported in Eimer (1996), which investigated the N2pc component of the visual event-related potential (ERP). The N2pc is commonly interpreted as a marker of attention, and the original study found that it can be elicited by different stimulus properties, namely shapes and colours. In addition, Eimer (1996) also found that the amplitude of the component was greater for shapes. Constant et al. conducted a replication of the original experiment as part of the EEGManyLabs project, and a total of 22 separate labs were involved in the replication. The N2pc effect using shapes was replicated, as well as the difference in amplitude between shapes and colours. The effect using colours was not replicated using the strict replication of the original pipeline and the ICA pipeline. It was, however, replicated using the collapsed localiser pipeline and in the additional exploratory analyses which didn’t use a fixed time window.  

Overall, this is a very ambitious project that was conducted with great care. The study provides a valuable contribution to the literature on the N2pc and ERPs more generally. I commend the authors on the impressive amount of work they undertook with this project, and I have only a few minor comments.

1. Transparency and openness statement

I would suggest adding links to the data, analysis scripts and stage 1 report in the transparency and openness statement so that they are easier to find.

2. Tables containing Bays factors

Apologies if this is a common notation I’m not familiar with, but I found the notation “BF-0” initially confusing.  I believe BF10 is the more commonly known notation, but if you prefer to go with BF-0  that’s completely fine, I’d just suggest adding a note to the tables to describe what it means.

3. Figure 17

Since each dot represents the average reaction time for each lab, I would suggest adding error bars to show the variability of the data (unless it all gets too messy).

In the description of the figure, I would suggest specifying that this shows reaction times on correct trials and trials kept after rejection.

4. Figure 18

Similar to my comments for Figure 17, if it isn’t too messy, it would be nice to add error bars, and describe what data was used in the figure description

5. Discussion

At the beginning of the discussion, you say that the colour manipulation is more influential because more studies use colour stimuli compared to shapes. Could you just provide some evidence to back this up?

Signed

Dr Reny Baykova

The paper does exactly what was described in the pre-registration, plus a lot of additional exploratory analysis that is very interesting. The results are not trivial and provide new insight on a seminal paper in cognitive neuroscience.

I have some comments; all are minor.

P2 – ‘Furthermore, the N2pc might reflect engagement at the location of the relevant stimulus rather…’ This is a bit hard to distinguish from the preceding hypotheses and needs some clarification. That is, an account of the N2pc that posits target enhancement is an account of the N2pc as engagement at the location of the relevant stimulus, not the shift of attention proper. Similarly, an account that posits distractor suppression is an account as engagement, not the shift of attention.

The N2pc is commonly used and discussed as an ‘index of the deployment of attention’ or of ‘attentional shifting’. This is not incorrect, but perhaps misleading… the N2pc is (mostly) an indirect index of these constructs. It reflects the large-scale change in striate and extrastriate cortical activity that is the consequence of attention. This was a salient observation in the early papers from Luck, Eimer and others that has become somewhat lost. The first proposal of the N2pc as a reflection of a control mechanism emerged in MEG work from Max Hopf’s group, where the involvement of posterior parietal cortex in generating the mN2pc was tentatively interpreted as reflecting the mechanistic implementation of attention, rather than its consequence on sensory activity.

p. 35 – ‘…the belief in fixed component timing is still widely held.’ Needs some sort of support for the authors’ impression / intuition / belief. I don’t have this same impression… people have been looking at effects on component latency in visual search, and on N2pc latency specifically. There are papers cited in the MS that demonstrate that, like Brisson et al 2007. Other ‘early’ latency effect papers looking at search or N2pc that come to mind:  

Smulders, Kok, Kenemans, Bashore (1995, Acta)
Wijers, Mulder & Mulder (1997, Psychopys)
Robitaille & Jolicoer, (2006, Neuroreport)
Kiesel, Miller, Jolicoeur, & Brisson (2008, Psychophys)
Hickey, van Zoest, & Theeuwes (2010, EBR)

More broadly, there were reports of P3 latency effects from Donchin and Kok in the ‘80s, probably the ‘70s too. I remember very early work from Falkenstein in the ‘90s with latency effects on Ne/ERN, either in the initial report or soon after. It could be more justifiable to say that the shift in N2pc timing observed here is larger than was expected across by the large group of authors contributing to this paper. 

Spelling etc:

p. 8 – ‘This copy was high-pass filtered at 2 Hz (passband edge)…’ for completeness, it would be good to have the same level of detail as for other filter descriptions.

p. 9 – ‘…that is, ERP amplitude at electrodes P07/9 are lower…’ to ‘…are more negative…’

p. 36 – The brightness of monitors has broadly increased over the last 40 years, and this could have something to do with the difference between the original report and these replications. CRT monitors were pretty dim by default, and and if you increased brightness the monitors degraded and became dimmer over time. If it was an early LED panel – unlikely as no one used them for vision research – then it’s even more probable, as the original fluorescent backlights were awful.

signed, Clayton Hickey