Recommendation

Is the N2pc a correlate of attentional selection? An #EEGManyLabs multi-lab registered replication of Eimer (1996)

ORCID_LOGO based on reviews by Reny Baykova and Clayton Hickey
A recommendation of:

A multilab investigation into the N2pc as an indicator of attentional selectivity: Direct replication of Eimer (1996)

Abstract

EN
AR
ES
FR
HI
JA
PT
RU
ZH-CN
Submission: posted 24 February 2023
Recommendation: posted 17 May 2023, validated 18 May 2023
Cite this recommendation as:
Sherman, M. (2023) Is the N2pc a correlate of attentional selection? An #EEGManyLabs multi-lab registered replication of Eimer (1996). Peer Community in Registered Reports, . https://rr.peercommunityin.org/articles/rec?id=411

Recommendation

The N2pc is a lateralised ERP component that is often interpreted as a marker of attentional allocation, so much so that it is frequently used a tool in the attention literature for inferring that a stimulus was attentionally processed. This interpretation of N2pc can be traced back to the seminal work of Eimer (1996), which has been conceptually replicated many times but has never been replicated directly.
 
This registered direct replication by Constant et al. (2023) forms part of a larger series of large-scale, multi-lab replications of highly influential EEG papers by the #EEGManyLabs project (Pavlov et al., 2021). Seven labs (with the potential for more to sign up later), will conduct high-powered replications of the critical Experiment 2 of Eimer (1996), where in the crucial conditions, participants discriminate a target letter (M vs W) or colour (blue vs green) in the presence of a distractor. Using four preprocessing pipelines, including the original, the authors will test whether the N2pc is observed over parieto-occipital electrodes contralateral to target presentation.  
 
The Stage 1 manuscript was evaluated over one round of in-depth review and one additional round of minor corrections. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/dw68r
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA (so-called “primary RR”)
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Constant, M., Mandal, A., Asanowicz, D., Yamaguchi, M., Gillmeister, H., Kerzel, D., Luque, D., Pesciarelli, F., Fehr, T., Mushtaq, F., Pavlov, Y. G. & Liesefeld, H. R. (2023). A multilab investigation into the N2pc as an indicator of attentional selectivity: Direct replication of Eimer (1996), in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/dw68r
 
2. Eimer, M. (1996). The N2pc component as an indicator of attentional selectivity. Electroencephalography and Clinical Neurophysiology, 99, 225-234. https://doi.org/10.1016/0013-4694(96)95711-9
 
3. Pavlov, Y. G., Adamian, N., Appelhoff, S., Arvaneh, M., Benwell, C. S., Beste, C., ... & Mushtaq, F. (2021). #EEGManyLabs: Investigating the replicability of influential EEG experiments. Cortex, 144, 213-229. https://doi.org/10.1016/j.cortex.2021.03.013
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Evaluation round #2

DOI or URL of the report: https://psyarxiv.com/3472y

Version of the report: 2

Author's Reply, 12 May 2023

Download tracked changes file

Dear Dr. Sherman,

Thank you for bringing to our attention the fact that we overlooked this small but indeed crucial detail about the priors. Thank you also for your suggestions and for the link.

We have discussed this among us and settled on using the default JASP settings (JZS prior). Our main analyses (in the context of the #EEGManyLabs project) do focus on frequentist statistics. As such, we want the BFs to be comparable to these results, while still allowing us to obtain evidence for the null. This can be achieved, we believe, by using the JZS prior.

We were not 100% sure of what you meant with "reporting the robustness region". We checked in JASP and the Bayesian paired-sample t test module does include a robustness check option which runs the analysis again with a "wide prior" (Cauchy width = 1.0) and "ultrawide prior" (Cauchy width = 1.4). We assumed that this is what you meant and have therefore decided to also report this robustness check.

We will also upload the .jasp files on our repository when we submit the stage 2 report. These files keep all the analyses and dataset within them, which allow any user to easily load them in JASP and change any settings (such as priors) in any analysis. So readers will easily be able to check whether Bayesian results hold with their prior of choice.

All changes are marked in red in the tracked changes manuscript and are located on page 6.

 

Kind regards,

Martin Constant and Heinrich Liesefeld

(on behalf of all listed co-authors)

 

Decision by ORCID_LOGO, posted 11 May 2023, validated 11 May 2023

Dear Martin Constant, Heinrich Liesefeld, and co-authors

Thank you for your thorough revision, which addresses all the reviewer comments. There is just one very minor issue outstanding, and then I'm very happy to recommend in-principle acceptance:

In the section Methods - Original pipeline, you write:

"In addition to these frequentist t tests, we will additionally perform directed Bayes Factor (BF) t tests with JASP (most recent version available at the time of data analysis; JASP Team, 2023; Love et al., 2019). A BF in favor of the null ≥ 3 or a BF in favor of the alternative ≥ 6 will be considered as sufficient evidence"

The text needs to include the prior you will use (e.g. "BFs will be calculated using the default Cauchy prior of width 0.707").

I'm assuming these will be calculated with the default Cauchy prior with scale parameter 0.7. This corresponds to a Cohen's d of approximately 2 (https://xeniaschmalz.blogspot.com/2019/09/justifying-bayesian-prior-parameters-in.html), which is very large - the predicted effect size is so large that it's relatively "easy" to find sensitive evidence for the null. Ideally the prior would be determined by the effect size of Eimer 1960, but (as was typical at the time) it's not reported, and descriptives aren't given/error bars aren't plotted. So, could you please make one of the following changes  -

1. Calculate the approximate standardised effect size from Eimer 1960, estimating the means (eyeballed from the plots) and estimating the SE from the ANOVA F stat and dfs, then use an informed prior (converting cohens d to a scale parameter using the information in that link above). Either a directional cauchy or half-normal prior is fine.

2. Use a standardised effect size from another, related study to inform your prior - doesn't matter whether you use a half-normal or cauchy

3. Use the default cauchy prior of 0.7 (again, one-tailed) and also report the robustness region (I'm 99% sure JASP can give you this). This way, readers can see whether the same effect would be found under their prior of choice

Whichever you choose, could you please describe your choice of prior in the manuscript. 

There is a small chance that the bayesian and frequentist results will diverge. It may be worth stating that you will draw your main inferences from the frequentist statistics (assuming that is indeed your preference).

 

Best wishes

Maxine

 

 


Evaluation round #1

DOI or URL of the report: https://psyarxiv.com/3472y

Version of the report: 1

Author's Reply, 08 May 2023

Decision by ORCID_LOGO, posted 11 Apr 2023, validated 11 Apr 2023

Dear Dr Constant,

Thank you for submission, and please accept my apologies for the delay in getting back to you. Your Stage One submission has received two high-quality reviews from Dr Clayton Hickey and Dr Reny Baykova, and I have provided a third review, which I have appended below. 

All reviewers agree that the submission is excellent, and request only very minor points of clarification. Once these have been completed (or rebutted) then I will go ahead recommend the report. 

Dr Baykova suggested that Eimer's third hypothesis in the original 1996 paper is also included as a target for replication here - it is not neccessary for you to do this, but if you decide against could you briefly explain why in the Introduction.

Kind regards,

Maxine Sherman

 

Review: Maxine Sherman

The submission is excellent and I have only two very minor comments.

1) The authors write:

 “The most representative result are the effects of contralaterality in Study 2 (which is the replicated study) for electrode pair OL (corresponding to PO7/8 in the 10-10 system) in the time range 220 – 300 ms for form discrimination F(1,9) = 57.10, p < .001 and color discrimination F(1,9) = 17.48, p = .002; thus the smallest of these two F values (17.48) is used to compute the effect size”

Could you clarify what is meant by representative result? Do you mean the result in Experiment 2 that is most representative of the interpretations of the original paper?

2) The authors write that they will report a meta analytic effect size for the original pipeline. Will this also be done for the alternative pipelines? If not, how will the across-labs results from these pipelines be reported, and how (if at all) will they be compared to results from the original pipeline?

 

Typos

  • p.2 “The N2pc to forms appeared to be larger in amplitude and in additional time windows compared to the N2pc for color patches.”

Should be "The N2pc forms appeared"

  • p.6 “The collapsed localizer usually consists in averaging all participants and conditions together, and then deciding on the analysis window based on this single waveform.”

‘…usually consists in’ should be ‘usually consists of’

  • p.7 “The most representative result are the effects of contralaterality in Study 2”

result should be results

 

Reviewed by , 29 Mar 2023

Reviewed by , 16 Mar 2023

The protocol is careful and entirely suitable for reproduction of the target study. I see no need for revision.

I have a couple of minor comments on the introduction:

P 2 – ‘Eimer’s finding does not exclude alternative interpretations… might be a composite reflecting both enhancement of the relevant stimulus and suppression of the irrelevant stimulus…’ There is nuance that is missed here. In Hickey et al., 2009, we suggested a couple of alternatives. As described in the RR, we concluded with some confidence that the Pd reflects direct suppression of the neural response to the distractor. We were less sure what computational mechanism the Nt represented. We offered the possibility that it might represent some sort of target enhancement. But we kept open the idea that it might reflect a mechanism of distractor suppression (in addition to the mechanism reflected in Pd). That is, the Nt could reflect activation of cells responsible for the ‘silencing’ of distractor input into cortex responsible for representation of the target (eg. by inhibiting cells in cortical layers receiving lateral or feedback input). This kind of sheltering mechanism would emerge in tissue that is physically proximal to cortex representing the target, and would emerge contralateral to the target (not the distractor).

This idea was broadly motivated by literature emerging at that time on the role of alpha in vision, and on the possibility of GABA-ergic circuits involved in attentional control. One reason to like the sheltering account is that it easily explains why the N2pc is so sensitive to the proximity, presence, and nature of distractor stimuli. It’s also broadly in line with the strong role for distractor suppression that emerges in animal electrophysiology studies of attention. To my knowledge, it remains a valid interpretation.

P 2 – ‘…only few N2pc studies have presented the relevant stimulus without surrounding elements…’ We went even a little further than this in Hilimire, Hickey, & Corballis, 2011, Psychophysiology, and only presented one stim at a time.

Signed, Clayton Hickey

User comments

No user comments yet