Dear Martin Constant, Heinrich Liesefeld, and co-authors

Thank you for your thorough revision, which addresses all the reviewer comments. There is just one very minor issue outstanding, and then I'm very happy to recommend in-principle acceptance:

In the section Methods - Original pipeline, you write:

"In addition to these frequentist t tests, we will additionally perform directed Bayes Factor (BF) t tests with JASP (most recent version available at the time of data analysis; JASP Team, 2023; Love et al., 2019). A BF in favor of the null ≥ 3 or a BF in favor of the alternative ≥ 6 will be considered as sufficient evidence"

The text needs to include the prior you will use (e.g. "BFs will be calculated using the default Cauchy prior of width 0.707").

I'm assuming these will be calculated with the default Cauchy prior with scale parameter 0.7. This corresponds to a Cohen's d of approximately 2 (https://xeniaschmalz.blogspot.com/2019/09/justifying-bayesian-prior-parameters-in.html), which is very large - the predicted effect size is so large that it's relatively "easy" to find sensitive evidence for the null. Ideally the prior would be determined by the effect size of Eimer 1960, but (as was typical at the time) it's not reported, and descriptives aren't given/error bars aren't plotted. So, could you please make one of the following changes  -

1. Calculate the approximate standardised effect size from Eimer 1960, estimating the means (eyeballed from the plots) and estimating the SE from the ANOVA F stat and dfs, then use an informed prior (converting cohens d to a scale parameter using the information in that link above). Either a directional cauchy or half-normal prior is fine.

2. Use a standardised effect size from another, related study to inform your prior - doesn't matter whether you use a half-normal or cauchy

3. Use the default cauchy prior of 0.7 (again, one-tailed) and also report the robustness region (I'm 99% sure JASP can give you this). This way, readers can see whether the same effect would be found under their prior of choice

Whichever you choose, could you please describe your choice of prior in the manuscript. 

There is a small chance that the bayesian and frequentist results will diverge. It may be worth stating that you will draw your main inferences from the frequentist statistics (assuming that is indeed your preference).

Best wishes

Maxine

Dear Dr Constant,

Thank you for submission, and please accept my apologies for the delay in getting back to you. Your Stage One submission has received two high-quality reviews from Dr Clayton Hickey and Dr Reny Baykova, and I have provided a third review, which I have appended below. 

All reviewers agree that the submission is excellent, and request only very minor points of clarification. Once these have been completed (or rebutted) then I will go ahead recommend the report. 

Dr Baykova suggested that Eimer's third hypothesis in the original 1996 paper is also included as a target for replication here - it is not neccessary for you to do this, but if you decide against could you briefly explain why in the Introduction.

Kind regards,

Maxine Sherman

Review: Maxine Sherman

The submission is excellent and I have only two very minor comments.

1) The authors write:

 “The most representative result are the effects of contralaterality in Study 2 (which is the replicated study) for electrode pair OL (corresponding to PO7/8 in the 10-10 system) in the time range 220 – 300 ms for form discrimination F(1,9) = 57.10, p < .001 and color discrimination F(1,9) = 17.48, p = .002; thus the smallest of these two F values (17.48) is used to compute the effect size”

Could you clarify what is meant by representative result? Do you mean the result in Experiment 2 that is most representative of the interpretations of the original paper?

2) The authors write that they will report a meta analytic effect size for the original pipeline. Will this also be done for the alternative pipelines? If not, how will the across-labs results from these pipelines be reported, and how (if at all) will they be compared to results from the original pipeline?

Typos

• p.2 “The N2pc to forms appeared to be larger in amplitude and in additional time windows compared to the N2pc for color patches.”

Should be "The N2pc forms appeared"

• p.6 “The collapsed localizer usually consists in averaging all participants and conditions together, and then deciding on the analysis window based on this single waveform.”

‘…usually consists in’ should be ‘usually consists of’

• p.7 “The most representative result are the effects of contralaterality in Study 2”

result should be results