Can EEG complexity measures discriminate between visual- and auditory-evoked differences in conscious contents?

Dear authors,

Thank you for submitting a revised version of your manuscript. I have received two positive reviews, one from a previous reviewer and one from a reviewer who has not seen your report before.

There is no doubt that your protocol is well-designed and well thought-through given the complexity of your investigation. 

I have already drafted the In-Principle Acceptance for your study and I will be ready to issue it as soon as you address some important points listed below: 

1. I need to know whether you have already received an ethics approval from a relevant review board. If not, then I can only do a provisional IPA. The provisional IPA will be promoted to a full IPA as soon as you inform me about an obtained ethical approval. For more information, please refer to point 3.4. on the policies and procedures page
2. See the letter from reviewer Marcin Koculak. The letter is very positive and provides some useful points for consideration. The reviewer states that these points are not necessary to be addressed prior to IPA, however, I would like you to consider if you would like to change anything in your Stage 1 report prior to its finalisation based on these comments. I leave this decision up to you except for one specific point I would like you to address:
   
   
   • Please address the last point raised by the reviewer asking for clarifications of the rationale for your hypotheses.
     
     
3. Regarding your sampling plan - thank you for adding more information here. It is good to have a more conservative estimate. However, in your report text, it is not clearly stated what is your target sample size - I understand that the maximum sample size is 51 because it's clearly stated in Tables 1-3. This is not quite the case for the text. So this needs to be edited for clarity. 
   
   
   • In addition to the above, I don't see a justification for why 36 is the number from which you'd start sequential analyses to check if sufficient power has been reached in your study. Some justification is needed. 
   • Plus you need to clarify what sequential analyses you will conduct. So far you state that you will check BFs for the alternative and null hypotheses (BF>3, BF<⅓ respectively), but are you going to obtain BFs for all of your 17 hypotheses after each participant? What if the results provide sufficient BFs only for PCI and not LZ? Or only for some of the hypotheses and not all. There needs to be a clearer statement of when exactly data collection will stop based on the different possible outcomes from the sequential analyses.
     
     
4. Regarding your analysis plans - I see that you have decided to add BFs to also aid your interpretation of findings. In the last paragraph of your report, you write:
   "we will compute Bayes factors (to compare model evidence) and probabilities of direction (to evaluate the likelihood of positive or negative effects".
   You should also state what criteria will be used for BFs and probabilities of direction (e.g. BF>3, BF<⅓ ) and add this information both in text and in Tables 1-3 in the column "Hypothesis Test Sensitivity Rationale"

Finally, I would like to signpost you to the policy for changes between Stage 1 and Stage 2. I know that I have made this point before, but just want to remind you before Stage 1 is finalised that no major changes will be accepted to the introduction and method sections after IPA - point 3.10 in policies and procedures.

Since one of the reviewers asked about open data, I also want to bring up the TOP guidelines. Adherence to the TOP guidelines will be checked at Stage 2 submission.

Best wishes,

Marta Topor

Dear authors, 

I am pleased to let you know that we have now received three positive reviews of your stage 1 RR. Please read them carefully and respond to each reviewer point by point.  

In addition to the peer reviews, here are a few important points I need you to address in your next draft before we can proceed with a recommendation. 

1. Please bear in mind that your submitted stage 1 manuscript with abstract, introduction and method should be in the format of a research article intended for publication. We generally advise only minor and clearly highlighted textual changes in these sections between Stage 1 and Stage 2 submissions. Therefore:
   • Your abstract is quite long and heavily focused on the background and rationale. You might want to shorten it, include more information about methodology and leave space to add information about results and conclusions at Stage 2. 
   • One of the reviewers suggests moving a part of your introduction into a future discussion section. Perhaps it’s a good idea to consider this suggestion a bit more broadly for your introduction. The introduction is very rich in information, and it is well structured, but maybe some of the detail could be saved for the discussion. I leave this decision up to you, but just want to note that just the theoretical background part is currently more than 5 000 words. I expect that your results and discussion sections will also be very extensive since you have many hypotheses and planned analyses. At this point, it’s good to consider how this will all look together in the end and make edits in accordance with what you would like the final manuscript to look like. 
   • I also want to highlight one of the reviewers’ comments, that the proof-of-concept section of the manuscript would be appropriate as a supplementary file. I agree with this, and I wanted to suggest that you could upload it to your OSF repository as a separate file. This will aid the clarity of the manuscript once you report the study results.
     
     
2. Tables with research questions and hypotheses.
   • Since the research questions are not stated in the text, it would help to have parentheses with the aim number next to each research question so it’s easier to relate these back to the aims. E.g., Do sPCIst and sLZc discriminate between brain responses to coarsegrained differences in visual stimuli? (Aim 1). 
     
     
3. Methods:
   • In the sampling section and inclusion/exclusion criteria, please state whether you have any age limits for participants.
   • Add space in the manuscript for a “participants” section, which will be added after you have collected data.
   • Please also add a section  with a clear explanation of what the procedure will look like, including ethical considerations and that two experimental sessions will be scheduled with each participant. This is currently unclear.
   • What demographic information will be recorded for each participant?
     
     
4. The sample size calculation
   • You write “we conducted power analyses using custom scripts”. Please provide details on software & version. Are you planning to share these scripts and the data? If not, please justify.
   • You provided a sample size calculation based on a previous study with three other experimental tasks and stimuli that are relevant to the aims of the proposed study. Your final sample size is an average taken from these calculations supplemented with a small note in footnote 24. According to your calculations, the study would be underpowered to detect 3 of the simulated effects. Instead of a footnote, it would be good to have a clear statement in the manuscript describing the outcome of the sample size calculation and a justification of why the sample size of 43 is adequate for your proposed study given that when looking at Table 5, it would be underpowered to detect 3 of the simulated effects and given the complexity of your study design with multiple analysis models.
     
     
5. Stimuli:
   • You write that you are adapting stimuli classes from Mensen et al., 2017. Are you using the exact same stimuli as Mensen et al. (2017) or are you preparing new stimuli? Please state how the stimuli can be accessed.
   • You state that custom scripts will be used to generate noise stimuli. Please state specific software and algorithms.
   • What software will be used for stimuli presentation?
   • Add information about the display of the visual stimuli – monitor size, refresh rate, size of the stimuli. Add information about the type and model of speakers used for the auditory stimuli.
   • Participants will make responses to rate the stimuli – what is the mode of responses? Using a keyboard, a mouse? Please specify.
   • How are correct and incorrect responses operationalised?
     
     
6. EEG pre-processing:
   • What method of baseline correction will be used?
     
     
7. Analysis plans
   • Your models correspond to research questions and hypotheses in Tables 1 and 2. What about the models for Table 3?
   • The use of the term ‘Meaningfulness’ in the statistical models is confusing because it’s the same as one of your subjective variables. You have a brief explanation of ‘meaningfulness’ in footnote 29, but this needs to be clearly explained in text, so that the reader can understand it’s not about the subjective rating. You had this described more clearly in the pilot analysis section – i.e. “Meaningfulness (with scrambled images coded as -0.5 and natural images coded as 0.5)” and “ObjectCategory (with cars coded as -0.5 and faces coded as 0.5)”. Please do the same for the main analyses and explain clearly for both the visual and the auditory task conditions.
   • In addition to the above, you have three levels of noise here, so I assume that the Meaningfulness coding -0.5 and 0.5 will not apply.
     
     
8. Minor corrections
   • P.23 and p.26  “A summary of all five dimensions, their operationalizations, examples, and rationales is given in Table 5”. This should be Table 6
   • Every time you mention the use of software and packages – Matlab, EEGLAB, Python, R etc. add the version of the software used or leave a blank space to complete at Stage 2

In your next version, please clearly mark all changes to the submitted manuscript. You may also choose to additionally upload a clean version of the manuscript to the OSF. 

I am looking forward to reading your revised manuscript.

Best wishes,

Marta Topor