Recommendation
DOI or URL of the report: https://osf.io/preprints/psyarxiv/sfyg6
Version of the report: 1
, posted 12 Mar 2025, validated 12 Mar 2025Dear Elena Karakashevska,
Apologies for the late response, I was waiting on one extra reviewer but given the positive response from both of our reviewers, I decided to move ahead with my response.
I have received reviews from one of our original reviewers and one new reviewer for your Stage 2 submission. Each reviewer carefully examined your submission against the Stage 1 IPA and felt the authors adhered to the original study design from the Stage 1 well.
Felix Klotzsche noted the exploratory analyses were extensive but had some confusion over the correlations between SPN amplitudes, perspective cost and behavioral performance. I agree that the interpretation of part E of Figure 16 is confusing and recommend some clarifications. Felix Klotzsche also suggested that the tenses be reexamined, and I support that the authors can edit to make the manuscript past tense throughout.
Tadamasa Sawada also commended the Stage 2 and had some minor comments regarding the discussion.
From my perspective as a recommender, I have a few suggestions regarding the formatting of the Stage 2 in line with the review criteria of a Stage 2 manuscript.
Please submit a point-by-point reply to our reviewers and my comments, revising your manuscript accordingly.
Best,
Grace
2A. Whether the data are able to test the authors’ proposed hypotheses (or answer the proposed research question) by passing the approved outcome-neutral criteria, such as absence of floor and ceiling effects or success of positive controls or other quality checks. All quality checks seem to have been upheld in the Stage 2 as outlined in the accepted Stage 1. In a couple of case the authors seem to have gone beyond the outlined quality checks, and I believe that some of these analyses should be moved to supplemental material as they were not officially registered analyses of the Stage 1. These include the analysis of the accuracy responses. In the Stage 1 you define an accuracy threshold for your participants to be included in the study, however you then go on to analyze differences in behavioral performance between conditions, which was not registered. Likewise, the analysis of the eyetracking data is also very thorough, but outside the scope of the planned analyses of the Stage 1. The Stage 1 indicated that the participants should maintain fixation bounded by 2.5 degrees, however in the Stage two authors report fixation consistency across multiple different fixation boundaries. These analyses are nicely confirmatory but should be in the supplemental.
2B. Whether the introduction, rationale and stated hypotheses (where applicable) are the same as the approved Stage 1 submission. All changes in the introduction, rationale and stated hypotheses maintain the original intention of the text. Some results are imbedded in the methods of the Stage 2, which is appropriate for readability. These results were quality checks of the data, planned in the Stage 1, however, there are a few extra quality checks which I requested to be moved to the supplemental (see 2A).
2C. Whether the authors adhered precisely to the registered study procedures. This criterion assesses compliance with protocol. From my reading of the Stage 2, the authors have complied with their planned design and analysis.
2D. Where applicable, whether any unregistered exploratory analyses are justified, methodologically sound, and informative. In my opinion, the exploratory analyses in the results section seem justified and further unpack some of the unexpected results.
2E. Whether the authors’ conclusions are justified given the evidence. The authors’ conclusions are well justified given the results of the planned study. However Dr. Sawada has some further suggestions regarding the discussion which should be considered.
Minor comments:
In Figure 2 the data in panel B) are labelled blue for frontoparietal and red for perspective, whereas in panel C) the data are labelled red for frontoparietal and blue for perspective. I understand that the authors may want the images to remain consistent from the original study, but they may want to note the color change from panel B) to C) in the legend for clarity.
Extra word in signal quality check section of methods: “These adjustments were only planned made if…”
Word missing from sentence in discussion: “This task effect consistent with many previous studies (Makin et al., 2022; Makin et al., 2024).”
Karakashevska et al. present results of a study in which they applied an immersive virtual reality (VR) setup to investigate view-independent representations of symmetry and their effect on an established EEG-marker of symmetry processing (the Sustained Posterior Negativity, SPN). The study was extensively and rigorously pre-registered and pre-reviewed. The results largely support the pre-registered hypotheses: centrally, the authors found evidence that (at least in a condition where participants discriminated symmetric from asymmetric stimuli) the SPN amplitude was not different when participants viewed the stimuli in the frontoparallel plane or from an angle (leading to retinotopic asymmetries of originally symmetric stimuli). To support the claim of a non-existing difference, the authors applied an equivalence testing strategy. For a second condition, in which participants focused on the luminance of the stimuli and symmetry processing was therefore incidental (aka. task-irrelevant), the authors found less conclusive results: the observed effect of the SPN was not significantly smaller than the pre-defined threshold for a small/negligible effect. However, as the authors discuss, this is likely to be a consequence of the optional stopping approach which was based on the observations in the first condition. The distribution of the data gets close to the authors’ prediction.
The authors also present convincing evidence supporting two additional analyses which comprise quality checks: there was a significant SPN in all conditions (supporting the claim that the SPN can be measured in a novel VR setup) and it was responsive to the task manipulation (symmetry vs luminance distinction) in a way which was expected based on previous studies (supporting the claim that the SPN was sensitive to experimental manipulations also in this study).
The authors stuck closely to the pre-registered analyses and hypotheses. Furthermore, I did not spot any substantial deviations of the introduction or rationale in the stage 2 manuscript from the stage 1 version.
Exploratory analyses were performed in a way which supports the main analyses. I particularly appreciated the extensive additional checks investigating differences between the experimental conditions in terms of eye movements (fixation breaks) and potential influences of the VR setup on the EEG signal. The correlations between SPNs, condition differences, and behavioral results (Fig. 16) presented themselves as less conclusive to me and I could not really make sense of the final sentence of the respective section in the discussion (“significant correlations between behavioural performance and ERP signals [p > .042, Figure 16E]”). This being said, the authors do not base relevant claims on these exploratory analyses.
Overall, the conclusions drawn by the authors seem to be well justified by the data and the supporting analyses.
After having reviewed the stage 1 version of this registered report, I enjoyed reading the results of the study and to get convinced that presenting stimuli in a more naturalistic way (adding stereoscopic depth information) may annihilate an alleged effect of “perspective cost”.
I want to thank the authors for the interesting read and wish them much success for the further publication journey of this very rigorous study and its results!
Beyond the scope of the review of the RR, here is one more observation/recommendation: The stage 2 article changes between future and past tense regularly which seems to be a consequence of the pre-registration (future tense) and the presentation of results and methods (past tense). For the reader this is difficult to digest, and I would recommend unification.
Best regards
Felix Klotzsche
This is the Stage-2 review of the registered report, entitled "They look virtually the same: extraretinal representation of symmetry in virtual reality", by Elena Karakashevska, Michael Batterley, and Alexis D.J. Makin. The authors tested the effect of the 3D orientation of a planar symmetrical figure on an ERP signal. The effect of the 3D orientation was observed and the effect was almost independent from the degree of visual information indicating the 3D orientation.
By comparing between the Stage-1 and Stage-2 versions of this manuscript, I confirmed that the authors collected data, analyzed the data, and reported the results by following the registered procedure.
The authors used a virtual reality head-mounted display as the apparatus used in the experiment reported in this study. Note that it is technically challenging to integrate the VR head-mounted display with a neurophysiological device. Note that this apparatus is the major difference of the current study from an earlier study by Karakashevska et al. (2025, Cortex), in which the effect of the 3D orientation was tested by using an ordinal computer display. This difference is primarily technical but I also consider it to be interesting.
I only have a few very-minor issues.
Regarding the mental representation of a retinal image itself, Moralex, Bax, and Firestone (2020, PNAS) also discussed that the representation of the retinal image remains in the visual system.
P. 27 (Discussion). > Given the current results, we considered whether the brain ever constructs extraretinal representations.
The authors discuss whether the brain constructs extraretinal representations in a binary way (a representation or no representation). This problem can be quantitative. Also, the retinal representations and the extraretinal representations are not exclusive to one another. The visual system can use both of them.
P. 27 (Discussion). Invariants discussed in Gibson (1979) and in Sawada and Pizlo (2008) are mentioned in the Discussion section. Sawada and Pizlo (2008) used the word “invariant” to mean a model-based invariant while Gibson (1979)’s usage of the word “invariant” is not very clear. This issue is closely discussed in the appendix of Sawada and Farshchi (2022, Visual Cognition).
