Recommendation
DOI or URL of the report: https://osf.io/8ctyu/
Version of the report: 1
Revised manuscript: https://osf.io/7ym9n
All revised materials uploaded to: https://osf.io/rwdn6/ , updated manuscript under sub-directory "PCIRR Stage 1\PCI-RR submission following R&R"
, posted 08 Mar 2024, validated 08 Mar 2024Dear authors,
Thank you very much for your submission. I have read your paper with great interest and received feedback from two reviewers. Given this feedback and my own reading of the paper, I recommend a minor revision to address the relatively minor concerns that Thibaut raises and that I also noted while reading your manuscript. Angela finds your manuscript ready for IPA.
Thibaut suggests to improve the presentation of the hypotheses (something that I also noted). He also suggests making a clearer distinction between the confirmatory and exploratory analyses, i.e., what you clearly replicate and what you explore. While you might have already stated this in some parts of the manuscript, it might be appropriate to increase its salience. Thibaut also suggests MHT correction if the extensions are part of your confirmatory analyses. More generally, even if you disagree, I think that it calls for making more explicit your decision rule (i.e., the significance threshold that you shall use for your registered hypotheses). Last, the referee is not convinced by the use of simulated data. I see it as a way to test your codes and understand why you included them. Please do not feel compelled to change the manuscript on this point.
I attach my own remarks. (As always, consider them with caution: I might not have understood everything well.)
I am looking forward to receiving the revised version of your (excellent) work.
Best regards,
Romain
Download recommender's annotationsThis report is very carefully designed.
1A. The scientific validity of the research question(s) - OK
1B. The logic, rationale, and plausibility of the proposed hypotheses (where a submission proposes hypotheses) - OK
1C. The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis or alternative sampling plans where applicable) - OK
1D. Whether the clarity and degree of methodological detail is sufficient to closely replicate the proposed study procedures and analysis pipeline and to prevent undisclosed flexibility in the procedures and analyses - OK
1E. Whether the authors have considered sufficient outcome-neutral conditions (e.g. absence of floor or ceiling effects; positive controls; other quality checks) for ensuring that the obtained results are able to test the stated hypotheses or answer the stated research question(s). - OK
I only have one remark. The authors specifically ask feedback on one improvement:
"We note that when reconstructing the materials we noticed that the conditions in Study 6,
at least as described in the target article, were not entirely equivalent, and seemed to conflate
expectations and outcome. For example, expectations for reputation seemed to conflate whether
the donation was private or public. We categorize this as a possible weakness in the experimental
design and decided to deviate and make an adjustment to the target’s stimuli to focus solely on
manipulation of expectations.
[Note to reviewers: We would appreciate feedback on our assessment of this issue and this
adjustment, and are open to changing it given a well-justified argument and/or clear editorial
guidelines.]"
However, there is not clear information about the comparision of the stimuli side by side, there is only a description and interpretation, so it's difficult to give any feedback.