DOI or URL of the report: https://osf.io/z95u6
Version of the report: 5
Revised manuscript: https://osf.io/x96kn
All revised materials uploaded to: https://osf.io/2sb7x/ , updated manuscript under sub-directory "PCIRR Stage 2\PCI-RR Stage 2 submission following R&R"
I have now obtained an evaluation from the one of the reviewers who assessed your Stage 1 submission, and I have decided that we can proceed based on this review and my own assessment. As expected from my own reading of the paper, the review is generally positive and there are few obstacles in the way to final Stage 2 acceptance. Within the comments you will find some interesting suggestions for clarifying the presentation of results and enhancing the discussion. I look forward to receiving your revision and response, which I will assess at desk before issuing a final recommendation.
The authors did what they said they would do, so this work should be approved.
What I say below are merely suggestions to improve the reporting of the results.
I found myself making multiple notes throughout the paper, which were then answered shortly afterwards (happily!).
My biggest concern is the low anthropomorphism scores in this replication. As far as I can tell, however, the authors do not discuss if it is low compared to the original.
What would be best is to present something like a 2x2 grid of density plots of this data (Gadget anthropomorphism, pet anthropomorphism, belief in supernatural, supernatural anthropomorphism) with the mean and 95%CI of the mean indicated, as well as a line of the mean of the original Epley data (where available). That may help elucidate if and how much the scores differ from then and now (I assume the original Epley data is not available).
A minor point is I would like to see more discussion of the results, what do we know now that we did not know before this rstudy was conducted? What insights can be gleaned for future use?
The final (and related) point is on the original material used. The authors used the original gadgets. But the results did not replicate.
A hostile individual could say "well, we all know you can't use the original materials as times change".
BUT
If the authors had used updated materials and still found non-significant results, the hostile individual could similarly say "Well, they changed the materials so it is not a replication".
I know the authors know about this problem.
It is a trap, laid by researchers desperate to vilify any nonsignificant replication.
I would love to see the authos say this, explicitly, and loudly, in their discussion.
There needs to be continued conversation about this 'updating materials trap', and this is a good place to continue to point it out.
Protzko