DOI or URL of the report: https://osf.io/hv4em
Version of the report: v1
Apologies, Zoltan. I have now edited Table 2 accordingly.
Best,
David
Thank you for you revision which has addressed most points. Just one thing. The analysis is set up as a 2X2 but the descriptives e.g. Table 2 present the groups one dimensionally with labels that do not match the IV names explicitly. For ease of relating the analyses to the descriptives, can you set up table 2 as a 2X2 table so it is easy to relate the IVs in the analysis to the descriptives.
DOI or URL of the report: https://osf.io/hv4em
First of all many apologies for the extraordinary delay on getting back to you on this mansucript (and your other one - you will hear about that soon). That was my fault for not keeping on top of it. I do have one review back from one of the original reviewers, who makes extensive comments but is very positive about the manuscript.
In dealing with the reviewer's comments, note that the Introduction, Method - and also the Design Table - should remain the same as Stage 1, except for changes in tense, or anything factually incorrect. You could add footnotes for further clarifications, noting these are Stage 2 additions. In the same way, the material in the Stage 1 "Analysis plan" onwards until the Results should be kept in. This section is vital for indicating the extent to which your non-singificant results count (or rather do not count) against predicted results.
Some further points:
Results: Present descriptives clearly in terms of each cell of low vs high risk by low vs high benefit.
2nd page of Results:
"However, contrary to predictions, the interaction term for risks and benefits was not significant, and the coefficient for risk was significant and negative."
A prediction is about the population states of affairs; here a non-significant interaction does not count against the prediction of the population state of affairs. Also this sentence implies a prediction was about risk; but the test for Risk does not feature in the Design Table. Include in the main results section only those tests you have indicated in the Design Table. The rest can go in a non-pre-registered section.
Similarly for discussion: You should be clear you have no grounds for either saying there was or was not an interaction, so the prediction is neither confirmed nor disconfirmed. So you have not failed to replicate the previous result of an interaction, in the sense you have provided no evidence that there was no interaction.
Overall, this manuscript makes an interesting and useful contribution to the literature - both in terms of advancing a novel methodology and exploring a challenging real world problem. I applaud the authors for their initiative in this area - it's a difficult issue to address empirically and certainly not a straightforward area of work. I also appreciate the transparent approach, with a focus on replicability, taken in the work.
The findings open a number of new questions for future research - and these are well-noted in the Discussion. I look forward to seeing how this line of research develops.
All my remaining comments and suggestions are noted in the attached manuscript document (as either comments or edits). The vast majority concern increasing the clarity or precision of the writing. While it's up to the authors whether they take these on board or not, my main purpose in taking the time to provide this kind of feedback is to maximise the potential of this work being accessed by a wider (and meaningful) audience. I hope the authors will view this in the spirit intended.
Thank you for allowing me to interact with this interesting project.
Download the review