DOI or URL of the report: https://doi.org/10.31219/osf.io/xmwgy
Version of the report: 1
Please find (i) my response to the recommender and reviewer comments and (ii) the manuscript file with all changes tracked separately attached below. Thank you!
All files (including a clean version of the manuscript with all changes confirmed) are available as one document via https://osf.io/3re4n. The final file has also been uploaded and will be available via https://doi.org/10.31219/osf.io/xmwgy (Version 2) upon approval by OSF moderators.
Sorry for the delay in getting back. We do have one review; and based on my own reading this is sufficient as you have followed the guidelines for a Stage 2 very well. The reviewer asks you optionally to consider wording in a few places.
best
Zoltan
I enjoyed reading this completed Stage 2 submission, having served as a (non-specialist) reviewer at Stage 1. In my view, the report already comes very close to meeting the Stage 2 criteria -- the authors adhered faithfully to their preregistered protocol, and as best as I can tell have documented all deviations very thoroughly and transparently. The reporting of the results is clear, with appropriate robustness tests where required, and the conclusions are justified by the evidence.
The judgment of findings and their importance does not form part of Stage 2 evaluation, so the following is pure commentary, but I do want to note for the record that I found the lack of compliance - and the apparent toothlessness of the relevant regulators - quite dispiriting, even if unsurprising. It is hard to see these findings and draw any conclusion other than that the regulation of loot boxes in the UK has failed. I hope this work has an impact in stimulating the necessary reforms.
I have one stylistic suggestion that I believe would further increase the impact of this work - although the authors may disagree and it will be the recommender's decision as to whether my comment falls within the purview of Stage 2 evaluation. At various places in the Stage 2 manuscript, I recommend replacing emotive/superlative language with more dispassionate phrasing, and keeping interpretations strictly in line with the evidence. Some examples (and there may be others) include "incredibly low", "compliance is abysmal”, "feel betrayed”, "destroyed their own reputation”, etc. These opinions are understandable, but the eventual Stage 2 RR will be harder for regulators and politicians to ignore if it meets the highest standards of discipline and sticks to the facts and their evidence-based interpretration - whereas emotive language may let them off the hook by pointing to the work as advocacy or a personal attack rather than what it is: rigorous preregistered scientific research. The more emotive language would be better suited, in my view, to an op-ed or other news-style article that the authors may want to write at a later date once the RR is recommended.
Minor points
Lines 1098-1105 – this is a very long sentence that I found difficult to parse; I suggest rephrasing or breaking into two sentences for clarity.