Do different screening instruments for ‘gaming disorder’ measure the same or different construct(s)?

I thank the authors for their submission and the recommender for the opportunity to review this interesting submission. I hope the following comments, suggestions, and questions help strengthen and clarify components of this submission:

General Comments and Summary:

1. The authors are proposing to study how different screening instruments (for gaming-related health problems) differ and perform in identifying risk groups. The authors astutley note that gaming-related health problems have been cashed out in a variety of different ways, each with unique ontological commitments. Each of the four measures discussed here, corresponds to a different one of these commitments.

Background and Justification:

1. The overarching research question and three sub-questions are clear and reasonable. The authors rightly note that the instruments may yield differing prevalence rates (RQ-A), but also those who are identified as such (RQ-B). Interestingly, the authors are proposing (RQ-C) to explore how different screening intsruments differ with respect to the health statuses of those identified as being at risk.

Design/Methodology:

1. I found each of the four sub-hypotheses for H1 to be clear (p. 4). I was initially hesitant, finding them to be poorly defined (i.e., what "remarkably lower" meant). However, I found the description of the interval-based method on p. 5 to be sufficiently clear to satisfy my concerns. I feel that the method used here is described in enough detail to be replicated by another set of researchers and further that it is an adequate method for assessing the hypotheses. The other two hypotheses (H2 and H3) are clear and testable - particularly H3, which clearly specifies the smallest effect size of interest (and justifies its selection).

2. Recruitment - p. 5 - The authors are recruiting using a company called Bilendi. I'd just like to see a little more detail about the recruitment timeline (expected start-finish dates) and how participants will 1) be incentivized, and 2) complete the survey (i.e., what software or platform will they use to complete the survey?).

3. Sampling plan - pp. 7-8 - The authors provide a reasonable sample size justification and the required sample size to estimate prevalence rates. 

Other Comments:

I don't have any other concerns at this time. I thank the authors for their clearly written Stage 1 submission and the recommender for their consideration of the above review.

This proposed study presents ba timely and important investigation into the ontological differences between screening instruments for gaming disorder, and the effects on prevalence. The following sections outline how the study adheres to the key questions by PCI RR:

1. Does the research question make sense in light of the theory or applications? Is it clearly defined? Where the proposal includes hypotheses, are the hypotheses capable of answering the research question?

The objectives and hypotheses outlined within are clearly defined. 

2. Is the protocol sufficiently detailed to enable replication by an expert in the field, and to close off sources of undisclosed procedural or analytic flexibility?

The methods section largely contains enough detail that replication would be possible. Sources of ambiguity may include how and from where participants will be recruited via the 3rd party (online advertisments), and how the survey data will actually be collected; i.e., whether in-person or online. These considerations may impact overall representativeness.

3. Is there an exact mapping between the theory, hypotheses, sampling plan (e.g. power analysis, where applicable), preregistered statistical tests, and possible interpretations given different outcomes?

Where possible, the hypotheses and proposed testing frameworks are adequately covered. The requisite sample size has been calculated appropriately. pre-registered tests and interpretations have been sufficiently provided.

4. For proposals that test hypotheses, have the authors explained precisely which outcomes will confirm or disconfirm their predictions?

In most cases, explanations are adequate. There may be quesitons about what constitutes exploratory investigation in some tests, but the authors have justified the need for exploratory analyses where necessary. 

5. Is the sample size sufficient to provide informative results?

Sample size appears to have been calculated appropriately.

6. Where the proposal involves statistical hypothesis testing, does the sampling plan for each hypothesis propose a realistic and well justified estimate of the effect size?

The effect sizes proposed are realistic and justified.

7. Have the authors avoided the common pitfall of relying on conventional null hypothesis significance testing to conclude evidence of absence from null results? Where the authors intend to interpret a negative result as evidence that an effect is absent, have authors proposed an inferential method that is capable of drawing such a conclusion, such as Bayesian hypothesis testing or frequentist equivalence testing?

All hypotheses have been clearly defined and tests of significance are adequate.

8. Have the authors minimised all discussion of post hoc exploratory analyses, apart from those that must be explained to justify specific design features? Maintaining this clear distinction at Stage 1 can prevent exploratory analyses at Stage 2 being inadvertently presented as pre-planned.
Have the authors clearly distinguished work that has already been done (e.g. preliminary studies and data analyses) from work yet to be done?

All post hoc explanatory analyses have been minimised where possible. The results from the pilot study have been sufficiently distinguished.

9. Have the authors prespecified positive controls, manipulation checks or other data quality checks? If not, have they justified why such tests are either infeasible or unnecessary? Is the design sufficiently well controlled in all other respects?

In some cases, for example the case of mischievious reporting has been accounted for. 

10. When proposing positive controls or other data quality checks that rely on inferential testing, have the authors included a statistical sampling plan that is sufficient in terms of statistical power or evidential strength?

All proposed statistical analyses are adequate.

11. Does the proposed research fall within established ethical norms for its field? Regardless of whether the study has received ethical approval, have the authors adequately considered any ethical risks of the research

All ethical considerations have been met. 

One future suggestion, that may be worth bearing in mind, is the digital convergence between gaming and gambling, with microtransactions and other predatory features that resemble gambling, including lootboxes. It strikes me that, given the overlap between gambling and gaming disorders in the DSM-IV and DSM-5 criteria, there might be the potential for confounding among participants. Controlling for this possibility might be something to think about.

Identifying Gaming Disorders by Ontology: 
A Nationally Representative Registered Report
 
This is a really interesting study that has been carefully considered. Beyond quantifying ontological overlap, I especially appreciate how the authors plan to compare mental and physical health differences between classificaitons from different measures. My comments are therefore relatively minor and perhaps reflect my own lack of understanding rather than any major issue. 
 
1. It would be useful if the Abstract and Introduction made it clear what ‘risk groups’ might look like. 
I assumed these were groups whereby their scores suggest some form of diagnosis/classification?  However, as I read on I realised this is also about other aspects of physical and mental health. Is a ‘risk group' both of these? A table summarising all measures would be helpful within the method section to provide further clarity in this regard.  
 
2. The research questions could also do with some clarification or possible re-ordering. 
‘Who' really refers to overlap (or lack of ontological overlap) in the sample as I understand it. It feels like it should be the first research question. The second question could then consider prevalence as this is making more population-based inferences (I think). I am not familiar with the interval-based method outlined so can’t comment further. Wondered a bit as to how this goes beyond a series of chi-squared tests that could also consider ontological similarities. Research question C is clearly about Health and that is clear. 
 
3. I suspect the results, regardless of direction, will have implications for many notions of technological ‘addiction' or related ‘disorders’. 
If ontologies are different, then this suggests a problem for comparisons across different studies. If they are all the same, then this doesn’t only indicate that scholars should direct efforts toward assessing the clinical relevance of multiple constructs as suggested. On the contrary, it begs the question as to why researchers have continued to re-invent the wheel and made little progress regarding a consensus. Creating a new disorder or ontology doesn't create new resources for pracititoners or clinics. What are the societal costs of stigmatising the most popular form of play in children and adolescents? I guess what I am getting is that results from this work might suggest the abandonment of such constructs altogether rather than any attempts at improved conceptualisation.