PCI Registered Reports

DUNLEAVY Daniel

Center For Translational Behavioral Science (College of Medicine), Florida State University, Tallahassee, United States of America
Medical Sciences, Social sciences
recommender

Recommendations: 0

Reviews: 2

Website https://scholar.google.com/citations?user=OWo5Q30AAAAJ&hl=en

Areas of expertise

I am a postdoctoral research fellow (PhD in Social Work, 2020). My primary research areas are in behavioral health (mental health, substance use), meta-research, and social welfare policy.

Reviews: 2

06 Jul 2022

STAGE 2
(Go to stage 1)

Ontological Diversity in Gaming Disorder Measurement: A Nationally Representative Registered Report

Veli-Matti Karhulahti, Jukka Vahlo, Marcel Martončik, Matti Munukka, Raine Koskimaa, Mikaela von Bonsdorff https://doi.org/10.31234/osf.io/qytrs

Different ontologies, different constructs? Instruments for gaming-related health problems identify different groups of people and measure different problems

Recommended by Charlotte Pennington based on reviews by Daniel Dunleavy and David Ellis

Screening instruments that aim to provide diagnostic classifications of gaming-related health problems derive from different ontologies and it is not known whether they identify equivalent prevalence rates of ‘gaming disorder’ or even the same individuals. Underpinned by this, Karhulahti et al. (2022) assessed how screening instruments that derive from different ontologies differ in identifying associated problem groups. A nationally representative sample of 8217 Finnish participants completed four screening measures to assess the degree of overlap between identified prevalence (how many?), who they identify (what characteristics?) and the health of their identified groups (how healthy?).

The results indicate that measures based on the ICD-11, DSM-5, DSM-IV, and self-assessment appear to be associated with lower mental health. However, these measures of gaming-related health problems differed significantly in terms of prevalence and/or overlap, suggesting that they identify different groups of people and that different problems or constructs are being measured by different instruments. These findings are important because they contribute to the rapidly growing literature on the ‘fuzziness’ of constructs and measures relating to technology use. The authors recommend that researchers working with these measures should: (a) define their construct of interest; and (b) evaluate the construct validity of their instruments. Being able to answer these questions will enhance research quality and contribute to strengthened meta-analyses. Importantly, this will prevent hype around gaming-related disorders, allowing researchers to communicate clearly and appropriately without risk of confusing related yet different constructs.

The Stage 2 manuscript was evaluated by two of the reviewers who assessed it at Stage 1. Following revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation. To ensure that the manuscript met the requirements of the PCI RR TOP guidelines, prior to this acceptance an email communication was sent to the authors by the recommender to ensure that study data were openly available on a temporary OSF link before the final data archive is full validated by the Finnish Social Sciences Data Archive (FSD). This is noted in the recommended preprint.

URL to the preregistered Stage 1 protocol: https://osf.io/usj5b

Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question existed prior to Stage 1 in-principle acceptance.

List of eligible PCI RR-friendly journals:

References

1. Karhulahti V.-M., Vahlo J., Martončik M., Munukka M., Koskimaa R. and Bonsdorff M. (2022). Ontological Diversity in Gaming Disorder Measurement: A Nationally Representative Registered Report. Peer-reviewed and recommended at Stage 2 by Peer Community in Registered Reports https://psyarxiv.com/qytrs

17 Jan 2022

STAGE 1

Identifying Gaming Disorders by Ontology: A Nationally Representative Registered Report

Veli-Matti Karhulahti, Jukka Vahlo, Marcel Martončik, Matti Munukka, Raine Koskimaa, Mikaela von Bonsdorff https://osf.io/mpz9q/

Do different screening instruments for ‘gaming disorder’ measure the same or different construct(s)?

Recommended by Charlotte Pennington based on reviews by Daniel Dunleavy, Linda Kaye, David Ellis and 1 anonymous reviewer

There is considerable debate regarding the relationship between excessive gaming and mental health problems. Whilst the diagnostic classification of “gaming disorder” has now been included in the WHO’s International Classification of Diseases (ICD-11), the APA decided not to include this diagnosis in their Diagnostic and Statistical Manual of Mental Disorders (DSM-5) because the literature “suffers from a lack of a standard definition from which to derive prevalence data” (APA 2013, p. 796). Furthermore, screening instruments that aim to provide diagnostic classifications derive from different ontologies and it is not known whether they identify equivalent prevalence rates of ‘gaming disorder’ or even the same individuals.

In this Stage 1 Registered Report, Karhulahti et al. (2022) aim to assess how screening instruments that derive from different ontologies differ in identifying associated problem groups. A nationally representative sample of 8000 Finnish individuals will complete four screening measures to assess the degree of overlap between identified prevalence (how many?), who they identify (what characteristics?) and the health of their identified groups (how healthy?). If these four ontologically diverse instruments operate similarly, this will support the notion of a single “gaming disorder” construct. If, however, the instruments operate differently, this will suggest that efforts should be directed toward assessing the clinical (ir)relevance of multiple constructs. This rigorous study will therefore have important implications for the conceptualisation and measurement of “gaming disorder”, contributing to the debate around the mixed findings of gaming-related health problems.

Four expert reviewers with field expertise assessed the Stage 1 manuscript over three rounds of in-depth review. Based on detailed and informed responses to the reviewers' comments, the recommender decided that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).

URL to the preregistered Stage 1 protocol: https://osf.io/usj5b

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.

List of eligible PCI RR-friendly journals:

References

APA (American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders (5th Edition). APA.
Karhulahti V-M, Vahlo J, Martončik M, Munukka M, Koskimaa R and Bonsdorff M (2022). Identifying Gaming Disorders by Ontology: A Nationally Representative Registered Report. OSF mpz9q, Stage 1 preregistration, in principle acceptance of version 4 by Peer Community in Registered Reports. https://osf.io/mpz9q/

DUNLEAVY Daniel

Center For Translational Behavioral Science (College of Medicine), Florida State University, Tallahassee, United States of America
Medical Sciences, Social sciences
recommender

Recommendations: 0

Reviews: 2

Website https://scholar.google.com/citations?user=OWo5Q30AAAAJ&hl=en

Areas of expertise

I am a postdoctoral research fellow (PhD in Social Work, 2020). My primary research areas are in behavioral health (mental health, substance use), meta-research, and social welfare policy.