Recommendation

How can the experiences of those who engage in video games in healthy and unhealthy ways be systematically organised?

ORCID_LOGO based on reviews by Michelle Carras, Lukas J. Gunschera and Christopher Ferguson
A recommendation of:
picture

Reconstructing Gaming Disorder: A Taxonomy by Registered Report

Abstract

EN
AR
ES
FR
HI
JA
PT
RU
ZH-CN
Submission: posted 10 October 2022
Recommendation: posted 15 April 2023, validated 15 April 2023
Cite this recommendation as:
Dienes, Z. (2023) How can the experiences of those who engage in video games in healthy and unhealthy ways be systematically organised?. Peer Community in Registered Reports, . https://rr.peercommunityin.org/PCIRegisteredReports/articles/rec?id=322

Recommendation

People are often drawn into intensive video game use in ways they or others may find troubling, harmless or even praiseworthy. Understanding these different experiences may help with integrating intensive technology use into everyday life in a healthy way.
 
In this programmatic submission, Karhulahti et al. (2023) will explore the gaming experiences of three groups of people (those who have sought treatment for gaming, esport players, and adolescents who play around two hours every day), using phenomenological and clinical interviews, and gaming diary logs every four months over three years. Around 200-300 participants will be recruited initially from Finland, Slovakia, and South Korea. In order to further increase cross-cultural range, the study will apply a new duplication method to collect similar data also in countries that have been studied little in the past. The aim will be to answer the questions of a) Is it possible to distinguish passionate from pathological gaming by the meanings and values that players attach to videogame play? and b) What are the design structures of videogames, which are played intensively and/or with gaming-related health problems? Ultimately, the study aims to synthesise all its data into a new taxonomic system, which can help better understand the differences and idiosyncrasies of gaming in lives across cultures.
 
This Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on the comprehensive responses to the reviewers' feedback, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/ekm8x
 
Level of bias control achieved: Level 4. At least some of the data/evidence that will be used to answer the research question already exists AND is accessible in principle to the authors (e.g. residing in a public database or with a colleague) BUT the authors certify that they have not yet accessed any part of that data/evidence
 
List of eligible PCI RR-friendly journals:
 
 
References
Karhulahti, V.-M., Martončik. M., Siutila, M., Park, S., Jin, J., Adamkovič, M., Auranen, T., Na, B., & Yoon, T.-J. (2023). Reconstructing Gaming Disorder: A Taxonomy by Registered Report​, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/ekm8x
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Evaluation round #2

DOI or URL of the report: https://osf.io/9nm6x

Version of the report: v2

Author's Reply, 12 Apr 2023

Decision by ORCID_LOGO, posted 03 Apr 2023, validated 03 Apr 2023

The reviewers are largely happy with the paper, with some minor changes for clarification. Concerning the smallest effect size of interest, I prefer to work in raw regression slopes, as they reflect units one should be interested in unaffected by criterion reliability (https://online.ucpress.edu/collabra/article/7/1/28202/118660/Obtaining-Evidence-for-No-Effect), but you may wish to argue,as you have done, that you will not be claiming an effect of interest exists vs does not, in which case, defining what is of interest is moot. Nonetheless, in addressing Ferguson's point, you may wish to alert readers about these issues in forming their own conclusions about what meaningfully exists, given that readers may well jump to conlusions you are are wise enough not to!

Reviewed by ORCID_LOGO, 30 Mar 2023

Review: Reconstructing Gaming Disorder: A Taxonomy by Registered Report (STAGE2)

I enjoyed reviewing this registered report and think the authors did well at implementing the suggested changes. I have added some minor comments below:

 

1)      Participants from groups A-C will provide frequent diary-like entries over multiple years. The mere act of monitoring one’s gaming activity may affect participants and their appraisal of their gaming. This is something that the data will be able to address, and it would be interesting to analyse and/or mention in the manuscript, given ‘thoughtless’ gaming may have distinct effects.

2)      The authors mention that “following our critical Bayesian epistemology, each discovered taxon will be established with a confidence level (0-1), and when evidence accumulates, confidence can decrease or decrease” (p. 14). It does not become clear whether these confidence intervals are based on data or on the subjective judgments of the authors. Arguably, the latter is insufficient to be termed “Bayesian”. In this case, the use of “Bayesian epistemology” is misleading, as it is traditionally used to refer to formal approaches of belief updating with high degrees of precision.

Moreover, the addition of a confidence level raises the question what level of confidence is acceptable. Please specify how these confidence levels will be used to guide later interpretation.

3)      Typo: “while 41.1 % think the Finns think gaming is harmful” (p. 23)

Reviewed by , 12 Mar 2023

I think these revisions mostly look good.

The only exception is that the authors MUST have a SESOI (smallest effect size of interest) of minimum of r (or beta) of .10.  I think the authors misunderstood me here...this has nothing to do with hypothesis testing or sample size, only avoiding misinterpreting noise for "true" effects. There's a risk of noise being misinterpreted below effect sizes of .10, as there's a general "hum" of garbage results below that effect size.  This is true whether a study is exploratory or confirmatory.  The only issue about sample size is that if the sample size is small, it won't be powerful enough to detect effects that low anyway.  Again, please read Ferguson and Heene (2021) for an explanation for why this is critical.  But this is an unambiguous MUST for me to sign off on this registered report.  

Reviewed by , 16 Mar 2023

The revised addresses all of our concerns well--I appreciate the opportunity to remain involved and very much look forward to your progress.

 

The attached copy of your tracked changes version includes a few more comments for slight changes to wording that could be considered.

 

Thank you again.

 

Michelle

Download the review

Evaluation round #1

DOI or URL of the report: https://osf.io/56ugm

Version of the report: v1

Author's Reply, 24 Feb 2023

Download author's reply Download tracked changes file

Dear Recommender and Reviewers,

Thank you for your patience. It took a lot of time with the revisions. Three remarks:

1. We reformatted the file for clarity. Therefore, the track changes file and the "clean" file have minor differences. We didn't mark track changes in the supplements because the changes were so major that basically everything was rewritten. OSF was also reorganized for clarity. A pdf-version is also available here: https://osf.io/xwubz

2. One reviewer asked further details, which we have promised to share confidentially via private repository link. Here is the link, which remains active until end of March (password sent to recommender):  

https://nextcloud.jyu.fi/index.php/s/c7DojEsYzLaNx5d 

3. We have started first steps in data collection in Finland and Slovakia, fully following this v2 protocol. First adolescents have been interviewed, and one treatment-seeker. This keeps our bias level at 4. 

One more thank you for the helpful feedback and time investments with this long manuscript. 

Sincerely, 

Veli-Matti Karhulahti

Decision by ORCID_LOGO, posted 27 Dec 2022, validated 27 Dec 2022

Dear Matti

 

I now have three thorough and thoughtful expert reviews. All are positive about your research and all ask for important clarifications.  I was not sure if you intended quantitiive analyses as part of the Stage 2s, and if so there would of course be a benefit of specifying them in the Stage 1.  You should also consider the extra clarity that would be provided by using a Desiggn Table, adapting it where possible for qualitative methods.

 

best

Zoltan

Reviewed by , 27 Nov 2022

Please see the attached file. 

Download the review

Reviewed by , 18 Dec 2022

I appreciated the opportunity to review this RR.  I have just a few comments below that I wonder if these may improve some of the ideas in here.

First, when mentioning the WHO "gaming disorder", it might be helpful to note the specific controversies around this.  Many scholars directly opposed the WHO (Aarseth et al., 2017) as did the APA and Psychological Society of Ireland's media psychology divisions in an open statement (https://div46amplifier.com/2018/06/21/an-official-division-46-statement-on-the-who-proposal-to-include-gaming-related-disorders-in-icd-11/).  The authors do a nice job discussing the wider controversies, but I think it's good to note specific criticisms of the WHO's decision.

Related to this conceptually, is the lack of interest (by the WHO and, frankly, most researchers) over the larger issues of behavioral overuse disorders...why is there so little interest in overeating, work, exercise, shopping or religion "addiction" (Griffiths, to his credit, has papers on dance and fishing "addiction".)  There's an important question about why we're always hyperfocused on technology (and I'm not convinced the answer is that technology is inherently more addictive than these other things.

This I think is also a practical issue for the paper...particularly given lack of clear guidance on what constitutes "gaming disorder"...responses by both clinicians and gamers may simply invoke societal moral panics in open response formats.  I'm thinking in particular of the clinician ratings here.  Given how easy it is to portray anything addictive...see the offline friends addiction scale (Satchell et al), I'd like to see a bit more attention to this.

I think the authors include a lot of good and important variables about gaming contexts and health, but I think it will be possible to find spurious correlations here, without controlling for other variables. I assume that T1 health/mental health data will be controlled.  However, particularly as we have some decent information to suggest that academic and family stress is causal for problematic gaming, I think some of these larger contexts would need to be measured and controlled.

The authors need to set a smallest effect size of interest (SESOI) for their quantitative analyses, otherwise they may inadvertently misinterpret "statistically significant" noise effects as hypothesis supportive when they should not be interpreted as such.  See Ferguson & Heene (2021) for discussion and guidance.

In conclusion, I support the authors' efforts here and hope my suggestions are constructive.

Signed,

Chris Ferguson

Reviewed by ORCID_LOGO, 13 Dec 2022

User comments

No user comments yet