Applying Harmful Dysfunction Analysis to social media usage in adolescents
The Harmful Dysfunction Analysis applied to the concept of behavioral addiction: A secondary analysis of data from the Health Behaviour in School - aged Children 2018
Abstract
Recommendation: posted 25 October 2024, validated 29 October 2024
Jones, A. (2024) Applying Harmful Dysfunction Analysis to social media usage in adolescents . Peer Community in Registered Reports, . https://rr.peercommunityin.org/PCIRegisteredReports/articles/rec?id=664
Recommendation
The Stage 1 manuscript was evaluated by three expert reviewers across two rounds of review. Following in-depth review and responses from the authors, the recommender judged that the Stage 1 criteria were met and awarded in-principle acceptance (IPA).
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence.
List of eligible PCI RR-friendly journals:
- Addiction Research & Theory
- Collabra: Psychology
- Peer Community Journal
- PeerJ
- Royal Society Open Science
- Swiss Psychology Open
1. Amendola, S., Hengartner, M. P., & Wakefield, J. C. (2024). The Harmful Dysfunction Analysis applied to the concept of behavioural addiction: A secondary analysis of data from the Health Behaviour in School-aged Children 2018. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/y3ub8
2. Wakefield, J. C. (1992). Disorder as Harmful Dysfunction: A Conceptual Critique of DSM-III-R’s Definition of Mental Disorder. Psychological Review, 99, 232–247. https://psycnet.apa.org/doi/10.1037/0033-295X.99.2.232
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.
Evaluation round #2
DOI or URL of the report: https://osf.io/j8dcu
Version of the report: 3
Author's Reply, 18 Oct 2024
Decision by Andrew Jones, posted 16 Aug 2024, validated 17 Aug 2024
Thanks to the reviewers for their comments. As I could not get a follow-up from one of the reviewers I invited a further reviewer to take a look. I think their comments are largely points for clarity which could be addressed and then we can hopefully recommend the report.
Many thanks.
Reviewed by Gemma Lucy Smart, 28 Jun 2024
Thanks once again for the opportunity to review this paper and the tracked changes. I appreciate the rigor with which the authors have responded to my comments and I am happy with the changes that they have made.
In response to their request for a couple of references from game studies, I have some references that *may* be of interest - although some come from within psychology:
Griffiths & Nuyens, 2017; Karlsen, 2013; and from game studies I think this paper discussing the structural elements of games is important for comparing to social media: Costello and Edmonds, 2007.
Many thanks again.
Reviewed by Josip Razum, 16 Aug 2024
Thank you for the opportunity to review this interesting report. I'd like to thank the previous reviewers in round 1 on their mindful and comprehensive suggestions, which I think have improved the manusript. However, I think there is still room for further improvement. I'll number my comments chronologically to make reading easier.
1. On page 5: "Other longitudinal studies present a confusing picture in which PSMU correlates with such conditions as anxiety, insomnia, and depression, but at an individual level is not necessarily causally related to such conditions (Chang et al., 2022; Lin et al., 2021)."
I'm not sure that this sentence accurately reflects the findings of the study by Chang et al. (2022). For example, Chang et al. (2022) found a reciprocical relationship between anxiety and future problematic social media use at the within person level. Please revise this sentence accordingly. I would not say that these studies can lead us to conclude that PSMU "at an individual level is not necessarily causally related to such conditions".
2. On page 5: "This suggests that, while time spent playing video games seems to play a major role in determining diagnosis under current approaches, it may not be an effective indicator for validly differentiating high versus pathological involvement, indicating a challenge to current approaches."
Altough these studies show that time spent playing video games does correlate with gaming disorder symptoms whereas not neccesarily being problematic, I would not say that it plays a major role in determining the Gaming Disorder diagnosis. None of the Gaming Disorder criteria, neither in DSM-5 nor in ICD-11, directly involve time spent playing video games. In DSM-5 it is mentioned within the diagnostic features of GD that problematic gamers typically spend upwards of 30 hours per week playing video games, but this is merely a description, not a criterion. Please revise this sentence accordingly.
3. (on page 6) Besides citing the two older studies by Charlton and Danforth, you could also mention systematic reviews that studied the tolerance and withdrawal criteria of Gaming Disorder across a broad range of studies.
A systematic review of the psychometric validity and the appropriateness of tolerance as a criterion for Gaming Disorder (Razum et al., 2023) has found that tolerance lacks relevance in measuring Gaming Disorder. Another review by Kaptsis et al. (2016) has found a paucity of studies investigating GD withdrawal, and in studies they reviewed "Five (50%) of the qualitative studies and six (86%) of the treatment studies reported no withdrawal symptoms in their samples."
Kaptsis, D., King, D. L., Delfabbro, P. H., & Gradisar, M. (2016). Withdrawal symptoms in internet gaming disorder: A systematic review. Clinical Psychology Review, 43, 58-66.
Razum, J., Baumgartner, B., & Glavak-Tkalić, R. (2023). Psychometric validity and the appropriateness of tolerance as a criterion for internet gaming disorder: A systematic review. Clinical psychology review, 101, 102256.
4. On page 7: "The ICD-11 definition of GD benefited from these suggestions and incorporated changes clarifying that the main symptom of GD is not excessive involvement itself but rather impaired control over gaming, with other classic symptoms of dependence included as possible additional clinical features."
In the ICD-11 definition of Gaming Disorder it is stated: "Gaming disorder is characterised by a pattern of persistent or recurrent gaming behaviour (‘digital gaming’ or ‘video-gaming’), which may be online (i.e., over the internet) or offline, manifested by: 1. impaired control over gaming (e.g., onset, frequency, intensity, duration, termination, context); 2. increasing priority given to gaming to the extent that gaming takes precedence over other life interests and daily activities; and 3. continuation or escalation of gaming despite the occurrence of negative consequences."
Therefore, Gaming Disorder is defined by these three symptoms: impaired control, increasing priority, and continuation or escalation of gaming, with a requirement that the gaming behavior results in marked distress or significant impairment. Nowhere is it stated that impaired control is the main or the only symptom. Please amend this.
5. On page 7: "However, despite providing some suggestions to differentiate GD from normal gaming behavior, the definition of the ICD-11 does not propose specific and effective indicators for discriminating between normal-range (e.g., functional, high-involvement gaming) and disordered pathological gaming."
This claim needs more elaboration. In my view, the ICD-11 criteria do provide a clear threshold between highly involved and disordered gaming: if the person meets all the criteria, they have Gaming Disorder. If they do not meet the criteria, they do not have the disorder no matter how much they play. ICD-11 also describes "boundary with normality", where they state that some gamers play a lot for different reasons, that they may play more in particular contexts such as holidays etc. There is also a category of "hazardous gaming", which involves increased risk while not meeting the full GD criteria - this category could perhaps be better defined.
In any case, the authors need to provide more arguments for their claim that: "the definition of the ICD-11 does not propose specific and effective indicators for discriminating between normal-range (e.g., functional, high-involvement gaming) and disordered pathological gaming.".
https://icd.who.int/browse/2024-01/mms/en#1448597234
6. On page 10: In my view, the authors need to provide a stronger argumentation for why they plan to use withdrawal symptoms as an indicator of dysfunction, as this criterion has been previously criticized as not valid for assessing gaming disorder. In the study by Castro Calvo et al. (2021), a low percentage of included experts thought that withdrawal symptoms have diagnostic validity, clinical utility or prognostic value in assessing Gaming Disorder.
Perhaps problematic social media use may be different, but then the authors need to cite relevant work and/or their rationale. The authors need to provide clear arguments for why they included withdrawal symptoms as an indicator of dysfunction.
Castro‐Calvo, J., King, D. L., Stein, D. J., Brand, M., Carmi, L., Chamberlain, S. R., ... & Billieux, J. (2021). Expert appraisal of criteria for assessing gaming disorder: An international Delphi study. Addiction, 116(9), 2463-2475.
7. On page 13: The authors thoroughly stated the study limitations and provided suggestions for which theory-driven aspects of dysfunction and harm could be investigated in future studies; however perhaps they could additionally elaborate what would constitute a "true" non-confirmatory application of HDA to problematic social media use. Would this involve interviews with problematic users where possible indicators of dysfunction and harm would be initially investigated?
Evaluation round #1
DOI or URL of the report: https://osf.io/efv56
Version of the report: 1
Author's Reply, 13 May 2024
Decision by Andrew Jones, posted 22 Mar 2024, validated 23 Mar 2024
The two reviewers have been comprehensive (I thank them).
They see plently of merit in this, but there are some suggestions to improve the rigour. Furthermore, they both seem in agreement about your framing and conceptualisation.
I hope you are able to address their points in a revision.
Reviewed by Veli-Matti Karhulahti, 24 Jan 2024
Thank you for inviting me to review this interesting manuscript. For context, I’m somewhat familiar with HDA and find it as one of the most interesting theoretical alternatives in the current field. My topic expertise is on gaming disorder, but I've also worked on related social media questions. In this review, I focus on the study design and its underlying philosophy. I leave details of the statistical analyses to be vetted by statisticians (I have a few remarks but am not fully qualified to propose specific statistics solutions, which should be done by statisticians). I number my comments chronologically, not in order of importance but to make reading easier.
1. I like the intro. It makes the state of the field clear and shows to the main problems that the design will tackle. There are some sentences and framings that are inaccurate, nonetheless. E.g., the first sentence “present study is an attempt to advance the validity of the diagnosis of behavioral addiction” is confusing: currently there is no diagnosis at all for PSMU so it’s strange to advance the validity of something that doesn’t exist (note that PSMU dominantly derives from problematic, not pathodological, social media use). I see what the authors want to say (improving related constructs) but it’s important to say things like this correctly. Likewise, on p. 4, it reads the study “uses a related behavioral addiction, PSMU, in a test of validity” but again PSMU is not a behavioral addiction albeit being often studied as such (e.g. Billieux et al. 2015 DOI 10.1007/s40429-015-0054-y “the evidence supporting PMPU [sic] as an addictive behavior is scarce” and thus not included in DSM nor ICD). I would carefully review each sentence to ensure correct framing.
2. Related to the above, as a larger structural issue, I see it problematic that even though the intro is informative, it focuses narrowly on the BA discourse and mostly on gaming; where the study itself then turns out to operate with PSMU data. There has been a lot of debate on the construct differences/issues regarding use of “social media”, “smartphones”, “social networking”, “screen time” etc. and what the related dynamics of (problem) behavior are (e.g., Mannell https://doi.org/10.1177/2050157918772864 mechanisms of disconnective affordances, Conroy et al. http://dx.doi.org/10.1037/ppm0000425 on smartphone overreliance, Vainio et al. https://doi.org/10.1037/ppm0000508 on perceptions on health changes etc.etc.). In brief, the PSMU literature is rich and not currently well represented by the limited BA lens, which mostly applies to gaming disorder that has a formal diagnostic status (adding two reviews here which are by no means exhaustive).
Bayer,J.B.,Triêụ,P.,&Ellison,N.B.(2020).Social media elements, ecologies, and effects. Annual Review of Psychology, 71(1), 471–497. https:// doi.org/10.1146/annurev-psych-010419-050944
Orben, A. (2020). Teenagers, screens and social media: a narrative review of reviews and key studies. Social psychiatry and psychiatric epidemiology, 55(4), 407-414. DOI 10.1007/s00127-019-01825-4
3. Following from the above, I am concerned about how HDA matches PSMU. Indeed, because PSMU is not an addiction but (by definition) a spectrum of problematic social media use patterns, it naturally involves diverse types of problems. Example: on p. 7 it is stated: “harmful consequences in the absence of a dysfunction do not qualify as a disorder. For example, obesity or postural problems may be consequences of inactivity or sedentary behaviors due to high amount of time spent gaming/using social media in absence of a dysfunction.” This could be seen as a straw man when applied to PSMU because PSMU is not a disorder, as noted earlier. The fact that PSMU may involve obesity or postural problems is not inconsistent with the idea of PSMU (as a non-disorder). Only if the addiction/disorder framing is applied to social media, the debate becomes relevant. Therefore, I would encourage the authors to carefully revisit the underlying philosophy on the constructs of the study, which currently seems to stand on the auxiliary hypothesis that PSMU is addiction/disorder and there is a need for a better disorder framing.
4. Moving to the next item on my list, I strongly recommend removing the hypothesis and doing this as a non-confirmatory study. Testing a hypotheses by the strict PCI RR guidelines would require major revisions and updates on the currently brief formulation, including assessment of practically meaningful SESOIs, null testing, effect size justifications, etc. Because HDA is being tested, the design should also be crafted in such way that clear criteria for falsifying HDA are outlined, preferably by equivalence testing (see author guidelines). In the current framework, to be honest, I do not see hypothesis testing feasible (also, as the data are already available, confidence level will be 1 anyway so the benefit of testing confirmatory hypotheses is very minor). If the authors really wish to do this, I’m ready re-review the improved hypotheses for the next version.
5. A few notes on methods. Regarding the heterogenous composite index, I don’t see good evidence/reasoning why it would produce (more) informative results. The authors themselves too address the issue (p. 11) but nonetheless decide to do it. What is the (good) reason for not modeling all these variables separately?
6. Finally, I have a bigger theoretical note. I am saying this because I’m actually a big fan of HDA and would love to see it carefully used in this field. I personally believe it can help explain some of the massive confusion in the current BA literature. In particular, a highly productive observation (which Wakefield made already in the influential 1992 paper) is that “whether a condition is a disorder is not determined by how the diagnosed individual subjectively happens to feel about the condition’s effects, but by more ‘objective’ standards determined by the culture’s value system” (cited in the MS p. 7). This is highly important especially for BAs (like gaming) which carry different stigma’s in different cultures, and the lost time/productivity is measured against different cultural norms of what kids/adults "should be doing instead”. For BAs, this culturally relative definition of harm has already been studied in-depth by many researchers and studies, e.g. Trent Bax’s extensive work on internet addiction in China (see e.g. the monograph) and Jeffrey Snodgrass team’s numerous papers on “cultural dissonance” with internet gaming/disoder (for a recent excellent investigation of culturally generated ‘harm’ in India, see https://doi.org/10.1086/717769).
--> in this theoretical context where a disorder is experienced through culture and “not determined by how the diagnosed individual subjectively happens to feel”, how does the current study tackle the paradox that its own harm inference derives from dichotomous self-report data (Table 1)? I continue on this below.
7. I see it as a meta-problem for this study that despite it being nicely designed to combat the unproductive “confirmatory appraoch” (confirming 6 criteria), instead of exploring the distinct or unique links of dysfunction and harm related to PSMU, it will carry out “another confirmatory appraoch” (confirming 2 criteria) by testing for HDA1/HDA2 via predefined items (Table 1). The logic is basically similar with the component model; the “components” are just different. To take a non-confirmatory approach (via the HDA framework) one would optimally explore the types and forms of problems that manifest in relation to social media use and see how they map out in HDA. E.g., a couple of years ago (https://www.nature.com/articles/s41599-023-01775-y ) we asked gaming treatment-seekers about the types of problems they have, some of which matched ICD criteria but many did not. Only 42% met DSM criteria, yet there were no differences in types of problems between DSM-meeting and non-meeting ones (and all were in self-sought treatment). My impresssion is that the current HBSC dataset is not very suitable for such non-confirmatory approach if the only available BA data are 7 predefined self-report items that derive from the component model. To be clear, I fully support the idea of exploring the relevance of different problem-items, which has previously yielded informative results (e.g., Colder Carras & Kardefelt-Winther https://doi.org/10.1007/s00787-018-1108-1 , Ballou & Zendle https://doi.org/10.1016/j.chb.2021.107140). I believe the currently planned study can produce likewise interesting results and reflect them usefully against the theoretical HDA framework, but one must carefully design the exploration to not to frame results as confirmatory (unless the hypothesis structure is completely rebuilt; that would then need to be reassessed).
8. As a sidenote, although I avoid commenting on the statistics, I didn’t notice a supplement for the code (assuming open software like R is used) or other description of the specific tools in the analysis. This should be included as per PCI RR guidelines.
Overall, this will be an interesting study and has the potential to produce informative results especially by exploring PSMU data in the HDA context. My major concerns are a) the mixing of addiction/disorder and problems on the construct level, b) need to remove or completely restructure the confirmatory element, and c) build a stronger bridge between theory and data/methodology to make a convincing exploration on which we can keep constructing robust (also confirmatory) studies later. Naturally, the recommender will assess to what degree these observations align with their/other views.
I always sign all of my reviews so I can be personally contacted in case my feedback feels unclear or unfair. I also add a default statement: some studies I have mentioned include me as an author; it is up the authors to assess whether they are worth citing and in case I will be re-reviewing this MS in the future, any citation or a lack thereof will not affect my assessment in any way.
Veli-Matti Karhulahti
Reviewed by Gemma Lucy Smart, 20 Mar 2024
Thank you for the opportunity to review this report. I have some suggestions to improve the overall rigor which I'll list below with their line numbers for reference.
Overall I'm a bit sceptical about the use of PSMU as a surrogate for IGD. They are very different uses of technology, and different types of activities. You're going to have to do some heavy conceptual lifting to make any claims that they are the same that anyone in Game Studies for instance would accept. I encourage you to look at the literature on types of Play in Games and tasks in Games critically examine whether the repeated tasks you are looking at are actually the same or similar enough to those of gamers.
One of the conceptual issues I have with IGD is that it lumps gamers into a homogenous category when they are doing heterogenous tasks. As you note, the potential for proliferation of behavioural addictions is something to be concerned about. My suggestion here is that we have that potential within a category because you're actually looking at people doing different things. It's a fundamental lack of understanding of gaming.
It may be that you can find some correlates here, and if so great. I think the HDA is a good model to apply to the concept of IGD, but the literature from Game Studies is so routinely ignored in this space that I encourage you to engage with it to improve the conceptual rigor of your surrogate here.
----
189 While I agree as an overall claim about the addition category in the DSM, I suggest the authors revise this section to avoid rhetoric. There are obvious limits to the theoretical framing, even within current DSM approach. Without such limits any substance would be a target of addictive pathology, including absurd candidates like water. Where authors have framed the debate as 'any type of behaviour will potentially on the table as a behavioural addiction' that's simply not true. We can have a more nuanced discussion than that. Proliferation is a conceptual issue, yes. But it's not so out of control that anything can be framed as addictive.
224 See Murphy & Smart (2018) for an overview of mechanistic models as theoretical approaches to the problem - I think they it would fit well enough with the HDA approach, mechanistic models, especially the work of Ross e al. (2008) agree with your view here, obviously moreso in the 'dysfunction' part of the equation'.
283 Addiction does pose a problem here as it's always been framed in such a way that allows for external actors to determine or identify that harm in a way that other disorders may or may not (I.e. the old fashioned criteria about addiction affecting marriage). This does mean that you may have to consider 'harm to others' not just 'harm identified by or harm to' the individual gamer. It depends how you conceptually frame harm in this context.
319 Interesting data set. The average age of gamers changes yearly, but it's around 30-35 years old. May be worth keeping in mind, especially as social media use would no doubt be quite different in that age group?