Recommendation

Different ways of describing problematic substance use and its treatment influence public stigma

ORCID_LOGO based on reviews by Nicholas Sinclair-House
A recommendation of:
toto

The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered Report

Abstract
Keywords
Submission: posted 21 October 2022
Recommendation: posted 10 February 2023, validated 10 February 2023
Cite this recommendation as:
Dienes, Z. (2023) Different ways of describing problematic substance use and its treatment influence public stigma. Peer Community in Registered Reports, 100330. https://doi.org/10.24072/pci.rr.100330

This is a stage 2 based on:

To help or hinder: Do the labels and models used to describe problematic substance use influence public stigma?
Charlotte R. Pennington, Rebecca L. Monk, Derek Heim, Abi K. Rose, Thomas Gough, Ross Clarke, Graeme Knibb, and Andrew Jones.
https://osf.io/7e2qm/

Recommendation

People experiencing problematic substance use are often stigmatised by the general public. This public stigma may impair such people obtaining help and the quality of help that they receive. For this reason, previous research has investigated the factors that may exacerbate or lessen stigma by focusing on the terminology used to describe problematic substance use. However, the evidence is not clear cut, with some studies suggesting that labelling the condition as a "chronically relapsing brain disease" vs a "problem" reduces certain elements of stigma and other studies finding absence of evidence. A closer look at these studies points to methodological differences that may explain their results, such as whether problematic substance use is compared with another health condition, whether the individual is described as seeking treatment or not, and whether general or discrete elements of stigma are measured.
 
In this Stage 2 Registered Report, Pennington et al. (2023) isolated these methodological differences to investigate if any of them influenced two different measures of stigma used in previous work. They found that greater social distance, danger and public stigma but lower blame were ascribed to drug use relative to a health concern, supporting previous research to suggest that problematic substance use is a highly stigmatised health condition. Furthermore, greater (genetic) blame was reported when drug use was labelled as a ‘chronically relapsing brain disease’ relative to a ‘problem’. The results for attributional judgement were either inconclusive or statistically equivalent. In summary, these findings suggest that the labels and models used to describe problematic substance use may impact upon public stigma in distinct ways. The authors suggest that future research should justify which measures are being used in line with theory. They also put forward the notion that addiction is a functional attribution, which may explain the mixed literature on the brain disease model of addiction to date.
 
The Stage 2 manuscript was evaluated over one round of specialist review and several rounds of discussion with the recommender. Based on comprehensive responses, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/4vscg
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
Pennington, C. R., Monk, R. L., Heim, D., Rose, A. K., Gough, T., Clarke, R., Knibb, G., Patel, R., Rai, P., Ravat, H., Ali, R., Anastasiou, G., Asgari, F., Bate, E., Bourke, T., Boyles, J., Campbell, A., Fowler, N., Hester, S., Neil, C., McIntrye, B., Ogilvy, E., Renouf, A., Stafford, J., Toothill, K., Wong, H. K., &  Jones, A. (2023). The labels and models used to describe problematic substance use impact discrete elements of stigma: A Registered Report. Stage 2 acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/z9bnf
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Evaluation round #3

DOI or URL of the report: https://osf.io/cvhz5

Version of the report: 3

Author's Reply, 10 Feb 2023

Download author's reply

Dear Zoltan,

Please see the attached PDF for our Response to the Editor letter.

Yours sincerly,

Charlotte

Decision by ORCID_LOGO, posted 01 Feb 2023, validated 01 Feb 2023

Dear Charlotte

We are getting there but a few points. In a few places you say things like:

"participants in the drug use condition allocated greater punishment compared to those in the health concern condition (d= .14, 99% CI = .02, .26), with the upper CI significantly outside of the equivalence range"

The upper limit of the CI is not "significantly outside" the equivalence range, it is just "outside" of it. That phrasing makes it sound like .26 is significantly different from .2 - but that has not been tested. What matters is what one can say about the population mean; and the sample mean is not significantly outside the equivalence region, so the set of possible population means, i.e. what is inside the CI and hence cannot be rejected, includes values deemed too small to be interesting i.e. practically equivalent to zero. You are allowed to say the drug use condition allocated greater punishment compared to those in the health concern condition; but the following clause should be deleted. (Also consider other similar sentences.) But given your claims about what effects are meaningful, and your Stage 1 decision procedure, even this phrasing is not the one you should use (see below).

You say "We refer to an effect as ‘significant’ if, given α = .01, the mean difference is
significantly different from zero and the 99% CI falls outside of the equivalence range" - but this is not the rule that has been applied, at least on a straightforward interpretation of what "falling outside" means, and given the logic of wanting to exclude all meaningless values in order to conclude there was a meaningful one; and also given how such reasoning has proceeded since Greenwald (1975) onwards; and given the need for consistency in a decision procedure for when a difference is regarded as good enough to be meaningful. If this rule were applied then you would declare a meaningful difference if the bottom limit of the CI were above the upper limit of the equivalence region (and vice versa, for a CI below the equivalence region). That rule, as I have just stated it, makes perfect sense. In the stage 1 you phrased it thus: "Equivalence will therefore be asserted if, given α = .01, the 99% confidence interval of the mean difference lies within this equivalence region, and rejected if the 99% CI lies outside of this region" and that states what I have just stated - equivalence is only rejected if the CI lies outside the equivalence region. Now I see you haven't distinguished "completely" from "partially" - but "partially" leads to self contradiction in what erffects are regarded as meaningful. Thus, the striaghtforward interptation of what you have stated that preserves self consistency is that outside means completely  outside - and I presume you wish to avoid self contradiction.  Conversely, equivalence only is declared when the CI lies within the equivalence region. THis means you suspend judgment in all other cases - namely if the CI overlaps both the equivalence region and the region of values deemed meaningful.  Notice this condition for suspending judgment is different from the one you have just stated; for example, if a sample mean is significantly different fropm zero, yet the CI spans equivalent and meaningful values, you suspend judgment. This rule needs to be consistently applied.

Concerning whether an effect size of .15 is meaningful for some tests but not others, in the absence of a principled argument for why, this seems arbitrary. So bear this in mind in how you interpet results.


Evaluation round #2

DOI or URL of the report: https://osf.io/ycz3q

Version of the report: 2

Author's Reply, 31 Jan 2023

Download author's reply

Please see attached PDF and thank you for your patience.

Decision by ORCID_LOGO, posted 31 Jan 2023, validated 31 Jan 2023

Thank you for your revision!  I still see some inconsistencies in applying the logic of a CI being within or outside a suitably defined equivalence region:

1) "For the Financial Discrimination Task, participants in the drug use condition allocated greater punishment compared to those in the health concern condition, with the observed effect size (d= .14, 99% CI = .02, .26) significantly outside of the equivalence range"

But the sample mean is not significantly outside the equivalence region. If it were signfiicantly outside the equivalence region, the lower limit of the CI would be equal to or above the upper limit of the equivalence region. This problem occurs in various sentences. If by assumption values within the equivalence region are too small to be of interest - that is what an equivalence region is - then a CI that includes such values does not establish that an effect of interest was found.

2) "The difference for continued care (d= .01, CI = -.15, .17) was not statistically different to zero and equivalent."  Also: "and for punishment (d= .007, 99% CI = -.16, .17) was equivalent" Yet, as you say elsewhere, these CIs include effect sizes previously regarded as of interest.  This point needs to be explicitly made. Also you should explicitly state that your equivalence region may include values of actual interest, at a point in the discussion where you conclude equivalence.

best

Zoltan


Evaluation round #1

DOI or URL of the report: https://osf.io/28jud

Version of the report: 1

Author's Reply, 31 Jan 2023

Download author's reply

Please see attached PDF.

Decision by ORCID_LOGO, posted 06 Dec 2022, validated 06 Dec 2022

One reviewer from the Stage 1 has responded and indicated their approval of the Stage 2. Incidentally their review read to me as possibly about the Stage 1 (a reviewer sees links to both Stage 1 and Stage 2), but I have double checked with them - they definitely read the Stage 2, they were positive and there were no problems.

Just one thing on my side. As per my comments for the Stage 1, equivalence cannot be concluded unless there is justification for the equivalence limit being so small that it is only just interesting/uninteresting from the point of view of the scientific theory tested. Resource limits obviously do not provide that justification, as they are arbitrarily related to theory. Indeed several of your CIs within the "equivalence region" include effect sizes that past studies have used as support for the theory. So remove references to having found equivalence, and just e.g. leave it as a case of having estimated the possible effect size.

Reviewed by , 21 Nov 2022

The rationale for the study and the scientific validity of the research question remain clearly demonstrated.

As with the previous draft, I note the improvements to the design and proposed analysis made in light of reviewer comments and the editor’s recommendations.

The authors have bolstered the justification for aligning their minimal effect of interest with the practical limitations outlined. The refinements made to the methodological approach appear to address all the substantive points raised in comments on the earlier draft.

I note the authors’ response on the point of licit vs illicit substances. Whilst I think it remains an interesting question, I quite accept that it is not one the authors have set out to answer, and therefore acknowledging it in passing (as the revised draft does) seems appropriate.

Considered as a whole, I believe this RR satisfies the relevant review criteria.

User comments

No user comments yet