DOI or URL of the report: https://osf.io/jzpc2?view_only=4238d2ee3d654c4f908a94efea82a027
Version of the report: v1 dated 02/02/2023
Please find our response letter attached, and the tracked changes manuscript document.
The three reviewers from Stage 1 returned to evaluate your Stage 2 manuscript, and I'm happy to see that their evaluations are unanimously positive. There are, however, some useful comments that should help further refine the final article, including suggestions for presentation of results, clarifying the interpretration, and new content for the Discussion. We look forward to receiving your revised submission in due course.
In this manuscript, the authors present the results of an observational study in which they examine the relationship with childhood adversity and ratings of subjective drug effects in response to pre-surgical administration of mu-agonist drugs. The authors hypothesized they would conceptually replicate their previous finding that childhood trauma is related to increased opioid liking. Contrary to their hypothesis, they report a significant negative association between childhood adversity and opioid liking, and no significant relationship between childhood trauma and ratings of “feeling good.” Overall, this is a well-written manuscript describing an important research question and the authors have made every effort to conduct the study in accordance with their pre-registered plan.
Intro/methods: The introduction and methods do not deviate substantially from the earlier submission.
Results/discussion: Overall, the results and discussion are clearly presented.
-It would be nice to have a figure depicting the main results for H1 and H2.
-One interesting thing the authors note in passing, but may deserve more attention, is the fact this study was conducted on participants in a high stress environment (just before a surgical procedure). Since some work by the authors has been done addressing the context in which opioids have different effects (Loseth, Guro E., Dan-Mikael Ellingsen, and Siri Leknes. "State-dependent μ-opioid modulation of social motivation." Frontiers in behavioral neuroscience 8 (2014): 430.) it may be worth pointing out this context more explicitly in the introduction, or at least referencing this idea of state-dependent differences in opioid effects in the discussion.
-In discussing the failure to replicate their previous findings, it would be helpful to include a paragraph on how the demographics of the two samples compare to one another.
The authors have carried out the analyses as planned, with additional analyses in a separate results section. The discusson draws reasonable conclusions from these results. It is clearly written. I just have a few points to make:
1) In the discussion a non-signfiicant result is taken to support the absence of a relationship ". Exploratory subgroup analysis restricted to patients matching the groups included in the original study (‘none’ or ‘severe’ in at least one CTQ domain) showed a non-significant pattern of change in feeling good that was consistent with the original study, providing very preliminary support for a non-linear effect of childhood adversity that should be examined with greater sample sizes. " Just a minor tweak - it is not that the non-significant result supports a non-linear relationship, as allows it as possible. So I suggest replacing "providing very preliminary support for" with "allowing for".
2) Raw regression slopes are given, which is good, but they are hard to interpret. The authors do explain the units for one of them - but that explanation does not seem to match the description of the scales. I always do the following which makes life a lot easier: report in terms of response averaged over items. So feeling good is on a 11-pt Likert scale and each item of the CTQ is on a 5-point Likert scale (not percentage points). I would report the slope as "X Likert units of liking/Likert unit of CTQ", and label the units as such everytime a slope, SE or CI is reported. Now we know what we are talking about in terms of the units the subjects responded with. It gives a clearer sense of how big or small an effect is. (Correspondingly, I report all means and SDs in the same averaged-over-items units - I don't want to tell other researchers to do this as the original total scale may have some meaning for them; but when it is my research I always do this. And I report corresponding effects from other papers in such units as well, for comparison, even though the original authors probably reported in some meaningless total scale. I think reporting total scores hides a lot.) But I leave the exact raw units the authors wish to use up to them; just notate them each time a raw slope is given so it is clear what they are.
3) Optional. In raw units, can the authors report a confidence interval on the difference in feeling good and liking between the two extreme groups, identified as in the original study, and compare what the difference was for the original study (e.g. about 18 points on a 0-100 euphoria scale? So about 2 units on a 0-10 feeling good scale). If the CI includes the original study's value then there is not a failure to replicate by an analysis close to the original study's. Just a quick rule of thumb for making sense of the results.
This is a Stage 2 manuscript in which I previously reviewed a Stage 1 manuscript and analyses were performed according to the protocol that IPA was granted. As I mentioned in Stage 1, I am not an expert on this topic, so I cannot make good points on the theoretical aspects. However, I can at least admit that the main hypothesis was not supported by this analysis, or rather that a weak opposite effect was observed. I agree with the authors about the possibility that the sample size issue in this case, including the exploratory analysis, may not have ensured its coverage or verified its linearity. On the other hand, it is worthwhile to publish this study because the exploratory investigation has identified some promising possibilities for future research. Minor points are noted below. Overall, no major problems were found.
- Some minor changes to the introduction and methods are observed, but most of them are probably only modifications to the citation numbers, so there is probably no problem.
- There are analyses that are not included in the Stage 1 manuscript as "Exploratory analyses", but it would be good to state clearly that these are not registered so that readers who do not rigorously check the Stage 1 manuscript will not be misled.