Recommendation

How does reward influence the effect of sleep on memory?

based on reviews by 2 anonymous reviewers
A recommendation of:
toto

Estimating the Effect of Reward on Sleep-Dependent Memory Consolidation – A Registered Report

Abstract
Submission: posted 16 May 2022
Recommendation: posted 18 November 2022, validated 18 November 2022

Recommendation

Sleep and reward each have an important role in human memory. According to the active system consolidation hypothesis, memory consolidation during sleep originates from the repeated reactivation of memory representations that were encoded during wake (Rasch & Born, 2013). Research has also consistently shown that memory performance is enhanced for items or stimuli associated with higher vs. lower rewards. While these lines of evidence are relatively clear, the role of sleep in shaping the interaction between reward and memory is more opaque, likely due to a combination of methodological variation between studies but also due to the field’s reliance on small-N designs and biased reporting practices. Clarifying this three-way relationship, and setting field benchmarks for effect sizes, is crucial not only for building richer neurocognitive models of memory, but for clinical applications such as targeted sleep interventions to treat addiction and other forms of mental illness. 
 
Using a large, stratified online German sample (N=1750), Morgan et al. (2022) will study the three-way relationship between sleep, reward and memory by asking whether, and if so how, reward influences the magnitude of sleep-dependent memory consolidation. Using an AM:PM-PM:AM design in combination with a motivated learning task, the authors will address three main questions: first, whether sleep yields greater memory performance compared to an equivalent period of wake; second whether information associated with higher reward leads to greater memory performance compared to lower reward; and third, the crucial interaction of whether sleep causes greater recognition memory performance for higher vs. lower reward items. The design also includes a series of rigorous positive controls to confirm testability of the hypotheses, while measuring a host of additional moderating variables for exploratory analyses (including age, education status, mental health, and more).
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/q5pk8
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Rasch, B. & Born, J. (2013). About Sleep's Role in Memory. Physiological Revews, 93, 681–766. https://doi.org/10.1152%2Fphysrev.00032.2012
 
2. Morgan, D. P., Nagel, J., Cagatay Gürsoy, N., Kern, S. & Feld, G. B. (2022). Estimating the effect of reward on sleep-dependent memory consolidation – A Registered Report, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/q5pk8
Cite this recommendation as:
Chris Chambers (2022) How does reward influence the effect of sleep on memory?. Peer Community in Registered Reports, . https://rr.peercommunityin.org/articles/rec?id=206
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Evaluation round #2

DOI or URL of the report: https://osf.io/4kfhu?view_only=c4a0bcdcbcee423e9e2e61b2e75ff490

Version of the report: v1

Author's Reply, 10 Nov 2022

Download tracked changes file

Dear Prof. Chambers, 

we would like to thank you and the reviewers for your feedback on our registered report. We have now amended the issues pointed out by the reviewers in the attached PDF document and have amended the link to the manuscript in the article data for PCIRR. 

We are very grateful to you for your support throughout this process and have had a very positive experience of the recommendation process at PCIRR.

Myself and the authors look forward to carrying our Stage 1 submission and delivering the eventual Stage 2 report. 

Please feel free to reach out to me if there are any other outstanding issues.

Yours sincerely,

David Philip Morgan (on behalf of the authors)

Decision by , posted 03 Nov 2022

Your revised manuscript has now been evaluated by the two reviewers from the first round. Both are satisifed with the changes. As you will see, there are just a couple of minor remaining issues to address. Once you have made these revisions and resubmitted, we will issue Stage 1 in-principle acceptance without further in-depth review.

Reviewed by anonymous reviewer, 23 Sep 2022

The authors have done an impressive job responding to the reviewers comments and I believe they have appropriately addressed all the comments I made in the previous round of reviews. The positive controls in particular greatly strengthen the research plans. 

One very minor issue: The bottom right panel of Figure 3 seems to have a formatting error with text on top of other text, so I suggest correcting this. 

I think this is a very well planned study that will make a strong contribution to the literature regardless of the outcome - a great example of what a registered report should be. 

Reviewed by anonymous reviewer, 28 Oct 2022

I thank the authors for their extremely thorough and concise responses to all my comments. I am satisfied that all of my prior concerns have been addressed and I am very pleased to recommend this study for stage-one acceptance. I look forward to seeing the results.  

I have no further comments apart from a typo on page 37 line 725: do the authors mean “timepoint x reward parameter” here?


Evaluation round #1

DOI or URL of the report: https://osf.io/7qf3p?view_only=c4a0bcdcbcee423e9e2e61b2e75ff490

Author's Reply, 20 Sep 2022

Download author's reply Download tracked changes file

Dear Recommenders, 

thank you for your critical comments on our stage 1 registered report. We have now responded in the attached files and believe that the revisions have made a substantial improvement to the quality of this registered report. 

We are sorry that you were unable to access the supplementary files, we have now rectified this so that you can evaluate the suitability of this manuscript for in principle acceptance. You can do so using the link and password provided in the attached response. Please follow the instructions provided below.

We look forward to hearing from you. Thank you for your serious consideration. 

__________________________________________________________________________________

Supplemental Instrucitions:

Please follow the link and use the password provided in the manuscript.

The images used in the motivated learning task and the motivated learning task itself underwent validation. The associated data and analyses can be accessed by clicking “download all files” (in the top right-hand corner) and then the plots and analyses can be accessed by opening the file named “mlt_validation_html.html” and opening it in your web browser*.

*The html file will not open correctly if you do not download the files into your machine

 

Decision by , posted 15 Jul 2022, validated 04 Nov 2022

I have now obtained two constructive and helpful reviews of your Stage 1 submission. Overall, the reviews are encouraging and the general consensus is that your proposal is a promising contender for eventual IPA. There are, however, a range of issues to address that cut across several of the Stage 1 criteria, including clarifications concerning the hypotheses (and contingencies between them), exclusion criteria and analysis plans, justification of including design features, and inclusion of positive controls. Another substantial concern raised is whether there is tension in your proposal between optimising it for unstated exploratory analyses vs. addressing the main confirmatory hypotheses. A Stage 1 RR should favour the latter as much as possible, but I am open to further discussion of this issue in your response in order to ideally reach a consensus.

On a purely technical note, please can you ensure that the main manuscript includes a direct URL to the Supplementary Information on the OSF so that it is easily accessible to the reviewers. When we invite to reviewers to evaluate RRs, we include only the link to the manuscript (rather than generic links to the parent OSF project), which avoids any risk of reviewers inadvertently assessing the wrong document or version. But because we only send the one direct URL to the manuscript, it is essential that the manuscript itself includes direct links to any such materials within the OSF project that are important for the review process.

On the basis of these reviews I'm happy to invite a Major Revision and response, which will be returned to both of the reviewers for a further look.

Reviewed by anonymous reviewer, 12 Jul 2022

Reviewed by anonymous reviewer, 09 Jul 2022

Summary

 

Memory consolidation is supported by sleep and recent theoretical approaches have suggested that sleep might work in a selective manner to strengthen information that is most relevant or salient to the individual. However, empirical evidence related to this selective memory benefit of sleep is fairly mixed, potentially due to a diversity of experimental methods and low statistical power. In this Registered Report, Morgan and colleagues plan to carry out a timely and well powered investigation of the effects of sleep on memories associated with high and low rewards. They will use an online experiment with a gamified task to determine whether a benefit of sleep for recognition memory is amplified for information associated with high relative to low reward. My overall assessment of the manuscript is positive, but I have a number of comments and queries, which are separated according to the subheadings of the reviewer guidance notes. I have also included some more general comments at the end of my review.

 

Please note that I was unable to retrieve the authors supplementary materials, which were referenced several times in the main manuscript. I will need to see this in the resubmission before providing another opinion on the suitability of the manuscript for stage one acceptance.

 

The scientific validity of the research question(s)

 

1)      From the overview of the literature in the Introduction, it appears that a potential source of inconsistency in previous findings relates to the memory system in question (e.g. declarative vs procedural memory). Is it the case that studies showing a positive effect of sleep on high vs low reward memories are in the procedural domain? Can the authors be sure that a declarative memory protocol is the optimal approach for studying effects of sleep and reward on memory consolidation?

 

2)      A more general point relates to the issue of whether reward actually has any impact of consolidation, irrespective of whether it occurs over sleep or wakefulness. In their Introduction, the authors state: “Either sleep selectively consolidates information associated with high rewards or reward related processes during encoding together with sleep-independent consolidation processes initiated shortly after learning are sufficient to enhance reward memory”. A third possibility is that the benefits of reward occur only at encoding and are not enhanced at all by consolidation. Can the authors rule this out?

 

The logic, rationale, and plausibility of the proposed hypotheses (where a submission proposes hypotheses)

 

3)      The use of a recognition paradigm raises questions about the plausibility of the authors proposed hypotheses. A number of previous studies have shown that recognition paradigms are fairly insensitive to sleep-memory effects, and this lack of sensitivity could pose problems for the authors predictions. Relatedly, H1 focuses on performance in the delayed test, whereas sleep-memory effects are typically shown via the change in performance between immediate and delayed tests.  

 

The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis or alternative sampling plans where applicable)

 

4)      One of my foremost concerns relates to the authors plan to collect a large amount of data for several exploratory analyses, seemingly at the cost of the main research question. In particular, the authors will not exclude participants on the basis of an existing mental health condition, sleep disorder or use of medication (among various other factors that are known to affect sleep) so that they can carry out exploratory analyses of how their findings are influenced by these factors. In my opinion, they should design their experiment in such a way that it is optimised for their main research goal (to test the selective effect of sleep (vs wake) on high (vs low) reward memories) and then carry out exploratory analyses where possible. Accordingly, they should extend their exclusion criteria to other factors that are known to affect sleep, so that they can maximise experimental control. Along the same lines, given the known effect of age on sleep architecture (such as slow-wave sleep, which has been linked to the preferential consolidation of salient information), I recommend that the authors restrict their sample to a narrower age range (e.g. young adults). This, again, will optimise their ability to test the main research question and provide a strong platform for future research in which the effects of age, mental health condition etc, can be studied in the context of the selective memory function of sleep.

 

5)      Along the same lines as the above comment, given that encoding strength is known to influence the effects of sleep on memory consolidation, keeping encoding strength consistent at learning would presumably be of benefit to the main research question.

 

6)      Related to the above point, I question how the authors can separate the effects of reward from encoding strength at learning? Presumably high reward items will be encoded more strongly at the learning phase than low reward items, which would influence their consolidation in sleep. How do the authors intend to control for this?

 

7)      At several points in the manuscript the authors say that data will be collected for exploratory analysis, but they do not provide much insight on the nature of the questions they intend to address. For example, it would be interesting to know the authors’ motivations for collecting data on participants’ experiences during the day, and the question(s) they intend to address in this exploratory analysis.

 

8)      The plan to use a within subjects is statistically optimal, but I am quite concerned about potential attrition rates, given the large number of questionnaires to complete at the recruitment stage, the lack of guaranteed financial compensation, the long delay between completing the sleep and wake conditions, and the delay between the initial recruitment screening and the first main experimental session (minimum of 24 h). The proposed use of a stratified sample could compound this even further, as the authors will need to “turn away” a lot of participants. Can the authors provide assurance that this plan is feasible?

 

9)      There is quite a bit of flexibility in the time windows that participants can complete the morning and evening sessions, which might be a further source of noise. For example, if in the PM (sleep) condition a participant completes the first test at 7pm, goes to bed at 11pm, wakes up at 8am and is tested again at 11am, there is 7 hours of wakefulness between the immediate and delayed tests. Now, if the same person in the AM (wake) condition does the first test at 11am and the second test at 7pm, there is nearly the same amount of wakefulness as there is in the PM (sleep) condition. Can the authors provide a tighter control of the timings to mitigate this issue?

 

10)   In subsidiary analyses, the authors plan to examine the sleep-memory effects at different levels of reward, with the prediction that an effect of sleep will be observed for the high rewards but not the low rewards. I recommend the use of Bayesian approaches to quantify evidence for the null for the low reward levels.

 

Whether the clarity and degree of methodological detail is sufficient to closely replicate the proposed study procedures and analysis pipeline and to prevent undisclosed flexibility in the procedures and analyses

 

 

11)   Will participants be excluded if they indicate that they have undertaken a prohibited action (e.g. napped in the wake group, consumed alcohol in the preceding 24 h)? Please make this clear.

 

12)   What will happen if the participants are continuously too slow on the flanker task? Will they be excluded?

 

13)   Are participants informed about the different levels of reward in the instructions or must they learn this adaptively? I think the former would be a better control.

 

14)   How long will encoding and retrieval take, given the large number of trials? This feeds into the attrition issue raised above.

 

15)   Why are participants asked to retrieve the reward if performance is based on the old/new decision? What is the purpose of the confidence scale? Why is the reward amount not with the boxes in the test phase?

 

16)   How will people document their wake experience every hour? Are they expected to do this independently and then type it in at the tests?

 

17)   The authors state: “To prevent p-hacking, p-values will only be calculated once a model with good convergence is identified” – how do they define “good convergence”?

 

 

Other Comments

 

Abstract

 

18)   It is not clear from the abstract whether the authors expect there to be a main effect of reward (i.e. better memory for high vs low rewards in both the sleep and wake conditions) in addition to the interaction (with the magnitude of this reward benefit being stronger in the sleep condition than the wake condition). I encourage the authors to make this clear, so that the reader can fully understand the expected direction of the results.

 

Introduction

 

19)   Some of the statements in the Introduction are quite strong and could be toned down slightly to reflect a more balanced view of the literature/current opinion. For example, “During sleep, memory traces that were encoded throughout prior wakefulness are replayed repeatedly and thereby strengthened” and “Importantly, sleep specific brain activity and especially the activity of hallmark oscillations (slow oscillations, hippocampal ripples and sleep spindles) that coordinate this replay drive greater memory performance in those tasks” refers to only one (albeit influential) theoretical perspective.

 

20)   The link between the basic scientific questions (consolidation of reward related memories) and societal issues (e.g. additive behaviours) is interesting, but the authors only briefly touch on this. I think it would be helpful if they could explain a little more clearly about why the impact of sleep on rewarded information can guide maladaptive behaviour such as unhealthy eating, smoking etc – what is the proposed mechanistic link here? Relatedly, how would targeted interventions make use of the findings of this study?

 

21)   The authors intend to use an online paradigm to study their research questions. The use of online paradigms to study sleep-memory effects has grown in recent years, with a number of studies showing a positive effect of sleep outside of the lab. I think the authors could mention this in their introduction, to highlight that the online method is an appropriate tool to measure the memory effects of sleep.

 

22)   Given that a lot of readers will not know what the AM:PM design is, I encourage the authors to elaborate on this point in the Introduction  

 

23)   What are the authors referring to by “retrieval function” – page 7 line 142?

 

24)   The authors state: “Regarding sleep, there is no consensus whether sleep enhances rewarded memories through additional dopaminergic neuromodulation during reactivation or rather dopamine sets a tag during learning that leads to enhanced reactivation without additional dopaminergic neuromodulation” – perhaps this should be removed or at least rephrased as it could give the impression that this is a question that the authors intend to address in their study.

 

Methods

 

25)   It looks like the light and dark green colours in Figure 2 represent participant sex, but this is not clear.

 

26)   The caption of figure 4 describes a 3-point Likert scale whereas the main text describes a 4-point scale.

 

27)   The authors state: We will use the maximal model to give us an indication of whether our prediction that the magnitude of decline in memory for high vs. low rewarded images will be greater after a period of wake compared to a period of sleep at delayed recognition.” - It should be clearer from the outset that the authors expect an overall decline in performance between the immediate and delayed tests, but that the magnitude of this decline will be smallest in the high reward/sleep condition.

User comments

No user comments yet