How does reward influence the effect of sleep on memory?

The authors have done an impressive job responding to the reviewers comments and I believe they have appropriately addressed all the comments I made in the previous round of reviews. The positive controls in particular greatly strengthen the research plans. 

One very minor issue: The bottom right panel of Figure 3 seems to have a formatting error with text on top of other text, so I suggest correcting this. 

I think this is a very well planned study that will make a strong contribution to the literature regardless of the outcome - a great example of what a registered report should be. 

I thank the authors for their extremely thorough and concise responses to all my comments. I am satisfied that all of my prior concerns have been addressed and I am very pleased to recommend this study for stage-one acceptance. I look forward to seeing the results.  

I have no further comments apart from a typo on page 37 line 725: do the authors mean “timepoint x reward parameter” here?

Summary

Memory consolidation is supported by sleep and recent theoretical approaches have suggested that sleep might work in a selective manner to strengthen information that is most relevant or salient to the individual. However, empirical evidence related to this selective memory benefit of sleep is fairly mixed, potentially due to a diversity of experimental methods and low statistical power. In this Registered Report, Morgan and colleagues plan to carry out a timely and well powered investigation of the effects of sleep on memories associated with high and low rewards. They will use an online experiment with a gamified task to determine whether a benefit of sleep for recognition memory is amplified for information associated with high relative to low reward. My overall assessment of the manuscript is positive, but I have a number of comments and queries, which are separated according to the subheadings of the reviewer guidance notes. I have also included some more general comments at the end of my review.

Please note that I was unable to retrieve the authors supplementary materials, which were referenced several times in the main manuscript. I will need to see this in the resubmission before providing another opinion on the suitability of the manuscript for stage one acceptance.

The scientific validity of the research question(s)

1)      From the overview of the literature in the Introduction, it appears that a potential source of inconsistency in previous findings relates to the memory system in question (e.g. declarative vs procedural memory). Is it the case that studies showing a positive effect of sleep on high vs low reward memories are in the procedural domain? Can the authors be sure that a declarative memory protocol is the optimal approach for studying effects of sleep and reward on memory consolidation?

2)      A more general point relates to the issue of whether reward actually has any impact of consolidation, irrespective of whether it occurs over sleep or wakefulness. In their Introduction, the authors state: “Either sleep selectively consolidates information associated with high rewards or reward related processes during encoding together with sleep-independent consolidation processes initiated shortly after learning are sufficient to enhance reward memory”. A third possibility is that the benefits of reward occur only at encoding and are not enhanced at all by consolidation. Can the authors rule this out?

The logic, rationale, and plausibility of the proposed hypotheses (where a submission proposes hypotheses)

3)      The use of a recognition paradigm raises questions about the plausibility of the authors proposed hypotheses. A number of previous studies have shown that recognition paradigms are fairly insensitive to sleep-memory effects, and this lack of sensitivity could pose problems for the authors predictions. Relatedly, H1 focuses on performance in the delayed test, whereas sleep-memory effects are typically shown via the change in performance between immediate and delayed tests.  

The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis or alternative sampling plans where applicable)

4)      One of my foremost concerns relates to the authors plan to collect a large amount of data for several exploratory analyses, seemingly at the cost of the main research question. In particular, the authors will not exclude participants on the basis of an existing mental health condition, sleep disorder or use of medication (among various other factors that are known to affect sleep) so that they can carry out exploratory analyses of how their findings are influenced by these factors. In my opinion, they should design their experiment in such a way that it is optimised for their main research goal (to test the selective effect of sleep (vs wake) on high (vs low) reward memories) and then carry out exploratory analyses where possible. Accordingly, they should extend their exclusion criteria to other factors that are known to affect sleep, so that they can maximise experimental control. Along the same lines, given the known effect of age on sleep architecture (such as slow-wave sleep, which has been linked to the preferential consolidation of salient information), I recommend that the authors restrict their sample to a narrower age range (e.g. young adults). This, again, will optimise their ability to test the main research question and provide a strong platform for future research in which the effects of age, mental health condition etc, can be studied in the context of the selective memory function of sleep.

5)      Along the same lines as the above comment, given that encoding strength is known to influence the effects of sleep on memory consolidation, keeping encoding strength consistent at learning would presumably be of benefit to the main research question.

6)      Related to the above point, I question how the authors can separate the effects of reward from encoding strength at learning? Presumably high reward items will be encoded more strongly at the learning phase than low reward items, which would influence their consolidation in sleep. How do the authors intend to control for this?

7)      At several points in the manuscript the authors say that data will be collected for exploratory analysis, but they do not provide much insight on the nature of the questions they intend to address. For example, it would be interesting to know the authors’ motivations for collecting data on participants’ experiences during the day, and the question(s) they intend to address in this exploratory analysis.

8)      The plan to use a within subjects is statistically optimal, but I am quite concerned about potential attrition rates, given the large number of questionnaires to complete at the recruitment stage, the lack of guaranteed financial compensation, the long delay between completing the sleep and wake conditions, and the delay between the initial recruitment screening and the first main experimental session (minimum of 24 h). The proposed use of a stratified sample could compound this even further, as the authors will need to “turn away” a lot of participants. Can the authors provide assurance that this plan is feasible?

9)      There is quite a bit of flexibility in the time windows that participants can complete the morning and evening sessions, which might be a further source of noise. For example, if in the PM (sleep) condition a participant completes the first test at 7pm, goes to bed at 11pm, wakes up at 8am and is tested again at 11am, there is 7 hours of wakefulness between the immediate and delayed tests. Now, if the same person in the AM (wake) condition does the first test at 11am and the second test at 7pm, there is nearly the same amount of wakefulness as there is in the PM (sleep) condition. Can the authors provide a tighter control of the timings to mitigate this issue?

10)   In subsidiary analyses, the authors plan to examine the sleep-memory effects at different levels of reward, with the prediction that an effect of sleep will be observed for the high rewards but not the low rewards. I recommend the use of Bayesian approaches to quantify evidence for the null for the low reward levels.

Whether the clarity and degree of methodological detail is sufficient to closely replicate the proposed study procedures and analysis pipeline and to prevent undisclosed flexibility in the procedures and analyses

11)   Will participants be excluded if they indicate that they have undertaken a prohibited action (e.g. napped in the wake group, consumed alcohol in the preceding 24 h)? Please make this clear.

12)   What will happen if the participants are continuously too slow on the flanker task? Will they be excluded?

13)   Are participants informed about the different levels of reward in the instructions or must they learn this adaptively? I think the former would be a better control.

14)   How long will encoding and retrieval take, given the large number of trials? This feeds into the attrition issue raised above.

15)   Why are participants asked to retrieve the reward if performance is based on the old/new decision? What is the purpose of the confidence scale? Why is the reward amount not with the boxes in the test phase?

16)   How will people document their wake experience every hour? Are they expected to do this independently and then type it in at the tests?

17)   The authors state: “To prevent p-hacking, p-values will only be calculated once a model with good convergence is identified” – how do they define “good convergence”?

Other Comments

Abstract

18)   It is not clear from the abstract whether the authors expect there to be a main effect of reward (i.e. better memory for high vs low rewards in both the sleep and wake conditions) in addition to the interaction (with the magnitude of this reward benefit being stronger in the sleep condition than the wake condition). I encourage the authors to make this clear, so that the reader can fully understand the expected direction of the results.

Introduction

19)   Some of the statements in the Introduction are quite strong and could be toned down slightly to reflect a more balanced view of the literature/current opinion. For example, “During sleep, memory traces that were encoded throughout prior wakefulness are replayed repeatedly and thereby strengthened” and “Importantly, sleep specific brain activity and especially the activity of hallmark oscillations (slow oscillations, hippocampal ripples and sleep spindles) that coordinate this replay drive greater memory performance in those tasks” refers to only one (albeit influential) theoretical perspective.

20)   The link between the basic scientific questions (consolidation of reward related memories) and societal issues (e.g. additive behaviours) is interesting, but the authors only briefly touch on this. I think it would be helpful if they could explain a little more clearly about why the impact of sleep on rewarded information can guide maladaptive behaviour such as unhealthy eating, smoking etc – what is the proposed mechanistic link here? Relatedly, how would targeted interventions make use of the findings of this study?

21)   The authors intend to use an online paradigm to study their research questions. The use of online paradigms to study sleep-memory effects has grown in recent years, with a number of studies showing a positive effect of sleep outside of the lab. I think the authors could mention this in their introduction, to highlight that the online method is an appropriate tool to measure the memory effects of sleep.

22)   Given that a lot of readers will not know what the AM:PM design is, I encourage the authors to elaborate on this point in the Introduction  

23)   What are the authors referring to by “retrieval function” – page 7 line 142?

24)   The authors state: “Regarding sleep, there is no consensus whether sleep enhances rewarded memories through additional dopaminergic neuromodulation during reactivation or rather dopamine sets a tag during learning that leads to enhanced reactivation without additional dopaminergic neuromodulation” – perhaps this should be removed or at least rephrased as it could give the impression that this is a question that the authors intend to address in their study.

Methods

25)   It looks like the light and dark green colours in Figure 2 represent participant sex, but this is not clear.

26)   The caption of figure 4 describes a 3-point Likert scale whereas the main text describes a 4-point scale.

27)   The authors state: We will use the maximal model to give us an indication of whether our prediction that the magnitude of decline in memory for high vs. low rewarded images will be greater after a period of wake compared to a period of sleep at delayed recognition.” - It should be clearer from the outset that the authors expect an overall decline in performance between the immediate and delayed tests, but that the magnitude of this decline will be smallest in the high reward/sleep condition.