Thank you for the invitation to re-review “Does retrieval practice protect memory against stress? A meta-analysis [Stage 1 Registered Report]”. My express philosophy for re-reviews is that I look at my comments from the previous review, and see whether they are addressed (as is an identified best practice; 10.1101/2022.12.20.521261).

Overall, the authors' responses are satisfactory and leave little more to be desired. It is clear they took the comments seriously and made good faith efforts in revising the stage 1 report. 

My only remaining comment, for a future version, is regarding the publication bias tests. As indicated in my original review, the previously proposed methods are also sensitive to heterogeneity:

> "There seems to be an oversight here, because the included publication bias tests are also sensitive to heterogeneity. I include references below from which that follows"

The authors did not alter their methods substantively from what I gather, as the original publication bias section remains largely unchanged, save a few additions. I would be remiss to not point this out in an otherwise fantastic revision. I strongly encourage the authors to revisit this as previously indicated - at the very least further discuss this in detail in the limitations if there is heterogeneity in the actual data.

All in all, I am grateful to see all points (minus 1) addressed and can recommend the manuscript to move forward. Thank you!

This is the second round of review regarding the meta-analysis “Does retrieval practice protect memory against stress?”

I will only provide some proofreading advice, since the authors resolved all of the comments I raised.

page 9 Retrival Practice main effect, the m and e of main effect could be capitalized.

Page 9 first section, all sentences begin by “this” (but also last sentence page 8). Reading could be improved by a rephrasing.

Page 14: 

The date last searched was _____. At Stage 2, we reran the searches at least twice to ensure all literature was up-to-date. The date last searched was ___. The outcome was a total of YY prospective articles. After deletion of doubles, we had a total of XX articles (Figure 1).

I don’t understand this sentence. You should conduct the search between stage 1 and stage 2 so why explicitly say that you will conduct the search at stage 2?

Also I think it is better to say duplicates than doubles. You also began two following sentences by “after” and could find a synonym.

This enabled us to ensure full coverage : you can delete the “enabled us to”

I may propose you to rephrase “study leader” to “main contributor” in the text.

Page 24 “we first illustrated” seems odd. You “display a funnel plot to check the existence of a publication bias.” Illustrated a publication biais would more mean that you draw a picture of a publication bias.

Besides these minors fix, I don’t see any problems and would be happy to see the results of the meta-analysis. I wish the best for the authors in this process.

Thank you for the kind invitation to review “Does retrieval practice protect memory against stress? A meta-analysis [Stage 1 Registered Report]”. It is my first time reviewing for PCI RR, so please let me know if this fits your expectations.

My philosophy for reviews is:

• To help build up the authors, not to tear them down.
• Provide honest feedback that the authors can decide what to do with.

Overall, I think this meta-analysis is set up in a meticulous way, and provides a great basis for a study with important findings regardless of outcomes. My review highlights areas that went well, and those that could still improve.

1. What went well

1. I am not a memory expert, and the introduction was extremely well written. It really helped set the stage for me - thank you! 🙌
2. Your predictions are clearly articulated and solid. Makes me very happy to see good predictions 😊
3. The commitment to documenting all conversions and coding decisions is really helpful. When I did this it taught me a lot and served as a helpful reference, so I was pleased to see it here too.
4. I am very glad to see risk of bias judgments included - thank you!

2. What could have gone better

1. The OSF project has not been registered and can be amended at any time (as of right now). Preregistration is not complete without explicit registration (until it is assigned a DOI is a decent heuristic here). Could you explain what I missed here?
2. Database search
   1. You mention you use MeSH terms, but these are for medical subject headings. I do not understand which would apply here. It would also help to explain to the reader what these are as these are not necessarily part of the common language. Could you maybe add a bit more of an explanation here?
   2. Could you explicitly mention the databases you use for what you call the grey literature search?
   3. Table 1 indicates “Variation of the above” for JSTOR and OSF preprints. If at all possible, could you be explicit about the search terms at this stage (or say why they can’t be explicated as such)?
3. Results
   1. It is confusing that results are included already. I am going to assume for now that this is just a placeholder, but can the authors confirm this? The manuscript does not say whether these are simulated data so it is confusing while reviewing.
   2. If I understand correctly, in the section “Statistical Power”, you seem to be hinting at post-hoc power, which is not a fruitful measure. Could you expand on whether this understanding is correct? Post-hoc power means that you’ll have high power if your meta-analysis finds an effect, and low power if there is none. The only statistical power you can calculate is a priori.
   3. Moderator analyses
      1. Stressor type - could you please add the statistics of the moderation effect results into the text (or where to find that information)?
      2. Same issue for the other learning strategies - mentioned results are best to be included in either the text directly or in the supporting materials (as you do so well for the rest!).
   4. You mention several tests for publication bias, and subsequently eliminate others because they wouldn’t work well in heterogeneous settings (e.g., p-uniform). There seems to be an oversight here, because the included publication bias tests are also sensitive to heterogeneity. I include references below from which that follows, primarily because these methods test/correct for small-study effects which can also be caused by a relation between heterogeneity and the size of the studies.
      1. Egger’s test: Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634.https://doi.org/10.1136/bmj.315.7109.629
      2. Trim + Fill: Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22(13), 2113–2126. https://doi.org/10.1002/sim.1461
      3. PET-PEESE: Stanley, T. D. (2017). Limitations of PET-PEESE and other meta-analysis methods. Social Psychological and Personality Science, 8(5), 581–591. https://doi.org/10.1177/1948550617693062
   5. A very minor point: It would help if you could label the results in relation to the predictions you made, to clarify which result relates to which (H1, H2, H3).

3. What to watch out for

1. You use the PsycNET, PubMed (Central?) and JSTOR databases. Did you look into any other databases? From my own experience with meta-analyses in psychology, you might risk losing coverage by restricting to these databases only. I understand that many databases are paywalled (e.g., Scopus, Web of Science) and it may not always be a willing choice, so any additional information on the choice would be helpful.
2. You call preprints grey literature; according to my understanding grey literature are e.g., blogs. Preprints are still part of the scholarly record. This is of course an ambiguous concept so it is up for the authors to decide what to do.
3. You choose random effects for the main effects, but fixed effects for the moderator analysis. Could you expand as to why? This seems rather arbitrary considering you just argued for including a distribution of effects a few paragraphs before (specifically referring to p25, first paragraph).

Summary

In my review, I found this manuscript to be succinct, to the point, and clear. I think there are no fundamental flaws or things that cannot be resolved by the authors. Thank you for doing this important work!