Thank you very much for this excellent revision that addressed all points that I mentioned. I have just one major concern left: In the revised version, it becomes clear that the recruiting strategy is different for the experimental group and the control group. “The experimental group will be recruited to participate in six hour meetings on climate education and fill online questionnaires, while the control group only to complete the questionnaires due to the difficulty of recruiting all 120 participants for the in-person meetings.” This would make it very difficult to infer causal effects from the study. Every difference between the two conditions could be attributed to the different recruiting strategy and the different motivation that participants had from the start. The authors describe that they would use initial motivation as a control variable in their analyses. However, this only partly addresses the problem. The study requires a lot of time and effort. It would be a pity if causal inferences would be severely restricted by the different sampling strategies. I would like to motivate the authors to consider a waitlist control group that receives the treatment later than the experimental group. With this design all participants would be recruited with the same information. I’m aware that it is more difficult to recruit participants for the in-person meetings than to just fill in online questionnaires. However, you planned five different studies with the above mentioned design. I would suggest to reduce the number of studies but to choose the more robust design.

The authors have addressed my comments (and also those of the other reviewers) thoroughly and appropriately. Especially the idea of the coding list I find to be a valuable addition! I appreciate this work for the numerous additions and endorse the publication of the registered report. I am eager to see the results and wish the researchers success in recruiting participants and carrying out their project! 

The registered report „A climate action intervention to boost individual and collective climate mitigation behaviors in young adults” proposes a set of five intervention studies with control groups and three measurement time points. I highly appreciate the attempt to investigate the causal effects of pro-environmental interventions in longitudinal studies. The design allows for pre-post comparisons, comparisons with a control group and the investigation of effects three months after the intervention. It is also fortunate that the studies will take place in different countries (Italy and the Netherlands). The study plan requires much effort and time. Such intervention studies are highly practically relevant but so far rare.

In the summary of the psychological obstacles of pro-environmental behavior I missed the point that most pro-environmental behaviors are embedded in a structure of a social dilemma (the behavior that provides the largest short term benefits for the individual is different from the behavior that benefits the collective in the long run; see Claessens et al., 2022; Steg et al., 2014).

As correlates of individual and collective pro-environmental behavior, you mention negative emotions, individual and collective efficacy, environmental identity, social norms, and cognitive alternatives. You may want to consider the role of positive emotions and activist identity in addition (see Landmann & Neumann, 2023).

The distribution of studies in Italy and the Netherlands is a bit one-sided. Only one study is planned in the Netherlands, the other four in Italy. I could not find an argument for why this isn’t more balanced.

I would prefer more information about the expected drop out rate - do you expect drop out only for the experimental group, not for the control group? I think you should calculate some drop outs in the control condition as well.

Please clarify whether participants are randomly assigned to the experimental or the control condition or whether they decide themselves if they want to participate in the intervention. You write that participants will receive a certificate for participating in the intervention. What about those in the control group? Do they have the same incentive for participation (including the certificate) or is the appeal for participation for those in the control group different?

I disagree that the comparison with the control group completely cancels the possible effect of social desirability. The perceived expectation of reporting pro-environmental behavior might be higher in the experimental condition, in which participants are repeatedly confronted with environmental issues. However, assessing pro-environmental behavior by self-report is still suitable for the planned studies. If people are asked very specific questions about their behavior as in the EMA, it is difficult to lie. Thus, I think the self-report measures of pro-environmental behavior are fine, I just suggest to discuss its limitations differently.

The mixed effects linear model seems suitable to test H1. I wondered why you chose a different analyses for testing H2 (repeated measures ANOVA) – mixed effects would be also suitable here.

For testing whether the effects of the intervention differs between the studies (H4) you need to add condition as predictor as well as its interactions with study ID. This would make your planned regression analyses very complex. It may therefore be more convenient to test H4 with mixed effects models including condition and study ID as predictors.

I’m already looking forward to seeing the results!

Claessens, S., Kelly, D., Sibley, C. G., Chaudhuri, A., & Atkinson, Q. D. (2022). Cooperative phenotype predicts climate change belief and pro-environmental behaviour. Scientific Reports, 12(1), 12730. https://www.nature.com/articles/s41598-022-16937-2.pdf

Landmann, H., & Naumann, J. (2023). Being positively moved by climate protest predicts peaceful collective action. Global Environmental Psychology. https://doi.org/10.23668/psycharchives.13186

Steg, L., Bolderdijk, J. W., Keizer, K., & Perlaviciute, G. (2014). An integrated framework for encouraging pro-environmental behaviour: The role of values, situational factors and goals. Journal of Environmental Psychology, 38, 104-115. https://doi.org/10.1016/j.jenvp.2014.01.002

Dear Anna and greetings to the entire research team,

I had the pleasure of reviewing your registered report in which you present a comprehensive long-term study involving approximately 50 participants undergoing an intervention, with effects compared to a control group of around 50 individuals. I believe the described approach is a valuable initiative to confront interested (yet not actively engaged) students and other young individuals with climate change, providing them with motivation for action. The theory-guided intervention based on the 12 factors is particularly promising. Additionally, the ideas for analyses considering interaction effects over time involving group, time, and person (as a random effect) levels are sensible.

While reading, some concerns and ideas arose, which I will describe below:

·         If I understand correctly, the control group is keeping diaries (EMS) for a total of 12 weeks (3 months). A 30% drop-out rate seems optimistic, especially given that there is a 3/100 chance (roughly) of winning €150 at the end. It might be worthwhile to research dropout rates from other environmentally focused diary methods, and adapt this rate, if necessary.

·         EMS has a significant drawback not yet mentioned: participants can mention anything, which, while acknowledged as an advantage, may lead to large subjective distortions when only frequency is queried. Personal biases (personality, attitudes, etc.) might not be reflected, potentially obscuring the results. For example, we conducted a study in which environmentally less conscious individuals reported partly mundane activities (e.g., using a lid while cooking to save energy), while others understood more elaborate behaviors (e.g., refraining from a flight and opting for a several-hour train journey): https://doi.org/10.3389/fpsyg.2022.883704. If both are equally weighted in terms of frequency, personal biases (personality, attitude, etc.) are not reflected, which could obscure the results. Honestly, I'm not sure how you could address this issue, but I wanted to inform you that such distortion could occur in EMS.

·         The effort required for participants seems substantial. Likely, only individuals already living environmentally friendly lives will persist, raising questions about the variance left for improving their behaviors – as you accordingly suggested in the report. Considering the anticipated ceiling effects and drop-out rates, I personally find the sampling size to be a bit small. Additionally, the power analysis mentions 14 replicates, while a minimum of 4 replicates is discussed on page 15. Should this number be reduced in the sampling accordingly?

·         It would be great to learn more about the sampling of the study you described on pages 2-3 (e.g., were the participants also young, and how many were there?).

·         Participants might be aware of what you are investigating, leading to biased responses in line with the expectation that the sessions should have an effect. How do you plan to account for this bias?

·         The focus on negative affects regarding climate change (anxiety) is noted. It would be beneficial if you could elaborate on why positive emotions like hope are excluded. Another crucial negative emotion seems to be anger, which doesn't paralyze but activates (see: https://doi.org/10.1016/j.joclim.2021.100003).

·         One last thought: considering your assumption that planning on one-day influences implementation on another, incorporating AR1 models might be beneficial.

Best regards and good luck with your research! I am looking forward to follow this research progress.

Jana

Thank you for the opportunity to review this programmatic registered report. The goals of this work are timely and important.

Major feedback:

The overarching hypotheses are sensible but not specific. I understand that this may be challenging given the iterative nature of the design but given the larger number of outcomes and psychological factors, it would be helpful to know more about: 1) how hypotheses will be refined over studies (e.g., if some outcomes but not others are altered in study 1, will this prior information be incorporated in some way in study 2?) and 2) how the false positive rate will be minimized across outcomes and moderators. 

This program of research has an implicit causal model—in which the intervention should increase the targeted psychological mechanisms, which in turn should change behavior—so it is unclear why methods for testing causal pathways are not actually proposed. In its present form, H1 maps onto the c path (X --> Y), H2 maps onto a path (X --> M), and H3 contains a combination of the b path (M --> Y) and the indirect path (X --> M --> Y). Rather than testing these hypotheses in separate models, it would be more parsimonious and informative (i.e., to actually test the implied indirect paths) to test these hypotheses in a single model path model. There are various methods to do this (see e.g. https://abcdworkshop.github.io/slides/ABCDSEMTheory_2021.pdf) and extensions that could be used for the EMA data through Bayesian multilevel modeling (in the brms package; see e.g. https://m-clark.github.io/models-by-example/bayesian-mixed-mediation.html). If a Bayesian framework were employed, this would also have the advantage of enabling the priors to be updated from one study to the next based on the results.

Regarding sample size and power, I am concerned that the estimated effect sizes for H2 and H3 are not realistic and will result in these tests being underpowered. Recent studies have shown that average published effects tend to be much higher than replication or preregistered effects (https://journals.sagepub.com/doi/full/10.1177/1745691612462588 ; https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2019.00813/full). In the former study for example, preregistered between-person experiments had a median effect size of just r = .12 versus r = .34 for non-preregistered studies. If power is not increased, this could also negatively affect the iterative nature of the research program. That is, observing a false negative might prompt changes to the intervention that are unnecessary or detrimental, thus affecting subsequent studies. One approach for overcoming this issue would be to power study 1 to detect smaller effect sizes (similar to H1) to get a realistic sense of the effect size for this intervention, and then power subsequent studies accordingly.

Regarding attrition, the authors state that they expect ~30% attrition, but as far as I could tell the target sample sizes do not include additional participants to make up for this. Given the very high level of engagement required for this study (and relatively low compensation), it seems likely that attrition will be high. It may also be wise to preemptively plan for even greater attrition in study 1 and then adjust as needed in subsequent studies.   

Another concern is the combination of iterative improvement and assessing generalizability different developmental and cultural samples. Each of these goals are useful, but when used in combination, they may make the result uninterpretable. That is, the authors expect that each iteration will improve the efficacy of the intervention (H4), but that may not be the case for more distal populations from the ones it was developed on. Thus, the final iteration of the intervention may not be the best because it simply works less well on high school students than it does for university students. Similarly, making changes in study 4 based on how well it performs in young professionals may result in a worse intervention for high school students who are in a different phase of life and have different affordances. An alternative approach would frame studies 3-5 as tests of generalizability and not use iterative improvement. 

Relatedly, the current design does not include many individual difference measures or have any planned moderation analyses. Although the hypotheses are centered around average effects between groups, there will likely be a lot of heterogeneity in how effective the interventions are for individuals. Including a broader array of individual difference measures may help contextualize for whom the intervention is effective (and explain attrition) and generate moderation hypotheses that could be tested in subsequent studies.  

Other suggestions that may enhance this work:

•  Include measures of hope and resilience/well-being in addition to climate anxiety

Other questions that arose while reviewing:

• The modules have solid coverage in topics but it is noteworthy that there is no focus on developing strategies for resilience and hope amidst the climate crisis. This seems to be an essential ingredient that buffers and enables people to continue to engage in climate action, particularly as they learn more and may feel anxiety, grief, hopelessness etc (see e.g. https://pubmed.ncbi.nlm.nih.gov/36502586/). 
• How many people will be in each intervention module group? If there are multiple groups, how will intervention fidelity of the module be tested (e.g. between different leaders)?
• What happens if the intervention does not affect the psychological targets or change the behaviors, and it needs to be drastically overhauled? The current model suggests small incremental improvements but in the absence of pilot data testing the intervention modules (e.g. with manipulation checks to ensure they’re targeting the processes intended), it seems possible that deeper changes may need to be made. 
• The current set of collective actions focuses primarily on traditional political targets, but what about collective actions in an individual’s more local sphere (e.g., organizing to change the food served at the school cafeteria to reduce climate impact, working with neighbors to plant trees)? These may be more accessible and potentially increase self-efficacy since the effects of their actions are more proximal.
• What happens if individuals do not plan any actions? Will their percentage scores be treat as NAs or 0s? And what happens if individuals engaged in unplanned actions? Will they be able to report those and will they be counted toward their action scores? 
• What is the justification for the exclusion thresholds listed? If possible, using methods that include all available data but adjust for differences between estimates of individuals with more versus less data (e.g. shrinkage in multilevel modeling) may reduce bias in the estimates.