Grateful or indebted? Revisiting the role of helper intention in gratitude and indebtedness

Dear authors and recommender, 

I have read both the authors' response to reviews and the revised manuscript. I am pleased to see that much work has been done to improve the proposed study protocol, and I am happy with how the authors responded to my comments. I am satisfied that a study based on this protocol would be of sound methodological quality, and would provide enough detail to be reproducible by other parties. I look forward to seeing the study when it is complete. 

Signed, Sarahanne M. Field

I appreciate the authors' responses to the reviews, they helped clarify some confusion and concerns I had with the research. Although I am hesitant about the within-subjects nature of the registered report study, as long as the authors collect enough data to examine order effects (which it appears they will), then I have no problem with it. Below are some small suggestions to further improve the research. 

- on p. 17 of the marked document, item #9, it might be clearer if, rather than using the label "selfish intentions" the authors used "benevolent motivations" so that it reads "Ratings of benevolent motivations are associated with gratitude . . ."  This suggestion also applies to item 10, and item 9 + 10. 

- related to the authors' proposed extension looking at indebtedness and gratitude on reciprocity, Nelson et al 2023 also looked at gratitude, indebtedness, and prosociality (https://doi.org/10.1080/17439760.2023.2190926)

- around p. 20, the authors report sensitivity and power analyses. The authors may want to explicitly state that the number of participants they plan on recruiting are more than enough to provide enough power for moderation analyses looking at order effects on the presentation of Study 2 vs. Study 3. 

- on p. 26, on the item "Study 2 and 3 are conducted seperately", many of the points raised in the "Reasons for change" column are unrelated to reasons for changing the original study to within-subjects. For example, the reasons "To reduce the order effect" and "to avoid the influence of decline to particular studies" are not the reasons why the authors moved to a within-subject design (although they are reasons for counterbalancing). The only reason in that section that explains a change from the original study is the statement, "to find potential consistency within participants’ answers (whether an answer is predictive of another answer". The other reasons, while good reasons for counterbalancing, are not relevant to the change from the between Ss nature of the original studies, and the within-Ss nature of the proposed study.

- on p. 37 on the table for the classification of replication, would it be appropriate to include in procedural details the fact that participants will see both Study 2 and Study 3 stimuli in the new research?

- on p. 46 the title for Figure 1 needs to be fixed.

- on p. 47 the title for Figure 2 needs to be fixed.

- In the authors' response letter, the third comprehension check for Study 3 appears to have incorrect answers. "a" (I know without doubt it is because my friend wanted to borrow my car.) should be selfish, "b" (It is not clear about the two being related, but the weekend after helping me this friend asked to borrow my car) sho- on uld be ambiguous, and "c" (My friend is really concerned about me​) should be benevolent. 

I look forward to seeing the authors' completed research. 

Thank the authors very much for clearly addressing my concerns in detail, which is impressive. I think the current manuscript is much more improved. I still have one small concern about the extension with Watkins et al (2006). I wonder whether perceived expectation for reciprocity is also a manipulation check rather than a DV. In Study 3 in the Selfish condition, the intention seems out of expectation for reciprocity. And in the vignette it actually has explicitly mentioned the higher expectation for reciprocity: “in order to borrow your car next weekend”.
 
Besides, some minor points. I saw a list of α in the Measures. What does α mean here? If they are Cronbach’s α, they seem incorrect (all below 0.1). If they are not, please specify. And Ames et al. (2004) was listed in the text (p.10) but not in the reference lists.
 
Good luck with your research!

I am excited to see this replication and extension of work on intention, gratitude, and indebtedness. I am glad that the authors are recruiting a bigger sample size, adding more manipulation checks, and measuring the additional dependent variable of reciprocity intentions. Below are some suggestions and questions that I have which I hope will helpful to the researchers in conducting their study. 

- I may have missed it, but I don't think the authors specified why they were replicating Studies 2 & 3, but not Study 1. When reading Tsang (2006), it is obvious that one would try and replicate Study 3 rather than Study 1, but readers might not be familiar with the methods of the original studies and therefore might miss this. 

- The hypotheses in Table 1 (p. 13) that were reworded from the null (1c, 3, 5, 8a, 8b, 8c, 10) were confusing to me. I understand the need to reframe the original null predictions, but the reframed hypotheses were making predictions that the original paper did not make. The combined hypotheses made more sense to me as a reframing. I'm not sure if there is a way to clarify this.

- at the bottom of page 15, the authors theorize about the relationship between intention and reciprocity, but then on p. 16 make predictions about gratitude and indebtedness. this was a little unclear to me; I was expecting the predictions to be about intention and reciprocity given the theorizing. perhaps there is more the authors can say about theory related reciprocity and gratitude/indebtedness before they get to those predictions that will make the argument a little more clear. 

- Method: I see the rationale for running Study 2 & 3 within-subjects, but I am worried that this may subtly influence the results. For example, if a participant is assigned to a benevolent intention condition in Study 2, but then is assigned to a selfish condition for Study 3, this may introduce a contrast effect--selfish favors may seem more selfish after writing about a benevolent favor. Participants who are asked to read a textbook scenario first, might then be influenced by this scenario when they recall their own received favors in the subsequent study. Additionally, gratitude or indebtedness may be primed in ways not primed by the original studies if the studies are run together. Thus, running these studies together might compromise the closeness of the replication. 

- oddly, given my previous comment, I am also worried about the directness of the replication of the scenario study, in that the population from which the authors are recruiting are not all students, and also given the passage of time from 2006 until now. Specifically, the scenario from Tsang (2006) was designed to be relevant to the student population from which the participants were recruited. However, the current authors plan to recruit from a broader population, and it is likely that students will be in the minority in their participant pool. Thus, the scenario may be less relevant to their participants, and induce less gratitude and indebtedness. The amounts used in the original scenario also mean something different today. For instance, $200--the amount lent to the protagonist in the original study--would be worth almost $300 today. Thus, using the same exact scenario today as Tsang used in 2006 would lead to participants reading about a less valuable favor. 

- it makes sense to measure reciprocation intentions as an extension. The authors might look at Peng et al. (2020) to inform predictions regarding gratitude, indebtedness, and reciprocation. 

The current paper intended to offer a replication and extension of Tsang (2006) investigating the effect of helper intention on gratitude and indebtedness. Tsang (2006) suggested that perceived benevolent intention would trigger higher gratitude but would not affect indebtedness. Tsang (2006) is indeed a pioneer work and inspired many later researchers to differentiate gratitude and indebtedness. However, there are severe problems of the current manuscript that blocks me from recommending to proceed to stage 2.
 
First, I think the current literature review on gratitude and indebtedness is very limited and far from comprehensive. There are accumulated literature suggesting the relation-oriented function of gratitude to promote intimate bonds (Algoe, 2012; Algoe et al., 2013; Bartlett et al., 2012; Gordon et al., 2012; Kubacka et al., 2011; Lambert et al., 2010; Ng et al., 2017; Peng et al., 2018; Williams & Bartlett, 2015), which is important to help clarify why benevolent intention is important to trigger gratitude. There are also accumulated literature suggesting the exchange-oriented function of indebtedness (Adams & Miller, 2022; Goyal et al., 2022; Naito & Sakata, 2010; Peng et al., 2018), which is important to clarify why beneveolent intention is NOT associated with indebtedness. Moreover, the current manuscript give me the impression that the authors lack to provide a systematic review of the literature and clear arguments but are listing literature loosely.
 
Second, the manuscript writing lacks basic scientific rigor. There are many literature presented in the introduction but not listed in the reference, and I could not find them either on google scholar to judge their validity (e.g., Gray et al., 2001 on p7; Ortony et al., 1988 and Mathews & Green, 2010 on p8; Maureen & Jeffrey, 2009 on p9; Ames et al., 2004 and Welsh et al., 2021 on p10). Meanwhile, in many cases, the authors fail to provide reference for certain claims (especially when the claims are big), making it difficult for me to make sense of it. Some examples are: The starting sentence in background on p7: “Gratitude and indebtedness are common emotions in response to receiving help. But studies suggested that they are experienced differently depending on situation”. And on p8 line 5, “These two emotions have often been equated in the early literature, yet evidence showing that these emotions are elicited in different situations suggested the need to differentiate them.”).  
 
Third, the current replication is making things too complicated to be qualified as a replication. I wonder why the authors considered to combine two separate studies in Tsang (2006) into one study rather than replicating them separately. This is not a replication anymore, as the design in either study may affect the other. For whatever results in the end, it would be very hard to interpret and compare with the original study. Let alone the authors are extending it to mix with the design of Watkins et al. (2006), making it even further from a replication. I think a good replication design should be as close and comparable to the original study as possible.

Dear authors, 

Thank you for giving me the opportunity to review your work. My apologies for the lateness of my review. I hope my comments are nevertheless helpful. 

I am generally very positive about this Stage 1.

- The replication protocol is very clearly set out, and it appears to me as though, providing the replication study is conducted closely to how it has been described here, the replication has a very good chance of reinforcing the effects in question, should they 'exist'. Importantly, I think this replication study protocol leaves little room for flexibility or bias, which is important for meaningful and high-quality replication studies. 

- The theoretical background is clear and follows logically, and motivates the study sufficiently. The target sample size is motivated also, and seems reasonable.

- The planned statistical approach seems appropriate to me (although I will freely admit that I am no expert on these kinds of analyses). 

- Although I typically suggest using Bayesian methods to compare replication targets (as they allow one to quantify pro-null evidence), I am interested to see how the authors use the LeBel method for this study. 

- Manipulation checks and controls seem sufficient for purpose, from what I can tell. All in all, I look forward to what the results show and whether Tsang's original findings are supported. 

I have only three tiny quibbles (in no particular order):

1. This article, while generally clearly written and free of obvious typing errors, is plagued by very tiny paragraphs. I'm not a nit-picker usually, but these paragraphs are so small as to be distracting and sometimes make the reading more difficult than it should be. Particular examples of where paragraphs could be merged are on pg 11 ("We chose..." might be merged with "The article has...") and pg 12 ("Tsang (2006) examined..." might be merged with "We focused our...") and so on. This isn't a huge deal-breaker, but readability would be improved, in my opinion, if the structure of the article were revised with this in mind. 

2. I find some of the figures a little hard to read. Violin plots are great and the figures generally look very good, however the raw data points on some of them are quite large and very transparent, which makes the distributions hard to see underneath the boxes. Figures 3, 4 and 8 for instance are great - the data points in those are smaller and you can clearly see the way the data are distributed, however Figures 5 and 6 (etc) are harder to make out. Not a huge issue, but given that the data in some of the figures seems quite evenly distributed, it's harder to see the distributions' nuances. 

3. The figures are numbered weirdly, unless I'm missing something. It appears as though there are two Figure 8s? 

As I mentioned above, this replication study plan looks very well thought-out to me, and will be a reasonable 'test' of Tsang's Study 2 and 3 (to the extent that a single replication can be, of course!). I wish the authors luck with the data collection, should they get the go-ahead!

I sign every review I undertake,

Sarahanne M. Field