Recommendation

Mixed evidence for the hypothesis that sunk cost effects are weaker for time than money

ORCID_LOGO based on reviews by Christopher Olivola and Dilip Soman
A recommendation of:
toto

Sunk cost effects for time versus money: Replication and extensions Registered Report of Soman (2001)

Abstract

EN
AR
ES
FR
HI
JA
PT
RU
ZH-CN
Submission: posted 18 April 2023
Recommendation: posted 21 May 2023, validated 21 May 2023
Cite this recommendation as:
Chambers, C. (2023) Mixed evidence for the hypothesis that sunk cost effects are weaker for time than money. Peer Community in Registered Reports, 100452. 10.24072/pci.rr.100452

This is a stage 2 based on:

Comparing time versus money in sunk cost effects: Replication of Soman (2001)
Nikolay Petrov, Wenkai Song, Yin Kan (Megan) CHAN, Cheuk Nam (Chris) LAU, Tin Ho (Donald) KWOK, Lok Ching (Estelle) CHOW, Wai Yan (Vivian) LO, Gilad Feldman (gfeldman@hku.hk)
Scheduled submission: 2022-04-29

Recommendation

The sunk cost fallacy is a cognitive bias in which people persist with a decision that is no longer optimal because of previous resources they have invested (now considered to be spent or “sunk”). Most of us will have heard sunk costs reflected in the saying “throwing good money after bad”, but sunk costs can, in theory, occur more broadly, whether for money, time or any other resource-limited investment. The sunk cost effect for money has been widely studied and appears robust; in contrast, the sunk cost effect for time is more uncertain, and is potentially moderated by the age of respondents (and likely resource availability), the fact that time is irreplaceable, and the tendency for people to account for time less easily than they do for money. In an impactful study, Soman (2001) found that the sunk cost effect for time was indeed weaker than for money, although this finding has not been widely replicated.
 
In the current study, Petrov et al. (2023) replicated three studies (1, 2 and 5) from Soman (2001), asking whether sunk costs are weaker for time than for money, and then testing whether the relative absence of a sunk time cost arises from the inability of participants to account for time or due to more rational beliefs in the evaluation of past time investments.
 
Results provided mixed support for the original findings. Consistent with Soman (2001), the sunk cost effect for money was reliably stronger than for time in Study 1; however, sunk costs for both money and time were comparable in Study 2 (and if anything, slightly stronger for time). The indirect replication of Study 5 from Soman (2001) found that the sunk cost effect for time was not significantly influenced by accounting for time, either using education or by highlighting opportunity costs. Robustness checks confirmed the main preregistered outcomes while also ruling out a range of potential alternative explanations. Overall, the results suggest that that sunk cost effects for time are more context-dependent and empirically volatile than sunk costs for money.
 
URL to the preregistered Stage 1 protocol: https://osf.io/65htv
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. 
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Soman, D. (2001). The mental accounting of sunk time costs: Why time is not like money. Journal of Behavioral Decision Making,14, 169-185. https://doi.org/10.1002/bdm.370
 
2. Petrov, N. B., Chan, Y. K., Lau, C. N., Kwok, T. H., Chow, L. C., Lo, W. Y. V, Song W., & Feldman, G. (2023). Sunk cost effects for time versus money: Replication and extensions Registered Report of Soman (2001), acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/q9s6p
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Evaluation round #1

DOI or URL of the report: https://osf.io/wb2vy

Version of the report: 5

Author's Reply, 17 May 2023

Download author's reply Download tracked changes file

Revised manuscript:  https://osf.io/q9s6p

All revised materials uploaded to: https://osf.io/pm264/  updated manuscript under sub-directory "PCIRR Stage 2\PCIRR-S2 submission following R&R"

Decision by ORCID_LOGO, posted 14 May 2023, validated 14 May 2023

I have now received evaluations of your Stage 2 manuscript from two of the reviewers who assessed the Stage 1 manuscript. As you can see, Soman is fully satisfied with your submission in its current state, while Olivola lists several points of concern. Given the specific and limited criteria by which Stage 2 submissions are assessed at PCI RR, I want to provide some guidance on how to respond to these points.

Concerning the reviewer's point 2a (table that summarising the subset of prior studies that examined sunk costs involving time), this was not a required element at Stage 1; therefore I am not going to require this at Stage 2, although I fully appreciate the reviewer's point that it would be a useful synthesis for readers; I will therefore leave this at your discretion to consider.

Concerning point 2b, factual accuracy and clarity is vital so please ensure that this issue is fully considered and revised as necessary (even if it involves some minor changes to the Introduction).

Concerning point 3: I agree with the reviewer that this would be an interesting exploratory analysis. Since it was not part of the preregistered and approved Stage 1 proposal, and since these analyses are not (in my view) required for the conclusions to be sufficiently supported by the evidence (Stage 2 criterion 2E), it is at your discretion to include these or not. Please consider the point carefully and include in your response to the reviewer a clear explanation for your decision (one way or the other).

Concerning point 4: There seems to be a general tension with online studies between favouring participants with greater experience vs. favouring participants with a lower chance of prior exposure to key interventions; therefore I think this could be a useful point to consider in the Discussion (particularly if there is evidence that can inform the debate).

I will evaluate your revision and responses to the above points at desk before issuing a final decision.

Reviewed by , 04 May 2023

Unfortunately, the authors have (again) failed to address many of my comments and concerns:

 

Comment 2a: I still don't fully understand why the authors refuse to provide a table that summarizes the (relatively small) subset of prior studies that examined sunk-costs involving time. Doing so is not a huge endeavor (there are far fewer studies involving investments of time), yet it would provide a summary of all previous attempts to test time sunk-cost effects, and give us an idea of how replicable those effects are. I won't insist on this point, though I do think the authors are doing their paper (and readers) a disservice by limiting the scope of their paper to a replication of just a single paper examining sunk-time effects.

 

Comment 2b: The authors incorrectly claim that Olivola (2018) did not contain any sunk time cost scenarios. To the contrary, that paper contained *multiple scenarios* involving sunk time costs (e.g., the "cello lessons" and "potluck cake" scenarios), including scenarios used in papers that the authors *did* choose to cite. Therefore, there is no reason for the authors to not cite that paper as demonstrating a sunk-cost effect for time. Moreover, that paper demonstrates both intra- and inter-personal versions of time sunk-cost effects (i.e., that we also honor *other people's* time investments), which is notable. This omission by the authors is far less justifiable (and requires even less work to correct) than their decision not to include a summary table.

 

Comment 3: I’m glad that the authors have agreed to increase (specifically: double) their sample sizes. However, it seems the authors are resisting my suggestion that they *also* (not solely) carry out *additional* purely between-subjects analyses that only compare the first study/condition that participants were exposed to. The reasons for these *additional* between-subjects analyses were clearly explained in my prior review(s) and I don’t see a reason not to include them as well (in addition to the analyses the authors had in mind). The authors’ argument that “[their] experience has been that order had very little to no implications in [their] other PCIRRs and replications” is a poor one, and I can quickly counter it with my own experience, which has shown that exposure to prior studies (especially those examining the same concepts) can often impact subsequent responses.

 

Comment 4: The authors are wrong to claim that participants with 100 studies have no commitment and little reputational concerns. I’ve carried out several MTurk studies in which I limit the sample to workers with 100 studies or less (but with a high reputation score from those studies they did complete), and found that they are no less attentive than MTurkers who have completed many hundreds of studies. I won’t insist on this point, though I think the authors risk not being in a strong position to claim their study is a proper replication of Soman (2001), since his studies involved participants who were likely totally unfamiliar with sunk-cost scenarios, whereas experienced MTurk workers are likely to have been exposed to those scenarios (and debriefed about them).

Reviewed by , 28 Apr 2023

The Stage 2 paper is ready for prime time! There were a lot of intriguing results - perhaps none more so than the differences in patterns of results across replicated Study 1 and Study 2. It does appear that time is a lot more malleable that most other resources, so the call for more research on what factors might drive the evaluation of time as a resource is a good one.

Thank you to the authors for their excellent work throughout the process!

User comments

No user comments yet