Are loneliness and free will beliefs associated with anthropomorphism?

I am fine with the changes the authors propose except one.

Their respectful diagreement about running a supplementary analysis removing the 'free will' item form the anthropomorphism scale. I, likewise, disagree (respectfully) with their response. It is not just a possibility of autocorrelation, but people who believe strongly in free will may be likely to give high ratings when asked about free will in general (a common method bias).

All I ask is a supplementary analysis removing that one item. I would be happy even if the authors gave it a caveat such as: "At the insistance of one reviewer, we run a completely exploratory and supplementary analysis where we eliminated the free will item from this anthropomorphism scale, for this lone reviewer's concern. This additional analysis shows..." and then fill in the rest.

Otherwise, I look forward to seeing the results of this manuscript.

Best;

Protzko

The authors propose to replicate a correlational study, and extend it with some new domains and some new measures.

Methodological Review

The authors make a large number of deviations from the original study. Deviations that are important and improve the quality of the study, but deviations nonetheless.

Attention check: none were used in the original and no justification is given for their inclusion here. There is a common belief that attention checks improve data quality, but the evience behind such claims is spotty at best.

Lonliness scale: the authors are using a longer version that contains the original 3 items. This is nice, but in their results section for only the direct replication of lonliness --> anthropomorphism they should show the results with both the oginial 3 items and the whole scale.

Throughout: there needs to be a little more information on the scales: are the questions for each scale administered in fixed order? Are all the response options labeled or only anchors? Do participants see the numbers on the response options?

Throughout: report omega as a measure of internal consistency instead of Cronbach's alpha

The authors should add a captcha at the end, to take into account the bots on mTurk.

The authors should make explicit any requirements (e.g. country, participant completion rate, number of HITs completed, are Masterworkers allowed, etc.) and justify them.

The authors randomly order the presentation of the scales, which is another deviation. This is nice and I agree with their decision, but in their results section for only the direct replication of lonliness --> anthropomorphism they should show the results with both the oginial order (lonliness then gadgets) and then the whole sample.

I like the extension with Free Will beliefs, as it is a field I work in and topic I am interested in. There is a problem, however, in that part of the measure of anthropomorphism includes averaging an item asking if participants believe something or a supernatural object has free will. This creates an unintereesting autocorrelation of people who believe in free will believe other objects have free will. To create a faithful replication, the authors have to include the question, but for analyses involving the belief in free will as a predictor, they will need to create an alternate measure of anthropomorphism that does not include the free will itme averaged in.

Anthropomorphic Traits rank: summing the ranks is not a numerical value, it is putting numbers on something masquerading as a number. While I know this is a bigger problem in psychology and it could easily apply to Likert scales as well, it is another bridge to try to draw inferences about the total of a bunch of numbers from a rank order. I recommend the authors cut this measure completely.

Multiple regression model: I'm not sure the purpose of this model. It does not appear as a test of the 3-factor model, more of a test of what variables of what the authors collected is the stronger predictor. If the authors wish to keep this analysis, I would recommend the following changes: 1) use robust standard errors to account for any data issues; 2) analyze the model as a large path model with all predictors and dependent variables modelled together (this one isn't necessary, but would be an improvement and can provide a very nice graph for readers to understand); 3) test models using a likelihood ratio test as opposed to partial F-tests. 

Content Review

Abstract: reference to supplementary information is not necessary and should be reviewed.

p.3: Given the effect size of the original study, what was the power of the OSC replication (N=78) to detect that relationship? It should be reported in the ms.

p. 6: Somewhere in the introduction before page 6 should be a 1-paragraph overview of the original study (e.g. participants (N = 20) filled out the lonliness scale, were asked about X in gadgets. The original authors showed Y...). The section that starts 'controllability' jumps out at the reader who don't intimately know the study the authors are replicating.

p. 10: for the "has consciouness" items, there would be a grammatical error for angels has consciousness (should be have) and ghosts has consciousness (should be have). Will the authors change it?

p.12: put the hypotheses in the main manuscript.

p. 18: There is no closed bracket on the paragraph about nonconvergence.

Overall, I look forward to seeing the authors response and the eventual study.

Protzko