Recommendation

What to say to help one's partners in crime

based on reviews by Tom Ormerod and Lorraine Hope
A recommendation of:
toto

Exploring How Members of Illicit Networks Navigate Investigative Interviews

Abstract
Submission: posted 20 December 2021
Recommendation: posted 02 June 2022, validated 02 June 2022

Recommendation

When interviewing members of a criminal network, what determines what information a given interviewee chooses to disclose, as guided by the network's collective planning? What principles could help inform a detective preparing for such interviews? In the current study, Neequaye et al. will recruit groups of people known to each other to assume the role of networks that run an illegal sports betting business, fronting as a chain of tanning salons. Although each network launders money, they have to come up with a strategy to convince investigators they are legit. The groups are motivated to disclose some information when individuals are interviewed, but only enough to appear cooperative. The relation of the amount of different sorts of information disclosed depending on estimated risks and benefits for the group will be tested.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review by two expert reviewers. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/n7ugr
 
Level of bias control achieved: Level 6. Data collection began during the final round of Stage 1 peer review. Since no further revisions were made after this review round, the risk of bias due to prior data observation remains zero, and the manuscript therefore qualifies for Level 6.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Neequaye, D. A., Granhag, P. A. & Luke, T. J. (2022). Exploring How Members of Illicit Networks Navigate Investigative Interviews, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/n7ugr
Cite this recommendation as:
Zoltan Dienes (2022) What to say to help one's partners in crime. Peer Community in Registered Reports, . https://rr.peercommunityin.org/articles/rec?id=154
Conflict of interest:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

Reviewed by , 21 Apr 2022

I am happy that the authors have made a robust response to each of the comments I made. We are now at a stage where whatever disagreements or misinterpretations  remain are a matter of opinion rather than material concern. Thus, it is appropriate I think to congratulate the authors for the work they have put in to get this pre-registered study into good shape, run the study and see what happens!

Reviewed by , 30 May 2022

First, my full and sincere apologies for the delay in responding/reviewing. It's been a particularly hectic period but I appreciate that's my (persistent) problem.

I read through the response letter and associated revisions in the draft. I'm glad the authors found the comments useful and as far as I'm concerned, the authors have addressed my comments satisfactorily. I'm also happy to roll with the notion that some issues/outcomes are unknowable until the data come in.

I have no further substantive comments at this time. I wish the authors the very best of luck - and I'm very interested to see what they find in this novel study.


Evaluation round #3

DOI or URL of the report: https://osf.io/2zn8r/?view_only=b2148cce80c5430baef8b31fc3ede111

Version of the report: https://osf.io/2zn8r/?view_only=b2148cce80c5430baef8b31fc3ede111

Author's Reply, 22 Mar 2022

Decision by , posted 12 Mar 2022

Dear David

 

I now have two expert reviews in, and I am very pleased with their thoughtful and constructive engagement, in the collaborative spirit that RRs help. Both reviewers are excited by an ambitious attempt to deal with a complex and important question. But both feel serious reworking is needed to enable the study to achieve its goals. The issues are partly methodological, whether the precise procedure is the best one to use; partly conceptual, thinking about what predictions are well motivated. Please give a detailed point by point response to each of their issues in your reply.

I very much look forward to seeing your revision,

best

Zoltan

Reviewed by , 12 Mar 2022

Reviewed by , 25 Feb 2022


Evaluation round #2

DOI or URL of the report: https://osf.io/2zn8r/?view_only=b2148cce80c5430baef8b31fc3ede111

Version of the report: https://osf.io/2zn8r/?view_only=b2148cce80c5430baef8b31fc3ede111

Author's Reply, 28 Jan 2022

Download tracked changes file

Dear Prof. Dienes,

Many thanks for your suggestions we have now edited the manuscript accordingly. We have phrased the language to imply estimation of potential effects. Additionally, we have identified the specific effects that will examine our hypotheses. The Analysis Plan and Table 1 reflect the changes. Apologies for the tardiness in returning with this revision.

Sincerely,

Authors

Decision by , posted 21 Jan 2022

Dear Dr Neequaye

Thank you for your revision. PCI RR has no power requirements; it is fine to calculate power for that minimally interesting effect which you have appropraitely identified, and indicate the power is too low for a non-signfiicant result to be taken as grounds for assserting Ho. That is just a matter of being explicit about what follows from a given inferential procedure. So do state the power which you do have for each effect of interest (if you are testing hypotheses).

Bear in mind that different journals, including the PCI RR friendly ones, that you may (or may not) wish to publish in afterwards sometimes do have power requirements, if power is used (though Royal Society Open Science does not, so that would be a possiblity).  But an alternative, as you say, is not to phrase your study in terms of hypothesis testing at all, but estimation. In that case your goals would not be stated as establishing whether or not there is an effect, and all such language would have to be carefully avoided; the goal is to estimate relevant effects with say a confidence or credibility interval (or similar).  The hypotheses could be phrased as implying a certain interaction etc is relevant (not present or absent, that language would be avoided). In fact, your phrasing is largely of the estimation type "to what extent", except for row 1 coloumn 1 of the Design table. You could change this, then the important results are, as I say, the confidence (etc) intervals for each effect of interest. In that case, power is not relevant, but you should indicate the expected precision, or width of the CI, that your N should provide.

Whichever way you go on that, thanks for adding the table; but each hypothesis is related in the table to a set of effects. This allows inferential flexibility. Can you identify a specific question with a specific effect?

In sum the changes needed for me to send out to review are minor, it is just a matter of being a bit clearer about your inferential goals and capabilities.

best

Zoltan

 

 


Evaluation round #1

DOI or URL of the report: https://osf.io/2zn8r/?view_only=b2148cce80c5430baef8b31fc3ede111

Author's Reply, 17 Jan 2022

Decision by , posted 06 Jan 2022

Dear Dr Neequaye

Thank you for your submission "Exploring How Members of Illicit Networks Navigate Investigative Interviews."  Before I send for review, there are a few issues to address:

1)  Include a  Study Design Template in the main manuscript. See here for details: https://rr.peercommunityin.org/help/guide_for_authors#h_27513965735331613309625021

This involves creating a table aligning specific theoretical questions with hypotheses with the specific statistical test that will test that hypothesis, and the conditions under which support will be inferred for the hypthesis, or rather the results will actively count against the hypothesis.

2) Make sure you remove all analytic flexibility from the analysis plan. Thus, be clear how convergence will be assessed, precisely when it will be decided to not have not occurred; precisely when parameters, like random slopes, will be dropped, etc.

3) Use an inferential procedure that can provide justification for the claim there was not an effect, i.e. the data counted against a hypothesis predicting an effect. For an overview, see https://psyarxiv.com/yc7s5/.  

4) In the light of 3), indicate how your N was determined, and what properties this gives your statistical procedure in terms of the likelihood of results counting for or against your hypothesis (e.g. power if you are using frequentist tests; see previously cited article for overview).

Let me know if there are any questions about how to proceed on any of these matters.

best

Zoltan

 

User comments

No user comments yet