Submit a report

Announcements

We are recruiting recommenders (editors) from all research fields!

Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.

Latest recommendations

IdTitle * Authors * Abstract * PictureThematic fields * RecommenderReviewersSubmission date
09 Sep 2024
STAGE 2
(Go to stage 1)

The effects of isolated game elements on adherence rates in food response inhibition training

Using gamification to improve food response inhibition training

Recommended by ORCID_LOGO based on reviews by Miguel Vadillo and Daniel Phipps
A poor diet has severe detrimental health effects, and attempts to reduce caloric intake often prove unsuccessful. Unhealthy foods, high in fat, sugar, and salt tend to be highly appetitive, and can undermine individuals’ ability to refrain themselves from consuming them. Computerized cognitive retraining techniques have shown promise in curbing the intake of unhealthy foods and promoting weight loss. However, in real-world scenarios, adherence to such retraining programs can be suboptimal, potentially diminishing their effectiveness.
 
In the present study, Maclellan et al. (2024) aimed to investigate whether the incorporation of gamified elements, transforming the cognitive retraining task into a game-like experience, can enhance adherence and overall intervention effectiveness by boosting engagement and motivation.
 
Upon testing the main hypotheses, the authors found mostly non-significant effects of adding gamified elements to adherence, motivation, or effectiveness of food response inhibition training programs. These results hold high relevance, as indeed there has been a push in introducing gamified elements to cognitive retraining programs. These findings should guide future developments in the field of cognitive retraining.
 
The Stage 2 manuscript was evaluated over one round of review and revision. Based on detailed evaluations by two expert reviewers, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/jspf3

Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
References
 
MacLellan, A., Pennington, C. R., Lawrence, N., Westwood, S. J., Jones, A., Slegrova, A., Sung, B., Parker, L., Relph, L., Miranda, J. O., Shakeel, M., Mouka, E., Lovejoy, C., Chung, C., Lash, S., Suhail, Y., Nag M., and Button​, K. S. (2024). The effects of isolated game elements on adherence rates in food response inhibition training​ [Stage 2]. Acceptance of Version 3 by Peer Community in Registered Reports.
The effects of isolated game elements on adherence rates in food response inhibition trainingAlexander MacLellan, Charlotte R. Pennington, Natalia Lawrence, Samuel J. Westwood, Andrew Jones, Anna Slegrova, Beatrice Sung, Louise Parker, Luke Relph, Jessica O. Miranda, Maryam Shakeel, Elizabeth Mouka, Charlotte Lovejoy, Chaebin Chung, Sabel...<p>Introduction: Poor diet and the consumption of foods high in fat, sugar and salt are common causes of numerous health conditions and premature mortality. Computerised food response inhibition training (food-RIT) is a type of intervention found ...Social sciencesMateo Leganes-Fonteneau2024-06-28 23:29:31 View
09 Sep 2024
STAGE 1

The effects of isolated game elements on adherence rates in food-based response inhibition training

Using gamification to improve food response inhibition training

Recommended by ORCID_LOGO based on reviews by Miguel Vadillo and 2 anonymous reviewers
A poor diet has severe detrimental health effects, and attempts to reduce caloric intake often prove unsuccessful. Unhealthy foods, high in fat, sugar, and salt tend to be highly appetitive, and can undermine individuals’ ability to refrain themselves from consuming them. Computerized cognitive retraining techniques have shown promise in curbing the intake of unhealthy foods and promoting weight loss. However, in real-world scenarios, adherence to such retraining programs can be suboptimal, potentially diminishing their effectiveness.
 
In the present study, Maclellan et al. (2023) aim to investigate whether the incorporation of gamified elements, transforming the cognitive retraining task into a game-like experience, can enhance adherence and overall intervention effectiveness by boosting engagement and motivation. These findings could deepen our understanding of the mechanisms through which cognitive retraining mitigates caloric intake and pave the way for more compelling and successful interventions.
 
The Stage 1 manuscript was evaluated over one round of review and revision. Based on detailed evaluations by two expert reviewers, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/jspf3 (under temporary private embargo)

Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 

References
 
1. MacLellan, A., Pennington, C., Lawrence, N., Westwood, S., Jones, A., & Button, K. (2023). The effects of isolated game elements on adherence rates in food-based response inhibition training. In principle acceptance of Version 1.6 by Peer Community in Registered Reports. https://osf.io/jspf3

The effects of isolated game elements on adherence rates in food-based response inhibition trainingAlexander MacLellan, Charlotte Pennington, Natalia Lawrence, Samuel Westwood, Andrew Jones, Katherine Button<p>Introduction: Poor diet, and the consumption of foods high in fat, sugar and salt are common causes of premature mortality and health conditions. Computerised response inhibition training has been proposed to devalue these foods by learning to ...Social sciencesMateo Leganes-Fonteneau2023-07-14 16:24:03 View
16 Nov 2023
STAGE 1

The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered Report

Are there oscillatory markers of pain intensity?

Recommended by ORCID_LOGO based on reviews by Markus Ploner and Björn Horing
Rhythmic changes in pain can lead to corresponding modulations of EEG amplitudes in theta, alpha, and beta bands. But the question remains open as to whether these modulations are actually tracking pain, or maybe rather saliency or stimulus intensity. The question is of some importance because a marker of pain per se could be useful for tracking felt pain without a verbal response, and could be useful in investigating interventions for treating pain (such as suggestion).  Here, Leu et al. (2023) will address the question of whether modulations reflect saliency or else the intensity of pain, by using an oddball paradigm in which most trials are a pain stimulus of a certain intensity, and oddball trials will sometimes occur, at either a higher intensity or a lower intensity than the baseline ones. If the modulations reflect salience, the modulation at the frequency of the oddball will be similar for high and low intensity oddballs. However, if the modulations reflect pain intensity, the modulations for the low rather than high oddball condition will be lower.
 
The Stage 1 manuscript was evaluated over three rounds of in-depth peer review, the first two consisting of substantial comments from two scholars with relevant expertise, and the third consisting of a close review by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/qbrf2
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Leu, C., Forest, S., Legrain, V., & Liberati, G. (2023). The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered Report. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/qbrf2
The effect of stimulus saliency on the modulation of pain-related ongoing neural oscillations: a Registered ReportChiara Leu, Sébastien Forest, Valéry Legrain, Giulia Liberati<p>Ongoing oscillations have been shown to be modulated in different frequency bands following phasic, tonic as well as periodic thermonociceptive stimulation. Yet, it remains unclear whether these modulations are related to pain perception, salie...Life Sciences, Medical SciencesZoltan Dienes2023-09-06 15:15:19 View
30 Sep 2024
STAGE 1

The effect of covert visual attention on pupil size during perceptual fading

Does pupil size track high-level attention?

Recommended by ORCID_LOGO and ORCID_LOGO based on reviews by Sander Nieuwenhuis, Martin Rolfs and 1 anonymous reviewer
Previous research has shown that higher-level cognitive processes like attention, memory or mental imagery can produce pupil responses, without any actual difference in luminance of the visual stimulus. Interestingly, the inverse scenario remains untested: when a difference in luminance is perceptually eliminated from awareness, does pupil size still track cognitive factors? In this study, Vilotijević and Mathôt (2024) seek to test this experimentally using a perceptual fading phenomenon where two Gaussian patches with different luminances fade from consciousness and are thus perceived as mid-level uniform grey. The researchers will present this fading manipulation and a control condition where no fading occurs in separate blocks. Participants are instructed to covertly attend one of the patches. If pupil size reflects attentional selection, pupil responses should be reduced in the fading condition, and this should evolve with time as the stimuli are perceptually fading.
 
The Stage 1 manuscript was evaluated by three reviewers and the Recommender over three rounds of review. One reviewer advises conducting robustness checks to rule out eye movements confounds. This constitutes exploratory analyses at Stage 2 that probably require some flexibility. Despite this, the Recommender decided to grant in-principle acceptance of this study. While these robustness tests can put the eventual results in context, the general hypotheses and predictions specified in this version are clear and straightforward.
 
URL to the preregistered Stage 1 protocol: https://osf.io/bmtp6
 
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Vilotijević, A. & Mathôt, S. (2024). The effect of covert visual attention on pupil size during perceptual fading. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/bmtp6
 
The effect of covert visual attention on pupil size during perceptual fadingAna Vilotijević, Sebastiaan Mathôt<p>Pupil size is modulated by various cognitive factors such as attention, working memory, mental imagery, and subjective perception. Previous studies examining cognitive effects on pupil size mainly focused on inducing or enhancing a subjective e...Social sciencesD. Samuel Schwarzkopf Martin Rolfs2024-01-18 13:28:02 View
30 Sep 2024
STAGE 2
(Go to stage 1)

The effect of covert visual attention on pupil size during perceptual fading

Does pupil size track high-level attention?

Recommended by ORCID_LOGO based on reviews by Sander Nieuwenhuis and Martin Rolfs
Higher-level cognitive processes like attention, memory, or mental imagery can produce pupil responses, without any actual difference in luminance of the visual stimulus. Interestingly, the inverse scenario remained untested to date: when a physical luminance difference is perceptually eliminated from awareness, does pupil size still track attention to the stimulus? In this Registered Report, Vilotijević and Mathôt (2024) sought to test this experimentally using a perceptual fading phenomenon where two Gaussian patches with different luminances fade from consciousness and are thus perceived as mid-level uniform grey (or at least the subjective difference is much reduced). This fading manipulation, as well as a control condition without perceptual fading, were presented in separate blocks. Participants were instructed to covertly attend one of the patches.

The authors hypothesised that if pupil size reflects attentional selection, these pupil responses in the fading condition​ should be eliminated or at least reduced, and this should evolve with time as the stimuli are perceptually fading. Their results show that pupil responses during covert attention are indeed reduced during perceptual fading - but they are not eliminated. Interestingly, this reduction did not depend on time or self-reports of the strength of perceptual fading. The findings therefore suggest that pupil dilation tracks subjective brightness differences.
 
One inherent issue with experiments like these is that the experimental and control conditions necessarily involve a physical difference in the stimulus. Here, the fading condition had the same spatial configuration of light and dark stimuli throughout a block while in the control (non-fading) condition the light and dark stimuli alternated sides between trials. It is therefore impossible to completely rule out that the physical difference affects the results. However, the only alternative to this would be an experimental design in which the stimuli never change, but only the subjective perceptual state varies. Such a design is completely at the mercy of the participant's subjective state and therefore loses experimental control and statistical sensitivity. The present results confirmed the authors' prediction that there are indeed differences in overall pupil responses during the fading and control conditions, irrespective of covert attention.
 
Critically, the fact that the attention effect did not vary with time or subjective self-reports of the illusion supports the authors' interpretation that this reflects higher-level cognition: the mere act of attending to the dark side - even if the actual appearance has faded - could cause a sustained pupil dilation. This would be consistent with the type of pupil effects for memory and mental imagery that motivated the present study. However, a simpler alternative is that the experience of perceptual fading was incomplete (as possibly suggested by Figures 2D and 2F) but that self-reports fail to capture this subjective experience accurately. Perhaps a future study could compare the magnitude of the attentional pupil effects when the initial stimulus is completely removed. If similar differences in pupil response persist this would suggest that the present results are due to high-level modulation or the residual low-level luminance difference.
 
The Stage 2 manuscript was evaluated by two reviewers and the recommender over two rounds of review. One reviewer again advised additional robustness checks to rule out eye movement confounds, an issue they had already raised during Stage 1 review. The researchers provide clear evidence that this is unlikely to have confounded their findings. This has been added to the supplementary data repository. Following this review and revision, the recommender judged that the Stage 2 criteria were met and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/bmtp6
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Vilotijević, A. & Mathôt, S. (2024). The effect of covert visual attention on pupil size during perceptual fading [Stage 2]. Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/ku8qc?view_only=f331df53b50f431386fabba9e386b387
 
The effect of covert visual attention on pupil size during perceptual fadingAna Vilotijević, Sebastiaan Mathôt<p>Pupil size is modulated by various cognitive factors such as attention, working memory, mental imagery, and subjective perception. Previous studies examining cognitive effects on pupil size mainly focused on inducing or enhancing a subjective e...Social sciencesD. Samuel Schwarzkopf2024-08-17 12:58:58 View
04 Dec 2023
STAGE 2
(Go to stage 1)

The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report

Brooding increases conspiracy beliefs but with practical significance to be determined

Recommended by ORCID_LOGO based on reviews by Matt Williams and Daniel Toribio-Flórez
The world is seemingly awash with conspiracy theories – from well-trodden examples such as fake Moon landings, the 9/11 truth movement, and Holocaust denial, to relative newcomers including COVID as a bioweapon, QAnon, and the belief that the science of climate change has been invented or falsified. While there is a public perception that conspiracy theories are becoming more prevalent, recent evidence suggests that the rate of conspiracism is relatively stable over time (Uscinski et al., 2022). At any point in history, it seems that a certain proportion of people find themselves vulnerable to conspiracy beliefs, but what distinguishes those who do from those who don’t, and what are the causal factors?
 
In the current study, Liekefett et al. (2023) investigated the critical role of rumination – a perseverative and repetitive focus on negative content leading to emotional distress. In particular, the authors asked whether one component of rumination referred to as brooding (dwelling on one’s worries and distressing emotions) has a specific causal role in the formation of conspiracy beliefs. In a series of preliminary experiments, the authors first established a procedure for successfully inducing rumination, identifying various boundary conditions and requirements for a successful design. In the main study (N=1,638 to 2,007 depending on the analysis), they asked whether the induction of brooding causes a significant increase in conspiracy beliefs. Manipulation checks were also included to confirm intervention fidelity (independently of this hypothesis), and exploratory analyses tested the effect of various moderators, as well as the causal role of a complementary manipulation of reflection – a component of rumination in which attention is focused on the issue at hand rather than one’s emotions.
 
As expected by the authors' preliminary work, manipulation checks independently confirmed the effectiveness of the brooding intervention. In answer to the main research question, participants who brooded over the worries and negative emotions associated with an issue were more susceptible to conspiracy beliefs compared to a control group. However, while this effect of brooding was statistically significant, the confidence interval of the effect size estimate overlapped with the authors' proposed smallest effect size of interest (d = 0.20), suggesting that the practical value of the effect remains to be determined.
 
Overall the findings are consistent with a range of psychological theories suggesting that rumination induces negative affect and/or narrows attention to negative information, which in turn may make conspiracy theories seem more probable and render individuals more vulnerable to cognitive bias. The authors note the importance of future work to define the smallest effect of practical significance, analagous to the criteria used to determine the 'minimal clinically important difference’ in medical research.
 
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/y82bs
 
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Uscinski, J., Enders, A., Klofstad, C., Seelig, M., Drochon, H., Premaratne, K. & Murthi, M. (2022) Have beliefs in conspiracy theories increased over time? PLOS ONE 17: e0270429. https://doi.org/10.1371/journal.pone.0270429

2. Liekefett, L. Sebben, S. & Becker, J. C. (2023). The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered Report. Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/3e8wc
The Effect of Brooding about Societal Problems on Conspiracy Beliefs: A Registered ReportLuisa Liekefett, Simone Sebben, Julia C. Becker<p>This Stage 2 Registered Report concerns the relationship between rumination, a repetitive style of negative thinking, and conspiracy beliefs (Stage 1 protocol: https://osf.io/y82bs, date of in-principle-acceptance: 23/05/2023). Based on four pi...Humanities, Social sciencesChris Chambers2023-10-19 17:46:59 View
22 Nov 2024
STAGE 2
(Go to stage 1)

The capacity of response training to help resist the consumption of sugary drinks

Gamified response training with sugary drinks does not facilitate adherence to a restrictive diet

Recommended by ORCID_LOGO based on reviews by Matthias Aulbach and Pieter Van Dessel
Many people nowadays struggle with maintaining a healthy diet. Despite the intentions to eat healthily, they often over-consume highly palatable yet nutrient-poor foods and drinks, which in turn can lead to many health problems. There is therefore a need to develop tools that can help people resist the (over-)consumption of such foods and drinks. 

Previous work has shown that training people to execute certain motor responses toward food items can modify their liking for these items, which may also influence their subsequent consumption behavior. Based on these findings, Najberg et al. (2023) developed a mobile game that combined two food-related response training tasks, namely the go/no-go training (Veling et al., 2017) and the cue-approach training (Schonberg et al., 2014). The experimental group was trained to consistently inhibit their responses toward sugary drinks in the go/no-go training, and consistently respond to water items in the cue-approach training (i.e., 100% consistent mapping). In the control group, the mapping between an item and response requirement was 50%, such that participants executed both go and no-go responses toward sugary drinks and water. Najberg et al. (2023) found that after the training, the experimental group reported more reduction in liking for sugary drinks and more increase in liking for water items compared to the control group. However, both groups showed equivalent reduction in self-reported consumption of sugary drinks. 

Using the same design (i.e., 100% vs. 50% consistency), in the current study, Najberg et al. (2024) further examined whether the combined go/no-go and cue-approach training game could help people resist the consumption of sugary drinks. Participants were divided into the experimental and control group (N = 100 and 92, respectively), and received the respective training for a minimum of seven days (and up to 20 days). After completing the training, they were asked to avoid the trained sugary drinks. The number of days in which they reported to successfully adhere to this restrictive diet was used as the main dependent variable. Contrary to their predictions, the two groups did not differ in how long they resisted the consumption of sugary drinks after training. Both groups showed equivalent reductions in liking for sugary drinks (contrary to the finding in Najberg et al., 2023), but this reduction in liking was not correlated with the number of successful days of diet in the experimental group. Lastly, those who trained for more days in the experimental group also adhered to the diet for a longer duration, but this correlation might be explained by differences in motivation across individuals.

Together, these results suggest that consistently withholding responses toward sugary drinks and responding to water items does not help people resist the consumption of sugary drinks, compared to a control intervention in which the mapping is 50%. More research is therefore needed to test the effectiveness of food-related response training in changing consumption behavior outside of laboratory contexts.

The Stage 2 manuscript was evaluated over three rounds of review by two expert reviewers who also assessed the Stage 1 manuscript. Following detailed responses to the recommender and the reviewers’ comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/97aez

Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA.
 
List of eligible PCI-RR-friendly journals: 
 
 
References

1. Najberg, H., Mouthon, M., Coppin, G., & Spierer, L. (2023). Reduction in sugar drink valuation and consumption with gamified executive control training. Scientific Reports, 13, 10659. https://doi.org/10.1038/s41598-023-36859-x
 
2. Veling, H., Lawrence, N. S., Chen, Z., van Koningsbruggen, G. M., & Holland, R. W. (2017). What Is Trained During Food Go/No-Go Training? A Review Focusing on Mechanisms and a Research Agenda. Current Addiction Reports, 4, 35–41. https://doi.org/10.1007/s40429-017-0131-5
 
3. Schonberg, T., Bakkour, A., Hover, A. M., Mumford, J. A., Nagar, L., Perez, J., & Poldrack, R. A. (2014). Changing value through cued approach: An automatic mechanism of behavior change. Nature Neuroscience, 17, 625–630. https://doi.org/10.1038/nn.3673

4. Najberg, H., Tapparel, M., & Spierer, L. (2024). The capacity of response training to help resist the consumption of sugary drinks [Stage 2]. Acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/eu7j4?view_only=4934c0215f2943cfb42e019792a30b53
The capacity of response training to help resist the consumption of sugary drinksHugo Najberg, Malika Tapparel, Lucas Spierer<p style="text-align: justify;">Food response training has been shown to reduce the reported value of palatable food items. These approaches may thus help to reduce unhealthy (over)consumption behaviors and its related diseases. Yet, whether and h...Medical Sciences, Social sciencesZhang Chen2024-06-26 11:41:16 View
18 Jul 2023
STAGE 1

Test-Retest Reliability of the STRAQ-1: A Registered Report

A reliable measure of physical closeness in interpersonal relationships?

Recommended by ORCID_LOGO based on reviews by Jacek Buczny and Ian Hussey
Attachment and interpersonal relationships are a major subject of research and clinical work in psychology. There are, accordingly, a proliferation of measurement instruments to tap into these broad constructs. The emphasis in these measures tends to be on the emotional dimensions of the relationships—how people feel about their partners and the support that they receive. However, that is not all there is to relationship quality. Increasing attention has been paid to the physical and physiological aspects of relationships, but there are few psychometrically sound measures available to assess these dimensions.
 
In the current study, Dujols et al. (2023) seek to assess the psychometric properties of the Social Thermoregulation and Risk Avoidance Questionnaire (STRAQ-1), a measure of physical relationships that targets social thermoregulation, or how physical proximity is used to promote warmth and closeness. The proposed project will be a thorough assessment of the measure’s reliability over time—that is, the degree to which the measure assesses the construct similarly across administrations. The authors will assess the test-retest reliability and longitudinal measurement invariance of the STRAQ-1, providing much-needed psychometric data that can build confidence in the utility of the measure.
 
The Stage 1 manuscript was evaluated over two rounds of in-depth review, the first round consisting of detailed comments from two reviewers and the second round consisting of a close read by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and was therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/pmnk2
 
Level of bias control achieved: Level 3. At least some data/evidence that will be used to the answer the research question has been previously accessed by the authors (e.g. downloaded or otherwise received), but the authors certify that they have not yet observed ANY part of the data/evidence
 
List of eligible PCI RR-friendly journals:
 
References
 
1. Dujols, O., Klein, R. A., Lindenberg, S., Van Lissa, C. J., & IJzerman, H. (2023). Test-Retest Reliability of the STRAQ-1: A Registered Report. In principle acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/pmnk2
Test-Retest Reliability of the STRAQ-1: A Registered ReportOlivier Dujols; Richard A. Klein; Siegwart Lindenberg; Hans IJzerman<p>This Registered Report provides the first test of measurement invariance across time points and estimates of test-retest reliability for the Social Thermoregulation, Risk Avoidance Questionnaire (STRAQ-1, Vergara et al., 2019). The scale was de...Social sciencesMoin Syed2023-03-01 16:07:17 View
25 Jun 2024
STAGE 2
(Go to stage 1)

Test-Retest Reliability of the STRAQ-1: A Registered Report

We may not be measuring physical closeness in interpersonal relationships as reliably as we think

Recommended by ORCID_LOGO based on reviews by Maanasa Raghavan and Jacek Buczny
Attachment and interpersonal relationships are a major subject of research and clinical work in psychology. There are, accordingly, a proliferation of measurement instruments to tap into these broad constructs. The emphasis in these measures tends to be on the emotional dimensions of the relationships—how people feel about their partners and the support that they receive. However, that is not all there is to relationship quality. Increasing attention has been paid to the physical and physiological aspects of relationships, but there are few psychometrically sound measures available to assess these dimensions.
 
In the current study, Dujols et al. (2024) assessed the psychometric properties of the Social Thermoregulation and Risk Avoidance Questionnaire (STRAQ-1), a measure of physical relationships that targets social thermoregulation, or how physical proximity is used to promote warmth and closeness. The project consists of a thorough assessment of the measure’s reliability over time—that is, the degree to which the measure assesses the construct similarly across administrations, in a sample of 183 French university students.
 
The authors assessed the longitudinal measurement invariance and test-restest reliability of the STRAQ-1. Longitudinal measurement invariance across two time points was only found for two of the four subscales. Similarly, test-retest reliability varied by subscale, ranging from poor to good. Taken together, the study suggests caution in using the STRAQ-1 scale as a reliable measure of physical relationships. The study highlights the need for continued assessment of the reliability of widely used measures, particularly reliability over time, and serves as a model for a rigorous analytic approach for doing so.
 
The Stage 2 manuscript was evaluated over two rounds of in-depth review, the first round consisting of comments from two reviewers and the second round consisting of a close read by the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and therefore awarded a positive recommendation.
 
URL to the preregistered Stage 1 protocol: https://osf.io/pmnk2
 
Level of bias control achieved: Level 3. At least some data/evidence that was used to the answer the research question was accessed by the authors prior to Stage 1 IPA (e.g. downloaded or otherwise received) but the authors certify that they had not observed ANY part of the data/evidence until after Stage 1 IPA.
 
List of eligible PCI RR-friendly journals:
 
 
References
 
1. Dujols, O., Klein, R. A., Lindenberg, S., Van Lissa, C. J., & IJzerman, H. (2024). Test-Retest Reliability of the STRAQ-1: A Registered Report [Stage 2]. Acceptance of Version 10 by Peer Community in Registered Reports. https://osf.io/preprints/psyarxiv/392g6
Test-Retest Reliability of the STRAQ-1: A Registered ReportOlivier Dujols, Siegwart Lindenberg, Caspar J. van Lissa, Hans IJzerman<p>This Registered Report provides the first test of measurement invariance across time points and estimates of test-retest reliability for the Social Thermoregulation, Risk Avoidance Questionnaire (STRAQ-1, Vergara et al., 2019). The scale was de...Social sciencesMoin Syed2024-01-11 16:01:41 View
09 Jul 2024
STAGE 1

Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task

Exploring determinants of test-retest reliabilty in fMRI

Recommended by ORCID_LOGO based on reviews by Xiangzhen Kong and 2 anonymous reviewers
Functional magnetic resonance imaging (fMRI) has been used to explore brain-behaviour relationships for many years, with proliferation of a wide range of sophisticated analytic procedures. However, rather scant attention has been paid to the reliability of findings. Concerns have been growing following failures to replicate findings in some fields, but it is hard to know how far this is a consequence of underpowered studies, flexible analytic pipelines, or variability within and between participants. 
 
Demidenko et al. (2023) plan a study that will be a major step forward in addressing these issues. They take advantage of the availability of three existing datasets, the Adolescent Brain Cognitive Development (ABCD) study, the Michigan Longitudinal Study, and the Adolescent Risk Behavior Study, which all included a version of the Monetary Incentive Delay task measured in two sessions. This gives ample data for a multiverse analysis, which will consider how within-subject and between-subject variance varies according to four analytic factors: smoothing (5 levels), motion correction (6 levels), task modelling (3 levels) and task contrasts (4 levels).  They will also consider how sample size affects estimates of reliability. This will involve a substantial amount of analysis.
 
The study is essentially focused on estimation, although specific predictions are presented regarding the combinations of factors expected to give optimal reliability.  The outcome will be a multiverse of results which will allow us to see how different pipeline decisions for this task affect reliability. In many ways, null results – finding that at least some factors have little effect on reliability – would be a positive finding for the field, as it would mean that we could be more relaxed when selecting an analytic pathway. A more likely outcome, however, is that analytic decisions will affect reliability, and this information can then guide future studies and help develop best practice guidelines. As one reviewer noted, we can’t assume that results from this analysis will generalise to other tasks, but this analysis with a widely-used task is an important step towards better and more consistent methods in fMRI.
 
The researchers present a fully worked out plan of action, with supporting scripts that have been developed in pilot testing. The Stage 1 manuscript received in-depth evaluation from three expert reviewers and the recommender. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
 
URL to the preregistered Stage 1 protocol: https://osf.io/nqgeh
 
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet sufficiently observed the key variables within the data to be able to answer the research questions AND they have taken additional steps to maximise bias control and rigour.
 
List of eligible PCI RR-friendly journals:
 

References

1. Demidenko, M. I., Mumford, J. A., & Poldrack, R. A. (2023). Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay Task. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/nqgeh
Test-Retest Reliability in Functional Magnetic Resonance Imaging: Impact of Analytical Decisions on Individual and Group Estimates in the Monetary Incentive Delay TaskMichael I. Demidenko, Jeanette A. Mumford, Russell A. Poldrack<p>Empirical studies reporting low test-retest reliability of individual neural estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods that may improve reliability in fMR...Life Sciences, Social sciencesDorothy Bishop2023-04-17 22:27:54 View