=============================================================================
IMPORTANT ANNOUNCEMENT: To accommodate reviewer and recommender holiday schedules, we will be closed to ALL submissions from 1st Jul - 1st Sep. During this time, reviewers can submit reviews and recommenders can issue decisions, but no new or revised submissions can be made by authors.
The one exception to this rule is that authors using the scheduled track who submit their initial Stage 1 snapshot prior to 1st Jul can choose a date within the shutdown period to submit their full Stage 1 manuscript.
We recommend that authors submit at least 1-2 weeks prior to commencement of the shutdown period to enable time to make any required revisions prior to in-depth review.
=============================================================================
We are recruiting recommenders (editors) from all research fields!
Your feedback matters! If you have authored or reviewed a Registered Report at Peer Community in Registered Reports, then please take 5 minutes to leave anonymous feedback about your experience, and view community ratings.
Latest recommendations
| Id | Title * ▲ | Authors * | Abstract * | Picture | Thematic fields * | Recommender | Reviewers | Submission date | |
|---|---|---|---|---|---|---|---|---|---|
Convenience Samples and Measurement Equivalence in Replication ResearchData from students and crowdsourced online platforms do not often measure the same thingRecommended by Corina Logan based on reviews by Benjamin Farrar and Shinichi Nakagawa
Comparative research is how evidence is generated to support or refute broad hypotheses (e.g., Pagel 1999). However, the foundations of such research must be solid if one is to arrive at the correct conclusions. Determining the external validity (the generalizability across situations/individuals/populations) of the building blocks of comparative data sets allows one to place appropriate caveats around the robustness of their conclusions (Steckler & McLeroy 2008). In the current study, Alley and colleagues (2023) tackled the external validity of comparative research that relies on subjects who are either university students or participating in experiments via an online platform. They determined whether data from these two types of subjects have measurement equivalence - whether the same trait is measured in the same way across groups. Although they use data from studies involved in the Many Labs replication project to evaluate this question, their results are of crucial importance to other comparative researchers whose data are generated from these two sources (students and online crowdsourcing). The authors show that these two types of subjects do not often have measurement equivalence, which is a warning to others to evaluate their experimental design to improve validity. They provide useful recommendations for researchers on how to to implement equivalence testing in their studies, and they facilitate the process by providing well annotated code that is openly available for others to use. After one round of review and revision, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation. URL to the preregistered Stage 1 protocol: https://osf.io/7gtvf
Level of bias control achieved: Level 2. At least some data/evidence that was used to answer the research question had been accessed and partially observed by the authors prior to Stage 1 IPA, but the authors certify that they had not yet observed the key variables within the data that were used to answer the research question AND they took additional steps to maximise bias control and rigour.
List of eligible PCI RR-friendly journals:
References
1. Pagel, M. (1999). Inferring the historical patterns of biological evolution. Nature, 401, 877-884. https://doi.org/10.1038/44766
2. Steckler, A. & McLeroy, K. R. (2008). The importance of external validity. American Journal of Public Health 98, 9-10. https://doi.org/10.2105/AJPH.2007.126847
3. Alley L. J., Axt, J., & Flake J. K. (2023). Convenience Samples and Measurement Equivalence in Replication Research [Stage 2 Registered Report] Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/s5t3v
| Convenience Samples and Measurement Equivalence in Replication Research | Lindsay J. Alley, Jordan Axt, Jessica Kay Flake | <p>A great deal of research in psychology employs either university student or online crowdsourced convenience samples (Chandler & Shapiro, 2016; Strickland & Stoops, 2019) and there is evidence that these groups differ in meaningful ways ... | Social sciences | Corina Logan | Alison Young Reusser | 2023-08-31 20:26:43 | ||
25 Jun 2023
STAGE 1
Cortical plasticity of the tactile mirror system in borderline personality disorderIs borderline personality disorder linked to impairment of the tactile mirror system?Recommended by Chris Chambers based on reviews by Zoltan Dienes and 2 anonymous reviewers
Borderline personality disorder (BPD) is a mental illness affecting ~1 in 100 people (Ellison et al., 2018), characterised by emotional dysregulation, impulsivity, a distorted sense of self, and a long-term pattern of unstable interpersonal relationships. Among this heterogenous range of symptoms is difficulty in the cognitive dimension of empathy, in particular understanding the perspectives of others, which in turn has been suggested to rely on the mirror neuron system, both in the motor and somatosensory domains. The integrity of the mirror system has therefore been a focus for understanding the possible causes or consequences of the disorder, with preliminary studies pointing to hypoactivity of neuronal areas associated with the mirror system in BPD (Mier et al., 2013).
In the current study, Zazio et al. (2023) will use crossmodal paired associative stimulation (cm-PAS) in which an image of a hand being touched is repeatedly paired with transcranial magnetic stimulation (TMS) of the primary somatosensory cortex (S1) to test the hypothesis that BPD is associated with a specific deficit in the tactile mirror system. In healthy controls, the close temporal coupling (20ms) between the visual depiction of tactile stimulation and TMS of S1 is expected boost tactile acuity and elevate the performance cost of incongruence in a task that manipulates visuo-tactile spatial congruity (VTSC) – effects that are thought to reflect the fidelity of the tactile mirror system.
In BPD patients, however, the authors make the crucial prediction that impairment of the tactile mirror system (if present) will lead to a reduced (or even non-existent) effect of cm-PAS on tactile acuity and VTSC task performance compared to healthy controls. To help ensure a severe test of this hypothesis, the design includes a variety of controls, including an attention check, control cm-PAS in which the inter-stimulus interval is increased to 100ms to break the close temporal coupling between visual stimulation and TMS, and a positive control to confirm that active cm-PAS (compared to control cm-PAS) produces the expected boost in tactile acuity in healthy controls.
The Stage 1 manuscript was evaluated over three rounds of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/sqnwd
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Ellison, W. D., Rosenstein, L. K., Morgan, T. A., & Zimmerman, M. (2018). Community and clinical epidemiology of borderline personality disorder. Psychiatric Clinics, 41, 561-573. https://doi.org/10.1016/j.psc.2018.07.008
2. Mier, D., Lis, S., Esslinger, C., Sauer, C., Hagenhoff, M., Ulferts, J., Gallhofer, B. & Kirsch, P. (2013). Neuronal correlates of social cognition in borderline personality disorder. Social Cognitive and Affective Neuroscience, 8, 531-537. https://doi.org/10.1093/scan/nss028 2. Zazio, A., Guidali, G., Rossi, R., Bolognini, N. & Bortoletto, A. (2023). Cortical plasticity of the tactile mirror system in borderline personality disorder, in principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/sqnwd
| Cortical plasticity of the tactile mirror system in borderline personality disorder | Agnese Zazio, Giacomo Guidali, Roberta Rossi, Nadia Bolognini, Marta Bortoletto | <p>People with borderline personality disorder (BPD) show alterations in empathic abilities, which may involve automatic simulation processes relying on mirror-like mechanisms in the somatosensory domain. In the tactile mirror system, the observat... | Life Sciences, Social sciences | Chris Chambers | 2023-01-05 21:50:15 | |||
27 Nov 2023
STAGE 1
Cortical voice processing in Autism Spectrum DisorderIs voice processing impacted in Autism Spectrum Disorder?Recommended by Grace Edwards and D. Samuel Schwarzkopf based on reviews by 2 anonymous reviewers
Vocal sounds, including both speech and non-speech sounds, have been found to activate the Superior Temporal Sulci and Gyri in comparison to non-vocal sounds. These regions, termed Temporal Voice Areas (TVAs), are considered to be involved in early voice processing and therefore critical for social interaction. TVA activation has been examined in Autism Spectrum Disorder (ASD) to determine if the characteristic difficulties in social communication and interaction are linked to an impaired early voice processing. Using functional magnetic resonance imaging (fMRI), one study found typical brain activation in TVAs for 15 out of 16 autistic participants (Schelinski et al., 2016), whereas another found atypical activation in 4 out of 5 autistic participants (Gervais et al., 2004).
Here, the inconsistencies in the previous literature propel Gautier et al. (2023) to examine brain activation of TVAs with a larger sample size (26 ASD and 26 non-ASD participants). Gautier et al. (2023) will present vocal sounds and non-vocal sounds to both groups of participants during fMRI and predict that fewer participants in the ASD group will show a preferential response to voices in TVAs compared to the non-ASD group. These results would suggest that symptoms of ASD interfere with early stages of social interaction, at the level of voice processing.
This Stage 1 manuscript was evaluated in an initial round by the co-recommenders and another two rounds of in-depth review by two expert reviewers. With these revisions, the recommenders judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/538m4
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1. Gautier, R., Houy-Durand, E., Barantin, L., Briend, F. & Latinus, M. (2023). Cortical voice processing in Autism Spectrum Disorder. In principle acceptance of Version 4 by Peer Community in Registered Reports. https://osf.io/538m4 | Cortical voice processing in Autism Spectrum Disorder | Raphaël Gautier, Emmanuelle Houy-Durand, Laurent Barantin, Frederic Briend, Marianne Latinus | <p>Voice processing is central to social functioning. A specific brain response to vocal sounds has been described and extensively characterized in the general population but remains critically unexplored in Autism Spectrum Disorder (ASD), a condi... | Life Sciences | Grace Edwards | 2023-03-28 09:51:48 | |||
21 Apr 2024
STAGE 1
Cross-cultural relationships between music, emotion, and visual imagery: A comparative study of Iran, Canada, and Japan [Stage 1 Registered Report]Testing cross-cultural difference in the emotionality and visual associations of musicRecommended by D. Samuel Schwarzkopf based on reviews by Elena Karakashevska, Juan David Leongómez and Nadine Dijkstra
For many of us, music is far more than an auditory experience. It can trigger emotional reactions, evoke memories, and wide-ranging associations with other sensory modalities and cognitive states. Music also varies between different cultures in several ways. It remains unclear in how far the broader associations music has differs between cultural contexts, both in terms of the music itself and the listener. This study by Hadavi et al. (2024) seeks to better understand these relationships. Using an online survey targeted at 72 participants from anglophone Canada, Farsi-speaking Iran, and Japan (24 from each location), the researchers aim to address two straightforward hypotheses.
First, does faster tempo of music increase ratings of emotional arousal? Second, do participants match faster tempo music with denser visual line patterns? This latter measure aims to quantify the visual imagery evoked by the musical pieces. Imagery is a loaded term that is not used consistently across the cognitive neuroscience literature; one could argue that what the researchers are actually here is in fact mainly an association between tempo and a visual representation of tempo (or frequency). It certainly seems doubtful that persons listening to a piece of music will form a mental image of a bundle of horizontal lines. Yet, irrespective of how to interpret this experimental variable, it quantifies something about the impression listeners have when experiencing music, and whether these associations differ cross-culturally. The experiment has a balanced design, incorporating excerpts of musical pieces from each of the three cultural contexts, including both solo and group music. While the sample size is comparably low, given the online nature of data collection, it is based on a power analysis and relatively large expected effect sizes.
The proposed study was evaluated by three expert reviewers and the recommender over three rounds of in-depth review, plus a final round ironing out smaller issues. Reviewer Juan David Leongómez was recruited as a co-author after the first round of revisions. Following this process, the recommender decided that the manuscript met Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/zdnkm
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References
1.Hadavi, S., Kuroda, J., Shimozono, T., Leongómez, J. D. & Savage, P. E. (2024). Cross-cultural relationships between music, emotion, and visual imagery: A comparative study of Iran, Canada, and Japan. In principle acceptance of Version 6 by Peer Community in Registered Reports. https://osf.io/zdnkm | Cross-cultural relationships between music, emotion, and visual imagery: A comparative study of Iran, Canada, and Japan [Stage 1 Registered Report] | Shafagh Hadavi, Junji Kuroda, Taiki Shimozono, Juan David Leongómez, Patrick E. Savage | <p>Many people experience emotions and visual imagery while listening to music. Previous research has identified cross-modal associations between musical and visual features as well as cross-cultural links between music and emotion and between mus... | | Humanities, Social sciences | D. Samuel Schwarzkopf | 2023-03-01 02:48:54 | ||
14 Aug 2023
STAGE 1
Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception?Understanding oscillatory correlates of pain expectationRecommended by Gemma Learmonth based on reviews by Zoltan Dienes, Chris Chambers and Markus Ploner
Recent studies using an EEG frequency tagging approach have reported modulations of alpha, beta and theta bands at the stimulation frequency during nociceptive/painful thermal stimulation compared to non-nociceptive/non-painful vibrotactile stimulation. Prior expectations of the intensity of upcoming painful stimuli are known to strongly modulate the subjective experience of those stimuli. Thus, modulating the expectation of pain should result in a change in the modulation of oscillations if these factors are indeed linked.
In this study, Leu, Glineur and Liberati will modulate expectations of pain (low or high intensity) prior to delivering thermal cutaneous stimulation (low, medium or high intensity). They will record how intense participants expect the pain to be, and how intense they felt it to be, as well as record EEG to assess oscillatory differences across the expectation and intensity conditions.
The Stage 1 manuscript was reviewed over 5 rounds by 3 reviewers. Based on detailed responses to the reviewers’ comments and edits to the Stage 1 report, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance.
URL to the preregistered Stage 1 protocol: https://osf.io/y6fb8
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals: References
1. Leu, C., Glineur, E. & Liberati, G. (2023). Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? In principle acceptance of Version 5 by Peer Community in Registered Reports. https://osf.io/y6fb8 | Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? | Chiara Leu, Esther Glineur, Giulia Liberati | <p>A promising stream of investigations is targeting ongoing neural oscillations and whether their modulation could be relevant for the perception of pain. Specifically, sustained periodic thermonociceptive stimuli have been shown to modulate ongo... | Medical Sciences | Gemma Learmonth | Zoltan Dienes | 2023-03-15 14:41:03 | ||
Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? A Registered ReportUnderstanding oscillatory correlates of pain expectationRecommended by Gemma Learmonth based on reviews by Zoltan Dienes, Chris Chambers and Markus Ploner
Recent studies using an EEG frequency tagging approach have reported modulations of alpha, beta and theta bands at the stimulation frequency during nociceptive/painful thermal stimulation compared to non-nociceptive/non-painful vibrotactile stimulation. Prior expectations of the intensity of upcoming painful stimuli are known to strongly modulate the subjective experience of those stimuli. Thus, modulating the expectation of pain should result in a change in the modulation of oscillations if these factors are indeed linked.
In this study, Leu, Glineur and Liberati modulated expectations of pain (low or high intensity) in 40 participants prior to delivering thermal cutaneous stimulation (low, medium or high intensity). They recorded how intense participants expected the pain to be, and how intense they felt it to be, as well as EEG to assess oscillatory differences across the expectation and intensity conditions.
The results confirmed that there was a strong effect of expectation on the perceived stimulus intensity. However, contrary to the hypotheses, this was not reflected in the cortical oscillations. Overall this indicates a possible dissociation between perceived pain and modulation of ongoing oscillations in the theta, alpha and beta bands.
The Stage 2 manuscript was evaluated over one round of in-depth review. Based on detailed responses to the reviewers' comments, the recommender judged that the manuscript met the Stage 2 criteria and awarded a positive recommendation.
URL to the preregistered Stage 1 protocol: https://osf.io/y6fb8
Level of bias control achieved: Level 6. No part of the data or evidence that was used to answer the research question was generated until after IPA. List of eligible PCI RR-friendly journals: References
1. Leu, C., Glineur, E. & Liberati, G. (2023). Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? [Stage 2] Acceptance of Version 2 by Peer Community in Registered Reports. https://osf.io/awrge | Cue-based modulation of pain stimulus expectation: do ongoing oscillations reflect changes in pain perception? A Registered Report | Chiara Leu, Esther Glineur, Giulia Liberati | <p style="text-align: justify;">A promising stream of investigations is targeting ongoing neural oscillations and whether their modulation could be related to the perception of pain. Using an electroencephalography (EEG) frequency tagging approach... | Life Sciences, Medical Sciences | Gemma Learmonth | 2024-01-23 19:35:39 | |||
19 Jun 2024
STAGE 1
Culture-Driven Neural Plasticity and Imprints of Body-Movement Pace on Musical Rhythm ProcessingThe interplay of music, movement, and culture on rhythm processingRecommended by Juan David Leongómez based on reviews by Anne Keitel and 1 anonymous reviewer
The interplay of music, movement, and cultural experience shapes rhythm perception. From bouncing babies and children at play, to tapping, clapping, and dancing, music often triggers synchronous body movements that can influence how we process rhythm. And, at the same time, long-term exposure to specific musical traditions shapes how we perceive and interpret rhythms.
However, direct behavioural and neuroscientific evidence on how these processes occur remains scarce. In this programmatic submission, comprising two complementary Stage 2 reports, Guérin et al. (2024) will investigate how body movements shape the processing of auditory information, and how previous short-term motor practice and long-term cultural experience interact to shape neural and behavioural responses to rhythmic stimuli. The authors will record in separate sessions both electroencephalography (EEG) and hand clapping, in response to rhythms from West/Central Africa. These recordings will be conducted before and after a session in which participants will clap/step to either a three or a four-beat metre that is expected to influence how they interpret a rhythm.
The first Stage 2 report, which will be conducted on African-enculturated participants, aims to demonstrate how body-movement pace flexibly imprints on human sensory processing. The authors build on various theoretical models that emphasise the role of motor production in metre perception, and predict that both neural and behavioural entrainment will improve following movement, matching the rhythmic pattern set by the previous body movements.
The second Stage 2 report aims to uncover how short-term motor practice and long-term cultural experience interact to shape responses to rhythmic stimuli, by integrating short-term motor practice and long-term cultural experience. For this, the authors will test separate groups of participants from distinct cultural backgrounds (African vs. Western-enculturated), which are predicted to show neural and behavioural differences in their preferred metric mapping before body movement, and expect that neural and behavioural entrainment will improve after movement, especially for the metre set by prior movements, and more significantly for the metre common in the participant's culture.
Together, the findings from the planned Stage 2 reports are expected to clarify how long-term cultural background and short-term motor practice imprint onto rhythm processing in humans. This research will enhance our understanding of how cultural experience, body movement, and neural plasticity interact in music processing.
The Stage 1 manuscript was evaluated over three rounds of in-depth review. Based on detailed responses to the reviewers' and recommender's comments, the recommender judged that the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/skuyc
Level of bias control achieved: Level 6. Data collection commenced during the later part of Stage 1 peer review; however, since no changes to the design were made after this point, the risk of bias due to prior data observation remains zero and the manuscript therefore qualifies for Level 6. List of eligible PCI RR-friendly journals: References
Guérin, S. M. R., Coulon, E., Lenc, T., Polak, R., Keller, P. E., & Nozaradan, S. (2024). Culture-Driven Neural Plasticity and Imprints of Body-Movement Pace on Musical Rhythm Processing. In principle acceptance of Version 2.1 by Peer Community in Registered Reports. https://osf.io/skuyc
| Culture-Driven Neural Plasticity and Imprints of Body-Movement Pace on Musical Rhythm Processing | Ségolène M. R. Guérin, Emmanuel Coulon, Tomas Lenc, Rainer Polak, Peter E. Keller, Sylvie Nozaradan | <p>The proposed programmatic registered report aims at capturing direct neuroscientific evidence for the rhythmic, movement-related shaping of auditory information with a cross-cultural perspective. Specifically, West/Central African- and Western-... | | Life Sciences | Juan David Leongómez | Anonymous | 2023-11-30 11:36:06 | |
31 May 2023
STAGE 1
Defacing biases in manual and automated quality assessments of structural MRI with MRIQCThe impact of removing facial features on quality measures of structural MRI scansRecommended by D. Samuel Schwarzkopf based on reviews by Catherine Morgan and Cassandra Gould van Praag
Data sharing is perhaps the most fundamental step for increasing the transparency and reproducibility of scientific research. However, the goals of open science must be tempered by ethical considerations, protecting the privacy and safety of research participants. Bridging this gap causes challenges for many fields, such as human neuroimaging. Brain images, as measured with magnetic resonance imaging (MRI), are unique to the participant and therefore contain identifying information by definition. One way to mitigate the risk to participants arising from public data sharing has been "defacing" the MRI scans, i.e., literally removing the part of the image that contains the face and surrounding tissue, while preserving the brain structure. This procedure however also removes information that is not (or at least minimally) identifiable. It also remains unclear whether defacing the images affects image quality and thus the information necessary for addressing many research questions.
The current study by Provins et al. (2023) seeks to address this question. Leveraging a publicly available "IXI dataset" comprising hundreds of T1-weighted structural MRI scans, they will assess the effect of defacing on manual and automatic estimates of image quality. Specifically, the researchers will compare image quality ratings by experts for a subset of 185 images. They hypothesise that images in which facial features have been removed are typically assigned higher quality ratings. Moreover, using a full data set of 580 images, which have been obtained across three scanning sites, they will also test the impact defacing MRI scans has on automated quality measures obtained with MRIQC software. The results of this study should have important implications for open science policy and for designing the optimal procedures for sharing structural MRI data in an ethical way. For example, if the authors' hypothesis is confirmed, studies relying on MRI quality measures might be better served by a custodianship model where identifiable data is shared under strict conditions, rather than relying on publishing defaced data. More generally, the outcome of this study may have significant legal implications in many jurisdictions.
The Stage 1 manuscript was evaluated at the inital triage stage by the Recommender and PCI:RR team, and another round of in-depth review by two experts. After a detailed response and substantial revisions, the recommender judged the manuscript met the Stage 1 criteria and awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/qcket (under temporary private embargo)
Level of bias control achieved: Level 2. At least some data/evidence that will be used to answer the research question has been accessed and partially observed by the authors, but the authors certify that they have not yet observed the key variables within the data that will be used to answer the research question AND they have taken additional steps to maximise bias control and rigour. List of eligible PCI RR-friendly journals:
References
1. Provins, C., Savary, E., Alemán-Gómez, Y., Richiardi, J., Poldrack, R. A., Hagmann, P. & Esteban, O. (2023). Defacing biases in manual and automated quality assessments of structural MRI with MRIQC, in principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/qcket
| Defacing biases in manual and automated quality assessments of structural MRI with MRIQC | Céline Provins, Yasser Alemán-Gómez, Jonas Richiardi, Russell A. Poldrack, Patric Hagmann, Oscar Esteban | <p>A critical requirement before data-sharing of human neuroimaging is removing facial features to protect individuals’ privacy. However, not only does this process redact identifiable information about individuals, but it also removes non-identif... | Medical Sciences | D. Samuel Schwarzkopf | Catherine Morgan, Cassandra Gould van Praag, Abiola Akinnubi | 2022-11-28 10:59:32 | ||
08 May 2025
STAGE 1
Detecting differences in conscious contents using EEG complexity measuresCan EEG complexity measures discriminate between visual- and auditory-evoked differences in conscious contents?Recommended by Marta Topor based on reviews by Michał Bola, Stefan Wiens, Marcin Koculak and 1 anonymous reviewer
It is a challenging task to measure consciousness. In this project, Ponce de Leon et al. (2025) propose the use of electroencaphalography (EEG) to evaluate the utility of two brain-based complexity measures – Lempel-Ziv complexity and the perturbational complexity index – in the study of conscious content. The overarching aim of the study is to investigate whether these two measures can discriminate between visual and auditory content varying in granularity levels. In addition to the main objectives, the authors plan to conduct a set of exploratory analyses.
The study will provide a significant contribution to the field by attempting to replicate effects previously reported in the literature and extending their generalisability through comparisons across varying configurations of the stimuli. The utility of these complexity measures within conscious content research will be further elucidated through exploratory regression analyses with behavioural variables and ratings of subjective experience.
The Stage 1 manuscript was evaluated over two rounds of in-depth review. Based on detailed responses to the recommender and reviewers' comments, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA). Ethics approval has not been granted yet, so this is a provisional IPA, which will be promoted to a full IPA once ethical approval is in place.
URL to the preregistered Stage 1 protocol: https://osf.io/kdsau (under temporary private embargo)
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA.
List of eligible PCI RR-friendly journals:
References Ponce de Leon, S., Backer, K. C., Monti, M. M., & Yoshimi, J. (2025). Detecting differences in conscious contents using EEG complexity measures. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/kdsau
| Detecting differences in conscious contents using EEG complexity measures | Sergio Ponce de Leon, Kristina C. Backer, Martin M. Monti, Jeff Yoshimi | <p>Measuring consciousness has been a longstanding problem. Even though behavioral responses are commonly used, converging evidence indicates that behavioral responsiveness and behavioral reports about consciousness dissociate from consciousness ... | Life Sciences, Social sciences | Marta Topor | 2024-04-29 01:01:44 | |||
14 Feb 2024
STAGE 1
Detecting DIF in Forced-Choice Assessments: A Simulation Study Examining the Effect of Model MisspecificationDeveloping differential item functioning (DIF) testing methods for use in forced-choice assessmentsRecommended by Amanda Montoya based on reviews by Timo Gnambs and 2 anonymous reviewers
Traditional Likert-type items are commonly used but can elicit response bias. An alternative approach, the forced-choice question, required respondents to rank order all items. Forced-choice questions boast some advantages but required advanced item response theory analysis to generate scores which are comparable across individuals and to evaluate the properties of those scales. However, there has been limited discussion of how to test differential item functioning (DIF) in these scales. In a previous study, Lee et al. (2021) proposed a method for testing DIF.
Here, Plantz et al. (2024) explore the implications of incorrect specification of anchors in DIF detection for forced choice items. The study proposes to use a Monte Carlo simulation which manipulates sample size, equality of sample size across groups, effect size, percentage of differentially functioning items, analysis approach, anchor set size, and percent of DIF blocks in the anchor set. This study aims to answer research questions about the type I error and power of DIF detection strategies under a variety of circumstances, both evaluating whether the results from Lee et al. (2021) generalize to misspecified models and expanding to evaluate new research questions. Results of this study will provide practical implications for DIF testing with forced-choice questions. An important limitation of the study is that it does not explore non-uniform DIF, only uniform DIF. Additionally, as with all simulation studies not all results can only apply to conditions which are simulated and so rely on the realistic selection of simulation conditions. The authors have selected conditions to match reality in circumstances where data is available, but relied on previous simulations in cases when data is not available.
This Stage 1 manuscript was evaluated over two rounds of review by two reviewers with expertise in psychometrics. An additional round of review was completed by the recommender only. Based on the merits of the original submission and responsiveness of the authors to requests from the reviewers, the recommender judged that the manuscript met the Stage 1 criteria and therefore awarded in-principle acceptance (IPA).
URL to the preregistered Stage 1 protocol: https://osf.io/p8awx
Level of bias control achieved: Level 6. No part of the data or evidence that will be used to answer the research question yet exists and no part will be generated until after IPA. List of eligible PCI RR-friendly journals:
References 1. Lee, P., Joo, S.-H. & Stark, S. (2021). Detecting DIF in multidimensional forced choice measures using the Thurstonian Item Response Theory Model. Organizational Research Methods, 24, 739–771. https://doi.org/10.1177/1094428120959822
2. Plantz, J. W., Brown, A., Wright, K. & Flake, J. K. (2024). Detecting DIF in Forced-Choice Assessments: A Simulation Study Examining the Effect of Model Misspecification. In principle acceptance of Version 3 by Peer Community in Registered Reports. https://osf.io/p8awx
| Detecting DIF in Forced-Choice Assessments: A Simulation Study Examining the Effect of Model Misspecification | Jake Plantz, Anna Brown, Keith Wright, Jessica K. Flake | <p>On a forced-choice (FC) questionnaire, the respondent must rank two or more items instead of indicating how much they agree with each of them. Research demonstrates that this format can reduce response bias. However, the data are ipsative, resu... | Social sciences | Amanda Montoya | 2023-09-06 22:43:32 |
