COGITATE Dataset
-The COGITATE dataset is a comprehensive collection of multimodal neuroimaging data, encompassing a total of 262 subjects. COGITATE employs three distinct neuroimaging techniques: fMRI, M-EEG, and iEEG/ECoG.
+The COGITATE dataset is a comprehensive collection of multimodal neuroimaging data, encompassing a total of 256 subjects. COGITATE employs three distinct neuroimaging techniques: fMRI, M-EEG, and iEEG/ECoG.
The MEG modality comprised 102 healthy subjects, also above the age of 18, with no known psychiatric or neurological issues. These participants were sourced from the Centre for Human Brain Health at the University of Birmingham (Birmingham, United Kingdom) and the Center for MRI Research of Peking University (Beijing, China).
-Similarly, the fMRI modality included 122 healthy volunteers, all of whom were above the age of 18 and predominantly right-handed. These participants had no known history of psychiatric or neurological disorders and were recruited from the Yale Magnetic Resonance Research Center (New Haven, CT, United States) and the Donders Centre for Cognitive Neuroimaging (Nijmegen, Netherlands).
+The MEG modality comprised 100 healthy subjects, also above the age of 18, with no known psychiatric or neurological issues. These participants were sourced from the Centre for Human Brain Health at the University of Birmingham (Birmingham, United Kingdom) and the Center for MRI Research of Peking University (Beijing, China).
+Similarly, the fMRI modality included 118 healthy volunteers, all of whom were above the age of 18 and predominantly right-handed. These participants had no known history of psychiatric or neurological disorders and were recruited from the Yale Magnetic Resonance Research Center (New Haven, CT, United States) and the Donders Centre for Cognitive Neuroimaging (Nijmegen, Netherlands).
In contrast, the iEEG modality involved a more specialized cohort of 38 patients diagnosed with pharmaco-resistant focal epilepsy. These participants ranged in age from 10 to 65 years, had an IQ above 70, and met specific health criteria. They were recruited from multiple medical centers specializing in epilepsy treatment, including the Comprehensive Epilepsy Center at New York University (New York, NY, United States), Brigham and Women’s Hospital, Boston Children’s Hospital (Boston, MA, United States), and the University of Wisconsin School of Medicine and Public Health (Madison, WI, United States).
@@ -291,10 +325,18 @@ File Type Glossary
Here are the various file formats used for each modality of the COGITATE dataset along with a short description of them.
Eye Tracking & Behavioral Data
-
-
- Unprocessed/Raw release format - - Filetype: ASC/CSV -
- BIDS Format - - Filetype: ASC/CSV +
-
+
Unprocessed/Raw release format
+-
+
- Filetype: ASC/CSV +
+ -
+
BIDS Format
+-
+
- Filetype: ASC/CSV +
The two eye trackers used within COGITATE are:
1. EyeLink eye tracker
@@ -379,8 +421,25 @@
MR/CT Data
DICOM (.dcm file extension) is a standard format utilized for storing CT (Computed Tomography) scans and MRI (Magnetic Resonance Imaging) data. These files encompass not only the image data but also essential metadata, including imaging parameters.
NIFTI (.nii.gz file extension) serves as another format employed for a subset of subjects where our standard procedure encountered challenges. With the exception of the MR and CT scans for 12 subjects within the iEEG data, all other datasets of similar nature are stored in DICOM format. Further details regarding these 12 problematic datasets are available in this section. NIFTI files encapsulate image data alongside metadata concerning spatial orientation, voxel dimensions, and additional imaging parameters.
+fMRI Data
+-
+
-
+
Unprocessed/Raw release format
+-
+
- Filetype: DICOM +
+ -
+
BIDS Format
+-
+
- Filetype: NIFTI +
+
DICOM (Digital Imaging and Communications in Medicine), with the file extension .dcm, is a standard format for storing anatomical and functional MRI (Magnetic Resonance Imaging) scans. These files contain both image data and crucial metadata, including imaging parameters.
+The BIDS format includes anatomical and functional images stored in NIFTI (Neuroimaging Informatics Technology Initiative) format with the .nii.gz file extension. These NIFTI files encompass image data, spatial orientation, voxel dimensions, and imaging parameters such as sequence details, field of view, repetition time (TR), echo time (TE), acquisition timing, and more.
Data Acquisition
-The Cogitate dataset encompasses three distinct neuroimaging modalities, along with synchronized eye-tracking and behavioral data linked to each of these modalities. Here we detail the acquisition protocol for each modality in the corresponding data release: M-EEG, iEEG
+The Cogitate dataset encompasses three distinct neuroimaging modalities, along with synchronized eye-tracking and behavioral data linked to each of these modalities. Here we detail the acquisition protocol for each modality in the corresponding data release: M-EEG, iEEG, and fMRI.
Stimuli
Stimuli belonged to four categories that naturally fell into two groups that were clearly distinct from each other: pictures (20 faces and 20 objects) and symbols (20 letters and 20 false-fonts). Face stimuli were created using the FaceGen Modeler 3.1 program and object stimuli were taken from the Object Databank (Tarr, 1996). Faces and objects were grey-scaled (RGB: 125, 125, 125), and manipulated to have similar size and equal luminance using the SHINE toolbox (Willenbockel et al., 2010). Equal proportions of male and female faces were presented. They all had hair and belonged to different ethnicities (e.g., Caucasian, Asian, African, American) to facilitate face individuation. The orientation of the stimuli was manipulated, such that half of the stimuli from each category had a side view (30° and -30°) and the other half a front view (0°). All letter stimuli and false fonts were generated with MAXON CINEMA 4D Studio (RC-R20) 20.059 on macOS 10.14, appearing in gray (RGB: 125, 125, 125). Three views were rendered for each font set (real font, false/pseudo font) at 0°, 30° and -30° horizontal viewing angle with the following settings: Extrusion depth 9.79% of character height, camera distance 5.65 times character height and 18° above the center of the letter (High Angle), with a simulated focal length of 135 mm (35 mm equiv.). All stimuli were presented on a rectangular aperture at an average visual angle of 6 ̊ by 6 ̊.
Procedure
@@ -419,6 +478,8 @@Procedure
Trial division was pseudorandomized with respect to two factors: the assignment of non-target stimuli to a miniblock, and the number of trials with specific durations within each task-relevance level. The assignment of non-targets was restricted by two constraints: i) Each stimulus was presented an equal number of times as a non-target stimulus (and as an irrelevant one) throughout the experiment. ii) Non-targets were, by definition, different from the targets in that specific miniblock. To meet these constraints, two pseudorandomized versions were made where the targets and non-target stimuli of each miniblock were determined. Three additional versions were prepared for each target/non-target division to balance the duration division between categories and orientation. In each version, slight imbalances (maximum difference of four trials) existed between the different durations for each condition (targets, non-targets, irrelevant). This imbalance was a necessary consequence of keeping the balance in duration and category across orientation and across categories as the numbers were not evenly divisible by three. The imbalances canceled out over the entire experiment so there was a perfect duration balance across the experiment: an equal amount of stimuli were presented for 500, 1000 and 1500 ms durations. The imbalances between durations within each relevant condition were balanced across the three versions. In total, six different versions were assigned to the subjects in consecutive order, repeating every six subjects. To get more details, please see the “Trial counts” section in COGITATE Preregistration, v4.
iEEG Related Modifications of the Design
Only half of the stimuli were used as targets for the iEEG experiments. The selection of target faces kept the balance of gender and ethnicity. Letters were chosen in equal amounts from the first and second parts of the alphabet, and the corresponding false-fonts were used. All stimuli were presented as relevant non-targets and irrelevant stimuli, matching the designs of the aforementioned procedure.
+fMRI Related Modifications of the Design
+For the fMRI experiment, timing was adjusted as follows: the random jitter between trials was increased (mean inter-trial interval of 3 s, jittered 2.5-10 s, with truncated exponential distribution), with each trial lasting approximately 5.5 s. This modification served to improve decoding and fitting of GLMs to the fMRI data (Glover, 1999). Second, to increase detection efficacy for amplitude-based analyses, three additional baseline periods (blank screen) of 12 s each were included per run (total = 24). The identity of the stimuli was randomized with the constraint that they appeared equally across durations and tasks conditions.
M-EEG Data Acquisition
M-EEG recordings were acquired at the Centre for Human Brain Health (CHBH) of University of Birmingham in the United Kingdom, and at the Center for MRI Research of Peking University (PKU) in China.
Hardware
@@ -566,8 +627,12 @@Deviations and Missing Data
For the subjects listed below, the anatomical MRI scans are missing. Consequently, there is no folder containing the “sub-CX???_ses-1_trans.fif” file under /derivatives/coreg for these subjects. Instead, the FreeSurfer standard template (fsaverage) was used.Subject_ID: CA101, CA102, CA104, CA110, CA111, CA152
Attention 2: -The following subjects are missing “Exit Questionnaires”:
-Subject_ID: CB003, CB035
+The “Exit Questionnaire” is not available for CB003. +Attention 3: +The “RestinEO” file is missing for the ET data of the below subjects:
+Subject_ID: CA172, CA176, CB999
+Attention 4: +The ET data is missing for CA151.
iEEG Data Acquisition
iEEG recordings were obtained from patients with pharmacologically resistant epilepsy undergoing invasive electrophysiological monitoring at the Comprehensive Epilepsy Center at New York University (NYU) Langone Health Center, Brigham and Women’s Hospital, Children’s Hospital Boston (Harvard Medical School), and University of Wisconsin School of Medicine and Public Health (WU).
Hardware
@@ -867,6 +932,95 @@Deviations and Missing Data
The following subjects are missing “Exit Questionnaires”:Subject_ID: CF103, CF105, CF106, CF110, CF112, CF113, CF116, CF117, CF119, CF120, CF121, CF122, CF124, CF125, CF126
Please note: +MR and CT data were collected for the subjects at Brigham and Women’s Hospital and Children’s Hospital Boston. However, due to the data protection policies, they are not included in the COGITATE Data Release.
++
fMRI Data Acquisition
+Imaging was conducted at the Yale Magnetic Resonance Research Center (MRRC) in New Haven and at the Donders Centre for Cognitive Neuroimaging (DCCN), of Radboud University Nijmegen in the Netherlands.
+Hardware
+Both centers have a Siemens 3T Prisma research scanner (Siemens, Erlangen, Germany) with high-performance gradients (max. gradient strength 80 mT/m, 200 mT/m/s rise time, 100% duty cycle), 32-channel parallel imaging, and a 32-channel head coil.
+Anatomical MRI Data Acquisition
+Anatomical images were acquired on a 3T Prisma scanner using a 32-channel head coil. The images were captured using a T1-weighted magnetization prepared rapid gradient echo sequence (MP-RAGE) with a GRAPPA acceleration factor of 2, TR/TE of 2300/3.03 ms, a voxel size of 1 mm isotropic, a field of view (FOV) of 256 x 256 x 192 mm (192 slices), and a 8° flip angle.
+Functional MRI Data Acquisition
+Functional images were acquired on the same scanner using a whole-brain T2-weighted multiband-4 sequence with TR/TE of 1500/39.6 ms, a voxel size of 2.019 x 2.019 x 2 mm, a 75° flip angle, an A/P phase encoding direction, a FOV of 210 x 210 x 136 mm (68 slices), a bandwidth (BW) of 2090 Hz/Px and different number of volumes. A single-band reference (SBRef) image was acquired before each run. The first three volumes of each run were discarded to allow for signal stabilization. Additionally, to correct for susceptibility distortions, extra scans using the same T2-weighted sequence with inverted phase encoding direction (inverted RO/PE polarity) were collected while the subject was at rest at multiple points throughout the experiments.
+Behavioral Data Acquisition
+The task was run on Matlab (DCCN and Yale: R2019b) using Psychtoolbox v.3 (Pelli, 1997). The fMRI version was executed on an MSI laptop at Yale and a Dell Desktop PC at DCCN. Both centers were equipped with behavioral testing apparatus, including a visual display projected onto a mirror fixed to the head coil, a button response system, and software for displaying and recording behavioral tasks synchronized with MRI data acquisition. At DCCN, stimuli were presented on an MRI-compatible Cambridge Research Systems BOLDscreen 32” IPS LCD monitor (1920 x 1080 pixels, 60 Hz; viewing distance ~134 cm). At Yale, stimuli were projected using a Psychology Software Tools Hyperion projection system (1920 x 1080 pixels, 60 Hz; viewing distance ~113 cm). Subjects responded with their right hand using a 2x2 Current Designs response box at Yale and a 1x4 Current Designs response box at DCCN.
+Eye Tracking Data Acquisition
+The fMRI labs used the MR-compatible EyeLink 1000 Plus eye-tracker system (SR Research Ltd., Ottawa, Canada) to collect data at 1000 Hz. Eye position was monitored monocularly, with data acquired from the left eye at DCCN and the right eye at Yale. Pupil and corneal reflection were sampled at 1000 Hz. A nine-point calibration was performed at the beginning of each experiment session and recalibrated as needed at the beginning of each block/run.
+Behavioral Data Code Scheme
+The behavioral code scheme aligns closely with the M-EEG modality, as detailed in this section.
+Eye Tracker Data Code
+The eye tracker code scheme used in fMRI follows a structure akin to the one outlined for M-EEG data. More detailed explanations can be found here.
+Task (tfMRI)
+For the fMRI protocol, there were 8 runs, each containing 4 blocks with 17-19 trials per block. Each trial lasted approximately 5.5 seconds, with 16 non-targets (4 per category) and 1-3 targets, totaling 576 trials. The timing was adjusted by increasing the random jitter between trials (mean inter-trial interval of 3 seconds, jittered 2.5-10 seconds with a truncated exponential distribution). Breaks and pauses were included between runs and blocks, with runs separated by short breaks. This process involved stopping and restarting the EPI acquisition sequences.
+| + | + | + | + | + |
|---|---|---|---|---|
| Task | +Runs | +Blocks | +Trials | +Total trials | +
| Experiment 1 | +8 | +4 | +17-19 per block | +576 | +
Full Structure of Session
+The complete session structure includes:
+Anatomical scan: Taken when subjects are at rest and not engaged in the experiment. +Single-band reference scan for distortion correction (DC1_SBRef): Conducted when subjects are at rest. +Distortion correction scans (DC1): Performed when subjects are at rest. +Single-band reference scan for functional scans (DurR1_SBRef): Administered at the start of the experiment. +Functional scans (DurR1) +Single-band reference scan for functional scans (DurR2_SBRef): Done while the subjects are at rest. +Functional scans (DurR2) +Single-band reference scan for functional scans (DurR3_SBRef) +Functional scans (DurR3) +Single-band reference scan for functional scans (DurR4_SBRef) +Functional scans (DurR4) +Single-band reference scan for distortion correction (DC2_SBRef): Conducted when subjects are at rest. +Distortion correction scans (DC2): Done when subjects are at rest. +Single-band reference scan for functional scans (DurR5_SBRef): Performed during the experiment. +Functional scans (DurR5) +Single-band reference scan for functional scans (DurR6_SBRef): Conducted during the experiment. +Functional scans (DurR6) +Single-band reference scan for functional scans (DurR7_SBRef) +Functional scans (DurR7) +Single-band reference scan for functional scans (DurR8_SBRef) +Functional scans (DurR8) +Single-band reference scan for distortion correction (DC3_SBRef): Conducted when subjects are at rest. +Distortion correction scans (DC3): Performed when subjects are at rest.
+For a comprehensive understanding of the fMRI session protocol, please refer to the fMRI Standard Operating Procedure document.
+Inclusion Criteria
+For the iEEG studies, subjects were 10-65 years old, able to provide informed consent, had IQ > 70, fluent in English, with self-reported normal hearing, normal or corrected-to-normal vision, and cognitive and language abilities within or above the normal range in formal neuropsychological testing performed before surgery. They must not have had an electrographic seizure within 3-hours prior to testing.
+Quality Check
+A comprehensive quality assessment was conducted on the iEEG data. The data underwent manual annotation by epileptologists, excluding channels within the epileptic onset zone, as well as those exhibiting artifacts or showing complete flatness due to electrode contact issues. Channel rejection was independently performed by both the data monitoring and iEEG teams, with results compared to ensure consistency. Additionally, electrode reconstruction was verified to align with subjects' CT scans. Finally, we inspected for significant disturbances in the spectra.
+Exclusion Criteria
+Subjects who were unable to complete a sufficient number of trials due to excessive muscular artifacts, movement, noisy recordings, or a decision by the subject to terminate the experiment were excluded. In addition, data was also excluded if it did not pass any of the pre-defined data quality checks.
+Deviations and Missing Data
+Attention 1: +The iEEG data for the seven subjects listed below do not have the correct sampling rate in the original EDF files. The correct sampling rate for these subjects is 2048 Hz. The raw data has not been updated with the correct sampling rate to avoid alignment issues when constructing the events. However, the same measurements in the BIDS release, provided in the BrainVision format, have the correct sampling rate as expected.
+Subject_ID: CE106, CE107, CE108, CE112, CE115, CE118, CE120
+Attention 2: +Subject “CG102” is missing the first “Run” and contains only DurR2, DurR3, DurR4, DurR5.
+Attention 3: +The following subjects are missing “Exit Questionnaires”:
+Subject_ID: CF103, CF105, CF106, CF110, CF112, CF113, CF116, CF117, CF119, CF120, CF121, CF122, CF124, CF125, CF126
+Please note: MR and CT data were collected for the subjects at Brigham and Women’s Hospital and Children’s Hospital Boston. However, due to the data protection policies, they are not included in the COGITATE Data Release.