Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/henneysq/meg-ahat
Browse files Browse the repository at this point in the history
  • Loading branch information
@henneysq
henneysq committed Mar 13, 2024
2 parents e19f2c2 + 2c6e26d commit a042d11
Show file tree
Hide file tree
Showing 3 changed files with 87 additions and 24 deletions.
24 changes: 19 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ this project aims to conduct data analysis on BIDS-compliant data from the get-g
the source data is converted to BIDS prior to data analysis. However, the conversion
takes place in two steps to incrementally enrich the data structure.

Data analysis is conducted solely on the completely enriched data set (raw2 below).

### Data Versions <a name="data-versions"></a>

The project pilot-data exists in several versions that can all be accessed in the MEG-AHAT
Expand All @@ -50,18 +52,29 @@ include:
respective recording devices and includes MEG scans, simultaneous eye-tracking
and presentation log-files, polhemous positional data, and MRI scans.
2. First iteration raw data (`/raw1/`) contains a BIDS-compliant version with minimal
changes
changes. One implication of this is that events are not yet alligned, as the recorded
behavioural data is not set on a time axis at the point of recording.
2. Second iteration raw data (`/raw2/`) contains a BIDS-compliant version of the data
in which the behevioural data recorded along with MEG has been moved to the /meg
directory, and files will are enriched and aligned in time.
directory, and MEG, eyetrack, and behavioural events are enriched and aligned in time.

### Converting Data <a name="converting-data"></a>

Converting data between versions is non-trivial and requires both commonly used public
tools and custom scripts. This projects uses the FieldTrip tool [`data2bids`](https://github.com/fieldtrip/fieldtrip/blob/master/data2bids.m)
and custom matlab scripts.

Source data is converted to raw1 using the
[`/data_curration/convert_source_to_raw1.m`](data_curration/convert_source_to_raw1.m) script.
[`/data_curration/convert_source_to_raw1.m`](https://github.com/henneysq/meg-ahat/blob/main/data_curration/convert_source_to_raw1.m) script.

Raw1 data is converted to raw2 using the
[`/data_curration/convert_raw1_to_raw2.m`](data_curration/convert_raw1_to_raw2.m) script.
[`/data_curration/convert_raw1_to_raw2.m`](https://github.com/henneysq/meg-ahat/blob/main/data_curration/convert_raw1_to_raw2.m) script.

#### Eyetrack data

Currently, eyetracking data is not covered by BIDS, but is added with the [BIDS extension proposal BEP020](https://docs.google.com/document/d/1eggzTCzSHG3AEKhtnEDbcdk-2avXN6I94X8aUPEBVsw/edit#heading=h.9tphvz6ot0j1).

With this change, eyetracker data is allocated to the `eyetrack/` scans directory and has the `_eyetrack` suffix.

#### Directory Layout

Expand All @@ -74,10 +87,12 @@ pilot-data
| |-- ses-001
| | |-- anat
| | |-- beh
| | |-- eyetrack
| | |-- meg
| | `-- sub-099_ses-001_scans.tsv
| |-- ses-002
| | |-- beh
| | |-- eyetrack
| | |-- meg
| | `-- sub-099_ses-002_scans.tsv
| `-- sub-099_sessions.tsv
Expand All @@ -99,7 +114,6 @@ pilot-data
`-- sub-099_ses-002-eyetracking.edf
```


## Experiment Management <a name="experiment-management"></a>

### Dependencies <a name="dependencies"></a>
Expand Down
62 changes: 48 additions & 14 deletions data_curration/convert_source_to_raw1.m
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@
% | |-- ses-001
% | | |-- anat
% | | |-- beh
% | | |-- eyetrack
% | | |-- meg
% | | `-- sub-099_ses-001_scans.tsv
% | |-- ses-002
% | | |-- beh
% | | |-- eyetrack
% | | |-- meg
% | | `-- sub-099_ses-002_scans.tsv
% | `-- sub-099_sessions.tsv
Expand All @@ -52,11 +54,13 @@
%% Add util dir to path
addpath('/project/3031004.01/meg-ahat/util')

%% Setup Fieldtrip
configure_ft

%% Set pilot data directory
data_dir = '/project/3031004.01/pilot-data';
diaryfile = strcat(data_dir, '/source_to_raw1.log');
diary (diaryfile)

%% Setup Fieldtrip
configure_ft

% Sebject (defined as cell array for future compatibility)
subj = {
Expand All @@ -65,7 +69,7 @@

% Tasks in the two runs
tasks = {
'visual';
'visualattention';
'workingmemory'
};

Expand Down Expand Up @@ -112,15 +116,18 @@
% HEAD POSITION (Polhemus)
%
% Specify source file path
head_pos_file = sprintf('sub-%s.pos', subj{subindx})
head_pos_file = sprintf('sub-%s.pos', subj{subindx});
source_ses1_dir = sprintf('%s/source/sub-%s/ses-001/', data_dir, subj{subindx});
head_pos_d = strcat(source_ses1_dir, head_pos_file);

% Specify destination file path
head_pos_file_raw1 = sprintf('sub-%s_ses-001_headshape.pos', subj{subindx})
head_pos_file_raw1 = sprintf('sub-%s_ses-001_headshape.pos', subj{subindx});
source_ses1_raw1_dir = sprintf('%s/sub-%s/ses-001/meg/', general_cfg.bidsroot, subj{subindx});
head_pos_raw1_d = strcat(source_ses1_raw1_dir, head_pos_file_raw1);

% Check that destination folder exists
mkdir (source_ses1_raw1_dir)

% Copy source to raw1 destination
copyfile(head_pos_d, head_pos_raw1_d);

Expand Down Expand Up @@ -157,6 +164,11 @@
cfg.dataset_description.BIDSVersion = 'v1.5.0';
cfg.TaskDescription = 'The experiment consisted of visual flicker stimulation combined with a visual attention discrimination task and an arithmetic task.';
cfg.task = 'flicker';
cfg.PowerLineFrequency = 50;
cfg.DewarPosition = 'upright';
cfg.SoftwareFilters = 'n/a';
cfg.DigitizedLandmarks = true;
cfg.DigitizedHeadPoints = false;

% Convert to BIDS in raw1
try
Expand All @@ -178,8 +190,8 @@
cfg.dataset_description.BIDSVersion = 'unofficial extension';
cfg.method = 'convert'; % the eyelink-specific format is not supported, convert it to plain TSV
cfg.dataset = fullfile(d.folder, d.name);
cfg.datatype = 'eyetracker';
cfg.suffix = 'meg';
cfg.datatype = 'eyetrack';
cfg.suffix = 'eyetrack';
cfg.Manufacturer = 'SR Research';
cfg.ManufacturerModelName = 'Eyelink 1000';
cfg.TaskDescription = 'The experiment consisted of visual flicker stimulation combined with a visual attention discrimination task and an arithmetic task.';
Expand All @@ -193,6 +205,12 @@
%
% Iterate over runs
%
% First, create a /beh/ dir
beh_dir = sprintf('%s/sub-%s/ses-00%d/beh/', general_cfg.bidsroot, subj{subindx}, sesindx);
if not(isfolder(beh_dir))
mkdir(beh_dir)
end

for runindx=1:2
%
% BEHAVIOURAL DATA
Expand All @@ -205,11 +223,17 @@
d = dir(strcat(behav_dir, filename));
% Load the behavioural data as a table
log = readtable(fullfile(d.folder, d.name));

% Add BIDS required columns
log_len = size(log, 1);
log.onset = repmat("n/a", log_len, 1);
log.duration = repmat("n/a", log_len, 1);
log = log(:, [end-1:end 1:end-3]);

% Specify the output path in the raw1 directory
out_dir = sprintf('%s/sub-%s/ses-00%d/beh/', general_cfg.bidsroot, subj{subindx}, sesindx);
out_file = sprintf('sub-%s_ses-00%d_run-00%d_beh.tsv', subj{subindx}, sesindx, runindx);
out_d = fullfile(strcat(out_dir, out_file));
filename = sprintf('sub-%s_ses-00%d_run-00%d_task-%s_events', subj{subindx}, sesindx, runindx, tasks{runindx});
out_file = strcat(filename, '.tsv');
out_d = fullfile(strcat(beh_dir, out_file));
% Write the behavioural data to destination
writetable(log,out_d, 'filetype','text', 'delimiter','\t')

Expand All @@ -236,13 +260,23 @@
json_txt = jsonencode(info, "PrettyPrint", true);

% Specify the destination
out_json_file = sprintf('sub-%s_ses-00%d_run-00%d_beh.json', subj{subindx}, sesindx, runindx);
out_json_d = fullfile(strcat(out_dir, out_json_file));
out_json_file = strcat(filename, '.json');
out_json_d = fullfile(strcat(beh_dir, out_json_file));
% Write the file
fid = fopen(out_json_d,'w');
fprintf(fid,'%s',json_txt);
fclose(fid);

end % for each run
end % for each ses
end % for each subject
end % for each subject

% Write entries to the .bidsignore file
bidsignore_text = ['*eyetrack.tsv' newline '*eyetrack.json' newline];
% Specify the destination
bidsignore_file = fullfile(strcat(general_cfg.bidsroot, '/.bidsignore'));
% Write the file
fid = fopen(bidsignore_file, 'w');
fprintf(fid, '%s', bidsignore_text);

diary off
25 changes: 20 additions & 5 deletions documentation/DATA_MANAGEMENT.stub
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ this project aims to conduct data analysis on BIDS-compliant data from the get-g
the source data is converted to BIDS prior to data analysis. However, the conversion
takes place in two steps to incrementally enrich the data structure.

Data analysis is conducted solely on the completely enriched data set (raw2 below).

### Data Versions

The project pilot-data exists in several versions that can all be accessed in the MEG-AHAT
Expand All @@ -17,18 +19,29 @@ include:
respective recording devices and includes MEG scans, simultaneous eye-tracking
and presentation log-files, polhemous positional data, and MRI scans.
2. First iteration raw data (`/raw1/`) contains a BIDS-compliant version with minimal
changes
changes. One implication of this is that events are not yet alligned, as the recorded
behavioural data is not set on a time axis at the point of recording.
2. Second iteration raw data (`/raw2/`) contains a BIDS-compliant version of the data
in which the behevioural data recorded along with MEG has been moved to the /meg
directory, and files will are enriched and aligned in time.
directory, and MEG, eyetrack, and behavioural events are enriched and aligned in time.

### Converting Data

Converting data between versions is non-trivial and requires both commonly used public
tools and custom scripts. This projects uses the FieldTrip tool [`data2bids`](https://github.com/fieldtrip/fieldtrip/blob/master/data2bids.m)
and custom matlab scripts.

Source data is converted to raw1 using the
[`/data_curration/convert_source_to_raw1.m`](data_curration/convert_source_to_raw1.m) script.
[`/data_curration/convert_source_to_raw1.m`](https://github.com/henneysq/meg-ahat/blob/main/data_curration/convert_source_to_raw1.m) script.

Raw1 data is converted to raw2 using the
[`/data_curration/convert_raw1_to_raw2.m`](data_curration/convert_raw1_to_raw2.m) script.
[`/data_curration/convert_raw1_to_raw2.m`](https://github.com/henneysq/meg-ahat/blob/main/data_curration/convert_raw1_to_raw2.m) script.

#### Eyetrack data

Currently, eyetracking data is not covered by BIDS, but is added with the [BIDS extension proposal BEP020](https://docs.google.com/document/d/1eggzTCzSHG3AEKhtnEDbcdk-2avXN6I94X8aUPEBVsw/edit#heading=h.9tphvz6ot0j1).

With this change, eyetracker data is allocated to the `eyetrack/` scans directory and has the `_eyetrack` suffix.

#### Directory Layout

Expand All @@ -41,10 +54,12 @@ pilot-data
| |-- ses-001
| | |-- anat
| | |-- beh
| | |-- eyetrack
| | |-- meg
| | `-- sub-099_ses-001_scans.tsv
| |-- ses-002
| | |-- beh
| | |-- eyetrack
| | |-- meg
| | `-- sub-099_ses-002_scans.tsv
| `-- sub-099_sessions.tsv
Expand All @@ -64,4 +79,4 @@ pilot-data
|-- pilot002_3031000.01_20231214_01.ds
|-- sub-099_ses-002-eyetracking.asc
`-- sub-099_ses-002-eyetracking.edf
```
```

0 comments on commit a042d11

Please sign in to comment.