Skip to content

Commit

Permalink
Merge pull request #23 from DouweHorsthuis/dashboard
Browse files Browse the repository at this point in the history 
first dashboard update
  • Loading branch information
@DouweHorsthuis
DouweHorsthuis authored Oct 17, 2022
2 parents bce5fc9 + 4d68bc9 commit 90e0d7d
Show file tree
Hide file tree
Showing 39 changed files with 3,014 additions and 203 deletions.
31 changes: 31 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,19 @@ home_path = 'path_where_to_load_in_pc';
save_path = 'path_where_to_save_in_pc';
blocks = 5;
```
10/14/2022 update
We added some quality control functions here. Both are 100% optional they will be used if the following variable are `'yes'`:
``` matlab
readme_file ='yes';
eye_tracking ='yes';
```

These functions come originally from [this project](https://github.com/DouweHorsthuis/EEG-quality-analysis) that we created to test the data while one collects it. They are also explained in-dept in the readme file there.

In short:
The `readme_to_EEG` function searches for a readme file (.txt file) that we use in our lab to describe data collection.
- this function can be addapted but currently is functioning only for the template of the readme files we use in our lab
The `edf_to_figure` function uses edf files created by our eye-tracker (sr-research eyelink 1000plus) and creates a gaze plot so you can see where the participant looked throughout the experiment.

[Back to top](#eeg-pipeline-using-eeglab)

Expand Down Expand Up @@ -319,6 +332,22 @@ downsample_to=256; % what is the sample rate you want to downsample to
lowpass_filter_hz=45; %45hz filter
highpass_filter_hz=1; %1hz filter
```
10/14/2022 update
### plotting_bridged_channels
We added this function, which uses `eBridge`. This is a function that uses the EEG structure and checks if there
are channels that are bridged. The full explanation can be found
[here](https://psychophysiology.cpmc.columbia.edu/software/eBridge/index.html).
Or in [this
paper](https://psychophysiology.cpmc.columbia.edu/pdf/alschuler2013a.pdf)
that was written and resulted in the function. `bridge=eBridge(EEG)`
gives us a structure in which `bridge.Bridged.Labels` gives us the
labels of all the bridged channels. Later we use this to plot a figure
of the location of bridged channels. **note that bridged channels are
not deleted**. In the `plotting_bridged_channels` function we use `eBridge` and plot the locations of the bridged channels.

### plot_deleted_chan_location
This function uses the locations of the deleted channels and creates a scalp map with these locations so one can see if too many channels close to each other are deleted.


[Back to top](#eeg-pipeline-using-eeglab)

Expand Down Expand Up @@ -780,6 +809,8 @@ with flat channels not being flat anymore.
6/17/2021- updating
[D_reref_exclextrn_avgref_ica_autoexcom](#D_reref_exclextrn_avgref_ica_autoexcom),
only deleting eye-components from now on.
10/17/2022
Working on a QA dashboard that will show you both individual subject and group related information to see without hassle how your data look

[Back to top](#eeg-pipeline-using-eeglab)

Expand Down
20 changes: 12 additions & 8 deletions Readme.Rmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,17 @@ changeable, but are pre-tested and worked for multiple publications.
### Publications

The following are the papers that we published using this pipeline.


[Francisco, A. A., Foxe, J. J., Horsthuis, D. J., & Molholm, S. (2022). Early visual processing and adaptation as markers of disease, not vulnerability: EEG evidence from 22q11. 2 deletion syndrome, a population at high risk for schizophrenia. Schizophrenia, 8(1), 1-12](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8938446/)(unique settings: epochs are −50 to 400ms using a baseline of −50 to 0ms.)

[Francisco, A. A., Berruti, A. S., Kaskel, F. J., Foxe, J. J., &
Molholm, S. (2021). Assessing the integrity of auditory processing and
sensory memory in adults with cystinosis (CTNS gene mutations). Orphanet
Journal of Rare Diseases, 16(1),
1-10.](https://link.springer.com/article/10.1186/s13023-021-01818-0)
(unique settings: re-referenced to TP8 and epochs are −100 to 400ms
using a baseline of −50 to 0ms.)

[Francisco, A. A., Foxe, J. J., Horsthuis, D. J., DeMaio, D., & Molholm,
S. (2020). Assessing auditory processing endophenotypes associated with
Schizophrenia in individuals with 22q11. 2 deletion syndrome.
Expand All @@ -121,13 +131,7 @@ negativity (MMN). NeuroImage: Clinical, 25,
(unique settings: re-referenced to TP8 and epochs are −100ms to 400ms
using a baseline of −100ms to 0ms)

[Francisco, A. A., Berruti, A. S., Kaskel, F. J., Foxe, J. J., &
Molholm, S. (2021). Assessing the integrity of auditory processing and
sensory memory in adults with cystinosis (CTNS gene mutations). Orphanet
Journal of Rare Diseases, 16(1),
1-10.](https://link.springer.com/article/10.1186/s13023-021-01818-0)
(unique settings: re-referenced to TP8 and epochs are −100 to 400ms
using a baseline of −50 to 0ms.)


[Back to top](#eeg-pipeline-using-eeglab)

Expand Down
13 changes: 13 additions & 0 deletions src/A_merge_sets.m
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@
filename = 'the_rest_of_the_file_name'; % if your bdf file has a name besides the ID of the participant (e.g. oddball_paradigm)
home_path = 'path_where_to_load_in_pc'; %place data is (something like 'C:\data\')
blocks = 5; % the amount of BDF files. if different participant have different amounts of blocks, run those participant separate
readme_file ='yes';%'yes' or 'no', if yes using [EEG]= readme_to_EEG(EEG, data_path) to read readme file
eye_tracking ='no';%'yes' or 'no', if yes using edf_to_figure(data_path)to create a figure with avg gaze
if strcmpi(readme_file,'yes') || strcmpi(eye_tracking,'yes')% if yes we need to add the functions to the file path
file_loc=[fileparts(matlab.desktop.editor.getActiveFilename),filesep];
addpath(genpath(file_loc));%adding path to your scripts so that the functions are found
end
for s = 1:length(subject_list)
clear ALLEEG
eeglab
Expand All @@ -25,7 +31,14 @@
EEG = pop_mergeset( ALLEEG, 1:blocks, 0);
end
%adding info to the EEG structure
if strcmpi(readme_file,'yes')
[EEG]= readme_to_EEG(EEG, data_path);
else
EEG.subject = subject_list{s}; %subject ID
end
if strcmpi(eye_tracking,'yes')
edf_to_figure(data_path)
end
%save the bdf as a .set file

EEG = pop_saveset( EEG, 'filename',[subject_list{s} '.set'],'filepath',data_path);
Expand Down
27 changes: 20 additions & 7 deletions src/B_downs_filter_chaninfo_exclchan.m
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,16 @@

% This defines the set of subjects
subject_list = {'some sort of ID' 'a different id for a different particpant'};
eeglab_location = fileparts(which('eeglab')); %needed if using a 10-5-cap
scripts_location = 'C:\\Scripts\'; %needed if using 160channel data
% Path to the parent folder, which contains the data folders for all subjects
home_path = 'the main folder where you store your data';
scripts_location = 'C:\\Scripts\'; %needed if using 160channel data
downsample_to=256; % what is the sample rate you want to downsample to
lowpass_filter_hz=45; %45hz filter
highpass_filter_hz=1; %1hz filter
avg_deleted_data=zeros(1, length(subject_list));
clean_data={'y'}; % if 'y' not only channels but also noisy moments in thedata get cleaned
individual_plots='yes';
%% variables that need to be created
eeglab_location = fileparts(which('eeglab')); %needed if using a 10-5-cap
avg_deleted_data=zeros(1, length(subject_list));
% Loop through all subjects
for s=1:length(subject_list)
fprintf('\n******\nProcessing subject %s\n******\n\n', subject_list{s});
Expand All @@ -34,11 +35,16 @@
EEG = pop_eegfiltnew(EEG, 'hicutoff',lowpass_filter_hz);
EEG = eeg_checkset( EEG );
EEG = pop_saveset( EEG, 'filename',[subject_list{s} '_downft.set'],'filepath', data_path);
%looking for bridged channels
if strcmpi(individual_plots,'yes')
EEG=plotting_bridged_channels(EEG, data_path); %plotting the location of bridged chan
end
close all
EEG.old_n_chan = EEG.nbchan;
old_samples=EEG.pnts;
%adding channel location
if EEG.nbchan >63 && EEG.nbchan < 95 %64chan cap (can be a lot of externals, this makes sure that it includes a everything that is under 96 channels, which could be an extra ribbon)
EEG=pop_chanedit(EEG, 'lookup',[eeglab_location 'plugins\dipfit\standard_BESA\standard-10-5-cap385.elp']); %make sure you put here the location of this file for your computer
EEG=pop_chanedit(EEG, 'lookup',[eeglab_location '\plugins\dipfit\standard_BESA\standard-10-5-cap385.elp'],'rplurchanloc',[1]); %make sure you put here the location of this file for your computer
EEG = pop_saveset( EEG, 'filename',[subject_list{s} '_info.set'],'filepath', data_path);
%deleting bad channels
%EEG = pop_rejchan(EEG,'elec', [1:64],'threshold',5,'norm','on','measure','kurt');
Expand All @@ -62,8 +68,15 @@
end
EEG.deleteddata_wboundries=100-EEG.pnts/old_samples*100;
end
EEG = pop_saveset( EEG, 'filename',[subject_list{s} '_exchn.set'],'filepath', data_path);
disp([num2str(EEG.deleteddata_wboundries) '% of the data got deleted for this participant']);
disp([num2str(EEG.deleteddata_wboundries) '% of the data got deleted for this participant']);
avg_deleted_data(s)=EEG.deleteddata_wboundries;


%% creating figures with deleted and bridged channels
if strcmpi(individual_plots,'yes')
EEG=plot_deleted_chan_location(EEG,data_path); %plotting the location of deleted chan
end
EEG = pop_saveset( EEG, 'filename',[subject_list{s} '_exchn.set'],'filepath', data_path);

end
disp(['on averages ' num2str(sum(avg_deleted_data)/length(subject_list)) ' % of the data got deleted']);
94 changes: 43 additions & 51 deletions src/D_reref_exclextrn_interp_avgref_ica_autoexcom.m
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,44 @@
% uses runica to do an Independent Component Analysis
% uses IClabel to define the eye component
% Deletes these and the components also get printed.
% last edits done on by Douwe 11/5/2021
% by Douwe Horsthuis updated on 12/22/2021
% ------------------------------------------------

clear variables

% This defines the set of subjects
subject_list = {'some sort of ID' 'a different id for a different particpant'};
% Path to the parent folder, which contains the data folders for all subjects
home_path = 'the main folder where you store all the data';
figure_path = 'the main folder where you store all the ic figures';
name_paradigm = 'nameofyourparadigm';%this will be added to the group file
eeglab
%% Subject info for each script
subject_list = {'some sort of ID' 'a different id for a different particpant'};% This defines the set of subjects
home_path = 'the main folder where you store your data';% Path to the parent folder, which contains the data folders for all subjects
%% info needed for this script specific
components = num2cell(zeros(length(subject_list), 8)); %prealocationg space for speed
refchan = { }; %if you want to re-ref to a channel add the name of the channel here, if empty won't re-ref to any specific channel (for example {'EXG3' 'EXG4'} or {'Cz'})
only_eye_ic='yes'; %'yes' or 'no' : if you want to delete only eye components 'yes' if you want more 'no' see line 73 for more info
%% Loop through all subjects
for s=1:length(subject_list)
fprintf('\n******\nProcessing subject %s\n******\n\n', subject_list{s});
% Path to the folder containing the current subject's data
data_path = [home_path subject_list{s} '\\'];
fprintf('\n\n\n**** %s: Loading dataset ****\n\n\n', subject_list{s});
EEG = pop_loadset('filename', [subject_list{s} '_exchn.set'], 'filepath', data_path);

%re-referencing, if refchan is empty this get's skipped
if isempty(refchan)~=1 %if no re-reference channels chose this gets skipped
for j=1:length(EEG.chanlocs)
if strcmp(refchan{1}, EEG.chanlocs(j).labels)
ref1=j; %stores here the index of the first ref channel
end
end
if length(refchan) ==1
EEG = pop_reref( EEG, ref1); % re-reference to the channel if there is only one input)
elseif length(refchan) ==2 %if 2 re-ref channels are chosen it needs to find the second one
for j=1:length(EEG.chanlocs)
if strcmp(refchan{2}, EEG.chanlocs(j).labels)
ref2=j;
end
end
EEG = pop_reref( EEG, [ref1 ref2]); %re-references to the average of 2 channels
end
end
EEG = eeg_checkset( EEG );
%deleting externals
EEG = pop_select( EEG,'nochannel',{'EXG1','EXG2','EXG3','EXG4','EXG5','EXG6','EXG7','EXG8' 'GSR1' 'GSR2' 'Erg1' 'Erg2' 'Resp' 'Plet' 'Temp'});
EEG = pop_saveset( EEG, 'filename',[subject_list{s} '_exext.set'],'filepath', data_path);
pca = EEG.nbchan-1; %the PCA part of the ICA needs stops the rank-deficiency
EEG_inter = pop_loadset('filename', [subject_list{s} '_info.set'], 'filepath', data_path);%loading participant file with all channels
EEG_inter = pop_select( EEG_inter,'nochannel',{'EXG1','EXG2','EXG3','EXG4','EXG5','EXG6','EXG7','EXG8' 'GSR1' 'GSR2' 'Erg1' 'Erg2' 'Resp' 'Plet' 'Temp'});
Expand All @@ -52,16 +67,15 @@
EEG = iclabel(EEG); %does ICLable function
ICA_components = EEG.etc.ic_classification.ICLabel.classifications ; %creates a new matrix with ICA components
%Only the eyecomponent will be deleted, thus only components 3 will be put into the 8 component
if strcmpi(only_eye_ic,'yes')
ICA_components(:,8) = ICA_components(:,3); %row 1 = Brain row 2 = muscle row 3= eye row 4 = Heart Row 5 = Line Noise row 6 = channel noise row 7 = other, combining this makes sure that the component also gets deleted if its a combination of all.
bad_components = find(ICA_components(:,8)>0.80 & ICA_components(:,1)<0.10); %if the new row is over 80% of the component and the component has less the 5% brain
%Still labeling all the other components so they get saved in the end
brain_ic = length(find(ICA_components(:,1)>0.80));
muscle_ic = length(find(ICA_components(:,2)>0.80 & ICA_components(:,1)<0.05));
eye_ic = length(find(ICA_components(:,3)>0.80 & ICA_components(:,1)<0.05));
hearth_ic = length(find(ICA_components(:,4)>0.80 & ICA_components(:,1)<0.05));
line_noise_ic = length(find(ICA_components(:,5)>0.80 & ICA_components(:,1)<0.05));
channel_ic = length(find(ICA_components(:,6)>0.80 & ICA_components(:,1)<0.05));
other_ic = length(find(ICA_components(:,7)>0.80 & ICA_components(:,1)<0.05));
elseif strcmpi(only_eye_ic,'no')
ICA_components(:,8) = sum(ICA_components(:,[2 3 4 5 6]),2);;
bad_components = (find((ICA_components(:,3)>0.70 | ICA_components(:,2)>0.80 | ICA_components(:,6)>0.70) & ICA_components(:,1)<0.10));
else
error('not correctly selected how many ICs should be deleted')
end
%Plotting all eye componentes and all remaining components
if isempty(bad_components)~= 1 %script would stop if people lack bad components
if ceil(sqrt(length(bad_components))) == 1
Expand All @@ -70,16 +84,16 @@
pop_topoplot(EEG, 0, [bad_components] ,subject_list{s},[ceil(sqrt(length(bad_components))) ceil(sqrt(length(bad_components)))] ,0,'electrodes','on');
end
title(subject_list{s});
print([figure_path subject_list{s} '_Bad_ICs_topos'], '-dpng' ,'-r300');
print([data_path subject_list{s} '_Bad_ICs_topos'], '-dpng' ,'-r300');
EEG = pop_subcomp( EEG, [bad_components], 0); %excluding the bad components
close all
else %instead of only plotting bad components it will plot all components
title(subject_list{s}); text( 0.2,0.5, 'there are no eye-components found')
print([figure_path subject_list{s} '_Bad_ICs_topos'], '-dpng' ,'-r300');
print([data_path subject_list{s} '_Bad_ICs_topos'], '-dpng' ,'-r300');
end
title(subject_list{s});
pop_topoplot(EEG, 0, 1:size(EEG.icaweights,1) ,subject_list{s},[ceil(sqrt(size(EEG.icaweights,1))) ceil(sqrt(size(EEG.icaweights,1)))] ,0,'electrodes','on');
print([figure_path subject_list{s} '_remaining_ICs_topos'], '-dpng' ,'-r300');
print([data_path subject_list{s} '_remaining_ICs_topos'], '-dpng' ,'-r300');
close all
%putting both figures in 1 plot saving it, deleting the other 2.
figure('units','normalized','outerposition',[0 0 1 1])
Expand All @@ -88,48 +102,26 @@
else
subplot(1,10,1);
end
imshow([figure_path subject_list{s} '_Bad_ICs_topos.png']);
imshow([data_path subject_list{s} '_Bad_ICs_topos.png']);
title('Deleted components')
if EEG.nbchan<65
subplot(1,5,2:5);
else
subplot(1,10,2:10);
end
imshow([figure_path subject_list{s} '_remaining_ICs_topos.png']);
imshow([data_path subject_list{s} '_remaining_ICs_topos.png']);
title('Remaining components')
print([figure_path subject_list{s} '_ICs_topos'], '-dpng' ,'-r300');
print([data_path subject_list{s} '_ICs_topos'], '-dpng' ,'-r300');
%deleting two original files
delete([figure_path subject_list{s} '_Bad_ICs_topos.png'])
delete([figure_path subject_list{s} '_remaining_ICs_topos.png'])
delete([data_path subject_list{s} '_Bad_ICs_topos.png'])
delete([data_path subject_list{s} '_remaining_ICs_topos.png'])
close all
EEG = pop_saveset( EEG, 'filename',[subject_list{s} '_excom.set'],'filepath', data_path);%save
%re-referencing, if refchan is empty this get's skipped
if isempty(refchan)~=1 %if no re-reference channels chose this gets skipped
for j=1:length(EEG.chanlocs)
if strcmp(refchan{1}, EEG.chanlocs(j).labels)
ref1=j; %stores here the index of the first ref channel
end
end
if length(refchan) ==1
EEG = pop_reref( EEG, ref1); % re-reference to the channel if there is only one input)
elseif length(refchan) ==2 %if 2 re-ref channels are chosen it needs to find the second one
for j=1:length(EEG.chanlocs)
if strcmp(refchan{2}, EEG.chanlocs(j).labels)
ref2=j;
end
end
EEG = pop_reref( EEG, [ref1 ref2]); %re-references to the average of 2 channels
end
end
EEG = pop_saveset( EEG, 'filename',[subject_list{s} '_reref.set'],'filepath', data_path);%save
subj_comps=[subject_list(s), num2cell(brain_ic), num2cell(muscle_ic), num2cell(eye_ic), num2cell(hearth_ic), num2cell(line_noise_ic), num2cell(channel_ic), num2cell(other_ic)];
components(s,:)=[subj_comps];
%this part saves all the bad channels + ID numbers
lables_del = setdiff(labels_all,labels_good); %only stores the deleted channels
All_bad_chan = strjoin(lables_del); %puts them in one string rather than individual strings
ID = string(subject_list{s});%keeps all the IDs
data_subj = [ID, All_bad_chan]; %combines IDs and Bad channels
participant_badchan(s,:) = data_subj;%combine new data with old data
end
save([home_path 'components'], 'components');
save([home_path name_paradigm '_participant_interpolation_info'], 'participant_badchan');
save([home_path 'participant_info'], 'participant_badchan');
Loading

0 comments on commit 90e0d7d

Please sign in to comment.