Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

QA #9

Open
fliem opened this issue Dec 23, 2016 · 24 comments
Open

QA #9

fliem opened this issue Dec 23, 2016 · 24 comments

Comments

@fliem
Copy link
Contributor

fliem commented Dec 23, 2016 •
edited
Loading

Is there anything I can do to revive the QA branch?
@chrisgorgo
Copy link
Contributor

It requires using a mini X server and I could not get it to work reliably under Docker and Singularity. It's an old and clunky solution - are there any other QA oriented FreeSurfer tools out there?

@fliem
Copy link
Contributor Author

fliem commented Dec 23, 2016

Mhh. I have a vague memory of @satra heading a past brainhack project about FS QA?

@satra
Copy link

satra commented Jan 1, 2017

@fliem - this depends on whether you are willing to create something from scratch or use some of the outputs in place.

for example, the brain is generally quite symmetric - so bad reconstructions often have abs(1 - L_struct/R_struct) > some number for each aparc/a2009 structure

but the things that need QA and are hard are:

  • meninges part of cortical ribbon
  • intensity bias affecting placement of surfaces in orbitofrontal cortex
  • ripples caused by interaction between motion and fat shifts.
  • outlines of subcortical boundaries (e.g., amygdala)

many of these should be detectable with enough training data.

@fliem
Copy link
Contributor Author

fliem commented Mar 4, 2017

Since surface plots now arrived in nilearn, and creating qc plots for freesurfer is such a drag, I suggest we revive this issue. As a starting point I have created a couple of plots in a notebook.
What do you think?

@chrisgorgo
Copy link
Contributor

This looks great - looping in @effigies who did some surface QC for fmriprep. This is how it looks:

image

@alexlicohen
Copy link
Collaborator

alexlicohen commented Mar 4, 2017 via email

@fliem
Copy link
Contributor Author

fliem commented Mar 6, 2017

Those look very pretty. @effigies could you please point me towards that fmriprep code. Did you do other things for qc as well (like calculating metrics...)? Thanks.

@fliem
Copy link
Contributor Author

fliem commented Mar 23, 2017

I have refined the surface and subcortical plots, borrowing heavily from nilearn/niworkflows.
Next, I'll look into how to assemble them in htmls

@effigies
Copy link

Hi @fliem sorry that I totally missed this thread.

It looks like you found the visualization bit in niworkflows. We haven't really done any metrics at this point, though; we've reconstructed surfaces using ANTs skull-stripping instead of mri_watershed, and are now using bbregister to perform functional alignment. At present, my focus (nipreps/fmriprep#391) is on sampling functional data to the surface and ensuring that these surfaces align properly in native T1 space, so that they can be used smoothly in non-FreeSurfer tools.

@fliem
Copy link
Contributor Author

fliem commented May 8, 2017

I have created first examples for single subjects and group html reports. Any suggestions?

@chrisgorgo
Copy link
Contributor

chrisgorgo commented May 8, 2017 via email

@effigies
Copy link

effigies commented May 9, 2017

I can't see the parcellation reports at all, either through Github or pulling locally and viewing...

But I do like @chrisfilo's idea of a separate module that would ease re-use in other projects. Since the base machinery for this is niworkflows-based, I think it'd be reasonable to pull this into niworkflows, itself. But I'd also entirely support an independent module that we then incorporate into ReconAllRPT.

@fliem
Copy link
Contributor Author

fliem commented May 10, 2017

  • re misalignment: this was partially due to a bug in the code. However after fixing this, the contours still don't align perfectly. I have reverted to ROI overlays for now.
    subcort_contours1

  • new example reports are here

  • Due to a nilearn issue with .add_contours, the white/pial-surface-plots show smaller brains than the subcortical overlays.

  • Making it an own module or part of niworkflows sounds good to me. For the time being, I have created a command line tool as part of my freesurfer repo, and a docker container. If anyone wants to try it:

docker run --rm -ti \
-v /data/sourcedata:/data/in \
-v /data/freesurfer:/data/out \
--entrypoint=/code/run_qc.py \
fliem/freesurfer:qc_nb \
/data/in /data/out participant

@alexlicohen
Copy link
Collaborator

alexlicohen commented May 10, 2017 via email

@satra
Copy link

satra commented May 10, 2017

@akeshavan - perhaps this can somehow be integrated with mindcontrol.

@effigies
Copy link

To be clear, the parcellation images are missing for everyone else, right? I'm seeing surfaces and segmentation only.

@alexlicohen
Copy link
Collaborator

alexlicohen commented May 10, 2017 via email

@fliem
Copy link
Contributor Author

fliem commented May 10, 2017 •
edited
Loading

Sorry, I can't figure out why the parcellations won't show. Here's one of them:
sub-01_parc_lh_lateral

@PeerHerholz
Copy link
Collaborator

Ahoi hoi gang,

should we revive this given the recent developments over at niworkflows ?

@satra
Copy link

satra commented Dec 16, 2020

this is an entire segment of our current nobrainer collaboration with the freesurfer folks to develop autoqc methods in general and with the short term goal for freesurfer specifically. initial goal discard samples that will not work "well" with freesurfer, medium term goal - modify the samples to run through freesurfer - long term goal make freesurfer faster and more robust to inputs.

this doesn't mean anybody should stop trying to improve manual quality verification. this will also be used in our distributed training mode, where freesurfer runs locally and the lab can help provide feedback as to the quality .

also kwyk can be used in the short term to generate a quality assessment metric for T1 scans in 30s on a GPU.

btw, if anyone is interested in joining the autoqc effort just drop me a line and i will keep you informed about when we meet next (likely in jan, i'm mostly offline over the next few weeks).

@PeerHerholz
Copy link
Collaborator

Thx for the update and pointers @satra!
I'm all for automatization and happy to not do QC manually (other than to train and inform a model), hehe.

Re the things in this thread: as far as I understood it, the idea would be to generate a BIDS App report (comparable to fmriprep and mriqc) focusing on FreeSurfer output QC, right? If so, the information generated through the autoqc methods could be included in this report besides some informative graphics, eh?

@satra
Copy link

satra commented Dec 24, 2020

he idea would be to generate a BIDS App report (comparable to fmriprep and mriqc) focusing on FreeSurfer output QC, right?

yes, and any useful thing created for the report itself could be used to improve autoqc.

If so, the information generated through the autoqc methods could be included in this report besides some informative graphics, eh?

yes. the autoqc stuff can benefit from the report and in turn benefit the report in the future.

@Shotgunosine
Copy link
Contributor

This is an old issue, but I think outputting QA reports would still be a big improvement.

@Remi-Gau
Copy link
Contributor

note that for QA based on freesurfer output there is this: https://github.com/Qoala-T/QC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@effigies @satra @chrisgorgo @Shotgunosine @fliem @Remi-Gau @alexlicohen @PeerHerholz