Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test collect_derivatives function #29

Merged
merged 31 commits into from
May 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
622bc78
Work on collect_derivatives.
tsalo May 16, 2024
9703bef
Work on queries.
tsalo May 16, 2024
85bec3a
Work on collect_derivatives.
tsalo May 16, 2024
0da1278
Tests look good now. Will still fail.
tsalo May 16, 2024
48360bd
Update conftest.py
tsalo May 16, 2024
b7729d5
Work on raw vs. derivative collection.
tsalo May 16, 2024
c2a5269
Update test_utils_bids.py
tsalo May 16, 2024
3efa4f2
Update bids.py
tsalo May 16, 2024
9991d18
Merge remote-tracking branch 'upstream/main' into testing
tsalo May 16, 2024
35f7f69
Try this.
tsalo May 16, 2024
3a7889b
Fix style issues.
tsalo May 16, 2024
88edabf
Update utils.py
tsalo May 16, 2024
ea500e9
Update conftest.py
tsalo May 16, 2024
93fb0d3
Update test_utils_bids.py
tsalo May 16, 2024
ca329af
Update test_utils_bids.py
tsalo May 16, 2024
1a02037
Don't index metadata.
tsalo May 16, 2024
85c3a1b
Update test_utils_bids.py
tsalo May 16, 2024
1d42fc1
Update test_utils_bids.py
tsalo May 16, 2024
68ea226
Fix up entrypoint and imports.
tsalo May 16, 2024
dbe3a25
Keep fixing.
tsalo May 16, 2024
867ff8c
Update.
tsalo May 16, 2024
6d5babe
Fix some stuff.
tsalo May 16, 2024
9f3f8e6
Keep trying to fix.
tsalo May 16, 2024
9d75012
Update base.py
tsalo May 16, 2024
f217d16
Get it building at least.
tsalo May 16, 2024
11bb3d9
Remove reporting nodes.
tsalo May 16, 2024
98ffee2
Whoops!
tsalo May 16, 2024
b4dd02d
Fix more stuff.
tsalo May 16, 2024
d6f3e36
Get the workflow to run all the way through.
tsalo May 16, 2024
f6b8ef3
Fix style issues.
tsalo May 16, 2024
33b624a
Update confounds.py
tsalo May 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ ENV IS_DOCKER_8395080871=1

RUN ldconfig
WORKDIR /tmp
ENTRYPOINT ["/opt/conda/envs/fmripost_aroma/bin/fmripost_aroma"]
ENTRYPOINT ["/opt/conda/envs/fmripost_aroma/bin/fmripost-aroma"]

ARG BUILD_DATE
ARG VCS_REF
Expand Down
2 changes: 1 addition & 1 deletion docs/spaces.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ stereotactic coordinate system.
The most extended standard space for fMRI analyses is generally referred to MNI.
For instance, to instruct *fMRIPost-AROMA* to use the MNI template brain distributed with
FSL as coordinate reference the option will read as follows: ``--output-spaces MNI152NLin6Asym``.
By default, *fMRIPost-AROMA* uses ``MNI152NLin2009cAsym`` as spatial-standardization reference.
By default, *fMRIPost-AROMA* uses ``MNI152NLin6Asym`` as spatial-standardization reference.
Valid template identifiers (``MNI152NLin6Asym``, ``MNI152NLin2009cAsym``, etc.) come from
the `TemplateFlow repository <https://github.com/templateflow/templateflow>`__.

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Issues = "https://github.com/nipreps/fmripost-aroma/issues"
Source = "https://github.com/nipreps/fmripost-aroma"

[project.scripts]
fmripost-aroma = "fmripost_aroma.cli:aroma"
fmripost-aroma = "fmripost_aroma.cli.run:main"

[tool.hatch.metadata]
allow-direct-references = true
Expand Down
20 changes: 17 additions & 3 deletions src/fmripost_aroma/cli/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,6 @@
'--derivatives',
action=ToDict,
metavar='PACKAGE=PATH',
type=str,
nargs='+',
help=(
'Search PATH(s) for pre-computed derivatives. '
Expand Down Expand Up @@ -332,6 +331,17 @@
default=False,
help='Skip generation of HTML and LaTeX formatted citation with pandoc',
)
g_outputs.add_argument(

Check warning on line 334 in src/fmripost_aroma/cli/parser.py

View check run for this annotation

Codecov / codecov/patch

src/fmripost_aroma/cli/parser.py#L334

Added line #L334 was not covered by tests
'--aggregate-session-reports',
dest='aggr_ses_reports',
action='store',
type=PositiveInt,
default=4,
help=(
"Maximum number of sessions aggregated in one subject's visual report. "
'If exceeded, visual reports are split by session.'
),
)

g_aroma = parser.add_argument_group('Options for running ICA_AROMA')
g_aroma.add_argument(
Expand All @@ -347,7 +357,7 @@
)
g_aroma.add_argument(
'--error-on-warnings',
dest='error_on_aroma_warnings',
dest='err_on_warn',
action='store_true',
default=False,
help=(
Expand All @@ -361,6 +371,7 @@
nargs='+',
choices=['aggr', 'nonaggr', 'orthaggr'],
default=None,
dest='denoise_method',
help='Denoising method to apply, if any.',
)

Expand Down Expand Up @@ -530,6 +541,9 @@
work_dir = config.execution.work_dir
version = config.environment.version

if config.execution.fmripost_aroma_dir is None:
config.execution.fmripost_aroma_dir = output_dir

Check warning on line 545 in src/fmripost_aroma/cli/parser.py

View check run for this annotation

Codecov / codecov/patch

src/fmripost_aroma/cli/parser.py#L545

Added line #L545 was not covered by tests

# Wipe out existing work_dir
if opts.clean_workdir and work_dir.exists():
from niworkflows.utils.misc import clean_directory
Expand Down Expand Up @@ -569,7 +583,7 @@
validate_input_dir(config.environment.exec_env, opts.bids_dir, opts.participant_label)

# Setup directories
config.execution.log_dir = config.execution.fmriprep_dir / 'logs'
config.execution.log_dir = config.execution.fmripost_aroma_dir / 'logs'

Check warning on line 586 in src/fmripost_aroma/cli/parser.py

View check run for this annotation

Codecov / codecov/patch

src/fmripost_aroma/cli/parser.py#L586

Added line #L586 was not covered by tests
# Check and create output and working directories
config.execution.log_dir.mkdir(exist_ok=True, parents=True)
work_dir.mkdir(exist_ok=True, parents=True)
Expand Down
12 changes: 4 additions & 8 deletions src/fmripost_aroma/cli/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,6 @@
)
errno = 0
finally:
from pkg_resources import resource_filename as pkgrf

# Code Carbon
if config.execution.track_carbon:
emissions: float = tracker.stop()
Expand All @@ -217,11 +215,9 @@

# Generate reports phase
failed_reports = generate_reports(
config.execution.participant_label,
config.execution.fmripost_aroma_dir,
config.execution.run_uuid,
config=pkgrf('fmripost_aroma', 'data/reports-spec.yml'),
packagename='fmripost_aroma',
subject_list=config.execution.participant_label,
output_dir=config.execution.fmripost_aroma_dir,
run_uuid=config.execution.run_uuid,
)
write_derivative_description(
config.execution.bids_dir, config.execution.fmripost_aroma_dir
Expand All @@ -233,7 +229,7 @@
'Report generation failed for %d subjects' % failed_reports,
level='error',
)
sys.exit(int((errno + failed_reports) > 0))
sys.exit(int((errno + len(failed_reports)) > 0))

Check warning on line 232 in src/fmripost_aroma/cli/run.py

View check run for this annotation

Codecov / codecov/patch

src/fmripost_aroma/cli/run.py#L232

Added line #L232 was not covered by tests


def migas_exit() -> None:
Expand Down
40 changes: 5 additions & 35 deletions src/fmripost_aroma/cli/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,13 @@
"""Create the Nipype Workflow that supports the whole execution graph."""
from pathlib import Path

from fmriprep.utils.bids import check_pipeline_version
from fmriprep.utils.misc import check_deps

Check warning on line 40 in src/fmripost_aroma/cli/workflow.py

View check run for this annotation

Codecov / codecov/patch

src/fmripost_aroma/cli/workflow.py#L39-L40

Added lines #L39 - L40 were not covered by tests
from niworkflows.utils.bids import collect_participants
from niworkflows.utils.misc import check_valid_fs_license
from pkg_resources import resource_filename as pkgrf

from fmripost_aroma import config
from fmripost_aroma.reports.core import generate_reports
from fmripost_aroma.utils.bids import check_pipeline_version
from fmripost_aroma.utils.misc import check_deps
from fmripost_aroma.workflows.base import init_fmripost_aroma_wf

config.load(config_file)
Expand Down Expand Up @@ -92,11 +91,9 @@
if config.execution.reports_only:
build_log.log(25, 'Running --reports-only on participants %s', ', '.join(subject_list))
retval['return_code'] = generate_reports(
config.execution.participant_label,
config.execution.fmripost_aroma_dir,
config.execution.run_uuid,
config=pkgrf('fmripost_aroma', 'data/reports-spec.yml'),
packagename='fmripost_aroma',
subject_list=config.execution.participant_label,
output_dir=config.execution.fmripost_aroma_dir,
run_uuid=config.execution.run_uuid,
)
return retval

Expand All @@ -112,37 +109,10 @@
if config.execution.derivatives:
init_msg += [f'Searching for derivatives: {config.execution.derivatives}.']

if config.execution.fs_subjects_dir:
init_msg += [f"Pre-run FreeSurfer's SUBJECTS_DIR: {config.execution.fs_subjects_dir}."]

build_log.log(25, f"\n{' ' * 11}* ".join(init_msg))

retval['workflow'] = init_fmripost_aroma_wf()

# Check for FS license after building the workflow
if not check_valid_fs_license():
from fmripost_aroma.utils.misc import fips_enabled

if fips_enabled():
build_log.critical(
"""\
ERROR: Federal Information Processing Standard (FIPS) mode is enabled on your system. \
FreeSurfer (and thus fMRIPost-AROMA) cannot be used in FIPS mode. \
Contact your system administrator for assistance."""
)
else:
build_log.critical(
"""\
ERROR: a valid license file is required for FreeSurfer to run. \
fMRIPost-AROMA looked for an existing license file at several paths, in this order: \
1) command line argument ``--fs-license-file``; \
2) ``$FS_LICENSE`` environment variable; and \
3) the ``$FREESURFER_HOME/license.txt`` path. \
Get it (for free) by registering at https://surfer.nmr.mgh.harvard.edu/registration.html"""
)
retval['return_code'] = 126 # 126 == Command invoked cannot execute.
return retval

# Check workflow for missing commands
missing = check_deps(retval['workflow'])
if missing:
Expand Down
15 changes: 12 additions & 3 deletions src/fmripost_aroma/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -225,13 +225,15 @@
for k, v in settings.items():
if k in ignore or v is None:
continue

if k in cls._paths:
if isinstance(v, list | tuple):
setattr(cls, k, [Path(val).absolute() for val in v])
elif isinstance(v, dict):
setattr(cls, k, {key: Path(val).absolute() for key, val in v.items()})
else:
setattr(cls, k, Path(v).absolute())

elif hasattr(cls, k):
setattr(cls, k, v)

Expand All @@ -250,17 +252,24 @@
for k, v in cls.__dict__.items():
if k.startswith('_') or v is None:
continue

if callable(getattr(cls, k)):
continue

if k in cls._paths:
if isinstance(v, list | tuple):
v = [str(val) for val in v]
elif isinstance(v, dict):
v = {key: str(val) for key, val in v.items()}
else:
v = str(v)

if isinstance(v, SpatialReferences):
v = ' '.join(str(s) for s in v.references) or None

if isinstance(v, Reference):
v = str(v) or None

out[k] = v
return out

Expand Down Expand Up @@ -533,7 +542,7 @@
class workflow(_Config):
"""Configure the particular execution graph of this workflow."""

err_on_warn = None
err_on_warn = False
"""Cast AROMA warnings to errors."""
melodic_dim = None
"""Number of ICA components to be estimated by MELODIC
Expand Down Expand Up @@ -729,8 +738,8 @@
spaces.checkpoint()

# Add the default standard space if not already present (required by several sub-workflows)
if 'MNI152NLin2009cAsym' not in spaces.get_spaces(nonstandard=False, dim=(3,)):
spaces.add(Reference('MNI152NLin2009cAsym', {}))
if 'MNI152NLin6Asym' not in spaces.get_spaces(nonstandard=False, dim=(3,)):
spaces.add(Reference('MNI152NLin6Asym', {}))

Check warning on line 742 in src/fmripost_aroma/config.py

View check run for this annotation

Codecov / codecov/patch

src/fmripost_aroma/config.py#L742

Added line #L742 was not covered by tests

# Ensure user-defined spatial references for outputs are correctly parsed.
# Certain options require normalization to a space not explicitly defined by users.
Expand Down
Loading