Provide more helpful error message if FreeSurfer executables cannot be found; consider amending install docs

[SophieHerbst]

Description of the problem

I try to compute BEM surfaces from a freshly recon-alled participant, but I get an error.

---

FileNotFoundError Traceback (most recent call last)
File /Volumes/neurospin/meg/meg_tmp/Deltaphase_shift_Valentine_2021/code/compute_BEM.py:2
1 #%%
----> 2 mne.bem.make_watershed_bem(subject, subjects_dir=None, overwrite=True)

File :12, in make_watershed_bem(subject, subjects_dir, overwrite, volume, atlas, gcaatlas, preflood, show, copy, T1, brainmask, verbose)

File ~/miniconda3/envs/mne/lib/python3.12/site-packages/mne/bem.py:1330, in make_watershed_bem(subject, subjects_dir, overwrite, volume, atlas, gcaatlas, preflood, show, copy, T1, brainmask, verbose)
1325 logger.info(
1326 "\nRunning mri_watershed for BEM segmentation with the following parameters:\n"
1327 f"\nResults dir = {ws_dir}\nCommand = {' '.join(cmd)}\n"
1328 )
1329 os.makedirs(op.join(ws_dir))
-> 1330 run_subprocess_env(cmd)
1331 del tempdir # clean up directory
1332 if op.isfile(T1_mgz):

File :12, in run_subprocess(command, return_code, verbose, *args, **kwargs)

File ~/miniconda3/envs/mne/lib/python3.12/site-packages/mne/utils/misc.py:144, in run_subprocess(command, return_code, verbose, *args, **kwargs)
142 control_stdout = "stdout" not in kwargs
143 control_stderr = "stderr" not in kwargs
--> 144 with running_subprocess(command, *args, **kwargs) as p:
145 if control_stdout:
...
-> 1953 raise child_exception_type(errno_num, err_msg, err_filename)
1954 else:
1955 raise child_exception_type(errno_num, err_msg)

FileNotFoundError: [Errno 2] No such file or directory: 'mri_watershed'

Steps to reproduce

#%%
import mne
import os.path as op
import mne_bids

subject = 'sub-22_new'
sb_dir = '/Volumes/neurospin/meg/meg_tmp/Deltaphase_shift_Valentine_2021/sophie_tmp/'

mne.set_config("FREESURFER_HOME", '/Applications/freesurfer/7.4.1/')
mne.set_config("SUBJECTS_DIR", sb_dir)
# print(mne.get_config())

#%%
mne.bem.make_watershed_bem(subject, subjects_dir=None, overwrite=True)

Link to data

No response

Expected results

Create BEM surfaces

Actual results

error message above

Additional information

Freesurfer is installed and working, mri_watershed exists in
/Applications/freesurfer/7.4.1/bin/mri_watershed

It seems like other users had the same issue recently: https://mne.discourse.group/t/mne-bem-make-watershed-bem-compatablity-with-freesurfer-7-3-2/5794/4

[larsoner]

Can you check for example ls -l $FREESURFER_HOME/bin/mri_* and see if it's there? And if you do:

$FREESURFER_HOME/bin/mri_watershed --help

do you get the help info?

[SophieHerbst]

Yes, the file is there: /Applications/freesurfer/7.4.1/bin/mri_watershed
and I get the help printed.
Also, when running it with the command line tools it works fine.

[hoechenberger]

Sorry if this is a bit annoying, but did you literally try to run mri_watershed by just entering this command? Without all the /Applications/... stuff in front of it :)

[SophieHerbst]

This is what I tried:

#%%
import mne
import os
import os.path as op
import mne_bids

subject = 'sub-23'
sb_dir = '/Volumes/neurospin/meg/meg_tmp/Deltaphase_shift_Valentine_2021/sophie_tmp/'

mne.set_config("FREESURFER_HOME", '/Applications/freesurfer/7.4.1/')
mne.set_config("SUBJECTS_DIR", sb_dir)
# print(mne.get_config())

#%%
mne.bem.make_watershed_bem(subject, subjects_dir=None, overwrite=True)

[hoechenberger]

No what I meant is, if you open a terminal in VS Code (assuming you're using VS Code to run your script?), just run:

mri_watershed

Does that work?

[hoechenberger]

mne.set_config("FREESURFER_HOME", '/Applications/freesurfer/7.4.1/')
mne.set_config("SUBJECTS_DIR", sb_dir)

Is this supposed to work? I would always do it as:

import os
os.environ["FREESURFER_HOME"] = ...

Or actually … I would ensure that the env var is set correctly even before I start Python :)
Are you sure your ~/.bashrc (or whatever shell you use!) has been updated? This is what I want to test by asking you to run mri_watershed (just like this) in a terminal

[SophieHerbst]

just running mri_watershed in the VScode terminal: not found

I got the set_config command from here: https://mne.tools/0.22/auto_tutorials/source-modeling/plot_background_freesurfer.html?highlight=mne%20set_config%20freesurfer_home

but using the os.environment gives the same error

os.environ["FREESURFER_HOME"] = '/Applications/freesurfer/7.4.1/'
os.environ["SUBJECTS_DIR"] = sb_dir

[hoechenberger]

just running mri_watershed in the VScode terminal: not found

This indicates a setup problem of FreeSurfer
We should try to fix this first before looking at MNE

Do you know which shell you're using? Bash? Zsh? What's the output of

echo $SHELL

?

[SophieHerbst]

I am using Zsh

[SophieHerbst]

After sourcing freesurfer from the VScode terminal, I can execute the code and it finds mri_watershed, but only when I run it through the terminal, not in the interactive jupyter notebook. Did I misunderstand how this is supposed to work?

[larsoner]

but only when I run it through the terminal, not in the interactive jupyter notebook. Did I misunderstand how this is supposed to work?

Did you make sure to launch the jupyter server from the same shell where you sourced the freesurfer setup script?

[SophieHerbst]

How do I find out other than checking that they use the same conda environment?

[hoechenberger]

I am using Zsh

I will send you my setup later this afternoon ... busy with something else right now!

you should NOT need to manually source FS

if you do, that's something we need to address

[larsoner]

How do I find out other than checking that they use the same conda environment?

By the order in which you execute commands. So if you do for BASH:

$ source $FREESURFER_HOME/SetUpFreeSurfer.sh
$ jupyter notebook

then open your notebook in that server things should work. But if you laurch jupyter notebook in a terminal where you have not yet set the FreeSurfer vars properly, it won't.

I thought we manually checked and added the correct bin paths so that things like mri_watershed would run via:

mne-python/mne/bem.py

Line 1245 in c63da99

env, mri_dir, bem_dir = _prepare_env(subject, subjects_dir)

It looks like we don't actually modify PATH. I guess we could here:

mne-python/mne/bem.py

Line 1910 in c63da99

env.update(SUBJECT=subject, SUBJECTS_DIR=str(subjects_dir), FREESURFER_HOME=fs_home)

But it's probably safest to make sure that people (either in a ZSH/BASH profile script) have $FREESURFER_HOME/<some_script> sourced in their terminal. It also indicates that we should probably improve our error message in MNE to mention the need to make sure FreeSurfer is properly set up / configured / sourced in whatever terminal you are running Python in (which for Jupyter means: the one where you launched the server).

[hoechenberger]

@SophieHerbst

So this is how I got it working on my Mac:

• Create a file ~/.zprofile with the following content (attention, it's .zprofile with a z, this is not a typo!):

  export FREESURFER_HOME=/Applications/freesurfer/7.4.1
  source $FREESURFER_HOME/SetUpFreeSurfer.sh

• Close your terminal and VS Code

• Start a new terminal

• Start VS Code from the terminal via code

• All should be working now

[SophieHerbst]

That works, thank you @hoechenberger!

[hoechenberger]

Great! I'm not sure it it works if you don't start VS Code from the command line, but by clicking on the app icon.

In any case, we should probably amend the installation instructions. What I told you above is, while present in the FreeSurfer docs, kind of hidden and hard to find.

We should also amend the error message.

Edit: I believe the FS docs only mention ~/.profile and not ~/.zprofile; ~/.profile however is ignored by Zsh, which is the default on macOS.

[SophieHerbst]

It seems to work also when clicking on the icon.
Indeed, I was looking for this information when I first encountered the problem, but did not find it.

[hoechenberger]

@larsoner It seems that currently, FREESURFER_HOME is retrieved via utils.get_config(). This searches the shell environment first and then falls back to the MNE config file.

I think this is problematic, as demonstrated above: @SophieHerbst set the variable via MNE's config mechanism, but the shell environment was never properly set up.

I suggest not to retrieve FREESURFER_HOME from MNE's config, and fail if it's not set in the shell.

[larsoner]

Another option would be to check to see if FREESURFER_HOME/bin is in PATH and if it's not, add it to the env we use to run FreeSurfer commands. I think we discussed that possibility at some point but can't find it. Using set_config rather than os.environ was at least discussed a bit in #6988 and implemented in the PR that fixed that issue. If it works, it's more convenient than having to source the correct FreeSurfer script! Might be fragile though, would need to be tested on macOS and Linux. At least to start, doing this works in an env without having sourced the FreeSurfer script:

$ ~/applications/freesurfer-6/bin/mri_watershed --help

So I'm cautiously optimistic we could work some magic here.

[hoechenberger]

Interesting, also considering that the SetUpFreeSurfer.sh script does quite some heavy lifting to set everything up correctly

[hoechenberger]

mne.set_config("FREESURFER_HOME", '/Applications/freesurfer/7.4.1/')

Also makes me wonder why this didn't work for @SophieHerbst, then?

[larsoner]

also considering that the SetUpFreeSurfer.sh script does quite some heavy lifting to set everything up correctly

Yeah I looked at that, I think most of it is probably for recon_all and/or GUI stuff like freeview moreso than something like mri_watershed. But that's why I think it's worth testing a bit!

Also makes me wonder why this didn't work for @SophieHerbst, then?

Someone would have to look back at the code and blame deeper, it's possible we used to modify PATH previously but stopped at some point. Or maybe we only used FREESURFER_HOME at that time to get a fsaverage directory or something, not sure.