Issue with "Getting Started" Code when Using 1000 Thousand Streamlines

[Wilbur2001]

Hi everyone,

I'm a complete newbie in both coding and dmri. I've encountered a problem with the "getting started" code, and I'd appreciate any help or guidance.

I'm working with a file containing 1000 thousand streamlines, and I'm facing an issue. The code runs smoothly for 200 and 500 thousand streamlines, but it gets stuck at loading data when using 1000 thousand streamlines. like below:

-> Creating the dictionary from tractogram:

   * Configuration:
	- Segment position = COMPUTE INTERSECTIONS
	- Coordinates shift in X = 0.500 (voxel-size units)
	- Coordinates shift in Y = 0.500 (voxel-size units)
	- Coordinates shift in Z = 0.500 (voxel-size units)
	- Min segment len  = 0.001 mm
	- Min streamline len    = 0.00 mm
	- Max streamline len    = 250.00 mm
	- Do not blur streamlines
	- Output written to "./COMMIT"
	- Temporary files written to "./COMMIT/temp"
	- Using parallel computation with 32 threads

   * Loading data:
	- Tractogram
		- geometry taken from "peaks_3.nii.gz"
		- 110 x 110 x 74
		- 2.0000 x 2.0000 x 2.0000
		- 1000000 streamlines
	- Filtering mask
		- 110 x 110 x 74
		- 2.0000 x 2.0000 x 2.0000
	- EC orientations
		- 110 x 110 x 74 x 9
		- 2.0000 x 2.0000 x 2.0000
		- ignoring peaks < 0.10 * MaxPeak
		- using affine matrix
		- flipping axes : [ x=False, y=False, z=False ]

And it just stuck here.

The problem occurs specifically when using 1000 thousand streamlines, and my computer seems to halt the subsequent steps. I have to manually close the terminal, which opens Spyder, to restart the process.

Here are some details about my system and my code:

CPU: i9-13900
GPU: RTX3060
RAM: 128GB
n_threads: 32 (default)

import commit

from commit import trk2dictionary

trk2dictionary.run(
    filename_tractogram = 'NC02_100.tck',
    filename_peaks      = 'peaks_3.nii.gz',
    filename_mask       = 'WMmaskindwispace.nii.gz',
    fiber_shift         = 0.5,
    peaks_use_affine    = True
)

For the 200/500 thousand streamlines, the code runs successfully in all "Getting Started" code with consuming times of 40 seconds and 100 seconds, respectively.

Any help or guidance would be highly appreciated!

Thank you in advance.

[nightwnvol]

Hi @Wilbur2001,

could you please try loading your tractogram using just one thread? You can achieve this by adding n_threads = 1 in your trk2dictionary.run() call.

[Wilbur2001]

Thanks @nightwnvol 's advice!

It's the same. I checked the CPU history and found one CPU running at full capacity for about 20 seconds. Afterward, all CPUs become inactive, similar to when I use n_threads = 16 or n_threads = 8.

[nightwnvol]

@Wilbur2001 I suspect there might be a problem in your 1M streamlines tractogram that COMMIT is unable to catch (as you mentioned, it works smoothly with your 200k and 500k streamlines tractograms). Would you mind sharing your data with us privately to help identify the issue?

[Wilbur2001]

@nightwnvol Certainly! I'd like to contribute to the development team. Should I reach out via your email, or is there another preferred method for sharing detailed information, including how to submit the raw files?

[nightwnvol]

Thanks @Wilbur2001, we appreciate your contribution!

You can reach me at [email protected]. Feel free to use the method you prefer to share your data with us.