AreTomo reconstruction fails after successful Unblur and AreTomo alignment (missing xf file)

[JuhaHuiskonen]

Hi,

I am running Preprocessing, using Unblur and AreTomo (both alignment and reconstruction). AreTomo tilt series alignment completes but then reconstruction fails with error
FileNotFoundError: [Errno 2] No such file or directory: 'lamella10_ts_002_Imod/lamella10_ts_002_st.xf'

I am rerunning this block as during the first run I ran out of memory. Any suggestions? The complete error message is below.

898	2024-02-17 11:04:14 [INFO] pyp/align/core.py:57 | Alignment took: 00h 01m 21s
899	
900	Progress:   0%|          | 0/1 [00:00<?, ?it/s]
901	
902	Progress: 100%|##########| 1/1 [01:22<00:00, 82.04s/it]
903	
904	Progress: 100%|##########| 1/1 [01:22<00:00, 82.74s/it]
905	2024-02-17 11:04:14 [INFO] /opt/pyp/bin/run/pyp:57 | Total time in tomo_swarm 00h 13m 47s seconds
906	joblib.externals.loky.process_executor._RemoteTraceback: 
907	"""
908	Traceback (most recent call last):
909	  File "/opt/pyp/src/pyp/align/core.py", line 5147, in align_tilt_series
910	    shutil.copy2(f"{name}_Imod/{name}_st.xf", f"{name}.xf")
911	  File "/usr/local/envs/pyp/lib/python3.8/shutil.py", line 435, in copy2
912	    copyfile(src, dst, follow_symlinks=follow_symlinks)
913	  File "/usr/local/envs/pyp/lib/python3.8/shutil.py", line 264, in copyfile
914	    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
915	FileNotFoundError: [Errno 2] No such file or directory: 'lamella10_ts_002_Imod/lamella10_ts_002_st.xf'
916	
917	During handling of the above exception, another exception occurred:
918	
919	Traceback (most recent call last):
920	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 428, in _process_worker
921	    r = call_item()
922	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 275, in __call__
923	    return self.fn(*self.args, **self.kwargs)
924	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 620, in __call__
925	    return self.func(*args, **kwargs)
926	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 288, in __call__
927	    return [func(*args, **kwargs)
928	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 288, in <listcomp>
929	    return [func(*args, **kwargs)
930	  File "/opt/pyp/src/pyp/system/mpi.py", line 174, in wrapper
931	    func(*arg)
932	  File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
933	    return func(*args, **kwargs)
934	  File "/opt/pyp/src/pyp/align/core.py", line 5155, in align_tilt_series
935	    raise Exception("AreTomo2 failed to run")
936	Exception: AreTomo2 failed to run
937	"""
938	
939	The above exception was the direct cause of the following exception:
940	
941	Traceback (most recent call last):
942	  File "/opt/pyp/bin/run/pyp", line 3490, in <module>
943	    tomo_swarm(args.path, args.file, args.debug, args.keep, args.skip)
944	  File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
945	    return func(*args, **kwargs)
946	  File "/opt/pyp/bin/run/pyp", line 1504, in tomo_swarm
947	    mpi.submit_function_to_workers(mpi_funcs, mpi_args, verbose=parameters["slurm_verbose"])
948	  File "/opt/pyp/src/pyp/system/mpi.py", line 185, in submit_function_to_workers
949	    parallel(delayed(wrapper)(func, *arg, current_directory=current_directory) for idx, func in enumerate(funcs) for arg in args[idx])
950	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 1098, in __call__
951	    self.retrieve()
952	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 975, in retrieve
953	    self._output.extend(job.get(timeout=self.timeout))
954	  File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 567, in wrap_future_result
955	    return future.result(timeout=timeout)
956	  File "/usr/local/envs/pyp/lib/python3.8/concurrent/futures/_base.py", line 444, in result
957	    return self.__get_result()
958	  File "/usr/local/envs/pyp/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
959	    raise self._exception
960	Exception: AreTomo2 failed to run
961	2024-02-17 11:04:14 [ERROR] /opt/pyp/bin/run/pyp:3495 | PYP (tomoswarm) failed

[tofushan]

Exception: AreTomo2 failed to run says areTomo does not run successfully. You might run out of memory, or something else prevents it from running correctly. I suggest turning on the verbose and see the actual areTomo command (AreTomo2 -InMrc xxx -OutMrc yyy) and outputs.

[JuhaHuiskonen]

I am unable to get AreTomo2 to run on my system. As that is probably a system specific issue, I am closing this.