[ERROR] PYP (cspswarm) failed when running a copy of the Particle refinement block

[JuhaHuiskonen]

I finished successfully Particle refinement. Then I went back and created a new, smaller set of tilt-series, excluding some bad ones. I made a copy of the successful run, gave the new set as input. But now I am getting this error and can't figure out why.

1	2024-02-04 15:00:57 [INFO] Job 3163561_1 (v0.6.2) launching on c3348.mahti.csc.fi using 64 task(s) and 200 GB of RAM
2	2024-02-04 15:01:06 [INFO] Filesystem                                  Size  Used Avail Use% Mounted on
3	2024-02-04 15:01:06 [INFO] 10.141.0.14@o2ib:10.141.0.13@o2ib:/scratch   40T  2.7T   38T   7% /scratch/project_2009485/nextPYP/tmp
4	2024-02-04 15:01:06 [INFO] First function is: load_files
5	2024-02-04 15:01:07 [INFO] First function is: load_files
6	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009.mrc
7	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009_raw.webp
8	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009_rec.webp
9	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009.rec
10	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009.webp
11	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009_ali.webp
12	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009.pkl
13	2024-02-04 15:01:07 [INFO] Retrieving GS8_TS27_ts_009_sides.webp
14	2024-02-04 15:01:14 [INFO] Executing coordinate extraction
15	2024-02-04 15:01:14 [INFO] Reading and converting coordinates took: 00h 00m 00s
16	2024-02-04 15:01:14 [INFO] Total time elapsed (csp_swarm): 00h 00m 07s
17	2024-02-04 15:01:14 [ERROR] PYP (cspswarm) failed

[abartesaghi]

It looks like you are showing the contents of the Stream tab which doesn't include the full error logs. You should look at the contents of the Logs tab to see the actual error.

How did you "copy the successful run"? Was this done using the Copy option from the pre-processing or the particle-refinement block menu, or did you do something else?

[JuhaHuiskonen]

I made a coupe of the Particle pre-processing block using the Copy function of that block. Here is the full error from Logs

17	Traceback (most recent call last):
18	  File "/opt/pyp/bin/run/pyp", line 3914, in <module>
19	    csp_swarm(args.file, parameters, int(args.iter), args.skip, args.debug)
20	  File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
21	    return func(*args, **kwargs)
22	  File "/opt/pyp/bin/run/pyp", line 2376, in csp_swarm
23	    [allboxes, allparxs] = csp_extract_coordinates(
24	  File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
25	    return func(*args, **kwargs)
26	  File "/opt/pyp/src/pyp/inout/metadata/core.py", line 1869, in csp_extract_coordinates
27	    [allboxes, allparxs] = tomo_extract_coordinates(
28	  File "/opt/pyp/src/pyp/inout/metadata/core.py", line 2007, in tomo_extract_coordinates
29	    refinement = project_params.resolve_path(parameters["refine_parfile_tomo"])
30	KeyError: 'refine_parfile_tomo'
31	2024-02-04 15:54:15 [ERROR] /opt/pyp/bin/run/pyp:3927 | PYP (cspswarm) failed

[yezhou867]

Probably need to specify the` input parameters` file in the new slot in the advanced option.

[JuhaHuiskonen]

This is strange as the original block did not have this file given. Where do I find this file?

[yezhou867]

This is iteration 2, right?
The input for the alignments from sub-volume averaging in the new version is separate from the original input parameter file. If this is from preprocessing, then the file is frealign/*_original_volumes.txt in the preprocessing job folder.