use workflow config to combine images into xarray

[dschneiderch]

In #708 I showed some code that I use to get my PSII images and then organize them into xarray format. it uses several key pieces of code from plantcv esp extracting metadata from the file. I copied the code into some new internal functions but I am wondering if this could possibly be done directly with plantcv.

this feels like a continuation of #538 (comment) and #423

@nfahlgren how were you imagining the grouping? or can you point me towards public code that implements #423 (comment)?

[nfahlgren]

I have some of it worked out (or at least an attempt at it) in a stale branch. I had added an attribute to WorkflowConfig: "group_by": None. The value for group_by is a list of one or more metadata terms that define a grouping. For example, "group_by": ["timestamp"] would group all the images taken at the same time, regardless of the image type.

To achieve the grouping, I had to modify how the metadata dictionary is built. In the current plantcv.parallel.metadata_parser the dictionary keys are the file path/name of each image. To extend this for groups I made a key out of the concatenation of the values of the terms in group_by (so the values of timestamp in the example above). Then the normal metadata for each image was stored in separate dictionaries under that key:

meta = {
    "2020-04-02 12:00:00": {
        "/path/snapshot1/VIS_SV_0_z1.png": {
            "IMAGE METADATA HERE"
        },
        "/path/snapshot1/NIR_SV_0_z1.png": {
            "IMAGE METADATA HERE"
        }
    }
}

[nfahlgren]

I didn't work out quite where to go from there though. Currently, we make a result file for each image in a "group" (what we call co-processing). So we could do that here, where the intention of the grouping is to analyze 2+ images together but collect data for them separately (VIS and NIR is sort of a good example of this). But alternatively, the grouping or series might have one set of outputs (like using the fluorescence images to calculate Fv/Fm for example), so then you only want one result file. For the latter then, we need to somehow merge the image metadata into one output. In some cases, I think this might be trivial but in others not? I started to think about adding another attribute to WorkflowConfig called merge_group that could control this...but didn't get that far.

[dschneiderch]

is it worth asking the question what type of implementation you want to see?
i see 2 options:

1. integrate the concatenation of different image files into one file into the metadata parser. eg intercepting the workflow if groupby is given to create the xarray object. then basically just start the metadata_parser over. at least, delete the original filename entries and create new ones with the new netcdf/zarr file
2. leave it to the user to do this before hand. we could create plantcv.util scripts to do this.

there are 2 use cases i can think of at the moment that could guide the discussion:

1. psii image series where there is an inherent relationship between each image
2. multicamera acquistions, e.g. nir and rgb and the nir needs the mask from rgb

EDIT: add your point about different outputs (individual vs merged) in different group circumstances

[dschneiderch]

since the metadata for each file is written to disk in that json file, couldn't we just keep that file name and write to the original metadata file if eg coprocess=True but use a new metadata file for the grouped object if coprocess=False
I think that would still work in the parallel workflow?

[nfahlgren]

Regarding your second idea about keeping coprocess in combination with grouping, I think that makes sense. I think it's similar to what I was thinking the "merge_group" would do (i.e. do you want an output per image or per group).

If you can join the meeting on Tuesday, maybe we could map out the options you listed above. I think you're right about the options and the current use cases.