Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As part of the PRWG discussions and unrelated discussions in the CAL team, we realized our "documentation" of the EuXFEL file structure is both the only one existing as well as out of date. In addition as part of the former, it would be useful to describe the file format in a more generic way and not only bound to files written by the DAQ from Karabo.
This is a first draft of this. I expect a lot of discussions and further work on this. The intention was to describe the structure as abstract as possible to how and what data actually ends up there, yet still with comments to how the "typical use case" in the form of recorded DAQ runs looks like.
I left some of my own open questions in the document, but a brief summary:
1.2
) and only track changes?RUN
top-level group even if there's noCONTROL
groupMETADATA
datasets? What about custom datasets, like forpycalibration
?first/last/status
datasets inINDEX/<source>
? Never saw those myself, move to appendix or ignore entirely?You can find a built version of this branch here.