-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[New Feature] Saving GUI & logic overhaul #150
Comments
I agree with adding some drop-in functionality to facilitate saving in GUIs. However it is not correct that you have no means to store metdata in the current state. The DataStorage objects have this option already. In case of text file saving, this information goes into the commented header. For other formats (e.g. numpy binary) you have an additional metadata file which is fine if you use the loading functionality if DataStorage to read it back into memory. HDF5 came up several times in the past but was always turned down by most users due to the unfamiliarity and non-human-readable nature. Many people use their own data post-processing scripts and a text file or numpy file simplifies handling. |
I see. I wasn't aware of the HDF5 history. Also thanks for the suggestion on how it could be implemented. With regards to the DataStorage objects supporting metadata, I meant the GUIs do not allow users to add metadata to the saved files (at least not that I know of). I wasn't being clear. |
Feature Description
There is no consistent saving interface across all modules (e.g. some give the option to have custom nametags, some do not). Also, the only way users can add additional metadata is in the filenames which is not ideal (difficult to parse, character limits etc).
We should have a common saving widget (and accompanying logic) that is imported into every GUI for consistent saving behaviour. There should also be an option to add custom metadata to the saved data (that is not appended to the filename but is instead stored within the file). This could simplify the organisation and sorting of large amounts of experimental data without having to solely rely on filename and autogenerated metadata.
Related Problem
No response
Considered Alternatives
This is a longterm overhaul given the scope of the work.
Additional Context
An additional feature could be the use of the HDF5 format (see Wikipedia here and Python library here) which is designed for neatly storing large amounts of (possibly mixed) data. HDF5 natively supports metadata as well. One can then easily sort and filter data based on metadata tags to retrieve only data they are interested in. A GUI tool for opening and accessing the HDF file could also be added to Qudi. There are examples here and here which are not Python-based but give an idea of what is possible.
Contact Details
No response
The text was updated successfully, but these errors were encountered: