Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration nexus file #147

Closed
wants to merge 111 commits into from
Closed

Integration nexus file #147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
9efc8df
updated plugin structure
aalbino2 Aug 19, 2024
6bc5efb
added pynxtools dependency
aalbino2 Aug 19, 2024
ccd1bce
Apply suggestions from Sarthak's code review
aalbino2 Aug 20, 2024
911b4f3
ruff linting
aalbino2 Aug 20, 2024
c7756da
Ruff linting 2
ka-sarthak Aug 20, 2024
8a28d66
Apply suggestions from code review
aalbino2 Aug 20, 2024
da400c0
changed xrd parser folder
aalbino2 Aug 21, 2024
a305ece
last fixes and descriptions
aalbino2 Aug 21, 2024
f320275
description of general schema
aalbino2 Aug 21, 2024
6eff9d4
changed general package into a module
aalbino2 Aug 21, 2024
eefe625
Implement write nexus section based on the populated nomad archive
ka-sarthak Jun 11, 2024
5f304f2
Update path of populate_nexus_subsection
ka-sarthak Aug 27, 2024
f813d22
app def missing. (#108)
RubelMozumder Aug 28, 2024
0ab0e6b
Run Python test action for all PR
ka-sarthak Aug 29, 2024
3aeb548
Fix for rebasing issues
ka-sarthak Sep 3, 2024
cd342fd
updated plugin structure
aalbino2 Aug 19, 2024
df6219b
added pynxtools dependency
aalbino2 Aug 19, 2024
ce1e60b
Apply suggestions from code review
aalbino2 Aug 20, 2024
de7b48e
Add sections for RSM and 1D which uses HDF5 references
ka-sarthak Sep 5, 2024
47198b9
Abstract out data interaction using setter and getter; allows to use …
ka-sarthak Sep 5, 2024
44ea11f
Use arrays, not references, in the `archive.results` section
ka-sarthak Sep 5, 2024
6b66448
Lock the state for using nexus file and corresponding references
ka-sarthak Sep 5, 2024
8d736b6
Populate results without references
ka-sarthak Sep 5, 2024
cd36d15
Make a general reader for raw files
ka-sarthak Nov 20, 2024
1d612f1
Remove nexus flags
ka-sarthak Nov 20, 2024
f796a79
Add quantity for auxialiary file
ka-sarthak Nov 20, 2024
6c59dad
Fix rebase
ka-sarthak Dec 3, 2024
ad98a38
Make integration_time as hdf5reference
ka-sarthak Dec 4, 2024
1072dc0
Reset results (refactor)
ka-sarthak Dec 4, 2024
65c8659
Add backward compatibility
ka-sarthak Dec 4, 2024
d5445ff
Refactor reader
ka-sarthak Dec 4, 2024
6f16f44
add missing imports
ka-sarthak Dec 4, 2024
d170d30
AttrDict class
ka-sarthak Dec 4, 2024
4f7bc83
Make concept map global
ka-sarthak Dec 4, 2024
a3295b7
Add function to remove nexus annotations in concept map
ka-sarthak Dec 4, 2024
89cdbc9
Move try block inside walk_through_object
ka-sarthak Dec 4, 2024
54902f8
Fix imports
ka-sarthak Dec 4, 2024
393c572
Add methods for generating hdf5 file
ka-sarthak Dec 4, 2024
d2d1d1e
Rename auxiliary file
ka-sarthak Dec 4, 2024
30caf99
Expect aux file to be .nxs in the beginning
ka-sarthak Dec 4, 2024
2f42f70
Add attributes for hdf5: data_dict, dataset_paths
ka-sarthak Dec 4, 2024
2fef457
Method for adding a quantity to hdf5_data_dict
ka-sarthak Dec 4, 2024
893d819
Abstract out methods for creating files based on hdf5_data_dict
ka-sarthak Dec 4, 2024
921a347
Add dataset_paths for nexus
ka-sarthak Dec 4, 2024
9f0cedf
Some reverting back
ka-sarthak Dec 4, 2024
f4fa2bd
Minor fixes
ka-sarthak Dec 4, 2024
97a4083
Refactor populate_hdf5_data_dict: store a reference to be made later
ka-sarthak Dec 5, 2024
191cd9d
Handle shift from nxs to hdf5
ka-sarthak Dec 5, 2024
8f4c4f4
Set hdf5 references after aux file is created
ka-sarthak Dec 5, 2024
5f378d2
Cleaning
ka-sarthak Dec 5, 2024
fdd2222
Fixing
ka-sarthak Dec 5, 2024
f49e168
Redefine result sections instead of extending
ka-sarthak Dec 5, 2024
f6f351c
Remove plotly plots from ELN
ka-sarthak Dec 6, 2024
7cd535e
Read util for hdf5 ref
ka-sarthak Dec 6, 2024
9d67aa2
Fixing
ka-sarthak Dec 6, 2024
9ad1efe
Move hdf5 handling into a util class
ka-sarthak Dec 6, 2024
8b8b9f4
Refactor instance variables
ka-sarthak Dec 6, 2024
382231b
Reset data dicts and reference after each writing
ka-sarthak Dec 6, 2024
f867a9a
Fixing
ka-sarthak Dec 6, 2024
2b75a79
Overwrite dataset if it already exists
ka-sarthak Dec 6, 2024
97f4a3f
Refactor add_dataset
ka-sarthak Dec 6, 2024
10bf72e
Reorganize and doctrings
ka-sarthak Dec 6, 2024
b5b5666
Rename variable
ka-sarthak Dec 6, 2024
88a9dde
Add read_dataset method
ka-sarthak Dec 6, 2024
bbd0a86
Cleaning
ka-sarthak Dec 6, 2024
94bdde7
Adapting schema with hdf5 handler
ka-sarthak Dec 6, 2024
a859be2
Cooments, minor refactoring
ka-sarthak Dec 6, 2024
fea1724
Fixing; add `hdf5_handler` as an attribute for archive
ka-sarthak Dec 6, 2024
df6b571
Reorganization
ka-sarthak Dec 6, 2024
ae96f5d
Fixing
ka-sarthak Dec 6, 2024
e6cd6b8
Refactoring
ka-sarthak Dec 6, 2024
7b7db2b
Cleaning
ka-sarthak Dec 6, 2024
11e4d38
Try block for using hdf5 handler: dont fail early, as later normaliza…
ka-sarthak Dec 7, 2024
de4f605
Extract units from dataset attrs when reading
ka-sarthak Dec 7, 2024
7f12438
Fixing
ka-sarthak Dec 7, 2024
d7f69f7
Linting
ka-sarthak Dec 7, 2024
be9dfa8
Make archive_path optional in add_dataset
ka-sarthak Dec 9, 2024
fb9f1c7
Rename class
ka-sarthak Dec 9, 2024
51915c3
attrs for add_dataset; use it for units
ka-sarthak Dec 9, 2024
090bc18
Add add_attribute method
ka-sarthak Dec 9, 2024
6b2a95e
Refactor add_attribute
ka-sarthak Dec 9, 2024
3207e8e
Add plot attributes: 1D
ka-sarthak Dec 9, 2024
dc3781c
Refactor hdf5 states
ka-sarthak Dec 10, 2024
24b4a9e
Add back plotly figures
ka-sarthak Dec 10, 2024
17a0088
rename auxiliary file name if changed by handler
ka-sarthak Dec 10, 2024
c3302c4
Add referenced plots
ka-sarthak Dec 10, 2024
e993deb
Allow hard link using internel reference
ka-sarthak Dec 10, 2024
d753f4a
Add sections for plots
ka-sarthak Dec 10, 2024
762fe86
Comment out validation
ka-sarthak Dec 10, 2024
18ed92c
Add archive paths for the plot subsections
ka-sarthak Dec 11, 2024
ff286c6
Add back validation with flag
ka-sarthak Dec 11, 2024
374b6bb
Use nexus flag
ka-sarthak Dec 11, 2024
9464f11
Add interpolated intensity data into h5 for qspace plots
ka-sarthak Dec 13, 2024
baac2da
Use prefix to reduce len of string
ka-sarthak Dec 13, 2024
d705957
Store regularized linespace of q vectors; revise descriptions
ka-sarthak Dec 13, 2024
47df0cc
Remove plotly plots
ka-sarthak Dec 13, 2024
fbc87d1
Bring plots to overview
ka-sarthak Dec 13, 2024
c3f2ff3
Fix tests
ka-sarthak Dec 13, 2024
3ba656f
Linting; remove attr arg from add_dataset
ka-sarthak Dec 13, 2024
4a0852c
Review: move none check into method
ka-sarthak Dec 16, 2024
864d53b
Review: use 'with' for opening h5 file
ka-sarthak Dec 17, 2024
122b65f
Review: make internal states as private vars
ka-sarthak Dec 17, 2024
67ee382
Add pydantic basemodel for dataset
ka-sarthak Dec 17, 2024
645348a
Use data from variables if available for reading
ka-sarthak Dec 17, 2024
64eae6c
Review: remove lazy arg
ka-sarthak Dec 18, 2024
e3164ff
Move DatasetModel outside Handler class
ka-sarthak Dec 18, 2024
0cd7593
Nexus file generation:
RubelMozumder Dec 18, 2024
6220b5a
Remove Nexus file before rereating it.
RubelMozumder Dec 19, 2024
fc598e3
Ruff.
RubelMozumder Dec 19, 2024
08c25bc
sourcery-ai reviewer.
RubelMozumder Dec 19, 2024
b6ecf01
Reference to the NeXus entry.
RubelMozumder Dec 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/python-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
branches: [ "*" ]

permissions:
contents: read
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ dependencies = [
"nomad-lab>=1.3.6",
"xmltodict==0.13.0",
"fairmat-readers-xrd>=0.0.3",
"pynxtools@git+https://github.com/FAIRmat-NFDI/pynxtools.git@master",
]
[project.urls]
"Homepage" = "https://github.com/FAIRmat-NFDI/nomad-measurements"
Expand Down
Loading