-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Add API tests for metadata handling #107
base: develop
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,91 @@ | ||
#!/usr/bin/env python3 | ||
|
||
# (C) Copyright 2020 ECMWF. | ||
# | ||
# This software is licensed under the terms of the Apache Licence Version 2.0 | ||
# which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. | ||
# In applying this licence, ECMWF does not waive the privileges and immunities | ||
# granted to it by virtue of its status as an intergovernmental organisation | ||
# nor does it submit to any jurisdiction. | ||
# | ||
|
||
import pytest | ||
|
||
from earthkit.data import from_source | ||
from earthkit.data.testing import earthkit_examples_file | ||
|
||
try: | ||
from earthkit.data.metadata import Metadata | ||
except ImportError: | ||
Metadata = None | ||
|
||
try: | ||
from earthkit.data.metadata import RawMetadata | ||
except ImportError: | ||
RawMetadata = None | ||
|
||
try: | ||
from earthkit.data.readers.grib.metadata import GribMetadata | ||
except ImportError: | ||
GribMetadata = None | ||
|
||
|
||
def test_abstract_metadata(): | ||
# Metadata properties: | ||
# * immutable | ||
# * behaves like a dict | ||
# * can be updated (creating a new object, either a copy or a view) | ||
# Updates can be made with compatible subclasses, e.g. update a GribMetadata with a RawMetadata | ||
assert Metadata is not None | ||
|
||
|
||
def test_raw_metadata(): | ||
assert RawMetadata is not None | ||
assert issubclass(RawMetadata, Metadata) | ||
|
||
md = RawMetadata({"shortName": "2t", "perturbationNumber": 5}) | ||
assert md["shortName"] == "2t" | ||
assert md["perturbationNumber"] == 5 | ||
assert md.get("shortName") == "2t" | ||
with pytest.raises(KeyError): | ||
md["nonExistentKey"] | ||
assert md.get("nonExistentKey", 12) == 12 | ||
|
||
md2 = md.update({"centre": "ecmf", "perturbationNumber": 8}) | ||
sentinel = object() | ||
assert md.get("centre", sentinel) is sentinel | ||
assert md["perturbationNumber"] == 5 | ||
assert md2["shortName"] == "2t" | ||
assert md2["perturbationNumber"] == 8 | ||
assert md2["centre"] == "ecmf" | ||
|
||
md3 = RawMetadata({"step": 24, "centre": "unknown"}).update(md2) | ||
assert md3["step"] == 24 | ||
assert md3["centre"] == "ecmf" | ||
assert md3["shortName"] == "2t" | ||
|
||
|
||
def test_grib_metadata(): | ||
assert GribMetadata is not None | ||
assert issubclass(GribMetadata, Metadata) | ||
|
||
with pytest.raises(TypeError): | ||
GribMetadata({"shortName": "u", "typeOfLevel": "pl", "levelist": 1000}) | ||
|
||
f = from_source("file", earthkit_examples_file("test_single.grib")) | ||
md = f[0].metadata() | ||
|
||
assert isinstance(md, GribMetadata) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you suggest only to change the behaviour of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, that was the idea. We keep the current API, but change the underlying metadata object, and we return it if the user is not asking for specific keys/namespaces |
||
|
||
assert str(md["dataDate"]) == "20170427" | ||
assert md["shortName"] == "2t" | ||
|
||
extra = RawMetadata({"shortName": "2ta"}) | ||
with pytest.raises(TypeError): | ||
extra.update(md) | ||
|
||
updated = md.update(extra) | ||
assert md["shortName"] == "2t" | ||
assert updated["shortName"] == "2ta" | ||
# When we write the GribMetadata, the GRIB keys should be set in the correct | ||
# order to produce the expected result |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What should happen for:
Should we raise
KeyError
or return None as adict
would do?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would return None, to match
dict
and the behaviour of the default value when givenThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @oiffrig! There is another question related to the type cast. On a
Field
we can callmetadata()
like this to specify the value type we want:How would the Metadata object support these? The last 4 calls are of course supported for GRIB by
[]
andget()
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not totally sure. What I would suggest is to have that as an overridable behaviour: the metadata object knows a few types and can convert from them (raising an error if the type is not supported), and specific implementations (e.g. GRIB) can extend the default behaviour by using functionalities of the underlying backend
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. Would that mean that
get
should work like that: