Minor re-design #82
Merged
Minor re-design #82
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CasperWA
force-pushed
the
issue_66_pagination
branch
from
ad61c98 to
c1719df
Compare
Codecov Report
@@ Coverage Diff @@
## main #82 +/- ##
==========================================
+ Coverage 88.00% 88.17% +0.17%
==========================================
Files 43 40 -3
Lines 1250 1192 -58
==========================================
- Hits 1100 1051 -49
+ Misses 150 141 -9
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
CasperWA
force-pushed
the
issue_66_pagination
branch
3 times, most recently
from
150d4e0 to
a3b70b2
Compare
CasperWA
force-pushed
the
issue_66_pagination
branch
from
a3b70b2 to
84aaf0a
Compare
CasperWA
force-pushed
the
issue_66_pagination
branch
from
c42de7d to
e216ffc
Compare
CasperWA
force-pushed
the
issue_66_pagination
branch
4 times, most recently
from
48a0231 to
6d02673
Compare
CasperWA
force-pushed
the
issue_66_pagination
branch
from
9e5c2ef to
d369865
Compare
A new data structure has been added to represent a gateway query response: `GatewayQueryResponse`. This new response changes the structure for the value of the top-level `data` field by always returning a dictionary/JSON object. The keys of the dictionary are the database `id`s and the value is a list of the returned resources from each database. In this way there is no need to alter/add anything to the returned resources, which has been the case up to now, where the `id` for each resource was prepended with the database `id`. This is however still in place, somewhat, if a `QueryResource` is not used for the query, but a straight-through result is desired. Then an OPTIMADE-compliant response is produced, where the database `id` is added as a `meta` entry to each returned resource. To handle this new approach a new utility function for processing an OPTIMADE database response has been written and implemented. An Enum for OPTIMADE endpoint entry types has been implemented with methods for returning the correct pydantic model for either the resources or response (both many and single), removing the need to pass the import path for these models. Make entry ID ambiguity a warning instead of an exception.
Only supply the pydantic model to `response_model` that represents the successful response. Add a dictionary of erroneous status codes with the ErrorResponse pydantic model to `responses` for all routes. Fix bugs caught by the tests introduced in the latest commit, modularizing out processing a database response. Use new specific exception classes from `optimade`.
CasperWA
force-pushed
the
issue_66_pagination
branch
from
d369865 to
125157d
Compare
- Minor renaming of routers in main.py. - Fix validation errors when returning error responses. - Minimize special meta fields for query response. - Utilize local `LinksMapper` in `DatabaseMapper`.
Instead of returning a `GatewayQueryResponse` for the single `QueryResource` endpoints as well as the `GET /search` endpoint, a `QueriesResponseSingle` is now returned. This is done to ensure all necessary information is communicated along with the actual data response, like the used query parameters and such. The example/test gateways have been updated to use "correct" database `id`s, where the provider is prepended, separating the provider `id` and the database `id` with a slash.
This is instead of redirecting to /gateways/{id}/structures as this is
now moving more towards *not* reproducing an actual OPTIMADE
implementation.
Add possibility to return responses from `GET /search` as an OPTIMADE entry-listing response or as the "standard" gateway query response. In doing this, some things have been optimized. Mainly the logic surrounding retrieving and naming databases in the `POST /search` endpoint. In order to return a query response as a valid OPTIMADE response, a method has been added to the `QueryResource` model, which works similarly to `GET /gateways/<id>/structures`, but working from the basis of the finished query. It updates the entry `id`s by prepending the `provider/database/` name, making the `id`s unique. Extra safety has been added to the creation of gateway resources, as the `resource_factory()` now expects a pre-treating `GatewayCreate` entity, where there are no "unknown" database `id`s in the `database_ids` attribute. Unknown meaning that all `database_ids` must be represented in the `databases` attribute.
Remove root validator from `Response` for `GatewayQueryResponse`. The `GatewayQueryResponse` is special, since it can allow `errors` to be present together with `data`. And `meta` is guaranteed to always be present, so there is no need for the validator. Also, update the test data to respect the new validators for `chemical_formula_anonymous`. Update optimade and pre-commit dependencies.
Remove the routers and tests. This also simplifies the query processing as a `QueryResource` will *always* be used, and hence all special cases where it is not used can be removed.
CasperWA
force-pushed
the
issue_66_pagination
branch
from
3cdbde5 to
0cd2480
Compare
CasperWA
force-pushed
the
issue_66_pagination
branch
from
0cd2480 to
36e5ace
Compare
Apparently its dependency `jsmin` can result in unstable builds/installations.
CasperWA
force-pushed
the
issue_66_pagination
branch
from
36e5ace to
784c5af
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR was originally intended to implement pagination according to the discussion in #66.
However, due to issues in the re-design mentioned below, this PR is now being remade to represent a minor re-design of the gateway.
To begin with, the response from a gateway query has been re-modelled:
A new data structure has been added to represent a gateway query response:
GatewayQueryResponse. This new response changes the structure for the value of the top-leveldatafield by always returning a dictionary/JSON object. The keys ofthe dictionary are the database
ids and the value is a list of the returned resources from each database. In this way there is no need to alter/add anything to the returned resources, which has been the case up to now, where theidfor each resource was prepended with the databaseid. This is however still in place, somewhat, if aQueryResourceis not used for the query, but a straight-through result is desired. Then an OPTIMADE-compliant response is produced, where the databaseidis added as ametaentry to each returned resource.To handle this new approach a new utility function for processing an OPTIMADE database response has been written and implemented.
An Enum for OPTIMADE endpoint entry types has been implemented with methods for returning the correct pydantic model for either the resources or response (both many and single), removing the need to pass the import path for these models.