Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/master' into update_readme
Browse files Browse the repository at this point in the history
  • Loading branch information
CasperWA committed Dec 21, 2020
2 parents 0c7c4a5 + 2673be6 commit 3737786
Show file tree
Hide file tree
Showing 18 changed files with 640 additions and 68 deletions.
9 changes: 4 additions & 5 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
name: Scheduled validator

on:
push:
on: [push, pull_request]

jobs:
validate:
name: OPTiMaDe validator
name: OPTIMADE validator
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 3.7
- name: Install OPTiMaDe
- name: Install OPTIMADE
run: |
python -m pip install --upgrade pip
pip install optimade
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ __pycache__/
*.py[cod]
*$py.class

*~
.DS_Store

# C extensions
*.so

Expand Down
22 changes: 17 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,12 @@ For convenience, you can also access the most recent list of providers in the fo

- [https://providers.optimade.org/providers.json](https://providers.optimade.org/providers.json)

You can contribute a new provider or amend the current information by creating a pull request to [the `providers` repository](https://github.com/Materials-Consortia/providers).
Additionally, there is a providers dashboard that includes the results of validation of listed implementations.
This can be accessed at the following URL:

- [https://optimade.org/providers-dashboard](https://optimade.org/providers-dashboard)

You can contribute a new provider or amend the current information by creating a pull request to [the `providers` repository](https://github.com/Materials-Consortia/providers).

## Repository organization

Expand All @@ -27,15 +31,23 @@ The repository is organized this way:

- `/src/info/<version>/info.json` is the proper response to the info endpoint formatted according to OPTIMADE version `<version>` and any later version that uses a format that is backward compatible with this version.

- `/_redirect` specify http rewrites to map index meta-database URLs `/<version>/info` and `/<version>/links` to the corresponding files under `src/`, as well as `/providers.json`.

- `/<version>` directories contain symlinks `info.html` and `links.html` to the corresponding files under `src/` to somewhat support hosting solutions that do not understand the instructions in `_redirect` but which support "pretty URLs".
- `/src/index-metadbs/<provider_name>/<version>/info.json` and `/src/index-metadbs/<provider_name>/<version>/links.json` are static Index Meta-Databases that are hosted in this repository for those providers that only have one main sub-database (or very few sub-databases) and do not wish to maintain one on their own.
See more details and instructions in `/src/index-metadbs/README.md`.

- `/providers.json` is a symbolic link continously updated to point at the latest version of the providers.json file under `/src/` to somewhat support hosting solutions that do not understand the instructions in `_redirect`.
- `/_redirects` specifies http rewrites to map index meta-database URLs `/<version>/info` and `/<version>/links` to the corresponding files under `src/`, as well as `/providers.json`.
It also creates `/index-metadbs/<provider_name>/<version>/info.json` and `/index-metadbs/<provider_name>/<version>/links.json` URLs to point to the corresponding files in the `/src/index-metadbs` subfolders.
This is used by the deploy tool Netlify.

Presently, only the `v1` version of the formats exist.
Hence, to update the list of providers, file a pull-request against the repository to edit `/src/links/v1/providers.json`.

## Requirements to be listed in this providers list

It is a policy of this providers list ([providers.optimade.org](http://providers.optimade.org)) that links inside `providers.json` must be links to an [OPTIMADE Index Meta-Database](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#32index-meta-database).

If you only have one or few databases in your implementation, and you do not want to host an Index Meta-Database yourself, you can host the Index Meta-Database directly in this repository.
You can find instructions [here](./src/index-metadbs).

## Fork this repository to set up an index meta-database for your own databases

- Click on the Fork button in GitHub and follow the instructions.
Expand Down
11 changes: 11 additions & 0 deletions _redirects
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
# The exclamation mark at the end of the HTTP code is to make sure
# the redirect is always followed even if there is a file at that page.
# See https://docs.netlify.com/routing/redirects/rewrites-proxies/#shadowing

# Returning 200 just returns the content without doing a HTTP redirect, i.e.
# without changing the URL, that is what we want to achieve here
# https://docs.netlify.com/routing/redirects/redirect-options/#http-status-codes
/providers.json /src/links/v1/providers.json 200!
/v1/links /src/links/v1/providers.json 200!
/v1/info /src/info/v1/info.json 200!

# Provide redirects for index meta-databases (removing the JSON at the end)
/index-metadbs/:providerid/:version/links /src/index-metadbs/:providerid/:version/links.json 200!
/index-metadbs/:providerid/:version/info /src/index-metadbs/:providerid/:version/info.json 200!
46 changes: 46 additions & 0 deletions src/index-metadbs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Static Index Meta-Databases

## Goal

This folder hosts static versions of Index Meta-Databases for those providers that only have one main sub-database (or very few sub-databases) and do not wish to maintain an [OPTIMADE Index Meta-Database](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#32index-meta-database) themselves.

Note: while providing an Index Meta-Database is not required by the OPTIMADE API specification, it is instead required in order to be listed in this List of Providers ([providers.optimade.org](http://providers.optimade.org)).

## When should I use a static Index Meta-Database hosted in this repository

If you add a new provider and do not wish to host an Index Meta-Database yourself, feel free to add an Index Meta-Database here.

However, if your Index Meta-Database is rapidly changing (e.g., because the list of sub-databases changes often), please host it yourself to avoid multiple commits and pull requests to this repository at every change.
Rare changes are instead of course welcome, e.g., if you have to change your DNS name or the description, or if you need just to add one static new sub-database, like a test database.

Note that "changes" here refer solely to changes to the *list of sub-databases*; the content of each sub-database can change at any time without the need to modify the Index Meta-Database.

## Instructions on how to add a new static Index Meta-Database

1. Go to the folder `src/index-metadbs`, copy the existing template folder (`exmpl`) to a new folder, where the new folder name should be the identifier of your provider (we will use `exmpl2` here and in the following as the name of the new provider).
In particular, this will contain a `v1` subfolder, with two files inside it: `info.json` and `links.json`.

2. Adapt the content of the `links.json` file.
In particular, provide a new `links` entry resource object for each sub-database that you want to link to.
In the special case in which you have a single "main" sub-database, just change the existing values as follows:

- change the `id` to your provider identifier: `"exmpl" -> "exmpl2"`
- change the attributes `name`, `description` and `homepage` to contain the correct content.
Please reuse the *same* content as the one you specified in the main `providers.json` file.
- point the `base_url` to the base URL of your OPTIMADE implementation.
Note, this should be the un-versioned base URL (see [the OPTIMADE API specification](https://github.com/Materials-Consortia/OPTiMaDe/blob/develop/optimade.rst#base-url) for more information on this).

3. Adapt the content of the `info.json` file.
In particular, you should change two fields:

- change the URL of the `available_api_versions` by replacing `exmpl` with your identifier: `http://providers.optimade.org/index-metadbs/exmpl2/v1/`.
- change the `id` inside `data -> relationships -> default -> data -> id` from `exmpl` to the correct ID from the list of links in the `links.json` file.
As [explained in the OPTIMADE API specification](https://github.com/Materials-Consortia/OPTiMaDe/blob/develop/optimade.rst#base-info-endpoint), this should be the ID of the database that should be considered as the "default" sub-database by clients.

If you only have one sub-database and you followed the instructions above, you should use here your provider identifier.
If you do not wish to have a default database, set the `relationships` value to an empty dictionary or set the value of `relationships -> default -> data` to `null`.

4. In the top-level `providers.json` file, point the `base_url` of your provider to `http://providers.optimade.org/index-metadbs/exmpl2/`.

5. Create a pull request, and check that all automated continuous-integration tests pass.
Also, you can check that the new Index Meta-Database properly works at the expected link using the Netlify preview (just click on the `netlify/optimade-providers/deploy-preview` entry of the GitHub checks that will appear in the GitHub PR Conversation page after a few seconds).
42 changes: 42 additions & 0 deletions src/index-metadbs/cod/v1/info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"meta" : {
"api_version" : "1.0.0",
"query" : {
"representation" : "/info"
},
"more_data_available" : false,
"schema" : "https://schemas.optimade.org/openapi/v1.0/optimade_index.json"
},
"data" : {
"attributes" : {
"available_api_versions" : [
{
"version" : "1.0.0",
"url" : "http://providers.optimade.org/index-metadbs/cod/v1/"
}
],
"is_index" : true,
"formats" : [
"json"
],
"entry_types_by_format" : {
"json" : []
},
"available_endpoints" : [
"info",
"links"
],
"api_version" : "1.0.0"
},
"type" : "info",
"relationships" : {
"default" : {
"data" : {
"id" : "cod",
"type" : "links"
}
}
},
"id" : "/"
}
}
34 changes: 34 additions & 0 deletions src/index-metadbs/cod/v1/links.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"meta" : {
"api_version" : "1.0.0",
"query" : {
"representation" : "/links"
},
"more_data_available" : false,
"schema" : "https://schemas.optimade.org/openapi/v1.0/optimade_index.json"
},
"data" : [
{
"id" : "cod",
"type" : "links",
"attributes" : {
"name": "Crystallography Open Database",
"description": "Open-access collection of crystal structures of organic, inorganic, metal-organics compounds and minerals, excluding biopolymers",
"base_url": "https://www.crystallography.net/cod/optimade",
"homepage": "https://www.crystallography.net/cod",
"link_type": "child"
}
},
{
"id" : "optimade-providers",
"type" : "links",
"attributes" : {
"name": "OPTIMADE Providers Index Meta-Database",
"description": "The list of OPTIMADE providers maintained by Materials-Consortia",
"base_url": "https://providers.optimade.org",
"homepage": "https://www.optimade.org",
"link_type": "providers"
}
}
]
}
42 changes: 42 additions & 0 deletions src/index-metadbs/exmpl/v1/info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"meta" : {
"api_version" : "1.0.0",
"query" : {
"representation" : "/info"
},
"more_data_available" : false,
"schema" : "https://schemas.optimade.org/openapi/v1.0/optimade_index.json"
},
"data" : {
"attributes" : {
"available_api_versions" : [
{
"version" : "1.0.0",
"url" : "http://providers.optimade.org/index-metadbs/exmpl/v1/"
}
],
"is_index" : true,
"formats" : [
"json"
],
"entry_types_by_format" : {
"json" : []
},
"available_endpoints" : [
"info",
"links"
],
"api_version" : "1.0.0"
},
"type" : "info",
"relationships" : {
"default" : {
"data" : {
"id" : "exmpl",
"type" : "links"
}
}
},
"id" : "/"
}
}
34 changes: 34 additions & 0 deletions src/index-metadbs/exmpl/v1/links.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"meta" : {
"api_version" : "1.0.0",
"query" : {
"representation" : "/links"
},
"more_data_available" : false,
"schema" : "https://schemas.optimade.org/openapi/v1.0/optimade_index.json"
},
"data" : [
{
"id" : "exmpl",
"type" : "links",
"attributes" : {
"name": "Example provider",
"description": "Provider used for examples, not to be assigned to a real database",
"base_url": null,
"homepage": "https://example.com",
"link_type": "child"
}
},
{
"id" : "optimade-providers",
"type" : "links",
"attributes" : {
"name": "OPTIMADE Providers Index Meta-Database",
"description": "The list of OPTIMADE providers maintained by Materials-Consortia",
"base_url": "https://providers.optimade.org",
"homepage": "https://www.optimade.org",
"link_type": "providers"
}
}
]
}
43 changes: 43 additions & 0 deletions src/index-metadbs/mp/v1/info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
"meta" : {
"api_version" : "1.0.0",
"query" : {
"representation" : "/info"
},
"more_data_available" : false,
"schema" : "https://schemas.optimade.org/openapi/v1.0/optimade_index.json"
},
"data" : {
"attributes" : {
"available_api_versions" : [
{
"version" : "1.0.0",
"url" : "http://providers.optimade.org/index-metadbs/mp/v1/"
}
],
"is_index" : true,
"formats" : [
"json"
],
"entry_types_by_format" : {
"json" : []
},
"available_endpoints" : [
"info",
"links"
],
"api_version" : "1.0.0"
},
"type" : "info",
"relationships" : {
"default" : {
"data" : {
"id" : "mp",
"type" : "links"
}
}
},
"id" : "/"
}
}

34 changes: 34 additions & 0 deletions src/index-metadbs/mp/v1/links.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"meta": {
"api_version": "1.0.0",
"query": {
"representation": "/links"
},
"more_data_available": false,
"schema": "https://schemas.optimade.org/openapi/v1.0/optimade_index.json"
},
"data": [
{
"id": "mp",
"type": "links",
"attributes": {
"name": "The Materials Project",
"description": "The Materials Project OPTIMADE endpoint",
"base_url": "https://optimade.materialsproject.org",
"homepage": "https://www.materialsproject.org",
"link_type": "child"
}
},
{
"id": "optimade-providers",
"type": "links",
"attributes": {
"name": "OPTIMADE Providers Index Meta-Database",
"description": "The list of OPTIMADE providers maintained by Materials-Consortia",
"base_url": "https://providers.optimade.org",
"homepage": "https://www.optimade.org",
"link_type": "providers"
}
}
]
}
Loading

0 comments on commit 3737786

Please sign in to comment.