Ensuring that links point to index meta-dbs

.github/workflows/validate.yml

@@ -1,7 +1,6 @@
 name: Scheduled validator
 
-on:
-  push:
+on: [push, pull_request]
 
 jobs:
   validate:

.gitignore

@@ -4,6 +4,9 @@ __pycache__/
 *.py[cod]
 *$py.class
 
+*~
+.DS_Store
+
 # C extensions
 *.so
 

README.md

@@ -35,3 +35,9 @@ The repository is organized this way:
 
 Presently, only the `v1` version of the formats exist.
 Hence, to update the list of providers, file a pull-request against the repository to edit `/src/links/v1/providers.json`.
+
+## Requirements to be listed in this providers list
+
+It is a policy of this providers list ([providers.optimade.org](http://providers.optimade.org)) that links inside `providers.json` must be links to an [OPTIMADE Index Meta-Database](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#32index-meta-database).
+
+If you only have one or few databases in your implementation, and you do not want to host an Index Meta-Database yourself, you can host the Index Meta-Database directly in this repository. You can find instructions [here](./src/index-metadbs).

_redirects

@@ -1,3 +1,14 @@
+# The exclamation mark at the end of the HTTP code is to make sure
+# the redirect is always followed even if there is a file at that page.
+# See https://docs.netlify.com/routing/redirects/rewrites-proxies/#shadowing
+
+# Returning 200 just returns the content without doing a HTTP redirect, i.e.
+# without changing the URL, that is what we want to achieve here
+# https://docs.netlify.com/routing/redirects/redirect-options/#http-status-codes
 /providers.json	/src/links/v1/providers.json	200!
 /v1/links	/src/links/v1/providers.json	200!
 /v1/info	/src/info/v1/info.json	200!
+
+# Provide redirects for index meta-databases (removing the JSON at the end)
+/index-metadbs/:providerid/:version/links /src/index-metadbs/:providerid/:version/links.json 200!
+/index-metadbs/:providerid/:version/info /src/index-metadbs/:providerid/:version/info.json 200!

src/index-metadbs/README.md

@@ -0,0 +1,41 @@
+# Static Index Meta-Databases
+
+## Goal
+
+This folder hosts static versions of Index Meta-Databases for those providers that only have one main sub-database (or very few sub-databases) and do not wish to maintain an Index Meta-Database themselves (Index Meta-Databases are described in the [OPTIMADE specifications](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst)).
+
+Note: while providing an (empty) Index Meta-Database is not required by the OPTIMADE API specifications, it is instead required in order to be listed in this List of Providers ([providers.optimade.org](http://providers.optimade.org)).
+
+## When should I use a static index meta-database hosted in this repository
+
+If you add a new provider and do not wish to host an Index Meta-Database yourself, feel free to add an Index Meta-Database here.
+
+However, if your Index Meta-Database is rapidly changing (e.g. because the list of sub-databases changes often), please host it yourself to avoid multiple commits and pull requests to this repository at every change.
+Rare changes are instead of course welcome, e.g., if you have to change your DNS name or the description, or if you need just to add one static new sub-database, like a test database.
+
+Note that "changes" here refer solely to changes to the *list of sub-databases*; the content of each sub-database can change at any time without the need to modify the Index Meta-Database.
+
+## Instructions on how to add a new static Index Meta-Database
+1. Go in the folder `src/index-metadbs`, copy the existing template folder (`exmpl`) to a new folder, where the folder name should be the identifier of your provider (we will use `exmpl2` here and in the following as the name of the new provider).
+   In particular, this will contain a `v1` subfolder, with two files inside it: `info.json` and `links.json`.
+
+2. Adapt the content of the `links.json` file. In particular, provide a new `link` dictionary entry for each sub-database that you want to refer to. 
+   In the special case in which you have a single "main" sub-database, just change the existing values as follows:
+
+   - change the `id` to your provider identifier: `"exmpl" -> "exmpl2"`
+   - change the attributes `name`, `description` and `homepage` to contain the correct content. Please reuse the *same* content as the one you specified in the main `providers.json` file.
+   - point the `base_url` to the base URL of your OPTIMADE implementation.
+
+3. Adapt the content of the `info.json` file. In particular, you should change two fields:
+
+   - change the URL of the `available_api_versions` by replacing `exmpl` with your identifier: `http://providers.optimade.org/index-metadbs/exmpl2/v1/`
+   - change the `id` inside `data->relationships->default->data->id` from `exmpl` to the correct ID from the list of links in the `links.json` file.
+     As explained in the OPTIMADE specifications, this should be the ID of the database that should be considered as the "default" sub-database by clients. 
+
+     If you only have one sub-database and you followed the instructions above, you should use here your provider identifier.
+     If you do not wish to have a default database, remove completely the `relationships` attribute.
+
+4. In the top-level `providers.json` file, point the `base_url` of your provider to `http://providers.optimade.org/index-metadbs/exmpl/`.
+
+5. Create a pull request, and check that all automated continuous-integration tests pass.
+   Also, you can check that the new Index Meta-Database properly works at the expected link using the Netlify preview (just click on the `netlify/optimade-providers/deploy-preview` entry of the GitHub checks that will appear in the GitHub PR Conversation page after a few seconds).

src/index-metadbs/cod/v1/info.json

@@ -0,0 +1,34 @@
+{
+   "data" : {
+      "attributes" : {
+         "available_api_versions" : [
+            {
+               "version" : "1.0",
+               "url" : "http://providers.optimade.org/index-metadbs/cod/v1/"
+            }
+         ],
+         "is_index" : true,
+         "formats" : [
+            "json"
+         ],
+         "entry_types_by_format" : {
+            "json" : []
+         },
+         "available_endpoints" : [
+            "info",
+            "links"
+         ],
+         "api_version" : "v1"
+      },
+      "type" : "info",
+      "relationships" : {
+         "default" : {
+            "data" : {
+               "id" : "cod",
+               "type" : "link"
+            }
+         }
+      },
+      "id" : "/"
+   }
+}

src/index-metadbs/cod/v1/links.json

@@ -0,0 +1,26 @@
+{
+   "data" : [
+      {
+         "id" : "cod",
+         "type" : "link",
+         "attributes" : {
+            "name": "Crystallography Open Database",
+            "description": "Open-access collection of crystal structures of organic, inorganic, metal-organics compounds and minerals, excluding biopolymers",
+            "base_url": "https://www.crystallography.net/cod/optimade",
+            "homepage": "https://www.crystallography.net/cod",
+            "link_type": "child"
+         }
+      },
+      {
+         "id" : "optimade-providers",
+         "type" : "link",
+         "attributes" : {
+            "name": "OPTIMADE Providers Index Meta-Database",
+            "description": "The list of OPTIMADE providers maintained by Materials-Consortia",
+            "base_url": "https://providers.optimade.org/",
+            "homepage": "https://www.optimade.org/",
+            "link_type": "providers"
+         }
+      }
+   ]
+}

src/index-metadbs/exmpl/v1/info.json

@@ -0,0 +1,34 @@
+{
+   "data" : {
+      "attributes" : {
+         "available_api_versions" : [
+            {
+               "version" : "1.0",
+               "url" : "http://providers.optimade.org/index-metadbs/exmpl/v1/"
+            }
+         ],
+         "is_index" : true,
+         "formats" : [
+            "json"
+         ],
+         "entry_types_by_format" : {
+            "json" : []
+         },
+         "available_endpoints" : [
+            "info",
+            "links"
+         ],
+         "api_version" : "v1"
+      },
+      "type" : "info",
+      "relationships" : {
+         "default" : {
+            "data" : {
+               "id" : "exmpl",
+               "type" : "link"
+            }
+         }
+      },
+      "id" : "/"
+   }
+}

src/index-metadbs/exmpl/v1/links.json

@@ -0,0 +1,26 @@
+{
+   "data" : [
+      {
+         "id" : "exmpl",
+         "type" : "link",
+         "attributes" : {
+            "name": "Example provider",
+            "description": "Provider used for examples, not to be assigned to a real database",
+            "base_url": null,
+            "homepage": "https://example.com",
+            "link_type": "child"
+         }
+      },
+      {
+         "id" : "optimade-providers",
+         "type" : "link",
+         "attributes" : {
+            "name": "OPTIMADE Providers Index Meta-Database",
+            "description": "The list of OPTIMADE providers maintained by Materials-Consortia",
+            "base_url": "https://providers.optimade.org/",
+            "homepage": "https://www.optimade.org/",
+            "link_type": "providers"
+         }
+      }
+   ]
+}

src/index-metadbs/tcod/v1/info.json

@@ -0,0 +1,34 @@
+{
+   "data" : {
+      "attributes" : {
+         "available_api_versions" : [
+            {
+               "version" : "1.0",
+               "url" : "http://providers.optimade.org/index-metadbs/tcod/v1/"
+            }
+         ],
+         "is_index" : true,
+         "formats" : [
+            "json"
+         ],
+         "entry_types_by_format" : {
+            "json" : []
+         },
+         "available_endpoints" : [
+            "info",
+            "links"
+         ],
+         "api_version" : "v1"
+      },
+      "type" : "info",
+      "relationships" : {
+         "default" : {
+            "data" : {
+               "id" : "cod",
+               "type" : "link"
+            }
+         }
+      },
+      "id" : "/"
+   }
+}

src/index-metadbs/tcod/v1/links.json

@@ -0,0 +1,26 @@
+{
+   "data" : [
+      {
+         "id" : "cod",
+         "type" : "link",
+         "attributes" : {
+            "name": "Theoretical Crystallography Open Database",
+            "description": "Open-access collection of theoretically calculated or refined crystal structures of organic, inorganic, metal-organic compounds and minerals, excluding biopolymers",
+            "base_url": "https://www.crystallography.net/tcod/optimade",
+            "homepage": "https://www.crystallography.net/tcod",
+            "link_type": "child"
+         }
+      },
+      {
+         "id" : "optimade-providers",
+         "type" : "link",
+         "attributes" : {
+            "name": "OPTIMADE Providers Index Meta-Database",
+            "description": "The list of OPTIMADE providers maintained by Materials-Consortia",
+            "base_url": "https://providers.optimade.org/",
+            "homepage": "https://www.optimade.org/",
+            "link_type": "providers"
+         }
+      }
+   ]
+}

src/links/v1/providers.json

@@ -26,7 +26,7 @@
       "attributes": {
         "name": "Crystallography Open Database",
         "description": "Open-access collection of crystal structures of organic, inorganic, metal-organics compounds and minerals, excluding biopolymers",
-        "base_url": "https://www.crystallography.net/cod/optimade",
+        "base_url": "http://providers.optimade.org/index-metadbs/cod",
         "homepage": "https://www.crystallography.net/cod"
       }
     },
@@ -36,7 +36,7 @@
       "attributes": {
         "name": "Example provider",
         "description": "Provider used for examples, not to be assigned to a real database",
-        "base_url": "https://example.com/optimade",
+        "base_url": "http://providers.optimade.org/index-metadbs/exmpl",
         "homepage": "https://example.com"
       }
     },
@@ -156,7 +156,7 @@
       "attributes": {
         "name": "Theoretical Crystallography Open Database",
         "description": "Open-access collection of theoretically calculated or refined crystal structures of organic, inorganic, metal-organic compounds and minerals, excluding biopolymers",
-        "base_url": "https://www.crystallography.net/tcod/optimade",
+        "base_url": "http://providers.optimade.org/index-metadbs/tcod",
         "homepage": "https://www.crystallography.net/tcod"
       }
     }