Major overhaul of the codebase to support the Traefik v2 API, followi…

…ng initial

work done in previous commit:-

41a5ef3

Also, see issue: jupyterhub#97

 - Relevant documentation has been updated in README.md, docs/source/file.md
   (renamed from toml.md)

 - KV providers (consul and etcd) have been updated to use new Traefik KV
   paths, as per:-
   https://doc.traefik.io/traefik/reference/dynamic-configuration/kv/

 - requirements.txt now makes toml an optional dependency as well as
   ruamel.yaml. This latter module is required by jupyterhub anyway, so should
   already be present on a system running JupyterHub.

 - The (thoroughly excellent) test system has had a bit of an overhaul:-

   - There were issues with repeated launching and shutting down of external
     `consul` and `etcd` servers, causing test failures. Added code to
     gracefully shutdown consul and wait for servers to launch (within
     `tests/conftest.py`) before continuing with the tests.

   - `traefik` v2 no longer has a 'storeconfig' command. Therefore, to load a
     pre-baked configuration file into the KV stores, have had to resort to
     loading configurations with either `etcdctl txn` or `consul kv import`
     commands.

   - The external file provider now watches a directory instead of a file, so
     have added a pre-baked dynamic_config directory (and file), where the
     rules.toml file will be saved and managed by the TraefikFileProviderProxy.

   - Removed the previous traefik_{consul_config,etcd_config}.toml files,
     (which acted as traefik static configuration files), and instead applied
     the static KV configuration using the CLI.

   - Refactored some of the text fixtures to try and re-use fixtures and make
     them (hopefully) a bit easier to follow.

   - Have duplicated the file_proxy and external_file_proxy pytest fixtures to
     test both toml and yaml files. (Would have preferred to parametrize the
     existing fixtures to avoid duplicating them, but couldn't figure out how).

   - `tests/test_traefik_api_auth.py` - Have had to make the test give traefik
     more of a chance to read its dynamic config before running the test.
     Previously, in traefik v1, the api authentication would be configured in
     the static config. Now the api 'middleware' is configured in the dynamic
     config, the previous wait condition of just waiting for the port to come
     up didn't give traefik enough time to set up the API authentication on
     that entrypoint / router.

 - All tests now pass on my dev system, woohoo!

README.md

@@ -19,7 +19,7 @@ depending on how traefik store its routing configuration.
 
 For **smaller**, single-node deployments:
 
-* TraefikTomlProxy
+* TraefikFileProviderProxy
 
 For **distributed** setups:
 
@@ -32,21 +32,13 @@ The [documentation](https://jupyterhub-traefik-proxy.readthedocs.io) contains a
 guide](https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/install.html)
 with examples for the three different implementations.
 
-* [For TraefikTomlProxy](https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/toml.html#example-setup)
+* [For TraefikFileProviderProxy](https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/toml.html#example-setup)
 * [For TraefikEtcdProxy](https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/etcd.html#example-setup)
 * [For TraefikConsulProxy](https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/consul.html#example-setup)
 
 
 ## Running tests
-There are some tests that use *etcdctl* command line client for etcd. Make sure
-to set environment variable `ETCDCTL_API=3` before running the tests, so that
-the v3 API to be used, e.g.:
-
-```
-$ export ETCDCTL_API=3
-```
-
-You can then run the all the test suite from the *traefik-proxy* directory with:
+You can run the all the test suite from the *traefik-proxy* directory with:
 
 ```
 $ pytest -v ./tests

docs/source/file.md

@@ -0,0 +1,194 @@
+# Using TraefikFileProviderProxy
+
+**jupyterhub-traefik-proxy** can be used with simple toml or yaml configuration files, for smaller, single-node deployments such as
+[The Littlest JupyterHub](https://tljh.jupyter.org).
+
+## How-To install TraefikFileProviderProxy
+
+1. Install **jupyterhub**
+2. Install **jupyterhub-traefik-proxy**
+3. Install **traefik**
+
+* You can find the full installation guide and examples in the [Introduction section](install.html#traefik-proxy-installation)
+
+## How-To enable TraefikFileProviderProxy
+
+You can enable JupyterHub to work with `TraefikFileProviderProxy` in jupyterhub_config.py, using the `proxy_class` configuration option.
+
+You can choose to:
+
+* use the `traefik_file` entrypoint, new in JupyterHub 1.0, e.g.:
+
+    ```python
+    c.JupyterHub.proxy_class = "traefik_file"
+    ```
+
+* use the TraefikFileProviderProxy object, in which case, you have to import the module, e.g.:
+
+    ```python
+    from jupyterhub_traefik_proxy import TraefikFileProviderProxy
+    c.JupyterHub.proxy_class = TraefikFileProviderProxy
+    ```
+
+
+## Traefik configuration
+
+Traefik's configuration is divided into two parts:
+
+* The **static** configuration (loaded only at the beginning)
+* The **dynamic** configuration (can be hot-reloaded, without restarting the proxy), 
+where the routing table will be updated continuously.
+
+Traefik allows us to have one file for the static configuration file (`traefik.toml` or `traefik.yaml`) and one or several files for the routes, that traefik would watch.
+
+```{note}
+  **TraefikFileProviderProxy**, uses two configuration files: one file for the routes (**rules.toml** or **rules.yaml**), and one for the static configuration (**traefik.toml** or **traefik.yaml**).
+```
+
+
+By **default**, Traefik will search for `traefik.toml` and `rules.toml` in the following places:
+
+* /etc/traefik/
+* $HOME/.traefik/
+* . the working directory
+
+You can override this in TraefikFileProviderProxy, by modifying the **toml_static_config_file** argument:
+
+```python
+c.TraefikFileProviderProxy.static_config_file="/path/to/static_config_filename.toml"
+```
+
+Similarly, you can override the dynamic configuration file by modifying the **dynamic_config_file** argument:
+
+```python
+c.TraefikFileProviderProxy.dynamic_config_file="/path/to/dynamic_config_filename.toml"
+```
+
+```{note}
+
+* **When JupyterHub starts the proxy**, it writes the static config once, then only edits the dynamic config file. 
+
+* **When JupyterHub does not start the proxy**, the user is totally responsible for the static config and 
+JupyterHub is responsible exclusively for the routes.
+
+* **When JupyterHub does not start the proxy**, the user should tell `traefik` to get its dynamic configuration
+from a directory. Then, one (or more) dynamic configuration file(s) can be managed externally, and `dynamic_config_file`
+will be managed by JupyterHub. This allows e.g., the administrator to configure traefik's API outside of JupyterHub.
+
+```
+
+## Externally managed TraefikFileProviderProxy
+
+When TraefikFileProviderProxy is externally managed, service managers like [systemd](https://www.freedesktop.org/wiki/Software/systemd/) 
+or [docker](https://www.docker.com/) will be responsible for starting and stopping the proxy.
+
+If TraefikFileProviderProxy is used as an externally managed service, then make sure you follow the steps enumerated below:
+
+1. Let JupyterHub know that the proxy being used is TraefikFileProviderProxy, using the *proxy_class* configuration option:
+    ```python
+    from jupyterhub_traefik_proxy import TraefikFileProviderProxy
+    c.JupyterHub.proxy_class = TraefikFileProviderProxy
+    ```
+
+2. Configure `TraefikFileProviderProxy` in **jupyterhub_config.py**
+
+   JupyterHub configuration file, *jupyterhub_config.py* must specify at least:
+   * That the proxy is externally managed
+   * The traefik api credentials
+   * The dynamic configuration file, 
+     if different from *rules.toml* or if this file is located 
+     in another place than traefik's default search directories (etc/traefik/, $HOME/.traefik/, the working directory)
+
+    Example configuration:
+    ```python
+    # JupyterHub shouldn't start the proxy, it's already running
+    c.TraefikFileProviderProxy.should_start = False
+
+    # if not the default:
+    c.TraefikFileProviderProxy.dynamic_config_file = "/path/to/somefile.toml"
+
+    # traefik api credentials
+    c.TraefikFileProviderProxy.traefik_api_username = "abc"
+    c.TraefikFileProviderProxy.traefik_api_password = "xxx"
+    ```
+
+3. Ensure **traefik.toml** / **traefik.yaml**
+
+   The static configuration file, *traefik.toml* (or **traefik.yaml**) must configure at least:
+   * The default entrypoint
+   * The api entrypoint (*and authenticate it in a user-managed dynamic configuration file*)
+   * The websockets protocol
+   * The dynamic configuration directory to watch
+    (*make sure this configuration directory exists, even if empty before the proxy is launched*)
+   * Check `tests/config_files/traefik.toml` for an example.
+
+## Example setup
+
+This is an example setup for using JupyterHub and TraefikFileProviderProxy managed by another service than JupyterHub.
+
+1. Configure the proxy through the JupyterHub configuration file, *jupyterhub_config.py*, e.g.:
+
+   ```python
+   from jupyterhub_traefik_proxy import TraefikFileProviderProxy
+
+   # mark the proxy as externally managed
+   c.TraefikFileProviderProxy.should_start = False
+
+   # traefik api endpoint login password
+   c.TraefikFileProviderProxy.traefik_api_password = "admin"
+
+   # traefik api endpoint login username
+   c.TraefikFileProviderProxy.traefik_api_username = "api_admin"
+
+   # traefik's dynamic configuration file, which will be managed by JupyterHub
+   c.TraefikFileProviderProxy.dynamic_config_file = "/var/run/traefik/rules.toml"
+
+   # configure JupyterHub to use TraefikFileProviderProxy
+   c.JupyterHub.proxy_class = TraefikFileProviderProxy
+    ```
+
+2. Create a traefik static configuration file, *traefik.toml*, e.g.:
+
+    ```
+    # the api entrypoint
+    [api]
+    dashboard = true
+
+    # websockets protocol
+    [wss]
+    protocol = "http"
+
+    # the port on localhost where traefik accepts http requests
+    [entryPoints.web]
+    address = ":8000"
+
+    # the port on localhost where the traefik api and dashboard can be found
+    [entryPoints.enter_api]
+    address = ":8099"
+
+    # the dynamic configuration directory
+    # This must match the directory provided in Step 1. above.
+    [providers.file]
+    directory = "/var/run/traefik"
+    watch = true
+   ```
+
+3. Create a traefik dynamic configuration file in the directory provided in the dynamic configuration above, to provide the api authentication parameters, e.g.
+
+    ```
+    # Router configuration for the api service
+    [http.routers.router-api]
+    rule = "Host(`localhost`) && PathPrefix(`/api`)"
+    entryPoints = ["enter_api"]
+    service = "api@internal"
+    middlewares = ["auth_api"]
+
+    # authenticate the traefik api entrypoint
+    [http.middlewares.auth_api.basicAuth]
+    users = [ "api_admin:$apr1$eS/j3kum$q/X2khsIEG/bBGsteP.x./",]
+    ```
+
+4. Start traefik with the configuration specified above, e.g.:
+    ```bash
+    $ traefik --configfile traefik.toml
+    ```