Decoupling the API and the UI

[chrismytton]

Problem

Currently the API and the UI are very tightly coupled. The logic for auth and instances lives in the UI (this repository), which makes it very tricky to provide things like a JSON list of instances (#777, #792) or allow instance creation over the API (#748). There are also two sets of popolo models which have slightly different behaviour, which has been the source of more than one bug in the past (#264).

Plan

The plan is to tease this repository and popit-api apart so that the main bulk of the logic sits in the API and the UI only interacts with the API using https requests. Once the logic for instance management and auth is in the API it makes it much easier to provide OAuth logins (#113) and the UI can essentially just be another OAuth client, probably using the Resource Owner Password Credentials Authorization Grant.

Steps

In order to achieve the above goals the following things will need to be tackled:

• Run popit-api as a separate vhost, probably on https://api.popit.mysociety.org
• Copy accounts, permissions and instances logic into the API, along with some thorough tests (currently the UI handles auth for the API)
• Add OAuth2 logins to the API
• Port the code in the UI over to use the API via server-side https calls, adding headless browser tests as we go to ensure there are no regressions
• Switch to using path-based instance names rather than subdomains for the API (Consider switching to path based instances #329)
• Merge the two sets of popolo models into one set of models which explicitly state the fields they contain rather than inferring them from schemas, more like django-popolo
• Proxy API calls so the current urls for API v0.1 continue to work correctly
• Eventually once the above is done we need to stop "embedding" popit-api in the UI repository
• Run the queue as part of the API deploy rather than the UI (since it's the API importer that uses it)
• Port emails which are currently sent via the UI for account creation etc over to the API, probably put them in a queue for background processing.