BSUoS import URLs change

[tlocke]

The URLs for the latest download of BSUoS data changes on every upload. Looks like we'll have to get Chellow to examine the page to get the URLs.

[tlocke]

I've made the following changes:

• It now doesn't wait until there's a full month's data to download, it'll use whatever it's got.
• It stores all the settlement runs (II, SF, RF).
• There's a global alert (red text on every page) if the importer fails.
• Since the URLs of the current data change on every upload, I've added a feature where the importer scrapes the page for the latest URLs.
• For loading in historical data there's a list of URLs in the configuration that the importer will additionally look at.

So, to install it follow these steps:

• Disable the bsuos importer by setting 'enabled' to 'false'.
• Execute the command: chellow stop
• pip install --upgrade chellow
• chellow start
• Delete all the BSUoS rate scripts except the first one.
• Make sure that the first rate script covers exactly one month.
• Update the BSUoS properties to be:

{
"discover_urls": true,
"enabled": true}

• Click the 'import now' button and the latest data should import.
• If you want to import historical data, you can use the URLs of historical data from the National Grid website (https://www.nationalgrid.com/uk/electricity/charging-and-methodology/balancing-services-use-system-bsuos-charges) and put them in the 'urls' list. For example:

{
"discover_urls": true,
"enabled": true,
"urls": ["https://www.nationalgrid.com/sites/default/files/documents/2016-2017%20SF.xls", "https://www.nationalgrid.com/sites/default/files/documents/8589938560-2015-2016%20SF.xlsx"]}

• Remember to remove the items from the 'urls' list after you've imported all the data, otherwise the importer will download them every time.