Skip to content

Commit

Permalink
Merge pull request #79 from dxw/docs/backfill-adrs
Browse files Browse the repository at this point in the history
Backfill some ADRs
  • Loading branch information
edavey authored Nov 5, 2024
2 parents bd04941 + dfda5a1 commit da11cae
Show file tree
Hide file tree
Showing 4 changed files with 230 additions and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ e.g.
## Architecture decision records

We use ADRs to document architectural decisions that we make. They can be found
in doc/architecture/decisions and contributed to with the
in [doc/architecture/decisions](./doc/architecture/decisions/) and contributed to with the
[adr-tools](https://github.com/npryce/adr-tools).

## Managing environment variables
Expand Down
39 changes: 39 additions & 0 deletions doc/architecture/decisions/0012-use-open-source-mapping-tools.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# 12. Use open source mapping tools

Date: 2024-09-25

## Status

Accepted

## Context

Representing air pollution and other environment characteristics visually on a
map is an important part of the UX in airTEXT. The [original
airTEXT implementation][] and the [sister project for York][] both used Google Maps.

However, using this commercial product with its fundamental data privacy
uncertainties is not necessary and should be challenged.

## Decision

We believe that excellent mapping and geospatial services can be built using
open source tools. These include:

- [Leaflet][]: a javascript library for presenting maps in HTML
- [OpenStreetMap][]: a public domain map, the base information

We will develop the service using these open source tools in the belief that:

- any usage fees will lower and more transparent
- tracking will be minimised and our users' personal information will be safer

## Consequences

If we are unable to deliver the user benefits which are required, together with
a top-quality user-experience, then we will review this approach.

[original airTEXT implementation]: https://www.airtext.info/
[sister project for York]: https://airqualityalerts.york.gov.uk/maps.php
[Leaflet]: https://leafletjs.com
[OpenStreetMap]: https://www.openstreetmap.org/about
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# 13. Use Hotwire tools for Javascript features

Date: 2024-10-09

## Status

Accepted

## Context

AirTEXT is in part a "single-page application". We are planning to offer users a
single view on which they can switch between 3 days (today, tomorrow and the day
after). On each day's tab they will be able to:

- view predictions and guidance about air quality, pollen, UV and temperature

- interact with a map comprising a number of overlays showing various aspects of
air quality, e.g. layers for nitrogen dioxide, particulate matter and ozone

This functionality will benefit from a front-end implementation where sections
of the page are reloaded as required, e.g. as a user chooses to load forecasts
for a different area or switches to a different day.

## Decision

We're going to use Rails' [Hotwire][] tools and conventions to:

- swap out chunks of pages using [Turbo][] (with HTML fragments prepared on the
server rather than by building views from JSON in the brower with custom JS)

- handle UI sprinkles with [Stimulus][] e.g. for setting classes on UI events

## Consequences

We believe that by adopting the Hotwire conventions we'll produce more
maintainable code, with a greatly reduced amount of Javascript.

Developers will need to become familiar with the Hotwire techniques. We've found
Pragmatic Bookshelf's "[Modern Front-End Development for Rails][]" by Noel
Rappin to be a good complement to the offical online docs.

[Hotwire]: https://hotwired.dev
[Turbo]: https://turbo.hotwired.dev
[Stimulus]: https://stimulus.hotwired.dev
[Modern Front-End Development for Rails]:
https://pragprog.com/titles/nrclient2/modern-front-end-development-for-rails-second-edition
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# 14. Cache CERC API responses using postgres

Date: 2024-10-29

## Status

Accepted

## Context

The airTEXT service we're building obtains forecasts from [CERC's API][]. These forecasts
are available for a particular day, from 50 areas or "zones". The forecasts are updated
occasionally during the day (we're not clear as to the exact schedule of updates).
A single request to the CERC API can be for:

- a particular zone or for all zones

- either 1, 2 or 3 days

Clearly we don't need to make an external API call to CERC every time a user
requests a forecast for a zone.

## Decision

We will implement a simple "cache" using Postgres:

### We will get forecasts for all zones in a single request

A request to the CERC API endpoint `/getforecast/all` without a zone param
returns forecasts for all zones (currently 50).

### We will request 3 days' worth of forecasts

We'll get forecasts for 3 days: today, tomorrow and the day after. This is how
the UI works currently. (However, our caching strategy won't be tied to this
number of forecasts being returned per zone.)

### We will use Postgres for our cache

This is simple in terms of infrastructure. It also will allow us to retain
cached forecasts to build an archive of historical forecasts. Research is
underway to properly understand users' needs for historical data.

### We will cache 3 days' worth of forecasts for each zone

Each entry in our cache will include these properties:

##### `obtained_at`

The timestamp of when the forecast was fetched. We will use this to expire our
cache after `CERC_API_CACHE_LIMIT_MINS` with the `CachedForecast.stale?` test.

##### `zone`

The zone to which it relates

##### `data`

A serialised dump of the 3 forecasts which are received for the zone. Note that
we can handle any quantity of forecasts if for example we added a feature
showing a 7 day view.

Note that we won't cache CERC's JSON response. We will cache our "built" domain
`Forecast` entities, like this:

```
#<Forecast
@obtained_at=2024-10-29 11:59:00 UTC
@date=2024-10-29
@zone=
#<ForecastZone
@id=9
@name=Ealing
@type=1>
@air_pollution=
#<AirPollutionPrediction
@forecasted_at=2024-10-29 10:09:00 UTC
@nitrogen_dioxide=2
@particulate_matter_10=2
@particulate_matter_2_5=2
@ozone=1
@value=2
@label=LOW>
@uv=
#<UvPrediction
@value=2>
@pollen=
#<PollenPrediction
@value=-999>
@temperature=
#<TemperaturePrediction
@min=10.3
@max=16.4>
>
```

### We will use a naive cache expiration strategy

We will expire the cache based purely on age, for example every 60 mins.

We could in theory get smarter as each forecast in an individual cache record
for a zone contains two "version" numbers, one for the air pollution prediction
and another for the other predictions (pollen, temp, uv etc):

```json
"forecasts": [
{
"forecast_date": "2024-10-30",
"non_pollution_version": null,
"pollution_version": 202410300809
...
},
{
"forecast_date": "2024-10-31",
"non_pollution_version": null,
"pollution_version": 202410300349
...
},
{
"forecast_date": "2024-11-01",
"non_pollution_version": null,
"pollution_version": 202410300349,
...
}
],
```

However:

- `non_pollution_version` is not being supplied by CERC

- the logic/mechanism for recording and comparing the 6 different version
numbers would be quite a bit more complex

## Consequences

Going forward we'll need to verify that the time limit for expiring the cache is
appropriate.

In the future we might want to refresh the cache in the background before it
expires to avoid having a couple of seconds of slow performance whilst the cache
is being refreshed.

[CERC's API]: https://www.airtext.info/API/#/default/get-forecast

0 comments on commit da11cae

Please sign in to comment.