Skip to content

Commit

Permalink
Merge pull request #53 from Police-Data-Accessibility-Project/mc-upda…
Browse files Browse the repository at this point in the history
…te-readme

Update README.md
  • Loading branch information
josh-chamberlain authored Jun 17, 2024
2 parents b564e9f + e12dbbf commit d768576
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ python3 mirror.py
Hourly

## How it works
We're taking advantage of a Python wrapper around the Airtable API to provide flat data joinable on an agency's `airtable_uid`. The script is scheduled to run via crontab on a droplet in DigitalOcean. On each run, the latest code from the main branch of the repo is fetched and current requirements are installed in order to make sure the script is up to date on the droplet. During execution, only rows from each table that are new or have been updated in the last two hours are fetched to update our Postgres database.
We're taking advantage of a Python wrapper around the Airtable API to provide flat data joinable on an agency's `airtable_uid`. The script is scheduled to run via crontab on the Automation Manager droplet in DigitalOcean, on an hourly basis. On each run, the latest code from the main branch of the repo is fetched and current requirements are installed in order to make sure the script is up to date on the droplet. During execution, only rows from each table that are new or have been updated in the last two hours are fetched to update our Postgres database.

The intent is to do as little transformation as we can, erring on the side of being true to what the API wrapper returns, while also being transparent about how we're doing things.

Expand Down

0 comments on commit d768576

Please sign in to comment.