All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Added
--include-libpostal
to includelibpostal
output alongside another geocoder's output.
- If we find a
\0
byte in cached data, try looking it up again. Also, report an error if the Smarty API ever returns a\0
byte.
- Roll back
serde_derive
to version 1.0.171 and pin it there, to avoid using the newserde_derive
that uses a pre-built binary compiler plugin. The binary plugin has not been successfully reproduced from source when I last looked, and it represents an unacceptable security risk, since it can't be audited without disassembling the binary code. We'll need a longer-term plan for this.
- In server mode, "prime" libpostal so that it doesn't take forever to process the first request.
- Fix build by ignoring test that requires libpostal data.
geocode-csv server
will now start a basic, experimental REST server.- The
--cache-hits-only
flag can be used to bypass geocoding for a given backend where we're only interested normalizing addresses we already know about.
- Use a
Justfile
to document and automate common maintainer tasks.
- Fixed an issue where libpostal would occasionally parse cities as suburbs. Thank you, Tom, for the PR!
- Skip records with empty streets after normalization.
- Fix broken release build caused by incorrect use of
cfg!
macro.
- Added lots of debugging assertions to help detect potential memory overflows. None of these were the actual cause of the recent bugs, but the ones that we left in were nice to have.
- Fixed hang/memory overflow where we forwarded
&[Address]
bocks with no addresses to the cache layer. - Updated to latest
cross
and installedprotoc
inside the Docker container.
- Update all dependencies to see if we can get a clean build with no security advisories. None of the advisories looked like they should affect normal usage, but it's better to fix them.
- Do not attempt to geocode empty addresses. This has recently started causing SmartyStreets to return an error.
- The build process now requires at least
cmake 3.12
or so, which breaks Ubuntu 18.04 unless you use a PPA. - We now require
protobuf-compiler
to build.
- Update dependencies to fix minor security advisories.
- Updated a bunch of dependencies, including BigTable client used for caching.
- Retries now use exponential backoff. This means that we should now give up after about 30 seconds, instead of 10 or 12 like before.
--max-retries
can now be used to control how many times we retry before giving up.
--max-addresses-per-second
allows specifying a rate limit for any external geocoder. This is particularly useful with services like Smarty that impose their own rate limits.
- Report metric
geocodecsv.selected_errors.count
, which breaks down particularly interesting errors bycomponent
andcause
. We focus on reporting this information for remote APIs, currently including BigTable and Smarty. This is meant to supplement the existinggeocodecsv.chunks_retried.total
andgeocodecsv.chunks_failed.total
metrics, which already report generic error statistics, but which can't say what failed or why.
- Renabled Mac M1 builds after fixing cross-compilation of
libpostal
.
- Disabled Mac M1 builds until we get cross-compilation working with
libpostal
. This will allow us to test other platforms.
- Renamed
--cache-record-keys
to--cache-output-keys
.
- Cleaned up packaging, build and CI.
Internal-only releases identified as either 1.0.2 or 2.0.0-alpha.1. No official binaries were ever built, and this exact release does not exist in git
history. Included for completeness.
- Support for "geocoding" using
libpostal
. This returns parsed and normalized address fields, but no lat/lon data. To use this, pass--geocoder=libpostal
. - Optionally normalize addresses using
libpostal
when using other geocoders. - We can now cache data using either Redis or BigTable.
- You can use
--cache-record-keys
to output the cache keys.
- You can use
- We now calculate extensive geocoding metrics, and can either print them to standard output, or send them to NewRelic.
SMARTY_AUTH_ID
is now preferred overSMARTYSTREETS_AUTH_ID
(though both will be supported).SMARTY_AUTH_TOKEN
is now preferred overSMARTYSTREETS_AUTH_TOKEN
(though both will be supported).--smarty-license
is now preferred over--license
(though both will be supported).
- New naming convention for release ZIP files.
- New binary builds for ARM/M1 Macs.
- The downloadable
*.zip
files now include both the CPU and the OS in the name. Downloading scripts will need to be adjusted. - We now use
rustls
instead of OpenSSL internally. This shouldn't change anything, but it's a significant change.
- Restored missing binaries for existing platforms by switching to GitHub CI.
Bumping number to v1.0.0 because this has been running fine in production for quite.
- Support
--match enhanced
. This only works for appropriate SmartyStreets plans.
- Updated dependencies to fix several CVEs reported by
cargo deny
. I do not believe that any of these CVEs actually affectedgeocode-csv
in practice, but better safe than sorry.
- Updated to latest
tokio
andhyper
libraries. This represents a major change to some of our core libraries, but it means that we're finally on stabletokio
and not a pre-release. - Replaced error-reporting code, so some error output might look different.
This build has released binaries, but the version number wasn't updated, at it still claims to be v0.3.0-beta.3. Since the code is identical, this isn't the end of the world.
- Fixed Travis CI build.
- Rebuild with modern Rust toolchain.
- Added a
--license
option to enable use of rooftop geocoding.
- Fixed a number of security advisories in supporting libraries. None of these appear to have been easily exploitable using an invalid CSV file as input.