v1.4.0
Release notes
We're pleased to introduce version 1.4 of immudb, which comes with two major features: FIPS-Compliant Builds and Synchronous Replication. And, of course, we’ve fixed bugs and made other improvements.
FIPS-Compliant Builds
Starting with v1.4.0, immudb can be compiled using the go-boringcrypto fork of the Go compiler, which uses the FIPS 140-2 compliant boringssl library. We now also officially provide FIPS-compliant binaries and Docker images.
What is it? The FIPS 140-2 standard prescribes the design and security requirements for cryptographic modules that may be approved for use by the United States government. FIPS-140 is a collection of computer security standards set by the National Institute of Standards and Technology (NIST) for the United States government. FIPS 140–2 defines the critical security parameters vendors must use for encryption implementations sold to the U.S government.
A detailed information about FIPS-compliant immudb build can be found in immudb's source code repository.
Synchronous Replication
Synchronous replication improves the reliability of data replicated across the immudb nodes in a cluster. When an immudb cluster uses synchronous replication, the primary node waits for sufficient transaction confirmations from replicas before it considers the transaction fully committed and durably stored by multiple nodes. This feature increases data durability by allowing any single node in an immudb cluster—including the primary node—to be lost without causing the database state tracked by the nodes and clients to diverge.
Prior to version 1.4, immudb only supports asynchronous replication. When a cluster is configured to use asynchronous replication, replica nodes can lag behind the primary node, and any given transaction committed on the primary node isn’t guaranteed to be replicated to other nodes within a reasonable time period. Thus, transactions committed to a primary node that have not been replicated can be lost if the primary node is irrecoverably lost. When this occurs, the state of the surviving nodes may diverge from the state computed by the immudb client, because the new primary node elected from the remaining replicas could be missing the most recent commits tracked by the clients.
To get more information about synchronous replication please refer to the documentation on docs.immudb.io.
Replication performance
Due to additional synchronization between nodes, commit throughput with synchronous replication will naturally be slower compared to a single-node cluster. The performance difference will depend on various factors such as the speed of disks, network latency, and the number of followers.
In this release we also worked on improvements to the replication mechanism itself, achieving 17 to 20 times larger TX/s replication throughput than what was available in 1.3.2 release.
SDK Updates
This release also comes with significant updates to immudb SDKs. We've brought brand new .Net SDK; added inline documentation to Go SDK; added inline documentation and increased functionality of Python SDK and Java SDK; and made significant updates to the node.js SDK 1.
Changelog
[v1.4.0] - 2022-10-12
Bug Fixes
- build: Do not publish official non-dev images on RC tags
- pkg/client: replace keepAlive context from the original one to the background, avoiding parent expiration
- build: Use correct binary download links
- embedded/store: edge-case calculation of precommitted tx
- embedded/watchers: Fix invariant breakage in watchers
- embedded/watchers: Fix invariant breakage in watchers
- pkg/database: any follower can do progress due to its prefech buffer
- pkg/replication: Do not crash on invalid tx metadata
- pkg/replication: handle replication already closed case
- pkg/replication: discard precommitted txs and continue from latest committed one
- pkg/replication: solve issues when follower diverged from master
- wmbedded/watchers: Correctly fix the original implementation
- Makefile: add fips build flag to test/fips
- Makefile: remove interactive flag from dist/fips command
- ci: fix regex pattern for fips binaries
- cmd/immuadmin: set correct data-type for replication-sync-followers flag
- embedded/store: fix size calculation of precommitted txs
- embedded/store: Fix checking for closed store when syncing TXs
- embedded/store: avoid attempts to commit in wrong order
- embedded/store: expose durable precommitted state
- embedded/store: include allowPrecommitted into tx reader construction
- embedded/store: ensure tx is released upon error
- embedded/store: aht up to precommited tx
- github: Update github actions after migration of Dockerfile's
- pkg/database: return master commit state if failing to read follower precommitted one
- pkg/database: Fix mutex lock in ExportTx
- pkg/database: set follower states holder when changing replication status
- pkg/server: add logs when replicator does not start
Changes
- Rename sync-followers to sync-acks
- cmd/immuclient: include precommit state when quering status
- pkg/server: Better error message when validating replication options
- embedded/watchers: Simplify and document cancellation path
- embedded/watchers: Simplify mutex locking code
- embedded/watchers: single-point for init and cleanup
- pkg/database: wait for tx when a non-existent or non-ready transaction is requested
- pkg/database: add TODO comment on replication passive waiting
- pkg/database: simplify follower's wait
- pkg/replication: Add TX gap metrics
- pkg/replication: Add basic replication metrics
- pkg/replication: improve replication logging
- add dependabot config
- Add empty line between license header and package
- Dockerfile.fips: add fips build changes
- cmd/immuadmin: add new replication flags
- cmd/immuadmin: revert default replication-master-port
- cmd/immuadmin: use default immudb port as default value for replication-master-port flag
- cmd/immuclient: flag replication-sync-enabled to enable sync replication
- cmd/immudb: deprecate replication-enabled towards replication-is-replica
- docker: Move main Dockerfile's to build folder
- docker: Simplify the main Dockerfile
- embedded/store: resolve pre-committed using clogbuf
- embedded/store: wip reduce allocations in exportTx
- embedded/store: mutexless export-tx
- embedded/store: enhanced tx discarding logic
- embedded/store: wip load precommitted txs
- embedded/store: method to dynamically switch to external allowance
- embedded/store: wip wait for precommitted txs
- embedded/store: explicit allowPrecommitted and restricted access to precommitted txs
- embedded/store: minor code simplification
- embedded/store: possibility to read tx header of precommitted txs
- embedded/store: support for concurrent replicated precommits
- embedded/store: tx parsing with sanity checks
- embedded/store: handle commit case when there is nothing new to commit
- embedded/store: tolerate partial data or inconsistencies when loading pre-committed txs
- embedded/store: explanatory comments added
- embedded/store: waits for durable precommitted txs
- embedded/store: minor renaming and comment additions
- embedded/store: add integrity checks when reading precommitted txs
- pkg/api: currentState endpoint includes precommitted info
- pkg/api: explicit sync replication setting
- pkg/api/schema: reformat schema.proto file
- pkg/database: minor typo in comment
- pkg/database: sync exportTx
- pkg/database: improve error comparison
- pkg/database: follower commit progress without additional waits
- pkg/database: handle special case related to sql initialization
- pkg/database: disable automatic sql init on older databases
- pkg/integration: add synchronous replication integration tests
- pkg/replication: allowPreCommitted only with sync replication enabled
- pkg/replication: speed up follower reconnection
- pkg/replication: use session-based authentication
- pkg/replication: handling a particular case in an optimized manner
- pkg/replication: backward compatible replication
- pkg/replication: check committedTxID from master
- pkg/replication: wip optimize concurrency in replicators
- pkg/replication: handle case when follower precommit state is up-to-date but commit state is lies behind
- pkg/replication: sync replication using follower state
- pkg/replication: configurable prefetchTxBufferSize and replicationCommitConcurrency
- pkg/replication: improve error comparison
- pkg/replication: graceful closing
- pkg/replication: further progress in sync replication
- pkg/replication: replicator with backward compatibility mode
- pkg/replicator: wip precommitted tx discarding when follower diverged from master
- pkg/server: explicit sync replication
- pkg/server: handle admin user creation with sync replication enabled
- pkg/server: support for systemdb with session-based auth
- pkg/server: display all replication settings
- pkg/server: include sync replication settings in options
- pkg/server: use replication settings
Features
- cmd/immuadmin: flag to set the number of sync followers
- cmd/immudb: flag to set the number of sync followers for systemdb and defaultdb
- embedded/store: functionality to discard precommitted txs
- embedded/store: core support for sync replication
- pkg/api: api extensions to support sync replication
- pkg/database: wip sync replication logic
- pkg/replication: mode to allow tx discarding on followers
- pkg/replication: wip replicator with support for sync replication
- pkg/server: sync replication logic
- pkg/server: Add ability to inject custom database management object
Downloads
Docker image
https://hub.docker.com/r/codenotary/immudb
immudb Binaries
| File | SHA256 |
|---|---|
| immudb-v1.4.0-darwin-amd64 | 0d0fd6700ac3f6686bd941f3c4a942ae8420ed3db926631165b844119ed6507f |
| immudb-v1.4.0-darwin-arm64 | d144db7dd9a230f81c69ce5a6efeda8d47ea5adce3e29275e4a2ebc3929c1f09 |
| immudb-v1.4.0-freebsd-amd64 | 49968893102c1e92caf668c403f43d47a268101c83ccde33bf0703606146b557 |
| immudb-v1.4.0-linux-amd64 | 6bfdbdd5a50bbd2d1d1ad576293a2b6d028091e2e56ba2ae078f1b6182a2320e |
| immudb-v1.4.0-linux-amd64-fips | f15b43896db970a1204e28e5ce7425993f822cd0d19bf9bde46d7486a0c72c77 |
| immudb-v1.4.0-linux-amd64-static | 883c735d94cd634dd4df026ac986688007ab8aa980ba49ebd75b3689019d8f72 |
| immudb-v1.4.0-linux-arm64 | 2991fcae7d6dcbe8ce95c9fdc675b7cdeb635c52e8887a84ef03b5ad8c180115 |
| immudb-v1.4.0-linux-s390x | 1fcafe7e3b14b9419ad1b6690bdae69a142d82e4c7fed135e98968377fab08e3 |
| immudb-v1.4.0-windows-amd64.exe | 9b65cf58d12a1f572cf0df60f27f8bd96c2e2aae2124187da24edab545517528 |
immuclient Binaries
| File | SHA256 |
|---|---|
| immuclient-v1.4.0-darwin-amd64 | f9d70ea80a9319aa85524eff93116f016ff429ec1806e316822156bc279c13eb |
| immuclient-v1.4.0-darwin-arm64 | f0bdbcef165c0faec34ee2affd820592d5a391c06c4058f5902d9a3c323896e4 |
| immuclient-v1.4.0-freebsd-amd64 | a9e10cf38226b7ada81e75528b066418ca0f0ccad80efce1a380d132b6f41021 |
| immuclient-v1.4.0-linux-amd64 | ed2b9e8dda6b7b46610a3cb1499cba6206ee291ee11ea283451a4b8d6a5e23b6 |
| immuclient-v1.4.0-linux-amd64-fips | 2f6995ae635ef4716c2d803352435723c4bb6a62fd0166a4d647e6660fc54a58 |
| immuclient-v1.4.0-linux-amd64-static | aef8a3dc6f40b0df4b2af5c95aa6759f38719dc38fe9fb3f29f7aa669f33a4cc |
| immuclient-v1.4.0-linux-arm64 | f908ecf7aca6fa68f6242220141d4198d4ab499ef5cadf3cd029e42e0b49ea2e |
| immuclient-v1.4.0-linux-s390x | e810e11543e5464e0378343cbbdfbe7eaab3b02ceb79ba1938705eca0b7b0ddd |
| immuclient-v1.4.0-windows-amd64.exe | 204563c884a99fc33d073e3cf9182833177d0966048145d11bef793e619dc376 |
immuadmin Binaries
| File | SHA256 |
|---|---|
| immuadmin-v1.4.0-darwin-amd64 | 644b1ef42cafed6a7d63aedc9496c6bc31be5abee06c7b44df40f4fe4a4d1cd1 |
| immuadmin-v1.4.0-darwin-arm64 | 55ec64fa781f5aa7b1c1731cee3c2ad6a11059a9a11aabe9eec72c19f09084eb |
| immuadmin-v1.4.0-freebsd-amd64 | d4f6222e500cfdf304b19be6671bfdedb93889c1fc951acce12c755370a39ca6 |
| immuadmin-v1.4.0-linux-amd64 | 8433ca56e8d631410cd54e6d832f509d1f27376d7d6c9685c4f57791f6d15959 |
| immuadmin-v1.4.0-linux-amd64-fips | e2522ffbf3b83dcbc134e35af06b2ba73baf2db4196c00566d2bd328ba1ef70c |
| immuadmin-v1.4.0-linux-amd64-static | c9a5323c6ad1a0ad2e246b936388143ee901283634547969b525cec7c7b861e9 |
| immuadmin-v1.4.0-linux-arm64 | 1a85623bf4fb675393ed1de1d474f9ed96cc6bc9869602e2de4d735de38853ce |
| immuadmin-v1.4.0-linux-s390x | 69c05e5598fbcbad447e447770f8870a4da0a7493f1f8f4236ac134ba7192a02 |
| immuadmin-v1.4.0-windows-amd64.exe | f192278e46b8cb99a813ff651cbed16ecf03f9cbc4e8a3af8e5af1db4b86af01 |
-
Some SDKs updates will be released a few days after the immudb release. ↩