RocksDB vs TuplDB counters

This test compares the performance of TuplDB to RocksDB on an Optane 900P drive, when issuing random counter increments. With 16 threads running, an 8-byte random key is selected from a range of 5 billion. If no corresponding value exists, a value of 0 is inserted. Otherwise, the current value is incremented by 1. Values are encoded as 8-byte integers.

Although RocksDB supports a merge operator for counters, it wasn't used. By issuing full read-modify-write operations, the test is better able to simulate a workload in which records are read as often as they are written.

RocksDB was configured with the same balanced set as used by the RocksDB vs TuplDB Optane test. In addition, the "cache index and filter blocks" Bloom filter option was enabled, which is recommended for large data sets.

The results show how quickly counters can be incremented compared to the total number of operations performed overall. The horizontal axis essentially shows time, but not in a linear fashion.

With TuplDB, the performance dips and speeds up as the operations shift from mostly inserts to mostly updates. TuplDB inserts are slower than updates, because inserts perform incremental rebalancing operations to improve storage efficiency.

With TuplDB on the Optane drive (block device, direct I/O, redo to EXT4 partition), the test completed in 46.5 hours and performed 35.6 billion operations. For reference, the same test was run on a Samsung 960 Pro SSD, and it took 120 hours (5.0 days) to complete. RocksDB on the Optane drive took 360 hours (15.0 days) to complete the test. Overall, RocksDB performance was about 7.7 times slower than TuplDB, when using the Optane drive.

In the TuplDB/Optane test, there's an odd performance dip and recovery in the 15 to 18 billion range. I don't know what could have caused this. On subsequent test runs, the initial performance is a bit lower, and then it picks up again at around the 11 billion mark. It's possible that the Optane drive performs some sort of background maintenance activity.

Another thing to note with RocksDB is that it doesn't really take advantage of the Optane drive. The next graph compares the earlier RocksDB/Optane data to the 960 Pro, although the test didn't run quite as long (power outage).

Interestingly, the test on the 960 Pro performed less erratically at first, but overall the outcome looks the same.