cargo chainhooks-install failed due to rocksdb build error

[qustavo]

When trying to build chainhooks, I'm getting the following error:

Compiling librocksdb-sys v0.10.0+7.9.2
The following warnings were emitted during compilation:

warning: In file included from rocksdb/table/block_based/data_block_hash_index.cc:5:
warning: rocksdb/table/block_based/data_block_hash_index.h:65:7: error: ‘uint8_t’ does not name a type
warning:    65 | const uint8_t kNoEntry = 255;

[...]

 error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.8.1/fused-src/" "-I" "snappy/" "-I" "/home/gchain/dev/chainhook/target/release/build/lz4-sys-5dba30cef2dbb318/out/include" "-I" "." "-Wall" "-Wextra" "-std=c++17" "-Wsign-compare" "-Wshadow" "-Wno-unused-parameter" "-Wno-unused-variable" "-Woverloaded-virtual" "-Wnon-virtual-dtor" "-Wno-missing-field-initializers" "-Wno-strict-aliasing" "-Wno-invalid-offsetof" "-msse2" "-std=c++17" "-DSNAPPY=1" "-DLZ4=1" "-DNDEBUG=1" "-DOS_LINUX" "-DROCKSDB_PLATFORM_POSIX" "-DROCKSDB_LIB_IO_POSIX" "-DROCKSDB_SUPPORT_THREAD_LOCAL" "-o" "/home/gchain/dev/chainhook/target/release/build/librocksdb-sys-3d8700161d2f94b8/out/rocksdb/table/block_based/data_block_hash_index.o" "-c" "rocksdb/table/block_based/data_block_hash_index.cc" with args "c++" did not execute successfully (status code exit status: 1).

error: failed to compile `chainhook v1.0.0 (/home/me/dev/chainhook/components/chainhook-cli)`, intermediate artifacts can be found at `/home/me/dev/chainhook/target`

I fixed it by bumping rocksdb dependency version:

diff --git a/Cargo.lock b/Cargo.lock
index 9338a06..ad5ef67 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -416,7 +416,7 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
 
 [[package]]
 name = "chainhook"
-version = "0.20.0"
+version = "1.0.0"
 dependencies = [
  "ansi_term",
  "atty",
diff --git a/components/chainhook-cli/Cargo.toml b/components/chainhook-cli/Cargo.toml
index 3c10818..18b38ae 100644
--- a/components/chainhook-cli/Cargo.toml
+++ b/components/chainhook-cli/Cargo.toml
@@ -41,7 +41,7 @@ rocket_okapi = { version = "0.8.0-rc.3", git = "https://github.com/hirosystems/o
 rocket = { version = "=0.5.0-rc.3", features = ["json"] }
 
 [dependencies.rocksdb]
-version = "0.20.1"
+version = "0.21.0"
 default-features = false
 features = ["lz4", "snappy"]

I'd create a PR to bump the version but I've just seen that rocksdb has been downgraded from 0.21.0 to 0.20.1 (0e88b53).

Any chance to revert that commit?

[lgalabru]

hey @qustavo!
thanks for opening this PR. The last time I tried updating rocksdb, I did experience a light regression in terms of performance.
i was testing a few different things simultaneously so it could have been something else.
this is not something that I can personally investigate, if you can, any help would be welcome!

[qustavo]

To give a little bit more context, my issue has been previously described here due to gcc 13.x issue. The suggested fix is described here.
I managed to switch rocksdb versions setting a ROCKSDB_LIB_DIR and using precompiled .so.

I'd like to know how did you benchmarked and detected the performance regressions, do you have some tests I can run and measure?

[SrushtiHaryan]

after cargo chainhook-install, when i run the chainhook command, the output remains empty, can anyone tell what could have gone wrong while setting up the repo locally?

[MicaiahReid]

@SrushtiHaryan were there any errors when you ran cargo chainhook-install? And to clarify, when you now run chainhook, there is no result at all? No error, nothing?

[SrushtiHaryan]

@SrushtiHaryan were there any errors when you ran cargo chainhook-install? And to clarify, when you now run chainhook, there is no result at all? No error, nothing?

yes there were errors,

 process didn't exit successfully: `C:\Srushti\Chainhook\chainhook\target\release\build\librocksdb-sys-b75e8e5ac7d7572f\build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at C:\Users\manav\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bindgen-0.64.0\./lib.rs:2393:31:
  Unable to find libclang: "couldn't find any valid shared libraries matching: ['clang.dll', 'libclang.dll'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `chainhook v1.1.1 (C:\Srushti\Chainhook\chainhook\components\chainhook-cli)`, intermediate artifacts can be found at `C:\Srushti\Chainhook\chainhook\target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

i installed clang and followed this issue for changing the dependency version:
i tried to change the dependency versions of:

1. Chainhook from v1.1.1 to v1.0.0 in Cargo.lock
2. rocksdb from 0.20.1 to v0.21.0 in components/chainhook-cli/Cargo.toml

after which the packages got installed without an error (but a few warnings) on cargo chainhook-install

Finished release [optimized] target(s) in 11m 45s
warning: the following packages contain code that will be rejected by a future version of Rust: rstest v0.11.0
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
  Replacing C:\Users\manav\.cargo\bin\chainhook.exe
   Replaced package `chainhook v1.1.1 (C:\Srushti\Chainhook\chainhook\components\chainhook-cli)` with `chainhook v1.0.0 

and just to clarify on the output, yes, it doesnt give errors too.

[radicleart]

Hey - I just tried cargo chainhook-install on Ubuntu 23.10 and end up with error ;

error: failed to run custom build command for `librocksdb-sys v0.10.0+7.9.2`

Caused by:
  process didn't exit successfully: `/mnt/bitcoin-mainnet/chainhook/target/release/build/librocksdb-sys-a85928e817e4688b/build-script-build` (exit status: 1)

followed by

  error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.8.1/fused-src/" "-I" "snappy/" "-I" "/mnt/bitcoin-mainnet/chainhook/target/release/build/lz4-sys-2c47c50287c793be/out/include" "-I" "." "-Wall" "-Wextra" "-std=c++17" "-Wsign-compare" "-Wshadow" "-Wno-unused-parameter" "-Wno-unused-variable" "-Woverloaded-virtual" "-Wnon-virtual-dtor" "-Wno-missing-field-initializers" "-Wno-strict-aliasing" "-Wno-invalid-offsetof" "-msse2" "-std=c++17" "-DSNAPPY=1" "-DLZ4=1" "-DNDEBUG=1" "-DOS_LINUX" "-DROCKSDB_PLATFORM_POSIX" "-DROCKSDB_LIB_IO_POSIX" "-DROCKSDB_SUPPORT_THREAD_LOCAL" "-o" "/mnt/bitcoin-mainnet/chainhook/target/release/build/librocksdb-sys-2e1f7850601fc859/out/725dd32d7edce1c8-string_util.o" "-c" "rocksdb/util/string_util.cc" with args "c++" did not execute successfully (status code exit status: 1)

I'm running ordhook on same server and its indexing and building rocksdb.

I have clang, libssl-dev versions

chainhook$ sudo apt install make clang pkg-config libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
make is already the newest version (4.3-4.1build1).
clang is already the newest version (1:16.0-57).
pkg-config is already the newest version (1.8.1-2).
libssl-dev is already the newest version (3.0.10-1ubuntu2.3).
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

[MicaiahReid]

Thanks for the info @radicleart. That's interesting that building ordhook is working. I will investigate!

[ECBSJ]

I'm also getting similar issues when trying to install via cargo on Windows:

error: failed to run custom build command for `librocksdb-sys v0.10.0+7.9.2`

Caused by:
  process didn't exit successfully: `C:\Users\xxx\Documents\stuff\chainhook\target\release\build\librocksdb-sys-cf32fdf9d0c4e618\build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at C:\Users\xxx\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bindgen-0.64.0\./lib.rs:2393:31:
  Unable to find libclang: "couldn't find any valid shared libraries matching: ['clang.dll', 'libclang.dll'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `chainhook v1.6.0 (C:\Users\xxx\Documents\stuff\chainhook\components\chainhook-cli)`, intermediate artifacts can be found at `C:\Users\xxx\Documents\stuff\chainhook\target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

Do I need to change the librocksdb-sys dependency version?

[MicaiahReid]

We now attach builds for Linux/mac/windows on each release: https://github.com/hirosystems/chainhook/releases/tag/v1.6.2

You can also install Chainhook via homebrew, snapcraft, and winget (install instructions coming soon).

With all of these options that should prevent users from having to install any dependencies, I'll go ahead and close this issue.

If you're trying to build Chainhook from source on Linux, there may be some additional dependencies you have to install, and the above comments are a good resource for what may need to be done.