This repository has been archived by the owner on Oct 25, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 67
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge remote-tracking branch 'origin/develop' into maciej/blockstore-v2
- Loading branch information
Showing
58 changed files
with
250 additions
and
299 deletions.
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,66 +26,38 @@ | |
| </a> | ||
| </p> | ||
|
|
||
| The Fuel indexer is a standalone service that can be used to index various components of the blockchain. These indexable components include blocks, transactions, receipts, and state within the Fuel network, allowing for high-performance read-only access to the blockchain for advanced dApp use-cases. | ||
|
|
||
| <font size="4">Want to get started right away? Check out our [Quickstart](https://fuellabs.github.io/fuel-indexer/master/getting-started/quickstart.html)!</font> | ||
|
|
||
| - [For Users](#for-users) | ||
| - [Dependencies](#dependencies) | ||
| - [`forc-index` Plugin](#forc-index-plugin) | ||
| - [WebAssembly (WASM) modules](#webassembly-wasm-modules) | ||
| - [For Contributors](#for-contributors) | ||
| - [Dev Dependencies](#dev-dependencies) | ||
| - [Building from Source](#building-from-source) | ||
| - [Testing](#testing) | ||
| - [Contributing](#contributing) | ||
| - [Read the book](#read-the-book) | ||
|
|
||
| # For Users | ||
|
|
||
| Users of the Fuel indexer project include dApp developers looking to write flexible data-based backends for their dApp frontends, as well as index operators who are interested in managing one or many indexer projects for dApp developers. | ||
| ### [➡️ Read the Quickstart! ➡️](https://docs.fuel.network/docs/indexer/getting-started/quickstart/) | ||
|
|
||
| ## Dependencies | ||
| ## What is the Fuel indexer? | ||
|
|
||
| ### `fuelup` | ||
|
|
||
| - We use fuelup in order to get the binaries produced by services in the Fuel ecosystem. Fuelup will install binaries related to the Fuel node, the Fuel indexer, the Fuel orchestrator (forc), and other components. | ||
| - fuelup can be downloaded [here](https://github.com/FuelLabs/fuelup). | ||
| The Fuel indexer is a standalone service that can be used to index various components of the blockchain. These indexable components include blocks, transactions, receipts, and state within the Fuel network, allowing for high-performance read-only access to the blockchain for advanced dApp use-cases. | ||
|
|
||
| ### `WebAssembly` | ||
| > TLDR: It's Infrastructure as a service (IaaS) for Web3 dApp backends. | ||
| Two additonal cargo components will be required to build your indexers: `wasm-snip` and the `wasm32-unknown-unknown` target. | ||
| ## Install | ||
|
|
||
| - To install `wasm-snip`: | ||
| Fuel's indexer supports Linux (x64 & arm64) and macOS (x64 & Apple Silicon). | ||
|
|
||
| ```bash | ||
| cargo install wasm-snip | ||
| ``` | ||
| > If you don't want to deal with dependency issues we recommend just using Fuel's indexer with Docker, [via the included docker-compose file](https://github.com/FuelLabs/fuel-indexer/blob/develop/scripts/docker-compose.yaml). | ||
| To install the `wasm32-unknown-unknown` target via `rustup`: | ||
| Install Fuel's toolchain manager - fuelup. | ||
|
|
||
| ```bash | ||
| rustup target add wasm32-unknown-unknown | ||
| curl --proto '=https' --tlsv1.2 -sSf https://install.fuel.network/fuelup-init.sh | sh | ||
| ``` | ||
|
|
||
| > IMPORTANT: Users on Apple Silicon macOS systems may experience trouble when trying to build WASM modules due to its `clang` binary not supporting WASM targets. If encountered, you can install a binary with better support from Homebrew (`brew install llvm`) and instruct `rustc` to leverage it by setting the following environment variables: | ||
| > | ||
| > - `AR=/opt/homebrew/opt/llvm/bin/llvm-ar` | ||
| > - `CC=/opt/homebrew/opt/llvm/bin/clang` | ||
| > The `fuel-indexer` and `forc-index` binaries should now be available in your `$PATH` | ||
| ## `forc-index` Plugin | ||
| ## Usage | ||
|
|
||
| The primary way of developing Fuel indexers for end users is via the `forc-index` plugin. The `forc-index` plugin, is a CLI tool that is bundled with Fuel's primary CLI tooling interface, [`forc`](https://github.com/FuelLabs/sway/tree/master/forc) ("Fuel Orchestrator"). | ||
| For development, users will primarily use the `forc index` command line utility made available after installing fuelup. | ||
|
|
||
| As mentioned in the [dependencies](#dependencies) section, the `forc-index` plugin is made available once you download [`fuelup`](#fuelup). | ||
|
|
||
| If you've successfully gone through the [Quickstart](#quickstart), you should already have `forc-index` installed and available in your `PATH`. | ||
|
|
||
| ```text | ||
| ```bash | ||
| forc index --help | ||
| ``` | ||
|
|
||
| ``` | ||
| ```text | ||
| forc index 0.0.0 | ||
| Fuel Indexer Orchestrator | ||
| USAGE: | ||
|
|
@@ -108,111 +80,11 @@ SUBCOMMANDS: | |
| remove Stop and remove a running indexer | ||
| start Standalone binary for the Fuel indexer service | ||
| status Check the status of a registered indexer | ||
| ``` | ||
|
|
||
| ## WebAssembly (WASM) modules | ||
|
|
||
| Within the context of the Fuel indexer, WebAssembly (WASM) modules are binaries that are compiled to a `wasm32-unknown-unknown` target, which can then be deployed to a running indexer service, and run as isolated runtime environments. | ||
|
|
||
| There are a few points that Fuel indexer users should know when using WASM: | ||
|
|
||
| 1. WASM modules are only used if the execution mode specified in your manifest file is `wasm`. | ||
|
|
||
| 2. Developers should be aware of what things may not work off-the-shelf in a module: file I/O, thread spawning, and anything that depends on system libraries. This is due to the technological limitations of WASM as a whole; more information can be found [here](https://rustwasm.github.io/docs/book/reference/which-crates-work-with-wasm.html). | ||
|
|
||
| 3. As of this writing, there is a small bug in newly built Fuel indexer WASM modules that produces a WASM runtime error due to an errant upstream dependency. For now, a quick workaround requires the use of `wasm-snip` to remove the errant symbols from the WASM module. More info can be found in the related script [here](https://github.com/FuelLabs/fuel-indexer/blob/develop/scripts/stripper.bash). | ||
|
|
||
| > IMPORTANT: Users on Apple Silicon macOS systems may experience trouble when trying to build WASM modules due to its `clang` binary not supporting WASM targets. If encountered, you can install a binary with better support from Homebrew (`brew install llvm`) and instruct `rustc` to leverage it by setting the following environment variables: | ||
| > | ||
| > - `export AR=/opt/homebrew/opt/llvm/bin/llvm-ar` | ||
| > - `export CC=/opt/homebrew/opt/llvm/bin/clang` | ||
| # For Contributors | ||
|
|
||
| Contributors of the Fuel indexer project are devs looking to help backends for their dApps. | ||
|
|
||
| ## Dev Dependencies | ||
|
|
||
| ### `docker` | ||
|
|
||
| > IMPORTANT: Docker is not required to run the Fuel indexer. | ||
| - We use Docker to produce reproducible environments for users that may be concerned with installing components with large sets of dependencies (e.g. PostgreSQL). | ||
| - Docker can be downloaded [here](https://docs.docker.com/engine/install/). | ||
|
|
||
| ### Database | ||
|
|
||
| At this time, the Fuel indexer requires the use of a database. We currently support a single database option: PostgreSQL. PostgreSQL is a database solution with a complex feature set and requires a database server. | ||
|
|
||
| #### PostgreSQL | ||
|
|
||
| > Note: The following explanation is for demonstration purposes only. A production setup should use secure users, permissions, and passwords. | ||
| On macOS systems, you can install PostgreSQL through Homebrew. If it isn't present on your system, you can install it according to the [instructions](https://brew.sh/). Once installed, you can add PostgreSQL to your system by running `brew install postgresql`. You can then start the service through `brew services start postgresql`. You'll need to create a database for your indexed data, which you can do by running `createdb [DATABASE_NAME]`. You may also need to create the `postgres` role; you can do so by running `createuser -s postgres`. | ||
|
|
||
| For Linux-based systems, the installation process is similar. First, you should install PostgreSQL according to your distribution's instructions. Once installed, there should be a new `postgres` user account; you can switch to that account by running `sudo -i -u postgres`. After you have switched accounts, you may need to create a `postgres` database role by running `createuser --interactive`. You will be asked a few questions; the name of the role should be `postgres` and you should elect for the new role to be a superuser. Finally, you can create a database by running `createdb [DATABASE_NAME]`. | ||
|
|
||
| In either case, your PostgreSQL database should now be accessible at `postgres://postgres@localhost:5432/[DATABASE_NAME]`. | ||
|
|
||
| ### SQLx | ||
|
|
||
| - After setting up your database, you should install `sqlx-cli` in order to run migrations for your indexer service. | ||
| - You can do so by running `cargo install sqlx-cli --features postgres`. | ||
| - Once installed, you can run the migrations by running the following command after changing `DATABASE_URL` to match your setup. | ||
|
|
||
| ## Building from Source | ||
|
|
||
| ### Clone repository | ||
|
|
||
| ```bash | ||
| git clone [email protected]:FuelLabs/fuel-indexer.git && cd fuel-indexer/ | ||
| ``` | ||
|
|
||
| ### Run migrations | ||
|
|
||
| #### PostgreSQL migrations | ||
|
|
||
| ```sh | ||
| cd packages/fuel-indexer-database/postgres | ||
| DATABASE_URL=postgres://postgres@localhost sqlx migrate run | ||
| ``` | ||
|
|
||
| ### Start the service | ||
|
|
||
| ```bash | ||
| cargo run --bin fuel-indexer | ||
| ``` | ||
|
|
||
| > If no configuration file or other options are passed, the service will default to a `postgres://postgres@localhost` database connection. | ||
| ## Testing | ||
|
|
||
| Fuel indexer tests are currently broken out by a database feature flag. In order to run tests with a PostgreSQL backend, use `--features postgres`. | ||
|
|
||
| ### Default tests | ||
|
|
||
| ```bash | ||
| cargo test --locked --workspace --all-targets | ||
| ``` | ||
|
|
||
| ### End-to-end tests | ||
|
|
||
| ```bash | ||
| cargo test --locked --workspace --all-targets --features postgres | ||
| ``` | ||
|
|
||
| ### `trybuild` tests | ||
|
|
||
| For tests related to the meta-programming used in the Fuel indexer, we use `trybuild`. | ||
|
|
||
| ```bash | ||
| RUSTFLAGS='-D warnings' cargo test -p fuel-indexer-tests --features trybuild --locked | ||
| ``` | ||
| #### [➡️ For more details on how to build Fuel indexers, read the docs! ➡️](https://docs.fuel.network/docs/indexer/) | ||
|
|
||
| ## Contributing | ||
|
|
||
| If you're interested in contributing PRs to make the Fuel indexer a better project, feel free to read [our contributors document](./CONTRIBUTING.md). | ||
|
|
||
| # Read the book | ||
|
|
||
| Whether you're a user or a contributor, for more detailed info on how the Fuel indexer service works, make sure you [**read the book**](https://fuellabs.github.io/fuel-indexer/master/). | ||
| If you're interested in contributing PRs to make the Fuel indexer a better project, feel free to read [our contributors document](./CONTRIBUTING.md). | ||
Oops, something went wrong.