[pull] main from linera-io:main #9
Open
+251,945
−59,725
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
## Motivation As we replace different existing functionality with blobs, we'll need different blob types. ## Proposal Make `BlobId` into an enum ## Test Plan CI
## Motivation `HashedBlob` isn't really a hashed blob anymore, as the `BlobId` now has more than just the hash. ## Proposal Change `Blob` to be `BlobContent`, and change `HashedBlob` to `Blob`. Hopefully this makes the code a bit less confusing. ## Test Plan CI
* Remove Python HTTP server from LLM example README It's no longer needed, now that the service can fetch it directly from the internet. * Fix the directory used in the commands It doesn't seem like the `llm` directory is entered at any time before that command.
…gStore`. (#2339) * Replace async_lock with sync_lock for the hashable_wrapper. * Replace the async_lock by sync lock for the LRU cache.
* Remove one store and restructure the test code for value-splitting. * Rename the "create_memory_context/store" as "create_test_memory_context/store". * Put the functions "create_test_memory_*" under "with_testing" control.
* `linera-version`: use a committed cache of the API hashes * `scripts/test_publish.sh`: allow publishing to an existing registry * `linera-version`: only check API hashes * `linera-version`: correct command in test failure output * Only enable the `tokio` feature for `fs-err` in `linera-service`
* Introduce the `MEMORY_STORES` for storing the memory stores. * Switch the memory stores from async `Mutex/RwLock` to sync ones for memory. * The `now_or_never` is removed from memory code. * The `MemoryStore` now satisfy the AdminKeyValueStore` and the testing framework for this trait has been improved. * The random namespaces are now used for the `MemoryStore`. * The `MemoryStorage` has been changed accordingly. * For testing cases, the namespace is killed on drop.
## Motivation We renamed this in a previous PR, but some names were still incorrect ## Proposal Use blob content wherever is appropriate ## Test Plan CI
## Motivation Right now we just have the NFT's image "payload" (the bytes of the image) loaded in the example app's state. ## Proposal Let's use blobs instead for that ## Test Plan e2e test was modified to work with blobs
* Simplify linera-sdk testing. * Fix typos, avoid unwrap. * Remove with_rejection.
* `linera-client`: put metrics and filesystem support behind a feature flag * `linera-client`: feature-flag the storage service * `linera-client`: build `persistent::LocalStorage` and conditionally use it on the Web * `linera-client`: add dummy memory-based `Persist` implementation to help linting * `.github/workflows`: test building `linera-client` for the browser instead of `linera-core` * Rename missed `storage_service` features * `linera-client`: add `web-default` feature set for ease of compilation --------- Co-authored-by: Andreas Fackler <[email protected]>
## Motivation
UI is now outdated since we're not using Blobs on the NFT example
## Proposal
Update UI to also use blobs
## Test Plan
<div class='graphite__hidden'>
<div>🎥 Video uploaded on Graphite:</div>
<a href="https://app.graphite.dev/media/video/HlciHFAoHZW62zn13apJ/86647f85-e5e0-4c14-b9a5-4bbd39e16443.mov">
<img src="https://app.graphite.dev/api/v1/graphite/video/thumbnail/HlciHFAoHZW62zn13apJ/86647f85-e5e0-4c14-b9a5-4bbd39e16443.mov">
</a>
</div>
<video src="https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HlciHFAoHZW62zn13apJ/86647f85-e5e0-4c14-b9a5-4bbd39e16443.mov">Screen Recording 2024-08-06 at 12.08.07.mov</video>
* feat: use "serde_json" for "BlobType" string conversion * chore: rebuild CLI.md * fix: use "serde_json" inside original string conversion code * fix: improve FromStr impl * fix: roll back "clap-markdown" to "0.1.3" * fix: roll back "clap" to "4.4.11" * fix: roll back "Cargo.lock" * fix: use "match" to avoid "unwrap" * chore: clippy
## Motivation This is a follow up suggested on #2354 ## Proposal Check if the `Blob` exists before actually minting the NFT ## Test Plan CI
* build: update the "Cargo.lock" * doc: rebuild the "CLI.md"
* feat: add "user_data" arg in "transfer" cli * doc: rebuild the CLI.md * fix: update the "UserData::from_option_string" impl * fix: correct the "UserData::from_option_string" * doc: add more documentation to "user_data" option * fix: handle the error case of parsing UserData from string * Update linera-service/src/linera/main.rs Co-authored-by: Andreas Fackler <[email protected]> Signed-off-by: duguorong009 <[email protected]> --------- Signed-off-by: duguorong009 <[email protected]> Co-authored-by: Andreas Fackler <[email protected]>
…antCollectionView`. (#2366) * Move the KeyValueStoreView to sync. * Replace the async_lock::Mutex by std::sync::Mutex. * Formatting. * Corrections from clippy.
## Motivation We don't want `BlobId` exposed to the SDK. Users will only be aware of `Data` blobs, the other types will be only internally used. ## Proposal Only expose `Data` blobs to the SDK, and assume when dealing with blobs that it's a `Data` blob. Because of that we don't need to use `BlobId` everywhere, and can just use the hash of the `Data` blob in most places. ## Test Plan CI + also retested the NFT UI
…omically. (#2343) * Atomic message bundles * Add a TransactionTracker; have one oracle responses list per txn. * Deduplicate bundle and operation handling. * Extract message execution from execute_block. * Move next_message_index to TransactionTracker. * Add to_posted test helper function. * Fix message_id_for_operation. * Return iterator from make_message_bundles_for. * Address review comments. * Address review comments; don't track messages twice.
* Create a `DeliveryNotifier` helper type Prepare to only allow the chain worker to manipulate it. * Use the `DeliveryNotifier` in `WorkerState` Use the new helper type as a first step to moving its control to the chain worker. * Pass a `DeliveryNotifier` to the chain worker Start moving it into the chain worker, so that the `WorkerState` only manages caching them, instead of actually using them. * Garbage collect delivery notifiers Avoid leaking delivery notifiers as the chain workers in the cache are evicted. * Move delivery notification to chain worker This is the first part to prevent a race between adding a listener to the `DeliveryNotifier` and notifying that messages have been delivered. * Move notifier registration to chain worker Finish preventing a race between adding a listener to the `DeliveryNotifier` and notifying that messages have been delivered. Now both steps are performed in the chain worker, which serializes them. * Protect registration and notification methods Ensure that only the chain worker is able to perform those operations. * Improve documentation links Don't link to `true` and use a slightly shorter link to external type. Co-authored-by: Andreas Fackler <[email protected]> Signed-off-by: Janito Vaqueiro Ferreira Filho <[email protected]> --------- Signed-off-by: Janito Vaqueiro Ferreira Filho <[email protected]> Co-authored-by: Andreas Fackler <[email protected]>
## Motivation CI was failing on PR #2646 with a JSON serialization failure. That was fixed by PR #2668 but we realized none of our tests covered this case. ## Proposal Create a test that tries to save a wallet with pending blobs in it. Verified that before #2668 this test fails, and now it succeeds. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle.
## Motivation #2668 was not necessarily the best way we can fix this issue. ## Proposal Always serialize `BlobId` as string if `is_human_readable`. Fixes #2671 ## Test Plan Reverted #2668, made sure the `test_save_wallet_with_pending_blobs` test failed, and made sure it went back to passing with the new serialization implementation. ## Release Plan - Nothing to do / These changes follow the usual release cycle.
## Motivation This step sometimes fails with API rate limit exceeded, but we don't actually need to run this from scratch everytime. Example failure: https://github.com/linera-io/linera-protocol/runs/31950311713 ## Proposal Cache this step ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle.
## Motivation Currently if a call to `blob_last_used_by` returns an `Err`, we'll print a `ERROR` level log. That is not very accurate because validators could still be not fully in sync and some of them could be missing some blobs. ## Proposal Change log level to `WARN` ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle.
…nic ≥ 0.12 (#2700) * `linera-rpc`: manually configure TLS for the client as required by Tonic ≥ 0.12 * `linera-rpc`: add CA roots from `webpki-roots`
## Motivation While getting #2683 across the finish line, I noticed some speed ups we could do to our Docker github workflow ## Proposal We already build the docker image in `compose.sh`, so no need to do it again before running it. ## Test Plan #2683 made Docker CI last 20+ minutes. This brings it back to around 10 minutes. Docker CI should be sufficient testing here ## Release Plan - Nothing to do / These changes follow the usual release cycle.
* Ensure received certificates' messages are delivered. * Remove unnecessary client mutex locking. * Add a comment: Why are we retrying?
* Scylla under docker-compose doesn't run in developer mode * Test changes to docker/docker-compose.yml. * Added upterm session * Running on gcp-large with updated aio-max-nr * Revert change of runner The custom runner isn't currently working. * Comment out filesystem tweak It doesn't work on GitHub's runners. * Patch `docker-compose.yml` file in CI Run it in developer mode in CI. * Comment out upterm session configuration Should only be needed in our custom runner. * Add TODOs for the steps to do for custom runners Ensure that the quick patches are easily remembered to be removed. --------- Co-authored-by: Christos Hadjiaslanis <[email protected]> Co-authored-by: Christos Hadjiaslanis <[email protected]>
…le (#2711) * Don't read environment variable unless needed Return sooner if the `faucet_url` was provided as an argument. * Avoid allocating a `String` unless needed Only allocate the fallback `String` if the environment variable was not configured. * Replace `.to_string()` with `.to_owned()` Make it explicit that we are moving the string to newly allocated heap memory. * Document the constructor Provide a quick description of how the `RemoteNetTestingConfig` is configured. * Refactor to reduce one nesting level Use only one final `unwrap` instead of nested `unwrap`s. Co-authored-by: Andreas Fackler <[email protected]> Signed-off-by: Janito Vaqueiro Ferreira Filho <[email protected]> * Tweak review suggestion Fix small typos to make it compile. --------- Signed-off-by: Janito Vaqueiro Ferreira Filho <[email protected]> Co-authored-by: Andreas Fackler <[email protected]>
The URL is currently incorrect, and changes often. It makes more sense to require the user to specify it when running the tests.
…2715) ## Motivation When using Docker Compose, we're not currently properly waiting for the service dependencies. ## Proposal Add a health check, and remove the `init_done` file ## Test Plan CI: we can see from the Docker CI logs that both the `proxy` and `shard`s services only get created after `shard-init` exits: ``` Network docker_default Creating Network docker_default Created Volume "docker_grafana-storage" Creating Volume "docker_grafana-storage" Created Volume "docker_linera-scylla-data" Creating Volume "docker_linera-scylla-data" Created Container scylla Creating Container prometheus Creating Container grafana Creating Container prometheus Created Container scylla Created Container grafana Created Container shard-init Creating Container shard-init Created Container proxy Creating Container docker-shard-4 Creating Container docker-shard-2 Creating Container docker-shard-1 Creating Container docker-shard-3 Creating Container docker-shard-1 Created <---- Creations Container proxy Created <---- Container docker-shard-4 Created <---- Container docker-shard-3 Created <---- Container docker-shard-2 Created <---- Container grafana Starting Container prometheus Starting Container scylla Starting Container scylla Started Container grafana Started Container shard-init Starting Container prometheus Started Container shard-init Started Container shard-init Waiting Container shard-init Waiting Container shard-init Exited <---- Exits Container proxy Starting <---- Starts Container shard-init Exited Container docker-shard-4 Starting <---- Container proxy Started Container docker-shard-4 Started Container docker-shard-1 Starting <---- Container docker-shard-1 Started Container docker-shard-3 Starting <---- Container docker-shard-3 Started Container docker-shard-2 Starting <---- Container docker-shard-2 Started Container prometheus Waiting Container grafana Waiting Container scylla Waiting Container proxy Waiting Container docker-shard-4 Waiting Container docker-shard-1 Waiting Container docker-shard-3 Waiting Container docker-shard-2 Waiting Container shard-init Waiting Container docker-shard-4 Healthy Container docker-shard-1 Healthy Container docker-shard-3 Healthy Container docker-shard-2 Healthy Container proxy Healthy Container shard-init Exited Container scylla Healthy Container grafana Healthy Container prometheus Healthy ``` ## Release Plan - Nothing to do / These changes follow the usual release cycle.
* Add a blob size limit. * Add a bytecode size limit. * Add unit tests for limits. * Don't enforce the limit for already published bytecode. * Simplify LimitedWriter; add unit test. * Add decompressed_size_at_most. * Update and copy comment about #2710.
* Add `forget_chain` method to wallet CLI wrapper Allow removing chains from the wallet in tests. * Forget irrelevant newly created chains in test The chains only serve to increase the Faucet chain's length in a way that simulates the Faucet from being executed for a long time, so they can be forgotten after they have been created.
Signed-off-by: Oleg <[email protected]>
* Don't cache blobs that are too large. * Check size first, then decompress.
* updated project template, includes tests * updating test case instantiation_argument * suggested changes
* Shard editing command * reduce version printing * add unit test * fix * fixes after review * remove write_json
## Motivation This README was just showing how to build the Block Explorer code ## Proposal Include also instructions on how to serve the web application and open it in your browser ## Test Plan Went through the steps myself ## Release Plan - Nothing to do / These changes follow the usual release cycle.
* increase logging for subscriptions * Update linera-rpc/src/grpc/client.rs Co-authored-by: Janito Vaqueiro Ferreira Filho <[email protected]> Signed-off-by: Mathieu Baudet <[email protected]>
## Motivation There's a lot of time in our release process that is used building docker images, so we should do something about that to speed up our process. ## Proposal Create a public registry with linera-io/linera-infra#97 And use a CI job to build and push docker images to this public registry ## Test Plan Modified the CI workflow to trigger for this PR, changed it to publish to a test image. Made sure CI was green, and the docker image was visible and pullable from the public registry:  Example successful run with the test code: https://github.com/linera-io/linera-protocol/actions/runs/11583023872/job/32247304661 Test code: 78c30d5 ## Release Plan - Nothing to do / These changes follow the usual release cycle.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )