Skip to content
This repository has been archived by the owner on Jan 11, 2024. It is now read-only.

FM-360: List snapshots #416

Merged
merged 12 commits into from
Nov 27, 2023
Merged

FM-360: List snapshots #416

merged 12 commits into from
Nov 27, 2023

Conversation

aakoshh
Copy link
Contributor

@aakoshh aakoshh commented Nov 9, 2023

Closes consensus-shipyard/ipc#181

  • Snapshot manifest type which we can store along the parts in a manifest.json
  • Export the manifest
  • List snapshots in a directory
  • Initialise the existing snapshots and add them to STM state
  • Add new snapshot manifests to the STM state
  • List snapshots from the STM data structure in the ABCI
  • Integration test that tests a snapshot being created and added to STM

@aakoshh aakoshh force-pushed the fm-360-list-snapshots branch 2 times, most recently from 67386ec to 495aec3 Compare November 13, 2023 15:14
@aakoshh aakoshh marked this pull request as ready for review November 13, 2023 22:30
// Not all data in the blockstore is traversable, e.g.
// Wasm bytecode is inserted as IPLD_RAW here: https://github.com/filecoin-project/builtin-actors-bundler/blob/bf6847b2276ee8e4e17f8336f2eb5ab2fce1d853/src/lib.rs#L54C71-L54C79
if cid.codec() == DAG_CBOR {
// XXX: Is it okay to panic?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are snapshots only read when starting up the peer? If this is the case I think it is OK to panic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you are looking at the code from @cryptoAtwill that streams the blockstore DAG. The stream either returns a pair of Some CID and content, or None to show that it's finished, but there is no way to signal errors.

There was a problem with it (I can't remember now how it manifested itself exactly - something about EOF) when I tried to export the data that the genesis put in place, because the Wasm files are aren't actual IPLD data structures, so the .unwrap() at line 223 panicked. I think that every encoded by DAG_CBOR was placed by ourselves in this part of the code, so it should be okay to panic, it should parse.

To answer your question: this isn't happening when we start the peer, this is happening at regular intervals when we are exporting the snapshots from the store.

Base automatically changed from fm-366-export-snapshot to main November 27, 2023 09:45
@aakoshh aakoshh merged commit 1d56a2e into main Nov 27, 2023
1 check failed
@aakoshh aakoshh deleted the fm-360-list-snapshots branch November 27, 2023 09:48
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

List completed snapshots in ABCI
2 participants