Spark RSR docs for FilOz review #6
Draft
+863
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Added initial spark retrieval success rate definiiton. This from a export from https://www.notion.so/protocollabs/Spark-Request-Based-Non-Committee-Global-Retrieval-Success-Rate-4c5e8c47c45f467f80392d00cac2aae4#1a5fc6a081904c88b6d9ac7752e652a8 I haven't fixed the anchor links yet. * Fixed the relative links. This was done with a script that was added as well. * Removed mangled markdown tables with a a request for a todo to give clean HTML * Added a versions and support/questions/feedback section * start porting result code table * add result status codes * fix rendering * fill 3rd column * more result code texts * finish table * empty lines in tables break formatting * ` -> <code /> * more formatting fixes * Finish Per Request (non-committee) Score vs. Committee Scoring * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> * Small table formating adjustments * Removed bolded headings * Apply suggestions from code review Co-authored-by: Julian Gruber <[email protected]> --------- Co-authored-by: Julian Gruber <[email protected]> Co-authored-by: patrickwoodhead <[email protected]>
* Added initial readme. This includes: * Project background * Service class definition and listing * Example service classes started for "warm" and "cold" * SLO definition * SLI definition and listing (referencing docs created in other PRs) * Project tenets * project conventions * Start of an FAQ * Typo / small cleanups * badge cleanup * more badge cleanup
* Added an initial sector health rate SLI definition * Moved to use propertly formula formatting * Typo fixes * More fixiups
BigLep
commented
Nov 10, 2024
BigLep
commented
Nov 10, 2024
BigLep
commented
Nov 10, 2024
| - Created by X on a daily basis as part of Y | ||
| - Consumed by anyone wanting to analyze or display SP retrievability performance. | ||
|
|
||
| ## Callouts/Concerns with this SLI |
There was a problem hiding this comment.
2024-11-10 sync conversation notes:
- Currently we are checking weighted on data stored. This means small-storing SPs won't get checked.
- @Stebalien had the idea of blending a weighting of how much data an SP has and the time since have last checked an SP.
- Idea of making it a two round thing
- Idea of having some known failures cases where you expect nodes to fail as a way to catch "dishonest actors"
There was a problem hiding this comment.
Let's discuss the ideas in space-meridian/roadmap#190
BigLep
commented
Nov 10, 2024
There was a problem hiding this comment.
2024-11-10 conversation between FilOz and Space Meridian in Bangkok:
- PayloadCID
- Label field should have the CID of the CAR that is stored.
- Spark wants to treat "payload" as the data that a user wants to store.
- Note: need to make it really clear on what an SP should do. Document the conventions.
- Really want to either
- Make deals with CARv2
- Deal should have the index of the data stored
- IPNI needs to do some sort of sticky cache to keep results for at least 20 minutes
- Is it needed in IPNI to get pieceCids to CIDs
- Spark needs to do more investigation on how to ensure spark checks are indiscerable from real requests. Right now a smart SP can see what are the Spark checks and then just be on their best behavior for the expected spark checks.
- Masih's video on CommPact: https://www.youtube.com/watch?v=ndLKjXQ1awI&list=PL_0VrY55uV19y0b03a-CXNuvlgJR3G0EG&index=15
- Steven idea of doing IPLD checks and pieceCid/range checks
- 🎬 Schedule recurring Spark RSR working group
How could a SP do that? |
@juliangruber : my understanding is that an SP can look at the Round Retrieval Task List, see which rows they are the listed "minerId" for, and then just make sure to serve the corresponding payloadCids in that 20 minute period. |
BigLep
added a commit
that referenced
this pull request
Nov 18, 2024
Incorporated some feedback from #6 These are mostly spellcheck / formatting updates. The only substantive change is the "callout/concern with this SLI" about how "checker traffic is pretty easy to differentiate from real traffic for a Storage Provider"
* Minor update to Spark RSR v1 docs Incorporated some feedback from #6 These are mostly spellcheck / formatting updates. The only substantive change is the "callout/concern with this SLI" about how "checker traffic is pretty easy to differentiate from real traffic for a Storage Provider" * Update service-level-indicators/spark-retrieval-success-rate.md Co-authored-by: Julian Gruber <[email protected]> --------- Co-authored-by: Julian Gruber <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Putting Spark RSR docs in a PR so easier to collect questions/suggestions from FilOz during their 2024-11-10 review before Space Meridian meeting.
FilOz: please add suggestions, ask contextual comments, etc.