Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2nd Community Review of Marshall Fil+ Allocator #216

Open
Marshall-btc opened this issue Nov 5, 2024 · 4 comments
Open

2nd Community Review of Marshall Fil+ Allocator #216

Marshall-btc opened this issue Nov 5, 2024 · 4 comments
Assignees
Labels
Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards Refresh Applications received from existing Allocators for a refresh of DataCap allowance

Comments

@Marshall-btc
Copy link

https://compliance.allocator.tech/report/f03012911/1730766444/report.md

  1. SPs have a good success rate for Spark retrieval, with high ones reaching 95% or more.
    image
    image

  2. However, there is not enough variety of customers, 1 enterprise dataset and 2 public datasets. I plan to follow up by encouraging more enterprise datasets to apply.

I will be planning to attend the Bangkok November Web3 meetup, my goal is to find 1-2 enterprise customers with real stored data and get them to try to enter the Filecoin space, looking forward to our efforts and progress.

I would like to get more guidance on Filecoin Allocator's work, thank you very much.

@Kevin-FF-USA Kevin-FF-USA self-assigned this Nov 7, 2024
@Kevin-FF-USA Kevin-FF-USA added Refresh Applications received from existing Allocators for a refresh of DataCap allowance Awaiting Governance/Watchdog Comment DataCap Refresh requests awaiting a public verification of the metrics outlined in Allocator App. labels Nov 7, 2024
@filecoin-watchdog
Copy link
Collaborator

@Marshall-btc
Allocator Application
Compliance Report
1st Review
1st Review score: 5PiB

5 PiB granted to existing clients:

Client Name DC status
USANASA 2PiB Existing
LiZhu Pharmaceutical Group Co. 2PiB Existing
NationalAstronomical Observatories, Chinese Academy of Sciences 1PiB Existing

USANASA

  • The client introduced 13 new SPs in this round(20 total), while only 6 were updated in the GitHub issue. Allocator did ask for the reason for doing so.
  • 4SPs have bad retrieval(~0%), while the rest have mixed; oscillating between 30-95%.

LiZhu Pharmaceutical Group Co

  • The client introduced 15 new SPs in this round(23 total) and updated some in the GitHub issue. Allocator did ask for the reason for doing so.
    The following SPs were not updated in the GitHub issue:
    f03178077
    f03179572
    f03178144
    f03214937
    f03229933
    f03173127
  • 3SPs has bad retrieval(~0%), while the rest have mixed; oscillating between 30-98%.

NationalAstronomical Observatories, Chinese Academy of Sciences

  • The client introduced 13 new SPs in this round(20 total) and updated 7 in the GitHub issue.
  • The Client declared 6 replicas, while 8 of them already exist.
  • 3SPs has bad retrieval(~0%), 3SPs uses DDO and the retrieval is unavailable, the rest have mixed; oscillating between 30-98%.

Overall, the allocator has implemented a new process of reviewing a random sample of SPs by asking their clients to prove the geographic location of selected SPs.
For all applications, the allocation schedule was compliant with the allocator application. The exception is the allocation at National Astronomical Observatories, where the allocator ran out of DC.

@filecoin-watchdog filecoin-watchdog added Awaiting Response from Allocator If there is a question that was raised in the issue that requires comment before moving forward. and removed Awaiting Governance/Watchdog Comment DataCap Refresh requests awaiting a public verification of the metrics outlined in Allocator App. labels Nov 19, 2024
@Marshall-btc
Copy link
Author

It's really amazing! Thank you @filecoin-watchdog for your objective and fair review.
Overall, our allocator operations have been positively acknowledged and praised by you. Based on your review, I believe there are mainly three areas that need improvement, which will also be the focus of my future review work.

1. Some of the newly introduced SPs were not updated on GitHub in a timely manner.
Solution: Although most of the newly added SPs were queried in github. However, since inadvertent oversight is inevitable, we will review each Datacap before sending it out to avoid omissions. In the meantime, we will ask Client to rectify the situation by e-mail, and to disclose the information in advance before starting each new package.

2.A small number of SPs had a low success rate of Spark retrieval;
Solution: Although some SPs have explained that they used the DDO order mode, to prevent SPs from cheating, we will limit and stop cooperation with SPs using DDO deals. Despite the fact that the Spark retrieval rate for well-performing SPs has exceeded 90%, for SPs with a Spark retrieval rate of 30%-50%, our team will assist in connecting and communicating with experienced successful SPs for guidance. If they still cannot improve the Spark retrieval rate, we will restrict and stop further approval of Datacap.

3. "The Client declared 6 replicas, while 8 of them already exist."
Solution: @filecoin-watchdog Shockingly diligent work! This is indeed an area that I overlooked in my previous review, and I will focus on it in the future. In the meantime, I will ask the other party to perform further explanation.

Additionally, regarding your mention of "For all applications, the allocation schedule was compliant with the allocator application. The exception is the allocation at National Astronomical Observatories, where the allocator ran out of DC."
Explanation: For all Datacap allocations, we are following the rules. However, for these two SPs f01518369 and f01889668, the Spark retrieval rate was only 0% without using the DDO order mode, so I asked the client for further explanation, and the client's explanation was that the data center malfunctioned, resulting in a broken network. Out of compliance caution, I limited the original 2 PiB Datacap approval to a 1 PiB Datacap approval.
image
image

Finally, as I mentioned earlier, our team traveled to Thailand for the Web 3 event in November. We met with a number of enterprise customers who have data storage needs and we plan to recommend them to apply for our Allocator application in the next 1-2 months, as well as introduce them to a number of reputable SPs.
Based on our past good performance in Allocator operation, we hope to get 10 - 15 PiB Datacap support. Thanks again to the official team for their hard work!

@filecoin-watchdog filecoin-watchdog added Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards and removed Awaiting Response from Allocator If there is a question that was raised in the issue that requires comment before moving forward. labels Nov 21, 2024
@Marshall-btc
Copy link
Author

image
Marshall-btc/Marshall-Fil-Data-Pathway#5 (comment)

The previously mentioned issue with the number of copies was reasonably explained by the customer and I assessed it as understandable. I will also focus on the backup issue in the follow-up.

@Marshall-btc
Copy link
Author

As I said, I have sent an email to the client about the important things about Data cap. Reminding them of the precautions again.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards Refresh Applications received from existing Allocators for a refresh of DataCap allowance
Projects
None yet
Development

No branches or pull requests

3 participants