NPM packages for artifacts

[cedoor]

This issue is part of the first version of a mechanism to handle circuit artifacts of PSE projects with ZK-Kit. Below is the idea:

• This repository acts as an aggregator of artifacts to keep track of files and manage NPM packages.
• There will be either one folder per project (e.g. /semaphore), or one branch per project so that one NPM artifacts package per project can be managed. Devs who want to add their artifacts could selectively clone the repo with their project folder/branch.
• Artifacts NPM packages will have the project name + -artifacts at the end, and the ZK-Kit NPM organization will be used. For example: @zk-kit/semaphore-artifacts.
• The version of the artifacts packages will follow that of the project packages. It is plausible that for current projects, a major will be bumped every time the circuits are updated and a trusted-setup will be run.
• Packages on NPM will be available on free CDNs such as https://unpkg.com. It will therefore be possible to use a URL of one of those CDNs to download the artefacts.
• On the utils package of ZK-Kit, a function will be available that will allow anyone to download the artifacts they want. By default, the latest versions will be downloaded, but there will be options to download specific versions.

The repository will include a README file and possibly scripts/instructions to allow people to easily add their artifacts.

[cedoor]

@sripwoud missing something?

[sripwoud]

I think this was the gist of it.
As you mentioned making this a kind of standard approach to manage and distribute the artifacts. I ll prepare a paper draft summarizing this in a next PR. Then we can get feedback on it before rolling this out.

[cedoor]

I think this was the gist of it. As you mentioned making this a kind of standard approach to manage and distribute the artifacts. I ll prepare a paper draft summarizing this in a next PR. Then we can get feedback on it before rolling this out.

Sounds great!

[sripwoud]

#7 would close this.

I must say I am still not 100% happy with what we have here though.
Especially I don't like that we are separating the src files (the circom circuits) from the built artifacts (zkey and wasm).
We aren't able to merge confidently here because zkey and wasm are just binary files. We will only know it works after the ci tests in the zk-kit repo fetches it and wait for it to fail or not...

Ideally you want your ci to generate whatever you "build" files are from your src files don't you?

Do you think it would be possible to do this?

• version control only circuits src files (in this case .circom files in zk-kit repo)
  Means this snark-artifacts repo becomes unecessary
• on each change in the circom files, have the ci
  • compile the circuit
  • perform a dummy phase1
  • perform a phase2 for each circuit
  • publish the artifacts as npm packages

Will probably take a long time, cause it is computation heavy.
But the benefits would be:

• confidence our artifacts match the src circuit files
• published artifacts as npm packages by the ci without having to version control the artifacts themselves (no repo bloat)
• artifacts available on unpkg

[cedoor]

@sripwoud sounds like a cleaner solution! We'd need to isolate the artifacts in a folder with a package.json, probably moved there during the ci.

It could require some time, but if you feel you can do it it's fine for me!

EDIT: we'll actually need real artifacts mostly. Generating them with a dummy trusted-setup works for testing, which will just be something we need in the first phase. Having a separate repo sounds still better then maybe 🤔

[sripwoud]

Maybe we can combine both.

• build dummy artifacts in ci on the fly in zk-kit repo so that we can test everything.
• have prod artifacts here. (I am not sure yet how to automate the generation of these prod artifacts though, as it involves big phase .ptau files)

[cedoor]

Maybe we can combine both.

* build dummy artifacts in ci on the fly in zk-kit repo so that we can test everything.

* have prod artifacts here. (I am not sure yet how to automate the generation of these prod artifacts though, as it involves big phase .ptau files)

After internal discussions with @sripwoud, we created some issues: https://github.com/privacy-scaling-explorations/snark-artifacts/issues.

TLDR:

• We need a clear versioning mechanism for the artifact packages: Define a clear versioning mechanism #14
• We should explore similar solutions: Explore similar projects and compare solutions #13
• We need to define a list of projects that may be interested in using this mechanism: Make a list of projects that may be interested in this project #12
• Artifacts need to be checked to be sure of their integrity: Check integrity of snark artifacts #11
• P0tion may be interested in automating the upload of the ceremony output files using this mechanism: Automate P0tion trusted-setup files upload #16
• ZK-Kit could add a new CI workflow to create dummy artifacts for testing: ci: create artifacts with a dummy trusted setup for test purposes on the fly in a GH action zk-kit#250
• Idea: Github action to trigger P0tion ceremonies based on changes in circuits

Agree to define an MVP and then work on low-priority issues later.

New board here: https://github.com/orgs/privacy-scaling-explorations/projects/45/views/1