Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CCIP gethwrappers with zkstack support (integrated with main flow) #15487

Open
wants to merge 50 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
f2c6a91
adding zk support script and makefile action
yashnevatia Dec 2, 2024
4d31dea
Adding zkstack go-gen files and wrapper entry point
yashnevatia Dec 2, 2024
0bf13d1
Adding initial framework
yashnevatia Dec 2, 2024
26eb91e
Merge 0bf13d10407d8d50da03e39d2895697e7ebc4127 into 86d5f9993ef208ef0…
yashnevatia Dec 2, 2024
058da9c
Update gethwrappers
app-token-issuer-infra-releng[bot] Dec 2, 2024
44912a2
adding more scaffolding
yashnevatia Dec 2, 2024
c58d1cf
Merge branch 'develop' of github.com:smartcontractkit/chainlink into …
yashnevatia Dec 2, 2024
78a9d08
Merge branch 'zksync-wrappers' of github.com:smartcontractkit/chainli…
yashnevatia Dec 2, 2024
8d21f2e
adding branch for zks addition
yashnevatia Dec 2, 2024
74d46ad
Merge 8d21f2e5859684d31c9833ffddb8b26e4872f033 into 86d5f9993ef208ef0…
yashnevatia Dec 2, 2024
79539d1
Update gethwrappers
app-token-issuer-infra-releng[bot] Dec 2, 2024
10128e9
adding zkbin insertion logic
yashnevatia Dec 3, 2024
a3c61ab
renaming py file, adding tx types in generated_zks
yashnevatia Dec 3, 2024
03a4a7a
adding everything
yashnevatia Dec 3, 2024
4b73bd3
integrating into main flow
yashnevatia Dec 3, 2024
4697679
adding to main flow
yashnevatia Dec 3, 2024
28bb116
changing selector name
yashnevatia Dec 3, 2024
fa1e1b2
optionally accept zkbinpath
yashnevatia Dec 4, 2024
118ec38
architecture
yashnevatia Dec 4, 2024
1cd6ec8
adding zksolc setup
yashnevatia Dec 4, 2024
b0957f2
setup zksolc
yashnevatia Dec 4, 2024
cb4d6e4
Merge b0957f2c7d3621c23f71b38d4c21399a097cb220 into 29e5b3b560a8576a4…
yashnevatia Dec 4, 2024
311ee9a
Update gethwrappers
app-token-issuer-infra-releng[bot] Dec 4, 2024
f46568d
restricting to ccip for debug
yashnevatia Dec 4, 2024
9f16729
Merge branch 'zksync-wrappers-integrated' of github.com:smartcontract…
yashnevatia Dec 4, 2024
ceae356
edit
yashnevatia Dec 4, 2024
a4c6d33
zksolc version
yashnevatia Dec 4, 2024
dce6eac
trying with wrappers
yashnevatia Dec 4, 2024
5359428
cleaning up
yashnevatia Dec 4, 2024
11bfbca
updated go mod
simsonraj Dec 4, 2024
86eaefe
parameterizing zksolc version
yashnevatia Dec 4, 2024
a0775fc
removing whitespace
yashnevatia Dec 4, 2024
66eb8ec
linting
yashnevatia Dec 4, 2024
c2927bb
updating mod files
yashnevatia Dec 4, 2024
1413c61
merging
yashnevatia Dec 4, 2024
e75485d
tidying
yashnevatia Dec 4, 2024
d8ff804
forgot this one
yashnevatia Dec 4, 2024
20c7c2e
adding wrappers manually
yashnevatia Dec 4, 2024
c7788f7
amd
yashnevatia Dec 4, 2024
1079efe
cleanup
yashnevatia Dec 4, 2024
0a2425d
removing metadata flag
yashnevatia Dec 5, 2024
cc40fa4
Merge 0a2425d0d370608acd29b9fd03baa23e9670cecc into 5ef07c836b9deb38c…
yashnevatia Dec 5, 2024
6873664
Update gethwrappers
app-token-issuer-infra-releng[bot] Dec 5, 2024
4d1c29d
building deployments
yashnevatia Dec 6, 2024
aa868d5
merging
yashnevatia Dec 6, 2024
6cc2be5
Merge aa868d56bdd2c47d04aa3572717a1cdaa23add29 into 699e172108fcda7ee…
yashnevatia Dec 6, 2024
fe7c5be
Update gethwrappers
app-token-issuer-infra-releng[bot] Dec 6, 2024
1188149
adding custom waitMined
yashnevatia Dec 6, 2024
9c4d285
Merge branch 'zksync-wrappers-integrated' of github.com:smartcontract…
yashnevatia Dec 6, 2024
ff6bdff
building integration tests
yashnevatia Dec 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/solidity-wrappers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ jobs:
contents: read
runs-on: ubuntu22.04-8cores-32GB
steps:
- name: Check architecture
yashnevatia marked this conversation as resolved.
Show resolved Hide resolved
run: uname -m

- name: Checkout the repo
uses: actions/[email protected]

Expand All @@ -49,11 +52,11 @@ jobs:
uses: ./.github/actions/setup-nodejs
with:
prod: "true"

- name: Run native compile and generate wrappers
run: make wrappers-all
working-directory: ./contracts

- name: Assume role capable of dispatching action
uses: smartcontractkit/.github/actions/setup-github-token@ef78fa97bf3c77de6563db1175422703e9e6674f # [email protected]
id: get-gh-token
Expand Down
1 change: 1 addition & 0 deletions contracts/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ artifacts
cache
node_modules
solc
zksolc
abi
coverage
coverage.json
Expand Down
8 changes: 6 additions & 2 deletions contracts/GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ pnpmdep: ## Install solidity contract dependencies through pnpm
abigen: ## Build & install abigen.
../tools/bin/build_abigen

.PHONY: setup_zksolc
setup_zksolc: ## Download and install zksolc
../tools/bin/setup_zksolc.sh

.PHONY: mockery
mockery: $(mockery) ## Install mockery.
go install github.com/vektra/mockery/[email protected]
Expand Down Expand Up @@ -76,15 +80,15 @@ ccip-lcov:
# make call example
# make FOUNDRY_PROFILE=llo-feeds wrappers
.PHONY: wrappers
wrappers: pnpmdep mockery abigen ## Recompiles solidity contracts and their go wrappers.
wrappers: pnpmdep mockery abigen setup_zksolc ## Recompiles solidity contracts and their go wrappers.
./scripts/native_solc_compile_all_$(FOUNDRY_PROFILE)
go generate ../core/gethwrappers/$(FOUNDRY_PROFILE)

# This call generates all gethwrappers for all products. It does so based on the
# assumption that native_solc_compile_all contains sub-calls to each product, and
# go_generate does the same.
.PHONY: wrappers-all
wrappers-all: pnpmdep mockery abigen ## Recompiles solidity contracts and their go wrappers.
wrappers-all: pnpmdep mockery abigen setup_zksolc ## Recompiles solidity contracts and their go wrappers.
# go_generate contains a call to compile all contracts before generating wrappers
go generate ../core/gethwrappers/go_generate.go

Expand Down
12 changes: 10 additions & 2 deletions contracts/scripts/native_solc_compile_all_ccip
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ echo " └───────────────────────
SOLC_VERSION="0.8.24"
OPTIMIZE_RUNS=26000
OPTIMIZE_RUNS_OFFRAMP=500
ZKSOLC_VERSION="1.5.6"
simsonraj marked this conversation as resolved.
Show resolved Hide resolved


SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"
Expand Down Expand Up @@ -38,6 +39,13 @@ compileContract () {
--bin-runtime --hashes --metadata --metadata-literal --combined-json abi,hashes,metadata,srcmap,srcmap-runtime \
--evm-version paris \
"$ROOT"/contracts/src/v0.8/"$1"

zksolc --overwrite -O3 --metadata-hash none \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the overwrite same as the "mode"?
& any other option to provide "fallback_to_optimizing_for_size"?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--fallback-Oz                 Try to recompile with -Oz if the bytecode is too large

i think unless you dont specify this flag, it should be false.
--overwrite is for overwriting a file if the wrapper already exists

-o "$ROOT"/contracts/zksolc/v$ZKSOLC_VERSION/"$contract" \
--bin --allow-paths "$ROOT"/contracts/src/v0.8 \
--metadata --metadata-literal \
--evm-version paris \
"$ROOT"/contracts/src/v0.8/"$1"
}


Expand Down Expand Up @@ -78,11 +86,11 @@ compileContract ccip/test/helpers/receivers/MaybeRevertMessageReceiver.sol
compileContract ccip/test/helpers/MultiOCR3Helper.sol
compileContract ccip/test/mocks/MockE2EUSDCTokenMessenger.sol
compileContract ccip/test/mocks/MockE2EUSDCTransmitter.sol
compileContract ccip/test/WETH9.sol
# compileContract ccip/test/WETH9.sol (error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can completely exclude these test contracts? no?

compileContract ccip/test/helpers/CCIPReaderTester.sol

# Encoding Utils
compileContract ccip/interfaces/encodingutils/ICCIPEncodingUtils.sol
# compileContract ccip/interfaces/encodingutils/ICCIPEncodingUtils.sol (error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, what is the error here? something we can fix? or is this file not being used by any contract?

Copy link
Author

@yashnevatia yashnevatia Dec 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It compiles

Compiler run successful. Artifact(s) can be found in directory "/Users/yashvardhan/chainlink/contracts/zksolc/v1.5.3/ICCIPEncodingUtils".

but does not generate the ICCIPEncodingUtils binary
image


# Customer contracts
compileContract ccip/pools/USDC/USDCTokenPool.sol
Expand Down
199 changes: 198 additions & 1 deletion core/gethwrappers/abigen.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
// AbigenArgs is the arguments to the abigen executable. E.g., Bin is the -bin
// arg.
type AbigenArgs struct {
Bin, ABI, Out, Type, Pkg string
Bin, ABI, Out, Type, Pkg, ZkBinPath string
}

// Abigen calls Abigen with the given arguments
Expand Down Expand Up @@ -72,6 +72,9 @@
}

ImproveAbigenOutput(a.Out, a.ABI)
if a.ZkBinPath != "" {
ImproveAbigenOutputZks(a.Out, a.ZkBinPath)
}
}

func ImproveAbigenOutput(path string, abiPath string) {
Expand Down Expand Up @@ -466,3 +469,197 @@
func addHeader(code []byte) []byte {
return utils.ConcatBytes([]byte(headerComment), code)
}

// ZK stack logic
func ImproveAbigenOutputZks(path string, zkBinPath string) {

Check failure on line 474 in core/gethwrappers/abigen.go

View workflow job for this annotation

GitHub Actions / lint

unnecessary leading newline (whitespace)

bs, err := os.ReadFile(path)
if err != nil {
Exit("Error while improving abigen output", err)
}

fset, fileNode := parseFile(bs)

contractName := getContractName(fileNode)

zkByteCode, err := os.ReadFile(zkBinPath)
if err != nil {
Exit("Error while improving abigen output", err)
}
zkHexString := string(zkByteCode)

// add zksync binary to the wrapper
fileNode = addZKSyncBin(fileNode, contractName, zkHexString)

// add zksync logic to the deploy function
fileNode = updateDeployMethod(contractName, fset, fileNode)

bs = generateCode(fset, fileNode)

err = os.WriteFile(path, bs, 0600)
if err != nil {
Exit("Error while writing improved abigen source", err)
}
}

// add zksync binary to the wrapper
func addZKSyncBin(fileNode *ast.File, contractName string, zkHexString string) *ast.File {
// zksync
newVarSpec := &ast.ValueSpec{
Names: []*ast.Ident{ast.NewIdent(fmt.Sprintf("%sZKBin", contractName))},
Type: ast.NewIdent("string"),
Values: []ast.Expr{
&ast.BasicLit{
Kind: token.STRING,
Value: fmt.Sprintf("(\"0x%s\")", zkHexString),
},
},
}
newVarDecl := &ast.GenDecl{
Tok: token.VAR,
Specs: []ast.Spec{newVarSpec},
}

// Insert the new variable declaration at the top of the file (before existing functions)
fileNode.Decls = append(fileNode.Decls, newVarDecl)
return fileNode
}

// add zksync logic to the deploy function
func updateDeployMethod(contractName string, fset *token.FileSet, fileNode *ast.File) *ast.File {

Check failure on line 529 in core/gethwrappers/abigen.go

View workflow job for this annotation

GitHub Actions / lint

unnecessary leading newline (whitespace)

return astutil.Apply(fileNode, func(cursor *astutil.Cursor) bool {
x, is := cursor.Node().(*ast.FuncDecl)
if !is {
return true
} else if x.Name.Name != "Deploy"+contractName {
return false
}

// only add this import if Deploy method found
astutil.AddImport(fset, fileNode, "github.com/smartcontractkit/chainlink/v2/core/gethwrappers/generated")

// Extract the parameters from the existing function x
paramList := getConstructorParams(x.Type.Params.List)
// get the `if zksync()` block
zkSyncBlock := getZKSyncBlock(contractName, paramList)
// insert the `if zksync()` block
addZKSyncBlock(*x, zkSyncBlock)
// update the return type in the function signature
updateTxReturnType(*x)
// update the actual return value
updateReturnStmt(*x)

return false
}, nil).(*ast.File)
}

// get the `if zksync()` block
func getZKSyncBlock(contractName, paramList string) string {
zkSyncBlock := `if generated.IsZKSync(backend) {
address, ethTx, contractBind, _ := generated.DeployContract(auth, parsed, common.FromHex(%sZKBin), backend, %params)
contractReturn := &%s{address: address, abi: *parsed, %sCaller: %sCaller{contract: contractBind}, %sTransactor: %sTransactor{contract: contractBind},%sFilterer: %sFilterer{contract: contractBind}}
return address, ethTx, contractReturn, err
}`
zkSyncBlock = strings.ReplaceAll(zkSyncBlock, "%s", contractName)
zkSyncBlock = strings.ReplaceAll(zkSyncBlock, "%params", paramList)
return strings.ReplaceAll(zkSyncBlock, "%s", contractName)
}

// Extract the parameters for constructor function
func getConstructorParams(contstructorParams []*ast.Field) string {
params := []string{}
for i, param := range contstructorParams {
if i > 1 { // Skip auth and backend
for _, name := range param.Names {
params = append(params, name.Name)
}
}
}
paramList := strings.Join(params, ", ")
return paramList
}

// insert the `if zksync()` block
func addZKSyncBlock(x ast.FuncDecl, zkSyncBlock string) ast.FuncDecl {
for i, stmt := range x.Body.List {

ifStmt, ok := stmt.(*ast.IfStmt)
if !ok {
continue
}
binaryExpr, ok := ifStmt.Cond.(*ast.BinaryExpr)
if !ok {
continue
}
if ident, ok := binaryExpr.X.(*ast.Ident); ok && ident.Name == "parsed" {
// Creating new statement to insert
newStmt := &ast.ExprStmt{
X: &ast.BasicLit{
Kind: token.STRING,
Value: zkSyncBlock,
},
}

// Insert the new statement after the current statement
x.Body.List = append(x.Body.List[:i+1], append([]ast.Stmt{newStmt}, x.Body.List[i+1:]...)...)
break
}
}
return x
}

// convert *types.Transaction to *generated_zks.Transaction
func updateTxReturnType(x ast.FuncDecl) {
x.Type.Results.List[1].Type = &ast.StarExpr{
X: &ast.SelectorExpr{
X: &ast.Ident{Name: "generated"},
Sel: &ast.Ident{Name: "Transaction"},
},
}
}

// convert tx to &Transaction{Transaction: tx, Hash_zks: tx.Hash()}
func updateReturnStmt(x ast.FuncDecl) {
for _, stmt := range x.Body.List {
returnStmt, is := stmt.(*ast.ReturnStmt)
if !is {
continue
}
if len(returnStmt.Results) < 3 {
continue
}

txExpr, ok := returnStmt.Results[1].(*ast.Ident)
if !ok {
return
}
if txExpr.Name != "tx" {
return
}

txField := &ast.KeyValueExpr{
Key: ast.NewIdent("Transaction"),
Value: ast.NewIdent("tx"),
}

hashField := &ast.KeyValueExpr{
Key: ast.NewIdent("Hash_zks"),
Value: &ast.CallExpr{
Fun: &ast.SelectorExpr{
X: ast.NewIdent("tx"),
Sel: ast.NewIdent("Hash"),
},
},
}
newRet := &ast.CompositeLit{
Type: &ast.SelectorExpr{
X: ast.NewIdent("generated"),
Sel: ast.NewIdent("Transaction"),
},
Elts: []ast.Expr{txField, hashField},
}
pointerRet := &ast.UnaryExpr{Op: token.AND, X: newRet}
returnStmt.Results[1] = pointerRet
}
}

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

15 changes: 9 additions & 6 deletions core/gethwrappers/ccip/generated/ccip_home/ccip_home.go

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

15 changes: 9 additions & 6 deletions core/gethwrappers/ccip/generated/fee_quoter/fee_quoter.go

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Loading
Loading