Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Constantine bindings for EIP196 #184

Merged
merged 33 commits into from
Aug 13, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
51496e8
constantine bindings
NickSneo Jun 27, 2024
079cfd3
C bindings and jni used
NickSneo Jul 12, 2024
535504d
C bindings added and build.gradle changed
NickSneo Jul 12, 2024
4d1c4e2
Merge branch 'main' into main
NickSneo Jul 12, 2024
c4c8eff
fixed errors and tests, modified build.gradle
NickSneo Jul 17, 2024
c84d5b5
added test cases from Gnark lib
NickSneo Jul 17, 2024
d6423b9
added nim install in CI
NickSneo Jul 18, 2024
4292e57
choose nim to install
NickSneo Jul 18, 2024
5c1570b
updated install nim command
NickSneo Jul 18, 2024
388eb98
Merge branch 'main' into main
NickSneo Jul 18, 2024
9ecea5e
fixed CI failing due to jfrog
NickSneo Jul 18, 2024
5cea62b
fix test task in CI
NickSneo Jul 19, 2024
6b8952f
fixed typo, corrected nimble
NickSneo Jul 19, 2024
f4877c2
Update constantine/build.gradle
NickSneo Jul 19, 2024
d2bfe1c
Update constantine/build.gradle
NickSneo Jul 19, 2024
17c57b6
Update constantine/build.gradle
NickSneo Jul 19, 2024
cf38680
Update constantine/build.gradle
NickSneo Jul 19, 2024
3dd0160
Update constantine/build.gradle
NickSneo Jul 19, 2024
52281fb
check CI/CD is passing
NickSneo Jul 19, 2024
bb7a57b
which nimble cmd test
NickSneo Jul 19, 2024
dac3084
prev implementation is working
NickSneo Jul 19, 2024
235c91a
just testing
NickSneo Jul 29, 2024
7c9f645
added constantine in build.sh
NickSneo Jul 29, 2024
b96360c
export PATH nimble
NickSneo Jul 29, 2024
18b8050
fix outout dir
NickSneo Jul 29, 2024
620b308
added prepare for linux-arm64
NickSneo Jul 29, 2024
f824860
skip for linux arm64
NickSneo Jul 29, 2024
f35318f
fix linkage and source not found
NickSneo Jul 29, 2024
23a7190
fix lib copy directories
NickSneo Jul 29, 2024
03a6057
fix for lib directory
NickSneo Jul 29, 2024
1423a90
Merge branch 'main' into main
NickSneo Aug 12, 2024
c5b504a
fixed shared lib for linux
NickSneo Aug 13, 2024
98035f1
Merge branch 'main' into main
garyschulte Aug 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ Cargo.lock

.gradle

*.DS_Store

## build files
*.com
*.class
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@
[submodule "secp256r1/besu-native-ec"]
path = secp256r1/besu-native-ec
url = [email protected]:ConsenSys/besu-native-ec.git
[submodule "constantine/constantine"]
path = constantine/constantine
url = https://github.com/mratsim/constantine.git
166 changes: 166 additions & 0 deletions constantine/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
plugins {
id 'java-library'
id 'maven-publish'
id 'com.jfrog.artifactory' version '4.20.0'
}

repositories {
mavenCentral()
}

dependencies {
implementation 'net.java.dev.jna:jna:5.12.1'
testImplementation 'junit:junit:4.13.2'
}

java {
toolchain {
languageVersion = JavaLanguageVersion.of(21)
}
}

//task runNimbleMakeLib(type: Exec) {
// description = 'Runs CTT_LTO=false nimble make_lib inside constantine folder'
// workingDir 'constantine'
// environment 'CTT_LTO', 'false'
// commandLine 'nimble', 'make_lib'
//}

task compileJavaSource(type: Exec) {
description = 'Compiles the Java source files'
commandLine 'javac', 'src/main/java/org/hyperledger/besu/nativelib/constantine/LibConstantineEIP196.java'
}

task generateJNIHeader(type: Exec) {
description = 'Generates the JNI header file'
commandLine 'javac', '-h', 'constantine/constantine-jni/include', 'src/main/java/org/hyperledger/besu/nativelib/constantine/LibConstantineEIP196.java'
}

task compileNativeLibrary(type: Exec) {
description = 'Compiles the native library'
commandLine 'clang', '-I', "${System.env.JAVA_HOME}/include", '-I', "${System.env.JAVA_HOME}/include/darwin", '-shared', '-o', 'constantine-jni/lib/libconstantine.jnilib', 'constantine-jni/src/main/c/ethereum_evm_precompiles.c', '-I', 'constantine/include', '-I', 'src/main/java/org/hyperledger/besu/nativelib/constantine', '-L', 'constantine-jni/lib', '-lconstantine'
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idiomatic besu-native builds incorporate the build of the library into the root build.sh script and use automake or a similar tool to ensure that the appropriate toolchain is configured (cc, gcc, clang, etc).

I think it is reasonable to use gradle for these tasks, but we should expect to be able to build on a variety of platforms.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, adding a dependsOn chain between these dependent tasks will make for an easier build ux.

NickSneo marked this conversation as resolved.
Show resolved Hide resolved

task runJavaProgram(type: Exec) {
description = 'Runs the Java program'
commandLine 'java', '-Djava.library.path=constantine-jni/lib', 'org.hyperledger.besu.nativelib.constantine.LibConstantineEIP196'
}

compileNativeLibrary.dependsOn generateJNIHeader
//generateJNIHeader.dependsOn compileJavaSource
//compileJavaSource.dependsOn runNimbleMakeLib
runJavaProgram.dependsOn compileNativeLibrary

tasks.withType(JavaCompile) {
options.encoding = 'UTF-8'
}

task macArmLibCopy(type: Copy) {
from 'constantine-jni/lib/libconstantine.dylib'
into 'build/resources/main/darwin-aarch64'
}

task macLibCopy(type: Copy) {
from 'constantine-jni/lib/libconstantine.dylib'
into 'build/resources/main/darwin-x86-64'
}

task linuxLibCopy(type: Copy) {
from 'constantine-jni/lib/libconstantine.so'
into 'build/resources/main/linux-x86-64'
}

task linuxArm64LibCopy(type: Copy) {
from 'constantine-jni/lib/libconstantine.so'
into 'build/resources/main/linux-aarch64'
}

NickSneo marked this conversation as resolved.
Show resolved Hide resolved
processResources.dependsOn macArmLibCopy, macLibCopy, linuxLibCopy, linuxArm64LibCopy

jar {
archiveBaseName = 'besu-native-constantine'
includeEmptyDirs = false
manifest {
attributes(
'Specification-Title': archiveBaseName,
'Specification-Version': project.version,
'Implementation-Title': archiveBaseName,
'Implementation-Version': project.version,
'Automatic-Module-Name': 'org.hyperledger.besu.nativelib.constantine'
)
}
}

task sourcesJar(type: Jar, dependsOn: classes) {
archiveBaseName = 'besu-native-constantine'
archiveClassifier = 'sources'
from sourceSets.main.allSource
}

task javadocJar(type: Jar, dependsOn: javadoc) {
archiveBaseName = 'besu-native-constantine'
archiveClassifier = 'javadoc'
from javadoc.destinationDir
}

publishing {
publications {
mavenJava(MavenPublication) {
groupId "org.hyperledger.besu"
artifactId 'constantine'
version "${project.version}"

from components.java
artifact sourcesJar
artifact javadocJar

pom {
name = "Besu Native - ${project.name}"
description = 'Adapter for native constantine library'
url = 'http://github.com/hyperledger/besu-native'
licenses {
license {
name = 'The Apache License, Version 2.0'
url = 'http://www.apache.org/licenses/LICENSE-2.0.txt'
}
}
scm {
connection = 'scm:git:git://github.com/hyperledger/besu-native.git'
developerConnection = 'scm:git:ssh://github.com/hyperledger/besu-native.git'
url = 'https://github.com/hyperledger/besu-native'
}
}
}
}
}

def artifactoryUser = project.hasProperty('artifactoryUser') ? project.property('artifactoryUser') : System.getenv('ARTIFACTORY_USER')
def artifactoryKey = project.hasProperty('artifactoryApiKey') ? project.property('artifactoryApiKey') : System.getenv('ARTIFACTORY_KEY')
def artifactoryRepo = System.getenv('ARTIFACTORY_REPO') ?: 'besu-maven'
def artifactoryOrg = System.getenv('ARTIFACTORY_ORG') ?: 'hyperledger'

artifactory {
contextUrl = "https://hyperledger.jfrog.io/${artifactoryOrg}"
publish {
repository {
repoKey = artifactoryRepo
username = artifactoryUser
password = artifactoryKey
}
defaults {
publications('mavenJava')
publishArtifacts = true
publishPom = true
}
}
}

test {
useJUnit()
}

task runJavaTests(type: Test) {
description = 'Runs the Java tests'
include '**/*Test.class'
}
runJavaTests.dependsOn runJavaProgram
1 change: 1 addition & 0 deletions constantine/constantine
Submodule constantine added at b13816
Binary file added constantine/constantine-jni/lib/libconstantine.a
NickSneo marked this conversation as resolved.
Show resolved Hide resolved
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
69 changes: 69 additions & 0 deletions constantine/constantine-jni/src/main/c/ethereum_evm_precompiles.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#include <jni.h>
#include "org_hyperledger_besu_nativelib_constantine_LibConstantineEIP196.h"
#include <constantine.h>
#include <stdio.h>

void printByteArray(const char* label, const byte* array, size_t len) {
printf("%s: [", label);
for (size_t i = 0; i < len; i++) {
printf("%02x", array[i]);
if (i < len - 1) {
printf(", ");
}
}
printf("]\n");
}

JNIEXPORT jint JNICALL Java_Constantine_ctt_1eth_1evm_1bn254_1g1add(JNIEnv *env, jobject obj, jbyteArray jr, jint r_len, jbyteArray jinputs, jint inputs_len) {
jbyte *r = (*env)->GetByteArrayElements(env, jr, NULL);
jbyte *inputs = (*env)->GetByteArrayElements(env, jinputs, NULL);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately I'm not familiar with Java at all but it seems like what is also being done in c-kzg-4844 so it's likely the expected way to wrap:

https://github.com/ethereum/c-kzg-4844/blob/1bccee0878ffc80efe8741afdb5793ef9105aa35/bindings/java/c_kzg_4844_jni.c#L148-L179


ctt_evm_status status = ctt_eth_evm_bn254_g1add((byte *)r, (ptrdiff_t)r_len, (const byte *)inputs, (ptrdiff_t)inputs_len);

if (status != cttEVM_Success) {
printf("ctt_eth_evm_bn254_g1add failed with status: %d\n", status);
} else {
printByteArray("Result", (const byte *)r, r_len);
NickSneo marked this conversation as resolved.
Show resolved Hide resolved
}

(*env)->ReleaseByteArrayElements(env, jr, r, 0);
(*env)->ReleaseByteArrayElements(env, jinputs, inputs, 0);

return (jint)status;
}

JNIEXPORT jint JNICALL Java_Constantine_ctt_1eth_1evm_1bn254_1g1mul(JNIEnv *env, jobject obj, jbyteArray jr, jint r_len, jbyteArray jinputs, jint inputs_len) {
jbyte *r = (*env)->GetByteArrayElements(env, jr, NULL);
jbyte *inputs = (*env)->GetByteArrayElements(env, jinputs, NULL);

ctt_evm_status status = ctt_eth_evm_bn254_g1mul((byte *)r, (ptrdiff_t)r_len, (const byte *)inputs, (ptrdiff_t)inputs_len);
Copy link
Contributor

@garyschulte garyschulte Jul 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very tidy that constantine already has precompile friendly endpoints 👍 . Is this the same for eip2537?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and SHA256 and MODEXP. The plan is to implement all cryptographic precompiles.

For the record, Constantine was actually started with the idea of having a clean from scratch BN254 backend for Nim but it was too hard/time-consuming at the time so a 1-1 port of https://github.com/zcash-hackworks/bn was made instead (https://github.com/status-im/nim-bncurve).
However performance is bad as even a simple add with carry instruction is transformed into 7 instructions: https://github.com/zcash-hackworks/bn/blob/master/src/arith.rs#L396-L405

The issue of zcash's bn were already mentioned in 2008 in EIP-1108 https://eips.ethereum.org/EIPS/eip-1108 that reduced gas cost price. And moving to Cloudflare bn256 improved speed by 10x:

and corroborated in a later note, the computational cost of ECADD, ECMUL, and pairing checks (excepting the constant) has dropped roughly an order of magnitude across the board.

However that Cloudflare library was about 2x slower than state-of-the-art in 2021 https://hackmd.io/@gnark/eccbench
https://user-images.githubusercontent.com/22738317/162632923-b057c0a2-6929-419e-b0de-4fd187a5e508.png
and Constantine there had a parameter passing bug which made it 2x slower than necessary on pairings (nim-lang/Nim#16897)

See full context in https://ethresear.ch/t/releasing-constantine-v0-1-0-a-modular-cryptography-stack-for-ethereum/19990


if (status != cttEVM_Success) {
printf("ctt_eth_evm_bn254_g1mul failed with status: %d\n", status);
} else {
printByteArray("Result", (const byte *)r, r_len);
}

(*env)->ReleaseByteArrayElements(env, jr, r, 0);
(*env)->ReleaseByteArrayElements(env, jinputs, inputs, 0);

return (jint)status;
}

JNIEXPORT jint JNICALL Java_Constantine_ctt_1eth_1evm_1bn254_1pairingCheck(JNIEnv *env, jobject obj, jbyteArray jr, jint r_len, jbyteArray jinputs, jint inputs_len) {
jbyte *r = (*env)->GetByteArrayElements(env, jr, NULL);
jbyte *inputs = (*env)->GetByteArrayElements(env, jinputs, NULL);

ctt_evm_status status = ctt_eth_evm_bn254_ecpairingcheck((byte *)r, (ptrdiff_t)r_len, (const byte *)inputs, (ptrdiff_t)inputs_len);

if (status != cttEVM_Success) {
printf("ctt_eth_evm_bn254_pairingCheck failed with status: %d\n", status);
} else {
printByteArray("Result", (const byte *)r, r_len);
}

(*env)->ReleaseByteArrayElements(env, jr, r, 0);
(*env)->ReleaseByteArrayElements(env, jinputs, inputs, 0);

return (jint)status;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package org.hyperledger.besu.nativelib.constantine;

public class LibConstantineEIP196 {
static {
System.loadLibrary("constantine");
}

public native int ctt_eth_evm_bn254_g1add(byte[] r, int r_len, byte[] inputs, int inputs_len);
public native int ctt_eth_evm_bn254_g1mul(byte[] r, int r_len, byte[] inputs, int inputs_len);
public native int ctt_eth_evm_bn254_pairingCheck(byte[] r, int r_len, byte[] inputs, int inputs_len);

public static void main(String[] args) {
LibConstantineEIP196 constInstance = new LibConstantineEIP196();

byte[] r = new byte[64];
byte[] inputs = new byte[128];
int status = constInstance.ctt_eth_evm_bn254_g1add(r, r.length, inputs, inputs.length);
System.out.println("ctt_eth_evm_bn254_g1add status: " + status + ", result: " + bytesToHex(r));

r = new byte[64];
inputs = new byte[96];
status = constInstance.ctt_eth_evm_bn254_g1mul(r, r.length, inputs, inputs.length);
System.out.println("ctt_eth_evm_bn254_g1mul status: " + status + ", result: " + bytesToHex(r));

r = new byte[32];
inputs = new byte[256];
status = constInstance.ctt_eth_evm_bn254_pairingCheck(r, r.length, inputs, inputs.length);
System.out.println("ctt_eth_evm_bn254_pairingCheck status: " + status + ", result: " + bytesToHex(r));
}

private static String bytesToHex(byte[] bytes) {
StringBuilder sb = new StringBuilder();
for (byte b : bytes) {
sb.append(String.format("%02x", b));
}
return sb.toString();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package org.hyperledger.besu.nativelib.constantine;

import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;

public class LibConstantineEIP196Test {
private LibConstantineEIP196 constInstance;

@Before
public void setUp() {
constInstance = new LibConstantineEIP196();
}

@Test
public void testG1Add() {
byte[] r = new byte[64];
byte[] inputs = new byte[128];
int status = constInstance.ctt_eth_evm_bn254_g1add(r, r.length, inputs, inputs.length);
assertEquals("Status should be cttEVM_Success", 0, status);
assertNotNull("Result array should not be null", r);
}

@Test
public void testG1Mul() {
byte[] r = new byte[64];
byte[] inputs = new byte[96];
int status = constInstance.ctt_eth_evm_bn254_g1mul(r, r.length, inputs, inputs.length);
assertEquals("Status should be cttEVM_Success", 0, status);
assertNotNull("Result array should not be null", r);
}

@Test
public void testPairingCheck() {
byte[] r = new byte[32];
byte[] inputs = new byte[256];
int status = constInstance.ctt_eth_evm_bn254_pairingCheck(r, r.length, inputs, inputs.length);
assertEquals("Status should be cttEVM_Success", 0, status);
assertNotNull("Result array should not be null", r);
}
}
1 change: 1 addition & 0 deletions settings.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@ include 'ipa-multipoint'
include 'secp256k1'
include 'secp256r1'
include 'gnark'
include 'constantine'
Loading