Merge pull request #18 from extism/updated-extism-host-api

- Loosely follow the java-sdk, expose ExtismHostFunction and use an internal
  ExtismValTypeList that avoids wrapping as much as possible.
- Hide the conversion between longs and high-level, types
- Add test case following the java-sdk
- Update the README

Signed-off-by: Edoardo Vacchi <[email protected]>

README.md

@@ -8,16 +8,187 @@ to complete a full Extism SDK. If anyone would like to work on it feel free to r
 > **Note**: If you are interested in a solid and working Java SDK, see our [Extism Java SDK](https://github.com/extism/java-sdk).
 > But if you have a need for pure Java solution, please reach out!
 
-## Example
+## Installation
+
+### Maven
+
+To use the Chicory java-sdk with maven you need to add the following dependency to your `pom.xml` file:
+```xml
+<dependency>
+    <groupId>org.extism.sdk</groupId>
+    <artifactId>chicory-sdk</artifactId>
+    <version>999-SNAPSHOT</version>
+</dependency>
+```
+
+
+### Gradle
+
+To use the Chicory java-sdk with maven you need to add the following dependency to your `build.gradle` file:
+
+```
+implementation 'org.extism.sdk:chicory-sdk:999-SNAPSHOT'
+```
+
+## Getting Started
+
+The primary concept in Extism is the [plug-in](https://extism.org/docs/concepts/plug-in). You can think of a plug-in as a code module stored in a `.wasm` file.
+Since you may not have a Extism plug-in on hand to test, let's load a demo plug-in from the web:
+
+```java
+var url = "https://github.com/extism/plugins/releases/latest/download/count_vowels.wasm";
+var wasm = ManifestWasm.fromUrl(url).build();
+var manifest = Manifest.ofWasms(wasm).build();
+var plugin = Plugin.ofManifest(manifest).build();
+```
+
+> **Note**: See [the Manifest docs](https://www.javadoc.io/doc/org.extism.sdk/extism/latest/org/extism/sdk/manifest/Manifest.html) as it has a rich schema and a lot of options.
+
+### Calling A Plug-in's Exports
+
+This plug-in was written in Rust and it does one thing, it counts vowels in a string. As such, it exposes one "export" function: `count_vowels`. 
+We can call exports using [Plugin#call](https://www.javadoc.io/doc/org.extism.sdk/extism/latest/org/extism/sdk/Plugin.html#call(java.lang.String,byte[]))
+
+```java
+var output = plugin.call("count_vowels", "Hello, World!".getBytes(StandardCharsets.UTF_8));
+System.out.println(new String(output, StandardCharsets.UTF_8));
+// => "{"count": 3, "total": 3, "vowels": "aeiouAEIOU"}"
+```
+
+All exports have a simple interface of bytes-in and bytes-out.
+This plug-in happens to take a string and return a JSON encoded string with a report of results.
+
+
+### Plug-in State
+
+Plug-ins may be stateful or stateless. Plug-ins can maintain state b/w calls by the use of variables.
+Our count vowels plug-in remembers the total number of vowels it's ever counted in the "total" key in the result.
+You can see this by making subsequent calls to the export:
+
+```java
+var output = plugin.call("count_vowels","Hello, World!".getBytes(StandardCharsets.UTF_8));
+System.out.println(output);
+// => "{"count": 3, "total": 6, "vowels": "aeiouAEIOU"}"
+
+var output = plugin.call("count_vowels", "Hello, World!".getBytes(StandardCharsets.UTF_8));
+System.out.println(output);
+// => "{"count": 3, "total": 9, "vowels": "aeiouAEIOU"}"
+```
+
+These variables will persist until this plug-in is freed or you initialize a new one.
+
+### Configuration
+
+Plug-ins may optionally take a configuration object. This is a static way to configure the plug-in.
+Our count-vowels plugin takes an optional configuration to change out which characters are considered vowels. Example:
 
 ```java
-        var manifest =
-        Manifest.ofWasms(
-                ManifestWasm.fromUrl(
-                                "https://github.com/extism/plugins/releases/download/v1.1.0/greet.wasm")
-                        .build()).build();
-var plugin = Plugin.Builder.ofManifest(manifest).build();
-var input = "Benjamin";
-var result = new String(plugin.call("greet", input.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8);
-assertEquals("Hello, Benjamin!", result);
-```
+var plugin = new Plugin(manifest, false, null);
+var output = plugin.call("count_vowels", "Yellow, World!");
+System.out.println(output);
+// => {"count": 3, "total": 3, "vowels": "aeiouAEIOU"}
+
+// Let's change the vowels config it uses to determine what is a vowel:
+var config = Map.of("vowels", "aeiouyAEIOUY");
+var manifest2 = Manifest.ofWasms(wasm)
+        .withOptions(new Manifest.Options().withConfig(config)).build();
+var plugin = Plugin.ofManifest(manifest2).build();
+var result = new String(plugin.call("count_vowels", "Yellow, World!".getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8);
+System.out.println(output);
+// => {"count": 4, "total": 4, "vowels": "aeiouyAEIOUY"}
+// ^ note count changed to 4 as we configured Y as a vowel this time
+```
+
+### Host Functions
+
+Let's extend our count-vowels example a little bit: Instead of storing the `total` in an ephemeral plug-in var,
+let's store it in a persistent key-value store!
+
+Wasm can't use our app's KV store on its own. This is where [Host Functions](https://extism.org/docs/concepts/host-functions) come in.
+
+[Host functions](https://extism.org/docs/concepts/host-functions) allow us to grant new capabilities to our plug-ins from our application.
+They are simply some java methods you write which can be passed down and invoked from any language inside the plug-in.
+
+Let's load the manifest like usual but load up this `count_vowels_kvstore` plug-in:
+
+```java
+var url = "https://github.com/extism/plugins/releases/latest/download/count_vowels_kvstore.wasm";
+var manifest = new Manifest(List.of(UrlWasmSource.fromUrl(url)));
+var plugin = new Plugin(manifest, false, null);
+```
+
+> *Note*: The source code for this plug-in is [here](https://github.com/extism/plugins/blob/main/count_vowels_kvstore/src/lib.rs)
+> and is written in rust, but it could be written in any of our PDK languages.
+
+Unlike our previous plug-in, this plug-in expects you to provide host functions that satisfy its import interface for a KV store.
+We want to expose two functions to our plugin, `kv_write(String key, Bytes value)` which writes a bytes value to a key and `Bytes kv_read(String key)` which reads the bytes at the given `key`.
+
+```java
+// Our application KV store
+// Pretend this is redis or a database :)
+var kvStore = new HashMap<String, byte[]>();
+
+ExtismFunction kvWrite = (plugin, params, returns) -> {
+    System.out.println("Hello from kv_write Java Function!");
+    var key = plugin.memory().readString(params.getRaw(0));
+    var value = plugin.memory().readBytes(params.getRaw(1));
+    System.out.println("Writing to key " + key);
+    kvStore.put(key, value);
+};
+
+ExtismFunction kvRead = (plugin, params, returns) -> {
+    System.out.println("Hello from kv_read Java Function!");
+    var key = plugin.memory().readString(params.getRaw(0));
+    System.out.println("Reading from key " + key);
+    var value = kvStore.get(key);
+    if (value == null) {
+        // default to zeroed bytes
+        var zero = new byte[]{0, 0, 0, 0};
+        returns.setRaw(0, plugin.memory().writeBytes(zero));
+    } else {
+        returns.setRaw(0, plugin.memory().writeBytes(value));
+    }
+};
+
+var kvWriteHostFn = ExtismHostFunction.of(
+        "kv_write",
+        List.of(ExtismValType.I64, ExtismValType.I64),
+        List.of(),
+        kvWrite
+);
+
+var kvReadHostFn = ExtismHostFunction.of(
+        "kv_read",
+        List.of(ExtismValType.I64),
+        List.of(ExtismValType.I64),
+        kvRead
+);
+```
+
+> *Note*: In order to write host functions you should get familiar with the methods on the [ExtismCurrentPlugin](https://www.javadoc.io/doc/org.extism.sdk/extism/latest/org/extism/sdk/ExtismCurrentPlugin.html) class.
+> The `plugin` parameter is an instance of this class.
+
+Now we just need to pass in these function references when creating the plugin:.
+
+```java
+var plugin = Plugin.ofManifest(manifest).withHostFunctions(kvReadHostFn, kvWriteHostFn).build();
+var output = plugin.call("count_vowels", "Yellow, World!".getBytes(StandardCharsets.UTF_8));
+var result = new String(output, StandardCharsets.UTF_8);
+// => Hello from kv_read Java Function!
+// => Reading from key count-vowels
+// => Hello from kv_write Java Function!
+// => Writing to key count-vowels
+System.out.println(output);
+// => {"count": 3, "total": 3, "vowels": "aeiouAEIOU"}
+```
+
+## Development
+
+# Build
+
+To build the Extism chicory-sdk run the following command:
+
+```
+mvn clean verify
+```
+

src/main/java/org/extism/chicory/sdk/CurrentPlugin.java

@@ -15,4 +15,17 @@ public HostEnv.Memory memory() {
         return plugin.memory();
     }
 
+
+    void setInput(byte[] input) {
+        plugin.setInput(input);
+    }
+
+    byte[] getOutput() {
+        return plugin.getOutput();
+    }
+
+    String getError() {
+        return plugin.getError();
+    }
+
 }

src/main/java/org/extism/chicory/sdk/ExtismFunction.java

@@ -0,0 +1,6 @@
+package org.extism.chicory.sdk;
+
+@FunctionalInterface
+public interface ExtismFunction {
+    void apply(CurrentPlugin currentPlugin, ExtismValueList args, ExtismValueList returns);
+}

src/main/java/org/extism/chicory/sdk/ExtismHostFunction.java

@@ -2,8 +2,6 @@
 
 import com.dylibso.chicory.runtime.HostFunction;
 import com.dylibso.chicory.runtime.Instance;
-import com.dylibso.chicory.wasm.types.Value;
-import com.dylibso.chicory.wasm.types.ValueType;
 
 import java.util.List;
 
@@ -12,39 +10,39 @@ public final class ExtismHostFunction {
 
     public static ExtismHostFunction of(
             String name,
-            List<ValueType> paramTypes,
-            List<ValueType> returnTypes,
-            Handle handle) {
-        return new ExtismHostFunction(DEFAULT_NAMESPACE, name, handle, paramTypes, returnTypes);
+            List<ExtismValType> paramTypes,
+            List<ExtismValType> returnTypes,
+            ExtismFunction extismFunction) {
+        return new ExtismHostFunction(DEFAULT_NAMESPACE, name, paramTypes, returnTypes, extismFunction);
     }
 
     public static ExtismHostFunction of(
             String module,
             String name,
-            Handle handle,
-            List<ValueType> paramTypes,
-            List<ValueType> returnTypes) {
-        return new ExtismHostFunction(module, name, handle, paramTypes, returnTypes);
+            ExtismFunction extismFunction,
+            List<ExtismValType> paramTypes,
+            List<ExtismValType> returnTypes) {
+        return new ExtismHostFunction(module, name, paramTypes, returnTypes, extismFunction);
     }
 
     private final String module;
     private final String name;
-    private final Handle handle;
-    private final List<ValueType> paramTypes;
-    private final List<ValueType> returnTypes;
+    private final ExtismFunction extismFunction;
+    private final ExtismValTypeList paramTypes;
+    private final ExtismValTypeList returnTypes;
     private CurrentPlugin currentPlugin;
 
-    ExtismHostFunction(
+    private ExtismHostFunction(
             String module,
             String name,
-            Handle handle,
-            List<ValueType> paramTypes,
-            List<ValueType> returnTypes) {
+            List<ExtismValType> paramTypes,
+            List<ExtismValType> returnTypes,
+            ExtismFunction extismFunction) {
         this.module = module;
         this.name = name;
-        this.handle = handle;
-        this.paramTypes = paramTypes;
-        this.returnTypes = returnTypes;
+        this.paramTypes = new ExtismValTypeList(paramTypes);
+        this.returnTypes = new ExtismValTypeList(returnTypes);
+        this.extismFunction = extismFunction;
     }
 
     public void bind(CurrentPlugin p) {
@@ -58,12 +56,13 @@ public void bind(CurrentPlugin p) {
 
     final HostFunction asHostFunction() {
         return new HostFunction(
-                module, name, paramTypes, returnTypes,
-                (Instance inst, long... args) -> handle.apply(this.currentPlugin, args));
+                module, name, paramTypes.toChicoryTypes(), returnTypes.toChicoryTypes(),
+                (Instance inst, long... args) -> {
+                    var params = paramTypes.toExtismValueList(args);
+                    var results = returnTypes.toExtismValueList();
+                    extismFunction.apply(this.currentPlugin, params, results);
+                    return results.unwrap();
+                });
     }
 
-    @FunctionalInterface
-    public interface Handle {
-        long[] apply(CurrentPlugin currentPlugin, long... args);
-    }
 }

src/main/java/org/extism/chicory/sdk/ExtismTypeConversionException.java

@@ -0,0 +1,9 @@
+package org.extism.chicory.sdk;
+
+public class ExtismTypeConversionException extends ExtismException {
+
+    public ExtismTypeConversionException(ExtismValType expected, ExtismValType given) {
+        super(String.format("Illegal type conversion, wanted %s, given %s", expected.name(), given.name()));
+    }
+
+}

src/main/java/org/extism/chicory/sdk/ExtismValType.java

@@ -0,0 +1,39 @@
+package org.extism.chicory.sdk;
+
+import com.dylibso.chicory.wasm.types.ValueType;
+
+public enum ExtismValType {
+    I32(ValueType.I32),
+    I64(ValueType.I64),
+    F32(ValueType.F32),
+    F64(ValueType.F64);
+
+    private final ValueType chicoryType;
+
+    ExtismValType(ValueType chicoryType) {
+        this.chicoryType = chicoryType;
+    }
+
+    ValueType toChicoryValueType() {
+        return chicoryType;
+    }
+
+    public ExtismValue toExtismValue(long v) {
+        switch (this) {
+            case I32:
+                return ExtismValue.i32(v);
+            case I64:
+                return ExtismValue.i64(v);
+            case F32:
+                return ExtismValue.f32FromLongBits(v);
+            case F64:
+                return ExtismValue.f64FromLongBits(v);
+            default:
+                throw new IllegalArgumentException();
+        }
+    }
+
+    public long toChicoryValue(ExtismValue value) {
+        return 0;
+    }
+}

src/main/java/org/extism/chicory/sdk/ExtismValTypeList.java

@@ -0,0 +1,33 @@
+package org.extism.chicory.sdk;
+
+import com.dylibso.chicory.wasm.types.ValueType;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+class ExtismValTypeList {
+    private final ExtismValType[] types;
+    private final List<ValueType> chicoryTypes;
+
+    ExtismValTypeList(List<ExtismValType> types) {
+        this.types = types.toArray(ExtismValType[]::new);
+        this.chicoryTypes = types.stream().map(ExtismValType::toChicoryValueType)
+                .collect(Collectors.toList());
+    }
+
+    List<ValueType> toChicoryTypes() {
+        return Arrays.stream(types)
+                .map(ExtismValType::toChicoryValueType)
+                .collect(Collectors.toList());
+    }
+
+    public ExtismValueList toExtismValueList(long[] args) {
+        return new ExtismValueList(this.types, args);
+    }
+
+    public ExtismValueList toExtismValueList() {
+        return new ExtismValueList(this.types, new long[this.types.length]);
+    }
+
+}