Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WASI: fs.FS as Preopen parameter with implementation of fd_prestat_get #362

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,25 @@ System Interface ([WASI](https://github.com/WebAssembly/WASI)). WASI defines
how WebAssembly programs interact with the host embedding them. For example,
WASI defines functions for reading the time, or a random number.

This repository includes several [examples](examples) that expose system
interfaces, via the module `wazero.WASISnapshotPreview1`. These examples are
tested and a good way to learn what's possible with wazero.
For example, wazero allows you to control which files a WebAssembly function
can read via `fs.FS`:
```golang
nullpo-head marked this conversation as resolved.
Show resolved Hide resolved
//go:embed your_directory
var embeddedFS embed.FS

--snip--
wasiConfig := wazero.NewWASIConfig().
// Mount the embedded filesystem read-only under the directory /static
WithPreopens(map[string]fs.FS{"/static": embeddedFS})

// Instantiate the wasm module with WASI APIs enabled.
wasi, err := r.InstantiateModule(wazero.WASISnapshotPreview1WithConfig(wasiConfig))
defer wasi.Close()
```

This repository includes several [examples](examples) that showcase this and
other system interfaces. These examples are tested and a good way to learn
what's possible with wazero.

## Runtime

Expand Down
65 changes: 38 additions & 27 deletions examples/file_system_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@ package examples

import (
"bytes"
_ "embed"
"embed"
"io"
"io/fs"
"os"
"testing"

"github.com/stretchr/testify/require"
Expand All @@ -16,42 +18,49 @@ import (
//go:embed testdata/file_system.wasm
var filesystemWasm []byte

func writeFile(fs wasi.FS, path string, data []byte) error {
f, err := fs.OpenWASI(0, path, wasi.O_CREATE|wasi.O_TRUNC, wasi.R_FD_WRITE, 0, 0)
if err != nil {
return err
}
// Embed a directory as FS.
//go:embed testdata/file_system_input/input.txt
var embeddedFS embed.FS

if _, err := io.Copy(f, bytes.NewBuffer(data)); err != nil {
return err
}

return f.Close()
}

func readFile(fs wasi.FS, path string) ([]byte, error) {
f, err := fs.OpenWASI(0, path, 0, 0, 0, 0)
if err != nil {
return nil, err
}
func writeFile(t *testing.T, fs wasi.OpenFileFS, path string, data []byte) {
f, err := fs.OpenFile(path, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, os.FileMode(0644))
require.NoError(t, err)

buf := bytes.NewBuffer(nil)
// fs.File can be casted to io.Writer when it's writable.
writer, ok := f.(io.Writer)
require.True(t, ok)

if _, err := io.Copy(buf, f); err != nil {
return buf.Bytes(), nil
}
_, err = io.Copy(writer, bytes.NewBuffer(data))
require.NoError(t, err)

return buf.Bytes(), f.Close()
err = f.Close()
require.NoError(t, err)
}

func Test_file_system(t *testing.T) {
r := wazero.NewRuntime()

memFS := wazero.WASIMemFS()
err := writeFile(memFS, "input.txt", []byte("Hello, file system!"))
// embeddedFS is a read-only embed.FS file system. Get the sub FS to shorten the nested directories.
embeddedFS, err := fs.Sub(embeddedFS, "testdata/file_system_input")
require.NoError(t, err)

wasiConfig := wazero.NewWASIConfig().WithPreopens(map[string]wasi.FS{".": memFS})
// WASIMemFS returns a in-memory file system that supports both read and write.
memFS := wazero.WASIMemFS()
// memFS is writable. Create another input file.
writeFile(t, memFS, "input.txt", []byte("Hello, "))

// Configure what resources you share with the WASI program.
wasiConfig := wazero.NewWASIConfig()
// Share fs.FS as pre-opened directories
wasiConfig = wasiConfig.WithPreopens(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like WithXXXMount will be easier for people to understand that "preopens" WDYT? Also, we can make it chained instead of a map. the map can be an internal detail

Ex.

// documented as read-only
wasiConfig.WithMount("/input", embeddedFS)

map[string]fs.FS{
".": memFS, // pre-open the writable in-memory directory at the working directory
"/input": embeddedFS, // pre-open the embeddedFS at "/input"
},
)
// the wasm module will concatenate the contents of the two input.txt and write to output.txt
wasiConfig = wasiConfig.WithArgs("./input.txt", "/input/input.txt", "./output.txt")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just overwrite a file instead? that's a lot simpler and less code.


wasi, err := r.InstantiateModule(wazero.WASISnapshotPreview1WithConfig(wasiConfig))
require.NoError(t, err)
defer wasi.Close()
Expand All @@ -61,7 +70,9 @@ func Test_file_system(t *testing.T) {
require.NoError(t, err)
defer mod.Close()

out, err := readFile(memFS, "output.txt")
// "input.txt" in the MemFS has "Hello, ", and "/testdata/file_system_input/input.txt" has "file system!\n".
// So, the result "output.txt" should contain "Hello, file system!\n".
out, err := fs.ReadFile(memFS, "output.txt")
require.NoError(t, err)
require.Equal(t, "Hello, file system!", string(out))
}
26 changes: 17 additions & 9 deletions examples/testdata/file_system.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,29 @@ import (
"os"
)

// This example reads the input file paths from args except the last argument,
// and writes the concatenated contents to the last file path.
func main() {
fileIn, err := os.Open("input.txt")
if err != nil {
panic(err)
}
defer fileIn.Close()
files := os.Args
inputFiles := files[:len(files)-1]
outputFile := files[len(files)-1]

fileOut, err := os.Create("output.txt")
fileOut, err := os.Create(outputFile)
if err != nil {
panic(err)
}
defer fileOut.Close()

_, err = io.Copy(fileOut, fileIn)
if err != nil {
panic(err)
for _, input := range inputFiles {
fileIn, err := os.Open(input)
if err != nil {
panic(err)
}
defer fileIn.Close()

_, err = io.Copy(fileOut, fileIn)
if err != nil {
panic(err)
}
}
}
Binary file modified examples/testdata/file_system.wasm
Binary file not shown.
1 change: 1 addition & 0 deletions examples/testdata/file_system_input/input.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
file system!
Loading