Golem 1.0 Launch Demo

Description

In this demo we implement a collaborative list editor application using Golem. The application can handle an arbitrary number of simultaneously open lists - each list consists of a list of string items. These list items can be appended, inserted and deleted simultaneously by multiple users; the current list state can be queried any time, as well as the active "editor" connections. Modification is only allowed for connected editors, and there is a poll function available for them which only returns the new changes since the last call.

Lists can be archived, in which case they are no longer editable and their contents are saved in a separate list archive. Then the list can be deleted, it's last state remains forever in archive.

An additional feature is that if a list is not archived and there are no changes for a certain period of time, all the connected editors are notified by sending an email to them. (Note: the demo does not actually implement the email sending, just prints a log line where it would do so.)

Demo script

Phase 1

In phase 1 we create the first version of the lst component, where each worker represents a stateful list and provides some basic manipulation and query functionalities.

Create the new component:

golem-cloud-cli new --lang ts --package-name demo:lst lst

Compile the initial version:

cd lst
npm install
npm run componentize

We have a Golem component in out/lst.wasm..

Apply the required changes to the WIT file (prepared/phase-1/lst/wit/main.wit) and regenerate the bindings:

npm run componentize

Then implement the first version (prepared/phase-1/lst/src/main.ts) and compile again:

npm run componentize

Let's set the project ID we are working with (Golem Cloud only) to an environment variable:

export PRJ=urn:project:b17d7bbf-9704-4578-bc25-8b1ad22f3f3a # Live demo project

Deploy the component and store it's ID in an environment variable:

golem-cloud-cli component add --project $PRJ --component-name lst out/lst.wasm
export LST=urn:component:4a3d6c13-9086-43d6-88c6-be4faeedc1f7

Try it out:

golem-cloud-cli worker invoke-and-await --component $LST --worker-name test1 --function 'demo:lst/api.{add}' --arg '"item 1"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test1 --function 'demo:lst/api.{add}' --arg '"item 3"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test1 --function 'demo:lst/api.{insert}' --arg '"item 1"' --arg '"item 2"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test1 --function 'demo:lst/api.{get}'

Phase 2

In phase 2 we add the concept of a connection and polling.

First we modify the WIT file (prepared/phase-2/lst/wit/main.wit) and regenerate the bindings:

npm run componentize

Then implement the changes (prepared/phase-2/lst/src/main.ts)

• Define EditorState
• Add a connected map and a lastConnectionId variable to State
• Write two helper functions: isConnected and addEvent
• Modify the existing exported functions
• Write the new ones

and compile again:

npm run componentize

Update the project:

golem-cloud-cli component update --component $LST out/lst.wasm

Note that this did not update the existing worker, but new workers will use the new version.

Try it out (using a new worker name to use the updated version):

golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{connect}' --arg '"[email protected]"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{connect}' --arg '"[email protected]"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 1"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 3"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{insert}' --arg '{id: 2}' --arg '"item 1"' --arg '"item 2"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{get}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{connected-editors}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{poll}' --arg '{id: 1}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{poll}' --arg '{id: 2}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 4"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{poll}' --arg '{id: 1}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 5"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{poll}' --arg '{id: 1}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test2 --function 'demo:lst/api.{poll}' --arg '{id: 2}'

Phase 3

In this step we implement the archive functionality.

First let's create a new component, now using the Go language:

golem-cloud-cli new --lang go --package-name demo:archive archive

compile the initial version:

cd archive
make build

Write the archive component's WIT definition (prepared/phase-3/archive/wit/archive.wit) and regenerate the bindings:

make bindings

Then implement it (prepared/phase-3/archive/src/main.go) and compile again:

make build

Generate a stub for the archive component:

cd ..
golem-cloud-cli stubgen build --source-wit-root archive/wit --dest-wasm archive-stub/archive-stub.wasm --dest-wit-root archive-stub/wit

And add the stub as a dependency to lst:

golem-cloud-cli stubgen add-stub-dependency --stub-wit-root archive-stub/wit --dest-wit-root lst/wit --overwrite

See how the wit/deps directory now contains demo-archive-stub. Modify the lst WIT definition to include the stub, and to export archive functionality (prepared/phase-3/lst/wit/main.wit).

Regenerate bindings for lst:

cd lst
npm run componentize

Implement the archive feature (prepared/phase-3/lst/src/main.ts):

• Add an archive flag to State
• Modify add, delete and insert to check it
• Implement archive and isArchived

Compile the lst component:

npm run componentize

Get back to the root and compose the lst.wasm with the archive-stub.wasm:

cd ..
golem-cloud-cli stubgen compose --source-wasm lst/out/lst.wasm --stub-wasm archive-stub/archive-stub.wasm --dest-wasm lst/out/lst-composed.wasm

Before trying it out, first upload the new archive component and save it's URN and ID:

golem-cloud-cli component add --project $PRJ --component-name archive archive/archive.wasm
export ARCHIVE=urn:component:c95c8c49-db39-4221-8721-f1f2b7e02a9d
export ARCHIVE_ID=c95c8c49-db39-4221-8721-f1f2b7e02a9d

Then update the list component with the new, composed version:

golem-cloud-cli component update --component $LST lst/out/lst-composed.wasm

And try it out! First we explicitly create a new list, passing the archive component's ID:

golem-cloud-cli worker start --component $LST --worker-name test3 --env "ARCHIVE_COMPONENT_ID=$ARCHIVE_ID"

Then invoke it a few times, then query if it's archived:

golem-cloud-cli worker invoke-and-await --component $LST --worker-name test3 --function 'demo:lst/api.{connect}' --arg '"[email protected]"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test3 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 1"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test3 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 3"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test3 --function 'demo:lst/api.{get}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test3 --function 'demo:lst/api.{is-archived}'

At this point the archive worker does not exist yet:

golem-cloud-cli worker list --component $ARCHIVE

Let's archive our list:

golem-cloud-cli worker invoke-and-await --component $LST --worker-name test3 --function 'demo:lst/api.{archive}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test3 --function 'demo:lst/api.{is-archived}'

And see the archive worker:

golem-cloud-cli worker list --component $ARCHIVE

Try to query it:

golem-cloud-cli worker invoke-and-await --component $ARCHIVE --worker-name archive --function 'demo:archive/api.{get-all}'

Phase 4

In the last step we implement the email sending functionality.

First we create a new component, this time using rust:

golem-cli new --lang rust --package-name demo:email email
cd email
cargo component build --release

Before we implement the email component, we are going to need to expose some functions from lst to be called from email.

Add the email-query interface to lst's WIT, then run

cd lst
npm run componentize

Then implement the two new functions in main.ts and compile it.

• Add the new emailQuery global
• Add emailDeadline and emailRecipients to State
• Add the updateEmailProperties helper function
• Call it from add, insert, delete, connect and disconnect

See if it compiles:

npm run componentize

Now we can implement the email component.

First we have to generate a stub for lst, so it can be called from email:

cd ..
golem-cloud-cli stubgen build --source-wit-root lst/wit --dest-wasm lst-stub/lst-stub.wasm --dest-wit-root lst-stub/wit

Then add lst as a dependency of email:

golem-cloud-cli stubgen add-stub-dependency --stub-wit-root lst-stub/wit --dest-wit-root email/wit --overwrite --update-cargo-toml

Try to build the email component:

cd email
cargo component build --release

It still compiles. See that wit/deps now contains demo_lst-stub and import it into email/wit/email.wit like we did with the archive stub before, then add it's API and regenerate the bindings:

cargo component build --release

Now it fails so implement it (prepared/phase-4/email/src/lib.rs) and build

cargo component build --release

Compose the result with the lst component's stub:

cd ..
golem-cloud-cli stubgen compose --source-wasm email/target/wasm32-wasi/release/email.wasm --stub-wasm lst-stub/lst-stub.wasm --dest-wasm email/target/wasm32-wasi/release/email-composed.wasm

At this point we have an email component but nobody calls it. We want to call it from the lst component whenever a new list is created. So we first need to generate a stub for email, so it can be called from lst:

golem-cloud-cli stubgen build --source-wit-root email/wit --dest-wasm email-stub/email-stub.wasm --dest-wit-root email-stub/wit

Then add email as a dependency of lst:

golem-cloud-cli stubgen add-stub-dependency --stub-wit-root email-stub/wit --dest-wit-root lst/wit --overwrite

Import the stub-email interface in lst/wit/main.wit and regenerate the bindings:

cd lst
npm run componentize

Because we don't have a better place to spawn the email component, we create an ensureInitialized method on State and call it from each exported function.

• Add the initialized, name, emailComponentId fields
• Add the method
• Call it from each exported function

Compile it

npm run componentize

Then compose it with both the archive and the email stubs:

cd ..
golem-cloud-cli stubgen compose --source-wasm lst/out/lst.wasm --stub-wasm archive-stub/archive-stub.wasm --dest-wasm lst/out/lst-composed1.wasm
golem-cloud-cli stubgen compose --source-wasm lst/out/lst-composed1.wasm --stub-wasm email-stub/email-stub.wasm --dest-wasm lst/out/lst-composed.wasm

Before trying it out, first we upload the email component to the cloud:

golem-cloud-cli component add --project $PRJ --component-name email email/target/wasm32-wasi/release/email-composed.wasm
export EMAIL_ID=59163ee3-95a2-4e35-b660-9feaee1e2163

Then we update the lst component with the composed WASM:

golem-cloud-cli component update --component $LST lst/out/lst-composed.wasm

Create a new list, now passing the email component id too:

golem-cloud-cli worker start --component $LST --worker-name test4 --env "ARCHIVE_COMPONENT_ID=$ARCHIVE_ID" --env "EMAIL_COMPONENT_ID=$EMAIL_ID"

Edit the list:

golem-cloud-cli worker invoke-and-await --component $LST --worker-name test4 --function 'demo:lst/api.{connect}' --arg '"[email protected]"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test4 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 1"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test4 --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 3"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test4 --function 'demo:lst/api.{get}'

See if it spawned the email worker:

golem-cloud-cli worker list --component urn:component:$EMAIL_ID

It shows it's Suspended, because it's sleeping until the deadline is reached.

We can also check it's logs

golem-cloud-cli worker connect --component urn:component:$EMAIL_ID --worker-name test4

End.

Demo Script for 1.1

Create a new project

golem-cloud-cli project add --project-name 'Golem 1.1 Demo'

Save it's urn:

export PRJ=urn:project:629fa7d7-84e5-4215-ba5c-7fd383ae8b40

Upload the components created on the 1.0 launch:

golem-cloud-cli component add --project $PRJ --component-name email precompiled/email-composed.wasm
export EMAIL_ID=064e104e-be4f-4fb8-935b-c4c5a775eac1
golem-cloud-cli component add --project $PRJ --component-name lst precompiled/lst-composed.wasm
export LST=urn:component:5826760e-216b-4341-a59a-cf3c177743d1
golem-cloud-cli component add --project $PRJ --component-name archive precompiled/archive.wasm
export ARCHIVE_ID=ec9715ba-feb1-4444-84c5-9ba9f9182dbd

Create a new list:

golem-cloud-cli worker start --component $LST --worker-name test --env "ARCHIVE_COMPONENT_ID=$ARCHIVE_ID" --env "EMAIL_COMPONENT_ID=$EMAIL_ID"

Do some invocations:

golem-cloud-cli worker invoke-and-await --component $LST --worker-name test --function 'demo:lst/api.{connect}' --arg '"[email protected]"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 1"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test --function 'demo:lst/api.{add}' --arg '{id: 1}' --arg '"item 3"'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test --function 'demo:lst/api.{get}'
golem-cloud-cli worker invoke-and-await --component $LST --worker-name test --function 'demo:lst/api.{archive}'

Show how we can check the worker's oplog:

golem-cloud-cli worker oplog --component $LST --worker-name test

Try only listing entries related to calling exported functions:

golem-cloud-cli worker oplog --component $LST --worker-name test --query 'exported-function'

Explain that this is a lucid query and can filter on many things.

Look for RPC calls:

golem-cloud-cli worker oplog --component $LST --worker-name test --query 'rpc'

Check the other workers:

golem-cloud-cli worker oplog --component-name email --project $PRJ --worker-name test
golem-cloud-cli worker oplog --component-name archive --project $PRJ --worker-name archive

Show how the idempotency key can be used in the query to match specific internal calls.