From 98284ef7ef95f5675b040ac49eabfaebe1701132 Mon Sep 17 00:00:00 2001 From: Oli Evans Date: Wed, 17 Jan 2024 12:02:04 +0000 Subject: [PATCH] feat: `w3 delegation create --base64` & `w3 space add ` (#158) **export** a delegation as base64 encoded identity CID with `w3 delegation create --base64` ```shell $ w3 delegation create did:key:z6MkviAsUfBwegmB57byQ7SZTFtX4jNjo315EvgurjWYoTRX --can 'store/add' --can 'upload/add' --base64 mAYIEAO0OEaJlcm9vdHOAZ3ZlcnNpb24BvQUBcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y6oYXNYRO2hA0CbPDJlxyrorHHdNAUnRUDA4xU7KHgHHstkM8tBxq+6KaQP5xLCknOh9TjkR0S0yuK/fiFxKwRDUHfECFEWQn4DYXZlMC45LjFjYXR0hqJjY2FuZ3NwYWNlLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmdzdG9yZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5odXBsb2FkLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmhhY2Nlc3MvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuamZpbGVjb2luLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2Nhbmd1c2FnZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXY2F1ZFgi7QHob+19JDMUBs+u1e646vN2MLovQUXA7xJeFs2THUcb+mNleHAaZzctGmNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0BL9X+p4Uyz05zSH0ol8TYPXpwU9EljNRo1O18uYbWlvljcHJmgL0FAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMuqGFzWETtoQNAmzwyZccq6Kxx3TQFJ0VAwOMVOyh4Bx7LZDPLQcavuimkD+cSwpJzofU45EdEtMriv34hcSsEQ1B3xAhRFkJ+A2F2ZTAuOS4xY2F0dIaiY2NhbmdzcGFjZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5nc3RvcmUvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuaHVwbG9hZC8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5oYWNjZXNzLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmpmaWxlY29pbi8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5ndXNhZ2UvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjZXhwGmc3LRpjZmN0gaFlc3BhY2WhZG5hbWVldG9vdHNjaXNzWCLtAS/V/qeFMs9Oc0h9KJfE2D16cFPRJYzUaNTtfLmG1pb5Y3ByZoDbAwFxEiB9iHpD1ttdKEQCvBZ8jJBD7Wqw1abOtYwNCKAKYALXMqhhc1hE7aEDQJ7U8I+a4Au/eb10r9T89weG/Nl2jccEUXHs8wq+i2tU0Iaik8KaKvovDqqE57JU8ZoY0JAzOBW7cMLGcV6/UwthdmUwLjkuMWNhdHSComNjYW5pc3RvcmUvYWRkZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5qdXBsb2FkL2FkZGR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B8YzaLs8NDe7oZt6rlpsW6iMh8XsoXZLvkHPqtftYRXZjZXhw9mNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjcHJmgtgqWCUAAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMu2CpYJQABcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y4 ``` ...yes, what if we put the CAR _in_ the CID! That way we can detect when the input has been truncted, and the screed is self describing... you can paste it into [cid.ipfs.tech](https://cid.ipfs.tech/#mAYIEAO0OEaJlcm9vdHOAZ3ZlcnNpb24BvQUBcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y6oYXNYRO2hA0CbPDJlxyrorHHdNAUnRUDA4xU7KHgHHstkM8tBxq+6KaQP5xLCknOh9TjkR0S0yuK/fiFxKwRDUHfECFEWQn4DYXZlMC45LjFjYXR0hqJjY2FuZ3NwYWNlLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmdzdG9yZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5odXBsb2FkLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmhhY2Nlc3MvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuamZpbGVjb2luLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2Nhbmd1c2FnZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXY2F1ZFgi7QHob+19JDMUBs+u1e646vN2MLovQUXA7xJeFs2THUcb+mNleHAaZzctGmNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0BL9X+p4Uyz05zSH0ol8TYPXpwU9EljNRo1O18uYbWlvljcHJmgL0FAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMuqGFzWETtoQNAmzwyZccq6Kxx3TQFJ0VAwOMVOyh4Bx7LZDPLQcavuimkD+cSwpJzofU45EdEtMriv34hcSsEQ1B3xAhRFkJ+A2F2ZTAuOS4xY2F0dIaiY2NhbmdzcGFjZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5nc3RvcmUvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuaHVwbG9hZC8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5oYWNjZXNzLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmpmaWxlY29pbi8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5ndXNhZ2UvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjZXhwGmc3LRpjZmN0gaFlc3BhY2WhZG5hbWVldG9vdHNjaXNzWCLtAS/V/qeFMs9Oc0h9KJfE2D16cFPRJYzUaNTtfLmG1pb5Y3ByZoDbAwFxEiB9iHpD1ttdKEQCvBZ8jJBD7Wqw1abOtYwNCKAKYALXMqhhc1hE7aEDQJ7U8I+a4Au/eb10r9T89weG/Nl2jccEUXHs8wq+i2tU0Iaik8KaKvovDqqE57JU8ZoY0JAzOBW7cMLGcV6/UwthdmUwLjkuMWNhdHSComNjYW5pc3RvcmUvYWRkZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5qdXBsb2FkL2FkZGR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B8YzaLs8NDe7oZt6rlpsW6iMh8XsoXZLvkHPqtftYRXZjZXhw9mNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjcHJmgtgqWCUAAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMu2CpYJQABcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y4) and it'll tell you it's CAR flavour identity hashed bytes! **import** a space from a stringified proof ```shell $ w3 space add $PROOF did:key:z6MkhfzTwZJ28aRobCp76uXRqzsjH6GNu18WFwMumkAF5ojW ``` see: https://github.com/web3-storage/w3cli/issues/154#issuecomment-1885042715 ## Usage **on your machine** - with `@web3-storage/w3cli` installed. - Set the space you want CI to upload to as the current space ```shell # create a key for ci $ w3 key create --json > ci.json # create a proof for that key. copy paste it to env var in CI $ w3 delegation create $(jq -r .did ci.json) -c 'store/add' -c 'upload/add' --base64 mA... ``` **in CI** - install `@web3-storage/w3cli`. No need to login, we pass it the proof to use. - copy paste the key we generated above `(jq -r .key ci.json)` as a secret in your ci workflow. _KEEP IT SECRET!_ - copy paste the proof we generated above as a variable or secret in your ci workflow. _it's ok to share this, only the holder of the secret key can use it._ ```shell # set key (jq -r .key ci.json) in env so w3cli uses it instead of generating one $ W3_PRINCIPAL=${{ secrets.w3key }} # import the space from the stringified proof $ w3 space add ${{ vars.w3proof }} # upload yer stuff $ w3 upload ./my/cool/stuff ``` License: MIT --------- Signed-off-by: Oli Evans --- README.md | 19 +++++++------- bin.js | 8 ++++-- index.js | 54 ++++++++++++++++++++++++++++++++++---- lib.js | 13 +++++++--- test/bin.spec.js | 67 ++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 141 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 04f8417..9604f7b 100644 --- a/README.md +++ b/README.md @@ -112,9 +112,11 @@ w3 open bafybeidluj5ub7okodgg5v6l4x3nytpivvcouuxgzuioa6vodg3xt2uqle/olizilla.png Print information about the current agent. -### `w3 space add ` +### `w3 space add ` -Add a space to the agent. The proof is a CAR encoded delegation to _this_ agent. +Add a space to the agent. The proof is a CAR encoded UCAN delegating capabilities over a space to _this_ agent. + +`proof` is a filesystem path to a CAR encoded UCAN, as generated by `w3 delegation create` _or_ a base64 identity CID string as created by `w3 delegation create --base64`. ### `w3 space create [name]` @@ -144,17 +146,14 @@ Create a delegation to the passed audience for the given abilities with the _cur - `--name` Human readable name for the audience receiving the delegation. - `--type` Type of the audience receiving the delegation, one of: device, app, service. - `--output` Path of file to write the exported delegation data to. +- `--base64` Format as base64 identity CID string. Useful when saving it as an environment variable. ```bash -# delegate space/info to did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN -w3 delegation create did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN --can space/info - -# delegate store/* and upload/* to did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN -w3 delegation create did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN --can 'store/*' --can 'upload/*' +# delegate space/info to did:key:z6M..., output as a CAR +w3 delegation create did:key:z6M... --can space/info --output ./info.ucan -# delegate all capabilities to did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN -# WARNING - this is bad practice and should generally only be done in testing and development -w3 delegation create did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN --can '*' +# delegate store/* and upload/* to did:key:z6M..., output as a string +w3 delegation create did:key:z6M... --can 'store/*' --can 'upload/*' --base64 ``` ### `w3 delegation ls` diff --git a/bin.js b/bin.js index 6ed89d1..fca4116 100755 --- a/bin.js +++ b/bin.js @@ -142,7 +142,7 @@ cli cli .command('space add ') .describe( - 'Add a space to the agent. The proof is a CAR encoded delegation to _this_ agent.' + 'Import a space from a proof: a CAR encoded UCAN delegating capabilities to this agent. proof is a filesystem path, or a base64 encoded cid string.' ) .action(addSpace) @@ -181,7 +181,7 @@ cli cli .command('delegation create ') .describe( - 'Create a delegation to the passed audience for the given abilities with the _current_ space as the resource.' + 'Output a CAR encoded UCAN that delegates capabilities to the audience for the current space.' ) .option('-c, --can', 'One or more abilities to delegate.') .option( @@ -201,6 +201,10 @@ cli '-o, --output', 'Path of file to write the exported delegation data to.' ) + .option( + '--base64', + 'Format as base64 identity CID string. Useful when saving it as an environment variable.' + ) .action(createDelegation) cli diff --git a/index.js b/index.js index da259fb..2cd8633 100644 --- a/index.js +++ b/index.js @@ -1,7 +1,9 @@ import fs from 'fs' import ora, { oraPromise } from 'ora' -import { Readable } from 'stream' +import { pipeline } from 'node:stream/promises' import { CID } from 'multiformats/cid' +import { base64 } from 'multiformats/bases/base64' +import { identity } from 'multiformats/hashes/identity' import * as DID from '@ipld/dag-ucan/did' import * as dagJSON from '@ipld/dag-json' import { CarWriter } from '@ipld/car' @@ -15,6 +17,7 @@ import { filesize, filesizeMB, readProof, + readProofFromBytes, uploadListResponseToString, startOfLastMonth, } from './lib.js' @@ -253,11 +256,33 @@ export async function createSpace(name) { } /** - * @param {string} proofPath + * @param {string} proofPathOrCid */ -export async function addSpace(proofPath) { +export async function addSpace(proofPathOrCid) { const client = await getClient() - const delegation = await readProof(proofPath) + + let cid + try { + cid = CID.parse(proofPathOrCid, base64) + } catch (/** @type {any} */ err) { + if (err?.message?.includes('Unexpected end of data')) { + console.error(`Error: failed to read proof. The string has been truncated.`) + process.exit(1) + } + /* otherwise, try as path */ + } + + let delegation + if (cid) { + if (cid.multihash.code !== identity.code) { + console.error(`Error: failed to read proof. Must be identity CID. Fetching of remote proof CARs not supported by this command yet`) + process.exit(1) + } + delegation = await readProofFromBytes(cid.multihash.digest) + } else { + delegation = await readProof(proofPathOrCid) + } + const space = await client.addSpace(delegation) console.log(space.did()) } @@ -339,6 +364,7 @@ Providers: ${providers || chalk.dim('none')}`) * @param {number} [opts.expiration] * @param {string} [opts.output] * @param {string} [opts.with] + * @param {boolean} [opts.base64] */ export async function createDelegation(audienceDID, opts) { const client = await getClient() @@ -367,7 +393,25 @@ export async function createDelegation(audienceDID, opts) { const { writer, out } = CarWriter.create() const dest = opts.output ? fs.createWriteStream(opts.output) : process.stdout - Readable.from(out).pipe(dest) + pipeline( + out, + async function* maybeBaseEncode(src) { + const chunks = [] + for await (const chunk of src) { + if (!opts.base64) { + yield chunk + } else { + chunks.push(chunk) + } + } + if (!opts.base64) return + const blob = new Blob(chunks) + const bytes = new Uint8Array(await blob.arrayBuffer()) + const idCid = CID.createV1(ucanto.CAR.code, identity.digest(bytes)) + yield idCid.toString(base64) + }, + dest + ) for (const block of delegation.export()) { // @ts-expect-error diff --git a/lib.js b/lib.js index 49b5cda..24cc359 100644 --- a/lib.js +++ b/lib.js @@ -139,16 +139,24 @@ export function getClient() { * @param {string} path Path to the proof file. */ export async function readProof(path) { + let bytes try { - await fs.promises.access(path, fs.constants.R_OK) + const buff = await fs.promises.readFile(path) + bytes = new Uint8Array(buff.buffer) } catch (/** @type {any} */ err) { console.error(`Error: failed to read proof: ${err.message}`) process.exit(1) } + return readProofFromBytes(bytes) +} +/** + * @param {Uint8Array} bytes Path to the proof file. + */ +export async function readProofFromBytes(bytes) { const blocks = [] try { - const reader = await CarReader.fromIterable(fs.createReadStream(path)) + const reader = await CarReader.fromBytes(bytes) for await (const block of reader.blocks()) { blocks.push(block) } @@ -156,7 +164,6 @@ export async function readProof(path) { console.error(`Error: failed to parse proof: ${err.message}`) process.exit(1) } - try { // @ts-expect-error return importDAG(blocks) diff --git a/test/bin.spec.js b/test/bin.spec.js index e498849..9479109 100644 --- a/test/bin.spec.js +++ b/test/bin.spec.js @@ -18,6 +18,7 @@ import { UCAN, Provider } from '@web3-storage/capabilities' import * as ED25519 from '@ucanto/principal/ed25519' import { sha256, delegate } from '@ucanto/core' import * as Result from '@web3-storage/w3up-client/result' +import { base64 } from 'multiformats/bases/base64' const w3 = Command.create('./bin.js') @@ -349,6 +350,34 @@ export const testSpace = { assert.ok(listSome.output.includes(spaceDID)) }), + 'w3 space add `base64 proof car`': test(async (assert, context) => { + const { env } = context + const spaceDID = await loginAndCreateSpace(context, { env: env.alice }) + const whosBob = await w3.args(['whoami']).env(env.bob).join() + const bobDID = SpaceDID.from(whosBob.output.trim()) + const res = await w3 + .args([ + 'delegation', + 'create', + bobDID, + '-c', + 'store/*', + 'upload/*', + '--base64' + ]) + .env(env.alice) + .join() + + const listNone = await w3.args(['space', 'ls']).env(env.bob).join() + assert.ok(!listNone.output.includes(spaceDID)) + + const add = await w3.args(['space', 'add', res.output]).env(env.bob).join() + assert.equal(add.output.trim(), spaceDID) + + const listSome = await w3.args(['space', 'ls']).env(env.bob).join() + assert.ok(listSome.output.includes(spaceDID)) + }), + 'w3 space add invalid/path': test(async (assert, context) => { const fail = await w3 .args(['space', 'add', 'djcvbii']) @@ -784,6 +813,44 @@ export const testDelegation = { assert.equal(delegate.status.success(), true) }), + 'w3 delegation create -c store/add -c upload/add --base64': test( + async (assert, context) => { + const env = context.env.alice + const { bob } = Test + const spaceDID = await loginAndCreateSpace(context) + const res = await w3 + .args([ + 'delegation', + 'create', + bob.did(), + '-c', + 'store/add', + '-c', + 'upload/add', + '--base64' + ]) + .env(env) + .join() + + assert.equal(res.status.success(), true) + + const identityCid = parseLink(res.output, base64) + const reader = await CarReader.fromBytes(identityCid.multihash.digest) + const blocks = [] + for await (const block of reader.blocks()) { + blocks.push(block) + } + + // @ts-expect-error + const delegation = importDAG(blocks) + assert.equal(delegation.audience.did(), bob.did()) + assert.equal(delegation.capabilities[0].can, 'store/add') + assert.equal(delegation.capabilities[0].with, spaceDID) + assert.equal(delegation.capabilities[1].can, 'upload/add') + assert.equal(delegation.capabilities[1].with, spaceDID) + } + ), + 'w3 delegation ls --json': test(async (assert, context) => { const { mallory } = Test