From 86c69ad2a49068e9882bf5d2cd81047e00f0740b Mon Sep 17 00:00:00 2001 From: marinahand Date: Mon, 26 Feb 2024 12:57:24 -0700 Subject: [PATCH] hydration doc simplify --- .../_build/advanced-features/hydration-cli.md | 93 +++++++++++ docs/_build/advanced-features/hydration.md | 144 ------------------ docs/_build/triggers/hook-trigger.md | 2 +- .../troubleshooting/error-non-object-array.md | 31 ++++ .../troubleshooting/error-non-object.md | 4 +- .../_manage/integration-changes/export-cli.md | 3 +- .../integration-publishing-requirements.md | 2 +- docs/_publish/launch-process/user-help.md | 4 +- 8 files changed, 133 insertions(+), 150 deletions(-) create mode 100644 docs/_build/advanced-features/hydration-cli.md delete mode 100644 docs/_build/advanced-features/hydration.md create mode 100644 docs/_build/troubleshooting/error-non-object-array.md diff --git a/docs/_build/advanced-features/hydration-cli.md b/docs/_build/advanced-features/hydration-cli.md new file mode 100644 index 00000000..1ecd5152 --- /dev/null +++ b/docs/_build/advanced-features/hydration-cli.md @@ -0,0 +1,93 @@ +--- +title: Use hydration in Platform CLI +order: 5 +layout: post-toc +redirect_from: + - /cli_tutorials/hydration + - /build/hydration +--- + +# Use hydration in Platform CLI + +## What is dehydration & hydration? + +The best answer to this lives in our [CLI docs](https://github.com/zapier/zapier-platform/blob/main/packages/cli/README.md#dehydration): + +> Dehydration, and its counterpart hydration, is a tool that can lazily load data that might be otherwise expensive to retrieve aggressively. + +From a developer's perspective, you only need to worry about dehydration—Zapier will cover the hydration side of things. + +## When to use dehydration? + +The two most common times you should use dehydration in a Zapier integration are when: + +1. You need to retrieve extra information from an API (e.g. a resource's list endpoint only returns IDs, but content must be retrieved per ID) +2. You need to provide access to a file (or files) + +## Why use dehydration? + +The core reason is reducing load to your API in case #1 above, where Zapier could fetch a list of known IDs of resources every 1-15 minutes per Zap, instead of the full definition of each of those resources. Putting any secondary requests behind a dehydration pointer means the request is made only once, although a Zap might see the same records again and again based on its polling cycle. + +Dehydration saves even more bandwidth with files. No polling trigger should return files without dehydration, because otherwise, your app will send that file to Zapier around 100-300 times per day. For file outputs, implementing dehydration means the file will only be accessed and downloaded when a later Zap step asks for it. + +The second reason is time. Your integration gets [30 seconds to run its API calls and any additional code](https://platform.zapier.com/build/troubleshoot-trigger-timeouts#trigger-runs-in-a-zap) each time a Zap step runs before the step would time out. If you are running into that time limit, consider if work could be offloaded to dehydration and hydration. + +## How to use dehydration? + +Check out our [example "files" app](https://github.com/zapier/zapier-platform/tree/main/example-apps/files) for an example of file dehydration in action with a working Zapier demo integration. You can even initialize a Zapier app based on that repo by entering ``zapier init . --template=files`` in Terminal to see it in your local code editor. + + +## Hydration in action + +Some key areas include `index.js`, `hydrators.js`, `triggers/newFile.js`, and `creates/uploadFile.js`. + +When building your integration, you'll likely be retrieving file info from a remote server. Instead, this example integration hard codes file urls to demonstrate. + +The `New File` Trigger returns those file urls. The method [`z.dehydrateFile`](https://github.com/zapier/zapier-platform/blob/master/packages/cli/README.md#zdehydratefilefunc-inputdata) is used to create a pointer to the `downloadFile` function. In order to pass those files to other apps in actions, we reference `hydrators.downloadFile`, our hydrating function given a file url. + +If you look at the `hydrators.js` file, you can see the `downloadFile` function. `downloadFile` calls the method[`z.stashFile`](https://github.com/zapier/zapier-platform/blob/master/packages/cli/README.md#zstashfilebufferstringstream-knownlength-filename-contenttype) to return a URL file pointer. + +All of these will work together to lazily fetch the trigger data only when needed, avoiding API calls during polling or for reuse. + +The only Action for this app is to upload the file, given a `bundle.inputData.file`. + +### Setup + +First, install the sample Zapier app `zapier init . --template=files` and `zapier push` it to Zapier. If you've not worked with the CLI before, start by checking out the [tutorial](https://platform.zapier.com/quickstart/cli-tutorial). + +![](https://cdn.zappy.app/c3e87ca1ba18ce915d23dedf055e61af.png) + +![](https://cdn.zappy.app/914893706d089af76bb445b3d885908d.png) + +Here's how the integration looks in [Zapier's developer dashboard](https://developer.zapier.com/). Add an optional icon to it if you like. + +![](https://cdn.zappy.app/13df6356d05b5bb3e4533666bbfcc680.png) + +Next, we'll want to add a Zap. Open the [Zap editor](https://zapier.com/editor), and select your integration's trigger. + +![](https://cdn.zappy.app/50a4c0399eef2728b1f3e2b67f7fb916.png) + +Select continue - you'll notice this app has no authentication, as the file urls are accessible without it. Select `Test trigger` to see the three sample urls pulled in and hydrated pointer for each. + +![](https://cdn.zappy.app/c68b2539d72c6c72da2ba3c35c9cef8d.png) + + +Now let's add the `Upload File` action to the Zap. Normally, we wouldn't want a setup like this (trigger off of new file / create a new file), because it would result in a [Zap loop](https://help.zapier.com/hc/en-us/articles/8496232045453-Zap-is-stuck-in-a-loop). But this is just a test—and be sure to turn the Zap off shortly after it's turned on. + +![](https://cdn.zappy.app/28ed4762db06d0dd95563a4480a8dc36.png) + +![](https://cdn.zappy.app/dd56ad225973829fbdc9fa1bcec5a2da.png) + +Above, you'll see the string that prompts Zapier to hydrate a file. When the Zap runner encounters a string like this, Zapier will call the defined hydration function with the proper arguments. + +After selecting `Test step`, you will see three new requests show in the `Monitoring` [tab of your integration](https://platform.zapier.com/build/test-monitoring): + +![](https://cdn.zappy.app/97152cd0012be0e7c35385a4f4b3f50a.png) + +The POST at the top was from the upload itself. The GET requests retrieve the file from the pointer provided by the trigger. + +Now the Zap is ready to be turned on! + +![](https://cdn.zappy.app/81cac4d9b52b6a76194d5af91949bfef.png) + +In this example app integration, the trigger will not run automatically due to the hard coded file urls used for illustrative purposes. Once you replace the `fileURLs` in the trigger `perform`, with a request to your API that returns the triggering file, you'll be able to test this out fully. diff --git a/docs/_build/advanced-features/hydration.md b/docs/_build/advanced-features/hydration.md deleted file mode 100644 index b0238a8b..00000000 --- a/docs/_build/advanced-features/hydration.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -title: Use hydration in Platform CLI -order: 5 -layout: post-toc -redirect_from: /cli_tutorials/hydration ---- - -# Use hydration in Platform CLI - -## What is Dehydration & Hydration? - -The best answer to this lives on our [CLI docs](https://platform.zapier.com/reference/cli-docs#dehydration): - -> Dehydration, and its counterpart Hydration, is a tool that can lazily load data that might be otherwise expensive to retrieve aggressively. - -From a developer's perspective, you only need to worry about dehydration—Zapier will cover the hydration side of things. - -## When Should I Use Dehydration? - -The two most common times you should use dehydration in a Zapier integration are when: - -1. You need to retrieve extra information from an API (e.g. a resource's list endpoint only returns IDs, but content must be retrieved per ID) -2. You need to provide access to a file (or files) - -## _Why_ Should I Use Dehydration? - -Fair question! The core reason is that dehydration could save you bandwidth in case #1 above, where Zapier could fetch a list of known IDs of resources every 5-15 minutes, instead of the full definition of each of those resources. Dehydration saves even more bandwidth with files. In general, no polling trigger should return files without dehydration, because otherwise, your app will send that file to Zapier around 100-300 times per day. - -The second reason is time. Your app's integration gets 30 seconds to run its API calls and any additional code each time a Zap step runs before Zapier pulls the plug. If you are running into that time limit, consider if work could be offloaded to dehydration and hydration. - -## How Do I use Dehydration? - -Check out our [example "files" app](https://github.com/zapier/zapier-platform/tree/master/example-apps/files) for an example of file dehydration in action with a working Zapier demo integration. You can even initialize a Zapier app based on that repo by entering ``zapier init . --template=files`` in Terminal. - -*** - -# Example Setup (optional) - -## The Server - -We've got a very simple file server set up with Express in this [Github repository](https://github.com/codebycaleb/express-files-example). The whole thing fits in one (hopefully) easy-to-read file, `index.js`. Let's take a quick look at what the API offers: - -* GET `/files` - list filenames for the directory (only filenames!) -* GET `/files/:name` - download the file with the given filename or return a 404 if it's not found -* GET `/files/:name/meta` - return file meta info (size, creation date, etc.) -* POST `/files` - upload a file - -The README in that repo offers installation instructions (the server can be up and running in 4 commands). - -## The App - -What would an example be without an example app? Check out [https://github.com/codebycaleb/zapier-hydration-example/](https://github.com/codebycaleb/zapier-hydration-example/) to see what is available to get started with. Some key areas include `index.js`, `hydrators.js`, `triggers/newFile.js`, and `creates/uploadFile.js`. - -The sole Trigger for this app will check our server's `/files` endpoint every time the Zap runs. As you may recall, the `/files` endpoint only returns filenames, but we want to offer the extra meta info (and the file itself!) to Zaps so that they can be passed to other apps in Actions. To achieve this, we reference `hydrators.fileMeta`, our function for hydrating metadata given a filename. - -The only Action for this app is to upload the file. This is nearly identical to the Action in the example files app, with the one difference being that we dehydrate the file in the response so it is available to future actions via `hydrators.downloadFile`. This wouldn't normally be necessary in a file upload action (after all, the file was already available to be uploaded initially, so it could be used again in a later step as well), but it's been added in for illustrative reasons. - -One final area to look at in this app is the hydrators defined in `hydrators.js`. If you look at this file, you can see our two methods (`downloadFile` and `fileMeta`). One thing that might not be necessarily common is that `fileMeta` also calls `downloadFile` - this is for illustrative purposes too, to show it's possible. Whenever `fileMeta` is called (in the Trigger), it will create a pointer to the meta data and call `downloadFile`, which will create a pointer to the file itself. - -`downloadFile` calls the method[`z.stashFile`](https://github.com/zapier/zapier-platform/blob/master/packages/cli/README.md#zstashfilebufferstringstream-knownlength-filename-contenttype) to return a URL file pointer. `fileMeta` calls the method `[z.dehydrateFile](https://github.com/zapier/zapier-platform/blob/master/packages/cli/README.md#zdehydratefilefunc-inputdata)` to create a pointer to the `downloadFile` function. The `New File` Trigger calls the method `[z.dehydrate](https://github.com/zapier/zapier-platform/blob/master/packages/cli/README.md#zdehydratefunc-inputdata)` to create a pointer to the `fileMeta` function. All of these will work together to lazily fetch data when needed! - -# Hydration in Action - -## Getting Started - -First, install the sample Zapier app and push it to Zapier. - -![](https://zappy.zapier.com/D1FDB653-A9B3-4C78-9739-B36E263FC396.png) - -Here's how the integration looks in [Zapier's developer dashboard](https://zapier.com/app/developer/) after adding your app icon to it: - -![](https://zappy.zapier.com/E834B7C3-46ED-48A8-B70C-D7AEA45BC432.png) - -Back in Terminal, you also need to install and start up the Express app. - -![](https://zappy.zapier.com/E494B908-65CE-4E54-B296-D745BAE5B983.png) - -Finally, I've started up ngrok to forward traffic to my computer on the port the Express app is running on (`ngrok http 3000`). You could actually host the Express server if you want, but I'm going with ngrok for ease of use: - -![](https://zappy.zapier.com/ABCEB99C-E530-454E-9C7E-4EB6BBF943EB.png) - -I'm also going to take a tip from the output above and open the Web Interface ([http://127.0.0.1:4040](http://127.0.0.1:4040/)): - -![](https://zappy.zapier.com/B7CB465F-EA89-4EFB-B175-CAA252D852CD.png) - -Finally, I'm ready to add a Zap. Open the [Zap creator](https://zapier.com/app/editor/), and select your integration's trigger. - -![](https://zappy.zapier.com/C0EC57F8-2011-4897-8DB4-2EE1745DF66D.png) - -Then add your ngrok URL to the authentication screen. - -![](https://zappy.zapier.com/B04C5076-40E6-4827-BFDF-826CD40E500F.png) - -After connecting an account, you'll see the first requests come in via ngrok: - -![](https://zappy.zapier.com/F9D1245F-901D-4341-94F3-3CA2BF8765A5.png) - -Now, add a file to your `/files` directory. I'll do a small image (<1MB). Then go back to Zapier, and have it pull in sample data for the Zap: - -![](https://zappy.zapier.com/C9834ECA-BED4-4487-B4A2-38AED0FF0FB0.png) - -And now we've got more requests! There's two new ones: the `/files` endpoint was hit, as was the meta endpoint for the one file in there: - -![](https://zappy.zapier.com/3CF58AEC-C7FD-4458-AB4E-88F6A2D462EA.png) - -![](https://zappy.zapier.com/0E456BE7-16FC-4C66-8019-156AB909FFD0.png) - -Exciting! Let's add the Upload File Action to the Zap. Normally, we wouldn't want a setup like this (trigger off of new file / create a new file), because it would result in a Zap loop! But this is just a test—and be sure to turn the Zap off shortly after it's turned on. - -![](https://zappy.zapier.com/0D6B2551-EAFA-4124-B148-66545B2C9DF3.png) - -![](https://zappy.zapier.com/042E4AC8-D00E-4821-AC0A-BDDF96327478.png) - -Above, you'll see the string that prompts Zapier to hydrate a file. When the Zap runner encounters a string like this, Zapier will call the defined hydration function with the proper arguments. - -After pressing the “Send Test” button, three new requests will show in ngrok: - -![](https://zappy.zapier.com/2A33D13E-D405-4034-B938-2AEC88148743.png) - -The POST at the top was from the upload itself. The GET `/files` under it was an authentication check that actually came in before the GET to download the file itself (ngrok isn't great at sorting same-second events; it doesn't keep track of milliseconds, so same-second requests aren't really sorted). Whenever performing hydration, Zapier will *always* run an authentication test. - -Now the Zap is ready to be turned on! - -![](https://zappy.zapier.com/C1C88FF3-4D66-4099-BC6D-CE16FCB8671B.png) - -(one minor note from above - I've added a “Delay For” step to introduce a delay between the Zap Triggering and the Action taking place; you'll see why in the near future!) - -After turning the Zap on, a new request to `/files` will come in to set up the deduplication baseline (i.e. anything returned in here shouldn't trigger the Zap if it's seen on a future poll). At this point, I'm going to clear my ngrok requests log for a clear view of what happens during Zap execution. I'm also going to add a new photo to the `/files` directory. And now... we wait! - -## A Zap at Work - -My Zap has fired! Here's a look: - -![](https://zappy.zapier.com/3EDC0260-0465-42D3-99D2-95674C4CA388.png) - -When the Trigger runs, it detects a new file, prompting the `fileMeta` function to be executed. This is a key difference between `z.dehydrate` and `z.dehydrateFile` – as soon as the Trigger ran, we hydrated the request for `fileMeta`; however, we won't hydrate the stored `downloadFile` until the moment it is needed. This is why I added the "Delay For" step previously - to highlight that we won't hydrate dehydrated information until needed. - -A short while later, ngrok receives three new requests: - -![](https://zappy.zapier.com/28C6F8FD-677C-4EA0-837B-E4FBBD36C776.png) - -![](https://zappy.zapier.com/0F46EF11-B50F-4EE1-A8C7-FDC82A743407.png) - -Now is the time that the `downloadFile` function was evaluated, and the file was downloaded (in order to be uploaded again!). One final point that you may notice is that `downloadFile` was only called once; we did not call it again after the upload, despite adding a dehydrated call to it. That's what's great about `z.dehydrateFile`, we only use it when necessary! diff --git a/docs/_build/triggers/hook-trigger.md b/docs/_build/triggers/hook-trigger.md index f6837f84..2f0a7b29 100644 --- a/docs/_build/triggers/hook-trigger.md +++ b/docs/_build/triggers/hook-trigger.md @@ -98,7 +98,7 @@ If, for architectural reasons, your webhook will receive some data that shouldn' For data sent to Zapier via REST Hook, most requests will be successful and return a 200 status code with some request-tracking data. This indicates that Zapier has accepted the data, but it is still possible for errors to occur within the Zap if the structure of the provided data is unexpected. -### Best practices when sending data to a Rest Hook trigger: +### Best practices when sending data to a REST Hook trigger - Be mindful of Zapier's [rate limits](https://zapier.com/help/troubleshoot/behavior/rate-limits-and-throttling-in-zapier#step-4). - If your app receives a 410 response, that webhook subscription is no longer active, and you should stop sending data to it. diff --git a/docs/_build/troubleshooting/error-non-object-array.md b/docs/_build/troubleshooting/error-non-object-array.md new file mode 100644 index 00000000..af1ec74e --- /dev/null +++ b/docs/_build/troubleshooting/error-non-object-array.md @@ -0,0 +1,31 @@ +--- +title: Got a non-object result in the array, expected only objects +order: 12 +layout: post-toc +redirect_from: +--- + +# Error: Got a non-object result in the array, expected only objects + +## Error shown + +When using a REST Hook trigger, the data returned by the perform must be an array. + +If an API returns a non-object result within the array, or an array of arrays, the following error will show. + +{% raw %} +`Got a non-object result in the array, expected only objects ( )` + {% endraw %} + +The non-object result will be wrapped in the parentheses for the error message. + +## Solution + +If the data included in the webhook needs to be transformed, or includes multiple objects, you can add custom code to parse the response data in `bundle.cleanedRequest` within the Perform into an array of objects. + +If your webhook already provides an array, remove the wrapping array that Zapier includes by default and simply return `bundle.cleanedRequest`. + +![image](https://cdn.zappy.app/26459a11e1630fe8318c341bf598ab5a.png) + + + \ No newline at end of file diff --git a/docs/_build/troubleshooting/error-non-object.md b/docs/_build/troubleshooting/error-non-object.md index 2627c2c5..66f1e562 100644 --- a/docs/_build/troubleshooting/error-non-object.md +++ b/docs/_build/troubleshooting/error-non-object.md @@ -1,11 +1,11 @@ --- -title: Error - Got a non-object result +title: Error - Got a non-object result, expected an object from create order: 11 layout: post-toc redirect_from: --- -# Error: Got a non-object result +# Error: Got a non-object result, expected an object from create ## Error shown diff --git a/docs/_manage/integration-changes/export-cli.md b/docs/_manage/integration-changes/export-cli.md index 01d7edbb..9a97f4df 100644 --- a/docs/_manage/integration-changes/export-cli.md +++ b/docs/_manage/integration-changes/export-cli.md @@ -41,7 +41,7 @@ Similarly, your version can be found there, and elsewhere throughout the Platfor ![](https://cdn.zappy.app/e31dd202b5e64bbcb13fc8b200518d17.png) -Using this example to create our project in the current directory our command would be: +Using this example, to create our project in the current directory our command would be: `zapier convert 1234 . --version=1.0.0` @@ -59,6 +59,7 @@ A couple of important notes before deploying: - When you push the CLI project to the server it will create a _new version_ of your integration. If you haven't gotten familiar with how versions work you might take a moment and learn about those [here](https://platform.zapier.com/manage/versions). - Take a look at the version number in your `package.json` file. When you created your project with the `convert` tool we automatically incremented the version you converted. You can change this to a different version number depending on your needs, but make sure a version with that number doesn't already exist in your integration. Run `zapier versions` from your project directory to see what's already been created. +- Integrations converted with the `zapier convert` command will include `convertedByCLIVersion` in the `package.json` for informational purposes. When you're ready to deploy your CLI version run: diff --git a/docs/_publish/launch-process/integration-publishing-requirements.md b/docs/_publish/launch-process/integration-publishing-requirements.md index 439fad76..7910401e 100644 --- a/docs/_publish/launch-process/integration-publishing-requirements.md +++ b/docs/_publish/launch-process/integration-publishing-requirements.md @@ -115,7 +115,7 @@ Do not add adjectives or phrases to your app name, and only include your company The logo must be a square, transparent PNG at least 256x256px in size. Please use a bigger version if you have one available. If your app has a solid, square background, round the corners 3% of the width and set the background as transparent. If your icon is not square, make a square transparent image and center your icon inside the transparent square. Do not include the app name or other copy in the logo as it will not be legible at small sizes. ### 2.7 Description -Write a short description (up to maximum of 140 characters) of your app’s core features and use cases in the form of ` is a...`. The copy should not include links or mentions of Zapier. Do not use flowery or overstated language, or make make it appear your integration is associated with or endorsed by Zapier. +Write a short description (up to maximum of 140 characters) of your app’s core features and use cases in the form of ` is a...`. The copy should not include links or mentions of Zapier. Do not use flowery or overstated language, or make it appear your integration is associated with or endorsed by Zapier. **Example**: diff --git a/docs/_publish/launch-process/user-help.md b/docs/_publish/launch-process/user-help.md index 8e7c2f57..8f6570d6 100644 --- a/docs/_publish/launch-process/user-help.md +++ b/docs/_publish/launch-process/user-help.md @@ -15,4 +15,6 @@ Optionally include links out to documentation on your app's site to help custome Once submitted, it takes roughly 2 weeks to get help docs published in the Help Center. -As you maintain and add functionality to your integration, use the same form to submit updates at any time to the Technical Writing team. \ No newline at end of file +As you maintain and add functionality to your integration, use the same form to submit updates at any time to the Technical Writing team. + +When adding Zapier logos on any help documentation or pages in your app, access the up-to-date branding assets [here](https://brand.zapier.com/). \ No newline at end of file