Skip to content
This repository has been archived by the owner on Nov 29, 2024. It is now read-only.

Commit

Permalink
Merge pull request #584 from zapier/hydration_doc_update
Browse files Browse the repository at this point in the history
PLATSUP-1020_hydration
  • Loading branch information
marinahand authored Feb 26, 2024
2 parents fdd623f + 86c69ad commit 6b55714
Show file tree
Hide file tree
Showing 8 changed files with 133 additions and 150 deletions.
93 changes: 93 additions & 0 deletions docs/_build/advanced-features/hydration-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: Use hydration in Platform CLI
order: 5
layout: post-toc
redirect_from:
- /cli_tutorials/hydration
- /build/hydration
---

# Use hydration in Platform CLI

## What is dehydration & hydration?

The best answer to this lives in our [CLI docs](https://github.com/zapier/zapier-platform/blob/main/packages/cli/README.md#dehydration):

> Dehydration, and its counterpart hydration, is a tool that can lazily load data that might be otherwise expensive to retrieve aggressively.
From a developer's perspective, you only need to worry about dehydration—Zapier will cover the hydration side of things.

## When to use dehydration?

The two most common times you should use dehydration in a Zapier integration are when:

1. You need to retrieve extra information from an API (e.g. a resource's list endpoint only returns IDs, but content must be retrieved per ID)
2. You need to provide access to a file (or files)

## Why use dehydration?

The core reason is reducing load to your API in case #1 above, where Zapier could fetch a list of known IDs of resources every 1-15 minutes per Zap, instead of the full definition of each of those resources. Putting any secondary requests behind a dehydration pointer means the request is made only once, although a Zap might see the same records again and again based on its polling cycle.

Dehydration saves even more bandwidth with files. No polling trigger should return files without dehydration, because otherwise, your app will send that file to Zapier around 100-300 times per day. For file outputs, implementing dehydration means the file will only be accessed and downloaded when a later Zap step asks for it.

The second reason is time. Your integration gets [30 seconds to run its API calls and any additional code](https://platform.zapier.com/build/troubleshoot-trigger-timeouts#trigger-runs-in-a-zap) each time a Zap step runs before the step would time out. If you are running into that time limit, consider if work could be offloaded to dehydration and hydration.

## How to use dehydration?

Check out our [example "files" app](https://github.com/zapier/zapier-platform/tree/main/example-apps/files) for an example of file dehydration in action with a working Zapier demo integration. You can even initialize a Zapier app based on that repo by entering ``zapier init . --template=files`` in Terminal to see it in your local code editor.


## Hydration in action

Some key areas include `index.js`, `hydrators.js`, `triggers/newFile.js`, and `creates/uploadFile.js`.

When building your integration, you'll likely be retrieving file info from a remote server. Instead, this example integration hard codes file urls to demonstrate.

The `New File` Trigger returns those file urls. The method [`z.dehydrateFile`](https://github.com/zapier/zapier-platform/blob/master/packages/cli/README.md#zdehydratefilefunc-inputdata) is used to create a pointer to the `downloadFile` function. In order to pass those files to other apps in actions, we reference `hydrators.downloadFile`, our hydrating function given a file url.

If you look at the `hydrators.js` file, you can see the `downloadFile` function. `downloadFile` calls the method[`z.stashFile`](https://github.com/zapier/zapier-platform/blob/master/packages/cli/README.md#zstashfilebufferstringstream-knownlength-filename-contenttype) to return a URL file pointer.

All of these will work together to lazily fetch the trigger data only when needed, avoiding API calls during polling or for reuse.

The only Action for this app is to upload the file, given a `bundle.inputData.file`.

### Setup

First, install the sample Zapier app `zapier init . --template=files` and `zapier push` it to Zapier. If you've not worked with the CLI before, start by checking out the [tutorial](https://platform.zapier.com/quickstart/cli-tutorial).

![](https://cdn.zappy.app/c3e87ca1ba18ce915d23dedf055e61af.png)

![](https://cdn.zappy.app/914893706d089af76bb445b3d885908d.png)

Here's how the integration looks in [Zapier's developer dashboard](https://developer.zapier.com/). Add an optional icon to it if you like.

![](https://cdn.zappy.app/13df6356d05b5bb3e4533666bbfcc680.png)

Next, we'll want to add a Zap. Open the [Zap editor](https://zapier.com/editor), and select your integration's trigger.

![](https://cdn.zappy.app/50a4c0399eef2728b1f3e2b67f7fb916.png)

Select continue - you'll notice this app has no authentication, as the file urls are accessible without it. Select `Test trigger` to see the three sample urls pulled in and hydrated pointer for each.

![](https://cdn.zappy.app/c68b2539d72c6c72da2ba3c35c9cef8d.png)


Now let's add the `Upload File` action to the Zap. Normally, we wouldn't want a setup like this (trigger off of new file / create a new file), because it would result in a [Zap loop](https://help.zapier.com/hc/en-us/articles/8496232045453-Zap-is-stuck-in-a-loop). But this is just a test—and be sure to turn the Zap off shortly after it's turned on.

![](https://cdn.zappy.app/28ed4762db06d0dd95563a4480a8dc36.png)

![](https://cdn.zappy.app/dd56ad225973829fbdc9fa1bcec5a2da.png)

Above, you'll see the string that prompts Zapier to hydrate a file. When the Zap runner encounters a string like this, Zapier will call the defined hydration function with the proper arguments.

After selecting `Test step`, you will see three new requests show in the `Monitoring` [tab of your integration](https://platform.zapier.com/build/test-monitoring):

![](https://cdn.zappy.app/97152cd0012be0e7c35385a4f4b3f50a.png)

The POST at the top was from the upload itself. The GET requests retrieve the file from the pointer provided by the trigger.

Now the Zap is ready to be turned on!

![](https://cdn.zappy.app/81cac4d9b52b6a76194d5af91949bfef.png)

In this example app integration, the trigger will not run automatically due to the hard coded file urls used for illustrative purposes. Once you replace the `fileURLs` in the trigger `perform`, with a request to your API that returns the triggering file, you'll be able to test this out fully.
144 changes: 0 additions & 144 deletions docs/_build/advanced-features/hydration.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/_build/triggers/hook-trigger.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ If, for architectural reasons, your webhook will receive some data that shouldn'

For data sent to Zapier via REST Hook, most requests will be successful and return a 200 status code with some request-tracking data. This indicates that Zapier has accepted the data, but it is still possible for errors to occur within the Zap if the structure of the provided data is unexpected.

### Best practices when sending data to a Rest Hook trigger:
### Best practices when sending data to a REST Hook trigger

- Be mindful of Zapier's [rate limits](https://zapier.com/help/troubleshoot/behavior/rate-limits-and-throttling-in-zapier#step-4).
- If your app receives a 410 response, that webhook subscription is no longer active, and you should stop sending data to it.
Expand Down
31 changes: 31 additions & 0 deletions docs/_build/troubleshooting/error-non-object-array.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
title: Got a non-object result in the array, expected only objects
order: 12
layout: post-toc
redirect_from:
---

# Error: Got a non-object result in the array, expected only objects

## Error shown

When using a REST Hook trigger, the data returned by the perform must be an array.

If an API returns a non-object result within the array, or an array of arrays, the following error will show.

{% raw %}
`Got a non-object result in the array, expected only objects ( )`
{% endraw %}

The non-object result will be wrapped in the parentheses for the error message.

## Solution

If the data included in the webhook needs to be transformed, or includes multiple objects, you can add custom code to parse the response data in `bundle.cleanedRequest` within the Perform into an array of objects.

If your webhook already provides an array, remove the wrapping array that Zapier includes by default and simply return `bundle.cleanedRequest`.

![image](https://cdn.zappy.app/26459a11e1630fe8318c341bf598ab5a.png)



4 changes: 2 additions & 2 deletions docs/_build/troubleshooting/error-non-object.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: Error - Got a non-object result
title: Error - Got a non-object result, expected an object from create
order: 11
layout: post-toc
redirect_from:
---

# Error: Got a non-object result
# Error: Got a non-object result, expected an object from create

## Error shown

Expand Down
3 changes: 2 additions & 1 deletion docs/_manage/integration-changes/export-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Similarly, your version can be found there, and elsewhere throughout the Platfor

![](https://cdn.zappy.app/e31dd202b5e64bbcb13fc8b200518d17.png)

Using this example to create our project in the current directory our command would be:
Using this example, to create our project in the current directory our command would be:

`zapier convert 1234 . --version=1.0.0`

Expand All @@ -59,6 +59,7 @@ A couple of important notes before deploying:

- When you push the CLI project to the server it will create a _new version_ of your integration. If you haven't gotten familiar with how versions work you might take a moment and learn about those [here](https://platform.zapier.com/manage/versions).
- Take a look at the version number in your `package.json` file. When you created your project with the `convert` tool we automatically incremented the version you converted. You can change this to a different version number depending on your needs, but make sure a version with that number doesn't already exist in your integration. Run `zapier versions` from your project directory to see what's already been created.
- Integrations converted with the `zapier convert` command will include `convertedByCLIVersion` in the `package.json` for informational purposes.

When you're ready to deploy your CLI version run:

Expand Down
Loading

0 comments on commit 6b55714

Please sign in to comment.