Support for deal history

[caitlinmoorman]

Currently, the tap only includes the most recent version of deal properties in the versions data. This is not super useful, as it simply matches the data available in the deal entity itself. The Hubspot API supports a parameter to include the full history of a deal's properties, which enables the recreation of a deal pipeline over time.

From Hupspot's API docs
&includePropertyVersions=true | By default, you will only get data for the most recent version of a property in the "versions" data. If you include this parameter, you will get data for all previous versions.

tap-hubspot/tap_hubspot/__init__.py

Line 494 in d168b50

def sync_deals(STATE, ctx):

[wss-chadical]

doh! I was just going to fire up the hubspot integration on Stitch Data Loader.... to specifically query deal property history via bigquery.

Darn it.

@caitlinmoorman Are you using Stitch Data Loader or are you running these taps on your own infrastructure?

[caitlinmoorman]

@wss-chadical I use Stitch.

Depending on what you need, you can still accomplish some history queries via deal properties - deals should contain fields for when the deal entered stage X and entered stage Y. However, you can't track the history of all deal properties (for example, no visibility into historical deal amount).

[wss-chadical]

@caitlinmoorman lol. Specifically date entering a specific stage is what we want.
That's good to know. I'll fire up the stitch for now as date entered stages is the ask.

What's your target?

[wss-chadical]

The core is that we want to be able to query the history of any property... so yeah.... &includePropertyVersions=True.

Sort of frustrating over the last 2 years using stitch primarily for Hubspot.

1. Still no support for new hubspot objects like Tickets, Quotes, Products
2. This tap still cannot handle when objects are deleted from Hubspot. (I know this is a hubspot issue - but vendors such as stitch do have some pull.)

[caitlinmoorman]

What's your target?
We're on Redshift.

One warning on using the date entering various stages from the deals object - I just discovered one of our deals isn't populating any of the entered_x_stage columns for some reason, so it's invisible in the reconstructed deal history. That makes this issue slightly more pressing for me, so maybe I can find some time to play around with it in the next couple of weeks - shouldn't be too hard to fix.

[wss-chadical]

@caitlinmoorman
So, looking at the code...
They are using the get_all_deals endpoint: https://developers.hubspot.com/docs/methods/deals/get-all-deals

Read this thread as well: https://community.hubspot.com/t5/APIs-Integrations/Deals-Endpoint-Returning-414/td-p/320123

Stitch is now using the undocumented allPropertiesFetchMode=latest_version parameter per hubspot's recommendation.

[wss-chadical]

@caitlinmoorman To get what you (we) need... the allPropertiesFetchMode needs to be set to 'all_versions'.

I've confirmed this works via postman api call.