More flexible aggregation of number event props

[btkostner]

What we are trying to accomplish

We are working on porting an eCommerce site from Google Analytics to Plausible. Right now we send the total order amount to Plausible, but it's hard to see the larger picture because most values have a super low count:

The things that would be super helpful here are:

• Grouping values and displaying a range (< 100, < 250, < 500, < 1000)
• Being able to see the total sum of the prop (The total amount of money we made from X campaign is $XXX)
• Seeing the avg of the prop (The average order created from X campaign is $XXX)

Having those values when we break down goals would allow us to track conversions and campaign sources much easier.

Initial ideas

Most of this can be done on the client side with no API changes. It just needs some design work to figure out the best way to display this.

The big thing I would say is allowing users to have a little say in the formatting. Automatically doing these things to all numbers would be a mess. I would probably add a second section under "Goals" site settings page that lets a user add props and select what type of value they are.

Update: We've now opened up a preview version of this to everyone. You can now assign monetary values to custom events and track the revenue attribution of your ecommerce store. Instructions at https://plausible.io/docs/ecommerce-revenue-tracking

[Vigasaurus]

Yeah this is something I've also been very interested in for my own stats; I might look into implementing it, soonish.

[ukutaht]

Agreed, this would be very useful. Related discussion: #256

I actually like better how this feature request is framed. A more general way to deal with numbers is needed and to run basic aggregations like sum and avg on an arbitrary property.

On top of this general feature, we can add special handling of currencies (like GA has) specifically to support monetary value tracking.

While it could be done on the client side I would prefer to implement this server-side so our API clients can make use of the same features. Also, if the field has a high cardinality (imagine 1m discrete values) then doing it on the client side is not very realistic. Meanwhile our DB can aggregate millions of data points in milliseconds.

Currently, all custom props are cast to strings in the database because Clickhouse arrays are statically typed and I had to choose one type for the field. Implementing this requires some schema work which will be interesting.

[Vigasaurus]

Yeah definitely agree that it being done on the Clickhouse side is the way to go.

My general idea (very similar to discussion starter) is to have a way to store the props for each goal (either of the most recently sent props of a goal, or a union of every prop ever sent to a goal (this could be problematic)) as part of the site settings in Postgres - and then in the goals settings a user can assign the Type (for now it'd just be String (default) or Number but that could expand to include currencies, times/dates, or even arrays/objects if it comes to it), as well as the aggregations to perform on each - for now probably would only offer sum, min, max, average, mode, median, and user-defined buckets (unless there are other ideas too). The benefit here is that the actual viewing UI can stay somewhat simple (it can already become hectic when there are a lot of meta values), and that the settings will be cleanly and evidently be shared for anyone viewing the site page, regardless of self-settings. If needed, the aggregations could even show up in the view pane with a Show/Hide Aggregations link.

Also, from what I can tell we wouldn't really need to change much in the CH schema, they can stay strings, CH offers very powerful type conversion (stuff like toInt64OrNull and more), as long as there's a consistent serialization of values to strings that is CH-friendly.

I might work on this in the coming weeks, will see.

[ukutaht]

Yeah, I can definitely see how the UI can get too busy. With goals we've seen that having to go through settings causes quite a bit of friction. My sense is that the vast majority would definitely prefer to 'fire and forget' with their goals and custom props. @metmarkosaric does more customer support, maybe you could share your view on this. Would it make it easier for users if they didn't have to go through settings to set up their goals?

Maybe we could just accept any custom prop type, store as that correct type on the backend, and when running aggregations we just ignore the prop values that are in the 'wrong' type for that aggregation?

My idea for the UI is to add dropdown arrows to labels. So you could choose Breakdown(url) or Sum(amount) or similar somewhere in the goals section of the dashboard from a dropdown. I feel like with some design work it could fit there but it's definitely easy to overcrowd the UI at this point... in that case I agree we should move it out of the dashboard.

With regard to DB schema, strings take a lot more space on disk than integers and also the aggregations would then need to do two passes over the whole dataset, once to convert the types and then to actually run the aggregations. So eventually as we scale I'm sure we will want to use the proper types in the database.

Having done a data migration for exit pages recently, I really want to avoid them in the future. So I would take the time to build a more future-proof schema to start with. Normally I try not to optimize things too prematurely because it's easy to change software but the CH schema is an exception. Getting it right the first time is super valuable.

[Vigasaurus]

Yeah I could totally see how having to hop into goal settings every time could be irksome. I feel like doing it in the UI wouldn't necessarily be that bad, we can even use some JS foolery to only show the menu when the values seem to be mostly numerical.

I feel like something like this (with multi-select in the first example, which is not necessary, but maybe kinda nice) could be appropriate - and maybe not that busy.

Or for a more simple setup (with more columns, probably):

Or something in between.

As for the CH side, from what I can gather the only really applicable datatype would be Nested, but there would need to be separate items for each data type in them. There is the Map but it's experimental and doesn't seem like exactly what this would need.

[metmarkosaric]

We don't have many complaints about goal settings at the moment but it would certainly help people such as WordPress plugin users if their 404 errors, external link clicks tracking and ecommerce conversion goals work out of the box with no manual work needed.

About this request:

Monetary values for goals and funnels are pretty much 2 main requests from ecommerce sites. The needs are pretty similar to this request. Here's my summary of what people look for in general:

Conversion rate
Number of conversions/transactions
Total revenue
Average order amount
Funnel steps from entry to conversion with focus on learning where people drop off
Number of abandoned carts
Total value of abandoned carts
All of the above in total but also per product
Filtering down of the dashboard for the total or the individual product so you can see referral sources

Not sure if there's a way to fit this in our existing goals UI or a new ecommerce goals report is more appropriate (something like the existing chart we have on the top of the dashboard: top line showing the 4-6 main metrics and the funnel steps below it - and an option to select total or per product)

[David-44Pixel]

Where did this land? Is it implemented or are there any plans for it?

I'm currently setting up plausible as tracking for our web-app.
We'd like to track video duration watched, and I imagine we will get a lot of different values, so summing and averages would be really useful.

[metmarkosaric]

there are no short term plans for this so we cannot make any promise on when it will be done. we are a small team with limited development resources so need to focus on one thing at a time. there are a few other more frequently requested features that are our priorities at the moment (cities/states, export/import...). but you've been heard and this is something we want to add so we'll update here as soon as there's some news. thanks for understanding

[metmarkosaric]

Update: We've now opened up a preview version of this to everyone. You can now assign monetary values to custom events and track the revenue attribution of your ecommerce store. Instructions at https://plausible.io/docs/ecommerce-revenue-tracking

[davidlim]

In the instructions for ecommerce revenue tracking, integrating with WooCommerce section, the code example provided shows properties for order Id and item count, but the screenshot below shows a breakdown by product. I'm interested in setting up the breakdown by product. Do you have a code example for that?

Thanks in advance.

[metmarkosaric]

Update: Latest version of our WordPress plugin introduces a WooCommerce integration for revenue attribution tracking. The WooCommerce integration automatically sets up several custom events (including add to cart and remove from cart), revenue tracking for completed purchases, custom properties (including product name, price and total cart items) and a checkout funnel to help you understand the cart abandonment rate. No need to manually set up anything nor to edit your code. Details at: https://plausible.io/wordpress-analytics-plugin