Site Search Terms

[metmarkosaric]

"It would be very helpful to be able to see the search terms users are using on your websites built in search. This is particularly helpful on ecommerce sites. I am talking about the search terms people use on my site. Not the ones they used in google to get to my site. For example, the search bar on an ecommerce site that people use to located a product.

It’d be interesting to connect this with the existing goals feature. E.g. allow the goals to have arguments."

Update: We'd like to support this out of the box in the future. In the meanwhile you might be able to use the new feature we released to get search terms too. See https://plausible.io/docs/custom-query-params

And the latest version of our WordPress plugin introduces site search tracking. You can see the popularity of different search terms and the number of search results for the individual terms. Details at: https://plausible.io/wordpress-analytics-plugin

[starchildluke]

+1 for this feature.

[neuromusic]

+1

I'd love to have this on https://www.napari-hub.org/ We briefly considered adding search terms as custom data, but did not pursue this b/c we don't want someone to accidentally paste PII or other sensitive info in the search box.

It would be 💯 if plausible could offer a special search event and aggregate on the dashboard something akin to Algolia's Out-of-the-box Search Analytics, with some guardrails to prevent sensitive data from bleeding through

• Overall search counts
• A “No search result” rate
• Top searches
• Top searches with no results
• Top results
• (all of the above broken out for other traffic segmentations)

[theseyi]

I think Site Search Metrics is an important feature to have. This is fundamental for many applications that have search as a primary user interface. I'm currently evaluation a web tracking tool for use in my startup's app, and this is the main limitation I see with Plausible. While it is possible to implement this myself, this isn't my core business and would be nice to have this prebuilt like others including Matomo, Algolia as mentioned above and others

[masteradhoc]

+1 for the feature. As this is a must-have for one of our clients we now have to work with matomo instead plausible. Would love to see this soon and switch 👍🏻

[ukutaht]

I'm curious, what does your client use for site search (e.g. Angolia, Wordpress plugin?) and how does it integrate with Matomo? Does matomo have a bunch of plugins to collect analytics from different search implementations?

[metmarkosaric]

I don't know how it works for all the different frameworks but for WordPress the behavior is simple and consistent for all the WordPress blogs no matter what they use. When a user searches, a ?s=search+keyword gets added to the URL so perhaps analytics just take it from there? The example here from my own WordPress blog after I search for the word analytics: https://markosaric.com/?s=analytics. If I search for google analytics it looks like this: https://markosaric.com/?s=google+analytics

[masteradhoc]

@ukutaht @metmarkosaric see Matomo Feature Set here: https://matomo.org/docs/site-search/#enable-site-search-tracking-for-your-website

We most often use the general user search, which works as Marko statet. Still we sometimes have to add custom parameters - but that depends on the site and multifunctional search. If the general ?s= would work already that would be amazing!

[ukutaht]

@masteradhoc Hey could you share what stats/metrics/reports are important to you? Do you care mostly about top search terms in general or something more advanced?

[masteradhoc]

@ukutaht thanks for the question. Love to elaborate a bit more on what i see could be useful for a lot of our projects. What we need is to
capture all search queries that happen on our site. In a statistical overview it would be great to use regular filters like date period & location of the user. Maybe also the option to exclude searches that are from logged in users / administrators.

As a second step i'd love to see where the users went based on the search query. But you see - im happy with the basic implementation already :)

[metmarkosaric]

We'd like to support this out of the box in the future. In the meanwhile you might be able to use the new feature we released to get search terms too. See https://plausible.io/docs/custom-query-params

[neuromusic]

ooooh this is cool. though I'm a bit hesitant to use this for search terms without guardrails (like only showing the top terms), in case a user accidentally posts their name and social security number in the search.

[metmarkosaric]

i see what you mean. not sure how or whether matomo or ga deal with this potential problem

[neuromusic]

yeah, I'm not sure how they handle it... and to be clear, my biggest concern is with the public dashboard. the solution there could be as simple as only showing terms that were searched for more than some threshold (1-10?), and/or only showing high-level summaries like the ones I mentioned here: #164 (comment)

[mlissner]

I think if this let queries through to a public dashboard, we'd have to choose between one and the other. I think we'd choose the public dashboard and disable search term logging.

I suppose it's worth mentioning that in 2006, AOL had an early privacy scandal related to search queries: https://en.wikipedia.org/wiki/AOL_search_log_release. In their case, they thought it'd be fine and they released the data for research purposes. Oops. Sounds a bit like assuming some kind of threshold would be good enough. It might be. Or not. :)

[ukutaht]

Many people have asked if there's a way to hide some data from the public dashboard but keep it in the private one. Sounds like a good use-case for this.

In general we tend to say that it's the websites' responsibility to make sure they don't send PII into our systems. But search terms are clearly different and we can still have sensible defaults like keeping Search Terms always private.

[ukutaht]

Hey all, we are thinking about tackling this one soon. I'd like to get everyone's feedback on the plan.

I've divided site search terms in two phases based on the readiness of our backend. Phase one can be completed with relatively little changes to the backend, whereas phase 2 relies on some improvements we need to make to the database schema and the UI. For now, we are just considering building phase 1. More advanced stuff is not really on the short term horizon at the moment.

Phase 1
This is very basic and really only gives you one interesting insight: top search keywords ranked by the number of searches. It can be enabled with a simple script extension:

<script src="https://plausible.io/js/plausible.search.js"></script>

When this script extension is loaded, and the page has a special query parameter defined, then a custom event is automatically triggered. The event is called Search with a prop keyword which is populated from the query parameter. By default ?s= and ?q= are used but this is possible to override as follows:

<script data-search-param="q,query" src="https://plausible.io/js/plausible.search.js"></script>

In the UI, it would be a custom event Search or something like that.

Phase 2
The next level of site search analytics will also include result tracking which makes it a lot more powerful. This will have to be based completely on custom events since we cannot infer the result set from the query parameters.

I guess the the canonical example would be Algolia/Typesense. Here's how you can hook into their search results: https://www.algolia.com/doc/api-reference/widgets/analytics/js/#plug-with-kissmetrics

Being able to track results opens up much more advanced insights like 'Top keywords with no results'. Or just 'Top results'. You could invert the question and ask 'Which search terms lead to this result most often?'. Even more advanced would be tracking clicks on results. This allows things like 'average result click position' etc. The video on this link is a wonderful example of really advanced analytics that can be done on search in Algolia.

This would probably require a specialized UI for Search Analytics.

Unfortunately doing these things is not currently possible for the same reason as this: #929. We definitely have to remove this limitation in the future but it's only needed when we want to go to phase 2 of search analytics.

[SemaphoreOxalis]

For now we have setup a small script for multiple clients. The main issue was about the CMS search form wich heavily use js and stop propagation command or use ajax. It works nicely.

[rubzie]

Any updates on this feature? Would love it, and to fully implement custom query parameters is quite a hassle. #929 is fixed and just phase 1 would be a nice simple but useful addition! :)

[holloway]

Any update?

[shornuk]

Looking for this as well. We use Algolia, but would be useful to see standard search terms as these are not tracked.

[JohnSharkey]

Would be amazing if this could be added. We use WordPress Divi on our sites and the results page is just [siteURL]/?s=search+term where S is the search parameter and "search+term" is the term searched for. No idea how to set this up with custom parameters as there is no set search page, just the detault site URL with the parameters.

[metmarkosaric]

Update: Latest version of our WordPress plugin introduces site search tracking. You can see the popularity of different search terms and the number of search results for the individual terms. Details at: https://plausible.io/wordpress-analytics-plugin

[JohnSharkey]

Hi Marko This is fantastic! I've been using the WP Search Insights plugin to get this data. That plugin also shows what page the search was made on, which is pretty important to have, see below: [image: image.png] Is there any way to see this in Plausible? I can't seem to find it... Kind regards, John

…

On Mon, Jul 22, 2024 at 6:05 AM Marko Saric ***@***.***> wrote: *Update:* Latest version of our WordPress plugin introduces site search tracking. You can see the popularity of different search terms and the number of search results for the individual terms. Details at: https://plausible.io/wordpress-analytics-plugin — Reply to this email directly, view it on GitHub <#164 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZD3SCASTHVYPBMCXQTK5W3ZNSHJNAVCNFSM44JP3W42U5DIOJSWCZC7NNSXTOSENFZWG5LTONUW63SDN5WW2ZLOOQ5TCMBRGEYDMMRT> . You are receiving this because you commented.Message ID: ***@***.***>

-- John Sharkey Clear Moon Studio <https://www.clearmoonstudio.com/> 01252 703605 07715 985840 <https://clearmoonstudio.com/>

[metmarkosaric]

good idea @JohnSharkey! this is something we will consider adding in the future release. thanks!

[jdevalk]

Hi @JohnSharkey i can't see the screenshot; what are you looking for exactly?

[jdevalk]

@metmarkosaric I think treating searches as a goal is a good intermediate step, but it's not what I'd call a good resolution. Searches are a part of site navigation, they're not the goal of a website.

[JohnSharkey]

For each search term, knowing the page the search was made. Kind regards,JohnOn 22 Jul 2024, at 3:37 PM, Joost de Valk ***@***.***> wrote: Hi @JohnSharkey i can't see the screenshot; what are you looking for exactly? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>