Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for the UBI plugin #7284

Merged
merged 117 commits into from
Jul 3, 2024
Merged
Show file tree
Hide file tree
Changes from 113 commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
29dabb9
gathering potential documentation attempts
RasonJ May 15, 2024
b2cce5b
considering the dashboard tutorial
RasonJ May 16, 2024
f5c509b
place holder for js data structure usage
RasonJ May 16, 2024
cbd8730
data-structures placeholder
RasonJ May 16, 2024
ff37482
Updating index links
RasonJ May 20, 2024
721fc81
adding old doc to be merged
RasonJ May 20, 2024
c98ca98
Starting to link things together
RasonJ May 22, 2024
80df271
fix broken link
RasonJ May 22, 2024
9ed7ea8
respond to vale
epugh May 22, 2024
ecfdac3
more vale violations
epugh May 22, 2024
b2e4923
name files consistently with docs site and fix links.
epugh May 22, 2024
9dc6e6b
vale
epugh May 22, 2024
0454234
Minor tweaks. Moved Ubi under SEARCH.
RasonJ May 22, 2024
14c2ee3
add label for versining of spec and OS version
epugh May 23, 2024
9917445
try to sort out vale error
epugh May 23, 2024
c453226
Converting mermaid diagrams to png's
RasonJ May 28, 2024
c0aa738
Updating query_id mermaid code
RasonJ May 28, 2024
95cb54c
Better way to ignore the mermaid scripts in the md files
RasonJ May 28, 2024
a8cd6a5
description updates
RasonJ May 29, 2024
0646b49
schema.md updating (still need to update the mermaid diagram)
RasonJ May 30, 2024
0fd39a0
schema updates
RasonJ May 30, 2024
566b8fc
updates
RasonJ May 31, 2024
ba4544d
Rebuilding main
RasonJ Jun 13, 2024
a2e3983
merging in images
RasonJ Jun 13, 2024
5a6e3b9
Updating UBI spec number
RasonJ Jun 14, 2024
9dee721
Use released version
epugh Jun 14, 2024
6d80d7f
Update _search-plugins/index.md
RasonJ Jun 19, 2024
d8cce37
Update _search-plugins/ubi/index.md
RasonJ Jun 19, 2024
70bf5c7
Update _search-plugins/ubi/index.md
RasonJ Jun 19, 2024
c1b953b
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 19, 2024
d8a02f2
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 19, 2024
89ec18b
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 19, 2024
cba4818
Addressing PR feedback
RasonJ Jun 19, 2024
1b03536
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 19, 2024
3d13bd3
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 19, 2024
1aca2fa
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 19, 2024
5fe1795
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 19, 2024
3cb7982
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 19, 2024
22748bb
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 19, 2024
eddb010
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 19, 2024
2928b83
Update _search-plugins/ubi/data-structures.md
RasonJ Jun 19, 2024
dad3c68
Update _search-plugins/ubi/data-structures.md
RasonJ Jun 19, 2024
bcb1d2f
Update _search-plugins/ubi/data-structures.md
RasonJ Jun 19, 2024
51a9cd6
Update _search-plugins/ubi/data-structures.md
RasonJ Jun 19, 2024
d4e64c7
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 19, 2024
63656a2
Adding dsl intro
RasonJ Jun 19, 2024
a3db1a5
Adding intro sentence
RasonJ Jun 19, 2024
1805c83
Title adjust
RasonJ Jun 19, 2024
299945f
Apply suggestions from code review
RasonJ Jun 19, 2024
923d568
Apply suggestions from code review
RasonJ Jun 19, 2024
4f8914b
Apply suggestions from code review
RasonJ Jun 19, 2024
402ab55
Apply suggestions from code review
RasonJ Jun 19, 2024
2a8c4f3
Apply suggestions from code review
RasonJ Jun 19, 2024
a995efa
Apply suggestions from code review
RasonJ Jun 19, 2024
75545c0
Apply suggestions from code review
RasonJ Jun 19, 2024
ab882f3
Addressing PR feedback
RasonJ Jun 19, 2024
7c1ab79
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 19, 2024
95878f0
Addressing pr feedback
RasonJ Jun 20, 2024
da22463
Addressing PR feedback
RasonJ Jun 20, 2024
7d70e10
Fixing vale errors
RasonJ Jun 20, 2024
35db038
Merge branch 'main' into main
epugh Jun 24, 2024
0f81289
Apply suggestions from code review
RasonJ Jun 24, 2024
c7809cb
Finishing initial pr feedback
RasonJ Jun 24, 2024
81b57d9
Next round of PR feedback
RasonJ Jun 25, 2024
ad9e9f4
Describing chorus workbench link
RasonJ Jun 25, 2024
78688e1
Adding captions for result tables
RasonJ Jun 27, 2024
4a99b1e
PR clean up
RasonJ Jun 27, 2024
93c235e
Apply suggestions from code review
RasonJ Jun 28, 2024
8c095d8
Adding a few more suggestions
RasonJ Jun 28, 2024
a95e43a
Apply suggestions from code review
RasonJ Jun 28, 2024
8346880
Apply suggestions from code review
RasonJ Jun 28, 2024
c201540
Apply suggestions from code review
RasonJ Jun 28, 2024
c2f2321
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
464fdd3
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
0527dd0
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
ee6e96c
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
90bc7ca
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
1c411b6
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
196329f
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
f5edb99
updating query filter for laptos
RasonJ Jun 28, 2024
0e32bda
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
c442e77
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
eeef786
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
0494381
PR feedback
RasonJ Jun 28, 2024
c51dcd0
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 28, 2024
ca45773
Update _search-plugins/ubi/ubi-dashboard-tutorial.md
RasonJ Jun 28, 2024
8b4dd1e
PR feedback
RasonJ Jun 28, 2024
321b98f
Merge branch 'main' of https://github.com/o19s/documentation-website
RasonJ Jun 28, 2024
6fe346c
Apply suggestions from code review
RasonJ Jun 28, 2024
a2f8133
Apply suggestions from code review
hdhalter Jun 28, 2024
2b27446
Apply suggestions from code review
hdhalter Jun 28, 2024
d4453d9
Update _search-plugins/ubi/sql-queries.md
hdhalter Jun 28, 2024
ecbf504
Apply suggestions from code review
hdhalter Jun 28, 2024
357ec1a
Apply suggestions from code review
hdhalter Jun 28, 2024
4978463
Update _search-plugins/ubi/dsl-queries.md
hdhalter Jun 28, 2024
bea46a9
Update _search-plugins/ubi/dsl-queries.md
hdhalter Jun 28, 2024
fbf213c
Apply suggestions from code review
hdhalter Jul 2, 2024
b9ad117
Update index.md
hdhalter Jul 2, 2024
4db066b
Update _search-plugins/ubi/dsl-queries.md
hdhalter Jul 2, 2024
f07ffc6
Apply suggestions from code review
hdhalter Jul 2, 2024
2a72b8c
Update index.md
hdhalter Jul 2, 2024
48c242f
Update schemas.md
hdhalter Jul 2, 2024
c03847d
Merge branch 'main' into main
hdhalter Jul 2, 2024
586086c
Update index.md
hdhalter Jul 2, 2024
282a1ba
Update index.md
hdhalter Jul 2, 2024
b142604
Update ubi-dashboard-tutorial.md
hdhalter Jul 2, 2024
d58d0ec
Update schemas.md
hdhalter Jul 2, 2024
d3e806a
Update sql-queries.md
hdhalter Jul 2, 2024
fe44e77
Apply suggestions from code review
hdhalter Jul 2, 2024
88f4813
Update _search-plugins/ubi/sql-queries.md
hdhalter Jul 2, 2024
9cbe990
Update _search-plugins/ubi/sql-queries.md
hdhalter Jul 2, 2024
510912b
Apply suggestions from code review
hdhalter Jul 3, 2024
d2a7de0
Update _search-plugins/ubi/schemas.md
kolchfa-aws Jul 3, 2024
3a6ec38
Apply suggestions from code review
kolchfa-aws Jul 3, 2024
409dcc0
Update schemas.md
hdhalter Jul 3, 2024
fed4d79
Update sql-queries.md
hdhalter Jul 3, 2024
b8d9bf6
Update sql-queries.md
hdhalter Jul 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/vale/styles/Vocab/OpenSearch/Plugins/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,5 @@ Search Relevance plugin
Security plugin
Security Analytics plugin
SQL plugin
Trace Analytics plugin
Trace Analytics plugin
User Behavior Insights
1 change: 1 addition & 0 deletions .ruby-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.3.2
2 changes: 2 additions & 0 deletions _search-plugins/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ OpenSearch provides the following search relevance features:

- [Querqy]({{site.url}}{{site.baseurl}}/search-plugins/querqy/): Offers query rewriting capability.

- [User Behavior Insights]({{site.url}}{{site.baseurl}}/search-plugins/ubi/): Links user behavior to user queries to improve search quality.

## Search results

OpenSearch supports the following commonly used operations on search results:
Expand Down
204 changes: 204 additions & 0 deletions _search-plugins/ubi/data-structures.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
---
layout: default
title: UBI client data structures
parent: User Behavior Insights
has_children: false
nav_order: 10
---

# UBI client data structures

Data structures are used to create events that follow the [User Behavior Insights (UBI) event schema specification](https://github.com/o19s/ubi).
For more information about the schema, see [UBI index schemas]({{site.url}}{{site.baseurl}}/search-plugins/ubi/schemas/).


You must provide an implementation for the following functions:
- `getClientId()`
- `getQueryId()`

You can also optionally provide an implementation for the following functions:
- `getSessionId()`
- `getPageId()`


RasonJ marked this conversation as resolved.
Show resolved Hide resolved
The following JavaScript structures can be used as a starter implementation to serialize UBI events into schema-compatible JSON:
```js
/*********************************************************************************************
* Ubi Event data structures
* The following structures help ensure adherence to the UBI event schema
*********************************************************************************************/



export class UbiEventData {
constructor(object_type, id=null, description=null, details=null) {
this.object_id_field = object_type;
this.object_id = id;
this.description = description;
this.object_detail = details;
}
}
export class UbiPosition{
constructor({ordinal=null, x=null, y=null, trail=null}={}) {
this.ordinal = ordinal;
this.x = x;
this.y = y;
if(trail)
this.trail = trail;
else {
const trail = getTrail();
if(trail && trail.length > 0)
this.trail = trail;
}
}
}


export class UbiEventAttributes {
/**
* Tries to prepopulate common event attributes
* The developer can add an `object` that the user interacted with and
* the site `position` information relevant to the event
*
* Attributes, other than `object` or `position` can be added in the form:
* attributes['item1'] = 1
* attributes['item2'] = '2'
*
* @param {*} attributes: object with general event attributes
* @param {*} object: the data object the user interacted with
* @param {*} position: the site position information
*/
constructor({attributes={}, object=null, position=null}={}) {
if(attributes != null){
Object.assign(this, attributes);
}
if(object != null && Object.keys(object).length > 0){
this.object = object;
}
if(position != null && Object.keys(position).length > 0){
this.position = position;
}
this.setDefaultValues();
}

setDefaultValues(){
try{
if(!this.hasOwnProperty('dwell_time') && typeof TimeMe !== 'undefined'){
this.dwell_time = TimeMe.getTimeOnPageInSeconds(window.location.pathname);
}

if(!this.hasOwnProperty('browser')){
this.browser = window.navigator.userAgent;
}

if(!this.hasOwnProperty('page_id')){
this.page_id = window.location.pathname;
}
if(!this.hasOwnProperty('session_id')){
this.session_id = getSessionId();
}

if(!this.hasOwnProperty('page_id')){
this.page_id = getPageId();
}

if(!this.hasOwnProperty('position') || this.position == null){
const trail = getTrail();
if(trail.length > 0){
this.position = new UbiPosition({trail:trail});
}
}
// ToDo: set IP
}
catch(error){
console.log(error);
}
}
}



export class UbiEvent {
constructor(action_name, {message_type='INFO', message=null, event_attributes={}, data_object={}}={}) {
this.action_name = action_name;
this.client_id = getClientId();
this.query_id = getQueryId();
this.timestamp = Date.now();

this.message_type = message_type;
if( message )
this.message = message;

this.event_attributes = new UbiEventAttributes({attributes:event_attributes, object:data_object});
}

/**
* Use to suppress null objects in the json output
* @param key
* @param value
* @returns
*/
static replacer(key, value){
if(value == null ||
(value.constructor == Object && Object.keys(value).length === 0)) {
return undefined;
}
return value;
}

/**
*
* @returns json string
*/
toJson() {
return JSON.stringify(this, UbiEvent.replacer);
}
}
```
{% include copy.html %}

# Sample usage

```js
export function logUbiMessage(event_type, message_type, message){
let e = new UbiEvent(event_type, {
message_type:message_type,
message:message
});
logEvent(e);
}

export function logDwellTime(action_name, page, seconds){
console.log(`${page} => ${seconds}`);
let e = new UbiEvent(action_name, {
message:`On page ${page} for ${seconds} seconds`,
event_attributes:{
session_id: getSessionId()},
dwell_seconds:seconds
},
data_object:TimeMe
});
logEvent(e);
}

/**
* ordinal is the number within a list of results
* for the item that was clicked
*/
export function logItemClick(item, ordinal){
let e = new UbiEvent('item_click', {
message:`Item ${item['object_id']} was clicked`,
event_attributes:{session_id: getSessionId()},
data_object:item,
});
e.event_attributes.position.ordinal = ordinal;
logEvent(e);
}

export function logEvent( event ){
// some configured http client
return client.index( index = 'ubi_events', body = event.toJson());
}

```
{% include copy.html %}
101 changes: 101 additions & 0 deletions _search-plugins/ubi/dsl-queries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
layout: default
title: Example UBI query DSL queries
parent: User Behavior Insights
has_children: false
nav_order: 15
---

# Example UBI query DSL queries

hdhalter marked this conversation as resolved.
Show resolved Hide resolved
You can use the OpenSearch search query language, [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/), to write User Behavior Insights (UBI) queries. The following example returns the number of times that each `action_name` event occurs.
For more extensive analytic queries, see [Example UBI SQL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/sql-queries/).
#### Example request
```json
GET ubi_events/_search
{
"size":0,
"aggs":{
"event_types":{
"terms": {
"field":"action_name",
"size":10
}
}
}
}
```
{% include copy.html %}

#### Example response

```json
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": []
},
"aggregations": {
"event_types": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "brand_filter",
"doc_count": 3084
},
{
"key": "product_hover",
"doc_count": 3068
},
{
"key": "button_click",
"doc_count": 3054
},
{
"key": "product_sort",
"doc_count": 3012
},
{
"key": "on_search",
"doc_count": 3010
},
{
"key": "type_filter",
"doc_count": 2925
},
{
"key": "login",
"doc_count": 2433
},
{
"key": "logout",
"doc_count": 1447
},
{
"key": "new_user_entry",
"doc_count": 207
}
]
}
}
}
```
{% include copy.html %}

You can run the preceding queries in the OpenSearch Dashboards [Query Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/).

A demo workbench with sample data can be found here:
[http://chorus-opensearch-edition.dev.o19s.com:5601/app/OpenSearch-query-workbench](http://chorus-OpenSearch-edition.dev.o19s.com:5601/app/OpenSearch-query-workbench).
50 changes: 50 additions & 0 deletions _search-plugins/ubi/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
layout: default
title: User Behavior Insights
has_children: true
nav_order: 90
redirect_from:
- /search-plugins/ubi/
---
# User Behavior Insights

**Introduced 2.15**
{: .label .label-purple }

**References UBI Specification 1.0.0**
{: .label .label-purple }

User Behavior Insights (UBI) is a plugin that captures client-side events and queries for the purposes of improving search relevance and the user experience.
It is a causal system, linking a user's query to all of their subsequent interactions with your application until they perform another search.

RasonJ marked this conversation as resolved.
Show resolved Hide resolved
UBI includes the following elements:
* A machine-readable [schema](https://github.com/o19s/ubi) that faciliates interoperablity of the UBI specification.
* An OpenSearch [plugin](https://github.com/opensearch-project/user-behavior-insights) that facilitates the storage of client-side events and queries.
* A client-side JavaScript [example reference implementation]({{site.url}}{{site.baseurl}}/search-plugins/ubi/data-structures/) that shows how to capture events and send them to the OpenSearch UBI plugin.

<!-- vale off -->

RasonJ marked this conversation as resolved.
Show resolved Hide resolved
The UBI documentation is organized into two categories: *Explanation and reference* and *Tutorials and how-to guides*:

*Explanation and reference*

| Link | Description |
| :--------- | :------- |
| [UBI Request/Response Specification](https://github.com/o19s/ubi/) | The industry-standard schema for UBI requests and responses. The current version references UBI Specification 1.0.0. |
| [UBI index schema]({{site.url}}{{site.baseurl}}/search-plugins/ubi/schemas/) | Documentation on the individual OpenSearch query and event stores. |


*Tutorials and how-to guides*

natebower marked this conversation as resolved.
Show resolved Hide resolved
| Link | Description |
| :--------- | :------- |
| [UBI plugin](https://github.com/opensearch-project/user-behavior-insights) | How to install and use the UBI plugin. |
| [UBI client data structures]({{site.url}}{{site.baseurl}}/search-plugins/ubi/data-structures/) | Sample JavaScript structures for populating the event store. |
| [Example UBI query DSL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/dsl-queries/) | How to write queries for UBI data in OpenSearch query DSL. |
| [Example UBI SQL queries]({{site.url}}{{site.baseurl}}/search-plugins/ubi/sql-queries/) | How to write analytic queries for UBI data in SQL. |
| [UBI dashboard tutorial]({{site.url}}{{site.baseurl}}/search-plugins/ubi/ubi-dashboard-tutorial/) | How to build a dashboard containing UBI data. |
| [Chorus Opensearch Edition](https://github.com/o19s/chorus-opensearch-edition/?tab=readme-ov-file#structured-learning-using-chorus-opensearch-edition) katas | A series of structured tutorials that teach you how to use UBI with OpenSearch through a demo e-commerce store. |

<!-- vale on -->
The documentation categories were adapted using concepts based on [Diátaxis](https://diataxis.fr/).
{: .tip }
Loading
Loading