Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON: function "query" #157

Merged
merged 55 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
d83c0b6
Function "query"
ralfhandl Sep 27, 2021
1641c49
Added "value" function, defined processing rules
ralfhandl Sep 28, 2021
b41aad8
Recommended and optional expressions
ralfhandl Sep 29, 2021
5f153de
Aligned terminology with draft RFC
ralfhandl Sep 29, 2021
b1e1ea9
Examples
ralfhandl Oct 1, 2021
ec1ec9c
Long description of the vocabulary
ralfhandl Oct 1, 2021
778d44c
Update Org.OData.JSON.V1.Schema-sample.xml
ralfhandl Oct 1, 2021
d761c8d
Minor edits
ralfhandl Oct 1, 2021
5ede455
Update Org.OData.JSON.V1.xml
ralfhandl Oct 1, 2021
f9d2a37
Function "value" isn't composable
ralfhandl Oct 1, 2021
b23a8bd
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Oct 14, 2021
aa6cb39
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Oct 15, 2021
c372fbc
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Nov 10, 2021
a3b0ed4
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Dec 6, 2021
7f02bee
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Dec 13, 2021
f8523ba
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Dec 17, 2021
c5b853f
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Feb 3, 2022
c98a1e6
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Feb 10, 2022
82e83fc
Merge remote-tracking branch 'origin/main' into ODATA-1336/json-query
HeikoTheissen May 6, 2022
270b5b4
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Aug 31, 2022
806188c
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Sep 7, 2022
d1848a3
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Dec 20, 2022
a661ad0
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Jan 10, 2023
bd12070
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Mar 21, 2023
470a277
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Apr 27, 2023
ab4a20f
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Jul 4, 2023
8b62d83
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Sep 7, 2023
99fa5ee
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Sep 7, 2023
c5b8c7f
Use JSONPath draft 20
ralfhandl Sep 22, 2023
b83f15a
Updated URLs
ralfhandl Sep 22, 2023
d9c8a69
Additional JSONPath constructs are examples only, no restriction
ralfhandl Sep 22, 2023
1b52e7f
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Dec 1, 2023
3fa98e6
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Feb 29, 2024
6f9faac
Typo
ralfhandl Feb 29, 2024
873715f
Typo
ralfhandl Feb 29, 2024
bb8577f
JSONPath is now an RFC
ralfhandl Feb 29, 2024
5b389f2
Update links
ralfhandl Feb 29, 2024
f2c28dd
Align with RFC text
ralfhandl Feb 29, 2024
d3a7336
MUST for minimum functionality
ralfhandl Mar 7, 2024
e2ac582
JSON data instead of JSON stream
ralfhandl Mar 7, 2024
24d78ab
name=value syntax for function calls
ralfhandl Mar 14, 2024
6c8dc89
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Mar 14, 2024
d41ef8d
Update vocabularies/Org.OData.JSON.V1.xml
ralfhandl Mar 14, 2024
ac67e0b
Update Org.OData.JSON.V1.xml
ralfhandl Mar 14, 2024
e0c93ac
Heiko's comment
ralfhandl Mar 14, 2024
1b5c571
Return-type-specific value functions
HeikoTheissen Apr 4, 2024
269ac57
Example for valueNumber
HeikoTheissen Apr 4, 2024
882237a
Apply suggestions from code review
ralfhandl Apr 4, 2024
f581770
Rebuilt
ralfhandl Apr 4, 2024
b1276f0
Formatting
ralfhandl Apr 4, 2024
b61160b
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Apr 8, 2024
52288cf
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Apr 11, 2024
ab134d0
Merge branch 'main' into ODATA-1336/json-query
ralfhandl Apr 11, 2024
f232900
Revert unintended line breaks
ralfhandl Apr 12, 2024
38c4827
Reference example, don't copy it
HeikoTheissen Apr 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions vocabularies/Org.OData.JSON.V1.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,58 @@
"@Core.Description": "The JSON Schema for JSON values of the annotated property, parameter, return type, term, or type definition",
"@Core.LongDescription": "The schema can be a schema reference, i.e. `{\"$ref\":\"url/of/schemafile#/path/to/schema/within/schemafile\"}`"
},
"query": [
{
"$Kind": "Function",
"@Core.Description": "Query stream values of media type `application/json`, returning a stream value of media type `application/json`",
"@Core.LongDescription": "Extracts a JSON value, such as an array, object, or a JSON scalar value (string, number, boolean, or `null`) from the `input` JSON value:\n- If `path` only consists of child member and single subscript operators, it returns the identified single node within `input`, or `null` if no node is identified. \n- If `path` potentially identifies multiple nodes within `input` (by using recursive descendant, wildcard, array subset, or array filter expressions), it returns an array containing the identified nodes, or an empty array if no node is identified. \n- If `input` is not a valid JSON value, the function returns `null`.\n- If `path` is `null`, not a valid [JSONPath expression](#Path), or does not match the structure of `input`, the function returns `null`. \n ",
"$Parameter": [
{
"$Name": "input",
"$Type": "JSON.JSON",
"$Nullable": true,
"@Core.Description": "JSON input"
},
{
"$Name": "path",
"$Type": "JSON.Path",
"$Nullable": true,
"@Core.Description": "JSONPath expression to be applied to value of `expr`"
}
],
"$ReturnType": {
"$Type": "JSON.JSON",
"$Nullable": true,
"@Core.Description": "JSON value resulting from applying `path` to `input`"
}
}
],
"value": [
{
"$Kind": "Function",
"@Core.Description": "Query stream values of media type `application/json`, returning an OData primitive value",
"@Core.LongDescription": "Extracts an OData primitive value from the `input` JSON value:\n- If `path` only consists of child member and single subscript operators and identifies a single scalar JSON value (string, number, boolean, or `null`) within `input`, it returns the identified single value cast to an OData primitive value (see below).\n- If `path` identifies multiple nodes within `input` (by using recursive descendant, wildcard, array subset, or array filter expressions), identifies an object or array, or does not identify any node, the function returns `null`.\n- If `input` is not a valid JSON value, the function returns `null`.\n- If `path` is `null`, not a valid [JSONPath expression](#Path), or does not match the structure of `input`, the function returns `null`.\n\nIf a single non-null scalar JSON value is identified by `path` within `input`, the function returns that value as a primitive value of type\n- `Edm.String` if the value is a JSON string\n- `Edm.Boolean` if the value is `true` or `false`\n- `Edm.Decimal` with unspecified precision and floating scale if the value is a JSON number\n ",
"$Parameter": [
{
"$Name": "input",
"$Type": "JSON.JSON",
"$Nullable": true,
"@Core.Description": "JSON input"
},
{
"$Name": "path",
"$Type": "JSON.Path",
"$Nullable": true,
"@Core.Description": "JSONPath expression to be applied to value of `expr`"
}
],
"$ReturnType": {
"$Type": "Edm.PrimitiveType",
"$Nullable": true,
ralfhandl marked this conversation as resolved.
Show resolved Hide resolved
"@Core.Description": "OData primitive value resulting from applying `path` to `input`"
}
}
],
"JSON": {
"$Kind": "TypeDefinition",
"$UnderlyingType": "Edm.Stream",
Expand All @@ -49,6 +101,12 @@
"@Core.AcceptableMediaTypes": [
"application/json"
]
},
"Path": {
"$Kind": "TypeDefinition",
"$UnderlyingType": "Edm.String",
"@Core.Description": "[JSONPath](https://datatracker.ietf.org/doc/html/draft-ietf-jsonpath-base-01) expression",
"@Core.LongDescription": "Implementations SHOULD support at least the following subset of JSONPath:\n\n_TODO: check which data sources support which subset, then reduce expressions_\n- SQL Server: https://docs.microsoft.com/en-us/sql/relational-databases/json/json-path-expressions-sql-server?view=sql-server-2017\n- SAP HANA: https://help.sap.com/products/SAP_HANA_PLATFORM/4fe29514fd584807ac9f2a04f6754767/3126ea33d50d42d19517a08fe22ec5a1.html?version=2.0.05#description\n- Google BigQuery references the Java implementation - all of it supported?\n\nJSONPath | Description | Examples\n---------|-------------|--------\n`$` | The root object, array, or value\n`.` | Child member operator | `$.foo`, `$.foo.bar`\n`..` | Recursive descendant operator: searches for the specified member name recursively and returns an array of all values with this property name | `$..foo`\n`*` | Wildcard matching all elements in an object or array | `$.foo.*`, `$.bar[*]`\n`[]` | Subscript operator, accepting names (single-quoted strings) or array indices (zero-based integers, negative integers count from the end of the array) | `$['foo']`, `$.foo['bar']`, `$.bar[0]`, `$.bar[-1]`\n`[,]` | Union operator for alternate names or array indices as a set | `$.foo['bar','baz']`, `$.bar[0,1,2,3,5,7,11]`\n`[start:end]` | Array subset by range of indices | `$.bar[2:4]`\n`[start:]` | Array subset from index to end of array | `$.bar[2:]`\n`[:end]` | Array subset from start of array to index | `$.bar[:4]`\n`[-start:]` | Array subset from _length-start_ to end of array | `$.bar[-3:]`\n`[?()]` | Filter expession | `$.bar[?(@.baz==42)]`\n`[()]` | Static expression | `$.bar[(@.length-1)]`\n`@` | in filter expressions: the current node being processed\n\n**References**\n- Current IETF draft: https://datatracker.ietf.org/doc/html/draft-ietf-jsonpath-base-01\n- Historic site: https://goessner.net/articles/JsonPath/\n- Node.js implementation: https://www.npmjs.com/package/jsonpath\n- Java implementation: https://github.com/json-path/JsonPath\n- Online evaluator: https://jsonpath.com/\n "
}
}
}
82 changes: 81 additions & 1 deletion vocabularies/Org.OData.JSON.V1.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,87 @@ Term|Type|Description
:---|:---|:----------
[Schema](./Org.OData.JSON.V1.xml#L67:~:text=<Term%20Name="-,Schema,-")|[JSON](#JSON)|<a name="Schema"></a>The JSON Schema for JSON values of the annotated property, parameter, return type, term, or type definition<br>The schema can be a schema reference, i.e. `{"$ref":"url/of/schemafile#/path/to/schema/within/schemafile"}`

## <a name="JSON"></a>[JSON](./Org.OData.JSON.V1.xml#L75:~:text=<TypeDefinition%20Name="-,JSON,-")

## Functions

### <a name="query"></a>[query](./Org.OData.JSON.V1.xml#L75:~:text=<Function%20Name="-,query,-")

Query stream values of media type `application/json`, returning a stream value of media type `application/json`

Extracts a JSON value, such as an array, object, or a JSON scalar value (string, number, boolean, or `null`) from the `input` JSON value:
- If `path` only consists of child member and single subscript operators, it returns the identified single node within `input`, or `null` if no node is identified.
- If `path` potentially identifies multiple nodes within `input` (by using recursive descendant, wildcard, array subset, or array filter expressions), it returns an array containing the identified nodes, or an empty array if no node is identified.
- If `input` is not a valid JSON value, the function returns `null`.
- If `path` is `null`, not a valid [JSONPath expression](#Path), or does not match the structure of `input`, the function returns `null`.


Parameter|Type|Description
:--------|:---|:----------
[input](./Org.OData.JSON.V1.xml#L85:~:text=<Function%20Name="-,query,-")|[JSON?](#JSON)|JSON input
[path](./Org.OData.JSON.V1.xml#L88:~:text=<Function%20Name="-,query,-")|[Path?](#Path)|JSONPath expression to be applied to value of `expr`
[&rarr;](./Org.OData.JSON.V1.xml#L91:~:text=<Function%20Name="-,query,-")|[JSON?](#JSON)|JSON value resulting from applying `path` to `input`


### <a name="value"></a>[value](./Org.OData.JSON.V1.xml#L96:~:text=<Function%20Name="-,value,-")

Query stream values of media type `application/json`, returning an OData primitive value

Extracts an OData primitive value from the `input` JSON value:
- If `path` only consists of child member and single subscript operators and identifies a single scalar JSON value (string, number, boolean, or `null`) within `input`, it returns the identified single value cast to an OData primitive value (see below).
- If `path` identifies multiple nodes within `input` (by using recursive descendant, wildcard, array subset, or array filter expressions), identifies an object or array, or does not identify any node, the function returns `null`.
- If `input` is not a valid JSON value, the function returns `null`.
- If `path` is `null`, not a valid [JSONPath expression](#Path), or does not match the structure of `input`, the function returns `null`.

If a single non-null scalar JSON value is identified by `path` within `input`, the function returns that value as a primitive value of type
- `Edm.String` if the value is a JSON string
- `Edm.Boolean` if the value is `true` or `false`
- `Edm.Decimal` with unspecified precision and floating scale if the value is a JSON number


Parameter|Type|Description
:--------|:---|:----------
[input](./Org.OData.JSON.V1.xml#L111:~:text=<Function%20Name="-,value,-")|[JSON?](#JSON)|JSON input
[path](./Org.OData.JSON.V1.xml#L114:~:text=<Function%20Name="-,value,-")|[Path?](#Path)|JSONPath expression to be applied to value of `expr`
[&rarr;](./Org.OData.JSON.V1.xml#L117:~:text=<Function%20Name="-,value,-")|PrimitiveType?|OData primitive value resulting from applying `path` to `input`


## <a name="JSON"></a>[JSON](./Org.OData.JSON.V1.xml#L122:~:text=<TypeDefinition%20Name="-,JSON,-")
**Type:** Stream

Textual data of media type `application/json`

## <a name="Path"></a>[Path](./Org.OData.JSON.V1.xml#L133:~:text=<TypeDefinition%20Name="-,Path,-")
**Type:** String

[JSONPath](https://datatracker.ietf.org/doc/html/draft-ietf-jsonpath-base-01) expression

Implementations SHOULD support at least the following subset of JSONPath:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to see MUST/SHOULD instead of SHOULD/MAY (here/line 141).

The use of MAY in "offerings" is in my understanding watering the danger that normally MAY indicates. As in

The Server MAY respond with random payloads on Tuesdays.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the SHOULD to MUST, because less is nothing 😄

Kept the MAY, because these are really optional.


_TODO: check which data sources support which subset, then reduce expressions_
- SQL Server: https://docs.microsoft.com/en-us/sql/relational-databases/json/json-path-expressions-sql-server?view=sql-server-2017
- SAP HANA: https://help.sap.com/products/SAP_HANA_PLATFORM/4fe29514fd584807ac9f2a04f6754767/3126ea33d50d42d19517a08fe22ec5a1.html?version=2.0.05#description
- Google BigQuery references the Java implementation - all of it supported?

JSONPath | Description | Examples
---------|-------------|--------
`$` | The root object, array, or value
`.` | Child member operator | `$.foo`, `$.foo.bar`
`..` | Recursive descendant operator: searches for the specified member name recursively and returns an array of all values with this property name | `$..foo`
`*` | Wildcard matching all elements in an object or array | `$.foo.*`, `$.bar[*]`
`[]` | Subscript operator, accepting names (single-quoted strings) or array indices (zero-based integers, negative integers count from the end of the array) | `$['foo']`, `$.foo['bar']`, `$.bar[0]`, `$.bar[-1]`
`[,]` | Union operator for alternate names or array indices as a set | `$.foo['bar','baz']`, `$.bar[0,1,2,3,5,7,11]`
`[start:end]` | Array subset by range of indices | `$.bar[2:4]`
`[start:]` | Array subset from index to end of array | `$.bar[2:]`
`[:end]` | Array subset from start of array to index | `$.bar[:4]`
`[-start:]` | Array subset from _length-start_ to end of array | `$.bar[-3:]`
`[?()]` | Filter expession | `$.bar[?(@.baz==42)]`
`[()]` | Static expression | `$.bar[(@.length-1)]`
`@` | in filter expressions: the current node being processed

**References**
- Current IETF draft: https://datatracker.ietf.org/doc/html/draft-ietf-jsonpath-base-01
- Historic site: https://goessner.net/articles/JsonPath/
- Node.js implementation: https://www.npmjs.com/package/jsonpath
- Java implementation: https://github.com/json-path/JsonPath
- Online evaluator: https://jsonpath.com/

84 changes: 84 additions & 0 deletions vocabularies/Org.OData.JSON.V1.xml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,53 @@
</Annotation>
</Term>

<Function Name="query">
<Annotation Term="Core.Description" String="Query stream values of media type `application/json`, returning a stream value of media type `application/json`" />
<Annotation Term="Core.LongDescription">
<String>Extracts a JSON value, such as an array, object, or a JSON scalar value (string, number, boolean, or `null`) from the `input` JSON value:
- If `path` only consists of child member and single subscript operators, it returns the identified single node within `input`, or `null` if no node is identified.
- If `path` potentially identifies multiple nodes within `input` (by using recursive descendant, wildcard, array subset, or array filter expressions), it returns an array containing the identified nodes, or an empty array if no node is identified.
- If `input` is not a valid JSON value, the function returns `null`.
- If `path` is `null`, not a valid [JSONPath expression](#Path), or does not match the structure of `input`, the function returns `null`.
ralfhandl marked this conversation as resolved.
Show resolved Hide resolved
</String>
HeikoTheissen marked this conversation as resolved.
Show resolved Hide resolved
</Annotation>
<Parameter Name="input" Type="JSON.JSON">
<Annotation Term="Core.Description" String="JSON input" />
</Parameter>
<Parameter Name="path" Type="JSON.Path">
<Annotation Term="Core.Description" String="JSONPath expression to be applied to value of `expr`" />
</Parameter>
<ReturnType Type="JSON.JSON">
<Annotation Term="Core.Description" String="JSON value resulting from applying `path` to `input`" />
</ReturnType>
</Function>

<Function Name="value">
<Annotation Term="Core.Description" String="Query stream values of media type `application/json`, returning an OData primitive value" />
<Annotation Term="Core.LongDescription">
<String>Extracts an OData primitive value from the `input` JSON value:
- If `path` only consists of child member and single subscript operators and identifies a single scalar JSON value (string, number, boolean, or `null`) within `input`, it returns the identified single value cast to an OData primitive value (see below).
- If `path` identifies multiple nodes within `input` (by using recursive descendant, wildcard, array subset, or array filter expressions), identifies an object or array, or does not identify any node, the function returns `null`.
- If `input` is not a valid JSON value, the function returns `null`.
- If `path` is `null`, not a valid [JSONPath expression](#Path), or does not match the structure of `input`, the function returns `null`.

If a single non-null scalar JSON value is identified by `path` within `input`, the function returns that value as a primitive value of type
- `Edm.String` if the value is a JSON string
- `Edm.Boolean` if the value is `true` or `false`
- `Edm.Decimal` with unspecified precision and floating scale if the value is a JSON number
ralfhandl marked this conversation as resolved.
Show resolved Hide resolved
</String>
HeikoTheissen marked this conversation as resolved.
Show resolved Hide resolved
</Annotation>
<Parameter Name="input" Type="JSON.JSON">
<Annotation Term="Core.Description" String="JSON input" />
</Parameter>
<Parameter Name="path" Type="JSON.Path">
<Annotation Term="Core.Description" String="JSONPath expression to be applied to value of `expr`" />
</Parameter>
<ReturnType Type="Edm.PrimitiveType">
<Annotation Term="Core.Description" String="OData primitive value resulting from applying `path` to `input`" />
</ReturnType>
</Function>

<TypeDefinition Name="JSON" UnderlyingType="Edm.Stream">
<Annotation Term="Core.Description" String="Textual data of media type `application/json`" />
<Annotation Term="Core.MediaType" String="application/json" />
Expand All @@ -82,6 +129,43 @@
</Annotation>
</TypeDefinition>


<TypeDefinition Name="Path" UnderlyingType="Edm.String">
<Annotation Term="Core.Description" String="[JSONPath](https://datatracker.ietf.org/doc/html/draft-ietf-jsonpath-base-01) expression" />
<Annotation Term="Core.LongDescription">
<String>Implementations SHOULD support at least the following subset of JSONPath:

_TODO: check which data sources support which subset, then reduce expressions_
- SQL Server: https://docs.microsoft.com/en-us/sql/relational-databases/json/json-path-expressions-sql-server?view=sql-server-2017
- SAP HANA: https://help.sap.com/products/SAP_HANA_PLATFORM/4fe29514fd584807ac9f2a04f6754767/3126ea33d50d42d19517a08fe22ec5a1.html?version=2.0.05#description
- Google BigQuery references the Java implementation - all of it supported?

JSONPath | Description | Examples
---------|-------------|--------
`$` | The root object, array, or value
`.` | Child member operator | `$.foo`, `$.foo.bar`
`..` | Recursive descendant operator: searches for the specified member name recursively and returns an array of all values with this property name | `$..foo`
`*` | Wildcard matching all elements in an object or array | `$.foo.*`, `$.bar[*]`
`[]` | Subscript operator, accepting names (single-quoted strings) or array indices (zero-based integers, negative integers count from the end of the array) | `$['foo']`, `$.foo['bar']`, `$.bar[0]`, `$.bar[-1]`
`[,]` | Union operator for alternate names or array indices as a set | `$.foo['bar','baz']`, `$.bar[0,1,2,3,5,7,11]`
`[start:end]` | Array subset by range of indices | `$.bar[2:4]`
`[start:]` | Array subset from index to end of array | `$.bar[2:]`
`[:end]` | Array subset from start of array to index | `$.bar[:4]`
`[-start:]` | Array subset from _length-start_ to end of array | `$.bar[-3:]`
`[?()]` | Filter expession | `$.bar[?(@.baz==42)]`
`[()]` | Static expression | `$.bar[(@.length-1)]`
`@` | in filter expressions: the current node being processed

**References**
- Current IETF draft: https://datatracker.ietf.org/doc/html/draft-ietf-jsonpath-base-01
- Historic site: https://goessner.net/articles/JsonPath/
- Node.js implementation: https://www.npmjs.com/package/jsonpath
- Java implementation: https://github.com/json-path/JsonPath
- Online evaluator: https://jsonpath.com/
</String>
</Annotation>
</TypeDefinition>

</Schema>
</edmx:DataServices>
</edmx:Edmx>