From d20c8049069dd9ece759f380b3eb8b127a09b82e Mon Sep 17 00:00:00 2001 From: Chandra Sanapala Date: Wed, 14 Aug 2024 09:41:08 +0530 Subject: [PATCH 1/6] Proposal for test file format --- tests/README.md | 138 ++++++++++++++++++++++ tests/cases/arithmetic/add.yaml | 31 +++++ tests/cases/arithmetic_decimal/power.yaml | 21 ++++ tests/cases/datetime/lt_datetime.yaml | 25 ++++ 4 files changed, 215 insertions(+) create mode 100644 tests/README.md create mode 100644 tests/cases/arithmetic/add.yaml create mode 100644 tests/cases/arithmetic_decimal/power.yaml create mode 100644 tests/cases/datetime/lt_datetime.yaml diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 000000000..66027e2f8 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,138 @@ +# Substrait Test Format + +This document describes the format for Substrait scalar test files. +A test file consists of the following elements: + +1. Version declaration +2. Optional include statements +3. One or more test groups, each containing one or more test cases + +## Syntax + +### Version Declaration +The version declaration must be the first line of the file. It specifies the version of the test file format. The version declaration must be in the following format: +``` +### SUBSTRAIT_SCALAR_TEST: V1 +``` + +### Include Statements +Include statements should have at least one include statement. The include statement specifies the path to substrait extension functions. The include statement must be in the following format: +``` +### SUBSTRAIT_INCLUDE: /extensions/functions_aggregate_approx.yaml +``` + +### Test Groups +A test group is a collection of test cases that are logically related. +- **description**: A string describing the test group or case. The description must start with a `#` character. + ```code + # Common Maths + ``` +### Test Cases +A test case consists of the following elements: + +- **function**: The name of the function being tested. The function name must be a string. +- **arguments**: Comma-separated list of arguments to the function. The arguments must be literals. +- **options**: Optional comma-separated list of options in `key:value` format. The options describe the behavior of the function. The test should be run only on dialects that support the options. If options are not specified, the test should be run for all permutations of the options. +- **result**: The expected result of the function. Either `SUBSTRAIT_ERROR` or a literal value. +- **literal**: In the format `::` +- **description**: A string describing the test case + + ```code + add(126::i8, 1::i8) = 127::i8 # addition of two numbers + ``` + +### Spec + +``` +doc := + ()+ + (()?()+\n)+ +version := ### SUBSTRAIT_SCALAR_TEST: +include := ### SUBSTRAIT_INCLUDE: +test_group := # +test_case := () ([])? = (#)? +description := string +function := string +arguments := , , ... +argument := +literal := :: +result := SUBSTRAIT_ERROR | +options := , , ... +optLiteral := : +``` + +**TODO:** use ANTLR to describe the grammar and generate parser +### Literals + +#### String +- **string**, **fixedchar**, **varchar**: A sequence of characters enclosed in single quotes. Example: 'Hello, world!' + +#### Integer +Integers are represented as sequences of digits. Negative numbers are preceded by a minus sign. +- **i8**: 8-bit integer, range: -128 to 127 +- **i16**: 16-bit integer, range: -32,768 to 32,767 +- **i32**: 32-bit integer, range: -2,147,483,648 to 2,147,483,647 +- **i64**: 64-bit integer, range: -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 + +#### Fixed Point decimals +- **decimal**: Fixed-point decimal number. Maximum 38 digits total, with up to 37 digits after the decimal point. + Example: 123.456 + +#### Floating Point numbers +- **float**: General floating-point number, can be represented as: + * Standard decimal notation: 123.456 + * Scientific notation: 1.23e4 + * Special values: `nan` (Not a Number), `+inf` (Positive Infinity), `-inf` (Negative Infinity) +- **float32**: Single-precision float, approximately 6 significant digits, range: ~1.2e-38 to ~3.4e38 +- **float64**: Double-precision float, approximately 15 significant digits, range: ~2.3e-308 to ~1.7e308 + +#### Boolean +- Valid values: TRUE, FALSE, NULL + +#### Date and Time +All date and time literals use ISO 8601 format: + +- **date**: `YYYY-MM-DD`, example: `2021-01-01` +- **time**: `HH:MM:SS[.fraction]`, example: `12:00:00.000` +- **timestamp**: `YYYY-MM-DD HH:MM:SS[.fraction]`, example: `2021-01-01 12:00:00` +- **timestamp_tz**: `YYYY-MM-DD HH:MM:SS[.fraction]±HH:MM`, example: `2021-01-01 12:00:00+05:30` +- **interval year**: `INTERVAL 'P[n]Y[n]M'`, example: `INTERVAL 'P2Y3M'` (2 years, 3 months) +- **interval days**: `INTERVAL 'P[n]DT[n]H[n]M[n]S'`, example: `INTERVAL 'P2DT3H2M9S'` (2 days, 3 hours, 2 minutes, 9 seconds) + +#### Other complex types +**TODO** Add support for complex types like arrays, structs, maps etc. + +### Data Types + +- **bool**: Boolean +- **i8**: 8-bit signed integer +- **i16**: 16-bit signed integer +- **i32**: 32-bit signed integer +- **i64**: 64-bit signed integer +- **f32**: 32-bit floating point number +- **f64**: 64-bit floating point number +- **dec**: Fixed-point `decimal` +- **str**: Variable-length string +- **fchar**: Fixed-length string `fixedchar` +- **vchar**: Variable-length string `varchar` +- **vbin**: Fixed-length binary `fixedbinary` +- **date**: Date +- **time**: Time +- **ts**: Timestamp +- **tstz**: Timestamp with timezone +- **iyear**: Interval year +- **iday**: Interval days + +### Example of a test file + +```code +### SUBSTRAIT_SCALAR_TEST:V1 +### SUBSTRAIT_INCLUDE: /extensions/functions_arithmetic.yaml + +# Common Maths +add(126::i8, 1::i8) = 127::i8 + +# Arithmetic Overflow Tests +add(127::i8, 1::i8) [overflow:ERROR] = #check overflow +``` +The above test file has two test groups "Common Maths" and "Arithmetic Overflow Tests". Each has one test case. The test case in the second group has a name whereas case in the first one does not. diff --git a/tests/cases/arithmetic/add.yaml b/tests/cases/arithmetic/add.yaml new file mode 100644 index 000000000..444647f25 --- /dev/null +++ b/tests/cases/arithmetic/add.yaml @@ -0,0 +1,31 @@ +### SUBSTRAIT_SCALAR_TEST: 1.0 +### SUBSTRAIT_INCLUDE: /extensions/functions_arithmetic.yaml + +# basic: Basic examples without any special cases +add(120::i8, 5::i8) = 125::i8 +add(100::i16, 100::i16) = 200::i16 +add(30000::i32, 30000::i32) = 60000::i32 +add(2000000000::i64, 2000000000::i64) = 4000000000::i64 + +# overflow: Examples demonstrating overflow behavior +add(120::i8, 10::i8) [overflow:ERROR] = error +add(30000::i16, 30000::i16) [overflow:ERROR] = error +add(2000000000::i32, 2000000000::i32) [overflow:ERROR] = error +add(9223372036854775807::i64, 1::i64) [overflow:ERROR] = error + +# overflow: Examples demonstrating overflow behavior tests: overflow with SATURATE +add(120::i8, 10::i8) [overflow:SATURATE] = 127::i8 +add(-120::i8, -10::i8) [overflow:SATURATE] = -128::i8 + +# overflow: Examples demonstrating overflow behavior tests: overflow with SILENT +add(120::i8, 10::i8) [overflow:SILENT] = undefined + +# floating_exception: Examples demonstrating exceptional floating point cases +add(1.5e+308::fp64, 1.5e+308::fp64) = inf::fp64 +add(-1.5e+308::fp64, -1.5e+308::fp64) = -inf::fp64 + +# rounding: Examples demonstrating floating point rounding behavior +add(4.5::fp32, 2.500001::fp32) [rounding:TIE_TO_EVEN] = 7.000001::fp32 + +# types: Examples demonstrating behavior of different data types +add(4.5::fp64, 2.5000007152557373::fp64) = 7.00000071525573::fp64 diff --git a/tests/cases/arithmetic_decimal/power.yaml b/tests/cases/arithmetic_decimal/power.yaml new file mode 100644 index 000000000..bfeba4672 --- /dev/null +++ b/tests/cases/arithmetic_decimal/power.yaml @@ -0,0 +1,21 @@ +### SUBSTRAIT_SCALAR_TEST: 1.0 +### SUBSTRAIT_INCLUDE: extensions/functions_arithmetic_decimal.yaml + +# basic: Basic examples without any special cases +power(8::decimal, 2::decimal<38, 0>) = 64::fp64 +power(1.0::decimal, -1.0::decimal<38, 0>) = 1.0::fp64 +power(2.0::decimal<38, 0>, -2.0::decimal<38, 0>) = 0.25::fp64 +power(13::decimal<38, 0>, 10::decimal<38, 0>) = 137858491849::fp64 + +# result_more_than_input_precison: Examples demonstrating result with more precision than input +power(16::decimal<2, 0>, 4::decimal<38, 0>) = 65536::fp64 + +# floating_exception: Examples demonstrating exceptional floating point cases +power(1.5e+10::decimal<38, 0>, 1.5e+20::decimal<38, 0>) = inf::fp64 +power(-16::decimal<4, 0>, 1001::decimal<4, 0>) = -inf::fp64 + +# complex_number: Examples demonstrating complex number output +power(-1::decimal, 0.5::decimal<38,1>) [complex_number_result:NAN] = nan + +# complex_number: Examples demonstrating complex number output tests: complex_number_result with ERROR +power(-1::decimal, 0.5::decimal<38,1>) [complex_number_result:ERROR] = error diff --git a/tests/cases/datetime/lt_datetime.yaml b/tests/cases/datetime/lt_datetime.yaml new file mode 100644 index 000000000..38cf4805b --- /dev/null +++ b/tests/cases/datetime/lt_datetime.yaml @@ -0,0 +1,25 @@ +### SUBSTRAIT_SCALAR_TEST: 1.0 +### SUBSTRAIT_INCLUDE: /extensions/functions_datetime.yaml + +# timestamps: examples using the timestamp type +lt(2016-12-31 13:30:15::timestamp, 2017-12-31 13:30:15::timestamp) = True::boolean +lt(2018-12-31 13:30:15::timestamp, 2017-12-31 13:30:15::timestamp) = False::boolean + +# timestamp_tz: examples using the timestamp_tz type +lt(1999-01-08 01:05:05-08:00::timestamp_tz, 1999-01-08 04:05:06-05:00::timestamp_tz) = True::boolean +lt(1999-01-08 01:05:06-08:00::timestamp_tz, 1999-01-08 04:05:06-05:00::timestamp_tz) = False::boolean + +# date: examples using the date type +lt(2020-12-30::date, 2020-12-31::date) = True::boolean +lt(2020-12-31::date, 2020-12-30::date) = False::boolean + +# interval: examples using the interval type +lt(INTERVAL 'P7D'::interval, INTERVAL 'P6D'::interval) = False::boolean +lt(INTERVAL 'P5D'::interval, INTERVAL '6D'::interval) = True::boolean +lt(INTERVAL 'P5Y'::interval, INTERVAL 'P6Y'::interval) = True::boolean +lt(INTERVAL 'P7Y'::interval, INTERVAL 'P6Y'::interval) = False::boolean + +# null_input: examples with null args or return +lt(None::interval, INTERVAL 'P5D'::interval) = Null::boolean +lt(None::date, 2020-12-30::date) = Null::boolean +lt(None::timestamp, 2018-12-31 13:30:15::timestamp) = Null::boolean From 22b186c6ab4c4404a367fdc16252223c0b8f625c Mon Sep 17 00:00:00 2001 From: Chandra Sanapala Date: Wed, 14 Aug 2024 09:48:37 +0530 Subject: [PATCH 2/6] rename test files --- tests/cases/arithmetic/add.yaml | 31 ----------------------- tests/cases/arithmetic_decimal/power.yaml | 21 --------------- tests/cases/datetime/lt_datetime.yaml | 25 ------------------ 3 files changed, 77 deletions(-) delete mode 100644 tests/cases/arithmetic/add.yaml delete mode 100644 tests/cases/arithmetic_decimal/power.yaml delete mode 100644 tests/cases/datetime/lt_datetime.yaml diff --git a/tests/cases/arithmetic/add.yaml b/tests/cases/arithmetic/add.yaml deleted file mode 100644 index 444647f25..000000000 --- a/tests/cases/arithmetic/add.yaml +++ /dev/null @@ -1,31 +0,0 @@ -### SUBSTRAIT_SCALAR_TEST: 1.0 -### SUBSTRAIT_INCLUDE: /extensions/functions_arithmetic.yaml - -# basic: Basic examples without any special cases -add(120::i8, 5::i8) = 125::i8 -add(100::i16, 100::i16) = 200::i16 -add(30000::i32, 30000::i32) = 60000::i32 -add(2000000000::i64, 2000000000::i64) = 4000000000::i64 - -# overflow: Examples demonstrating overflow behavior -add(120::i8, 10::i8) [overflow:ERROR] = error -add(30000::i16, 30000::i16) [overflow:ERROR] = error -add(2000000000::i32, 2000000000::i32) [overflow:ERROR] = error -add(9223372036854775807::i64, 1::i64) [overflow:ERROR] = error - -# overflow: Examples demonstrating overflow behavior tests: overflow with SATURATE -add(120::i8, 10::i8) [overflow:SATURATE] = 127::i8 -add(-120::i8, -10::i8) [overflow:SATURATE] = -128::i8 - -# overflow: Examples demonstrating overflow behavior tests: overflow with SILENT -add(120::i8, 10::i8) [overflow:SILENT] = undefined - -# floating_exception: Examples demonstrating exceptional floating point cases -add(1.5e+308::fp64, 1.5e+308::fp64) = inf::fp64 -add(-1.5e+308::fp64, -1.5e+308::fp64) = -inf::fp64 - -# rounding: Examples demonstrating floating point rounding behavior -add(4.5::fp32, 2.500001::fp32) [rounding:TIE_TO_EVEN] = 7.000001::fp32 - -# types: Examples demonstrating behavior of different data types -add(4.5::fp64, 2.5000007152557373::fp64) = 7.00000071525573::fp64 diff --git a/tests/cases/arithmetic_decimal/power.yaml b/tests/cases/arithmetic_decimal/power.yaml deleted file mode 100644 index bfeba4672..000000000 --- a/tests/cases/arithmetic_decimal/power.yaml +++ /dev/null @@ -1,21 +0,0 @@ -### SUBSTRAIT_SCALAR_TEST: 1.0 -### SUBSTRAIT_INCLUDE: extensions/functions_arithmetic_decimal.yaml - -# basic: Basic examples without any special cases -power(8::decimal, 2::decimal<38, 0>) = 64::fp64 -power(1.0::decimal, -1.0::decimal<38, 0>) = 1.0::fp64 -power(2.0::decimal<38, 0>, -2.0::decimal<38, 0>) = 0.25::fp64 -power(13::decimal<38, 0>, 10::decimal<38, 0>) = 137858491849::fp64 - -# result_more_than_input_precison: Examples demonstrating result with more precision than input -power(16::decimal<2, 0>, 4::decimal<38, 0>) = 65536::fp64 - -# floating_exception: Examples demonstrating exceptional floating point cases -power(1.5e+10::decimal<38, 0>, 1.5e+20::decimal<38, 0>) = inf::fp64 -power(-16::decimal<4, 0>, 1001::decimal<4, 0>) = -inf::fp64 - -# complex_number: Examples demonstrating complex number output -power(-1::decimal, 0.5::decimal<38,1>) [complex_number_result:NAN] = nan - -# complex_number: Examples demonstrating complex number output tests: complex_number_result with ERROR -power(-1::decimal, 0.5::decimal<38,1>) [complex_number_result:ERROR] = error diff --git a/tests/cases/datetime/lt_datetime.yaml b/tests/cases/datetime/lt_datetime.yaml deleted file mode 100644 index 38cf4805b..000000000 --- a/tests/cases/datetime/lt_datetime.yaml +++ /dev/null @@ -1,25 +0,0 @@ -### SUBSTRAIT_SCALAR_TEST: 1.0 -### SUBSTRAIT_INCLUDE: /extensions/functions_datetime.yaml - -# timestamps: examples using the timestamp type -lt(2016-12-31 13:30:15::timestamp, 2017-12-31 13:30:15::timestamp) = True::boolean -lt(2018-12-31 13:30:15::timestamp, 2017-12-31 13:30:15::timestamp) = False::boolean - -# timestamp_tz: examples using the timestamp_tz type -lt(1999-01-08 01:05:05-08:00::timestamp_tz, 1999-01-08 04:05:06-05:00::timestamp_tz) = True::boolean -lt(1999-01-08 01:05:06-08:00::timestamp_tz, 1999-01-08 04:05:06-05:00::timestamp_tz) = False::boolean - -# date: examples using the date type -lt(2020-12-30::date, 2020-12-31::date) = True::boolean -lt(2020-12-31::date, 2020-12-30::date) = False::boolean - -# interval: examples using the interval type -lt(INTERVAL 'P7D'::interval, INTERVAL 'P6D'::interval) = False::boolean -lt(INTERVAL 'P5D'::interval, INTERVAL '6D'::interval) = True::boolean -lt(INTERVAL 'P5Y'::interval, INTERVAL 'P6Y'::interval) = True::boolean -lt(INTERVAL 'P7Y'::interval, INTERVAL 'P6Y'::interval) = False::boolean - -# null_input: examples with null args or return -lt(None::interval, INTERVAL 'P5D'::interval) = Null::boolean -lt(None::date, 2020-12-30::date) = Null::boolean -lt(None::timestamp, 2018-12-31 13:30:15::timestamp) = Null::boolean From b63c4d2ee733f3f1504ae1ad8fbb33d9033662f8 Mon Sep 17 00:00:00 2001 From: Chandra Sanapala Date: Thu, 15 Aug 2024 21:11:29 +0530 Subject: [PATCH 3/6] use shortType name --- tests/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/README.md b/tests/README.md index 66027e2f8..6c29dc842 100644 --- a/tests/README.md +++ b/tests/README.md @@ -96,8 +96,8 @@ All date and time literals use ISO 8601 format: - **time**: `HH:MM:SS[.fraction]`, example: `12:00:00.000` - **timestamp**: `YYYY-MM-DD HH:MM:SS[.fraction]`, example: `2021-01-01 12:00:00` - **timestamp_tz**: `YYYY-MM-DD HH:MM:SS[.fraction]±HH:MM`, example: `2021-01-01 12:00:00+05:30` -- **interval year**: `INTERVAL 'P[n]Y[n]M'`, example: `INTERVAL 'P2Y3M'` (2 years, 3 months) -- **interval days**: `INTERVAL 'P[n]DT[n]H[n]M[n]S'`, example: `INTERVAL 'P2DT3H2M9S'` (2 days, 3 hours, 2 minutes, 9 seconds) +- **interval year**: `'P[n]Y[n]M'`, example: `'P2Y3M'` (2 years, 3 months) +- **interval days**: `'P[n]DT[n]H[n]M[n]S'`, example: `'P2DT3H2M9S'` (2 days, 3 hours, 2 minutes, 9 seconds) #### Other complex types **TODO** Add support for complex types like arrays, structs, maps etc. From 6e3e96adc11ac2a4f42cf713881d447cc1f2bc4d Mon Sep 17 00:00:00 2001 From: Chandra Sanapala Date: Fri, 16 Aug 2024 17:26:43 +0530 Subject: [PATCH 4/6] Address review comments. --- tests/README.md | 43 +++++++++++++++++++++++++++++++------------ 1 file changed, 31 insertions(+), 12 deletions(-) diff --git a/tests/README.md b/tests/README.md index 6c29dc842..e6eb8c391 100644 --- a/tests/README.md +++ b/tests/README.md @@ -22,7 +22,7 @@ Include statements should have at least one include statement. The include state ``` ### Test Groups -A test group is a collection of test cases that are logically related. +A test group is a collection of test cases that are logically related. Test groups are purely for categorization purposes and do not affect the execution or meaning of tests. - **description**: A string describing the test group or case. The description must start with a `#` character. ```code # Common Maths @@ -30,11 +30,11 @@ A test group is a collection of test cases that are logically related. ### Test Cases A test case consists of the following elements: -- **function**: The name of the function being tested. The function name must be a string. +- **function**: The name of the function being tested. The function name must be an identifier alphanumeric string. - **arguments**: Comma-separated list of arguments to the function. The arguments must be literals. - **options**: Optional comma-separated list of options in `key:value` format. The options describe the behavior of the function. The test should be run only on dialects that support the options. If options are not specified, the test should be run for all permutations of the options. - **result**: The expected result of the function. Either `SUBSTRAIT_ERROR` or a literal value. -- **literal**: In the format `::` +- **literal**: In the format `::` - **description**: A string describing the test case ```code @@ -55,24 +55,32 @@ description := string function := string arguments := , , ... argument := -literal := :: -result := SUBSTRAIT_ERROR | +literal := :: +result := | options := , , ... optLiteral := : +lietral_value := string | integer | decimal | float | boolean | date | time | timestamp | timestamp_tz | interval year | interval days | null +datatype := | | +basic_type := bool | i8 | i16 | i32 | i64 | f32 | f64 | str | date | time | ts | tstz | iyear | iday | +parametrized_type := dec | fchar | vchar | vbin +complex_type := | | +substrait_error := | ``` **TODO:** use ANTLR to describe the grammar and generate parser ### Literals +`` described in this section. + #### String - **string**, **fixedchar**, **varchar**: A sequence of characters enclosed in single quotes. Example: 'Hello, world!' #### Integer Integers are represented as sequences of digits. Negative numbers are preceded by a minus sign. - **i8**: 8-bit integer, range: -128 to 127 -- **i16**: 16-bit integer, range: -32,768 to 32,767 -- **i32**: 32-bit integer, range: -2,147,483,648 to 2,147,483,647 -- **i64**: 64-bit integer, range: -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 +- **i16**: 16-bit integer, range: -32768 to 32767 +- **i32**: 32-bit integer, range: -2147483648 to 2147483647 +- **i64**: 64-bit integer, range: -9223372036854775808 to 9223372036854775807 #### Fixed Point decimals - **decimal**: Fixed-point decimal number. Maximum 38 digits total, with up to 37 digits after the decimal point. @@ -82,12 +90,12 @@ Integers are represented as sequences of digits. Negative numbers are preceded b - **float**: General floating-point number, can be represented as: * Standard decimal notation: 123.456 * Scientific notation: 1.23e4 - * Special values: `nan` (Not a Number), `+inf` (Positive Infinity), `-inf` (Negative Infinity) + * Special values: `+inf` (Positive Infinity), `-inf` (Negative Infinity), `+0` (Positive Zero), `-0` (Negative Zero), `nan` (Not a Number), `snan` (Signaling NaN) - **float32**: Single-precision float, approximately 6 significant digits, range: ~1.2e-38 to ~3.4e38 - **float64**: Double-precision float, approximately 15 significant digits, range: ~2.3e-308 to ~1.7e308 #### Boolean -- Valid values: TRUE, FALSE, NULL +- Valid values: true, false #### Date and Time All date and time literals use ISO 8601 format: @@ -97,13 +105,16 @@ All date and time literals use ISO 8601 format: - **timestamp**: `YYYY-MM-DD HH:MM:SS[.fraction]`, example: `2021-01-01 12:00:00` - **timestamp_tz**: `YYYY-MM-DD HH:MM:SS[.fraction]±HH:MM`, example: `2021-01-01 12:00:00+05:30` - **interval year**: `'P[n]Y[n]M'`, example: `'P2Y3M'` (2 years, 3 months) -- **interval days**: `'P[n]DT[n]H[n]M[n]S'`, example: `'P2DT3H2M9S'` (2 days, 3 hours, 2 minutes, 9 seconds) +- **interval days**: `'P[n]DT[n]H[n]M[n]S[n]F'`, Valid values for F are 1-9, representing fraction of a second. example: `'P2DT3H2M9S'` (2 days, 3 hours, 2 minutes, 9 seconds) + ex2: 'P1DT2H3M4S99::iday<3>' (1 day, 2 hours, 3 minutes, 4 seconds, 99 milliseconds)` #### Other complex types **TODO** Add support for complex types like arrays, structs, maps etc. ### Data Types +Use short names listed in https://substrait.io/extensions/#function-signature-compound-names + - **bool**: Boolean - **i8**: 8-bit signed integer - **i16**: 16-bit signed integer @@ -123,6 +134,14 @@ All date and time literals use ISO 8601 format: - **iyear**: Interval year - **iday**: Interval days + +### Options + +**TODO** Add option names and values to the grammar. Option names and values can be found in extension files. + +### Errors + +If return type is `` then the test case should fail with an error. If the return type is `` then the test case should not fail result could be anything. ### Example of a test file ```code @@ -133,6 +152,6 @@ All date and time literals use ISO 8601 format: add(126::i8, 1::i8) = 127::i8 # Arithmetic Overflow Tests -add(127::i8, 1::i8) [overflow:ERROR] = #check overflow +add(127::i8, 1::i8) [overflow:ERROR] = ERROR #check overflow ``` The above test file has two test groups "Common Maths" and "Arithmetic Overflow Tests". Each has one test case. The test case in the second group has a name whereas case in the first one does not. From ad24c4d626b8e84df44941b062bc74dd62ba6edb Mon Sep 17 00:00:00 2001 From: Chandra Sanapala Date: Mon, 26 Aug 2024 11:05:26 +0530 Subject: [PATCH 5/6] Address review comments --- tests/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/README.md b/tests/README.md index e6eb8c391..81d9a850b 100644 --- a/tests/README.md +++ b/tests/README.md @@ -57,9 +57,9 @@ arguments := , , ... argument := literal := :: result := | -options := , , ... -optLiteral := : -lietral_value := string | integer | decimal | float | boolean | date | time | timestamp | timestamp_tz | interval year | interval days | null +options :=