Skip to content

Latest commit

 

History

History
570 lines (478 loc) · 9.78 KB

README.md

File metadata and controls

570 lines (478 loc) · 9.78 KB
Raw

jetro

GitHub

Jetro is a library which provides a custom DSL for transforming, querying and comparing data in JSON format. It is easy to use and extend.

Jetro has minimal dependency, the traversal and eval algorithm is implemented on top of serde_json.

Jetro can be used inside Web Browser by compiling down to WASM. Clone it and give it a shot.

Jetro can be used in command line using Jetrocli.

Jetro combines access path with functions which operate on values matched within the pipeline. Access path uses / as separator similar to structure of URI, the start of access path should denote whether the access starts from root by using >, it is possible to traverse from root in nested paths by using <.

Jetro expressions support line breaks and whitespace, the statements can be broken up into smaller parts.

By convention, functions are denoted using # operator. Functions can be composed.

Function Action
#pick('string' | expression, ...) [ as | as* 'binding_value' ] Select a key from an object, bind it to a name, select multiple sub queries to create new object
#head Head of the list
#tail Tail of the list
#keys Keys associated with an object
#values Values associated with an object
#reverse Reverse the list
#min Min value of numbers
#max Max value of numbers
#all Whether all boolean values are true
#sum Sum of numbers
#formats('format with placeholder {} {}', 'key_a', 'key_b') [ -> | ->* 'binding_value' ] Insert formatted key:value into object or return it as single key:value
#filter('target_key' (>, <, >=, <=, ==, ~=, !=) (string, boolean, number)) Perform Filter on list
#map(x: x.y.z | x.y.z.some_method()) Map each item in a list with the given lambda
#zip Zip two or more arrays together 
let data = serde_json::json!({
  "name": "mr snuggle",
  "some_entry": {
    "some_obj": {
      "obj": {
        "a": "object_a",
        "b": "object_b",
        "c": "object_c",
        "d": "object_d"
      }
    }
  }
});

let mut values = Path::collect(data, ">/..obj/#pick('a','b')");

#[derive(Serialize, Deserialize)]
struct Output {
   a: String,
   b: String,
}

let output: Option<Output> = values.from_index(0);

structure

Jetro consists of a parser, context wrapper which manages traversal and evaluation of each step of user input and a runtime for dynamic functions. The future version will support user-defined functions.

example

{
  "customer": {
    "id": "xyz",
    "ident": {
      "user": {
        "isExternal": false,
        "profile": {
          "firstname": "John",
          "alias": "Japp",
          "lastname": "Appleseed"
        }
      }
    },
    "preferences": []
  },
  "line_items": {
    "items": [
      {
        "ident": "abc",
        "is_gratis": false,
        "name": "pizza",
        "price": 4.8,
        "total": 1,
        "type": "base_composable"
      },
      {
        "ident": "def",
        "is_gratis": false,
        "name": "salami",
        "price": 2.8,
        "total": 10,
        "type": "ingredient"
      },
      {
        "ident": "ghi",
        "is_gratis": false,
        "name": "cheese",
        "price": 2,
        "total": 1,
        "type": "ingredient"
      },
      {
        "ident": "uip",
        "is_gratis": true,
        "name": "chilli",
        "price": 0,
        "total": 1,
        "type": "ingredient"
      },
      {
        "ident": "ewq",
        "is_gratis": true,
        "name": "bread sticks",
        "price": 0,
        "total": 8,
        "type": "box"
      }
    ]
  }
}

Queries

Get value associated with line_items.

>/line_items
See output

result

{
"items": [
  {
    "ident": "abc",
    "is_gratis": false,
    "name": "pizza",
    "price": 4.8,
    "total": 1,
    "type": "base_composable"
  },
  {
    "ident": "def",
    "is_gratis": false,
    "name": "salami",
    "price": 2.8,
    "total": 10,
    "type": "ingredient"
  },
  {
    "ident": "ghi",
    "is_gratis": false,
    "name": "cheese",
    "price": 2,
    "total": 1,
    "type": "ingredient"
  },
  {
    "ident": "uip",
    "is_gratis": true,
    "name": "chilli",
    "price": 0,
    "total": 1,
    "type": "ingredient"
  },
  {
    "ident": "ewq",
    "is_gratis": true,
    "name": "bread sticks",
    "price": 0,
    "total": 8,
    "type": "box"
  }
]
}

Get value associated with first matching key which has a value and return its id field.

>/('non-existing-member' | 'customer')/id
See output

result

"xyz"

Recursively search for objects that has key with specified value.

>/..('type'='ingredient')
See output

result

[
  {
	"ident": "ghi",
	"is_gratis": false,
	"name": "cheese",
	"price": 2,
	"total": 1,
	"type": "ingredient"
  },
  {
	"ident": "def",
	"is_gratis": false,
	"name": "salami",
	"price": 2.8,
	"total": 10,
	"type": "ingredient"
  }
]
>/..items/#tail
See output

result

[
  {
    "ident": "def",
    "is_gratis": false,
    "name": "salami",
    "price": 2.8,
    "total": 10,
    "type": "ingredient"
  },
  {
    "ident": "ghi",
    "is_gratis": false,
    "name": "cheese",
    "price": 2,
    "total": 1,
    "type": "ingredient"
  },
  {
    "ident": "uip",
    "is_gratis": true,
    "name": "chilli",
    "price": 0,
    "total": 1,
    "type": "ingredient"
  },
  {
    "ident": "ewq",
    "is_gratis": true,
    "name": "bread sticks",
    "price": 0,
    "total": 8,
    "type": "box"
  }
]
>/..items/#filter('is_gratis' == true and 'name' ~= 'ChILLi')
See output

result

[
  {
    "ident": "uip",
    "is_gratis": true,
    "name": "chilli",
    "price": 0,
    "total": 1,
    "type": "ingredient"
  }
]
>/..items/#filter('is_gratis' == true and 'name' ~= 'ChILLi')/#map(x: x.type)
See output

result

[
  "ingredient"
]

Create a new object with scheme {'total': ..., 'fullname': ...} as follow:

  • recursively search for line_items, dive into any matched object, filter matches with is_gratis == false statement, recursively look for their prices and return the sum of prices
  • recursively search for object user, select its profile and create a new object with schema {'fullname': ...} formated by concatenating values of keys ('firstname', 'lastname')
>/#pick(
  >/..line_items
   /*
   /#filter('is_gratis' == false)/..price/#sum as 'total',

  >/..user
   /profile
   /#formats('{} {}', 'firstname', 'lastname') ->* 'fullname'
)
See output

result

{
  "fullname": "John Appleseed",
  "total": 9.6
}

Select up to 4 items from index zero of array items

>/..items/[:4]
See output

result

[
  {
    "ident": "abc",
    "is_gratis": false,
    "name": "pizza",
    "price": 4.8,
    "total": 1,
    "type": "base_composable"
  },
  {
    "ident": "def",
    "is_gratis": false,
    "name": "salami",
    "price": 2.8,
    "total": 10,
    "type": "ingredient"
  },
  {
    "ident": "ghi",
    "is_gratis": false,
    "name": "cheese",
    "price": 2,
    "total": 1,
    "type": "ingredient"
  },
  {
    "ident": "uip",
    "is_gratis": true,
    "name": "chilli",
    "price": 0,
    "total": 1,
    "type": "ingredient"
  }
]

Select from 4th index and consume until end of array items

>/..items/[4:]
See output

result

[
  {
    "ident": "ewq",
    "is_gratis": true,
    "name": "bread sticks",
    "price": 0,
    "total": 8,
    "type": "box"
  }
]

Create a new object with schema {'total_gratis': ...} as follow:

  • Recursively look for any object containing items, and then recursively search within the matched object for is_gratis and length of matched values
>/#pick(>/..items/..is_gratis/#len as 'total_gratis')
See output

result

{
  "total_gratis": 2
}

Recursively search for object items, select its first item and return its keys

>/..items/[0]/#keys
See output

result

[
  "ident",
  "is_gratis",
  "name",
  "price",
  "total",
  "type"
]

Recursively search for object items, select its first item and return its values

>/..items/[0]/#values
See output

result

[
  "abc",
  false,
  "pizza",
  4.8,
  1,
  "base_composable"
]

Zip two or more arrays together.

>/#pick(>/..name as 'name',
        >/..nested as 'field',
        >/..b as 'release')/#zip

JSON:

{
  "a": [
    {
      "name": "tool",
      "value": {
        "nested": "field"
      }
    },
    {
      "name": "pneuma",
      "value": {
        "nested": "seal"
      }
    }
  ],
  "b": [
    2000,
    2100
  ]
}
See output

result

[
  {
    "field": "field",
    "name": "tool",
    "release": 2000
  },
  {
    "field": "seal",
    "name": "pneuma",
    "release": 2100
  }
]