PATCH and partial updates

Folks and Context

• Adrian - change sparse data structure
• Steve - Investigating PATCH
• Brandur - Heroku
• Matt - JSON Patch for PHP
• Jeff - DNS
• Lukas - Tooling for Apis
• Rob - changing small items, or pieces of items from larger data existing data sets, while keeping an audit trail

JSON PATCH Overview

• Standardized in April
• always an array of operations
• ex `{ "op": "add", "path": "/a/foo", "value" : any json
  • path is a json pointer
• ops

{ "foo" : { "bar" : 42 } }

* test - ex. make sure something before proceeding
* add - only works if the object already exists
   * /foo/b succeeds, but /a/b fails, as the parent doesn't exist
   * /foo/bar will replace
* replace - differs from add where the entire path must exist
* move - 
* copy - 
* remove -
   * for arrays, you need to know the index.  ex. /foo/1

• failure modes are not a part of the spec
• a little similar to XML patch or webdav

Implementation notes

• about a month for Matt to write the pointer, then patch api
  • Matt said he would open source the PHP lib in a non GPL license
• add and remove are the only base ops, all others are derived from that
• empty string in patch means the entire object
• watch out for pointer implementation wrt looking up parent (basename) to ensure replace is valid.
• Implement pointer spec first.
  • be careful with escaping, ex. resolve ~ last.
• consider cucumber for language families
• Should be atomic (wrt the update)
  • Use 202 like you do with normal REST, if the patch was accepted vs completed.
• it is possible to undo a patch
  • if your HTTP response returns an "undo" patch

Use cases

• Geo DNS: change territories as opposed to listing all in an update (more concise)
• Insurance language: hard to update what are otherwise very large strings (Emmanuel)
• Logging/Auditing:
• Mobile: GET a large structure is good for reads, but less so for PATCH
• Brandur: Should we use this for batch ops?
  • Rob: watch out for business rules; how this object is composed on the backend
  • Jeff: tried to do it, but the data size was too big.

Patch in the wild

public apis

• GitHub: PATCH, but not JSON patch
• Google Storage PATCH, but not JSON patch
• OpenStack

public sdks

• Java/Jackson: watch out LGPL
• Python: BSD
• Node/JavaScript: GPL
• watch https://github.com/quickenloans for potential OSS
• socialize your efforts if you make your own!

Questions/Concerns

• How to deal with the relationship between front-end datastructure and backend resources Ex. when a subtree of the json can be updated atomically, but other pieces not A: (Matt) might lead to data model refactoring
• Rob: Why have a spec for this?
  • Matt:
    • Helps to not have to know all the keys when doing an update
    • Helps with debugging or distributing change across nodes
• Brandur: Relationship between this spec and others?
  • Where url path and json path are related, this implies there are options for how PATCH is implied.
• Lukas/Jeff: queries are not stable. ex. list index or matching on values.
• Steve: point-cuts and patch. white list supported patch ops similar to HTTP 405

Interesting bits

• Arrays what happens when you add /foo/1?

{ "foo" : [ 42, 43 ] }

inserts at position 1!

{ "foo" : [ 42, 44, 43 ] }

what if you want to affect the tail? use index - ex. /foo/-

• unknown keys are fine Ex. you could add a reason for the change