Batch changes requests to resources

Convener

Jeff Damick @jdamick

Attendees

lots..add yourself

@pedro has good notes

Notes

• Problems:
• Want to be able to update multiple resources atomically
• Want to be able to retrieve multiple resources in a single call
• Explored some examples
• https://developers.facebook.com/docs/reference/api/batch/
• https://developers.google.com/storage/docs/json_api/v1/how-tos/batch
• http://docs.neo4j.org/chunked/milestone/rest-api-batch-ops.html
• http://docs.aws.amazon.com/cloudsearch/latest/developerguide/DocumentsBatch.JSON.html

Question came up: Do you need the operations to atomic?

• One example came out

  • Asynchronous
  • Return 202, ticket to get result of each operation

• Rest.li - supports batch operations

• they tolerate individual failures within the batch

• Another suggestion: remodel data to not span multiple resource or virtual model over both

• Why are people using batch operations?

• Performance considerations for mobile or internally to relieve pressure on backend services

• Caching: can't use HTTP caches, need to user server-side caching techniques for parts of the response

• Cassandra 2.0 supports a light transactional model, as well as DBs, so consumers want parity in their APIs with their backends.

• Suggestion:

• Rather than support atomicity, write the children & children of children first, then remove the old ones.

• Rest.li Approach

• Not strictly RESTful

• minimal envelope

• serial / ordered operations

• 1 set of headers (can be an issue)

• Acts on 1 Resource

• Purely an optimization

• GET Example: /url?ids=1&ids=2&ids=3

• response contains the id mapped to entity response & response code

• Batch support for PUT, POST and DELETE as well

Protocol level support? using SPDY

• no one (admitted) to running SPY on the services
• Possible solution, since operations can be pipelined, cached, and other protocol level benefits

Why else to use SPDY?

• header compression

• multiple resource retrieval

• Another use came out:

• Migrating data across entire systems

• Do I create children and then children of children in the new system or just move the whole thing in bulk?

• Is bulk loading a special case?

• create the new data, hyperlink as appropriate

• Bulk loading

• most agreed that bulk loading should be treated as a special case, and you can do it async

• could use an orchestrator like service-mix or mule

• make sure your resources have unique id's

• question: can you just hit the next resource up a level?

• we were exposing metadata at that level, otherwise the data size can get unwiedly