Tools for parsing and viewing PITCH messages
Data file to upload for testing with online demo - data_med
CBOE provide services for rapidly trading stocks, and they publish specifications detailing their proprietary message formats - only available as PDFs :/ they also do not have software SDKs for public use. This repo addresses the issue by offering:
- YAML files representing messages as defined from the official PDFs (only a few are done, as examples)
- Kaitai Struct tooling, turning the YAMLs into SDKs for parsing data in Python/Javascript/C++/more
- A Vue app that uses our new sdk for parsing uploaded data and viewing it in a debug environment
Example PITCH data feed:
S28800012ABK27GA00000LB001000SSO 0000763600Y
S28800012ABK27GA00000MS001000SSO 0000764800Y
S28800012AAK27GA0000DUS001000SDS 0000549300Y
S28800012AAK27GA0000DVB001000SDS 0000548000Y
...
The first line is an Add Order (short) 4.3 message, for which we have:
- structs/specs/4.3-short.yml file defineing it's fields and their types
- a javascript file with functions to parse 4.3 formatted data. the javascript file is generated by Kaitai
- The Vue debugging tool will parse uploaded data and display the 4.3 message as:
OS X & Linux:
$ cd /client
$ yarn install
yarn install v1.7.0
[1/4] Resolving packages...
[2/4] Fetching packages...
...
$ npm run serve
...
App running at:
- Local: http://localhost:8080/
- Network: http://10.0.0.200:8080/The code is split into the SDK building, and then the Viewing app:
- Converting the PITCH Message Specifications into usage python/javascript parsers/models
/structs/specs- YAML's built by referencing the official PDF specsnotebooks/Building Structs.ipynb- Notebook to build YAMLs into py/js models
- Web interface/app/viewer to view uploaded message data
/client/- Vue.js app (pictured above)structs/compiled/Cboe.js- Generated in step 1 , imported by web app
Initially I went the route of making the python models and types for a backend parsing service, as I'm primarily a python developer. But copy-pasting each value into hardcoded classes when there's clearly an existing spec somewhere isn't what I'd normally do. So I did the YAMLs, and referenced those as the specs in the vue app. More about this in notebooks/Building Structs.ipynb. I left code from initial run in /cboe-sdk/ and notebooks/Modeling.ipynb.
This feels very similar to ACORD AL3, which is a popular fixed length messaging standard used by insurance software for the transmission of policies and claims. I created the AL3 implementation during my time at BriteCore .
The idea for the viewer comes from working with AL3, as there are thousands of different data/message types and no straightforward tool for displaying a feed of messages and their fields.