Requesting Data v1

Butterfly can support multiple query schemes, which can be accessed directly via a web browser, or software clients like VAST.

Default

The default query scheme is a standard url query string with the following required queries:

Structure:

servername:port/data/?datapath=DATAPATH&start=x,y,z&size=dx,dy,dz

• DATAPATH
  • The complete path to the data on the physical server. You can add physical storage with data without restarting the server, and clients can simply provide the path upon request.
• x,y,z
  • The starting coordinates of the volume being requested. Indexing begins at 0, and coordinates are comma-separated without spaces.
• dx,dy,dz
  • The size of the volume requested. This is relative to the current zoom (mipmap) level in the xy plane. For example, size=512,512,10&mip=0 and size=256,256,10&mip=1 will return the same physical volume.

Additional queries:
Queries are separated by '&', and are assigned their value by '='. The order does not matter. The following optional queries are supported:

• &output=FORMAT
  • Currently supported formats:
  • zip - binary zipped data
  • (png, jpg, jpeg, tiff, tif, bmp) - Single slice return as image. More formats can be configured in settings.py
• &mip=LEVEL
  • Zoom by mipmap level. 0 is full resolution, 1 is half resolution, etc.

The following flags will take "yes", "y", or "true" as FLAG to turn them on, but are off by default.

• &fit=FLAG
  • The volume will be fitted to the existing data in case of requests that are out of bounds. If this is not specified or any other value, empty areas are filled with 0 and the entire requested volume is returned.
• &segmentation=FLAG
  • If available, the segmentation will be returned instead.
• &segcolor=FLAG
  • For segmentations, segment IDs will be converted to RGB colors based on the colormap in the data set.

Open Connectome Project format (RESTful Arguments)

Butterfly also supports the Open Connectome Project volume cutout service format. The format is essentially identical:

Structure:

servername:port/data/DATAPATH/ocp/FORMAT/resolution/xlow,xhigh/ylow,yhigh/zlow,zhigh/

Note that the entire "folder to the data" is considered /data/DATAPATH/ocp/, with the standard arguments following after. As in the case of the default query, DATAPATH should not be quoted (slashes in the url are okay, as well as datapaths from windows file systems).

Currently resolution = mipmap level, which means 0 is full resolution. This can be changed if necessary.

Use with VAST

Integration with VAST is possible in two ways. Since butterfly supports the OCP format, we can use this directly in a VAST online server configuration file.

For example, let's say butterfly is at testserver.me/butterfly, and our datapath is /home/user/EM/mojo. This would usually mean that the OCP request is testserver.me/butterfly/data/home/user/EM/mojo/ocp/[ARGUMENTS]. We make a configuration file like so:

{
  "Comment": "Source: http://testserver.me/butterfly/data/home/user/EM/mojo/ocp",
  "ServerType": "openconnectome",
  "ServerName": "testserver.me",
  "ServerFolder": "/butterfly/data/home/user/EM/mojo/ocp",
  "SourceDataSizeX": 21504, 
  "SourceDataSizeY": 26624,
  "SourceDataSizeZ": 1850,
  "TargetDataSizeX": 10747,
  "TargetDataSizeY": 12895,
  "TargetDataSizeZ": 1850,
  "OffsetX": 0,
  "OffsetY": 0,
  "OffsetZ": 0,
  "OffsetMip": 1,
  "TargetVoxelSizeXnm": 6,
  "TargetVoxelSizeYnm": 6,
  "TargetVoxelSizeZnm": 30,
  "TargetLayerName": "testdata"
}

In other words, simply take the entire "folder to the data" name (as defined in the OCP section of this page) as the ServerFolder.

As of VAST version 1.01c, butterfly is also natively supported as a folder type, offering more clarity in the configuration file and explicit port designation. The above example using the butterfly ServerType is:

{
  "Comment": "Source: http://testserver.me/butterfly/data/?datapath=/home/user/EM/mojo/",
  "ServerType": "butterfly",
  "ServerName": "testserver.me",
  "ServerPort": 80,
  "ServerFolder": "/butterfly/data/",
  "DataPath": "/home/user/EM/mojo/"
  "SourceDataSizeX": 21504, 
  "SourceDataSizeY": 26624,
  "SourceDataSizeZ": 1850,
  "TargetDataSizeX": 10747,
  "TargetDataSizeY": 12895,
  "TargetDataSizeZ": 1850,
  "OffsetX": 0,
  "OffsetY": 0,
  "OffsetZ": 0,
  "OffsetMip": 1,
  "TargetVoxelSizeXnm": 6,
  "TargetVoxelSizeYnm": 6,
  "TargetVoxelSizeZnm": 30,
  "TargetLayerName": "testdata"
}

Note the addition of the ServerPort and DataPath parameters.