Vizier Architecture

Vizier Architecture

Vizier's code is organized into three general categories:

• Frontend: This code is compiled with ScalaJS and run in the browser
• Backend: This code is compiled with Scala and run in the JVM on the command line
• Shared: This code is designed to be generic and used in the Front- and Back-ends

Vizier's design is heavily informed by the MVC paradigm:

• Model:
  • At rest in the database: The Backend's info.vizierdb.catalog.Schema class
  • In flight: The Shared info.vizierdb.serialized package
• View:
  • The Frontend
• Controller:
  • info.vizierdb.api
  • info.vizierdb.viztrails

Frontend

The main entrypoint into Vizier's frontend is the ui/html directory: Each HTML file defines a 'view'. The HTML file calls into info.vizierdb.ui.Vizier to actually render the view.

Styling is defined by ui/css/vizier.scss; The build process renders this into regular css.

ScalaTags

Most DOM rendering is done with ScalaTags. This is a set of methods that provide Scala-ish constructors for DOM nodes.

• Tags (e.g., a(...)) are methods that take attributes and content as arguments
  • Tags can be manually defined with the tag("tag_name")(arguments, ...) method.
• Attribute/Value pairs (e.g., href := "http://foo.com") are defined with the := operator.
  • Attributes that are reserved scala keywords (e.g., class) can be defined using backticks e.g., `class`)
  • Attributes can be manually defined with the attr("attr_name") := value method
• Content can be:
  • Another scalatag
  • A string
  • A dom node

The .render method on a scalatag converts it into a dom node. Note that .render will generate a fresh DOM node on each call.

ScalaRx

Reactive content is implemented with ScalaRX. Reactive programming can be a bit of an unusual paradigm for folks who haven't been exposed to it. The basic idea is that you draw the UI as if the state were completely static and unchanging. ScalaRX provides a few gimmicks that allow state changes to automatically re-generate only the portions of the DOM that need to be re-generated.

ScalaRX is centered on the rx.Rx trait. This denotes a reactive value that can be used inside a reactive block. The following pattern appears throughout the Vizier codebase:

Rx { 
  div(
    content
  ) 
}.reactive

The value of objects of trait val myValue:rx.Rx can be referenced as myValue() within an Rx block. These are reactive references. If one of those values changes, the entire Rx block will be re-evaluated. The .reactive extension method (defined in info.vizierdb.ui.rxExtras.implicits) generates a 'reactive' dom element that will automatically refresh itself when the Rx block is updated.

Widgets (package info.vizierdb.ui.widgets)

Widgets are helpful utility constructs to generate on-screen widgets that are not tied to a specific part of Vizier's model. They are grouped together to avoid code duplication, and to ensure consistent styling. Examples include:

• Expander: A content area + summary with a button that lets you toggle between them
• FontAwesome: A FontAwesome glyph
• PopUpButton: A button with a drop-down menu of actions. Unlike a regular drop-down menu, the resulting button is not a 'pick one' but rather a hidden menu of actions.
• ScrollIntoView: A convenience function to scroll a particular anchor into view.
• SearchWidget: A Trie-based autocompleting search tool.
• ShowModal: A modal dialogue box.
• Spinner: A "wait for it" spinner.
• SystemNotification: Utility methods to generate Browser Notifications.
• Toast: A quick informational (e.g., error) blurb at the top of the screen.
• Tooltip: A mouse-over tooltip.

Components (package info.vizierdb.ui.components)

Components are Views for specific Vizier constructs; In contrast to Widgets which are state-agnostic, components display specific state.

By convention, every component has a field named root that is a DOM node (i.e., a rendered ScalaTag). This field represents the component in the UI.

Network (package info.vizierdb.ui.network)

Code involved in communicating over the network.

Backend

The main entrypoint into Vizier's backend is info.vizierdb.Vizier.

Commands (package info.vizierdb.commands)

Commands are the core of Vizier; Each cell executes a command.

• Each command implements the info.vizierdb.commands.Command trait.
• Making a command available to Vizier requires adding an initializer (these are primarily in the info.vizierdb.commands.Commands object)

See the Cell Dev Guide for more details.

API (package info.vizierdb.api)

Vizier uses Scala Play to host its webserver. However, virtually all code that interacts with Play is automatically synthesized from vizier/resources/vizier-routes.txt. Adding a new route requires:

1. Add a route line to vizier-routes.txt
2. Implement a handler

The build script will automatically synthesize play code through scripts/build_routes.sc

Route lines are formatted in whitespace-separated fields:

1. The route's path (i.e., https://localhost:5050/vizier-db/api/v1[path listed here])

• A variable path element may be specified as {varname:type} (see types below)
• (optional) Get arguments may be provided as ?varname:type&varname:type&... (see types below)

2. HTTP Verb: GET, POST, PUT, or DELETE usually)
3. Category: These are used for organizational purposes (documentation, authentication, etc...). The value is not semantically meaningful, but you are discouraged from adding new categories.
4. Stub: This must be a unique name within the category. The category and the stub are used to generate accessor methods for the frontend, so it should be reasonably descriptive.
5. Handler: The name of the handler object. Customarily, these are all objects in the info.vizierdb.api package, and this prefix can be omitted.
6. Return Type: The value 'returned' by the API call

• Unit if no value is returned
• FILE[mime/type] or FILE[*] to stream back a file with a static or dynamic mime type, respectively.
• STRING: There's a reason for this, but I can't remember exactly why. I think it's something to do with Play.
• Any Scala type (The type must have a defined Play-JSON Format)

7. POST arguments:

• An underscore (_) to signify no post arguments
• A list of semicolon (;) delimited name:type pairs, where types are as in Return Type
• A type may be UndefOr[type] to denote an optional type. This is required because optional types require special handling at invocation sites. UndefOr makes the parameter optional in frontend accessor methods without typing hacks.

For path and get arguments, the routes file generator supports...

• long
• int
• string
• subpath: Unlike the other types, this may only be used at the end of a path, and generates a string containing all path elements following the prefix.

The routes file generates:

• Backend:
  • info.vizierdb.api.akka.RoutesFor*: One file per category including Play routing
  • info.vizierdb.api.akka.AllRoutes: Merges the RoutesFor* routes together
  • info.vizierdb.api.websocket.BranchWatcherAPIRoutes: Backend support to allow routes to be invoked directly over the websocket (sometimes helpful for sequencing operations).
• Frontend:
  • info.vizierdb.ui.network.API: Thin wrapper around the API.
  • info.vizierdb.ui.network.BranchWatcherAPIProxy: Thin wrapper around the websocket version of the API.

Scheduler (package info.vizierdb.viztrails)

This is how workflow execution is managed. Key classes include:

• info.vizierdb.viztrails.Scheduler: The main entrypoint.
• info.vizierdb.viztrails.RunningWorkflow: A thread that manages workflow execution.
• info.vizierdb.viztrails.RunningCell: A thread that manages the execution of one cell.

Commands optionally provide a ProvenancePrediction which is intended to bound the set of possible reads/writes that the command will perform (given specific input arguments). The scheduler (Specifically RunningWorkflow) uses this information to decide when it is possible to run two cells in parallel and/or which cells are invalidated and which can be re-used. See The Cell State Model for more details.

Catalog (package info.vizierdb.catalog)

The catalog contains all of Vizier's state. It's a thin ORM-style wrapper around a database (Sqlite by default). Access to the database is through ScalikeJDBC. See the overview for details.

In general, changes to the catalog should not be needed.