A tool for visualising and exploring Bazel Skyframe graphs.
Has support for arbitrary dot
graphs too.
As a prerequisite you will need the graphviz
package installed and
PATH
set so that dot
can be found. Use your preferred package
manager to do this, e.g. for Ubuntu:
apt install graphviz
Or for MacOS:
brew install graphviz
If you're running Linux, you can download Skyscope as a prebuilt statically linked binary:
-
Go to the releases page and pick the version you want. The latest is v0.4.3.
-
Download the zip archive and unpack it somewhere; e.g.
unzip ~/Downloads/skyscope.zip -d ~/.local/
- Update your
PATH
variable appropriately; e.g. append this to~/.bashrc
:
export PATH="$HOME/.local/skyscope/bin:$PATH"
Option 2: Build and run it from source (requires Nix)
If you're running MacOS, you'll need to build Skyscope yourself:
- Clone the repository:
git clone https://github.com/tweag/skyscope.git
- Build the backend binary:
nix develop --command bazel build //backend:skyscope
After the build completes the binary can be found at bazel-bin/backend/skyscope
.
The Skyframe graph is stateful. Most bazel
commands you can run will have
some effect on it.
To view a Skyframe graph you must first import a snapshot of its current state
into Skyscope. If you have skyscope
in your PATH
variable, this can be done
by running skyscope import
under the workspace.
Depending on the size of the Skyframe graph, the import process might take a
few minutes. Note that the graph is initially empty when the Bazel server
starts and nodes are added as required when you run a Bazel command. So if
importing is taking too long, try doing bazel shutdown
followed by a minimal
sequence of commands to repopulate only the parts of the graph you are
interested in.
When the import process is complete you will be prompted to open a link in your browser. A list of previously imported graphs can be found at http://localhost:28581 and you can also delete imports from here when they are no longer wanted.
When you first view a newly imported graph, all its nodes will be hidden. So
you must use the search box to find and display nodes of interest. The pattern
you enter here is matched against node keys, as they are printed by bazel dump --skyframe
. You may use %
as a wildcard.
As you type, the list of results will be dynamically updated and the matching part of each key highlighted. To keep the interface responsive only a few hundred results are shown; so if you do not see the node you want, try making the search pattern more specific.
Clicking on an entry will toggle its visibility and the graph will be immediately updated behind the search box. The search box will stay open until you press escape or click elsewhere so you can continue adding nodes.
By default nodes are in the collapsed state, which means only edges connected to other visible nodes are displayed. This helps keep the complexity of the graph manageable. Visible nodes may be toggled between the collapsed and expanded states by clicking on them.
When a node has been expanded all its edges are displayed, including edges connected to hidden nodes. Hidden nodes are represented by small unlabelled circles. You can click on these circles to make the hidden nodes visible. To make all the hidden neighbours of an expanded node visible at once, double click on it (this may take several seconds for a highly connected node).
If one connected component of the graph depends on another, the dependency path will be represented by a dotted edge between the components. You can make the nodes on the dependency path visible by clicking Open:
This feature can be used to discover how a particular target depends on
another, or how an action depends on a file. It works much like a somepath
Bazel query.
Collapsing nodes can help keep the size of the graph manageable, but it will still grow too complex from time to time. When this happens you can crop the graph to a smaller selection of nodes:
To do this, press and hold the shift key while you make your selection. Upon releasing the shift key, the graph will be updated and only the selected nodes will be visible. You can also hide individual nodes by ctrl clicking.
Every action you take in exploring the graph is added to your browser history. So if you make a mistake, you can undo it by hitting the back button (the forward button will allow you to redo an undone action). This feature is also useful when an action causes a large transformation to the graph. Nodes are animated into their new positions, so jumping back and forth a few times with keyboard shortcuts1 and watching how the nodes move can help you orientate yourself in the new layout.
For graphs with a small number of visible nodes, rendering is fast enough to be nearly seamless. For larger graphs it might take several seconds to finish rendering an update. If rendering is taking too long, you can click on the spinning hourglass in the lower right corner to interrupt it.
The graph will then be restored to the last checkpoint. A checkpoint is created whenever rendering completes successfully. It will also be restored when you reopen a closed graph tab (e.g. after restarting your browser). Checkpoints are stored in local storage, so clear that if you want to start afresh.
Unless Skyscope is in the middle of rendering a graph update, the lower right corner will contain a save icon. Click it to save a static copy of the graph as an SVG image in your downloads folder. This image is best viewed in a web browser (not all image viewers fully support the CSS embedded in it).
When you run skyscope import
the Skyscope server is automatically started (or
restarted if it was already running). If you wish to start the server without
importing a new graph, just run skyscope server
.
The output produced by bazel dump --skyframe
is sufficient to determine the
Skyframe graph topology, but for many node types it does not provide the full
context. For example, an ACTION_EXECUTION
node only has a reference to the
CONFIGURED_TARGET
that created it, a BUILD_CONFIGURATION
, and an
actionIndex
:
ACTION_EXECUTION:ActionLookupData{
actionLookupKey=ConfiguredTargetKey{label=//src/main/java/com/google/devtools/build/lib/bazel:BazelServer,
config=BuildConfigurationValue.Key[29162d16f36425edb5387766d6f9e873585a5b890f0ae3a9e778941f90411445]},
actionIndex=5
}
The actionIndex
field differentiates this ACTION_EXECUTION
from others
created by the same CONFIGURED_TARGET
, but on its own is not very
illuminating. We can use bazel aquery
to get a list of actions and then
correlate it with actionIndex
to recover the missing context:
action 'Creating runfiles tree bazel-out/k8-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer.runfiles'
Mnemonic: SymlinkTree
Target: //src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Configuration: k8-fastbuild
Execution platform: //:default_host_platform
ActionKey: 7d501d61ad3623eb44a3f87523c2d681961adf84fb47759e558af0df2d5249d1
Inputs: [bazel-out/k8-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer.runfiles_manifest]
Outputs: [bazel-out/k8-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer.runfiles/MANIFEST]
A similar process with bazel query
provides additional context for
CONFIGURED_TARGET
nodes. By default Skyscope will attempt to import this
information by running the following when you do skyscope import
:
bazel aquery 'deps(//...)' # Get additional context for action executions
bazel query 'deps(//...)' --output build # Additional context for targets
If either of these commands fail (e.g. because some matching targets are
broken) the import will still continue, but the additional context will be
missing. In this situation you can use the --aquery
and --query
parameters to specify the queries Skyscope should run:
skyscope import --query='//src/...' --aquery='//src/main/...'
You can also pass --no-query
and --no-aquery
to disable importing of
additional context entirely. This is useful when you want to import a Skyframe
graph without affecting it at all (since the bazel query
and bazel aquery
commands can themselves cause nodes to be added to the graph).
The default theme is embedded in the Skyscope binary, but you can make it use a
local file instead by setting the SKYSCOPE_THEME_CSS
environment variable
(probably in ~/.bashrc
or equivalent). Note that if the Skyscope server is
already running, it will not pick up the new value until it is restarted
(pkill -f 'skyscope .* server'
).
It is recommended that you begin by making a local copy of the default theme and edit that as needed. Any changes you make can be checked immediately by refreshing your browser. The relevant section for colours is this:
div.ResultRow.ActionExecution span.NodeTitle { color: hsl(318, 55%, 29%); }
div.ResultRow.ConfiguredTarget span.NodeTitle { color: hsl(117, 55%, 29%); }
div.ResultRow.FileState span.NodeTitle { color: hsl(271, 55%, 29%); }
g.node.ActionExecution text.NodeTitle { fill: hsl(318, 55%, 29%); }
g.node.ConfiguredTarget text.NodeTitle { fill: hsl(117, 55%, 29%); }
g.node.FileState text.NodeTitle { fill: hsl(271, 55%, 29%); }
Set color
for div.ResultRow
selectors to change the colour of a particular
node type in the search box results. The g.node
selectors set the fill
colour of the node title in the graph. While div.ResultRow
can be set
independently from g.node
, for consistency they should be the same.
The majority of the frontend is written in Purescript, but the formatting of
node labels is done in Javascript to allow easy customisation. If you wish to
change this formatting, you can make a local copy of the default format
file and
set the SKYSCOPE_FORMAT_JS
environment variable appropriately. For reference,
a minimal format.js
would be:
// You have access to a `node` object with the following fields:
// node.type Camelcase version of SkyKey.functionName()
// node.data Raw SkyKey as printed by bazel dump
// node.label First label extracted from node data (possibly empty)
// node.context Extra context, if there is any (e.g. from query or aquery)
const title = "" // Use default (which is node.type)
const detail = "" // Use default (which is node.label if non-empty, or node.data otherwise)
const tooltip = "" // Use default (which is node.data)
return { title, detail, tooltip };
Returning an empty string for one of the fields will cause the default
formatting to be used for that field. The four input fields can be parsed and
combined as you wish to produce the output fields. You might want to add a
temporary console.log(node)
statement to see what the input fields look like
in your browser console.
Variable Name | Description |
---|---|
SKYSCOPE_PORT |
Port on which the Skyscope HTTP server should listen. Defaults to 28581 . |
SKYSCOPE_DATA |
Directory where imports, server pid file, and other data should be stored. Defaults to $HOME/.skyscope if not set. |
SKYSCOPE_DEBUG |
Execution tracing is enabled for the wrapper scripts when this variable is set. |
SKYSCOPE_FORMAT_JS |
Set this to the path of a Javascript file to override the embedded node formatting. |
SKYSCOPE_THEME_CSS |
Set this to the path of a CSS file to override the embedded theme. |
SKYSCOPE_BAZEL_BIN |
Use this bazel binary instead of whatever is found in your PATH. |
In addition to importing Bazel Skyframe graphs, Skyscope also supports
importing arbitrary graphs in Graphviz format (provided they can be parsed by
the Haskell Graphviz library).
Just send the graph on standard input to skyscope import-graphviz
, e.g.
skyscope import-graphviz < graph.dot
Or via a pipe:
echo 'digraph G { A -> B; }' | skyscope import-graphviz
You can give labels to nodes, and by convention the first line becomes the node title. For example, the following graph definition:
digraph G {
A [ label="Alpha\nDetail for alpha" ];
B [ label="Beta\nDetail for beta" ];
A -> B;
}
Looks like this in Skyscope:
The backend is split into separate server and import processes. This allows a single server process to be used across multiple workspaces. The import process extracts the Skyframe data from Bazel and inserts it into an Sqlite database. After this is done it notifies the server of the newly imported graph. The server keeps track of all imported graphs in a central Sqlite database.
The frontend is responsible for storing the node configuration (i.e. which nodes are visible) and whenever this changes it sends a request to the backend to rerender the graph (Graphviz is used for this). The response is an unstyled SVG image which the frontend then decorates with CSS and various event handlers.
You can contribute by reporting bugs or requesting features you would like to see in the GitHub issue tracker. Pull requests are also welcome, but for non-trivial changes please discuss the change you have in mind first so we can agree on an approach. If there is an existing issue you can comment on that, otherwise you can open a new issue.
Footnotes
-
Usually Alt + Left Arrow to go back and Alt + Right Arrow to go forward. ↩