Home
Geoflow is a spatial ETL tool for processing 3D geo-information such as point clouds and 3D city models.
I started developing Geoflow while was working on an algorithm for automatic 3D building reconstruction from point clouds (what currently underpins 3DBAG.nl )). This turned out to be a complex process consisting of many small steps that are executed sequentially. The first step is to read the input datasets (eg. point cloud) and the last step to write the 3D building model to a file. And, most importantly, in between there are many steps that take some data, process it and output new data. At the beginning of my work I did not know what the final algorithm would look like. And as I was working on the reconstruction algorithm and trying different ideas I quickly realised that it was very important to me to be able to quickly
- (visually) inspect and validate the output of each step in the process,
- reconfigure the behaviour of one step and immediately see the effects on the rest of the pipeline, and
- replace the operations in one steps with an alternative implementation, or remove and add steps all-together.
And thus Geoflow was born. I built it to be fast, interactive, and modular. It consists of a GUI application that allows one to build and run 'flowcharts'. Flowcharts contains pipelines of 'nodes' (each node is a 'step' that performs a certain operation as described above) and connections between nodes (to direct how the data flows). A command line application also exists that can be used to run flowchart files (human readable JSON) on headless servers.
Geoflow is designed around the idea of the flowchart. You create a flowchart by building graphs of nodes and connections.
- A node is a processing unit. It takes data from its input terminals (on the left side of the node), performs some processing, and delivers the results on the output terminals (on the right side of the node). Nodes can be created thought the menu that appears when right clicking on the flowchart canvas.
- With connections one can connect output terminals to input terminals of other nodes. Connections can be created by dragging from one terminal to another terminal
You can start a node in Geoflow by clicking on the Run button in the menu that appears when you right click on a node. The node will then start processing. Once a node has finished processing it will turn green and the output results are pushed to the output terminals. When a terminal has data pushed to it, it wil also turn green. In case there is a connection on an output terminal, the input terminal on the other end of the connection will also be notified that data is available, and it will also turn green. A node can only start processing when all its input terminals have data/are green.
The example above shows a simple flowchart with three nodes. There are two Number nodes on the left and they are connected to an Adder node (which reads the numbers on the input terminals and outputs their sum). The two number nodes are special in the sense that they do not have input terminals. They are therefore called root nodes. In this example the Number nodes simply set a user-configurable number (a parameter) on the output terminal. In practice, root nodes often read data from disk and output that data.
You can see that the Adder node automatically turns green once we manually run the two Number nodes. This is the default behaviour: once the input terminals are green a node automatically starts processing.
But, beware that a node can only start processing when all its input terminals are ready, ie. there is data available on each terminal.
- nodes and connections.
- processing order
- node parameters and globals*
- saving and loading a flowchart
Loops are forbidden Connect/disconnect events immediately affect status of child nodes If all inputs are set, a node will automatically run Only nodes without inputs need to be ran manually Re-running a node will also re-run all child nodes
geof <flowchart file> [--config <TOML config file with globals>] [--GLOBAL1 <value> --GLOBAL2 <value> ...]
You can also simply print just information on the plugins that are loaded with:
geof info
Takes the same parameters as geof on the command line, but opens a GUI instead of simply running a flowchart.
Warning: the GUI is currently experimental.
- Right click to open the menu to create new nodes
- Drag from input/output terminals to make connections
- Right click on a node to access its context menu
- Translate in the 3D viewer by left-mouse dragging while holding
ctrl, faster zooming by holdingctrlwhile scrolling.
The geoflow-bundle repository bundles geoflow together with the actively developed plugins and some example flowcharts (eg. for building reconstruction). With each release of this repository some binary packages become available:
Geoflow can be compiled for all major platforms (Linux, Mac, Windows). We prioritised the above packages, but we aim to provide readily usable binaries for other platforms too in the future.