Home

One of the central components of the Eddy platform is represented by the Eddy User Interface. Its main functionality consists of web application allowing projects management, data pipelines building and dashboards creation. The project is built using the Vue.js JavaScript framework and acts as a client for the GraphQL API endpoints of the Eddy Kafka Bridge and Eddy Admin components.

Considering the extensive set of features that Vue.js provides, the flexible way of structuring the project imposed, and the fact that it is the most popular JavaScript front-end framework at the moment, creating Eddy UI based on Vue provides a solid and scalable foundation for an open-source project. The project is a single-page web application (SPA) structured into reusable single-file components, singleton services, a shared data store, and a router. The components are used in combination to create “pages” that are linked together by a virtual router which still provides predictable resourceful URLs despite the fact that the entire application is a single HTML page. The communication with the backend services is established by using Vue Apollo, a wrapper for Apollo, a popular client for GraphQL APIs.

The choice of using GraphQL over a more traditional REST architecture was mainly driven by the flexibility of performing custom queries requesting only the data needed in the response and the ability to establish subscriptions. Keeping long-lived requests as subscriptions between the frontend and the backend over WebSockets allows the server to push new data to the client when available. In contrast, a pulling approach, usually used in combination with REST, involves constantly making requests on a time interval, despite the fact that new data might not be available. Furthermore, GraphQL allows performing queries and mutations validation against a predefined schema improving the development experience and ensuring that the queries performed are always valid. The features described above, the increasing popularity and the good support from the community determined to choose GraphQL as the main communication protocol between our UI and the other services.

From a functional point of view, the application can be divided into the following parts: resources management, pipeline builder and dashboard builder. The resources management part allows performing CRUD operations on projects, integrations, data connectors, pipelines and dashboards. The screen capture inserted below displays a project with a MySQL Server data connector added as a data source, different pipelines for data transformations and two dashboards for presenting the transformation results. In this screen, the users that have access to the project can add, edit (rename) and remove data connectors, pipelines and dashboards.

Project Management View - Screen Capture

The pipeline builder consists of a visual interface that allows creating DAGs of building blocks by dragging and dropping the blocks and drawing directed edges (arcs) between them. Based on the structure of the DAGs Flink or Beam jobs are generated. Walking the graph generates parts of the jobs configuration from the properties of the reachable blocks by starting from transformation blocks. By pressing the execute button the pipeline is being executed by submitting the generated jobs to a GraphQL endpoint.

Pipeline Builder View - Screen Capture

After adding a block, specific properties can be edited depending on the block type. For example, a DB Connector Stream block allows the selection of a data connector, a database and a table and generates the input topic corresponding to the selection. It also allows viewing sample events and defining the schema of the stream by either inferring it from a JSON or CSV sample event or defining it manually using the dropdown controls.

Block Properties Modals Examples - Screen Captures

The dashboard builder part enables the creation of real-time dashboards consisting of multiple widgets mainly represented by charts: pie, bar, area, etc. The widgets can be resized and moved around according to the layout preferences of the user. The data plotted in the widgets is received by subscribing to a Kafka topic via the Kafka GraphQL Bridge. It is also possible to subscribe to data starting from a certain point in time by selected a starting date in the date picker. A dashboard can be also shared in a view-only mode which does not allow adding or editing widgets but allows moving and resizing.

Dashboard Builder View - Screen Capture