Skip to content

jjbrosnan/deephaven-core

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Deephaven Community Core

Deephaven Data Labs Logo

Deephaven Community Core is a real-time, time-series, column-oriented analytics engine with relational database features. Queries can seamlessly operate upon both historical and real-time data. Deephaven includes an intuitive user experience and visualization tools. It can ingest data from a variety of sources, apply computation and analysis algorithms to that data, and build rich queries, dashboards, and representations with the results.

Deephaven Community Core is the open version of Deephaven Enterprise, which functions as the data backbone for prominent hedge funds, banks, and financial exchanges.

  • Build CI
  • Quick CI
  • Docs CI
  • Check CI
  • Nightly Check CI

This README is intended to provide a high-level overview of the installation and use of Deephaven Community Core. For more detailed guides on the topics presented below, see our Community documentation.

Supported Languages

Language Server Application Client Application
Python Yes Yes
Java / Groovy Yes Yes
C++ No Yes
JavaScript No Yes
Go No Yes
R No Yes

Deephaven's client APIs use gRPC, protobuf, Apache Arrow Flight, and Barrage to handle ticking data. Users who wish to build their own client APIs can use these tools to do so.

The following list contains documentation links for installation instructions and more:

Install and run Deephaven

The Deephaven server can be installed and instantiated from Docker, from Python, or from source code.

From Docker

This is the easiest way to get started with Deephaven. For complete instructions, see our quickstart for Docker. The table below shows installation dependencies.

Dependency Version OS Required/Recommended
Docker ^20.10.8 All Required
Docker compose ^2 All Recommended
Windows 10+ Windows Required
WSL ^2 Windows Required

The quickest way to install and run Deephaven from Docker is with a single Docker command:

Python without Docker Compose

# Python
docker run --rm --name deephaven -p 10000:10000 ghcr.io/deephaven/server:latest

Groovy without Docker Compose

# Groovy
docker run --rm name deephaven -p 10000:10000 ghcr.io/deephaven/server-slim:latest

Users who wish to customize their deployment should use Docker Compose. Deephaven offers a multitude of pre-made docker-compose.yml files to choose from. To get started, all that's required is to download a file, pull the images, and start the server.

Python with Docker Compose

The base Python docker-compose.yml file can be found here.

mkdir deephaven-deployment
cd deephaven-deployment

curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/base/docker-compose.yml

docker compose pull
docker compose up

Groovy with Docker Compose

The base Groovy docker-compose.yml file can be found here.

mkdir deephaven-deployment
cd deephaven-deployment

curl -O https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/groovy/docker-compose.yml

docker compose pull
docker compose up

pip-installed Deephaven

Users who wish to use Python but not Docker should use pip-installed Deephaven. For users with Windows operating systems, WSL is not required to use Deephaven this way.

pip install --upgrade pip setuptools wheel
pip install deephaven-server deephaven-ipywidgets

Then, from Python:

from deephaven_server import Server
s = Server(port=10000, jvm_args=["-Xmx4g"]).start()

The input arguments to Server specify to bind to the Deephaven server on port 10000 and to allocate 4GB of memory to the server JVM.

Built from source

Users who wish to modify source code and contribute to the project should build Deephaven from source. For complete instructions, see How to build Deephaven from source.

Building and running Deephaven requires a few software packages.

Package Version OS Required/Recommended
git ^2.25.0 All Required
java >=11, <=22 All Required
docker ^20.10.8 All Required
docker compose ^2 All Recommended
Windows 10 (OS build 20262 or higher) Only Windows Required
WSL 2 Only Windows Required

You can check if these packages are installed and functioning by running:

git version
java -version
docker version
docker compose version
docker run hello-world

NOTE: Internally, the Java build process will use Gradle Auto Provisioning to download and use the appropriate Java version for building and testing.

NOTE: On Windows, all commands must be run inside a WSL 2 terminal.

Python

A Python virtual environment is highly recommended for building Deephaven from source. Additionally, the wheel is installed with pip and built with Gradle.

git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core
python3 -m venv /tmp/my-dh-venv
source /tmp/my-dh-venv/bin/activate
./gradlew py-server:assemble
pip install "py/server/build/wheel/deephaven_core-<version>-py3-non-any.whl[autocomplete]
./gradlew server-jetty-app:run

Groovy

The Groovy server is built with Gradle. -Pgroovy builds the Groovy server instead of Python.

git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core
./gradlew server-jetty-app:run -Pgroovy

Debugging

You can debug the server by adding the -Pdebug flag, and then attaching a debugger to port 5005. This can be used in conjunction with other flags. For example, if you wanted to debug a server and startup with Groovy:

./gradlew server-jetty-app:run -Pgroovy -Pdebug

Get the authentication key

Deephaven, by default, uses pre-shared key authentication to authenticate against unauthorized access.

Deephaven run from Docker

The pre-shared key is printed to the Docker logs when the server is started. Set your own key with the configuration parameter -Dauthentication.psk=<YourKey>. For users running Deephaven via Docker, this is set in the environment section of a docker-compose.yml file, or as a space-separated configuration parameter at the end of the docker run command.

To find the pre-shared key in the Docker logs:

docker compose logs -f | grep "access through pre-shared key"

Deephaven run from Python

When a Deephaven server is started from Python, executing Deephaven queries from Python does not require the key. However, if you wish to connect to the IDE via your web browser, you will need the pre-shared key. You will not be able to get the pre-shared key unless you set it yourself. To set the pre-shared key, add "-Dauthentication.psk=<YourKey>" as an additional JVM parameter to the server. The following example sets the key to MyPreSharedKey:

from deephaven_server import Server
s = Server(port=10000, jvm_args=["-Xmx4g", "-Dauthentication.psk=MyPreSharedKey"]).start()

Client APIs

Clients that attempt to connect to a server using pre-shared key authentication will need to supply the key to complete the connection. The key is the same for a client connection as it is for connecting directly to the server. For instance, in the above example, the key for a client connection would also be MyPreSharedKey.

Connect to the server

The Deephaven UI is accessible from a web browser. For a server running locally on port 10000, it can be connected to via https://localhost:10000/ide. For a server running remotely on port 10000, it can be connected to via https://<hostname>:10000/ide. If using authentication, enter credentials to gain access to the IDE. For information on supported browsers, see here.

First query

From the Deephaven IDE, you can perform your first query.

The scripts below create two small tables: one for employees and one for departments. They are joined on the DeptID column to show the name of the department where each employee works.

Python

from deephaven import new_table
from deephaven.column import string_col, int_col
from deephaven.constants import NULL_INT

left = new_table([
        string_col("LastName", ["Rafferty", "Jones", "Steiner", "Robins", "Smith", "Rogers"]),
        int_col("DeptID", [31, 33, 33, 34, 34, NULL_INT]),
        string_col("Telephone", ["(347) 555-0123", "(917) 555-0198", "(212) 555-0167", "(952) 555-0110", None, None])
    ])

right = new_table([
        int_col("DeptID", [31, 33, 34, 35]),
        string_col("DeptName", ["Sales", "Engineering", "Clerical", "Marketing"]),
        string_col("Telephone", ["(646) 555-0134", "(646) 555-0178", "(646) 555-0159", "(212) 555-0111"])
    ])

t = left.join(right, "DeptID", "DeptName, DeptTelephone=Telephone")

alt_text

Groovy

left = newTable(
        string_col("LastName", "Rafferty", "Jones", "Steiner", "Robins", "Smith", "Rogers"),
        int_col("DeptID", 31, 33, 33, 34, 34, NULL_INT),
        string_col("Telephone", "(347) 555-0123", "(917) 555-0198", "(212) 555-0167", "(952) 555-0110", null, null)
    )

right = newTable(
        intCol("DeptID", 31, 33, 34, 35),
        stringCol("DeptName", "Sales", "Engineering", "Clerical", "Marketing"),
        stringCol("Telephone", "(646) 555-0134", "(646) 555-0178", "(646) 555-0159", "(212) 555-0111")
    )

t = left.join(right, "DeptID", "DeptName, DeptTelephone=Telephone")

alt_text

Resources

Contributing

See CONTRIBUTING for full instructions on how to contribute to this project.

Code Of Conduct

This project has adopted the Contributor Covenant Code of Conduct. For more information see the Code of Conduct or contact [email protected] with any additional questions or comments.

License

Copyright © 2016-2023 Deephaven Data Labs and Patent Pending. All rights reserved.

Provided under the Deephaven Community License.

About

Deephaven Community Core

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Languages

  • Java 93.6%
  • Python 2.9%
  • C++ 1.2%
  • R 0.6%
  • FreeMarker 0.5%
  • C# 0.4%
  • Other 0.8%