Skip to content

Latest commit

 

History

History
288 lines (231 loc) · 11.7 KB

Development-Guide.md

File metadata and controls

288 lines (231 loc) · 11.7 KB
Raw

Development Guide

This document explains the structure of the kRPC codebase, how to build it and the various tools.

If you are not already familiar with how kRPC works, it is recommended that you first read the documentation on the website:

Code Structure and Components

kRPC consists of many components, including the server (which runs in the game), service DLLs (which add RPCs to the server), client libraries for each language (Python, C#, C++, ...), documentation and various tools (for client developers and kRPC developers).

The codebase is structured as follows:

  • server/... - C# implementation of the main server that runs in the game.
  • service/... - C# implementations of the various services that run in the game, and add RPCs to the server.
  • lib/... - symlinks (or copies) of DLLs required to build the C# projects.
  • client/... - the client libraries, with one directory for each language.
  • client/serialio/... and client/websockets/... - tests for the serialio and websockets communication protocols.
  • doc/... - source code for the documentation, from which the website and pdf are built. Includes auto-generate API documentation for all of the client libraries, generated by the docgen tool (see below).
  • protobuf/... - the schema for the protobuf communication protocol.
  • tools/TestServer/... - implementation of a standalone C# executable that can be used to run a server without needing the run the game. This can be used by client developers to test that the client can communicate with the server correctly.
  • tools/ServiceDefinitions/... - a C# binary that can be used to generate a service definition file (in json format) from a service DLL.
  • tools/krpctools/... - a Python package with various tools for client libraries. Includes docgen for auto-generating client documentation, and clientgen for auto-generating client stubs.
  • tools/krpctest/... - a Python package with various tools for automated integration testing of RPCs running in the game.
  • tools/TestingTools/... - a service DLL that can be loaded into the game, to add RPCs that can be used for automatic testing. For example, adds an RPC that allows you to teleport a vessel into orbit. This is used by krpctest.
  • tools/build/... - scripts for the Bazel build system
  • tools/docker/... - scripts to create the buildenv docker image

Compiling

kRPC uses the Bazel build system. This provides us with fast, repeatable builds, and support for many languages. (See below for a Bazel cheat sheet.)

Note: we don't currently support building on Windows, although this is something we are working on. However, you can build the C# projects on Windows. See the section below named "Building the C# projects using an IDE". You can also build the project using a docker container running on Windows. See the section below named "Building using Docker".

Setting up your Environment

The Bazel build scripts will automatically download most of the required dependencies to build the project, but the following needs to be installed on your system:

  • Bazel
  • C# compiler, runtime and tools (for example Mono)
  • Python 3.7+
  • Autotools
  • LuaRocks
  • Maven
  • pdflatex and svg tools (for building the documentation)

On Ubuntu, these can be installed via apt as follows:

  • First follow the instructions on Mono's website to add their apt repository.
  • Then run the following command: 
    sudo apt-get install mono-complete python-is-python3 python3-dev python3-setuptools \
  python3-virtualenv autoconf libtool luarocks maven latexmk texlive-latex-base \
  texlive-latex-recommended texlive-fonts-recommended texlive-latex-extra texlive-fonts-extra \
  tex-gyre libxml2-dev libxslt1-dev librsvg2-bin libenchant-2-2 build-essential make

You also need to set up the necessary libraries in the lib diretory:

  • lib/ksp should contain a copy of the game. You can either create this directory, and copy the game files into it, or create a symlink to point to an existing copy of the game. For example, run the following command from the root of the krpc source to point to KSP installed in the default Steam location: ln -s "$HOME/.local/share/Steam/steamapps/common/Kerbal Space Program" ksp lib/ksp

You may also need to modify the symlink at lib/mono-4.5 to point to the correct location of your Mono installation. On Ubuntu 22.04 with the latest version of Mono, lib/mono-4.5 should be a symlink pointing to /usr/lib/mono/4.5

Building using Bazel

To build the kRPC mod release archive, containing the server DLL and service DLLs, run bazel build //:krpc. The resulting archive containing the GameData directory will be created at bazel-out/krpc-<version>.zip

The build scripts define various "targets" that can be used to build different parts of the project, by running bazel build <target> These targets are available:

  • //server - builds the server plugin and associated files
  • Client libraries:
    • //client/csharp
    • //client/cpp
    • //client/java
    • //client/lua
    • //client/python
    • //client/python:python-base - builds the python client, without pre-generated stubs
  • Services
    • //service/SpaceCenter
    • //service/Drawing
    • //service/UI
    • //service/InfernalRobotics
    • //service/KerbalAlarmClock
    • //service/RemoteTech
  • Protobuf definitions:
    • //protobuf:csharp
    • //protobuf:cpp
    • //protobuf:java
    • //protobuf:lua
    • //protobuf:python
  • Documentation:
    • //doc:html
    • //doc:pdf

Building the C# projects using an IDE

A C# solution file (kRPC.sln) is provided in the root of the project for use with Visual Studio, MonoDevelop or a similar C# IDE.

Before it can be used, some generated C# source files need to be built using the Bazel build scripts. This can be done by running bazel build //:csproj

Alternatively, if you are unable to run Bazel to build these files (for example on Windows), you can get a copy of these files from the "genfiles" archive on the (GitHub releases page)[https://github.com/krpc/krpc/releases]. Download the latest krpc-genfiles-<version>.zip and extract the archive over your copy of the source. This should give you a bazel-bin and bazel-krpc directory containing all the necessary files.

Building using Docker

A docker image called buildenv is also provided with the necessary build environment already set up. It can be used to build the entire project.

Pull the image using:

docker pull ghcr.io/krpc/buildenv:latest

Then run a container using:

docker run -it ghcr.io/krpc/buildenv:latest

This will drop you into a command line where you can then clone the repository, set up the library symlinks, build everything and run the tests, using the following commands:

git clone https://github.com/krpc/krpc.git
cd krpc
ln -s /usr/local/lib/ksp lib/ksp
ln -s /usr/lib/mono/4.5 lib/mono-4.5
bazel build //...
bazel test //:test

Note: copies of the KSP libraries are included in the docker image in /usr/local/lib/ksp. These DLLs have had their implementation stripped out. This is sufficient to build the code, without needing to publicly distribute the original KSP DLLs.

Tools

Included in the project are various tools to aid development.

  • tools/serve-docs.sh
    • Running this script build the documentation website, and starts a local webserver. You can use this to try out building the documentation and then view it by pointing a web browser at http://localhost:8080
  • tools/install.sh
    • Running this script builds the mod, and the TestingTools DLL, and installs them into the GameData directory of the copy of KSP in lib/ksp.
  • tools/run-ksp.sh
    • This script uses tools/install.sh to install the mod (as above) and then launches the game. It also pipes the game's log output to the terminal for ease of debugging.

Running the Tests

kRPC contains a suite of tests for the server plugin, services, client libraries and other parts of the project.

To run the tests, the following dependencies need to be installed. Without them, some of the tests will fail.

  • Python 3.7+ development files
  • CppCheck
  • socat

To install these dependencies via apt on Ubuntu run the following command:

sudo apt-get install cppcheck socat

The unit tests can be run using: bazel test //:test

Note: these tests do not require the game to be running. They test the various components without using the game. For communication tests, a tool called "TestServer" is used. This runs a version of the server plugin without needing to launch the game.

kRPC also includes a suite of tests that test interaction with the KSP. This requires the game to be running. To run these tests:

  • First run tools/install.sh This script builds kRPC and the TestingTools DLL, and installs them into the GameData directory of the copy of KSP found at lib/ksp.
  • Then run KSP (in lib/ksp), or run tools/run-ksp.sh to load up the game automatically.
  • When the game loads, TestingTools will automatically load up a save game and place you in the flight scene.
  • Install the python client package, the krpctest package. This can be done using: 
    bazel build //client/python //tools/krpctest
pip install --upgrade bazel-bin/client/python/krpc.<version>.zip
pip install --upgrade bazel-bin/tools/krpctools/krpc-tools.<version>.zip
    These python packages are also available from the GitHub releases page.
  • Now you are ready to run the tests.

For example, there are python scripts in service/SpaceCenter/test/... that can be used to test the space center service. These tests use the krpctools package to automatically load a save game called krpctest and launch a vessel. They then use the python client to communicate with the game, and test the various RPCs.

Bazel cheat sheet

Temporary build files are stored in bazel-krpc and bazel-out. Build outputs are stored in bazel-bin and test logs in bazel-testlogs.

Build the entire project:

bazel build //...

Build a specific "target", for example the python client:

bazel build //client/python:python

You can also pass options to bazel build to change the way it behaves:

  • Pass --subcommands to see the actual commands that are being run.
  • Pass --jobs 1 to build using a single thread. Defaults to using all threads.

Clean the entire project (but don't delete downloaded dependencies). This deletes the bazel-* directories:

bazel clean

Clean the entire project (including deleting downloaded dependencies):

bazel clean --expunge

Download all dependencies. It's no necessary to do this before building, as build will download the dependencies it needs, but doing this is useful if you want to work offline:

bazel fetch //...

Run all tests:

bazel test //:test

Note: bazel test //... does not work!

Run a specific test, e.g. the python client tests:

bazel test //client/python:test

You can also pass options to bazel test to change the way it behaves:

  • Pass --subcommands to see the actual commands that are being run.
  • Pass --jobs 1 to test using a single thread. Defaults to using all threads.
  • By default bazel will cache test results. To force it to always run the tests use --cache_test_results=false
  • By default bazel will only print out logs for failed tests. To print out all logs use --test_output=all
  • To run tests sequentially, and incrementally output logs, use --test_output=streamed