Go for beginners: Getting started

[jyyi1]

Introduction

Outline SDK is a powerful Go¹ library that can be used to build cross-platform VPN applications. However, getting started with the cross platform compilation can be a bit tricky, especially if you are new to Go.

This guide walks you through the process of setting up your Go environment and using the Outline SDK. It also covers how to create Go libraries that can be used on a variety of platforms, including Android, iOS/macOS, Windows and Linux.

If you have any questions about the environment setup, please feel free to post them here or in a new discussion thread.

Set up Go¹

This section describes how to install Go. After your environment is set up, you will be able to create an executable that splits TCP streams into two streams: The first stream contains the first 3 bytes, and the second stream contains the rest of the data. You will be using the Outline SDK to create this executable.

Install Go

You can follow the official installation guide at https://go.dev/doc/install ². Alternatively, if you are using a package manager, you can also use it to install Go:

macOS Homebrew

brew install go

³

Windows WinGet

winget install -e --id GoLang.Go

⁴

Ubuntu apt

sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt update
sudo apt install golang-go

⁵

After Go is installed, you can verify that it is installed correctly by running the following command (make sure the version is ≥ 1.20):

go version

If you want to learn some basic Go syntax, "A Tour of Go"⁶ is a great online tool.

Create splitfetch Application

This section describes how to create a cross-platform split-fetcher application that will fetch a page using HTTP/HTTPS protocol. Instead of sending the entire request in a single network packet, it will split it into two packets. This helps the app to circumvent deep-packet inspection ⁷. This application will run on desktop platforms. For mobile usage, see the following sections.

To set up the splitfetch project, follow these steps:

Details

1. Create a folder to contain your code; initialize a Go project and add the Outline SDK dependency (this will add go.mod and go.sum file to this folder):

   mkdir splitfetch
   cd splitfetch
   go mod init local/example/splitfetch
   go get github.com/Jigsaw-Code/outline-sdk@latest

   Note: If you need to push the code to a version control system such as a GitHub repository, we recommend you to use a fully qualified package name "github.com/your-org/your-repo" instead of the one used here.

2. Create two files: fetcher.go and main/main.go. fetcher.go will include a helper function that fetches a page using Outline SDK's split.NewStreamDialer. This function will be used by main.go, which is the entrypoint of the application:

   touch fetcher.go
   mkdir main
   touch main/main.go

Then, use your favorite code editor⁸ to paste the code below into fetcher.go and main/main.go:

fetcher.go

package splitfetch

import (
	"context"
	"fmt"
	"io"
	"net"
	"net/http"
	"strings"

	"github.com/Jigsaw-Code/outline-sdk/transport"
	"github.com/Jigsaw-Code/outline-sdk/transport/split"
)

const splitPacketSize = 3

func FetchPage(url string) (string, error) {
	dialer, err := split.NewStreamDialer(&transport.TCPStreamDialer{}, splitPacketSize)
	if err != nil {
		return "", err
	}

	httpClient := &http.Client{
		Transport: &http.Transport{
			DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
				if !strings.HasPrefix(network, "tcp") {
					return nil, fmt.Errorf("protocol not supported: %v", network)
				}
				return dialer.Dial(ctx, addr)
			},
		},
	}

	resp, err := httpClient.Get(url)
	if err != nil {
		return "", err
	}
	defer resp.Body.Close()

	buf := &strings.Builder{}
	_, err = io.Copy(buf, resp.Body)
	if err != nil {
		return "", nil
	}
	return buf.String(), nil
}

main/main.go

package main

import (
	"fmt"
	"local/example/splitfetch"
	"log"
	"os"
)

func main() {
	page, err := splitfetch.FetchPage(os.Args[1])
	if err != nil {
		log.Fatalf("%v", err)
	}
	fmt.Println(page)
}

After you have copied the code into the files, you can run go mod tidy to update the go.mod file.

Run splitfetch Application

You can now run the splitfetch application by:

go run main/main.go https://ipinfo.io

This will compile and run the application. If you want to create a distributable binary, you can use the go build command:

Windows

go build -o splitfetch.exe main/main.go

macOS or Linux

go build -o splitfetch main/main.go

Create a Go library

In some cases, it is more desirable to use the splitfetch.FetchPage as a library rather than a standalone executable binary. For example, when developing an Android app.

This section describes the process of creating a reusable library for splitfetch.FetchPage. It demonstrates how to build this library for different platforms that can be readily integrated into your applications.

Mobile Platforms

This tutorial uses Go Mobile⁹ to create the library for Android and iOS/macOS. First, you will need to install the gomobile command line tool ¹⁰ using the following command:

go install golang.org/x/mobile/cmd/gomobile@latest

By default, it will be installed to the "$(go env GOPATH)/bin" folder (typically it expands to "$HOME/go/bin" on Linux/macOS, or "%HOMEPATH%\go\bin" on Windows). You can override this location by export GOBIN="<customized-bin-folder>" before running the go install command.

You must also add the bin folder mentioned above to the PATH environment variable, because gomobile will try to install and discover some additional command line tools:

Linux/macOS

export PATH="$PATH:$(go env GOPATH)/bin"

Windows (PowerShell)

$Env:PATH+=";$(go env GOPATH)\bin"

After you have completed these steps, you can initialize the gomobile command line tools ¹⁰:

gomobile init

Build Android Library

To build an Android library (AAR, Android Archive) for the FetchPage function, you need to install the Android SDK (API Level 18+) and JDK (8+). We recommend installing the latest Android Studio ¹¹, which includes both of these requirements.

After you have installed Android Studio, check that JDK is installed correctly by running javac --version in the terminal. If you see any errors, install the JDK from a vendor such as Oracle¹² or OpenJDK¹³. It is also usually available in package managers such as brew¹⁴, WinGet¹⁵, and apt¹⁶.

You must also set the ANDROID_HOME environment variable to the location of your Android SDK. You can find this location in Android Studio ¹⁷:

export ANDROID_HOME="$HOME/Android/Sdk"    ## Replace it with your actual Android SDK path ##

You can now build the Android library using the following commands (for more gomobile options, see their documentation ¹⁸):

go get golang.org/x/mobile/bind
gomobile bind -target android -o splitfetch.aar .

💡 golang.org/x/mobile/bind is needed when running gomobile, but go mod tidy will remove it because it is not used by any Go source code. One way to fix this is to import ( _ "golang.org/x/mobile" ) in a dedicated tools.go file ¹⁹.

To use this library in your Android app, follow these steps:

1. Create a new project in Android Studio.
2. Import the splitfetch.aar file into your project ²⁰.
3. In the Manifest file ²¹, declare the internet permission ²².
4. Use the following Java code to call the FetchPage function:

   import splitfetch.Splitfetch;
   // ... unrelated code omitted ...
   try {
       String resp = Splitfetch.fetchPage("https://ipinfo.io");
       // resp contains the response content string
   } catch (Exception ex) {
       // handle errors
   }

Build iOS/macOS Library

To create an XCFramework bundle ²³ for the FetchPage function, you need a macOS machine with Xcode²⁴ installed. After Xcode²⁴ is installed, you can build the XCFramework bundle using the following command (for more gomobile options, see their documentation ¹⁸):

go get golang.org/x/mobile/bind
gomobile bind -target=ios,macos -o splitfetch.xcframework .

💡 golang.org/x/mobile/bind is needed when running gomobile, but go mod tidy will remove it because it is not used by any Go source code. One way to fix this is to import ( _ "golang.org/x/mobile" ) in a dedicated tools.go file ¹⁹.

To use this multiplatform XCFramework library in your iOS or macOS app, you must first create a Swift Package ²⁵ for the binary ²⁶. To do this:

1. Create a new "Swift Package" project in Xcode (e.g. "SplitFetchLib").
2. Modify the Package.swift file:

   // ... unchanged ...
   let package = Package(
       // ... unchanged ...
       products: [
           .library(
               name: "SplitFetchLib",
               targets: ["SplitFetchLib", "splitfetch"]),  // add "splitfetch" target
       ],
       // ... unchanged ...
       targets: [
           // ... unchanged ...
           // add the xcframework binary (must use relative path)
           .binaryTarget(
               name: "splitfetch",
               path: "<path-relative-to-this-swift-file>/splitfetch.xcframework"),
       ]
   )

3. Product > Build this project to make sure everything is good.

Next, close the package project. The steps below create an app to use the package:

1. Create a new "App" project in Xcode (e.g. "FetchingApp").
2. Add the "SplitFetchLib" dependency to the project: File > Add Packages... > Add Local... > choose "SplitFetchLib" folder > Add Package ²⁷.
3. Make sure "SplitFetchLib" is included in the build process: select project in the left nav pane > under "TARGETS", select project > General Pane > Frameworks, Libraries, and Embedded Content > + > select the leaf node of "SplitFetchLib" > Add.
4. Enable internet capability by checking the Outgoing Connections (Client) checkbox on the Signing & Capabilities page of the project. ²⁸²⁹
5. Use the following Swift code to call the FetchPage function:

   import Splitfetch
   // ... unrelated code omitted ...
   let err: NSErrorPointer = nil
   let response = SplitfetchFetchPage("https://ipinfo.io", err)
   if err != nil {
       // handle errors
   }

6. Product > Build to make sure everything is good. If you get any errors, double-check step 2 and step 3 above. Sometimes File > Packages > Reset Package Caches and Product > Clean Build Folder might also help.

💡 If you don't need a multiplatform app, you can use the library even more easily (no need to create a Swift package wrapper): just drag a corresponding Splitfetch.framework folder in the splitfetch.xcframework to your project, and call the FetchPage function as specified in Step 5 above. Here is an example ³⁰.

For Desktop Platforms

The best way to consume this package on desktop platforms such as Linux and Windows is to use Go directly (you can even write GUIs in Go ³¹). However, if this is not possible (for example, if you have an existing application written in another language, or if you are creating an embedded application), you can still create a C library using cgo³²³³. The C library is a glue that can be used to connect different programming languages together ^34353637. You need to install a native C/C++ compiler on your platform before using cgo. For example, on Linux you need to install gcc/g++ ³⁸³⁹, on Windows you need to install MinGW-w64 ⁴⁰⁴¹ (MSVC is not currently supported ⁴²), and on macOS you need to install Xcode ²⁴.

💡 Using a native C/C++ compiler means you can only compile your program to your computer's platform. It is possible to cross-compile to different platforms, but it is more complicated and is not covered in this document. If you want to cross-compile, you need to install a cross-compile C/C++ compiler (such as llvm/clang⁴³) and have the target platform's sysroot ⁴⁴ ready.

To compile this package to a C library, you need to add another main package that defines C-compatible functions (you can be more creative than us when defining the parameters and return type of the functions, see more in cgo documentation ³³):

mkdir clib
touch clib/main.go

clib/main.go

package main

// #include <stdlib.h>
import "C"
import (
	"local/example/splitfetch"
	"unsafe"
)

//export fetch_page
func fetch_page(url *C.char) *C.char {
	resp, err := splitfetch.FetchPage(C.GoString(url))
	if err != nil {
		resp = err.Error()
	}
	return C.CString(resp)
}

//export free_string
func free_string(s *C.char) {
	C.free(unsafe.Pointer(s))
}

func main() { /* dummy to make `go build` happy */ }

After you have completed these steps, you will be able to build it into a C shared library:

Linux/macOS

go build -buildmode c-shared -o lib/splitfetch.so ./clib/main.go

Windows (PowerShell)

# make sure gcc.exe is available through PATH environment variable
go build -buildmode c-shared -o lib/splitfetch.dll ./clib/main.go

After running the previous command, you will have two files in the lib folder: a shared library binary (for example, splitfetch.so/splitfetch.dll) and a header file (splitfetch.h).

Use the shared library (C/C++)

This section creates a native C++ application to demonstrate how to use this shared library. Create a new folder and copy both files (i.e. splitfetch.so/splitfetch.dll and splitfetch.h) into it. Then create a C file called main.cpp in this folder.

main.cpp

#include <iostream>

#include "splitfetch.h"

int main(void)
{
    char *resp = fetch_page(const_cast<char*>("https://ipinfo.io"));
    std::cout << "Result:\n" << resp << std::endl;
    free_string(resp);
    return 0;
}

To compile and link the C++ code with the shared library, and run the final executable file (it's important to make sure that the shared library and the executable file are in the same folder):

Linux/macOS

g++ -o fetching main.cpp splitfetch.so
./fetching

Windows (PowerShell)

# make sure g++.exe is available through environment variable
g++ -o fetching main.cpp splitfetch.dll
./fetching.exe

Use the shared library (C#)

This section demonstrates how to use this shared library in C#. First, you will need to have the .NET SDK ⁴⁵ installed on your system. Then you can create, build and run your C# application:

dotnet new console -n fetcher
cd fetcher
# copy the file content below into Program.cs
dotnet build -o ./bin
# copy "splitfetch.so"/"splitfetch.dll" to "./bin" folder
# (you do not need to copy the header file)
./bin/fetcher.exe

Program.cs

using System.Runtime.InteropServices;

[DllImport("splitfetch")]
static extern string fetch_page(string url);

[DllImport("splitfetch")]
static extern void free_string(string s);

var result = fetch_page("https://ipinfo.io");
Console.WriteLine(result);
free_string(result);

Footnotes

Footnotes

1. The Go Programming Language ↩ ↩²

2. Download and Install Go ↩

3. Homebrew: go ↩

4. WinGet: Golang.Go.1.20 ↩

5. Install Go on Ubuntu ↩

6. A Tour of Go ↩

7. New Feature: Packet Splitting ↩

8. Go Editors and IDEs ↩

9. Go Mobile ↩

10. gomobile CLI ↩ ↩²

11. Install Android Studio ↩

12. Oracle JDK 21 ↩

13. Open JDK 21 ↩

14. brew install openjdk ↩

15. WinGet Oracle JDK 17 ↩

16. apt install default-jdk ↩

17. Find Android SDK Location ↩

18. gomobile Build a library for Android and iOS ↩ ↩²

19. Go Modules: how can I track tool dependencies for a module ↩ ↩²

20. Android Studio: add AAR or JAR as a dependency ↩

21. Android Studio: declaring app permissions ↩

22. Network permissions for Android app ↩

23. Apple: multiplatform binary framework bundle ↩

24. Xcode on the Mac App Store ↩ ↩² ↩³

25. Creating a standalone Swift package with Xcode ↩

26. Distributing binary frameworks as Swift packages ↩

27. Adding package dependencies to your app ↩

28. Adding capabilities to your app ↩

29. Configuring the macOS App Sandbox ↩

30. Calling Go code from Swift on iOS and vice versa with Gomobile ↩

31. A list of Go GUI projects ↩

32. C? Go? Cgo! ↩

33. cgo command ↩ ↩²

34. NodeJS: C++ addons ↩

35. C#: Platform Invoke ↩

36. Extending Python with C or C++ ↩

37. Java: Guide to JNI ↩

38. Installing GCC ↩

39. How to Install GCC Compiler on Ubuntu ↩

40. MinGW-w64 Downloads ↩

41. MSYS2 ↩

42. Go cmd/link: support msvc object files ↩

43. LLVM/Clang ↩

44. Cross compiling made easy, using Clang and LLVM ↩

45. Install .NET on Windows, Linux, and macOS ↩

[SiteSplat]

Waiting for the: "For Desktop Platforms" 🙏

[maddyhof]

Coming soon!! Sorry for the delay