Skip to content

Commit

Permalink
tmp
Browse files Browse the repository at this point in the history
  • Loading branch information
BartSte committed Jan 14, 2024
1 parent 284042f commit dd9cb40
Show file tree
Hide file tree
Showing 7 changed files with 316 additions and 21 deletions.
68 changes: 68 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Release

on:
push:
tags:
- 'v[0-9]+\.[0-9]+\.[0-9]+'

jobs:
build-release:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v2

- name: Update apt-get database
run: |
sudo apt-get update -y
sudo apt-get upgrade -y
- name: Install build dependencies
run: sudo apt-get install -y clang ninja-build git cmake llvm pkg-config

- name: Install vulkan
run: |
wget -qO- https://packages.lunarg.com/lunarg-signing-key-pub.asc | sudo tee /etc/apt/trusted.gpg.d/lunarg.asc
sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-jammy.list http://packages.lunarg.com/vulkan/lunarg-vulkan-jammy.list
sudo apt update
sudo apt -y install vulkan-sdk
- name: Install project dependencies
run: |
sudo apt-get install -y libboost-all-dev libspdlog-dev
sudo apt-get install -y '^libxcb.*-dev' build-essential libgl1-mesa-dev libgles2-mesa-dev libglu1-mesa-dev libpulse-dev libqt5x11extras5 libssl-dev libssl1.0 libwayland-dev libwayland-egl1-mesa libwayland-server0 libx11-xcb-dev libxi-dev libxkbcommon-dev libxkbcommon-dev libxkbcommon-x11-dev libxrender-dev openssl ffmpeg
- name: Install ffmpeg
run: |
./scripts/install-ffmpeg
- name: Download Qt source
env:
QT_INSTALLER_JWT_TOKEN: ${{ secrets.QT_INSTALLER_JWT_TOKEN }}
run: |
mkdir /tmp/Qt
./scripts/install-qt --root /tmp/Qt qt.qt6.661.src
- name: Build Qt from source with static linking
env:
CC: gcc
CXX: g++
CXXFLAGS: -Wall
run: |
src=/tmp/Qt/6.6.1/Src
$src/configure -release -static -prefix $PWD/3rdparty/Qt/ -submodules qtbase,qtmultimedia,qtwayland -no-pch -no-gstreamer -- -S $src -B $src/build -Wdev -DPC_FFMPEG_LIBRARY_DIRS=$PWD/3rdparty/ffmpeg/lib -DPC_FFMPEG_INCLUDE_DIRS=$PWD/3rdparty/ffmpeg/include
cmake --build $src/build
cmake --install $src/build
- name: Build project
run: |
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DPC_FFMPEG_LIBRARY_DIRS=$PWD/3rdparty/ffmpeg/lib -DPC_FFMPEG_INCLUDE_DIRS=$PWD/3rdparty/ffmpeg/include
cmake --build ./build --target package
- name: Publish
uses: "marvinpinto/action-automatic-releases@latest"
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
draft: true
files: ./build/*.tar.gz
13 changes: 10 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,20 @@ jobs:
- name: Install build dependencies
run: sudo apt-get install -y clang ninja-build git cmake

- name: Install vulkan
run: |
wget -qO- https://packages.lunarg.com/lunarg-signing-key-pub.asc | sudo tee /etc/apt/trusted.gpg.d/lunarg.asc
sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-jammy.list http://packages.lunarg.com/vulkan/lunarg-vulkan-jammy.list
sudo apt update
sudo apt install vulkan-sdk
- name: Install project dependencies
env:
QT_INSTALLER_JWT_TOKEN: ${{ secrets.QT_INSTALLER_JWT_TOKEN }}
run: |
sudo apt-get install -y libboost-all-dev libspdlog-dev
sudo apt-get install -y build-essential openssl libssl-dev libssl1.0 libgl1-mesa-dev libqt5x11extras5 '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev libpulse-dev
./scripts/install-qt
sudo apt-get install -y libboost-all-dev libspdlog-dev
sudo apt-get install -y build-essential openssl libssl-dev libssl1.0 libgl1-mesa-dev libqt5x11extras5 '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev libpulse-dev libwayland-dev libwayland-egl1-mesa libwayland-server0
./scripts/install-qt --root $PWD/3rdparty/Qt qt.qt6.661.gcc_64 qt.qt6.661.addons.qtmultimedia
- name: Build project
run: |
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Session.vim
nvim.shada
include/version.h
3rdparty/Qt
3rdparty/ffmpeg

# This file is used to ignore files which are generated
# ----------------------------------------------------------------------------
Expand Down
55 changes: 51 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# README

[![Tests](https://github.com/BartSte/snapshot/actions/workflows/tests.yml/badge.svg)](https://github.com/BartSte/snapshot/actions/workflows/tests.yml)
[![Release](https://github.com/BartSte/snapshot/actions/workflows/release.yml/badge.svg)](https://github.com/BartSte/snapshot/actions/workflows/release.yml)

> WORK IN PROGRESS!
> This project is still under development. The code is still experimental and
> is subject to change.
Expand Down Expand Up @@ -414,7 +417,24 @@ more information.
to run the app.
- [ ] Publish the directory tree as an archive as a release. Other more
sophisticated release methods can be used later.
- Issues:
- Issue FFmpeg:

- On ubuntu, the includes (headers I guess) are not included in the apt
package, in contrast to the arch package. As a solution, I build ffmpeg
from source. I build them with static linking which gives some new
issues when building the project. Now I am thinking.. I guess I have 3
options:
1. statical link ffmpeg to the project.
2. dynamically link ffmpeg to the project and include the ffmpeg libs in
the release.
3. dynamically link ffmpeg to the project and let the user install ffmpeg
themselves. This could be useful when the user needs different
versions of ffmpeg to make it work with their hardware.

On arch, I can just install static ffmpeg using AUR (its located at
/opt/ffmpeg063).

- Issues graphics:

- On WSL I get the following error:

Expand All @@ -425,11 +445,38 @@ more information.
libEGL warning: egl: failed to create dri2 screen
```
- On ubuntu, fresh install, I get the following error:
According to bing:
```
barts@zbook:~$ ./snapshot-0.0.0-Linux/snapshot -h
1012: __vdso_timeSegmentation fault
It seems that you are encountering an error while running your Qt app on
Arch Linux with WSL. The error message you provided indicates that the
vkCreateInstance function failed with the error code
VK_ERROR_INCOMPATIBLE_DRIVER. This error is usually caused by a mismatch
between the Vulkan driver and the hardware it is running on 1.
One possible solution is to install the latest graphics drivers for your
system 2. Another possible solution is to try running the app without any
Vulkan layers at all 3.
It’s also worth noting that there is an ongoing issue with
hardware-accelerated Vulkan in WSL2 4. You may want to check if this issue
is related to your problem.
I hope this helps!
```
An idea is to check for differences in the output of `pldd` when comparing
an snapshot app that uses a dynamic Qt6 and one that uses a static Qt6. I
the past, a dynamic Qt6 worked on WSL, so it is likely that something goes
wrong during the static linking.
I think it has to do with support for the GPU. I think it is not possible
to make a stand-alone structure that support all GPU's. So it is likely to
be better that the user installs the GPU drivers themselves. I am not sure
what the best way is to do this. It does explain why the build on the
ubuntu server fails on my arch machine, but when I build it myself, it
works. This may also be the case on my WSL+arch machine when I install the
GPU drivers as is explained
[here](https://github.com/lutris/docs/blob/master/InstallingDrivers.md#amd--intel).
- [ ] Cross compile for raspberry pi
149 changes: 149 additions & 0 deletions scripts/install-ffmpeg
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
set -euo pipefail

this_dir=$(dirname "$(realpath "${BASH_SOURCE:-$0}")")
usage="$(basename "$0") [-h] [-s, --source-dir] [-b, --build-dir] -- Compiles ffmpeg from source
The packages managers \`apt\` and \`pacman\` are supported.
where:
-h, --help show this help text
-s, --source-dir ffmpeg source directory (default: $HOME/ffmpeg_sources)
-b, --build-dir ffmpeg build directory (default: $this_dir/../3rdparty/ffmpeg)"

source_dir="$HOME/ffmpeg_sources"
build_dir="$this_dir/../3rdparty/ffmpeg"
while [[ $# -gt 0 ]]; do
key="$1"
case $key in
-h | --help)
echo "$usage"
exit
;;
-s | --source-dir)
source_dir="$2"
shift
shift
;;
-b | --build-dir)
build_dir="$2"
shift
shift
;;
*)
echo "Unknown option: $key"
echo "$usage"
exit 1
;;
esac
done
bin_dir="$build_dir/bin"

install_dependencies() {
if command -v apt-get &>/dev/null; then
echo "apt-get found"
install_dependencies_apt
elif command -v pacman &>/dev/null; then
echo "pacman found"
install_dependencies_pacman
else
echo "No supported package manager found"
exit 1
fi
}

install_dependencies_apt() {
echo "Updating apt-get"
sudo apt-get update -qq
echo "Installing build tools and dependencies"
sudo apt-get -y install autoconf automake build-essential cmake git-core libass-dev libfreetype6-dev libgnutls28-dev libmp3lame-dev libsdl2-dev libtool libva-dev libvdpau-dev libvorbis-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev meson ninja-build pkg-config texinfo wget yasm zlib1g-dev gnutls-bin libunistring-dev libaom-dev libdav1d-dev tar

echo "Installing ffmpeg dependencies"
sudo apt-get -y install nasm libx264-dev libx265-dev libvpx-dev libfdk-aac-dev libopus-dev libdav1d-dev libnuma-dev
}

install_dependencies_pacman() {
echo "Installing build tools and dependencies"
sudo pacman -S --noconfirm --needed autoconf automake cmake git libass freetype2 sdl2 libtool libva libvdpau libvorbis libxcb meson ninja pkgconf texinfo wget yasm zlib gnutls libunistring aom dav1d tar

echo "Installing ffmpeg dependencies"
sudo pacman -S --noconfirm --needed nasm x264 x265 libvpx fdkaac opus numactl
}

compile_libaom() {
echo "Compiling libaom"
cd ~/ffmpeg_sources &&
git -C aom pull 2>/dev/null || git clone --depth 1 https://aomedia.googlesource.com/aom &&
mkdir -p aom_build &&
cd aom_build &&
PATH="$bin_dir:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$build_dir" -DENABLE_TESTS=OFF -DENABLE_NASM=on ../aom &&
PATH="$bin_dir:$PATH" make -j"$(nproc)" &&
make install
}

compile_libsvtav1() {
echo "Compiling libsvtav1"
cd ~/ffmpeg_sources &&
git -C SVT-AV1 pull 2>/dev/null || git clone https://gitlab.com/AOMediaCodec/SVT-AV1.git &&
mkdir -p SVT-AV1/build &&
cd SVT-AV1/build &&
PATH="$bin_dir:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$build_dir" -DCMAKE_BUILD_TYPE=Release -DBUILD_DEC=OFF -DBUILD_SHARED_LIBS=OFF .. &&
PATH="$bin_dir:$PATH" make -j"$(nproc)" &&
make install
}

compile_libvmaf() {
echo "Compiling libvmaf"
cd ~/ffmpeg_sources &&
wget https://github.com/Netflix/vmaf/archive/v2.3.1.tar.gz && tar xvf v2.3.1.tar.gz &&
mkdir -p vmaf-2.3.1/libvmaf/build &&
cd vmaf-2.3.1/libvmaf/build &&
meson setup -Denable_tests=false -Denable_docs=false --buildtype=release --default-library=static .. --prefix "$build_dir" --bindir="$bin_dir" --libdir="$build_dir/lib" &&
ninja &&
ninja install
}

compile_ffmpeg() {
echo "Compiling ffmpeg"
cd ~/ffmpeg_sources &&
wget -O ffmpeg-snapshot.tar.bz2 https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2 &&
tar xjvf ffmpeg-snapshot.tar.bz2 &&
cd ffmpeg &&
PATH="$bin_dir:$PATH" PKG_CONFIG_PATH="$build_dir/lib/pkgconfig" ./configure \
--prefix="$build_dir" \
--pkg-config-flags="--static" \
--extra-cflags="-I$build_dir/include" \
--extra-ldflags="-L$build_dir/lib" \
--extra-libs="-lpthread -lm" \
--ld="g++" \
--bindir="$bin_dir" \
--enable-gpl \
--enable-gnutls \
--enable-libaom \
--enable-libass \
--enable-libfdk-aac \
--enable-libfreetype \
--enable-libmp3lame \
--enable-libopus \
--enable-libsvtav1 \
--enable-libdav1d \
--enable-libvorbis \
--enable-libvpx \
--enable-libx264 \
--enable-libx265 \
--enable-nonfree &&
PATH="$bin_dir:$PATH" make -j"$(nproc)" &&
make install &&
hash -r
}

dir=$(pwd)

mkdir -p "$source_dir" "$bin_dir"
install_dependencies
compile_libaom
compile_libsvtav1
compile_libvmaf
compile_ffmpeg

cd $dir
31 changes: 21 additions & 10 deletions scripts/install-qt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail

usage="Usage: $0 <optional packages>
usage="Usage: $0 <packages>
Installs Qt 6.6.1 to 3rdparty/Qt. For this script to work, you need to set the
QT_INSTALLER_JWT_TOKEN environment variable to your jwt token. Check the
Expand All @@ -11,21 +11,32 @@ Optionally, you can add more Qt packages to install by adding them as
arguments.
Options:
-r, --root The root directory to install Qt to.
-h, --help Show this help message and exit"

if [[ "$*" == *-h* ]] || [[ "$*" == *--help* ]]; then
echo "$usage"
exit 0
fi
packages=""
while [[ $# -gt 0 ]]; do
case "$1" in
-r | --root)
install_dir="$2"
shift
shift
;;
-h | --help)
echo "$usage"
exit 0
;;
*)
packages="$packages $1"
shift
;;
esac
done

tmp_dir=$(mktemp -d)
this_dir=$(dirname "$(realpath "${BASH_SOURCE:-$0}")")
install_dir="$this_dir/../3rdparty/Qt"
base="qt.qt6.661.gcc_64"
multimedia="qt.qt6.661.addons.qtmultimedia"

curl "https://d13lb3tujbc8s0.cloudfront.net/onlineinstallers/qt-unified-linux-x64-4.6.1-online.run" -o "$tmp_dir/qt-installer.run"
chmod +x "$tmp_dir/qt-installer.run"
"$tmp_dir/qt-installer.run" --root "$install_dir" --accept-licenses --accept-obligations --default-answer --confirm-command install $base $multimedia $@
"$tmp_dir/qt-installer.run" --root "$install_dir" --accept-licenses --accept-obligations --default-answer --confirm-command install $packages

rm -rf "$tmp_dir"
Loading

0 comments on commit dd9cb40

Please sign in to comment.