build failed due to mismatches with linuxdeploy inclusion tree

[mdear]

In my attempt to build PR #157 I got the following compilation error:

Any idea how to proceed, @SushiTee ?

I tried doing Docker build on x86_64 build VM for target ARCH=aarch64 but it failed (apparently cross-compilation is not supported).

My QEMU running aarch64 is too resource constrained to run docker, and my physical aarch64 device is even more constrained. The QEMU is running nested virtualization inside an x86_64 Ubunto 20.046 LTS VM, which itself is running as a guest on a Windows 11 host.

The head of my git log for each repo:

qt plugin:

commit 2b73a2173f8acfc0269e681bdb28ebf65b0b4b48 (HEAD -> mdear_elf_fix, tag: 1-alpha-20240109-1, origin/master, origin/HEAD, master)
Author: TheAssassin [email protected]
Date: Wed Jan 3 11:48:25 2024 +0100

Improve logging of failed subprocesses

base:

commit 2b73a2173f8acfc0269e681bdb28ebf65b0b4b48
Author: TheAssassin [email protected]
Date: Wed Jan 3 11:48:25 2024 +0100

Improve logging of failed subprocesses

In file included from /home/ccs/workspaces/git/linuxdeploy-plugin-qt/src/util.cpp:5:
/home/ccs/workspaces/git/linuxdeploy-plugin-qt/src/linuxdeploy/util/util.h:13:10: fatal error: linuxdeploy/core/log.h: No such file or directory
   13 | #include <linuxdeploy/core/log.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Apaprently, this is a dependency that reaches back to the main linuxdeploy repository, which does not have that header file at the requested location.

I tried adding the following line to the end of the top level CMakeLists.txt but didn't see an -I argument showing up when compiling with --verbose flag:
include_directories(/home/ccs/workspaces/git/linuxdeploy/include)

so I added the following hack:

ln -s /home/ccs/workspaces/git/linuxdeploy/lib/linuxdeploy-desktopfile/include/linuxdeploy/desktopfile /home/ccs/workspaces/git/linuxdeploy/include/linuxdeploy/desktopfile

cd /home/ccs/workspaces/git/linuxdeploy-plugin-qt/src/linuxdeploy/core

cp /home/ccs/workspaces/git/linuxdeploy/include/linuxdeploy/log/log.h .

# Change the namespace in log.h to linuxdeploy::core::log

# Allow qt plugin to reach into linuxdeploy's include tree under already established
# include dir.
ln -s /home/ccs/workspaces/git/linuxdeploy/include/linuxdeploy /home/ccs/workspaces/git/linuxdeploy-plugin-qt/src/linuxdeploy

# qt plugin depends on static libraries:
# linuxdeploy_core rquested via -llinuxdeploy_core linker argument
../../linuxdeploy/build/src/core/liblinuxdeploy_core.a
# Remove -largs since it's a single-file included library, resolved at build time
# not link time.
# Add the following to src/CMakeFiles/linuxdeploy-plugin-qt.dir/link.txt
-L/home/ccs/workspaces/git/linuxdeploy/build/src/core

# I duplicate the main clause in linuxdeploy/src/log/log.cpp and make it under namespace linuxdeploy::core::log but I still get tons of undefined symbols in my link of the qt plugin.

I think I'll have to give up trying to build the qt plugin for now.

[SushiTee]

I'm sorry. I didn't get my application work using appimage in combination with Ubuntu 20.04. I don't remember that error though. I switched to Ubuntu 22.04 as I don't care about old Ubuntu distributions.

Also I never tried to build my app for ARM.

[mdear]

Please excuse the hack-worthy quality of this code, as my neck is on the line I had to get this working, but I'm sure there are many ways of doing it with more civility.

With that said, here is a summary of what I had to do to get
linuxdeploy-plugin-qt:
commit 52d9a4cd0f6ff20704953bfe223f057a6ca6cff3 (HEAD -> master, tag: 1-alpha-20240109-1, origin/master, origin/arm, origin/HEAD)

compiled against base linuxdeploy:
commit 2b73a2173f8acfc0269e681bdb28ebf65b0b4b48 (HEAD -> mdear_elf_fix, tag: 1-alpha-20240109-1, origin/master, origin/HEAD, master)

with the following patches to linuxdeploy main repo:
(please refer to linuxdeploy/linuxdeploy#258 (comment) for changes I did to the exclusion list to promote portability on top of multiple OS versions) :

Duplicate include/linuxdeploy/log/log.h to include/linuxdeploy/core/log.h and change the namespace from linuxdeploy::log to linuxdeploy::core::log

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 14d7fcf..a4cddea 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -48,7 +48,7 @@ include(CTest)
 
 if(BUILD_TESTING)
     # including this before including lib/ makes sure that the top level project's gtest is used everywhere
-    include(${PROJECT_SOURCE_DIR}/lib/cmake-scripts/include-or-build-gtest.cmake)
+    #include(${PROJECT_SOURCE_DIR}/lib/cmake-scripts/include-or-build-gtest.cmake)
 endif()
 
 add_subdirectory(lib)
diff --git a/lib/linuxdeploy-desktopfile b/lib/linuxdeploy-desktopfile
--- a/lib/linuxdeploy-desktopfile
+++ b/lib/linuxdeploy-desktopfile
@@ -1 +1 @@
-Subproject commit aa7423539d6ecdb6eb3bc6c6607247d3f7ff8bb3
+Subproject commit aa7423539d6ecdb6eb3bc6c6607247d3f7ff8bb3-dirty
diff --git a/src/core/elf_file.cpp b/src/core/elf_file.cpp
index f199b21..89101e4 100644
--- a/src/core/elf_file.cpp
+++ b/src/core/elf_file.cpp
@@ -173,7 +173,7 @@ namespace linuxdeploy {
                 ifs.read(magicBytes.data(), 4);
 
                 if (strncmp(magicBytes.data(), "\177ELF", 4) != 0)
-                    throw ElfFileParseError("Invalid magic bytes in file header");
+                    throw ElfFileParseError("Invalid magic bytes in file header" + path.string() + magicBytes.data());
 
                 d = new PrivateData(path);
                 d->readDataUsingElfAPI();
diff --git a/src/core/generate-excludelist.sh b/src/core/generate-excludelist.sh
index d92cb13..5c4277b 100644
--- a/src/core/generate-excludelist.sh
+++ b/src/core/generate-excludelist.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#/bin/bash
 
 # Copyright 2018 Alexander Gottwald (https://github.com/ago1024)
 # Copyright 2018 TheAssassin (https://github.com/TheAssassin)
@@ -64,8 +64,9 @@ echo "};" >> "$tempfile"
 # avoid overwriting if the contents have not changed
 # this prevents CMake having to recompile half of linuxdeploy even if nothing changed
 if [ "$(sha256sum $filename | awk '{print $1}')" != "$(sha256sum $tempfile | awk '{print $1}')" ]; then
-    echo "$log_prefix changes detected, updating $filename"
-    cp "$tempfile" "$filename"
+    echo "KTECHNOLOGY : $log_prefix changes detected, not touching $filename to keep local changes intact."
+    #echo "$log_prefix changes detected, updating $filename"
+    #cp "$tempfile" "$filename"
 else
     echo "$log_prefix no changes detected, not touching $filename"
 fi
diff --git a/src/log/log.cpp b/src/log/log.cpp
index 65c2f63..733e8ce 100644
--- a/src/log/log.cpp
+++ b/src/log/log.cpp
@@ -1,5 +1,6 @@
 // local includes
 #include "linuxdeploy/log/log.h"
+#include "linuxdeploy/core/log.h"
 
 namespace linuxdeploy::log {
     LD_LOGLEVEL ldLog::verbosity = LD_INFO;
@@ -125,3 +126,128 @@ namespace linuxdeploy::log {
         stream.write(s, n);
     }
 }
+
+namespace linuxdeploy::core::log {
+    LD_LOGLEVEL ldLog::verbosity = LD_INFO;
+
+    void ldLog::setVerbosity(LD_LOGLEVEL verbosity) {
+        ldLog::verbosity = verbosity;
+    }
+
+    ldLog::ldLog() {
+        prependSpace = false;
+        currentLogLevel = LD_INFO;
+        logLevelSet = false;
+    };
+
+    ldLog::ldLog(bool prependSpace, bool logLevelSet, LD_LOGLEVEL logLevel) {
+        this->prependSpace = prependSpace;
+        this->currentLogLevel = logLevel;
+        this->logLevelSet = logLevelSet;
+    }
+
+    void ldLog::checkPrependSpace() {
+        if (prependSpace) {
+            stream << " ";
+            prependSpace = false;
+        }
+    }
+
+    bool ldLog::checkVerbosity() {
+//                std::cerr << "current: " << currentLogLevel << " verbosity: " << verbosity << std::endl;
+        return (currentLogLevel >= verbosity);
+    }
+
+    ldLog ldLog::operator<<(const std::string& message) {
+        if (checkVerbosity()) {
+            checkPrependSpace();
+            stream << message;
+        }
+
+        return ldLog(true, logLevelSet, currentLogLevel);
+    }
+    ldLog ldLog::operator<<(const char* message) {
+        if (checkVerbosity()) {
+            checkPrependSpace();
+            stream << message;
+        }
+
+        return ldLog(true, logLevelSet, currentLogLevel);
+    }
+
+    ldLog ldLog::operator<<(const std::filesystem::path& path) {
+        if (checkVerbosity()) {
+            checkPrependSpace();
+            stream << path.string();
+        }
+
+        return ldLog(true, logLevelSet, currentLogLevel);
+    }
+
+    ldLog ldLog::operator<<(const int val) {
+        return ldLog::operator<<(std::to_string(val));
+    }
+
+    ldLog ldLog::operator<<(const size_t val) {
+        return ldLog::operator<<(std::to_string(val));
+    }
+
+    ldLog ldLog::operator<<(const double val) {
+        return ldLog::operator<<(std::to_string(val));
+    }
+
+    ldLog ldLog::operator<<(stdEndlType strm) {
+        if (checkVerbosity()) {
+            checkPrependSpace();
+            stream << strm;
+        }
+
+        return ldLog(false, logLevelSet, currentLogLevel);
+    }
+
+    ldLog ldLog::operator<<(const LD_LOGLEVEL logLevel) {
+        if (logLevelSet) {
+            throw std::runtime_error(
+                "log level must be first element passed via the stream insertion operator");
+        }
+
+        logLevelSet = true;
+        currentLogLevel = logLevel;
+
+        if (checkVerbosity()) {
+            switch (logLevel) {
+                case LD_DEBUG:
+                    stream << "DEBUG: ";
+                    break;
+                case LD_WARNING:
+                    stream << "WARNING: ";
+                    break;
+                case LD_ERROR:
+                    stream << "ERROR: ";
+                    break;
+                default:
+                    break;
+            }
+        }
+
+        return ldLog(false, logLevelSet, currentLogLevel);
+    }
+
+    ldLog ldLog::operator<<(const LD_STREAM_CONTROL streamControl) {
+        bool prependSpace = true;
+
+        switch (streamControl) {
+            case LD_NO_SPACE:
+                prependSpace = false;
+                break;
+            default:
+                break;
+        }
+
+        return ldLog(prependSpace, logLevelSet, currentLogLevel);
+    }
+
+    void ldLog::write(const char* s, const size_t n) {
+        stream.write(s, n);
+    }
+}

... and the following modifications to the linuxdeploy-plugin-qt repo:

Build-time depencency of linuxdeploy-plugin-qt
Add nlohmann json.hpp (one file library) to local system
cp ${GIT_REPO_BASE}json/single_include/nlohmann/json.hpp /usr/include
Build-time depencency of linuxdeploy-plugin-qt
Add args.hxx (one file library) to local System

cd ${GIT_REPO_BASE}
#git clone https://github.com/Taywee/args.git
sudo cp args/args.hxx /usr/include

NOTE : When cloning linuxdeploy and linuxdeploy-plugin-qt and wanting to build the qt plugin, it has a broken dependency
on linuxdeploy include directory and library import paths.

set GIT_REPO_BASE="/path/to/my/git/repos/"
cd ${GIT_REPO_BASE}/linuxdeploy-plugin-qt/src/linuxdeploy/core
cp ${GIT_REPO_BASE}/linuxdeploy/include/linuxdeploy/log/log.h .

Change the namespace in log.h to linuxdeploy::core::log
Allow qt plugin to reach into linuxdeploy's include tree under already established include dir.

ln -s ${GIT_REPO_BASE}/linuxdeploy/include/linuxdeploy ${GIT_REPO_BASE}/linuxdeploy-plugin-qt/src/linuxdeploy
ln -s ${GIT_REPO_BASE}/linuxdeploy/build/src ${GIT_REPO_BASE}/linuxdeploy-plugin-qt/src/linuxdeploy_build
ln -s ${GIT_REPO_BASE}/linuxdeploy/build/lib/linuxdeploy-desktopfile/src/ ${GIT_REPO_BASE}/linuxdeploy/build/src/desktopfile
ln -s ${GIT_REPO_BASE}/linuxdeploy/lib/linuxdeploy-desktopfile/include/linuxdeploy/desktopfile ${GIT_REPO_BASE}/linuxdeploy/include/linuxdeploy/desktopfile

NOTE : It's also necessary to go into ${GIT_REPO_BASE}/linuxdeploy/src/log/log.cpp
and duplicate the log code block but make it under namespace linuxdeploy::core::log

apply the following patch (see the softlinks I created from the qt plugin back to the core repositoriy)

Removed args since it's a single-file included library, resolved at build time, not link time.

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 3ff6634..24d32bc 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -29,10 +29,19 @@ execute_process(
 # TODO: CMake <= 3.7 (at least!) doesn't allow for using OBJECT libraries with target_link_libraries
 add_library(linuxdeploy-plugin-qt_util STATIC util.cpp util.h)
 target_include_directories(linuxdeploy-plugin-qt_util PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
-target_link_libraries(linuxdeploy-plugin-qt_util linuxdeploy_core args Threads::Threads)
+
+link_directories(
+    linuxdeploy_build/subprocess
+    linuxdeploy_build/plugin
+    linuxdeploy_build/desktopfile
+    linuxdeploy_build/core/copyright
+    linuxdeploy_build/core
+)
+target_link_libraries(linuxdeploy-plugin-qt_util linuxdeploy_core linuxdeploy_core_copyright linuxdeploy_plugin linuxdeploy_subprocess linuxdeploy_desktopfile_static jpeg png Threads::Threads)
+
 
 add_executable(linuxdeploy-plugin-qt main.cpp qt-modules.h qml.cpp qml.h deployment.h)
-target_link_libraries(linuxdeploy-plugin-qt linuxdeploy_core args nlohmann_json::nlohmann_json linuxdeploy-plugin-qt_util Threads::Threads)
+#target_link_libraries(linuxdeploy-plugin-qt linuxdeploy_core args linuxdeploy-plugin-qt_util Threads::Threads)
 set_target_properties(linuxdeploy-plugin-qt PROPERTIES RUNTIME_OUTPUT_DIRECTORY "${PROJECT_BINARY_DIR}/bin")
 target_compile_definitions(linuxdeploy-plugin-qt
     PRIVATE -DLD_GIT_COMMIT="${GIT_COMMIT}"

[mdear]

@TheAssassin Please respond to these rough proposed changes and, if you agree, I'll clean them up and submit a PR for the same. Thanks.