included venv activation for python scripts (#2233)

* add setup script for rfid reader. moved python script

* add check for venv activation

* fix

* fix cd fail behavior (dont start subprocess)

* add script for run jukebox for venv.
updated docs.

* add setup script for audio sink. moved python script

* add script for rpc tool. moved python script

* add script for sniffer tool. moved python script

* fixed parameter passing

* fix paths

* fix flake8

* fix comments and docs

* fixed parameter passing

* some fixes

* fixed execution rights

* some fixes

* move scripts to python root again (failing import paths).
fixed absolute path in caller scripts

* fix relative paths

* update notes about config file paths

* update caller scripts path handling

* update comments

* add venv activation to tests

* explicit installation of python3-venv not needed

* renamed audio setup file. 
updated docs path.

* Update configuration.md

* Update coreapps.md

* update motd message

* fix execution rights for motd file.
refactored into own postinstall script

* fix execution right for audio setup

---------

Co-authored-by: pabera <[email protected]>

.githooks/post-merge

@@ -43,6 +43,7 @@ warn_python_requirements() {
   echo "ATTENTION: Python requirements have changed since last pull!"
   echo ""
   echo "To update python requirements on the RPi run"
+  echo "$ source .venv/bin/activate"
   echo "$ python -m pip install --upgrade -r requirements.txt"
   echo "************************************************************"
   echo -e "\n"

.github/workflows/codeql-analysis_v3.yml

@@ -45,6 +45,9 @@ jobs:
       run: |
         # Install necessary packages
         sudo apt-get install libasound2-dev pulseaudio
+        python3 -m venv .venv
+        source ".venv/bin/activate"
+
         python -m pip install --upgrade pip
         pip install -r requirements.txt
         # Set the `CODEQL-PYTHON` environment variable to the Python executable

.github/workflows/pythonpackage_future3.yml

@@ -31,6 +31,9 @@ jobs:
       run: |
         sudo apt-get update
         sudo apt-get install -y libasound2-dev pulseaudio
+        python3 -m venv .venv
+        source ".venv/bin/activate"
+
         python3 -m pip install --upgrade pip
         pip3 install -r requirements.txt
         # For operation of the Jukebox, ZMQ must be compiled from sources due to Websocket support
@@ -51,7 +54,6 @@ jobs:
         parallel: true
     - name: Lint with flake8
       run: |
-        pip3 install flake8
         # Stop the build if linting fails
         ./run_flake8.sh
 

documentation/builders/configuration.md

@@ -1,15 +1,15 @@
 # Jukebox Configuration
 
 The Jukebox configuration is managed by a set of files located in `shared/settings`.
-Some configuration changes can be made through the WebUI and take immediate effect.
+Some configuration changes can be made through the Web App and take immediate effect.
 
-The majority of configuration options is only available by editing the config files -
+The majority of configuration options are only available by editing the config files -
 *when the service is not running!*
 Don't fear (overly), they contain commentaries.
 
-For several aspects we have [configuration tools](../developers/coreapps.md#configuration-tools) and [detailed guides](./README.md#features).
+For several aspects, we have [configuration tools](../developers/coreapps.md#configuration-tools) and [detailed guides](./README.md#features).
 
-Even after running the tools, certain aspects can only be changed by modifying the configuration files directly.
+Even after using the tools, certain aspects can only be changed by directly modifying the configuration files.
 
 ## Best practice procedure
 
@@ -21,19 +21,17 @@ $ systemctl --user stop jukebox-daemon
 $ nano ./shared/settings/jukebox.yaml
 
 # Start Jukebox in console and check the log output (optional)
-$ cd src/jukebox
-$ ./run_jukebox.py 
+$ ./run_jukebox.sh
 # and if OK, press Ctrl-C and restart the service
 
 # Restart the service
 $ systemctl --user start jukebox-daemon
 ```
-
-To try different configurations, you can start the Jukebox with a custom config file.
+To try different configurations, you can start the Jukebox with a custom config file. 
 This could be useful if you want your Jukebox to only allow a lower volume when started
-at night time when there is time to go to bed :-)
+at nighttime, signaling it's time to go to bed. :-) 
+The path to the custom config file must be either absolute or relative to the folder `src/jukebox/`.
 
 ```bash
-$ cd src/jukebox
-$ ./run_jukebox.py --conf path/to/custom/config.yaml
+$ ./run_jukebox.sh --conf /absolute/path/to/custom/config.yaml
 ```

documentation/builders/troubleshooting.md

@@ -64,12 +64,10 @@ on the console log.
 $ systemctl --user stop jukebox-daemon
 
 # Start the Jukebox in debug mode:
-$ cd src/jukebox
-
 # with default logger:
-$ ./run_jukebox.py
+$ ./run_jukebox.sh
 # or with custom logger configuration:
-$ ./run_jukebox.py --logger path/to/custom/logger.yaml
+$ ./run_jukebox.sh --logger path/to/custom/logger.yaml
 ```
 
 ### Fallback configuration
@@ -79,8 +77,7 @@ Attention: This only emits messages to the console and does not write to the log
 This is more a fallback features:
 
 ```bash
-$ cd src/jukebox
-$ ./run_jukebox.py -vv
+$ ./run_jukebox.sh -vv
 ```
 
 ### Extreme cases

documentation/developers/coreapps.md

@@ -1,7 +1,6 @@
 # Jukebox Apps
 
-The Jukebox\'s core apps are located in `src/jukebox`. Run the following
-command to learn more about each app and its parameters:
+The Jukebox's core apps are located in `src/jukebox`. To learn more about each app and its parameters, run the following command:
 
 ``` bash
 $ cd src/jukebox
@@ -10,13 +9,13 @@ $ ./<scriptname> -h
 
 ## Jukebox Core
 
-**Scriptname:** [run_jukebox.py](../../src/jukebox/run_jukebox.py)
+**Scriptname:** [run_jukebox.sh](../../run_jukebox.sh)
 
-This is the main app and starts the Jukebox Core.
+This is the main app. It starts the Jukebox Core.
 
-Usually this runs as a service, which is started automatically after boot-up. At times, it may be necessary to restart the service. For example after a configuration change. Not all configuration changes can be applied on-the-fly. See [Jukebox Configuration](../builders/configuration.md#jukebox-configuration).
+This runs as a service, which starts automatically after boot-up. At times, it may be necessary to restart the service, for example, after a configuration change. Not all configuration changes can be applied on-the-fly. See [Jukebox Configuration](../builders/configuration.md#jukebox-configuration).
 
-For debugging, it is usually desirable to run the Jukebox directly from the console rather than as service. This gives direct logging info in the console and allows changing command line parameters. See [Troubleshooting](../builders/troubleshooting.md).
+For debugging, it's best to run Jukebox directly from the console rather than as a service, as this provides direct logging information in the console and allows for changing command line parameters. See [Troubleshooting](../builders/troubleshooting.md).
 
 ## Configuration Tools
 
@@ -25,41 +24,39 @@ See [Best practice procedure](../builders/configuration.md#best-practice-procedu
 
 ### Audio
 
-**Scriptname:** [run_configure_audio.py](../../src/jukebox/run_configure_audio.py)
+**Scriptname:** [setup_configure_audio.sh](../../installation/components/setup_configure_audio.sh)
 
-Setup tool to register the PulseAudio sinks as primary and secondary audio outputs.
+A setup tool to register the PulseAudio sinks as primary and secondary audio outputs.
 
-Will also setup equalizer and mono down mixer in the pulseaudio config file. Run this once after installation. Can be re-run at any time to change the settings. For more information see [Audio Configuration](../builders/audio.md).
+This will also set up an equalizer and mono downmixer in the PulseAudio configuration file. Run this once after installation. It can be re-run at any time to change the settings. For more information see [Audio Configuration](../builders/audio.md).
 
 ### RFID Reader
 
-**Scriptname:** [run_register_rfid_reader.py](../../src/jukebox/run_register_rfid_reader.py)
+**Scriptname:** [setup_rfid_reader.sh](../../installation/components/setup_rfid_reader.sh)
 
 Setup tool to configure the RFID Readers.
 
-Run this once to register and configure the RFID readers with the Jukebox. Can be re-run at any time to change the settings. For more information see [RFID Readers](./rfid/README.md).
+Run this once to register and configure the RFID readers with Jukebox. It can be re-run at any time to change the settings. For more information see [RFID Readers](./rfid/README.md).
 
 > [!NOTE]
-> This tool will always write a new configurations file. Thus, overwrite the old one (after checking with the user). Any manual modifications to the settings will have to be re-applied
+> This tool will always create a new configuration file, thereby overwriting the old one (after confirming with the user). Any manual modifications to the settings will need to be reapplied.
 
 ## Developer Tools
 
 ### RPC
 
-**Scriptname:** [run_rpc_tool.py](../../src/jukebox/run_rpc_tool.py)
+**Scriptname:** [run_rpc_tool.sh](../../tools/run_rpc_tool.sh)
 
 Command Line Interface to the Jukebox RPC Server.
 
-A command line tool for sending RPC commands to the running jukebox app. This uses the same interface as the WebUI. Can be used for additional control or for debugging. Use `./run_rpc_tool.py` to start the tool in interactive mode.
+A command-line tool for sending RPC commands to the running Jukebox app, utilizing the same interface as the Web App, provides additional control or debugging capabilities. Start the tool in interactive mode with `./run_rpc_tool.sh`.
 
-The tool features auto-completion and command history.
+Features include auto-completion and command history, with available commands fetched from the running Jukebox service.
 
-The list of available commands is fetched from the running Jukebox service.
-
-The tool can also be used to send commands directly, when passing a `-c` argument, e.g. `./run_rpc_tool.py -c host.shutdown`.
+For direct command execution, use the `-c` argument, e.g., `./run_rpc_tool.sh -c host.shutdown`.
 
 ### Publicity Sniffer
 
-**Scriptname:** [run_publicity_sniffer.py](../../src/jukebox/run_publicity_sniffer.py)
+**Scriptname:** [run_publicity_sniffer.sh](../../tools/run_publicity_sniffer.sh)
 
-A command line tool that monitors all messages being sent out from the Jukebox via the publishing interface. Received messages are printed in the console. Mainly used for debugging.
+This command-line tool monitors all messages sent from Jukebox through the publishing interface, printing received messages in the console. It is primarily used for debugging.

documentation/developers/known-issues.md

@@ -21,6 +21,6 @@ RUN cd ${HOME} && mkdir ${ZMQ_TMP_DIR} && cd ${ZMQ_TMP_DIR}; \
 ## Configuration
 
 In `jukebox.yaml` (and all other config files):
-Always use relative path from settingsfile `../../`, but do not use relative paths with `~/`.
+Always use relative path from folder `src/jukebox` (`../../`), but do not use relative paths with `~/`.
 
 **Sole** exception is in `playermpd.mpd_conf`.

installation/components/setup_configure_audio.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+# Runner script to ensure
+# - correct venv activation
+# - independent from working directory
+
+# Change working directory to project root
+SOURCE=${BASH_SOURCE[0]}
+SCRIPT_DIR="$(dirname "$SOURCE")"
+PROJECT_ROOT="$SCRIPT_DIR"/../..
+cd "$PROJECT_ROOT" || { echo "Could not change directory"; exit 1; }
+
+source .venv/bin/activate || { echo "ERROR: Failed to activate virtual environment for python"; exit 1; }
+
+cd src/jukebox || { echo "Could not change directory"; exit 1; }
+python run_configure_audio.py $@

installation/components/setup_rfid_reader.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+# Runner script to ensure
+# - correct venv activation
+# - independent from working directory
+
+# Change working directory to project root
+SOURCE=${BASH_SOURCE[0]}
+SCRIPT_DIR="$(dirname "$SOURCE")"
+PROJECT_ROOT="$SCRIPT_DIR"/../..
+cd "$PROJECT_ROOT" || { echo "Could not change directory"; exit 1; }
+
+source .venv/bin/activate || { echo "ERROR: Failed to activate virtual environment for python"; exit 1; }
+
+cd src/jukebox || { echo "Could not change directory"; exit 1; }
+python run_register_rfid_reader.py $@

installation/routines/install.sh

@@ -15,6 +15,6 @@ install() {
   setup_rfid_reader
   optimize_boot_time
   setup_autohotspot
-  setup_login_message
+  setup_postinstall
   cleanup
 }

installation/routines/set_raspi_config.sh

@@ -31,7 +31,3 @@ _run_set_raspi_config() {
 set_raspi_config() {
     run_with_log_frame _run_set_raspi_config "Set default raspi-config"
 }
-
-setup_login_message() {
-    sudo cp -f "${INSTALLATION_PATH}/resources/system/99-rpi-jukebox-rfid-welcome" "/etc/update-motd.d/99-rpi-jukebox-rfid-welcome"
-}

installation/routines/setup_postinstall.sh

@@ -0,0 +1,13 @@
+_setup_login_message() {
+    local login_message_welcome_file="/etc/update-motd.d/99-rpi-jukebox-rfid-welcome"
+    sudo cp -f "${INSTALLATION_PATH}/resources/system/99-rpi-jukebox-rfid-welcome" "$login_message_welcome_file"
+    sudo chmod +x "$login_message_welcome_file"
+}
+
+_run_setup_postinstall() {
+    _setup_login_message
+}
+
+setup_postinstall() {
+    run_with_log_frame _run_setup_postinstall "Post install"
+}

installation/routines/setup_rfid_reader.sh

@@ -1,7 +1,9 @@
 #!/usr/bin/env bash
 
 _run_setup_rfid_reader() {
-    run_and_print_lc python "${INSTALLATION_PATH}/src/jukebox/run_register_rfid_reader.py"
+    local script="${INSTALLATION_PATH}"/installation/components/setup_rfid_reader.sh
+    sudo chmod +x "$script"
+    run_and_print_lc "$script"
 }
 
 setup_rfid_reader() {

resources/default-services/jukebox-daemon.service

@@ -10,8 +10,8 @@ After=network.target sound.target mpd.service pulseaudio.service
 Requires=mpd.service pulseaudio.service
 
 [Service]
-WorkingDirectory=%%INSTALLATION_PATH%%/src/jukebox
-ExecStart=/bin/bash -c 'source %%INSTALLATION_PATH%%/.venv/bin/activate && python run_jukebox.py'
+WorkingDirectory=%%INSTALLATION_PATH%%
+ExecStart=/bin/bash -c '%%INSTALLATION_PATH%%/run_jukebox.sh'
 StandardOutput=inherit
 StandardError=inherit
 Restart=always

resources/default-settings/jukebox.default.yaml

@@ -1,5 +1,5 @@
 # IMPORTANT:
-# Always use relative path from settingsfile `../../`, but do not use relative paths with `~/`.
+# Always use relative path from folder `src/jukebox` (`../../`), but do not use relative paths with `~/`.
 # Sole (!) exception is in playermpd.mpd_conf
 system:
   box_name: Jukebox

resources/default-settings/pulseaudio.default.pa

@@ -147,5 +147,5 @@ load-module module-filter-apply
 #set-default-source input
 
 ### Configuration by Jukebox's Tool may come below
-# Run ./run_configure_audio.py for configuration
+# Run ./installation/components/setup_configure_audio.sh for configuration
 

resources/system/99-rpi-jukebox-rfid-welcome

@@ -1,16 +1,12 @@
 #!/usr/bin/env bash
 
 echo "
-#########################################################
+##################################################
    ___  __ ______  _  __________ ____   __  _  _
   / _ \/ // / __ \/ |/ /  _/ __/(  _ \ /  \( \/ )
  / ___/ _  / /_/ /    // // _/   ) _ ((  O ))  (
 /_/  /_//_/\____/_/|_/___/____/ (____/ \__/(_/\_)
-future3
 
-If you want to run a python script from the project
-activate the venv before with 'source .venv/bin/activate'
-See also https://github.com/MiczFlor/RPi-Jukebox-RFID/
-blob/future3/main/documentation/developers/python.md
-
-#########################################################"
+            Welcome to your Phoniebox
+##################################################
+"

run_docgeneration.sh

@@ -1,12 +1,16 @@
 #!/usr/bin/env bash
 
-# Runner script for pydoc-markdown to ensure
+# Runner script to ensure
+# - correct venv activation
 # - independent from working directory
 
-# Change working directory to location of script
+# Change working directory to project root
 SOURCE=${BASH_SOURCE[0]}
 SCRIPT_DIR="$(dirname "$SOURCE")"
-cd "$SCRIPT_DIR" || (echo "Could not change to top-level project directory" && exit 1)
+PROJECT_ROOT="$SCRIPT_DIR"
+cd "$PROJECT_ROOT" || { echo "Could not change directory"; exit 1; }
+
+source .venv/bin/activate || { echo "ERROR: Failed to activate virtual environment for python"; exit 1; }
 
 # Run pydoc-markdown
 # make sure, directory exists

run_flake8.sh

@@ -1,13 +1,16 @@
 #!/usr/bin/env bash
 
-# Runner script for flak8 to ensure
-# - correct config file
+# Runner script to ensure
+# - correct venv activation
 # - independent from working directory
 
-# Change working directory to location of script
+# Change working directory to project root
 SOURCE=${BASH_SOURCE[0]}
 SCRIPT_DIR="$(dirname "$SOURCE")"
-cd "$SCRIPT_DIR" || (echo "Could not change to top-level project directory" && exit 1)
+PROJECT_ROOT="$SCRIPT_DIR"
+cd "$PROJECT_ROOT" || { echo "Could not change directory"; exit 1; }
+
+source .venv/bin/activate || { echo "ERROR: Failed to activate virtual environment for python"; exit 1; }
 
 # Run flake8
 flake8 --config .flake8 "$@"

run_jukebox.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+# Runner script to ensure
+# - correct venv activation
+# - independent from working directory
+
+# Change working directory to project root
+SOURCE=${BASH_SOURCE[0]}
+SCRIPT_DIR="$(dirname "$SOURCE")"
+PROJECT_ROOT="$SCRIPT_DIR"
+cd "$PROJECT_ROOT" || { echo "Could not change directory"; exit 1; }
+
+source .venv/bin/activate || { echo "ERROR: Failed to activate virtual environment for python"; exit 1; }
+
+cd src/jukebox || { echo "Could not change directory"; exit 1; }
+python run_jukebox.py $@

run_pytest.sh

@@ -1,13 +1,16 @@
 #!/usr/bin/env bash
 
-# Runner script for pytest to ensure
-# - correct config file
+# Runner script to ensure
+# - correct venv activation
 # - independent from working directory
 
-# Change working directory to location of script
+# Change working directory to project root
 SOURCE=${BASH_SOURCE[0]}
 SCRIPT_DIR="$(dirname "$SOURCE")"
-cd "$SCRIPT_DIR" || (echo "Could not change to top-level project directory" && exit 1)
+PROJECT_ROOT="$SCRIPT_DIR"
+cd "$PROJECT_ROOT" || { echo "Could not change directory"; exit 1; }
+
+source .venv/bin/activate || { echo "ERROR: Failed to activate virtual environment for python"; exit 1; }
 
 # Run pytest
 pytest -c pytest.ini $@