Fix Docker for pyzmq #2167

s-martin · 2023-12-17T23:19:43Z

g++ was missing to create pyzmq.

This happened to me under Windows

docker/jukebox.Dockerfile

s-martin · 2023-12-20T21:59:40Z

Building the container works now, but #1838 still doesn't work.

docker/jukebox.Dockerfile

Groovylein

Worked for me on Linux as well

Groovylein

OK the build works, but staring throws an error:

mpd exited with code 0
mpd      | exception: Failed to open '/home/pi/.config/mpd/mpd.conf': No such file or directory
[...]
jukebox  | run_jukebox.py: error: argument -c/--conf: can't open '/home/pi/RPi-Jukebox-RFID/shared/settings/jukebox.yaml': [Errno 2] No such file or directory: '/home/pi/RPi-Jukebox-RFID/shared/settings/jukebox.yaml'

AlvinSchiller · 2023-12-21T06:20:02Z

OK the build works, but staring throws an error:

mpd exited with code 0
mpd      | exception: Failed to open '/home/pi/.config/mpd/mpd.conf': No such file or directory
[...]
jukebox  | run_jukebox.py: error: argument -c/--conf: can't open '/home/pi/RPi-Jukebox-RFID/shared/settings/jukebox.yaml': [Errno 2] No such file or directory: '/home/pi/RPi-Jukebox-RFID/shared/settings/jukebox.yaml'

Did you copy the yaml? docker.md#prerequisites

Groovylein · 2023-12-21T06:22:06Z

OK the build works, but staring throws an error:

mpd exited with code 0
mpd      | exception: Failed to open '/home/pi/.config/mpd/mpd.conf': No such file or directory
[...]
jukebox  | run_jukebox.py: error: argument -c/--conf: can't open '/home/pi/RPi-Jukebox-RFID/shared/settings/jukebox.yaml': [Errno 2] No such file or directory: '/home/pi/RPi-Jukebox-RFID/shared/settings/jukebox.yaml'

Did you copy the yaml? docker.md#prerequisites

Yes, everything is there.

Changing these volume paths to /home/pi let's the jukebox start:
https://github.com/s-martin/RPi-Jukebox-RFID/blob/42c2e8db97d98599e153cb76763edd3891e5368d/docker/docker-compose.yml#L42

The same applies for mpd:
https://github.com/s-martin/RPi-Jukebox-RFID/blob/42c2e8db97d98599e153cb76763edd3891e5368d/docker/docker-compose.yml#L17

AlvinSchiller · 2023-12-21T06:58:10Z

The same applies for mpd:
https://github.com/s-martin/RPi-Jukebox-RFID/blob/42c2e8db97d98599e153cb76763edd3891e5368d/docker/docker-compose.yml#L17

Shouldn't they be overwritten from *linux.yml (but i really know docker compose)? Looks like- ../src/jukebox:xxx is missing here?

docker/jukebox.Dockerfile

docker/docker-compose.linux.yml

docker/jukebox.Dockerfile

fixes docker

s-martin · 2023-12-30T08:26:12Z

Reference: https://github.com/zeromq/pyzmq/wiki/Building-and-Installing-PyZMQ

…tin/RPi-Jukebox-RFID into future3/fix-docker-windows

s-martin · 2023-12-31T13:00:27Z

I got the webapp running under a Github Codespace using Ubuntu.
So I guess my issues on Windows are Windows-related

s-martin · 2024-01-02T10:45:36Z

Todo: check pyzmq and libzmq version on a working raspberry instance

Groovylein · 2024-01-22T20:41:10Z

My problem now is that I still get the errors:

jukebox  | 22.01.2024 20:32:31 -   67:server.py          - jb.rpc.server        - MainThread      - INFO     - Init RPC Server (Pyzmq version: 21.0.2; ZMQ version: 4.3.3; has draft API: False)
jukebox  | 22.01.2024 20:32:31 -   75:server.py          - jb.rpc.server        - MainThread      - DEBUG    - Connected to address 'inproc://JukeBoxRpcServer'
jukebox  | 22.01.2024 20:32:31 -   81:server.py          - jb.rpc.server        - MainThread      - DEBUG    - Connected to address 'tcp://*:5555'
jukebox  | Traceback (most recent call last):
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/run_jukebox.py", line 67, in <module>
jukebox  |     main()
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/run_jukebox.py", line 63, in main
jukebox  |     myjukebox.run()
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/jukebox/daemon.py", line 227, in run
jukebox  |     self.rpc_server = RpcServer()
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/jukebox/rpc/server.py", line 86, in __init__
jukebox  |     self.socket.bind(websocket_address)
jukebox  |   File "/root/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/zmq/sugar/socket.py", line 172, in bind
jukebox  |     super().bind(addr)
jukebox  |   File "zmq/backend/cython/socket.pyx", line 540, in zmq.backend.cython.socket.Socket.bind
jukebox  |   File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
jukebox  | zmq.error.ZMQError: Protocol not supported
jukebox  | 22.01.2024 20:32:31 -   31:daemon.py          - jb.daemon            - MainThread      - DEBUG    - Active Threads = [<_MainThread(MainThread, stopped 139642232436544)>, <Thread(PlayerEventLoop, started daemon 139642181654272)>, <MultiTimer(host.timer.cputemp, started daemon 139642154718976)>]
jukebox exited with code 0

Groovylein · 2024-01-22T21:30:19Z

When having one line:
RUN export ZMQ_PREFIX=${PREFIX} && export ZMQ_DRAFT_API=1 && pip install -v --no-binary pyzmq --pre pyzmq

I get another error:

jukebox  | 22.01.2024 21:27:48 -   67:server.py          - jb.rpc.server        - MainThread      - INFO     - Init RPC Server (Pyzmq version: 25.1.2; ZMQ version: 4.3.4; has draft API: True)
jukebox  | 22.01.2024 21:27:48 -   75:server.py          - jb.rpc.server        - MainThread      - DEBUG    - Connected to address 'inproc://JukeBoxRpcServer'
jukebox  | 22.01.2024 21:27:48 -   81:server.py          - jb.rpc.server        - MainThread      - DEBUG    - Connected to address 'tcp://*:5555'
jukebox  | Traceback (most recent call last):
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/run_jukebox.py", line 67, in <module>
jukebox  |     main()
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/run_jukebox.py", line 63, in main
jukebox  |     myjukebox.run()
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/jukebox/daemon.py", line 227, in run
jukebox  |     self.rpc_server = RpcServer()
jukebox  |   File "/root/RPi-Jukebox-RFID/src/jukebox/jukebox/rpc/server.py", line 86, in __init__
jukebox  |     self.socket.bind(websocket_address)
jukebox  |   File "/root/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/zmq/sugar/socket.py", line 302, in bind
jukebox  |     super().bind(addr)
jukebox  |   File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
jukebox  |   File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
jukebox  | zmq.error.ZMQError: Protocol not supported (addr='ws://*:5556')
jukebox  | 22.01.2024 21:27:48 -   31:daemon.py          - jb.daemon            - MainThread      - DEBUG    - Active Threads = [<_MainThread(MainThread, stopped 140644536080192)>, <Thread(PlayerEventLoop, started daemon 140644485297920)>, <MultiTimer(host.timer.cputemp, started daemon 140644391253760)>]
jukebox exited with code 1

s-martin · 2024-02-18T20:52:03Z

#2268 may be related

s-martin · 2024-02-19T20:48:32Z

Lots of comments here now. :-D
But with the latest changes it seems to work on Windows itself and Linux (WSL Ubuntu).

@AlvinSchiller @pabera @Groovylein if anyone of you could also check, would be nice.

AlvinSchiller · 2024-02-19T22:21:52Z

Did we change anything for this part? Or is it maybe also the prerelease of pyzmq which makes a change?
Could you test it without the --pre?

Nevermind, didn't see the last commit

pabera · 2024-02-20T22:24:15Z

This is not entirely fixed. On my Windows machine, it even doesn't work with the updated pip command. Reason for this is that the downloaded, pre-compiled libzmq does not work on my system (x86_64), even though I used the same machine to compile it before. When I compile libzmq manually in the Dockerfile, it works.

This means, we need to enable a way to build libzmq locally in order to use it in case the pre-built versions do not work.

I am working on that.

AlvinSchiller · 2024-02-20T22:50:47Z

Could we use the bundled libzmq for that (ZMQ_PREFIX="bundled") ? No further setup of libzmq would be needed if it works.

pabera · 2024-02-21T08:05:50Z

Not sure. In my environment, the problem is that pyzmq (actually g++) cannot compile the lib properly and fails. It then falls back on the bundled, none-draft version that pyzmq is shipped with.

AlvinSchiller · 2024-02-21T12:55:46Z

Ah ok.
Maybe settings the libzmq version explicitly could help in a first step -> export PYZMQ_LIBZMQ_VERSION=4.3.5

Or turn off the fallback? -> export PYZMQ_NO_BUNDLE=ON

AlvinSchiller · 2024-03-21T00:48:23Z

Might be the cause for the problems here aswell? #2302

pabera · 2024-03-21T07:36:46Z

Pre-building the libzmq does not work well as many users have different host architectures and OS.
The goal should be to build an individual Docker image that just builds libzmq for the host it's running on (1-time build)
Once built, it can be used in docker-compose instead of downloading the pre-built version from our repos.

I am working on this in a branch, but haven't had the time to bring it to a draft.

notapirate · 2024-03-24T11:50:32Z

In commit 1ae54c7 changes where made to adjust docker/docker-compose.linux.yaml to docker/docker-compose.mac.yaml. IMHO this doesn't work because other than on mac or windows, on a linux host the already running pulseaudio is used, but for that to work, the UID within the docker container must match the UID of the user outside the container, which doesn't work anymore with 1ae54c7, so I vote to undo that commit.

My docker tests show that based on current branch future3/develop, 2 things need to be changed to make it work again:

In jukebox.Dockerfile move the commands to untar libzmq up to the area before the User is changed, because on a linux-based host this user is limited and can't write to /usr/local.
As @pabera already noted on Feb 20 (I created a separate issue, 🐛 | docker: prebuild libzmq incompatible with debian-bullseye #2302), the prebuilt libzmq doesn't work on x86_64 on bullseye. For testing, one can use the debian:bookworm-slim image for jukebox (netcat package also needs to change to netcat-openbsd) and see if pyzmq works. This might cause other problems (see Sharing tips for future3 on bookworm/kernel 6.6 and upstreaming fixes #2295 for a summary of bookworm experiences), but it should show if the pyzmq problem is solved then.

On my local build I did both changes and it works fine.

s-martin · 2024-03-24T12:34:18Z

Will check this :)

pabera · 2024-03-26T20:32:05Z

@s-martin Could you check if the pyzmq problem is solved for you too? This issue has been the reason I provided #2307 :-)

s-martin · 2024-03-26T23:07:06Z

#2307 provides a working solution

add g++ to create pyzmq

39432f3

s-martin requested a review from pabera December 17, 2023 23:20

s-martin added bug future3 Relates to future3 development labels Dec 17, 2023

pabera reviewed Dec 18, 2023

View reviewed changes

docker/jukebox.Dockerfile Outdated Show resolved Hide resolved

AlvinSchiller added 2 commits December 18, 2023 19:22

fix user rights for venv creation

58e1222

remove static /home/pi/ paths

66c8a79

Groovylein approved these changes Dec 18, 2023

View reviewed changes

docker/jukebox.Dockerfile Outdated Show resolved Hide resolved

pabera reviewed Dec 18, 2023

View reviewed changes

docker/jukebox.Dockerfile Outdated Show resolved Hide resolved

s-martin added 2 commits December 20, 2023 22:23

fix paths (independent of pi)

68e1d8b

use only g++ and fix home dir and user

42c2e8d

s-martin marked this pull request as ready for review December 20, 2023 21:59

AlvinSchiller reviewed Dec 20, 2023

View reviewed changes

docker/jukebox.Dockerfile Outdated Show resolved Hide resolved

Groovylein approved these changes Dec 21, 2023

View reviewed changes

Groovylein requested changes Dec 21, 2023

View reviewed changes

Groovylein reviewed Dec 21, 2023

View reviewed changes

docker/jukebox.Dockerfile Show resolved Hide resolved

pabera reviewed Dec 21, 2023

View reviewed changes

docker/docker-compose.linux.yml Outdated Show resolved Hide resolved

docker/docker-compose.linux.yml Outdated Show resolved Hide resolved

docker/jukebox.Dockerfile Outdated Show resolved Hide resolved

s-martin and others added 2 commits December 21, 2023 22:37

just in one command

55048fc

Merge pull request #30 from AlvinSchiller/future3/fix-docker-windows

3681460

fixes docker

Merge branch 'future3/fix-docker-windows' of https://github.com/s-mar…

b0eaa8c

…tin/RPi-Jukebox-RFID into future3/fix-docker-windows

s-martin mentioned this pull request Jan 2, 2024

Websocket connections fail on firefox #2191

Closed

s-martin mentioned this pull request Jan 2, 2024

future3: Jukebox.Dockerfile - Reorder 'USER' command #2199

Merged

add temporary tasks

3145824

s-martin modified the milestones: v3.5, v3.6 Jan 29, 2024

pabera mentioned this pull request Feb 5, 2024

Release v3.5.1 #2241

Merged

2 tasks

AlvinSchiller mentioned this pull request Feb 14, 2024

Release v3.6.0 #2258

Open

s-martin added 2 commits February 19, 2024 21:22

Merge branch 'future3/develop' into future3/fix-docker-windows

b2bdf2b

remove --pre

b1b321c

Merge branch 'future3/develop' into future3/fix-docker-windows

4eb3c69

pabera self-assigned this Mar 21, 2024

AlvinSchiller mentioned this pull request Mar 24, 2024

installation: untar libzmq in venv folder instead of /usr/local #2305

Closed

pabera mentioned this pull request Mar 25, 2024

fix: Build libzmq locally to work on all host types #2307

Merged

3 tasks

s-martin removed this from the v3.6 milestone Mar 26, 2024

s-martin closed this Mar 26, 2024

s-martin deleted the future3/fix-docker-windows branch April 7, 2024 18:49

pabera added this to the v3.6 milestone Apr 8, 2024

s-martin removed this from the v3.6 milestone Apr 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Docker for pyzmq #2167

Fix Docker for pyzmq #2167

s-martin commented Dec 17, 2023

s-martin commented Dec 20, 2023

Groovylein left a comment

Groovylein left a comment •

edited

Loading

AlvinSchiller commented Dec 21, 2023

Groovylein commented Dec 21, 2023 •

edited

Loading

AlvinSchiller commented Dec 21, 2023

s-martin commented Dec 30, 2023

s-martin commented Dec 31, 2023

s-martin commented Jan 2, 2024 •

edited

Loading

Groovylein commented Jan 22, 2024

Groovylein commented Jan 22, 2024

s-martin commented Feb 18, 2024

s-martin commented Feb 19, 2024

AlvinSchiller commented Feb 19, 2024 •

edited

Loading

pabera commented Feb 20, 2024 •

edited

Loading

AlvinSchiller commented Feb 20, 2024 •

edited

Loading

pabera commented Feb 21, 2024

AlvinSchiller commented Feb 21, 2024

AlvinSchiller commented Mar 21, 2024

pabera commented Mar 21, 2024

notapirate commented Mar 24, 2024 •

edited

Loading

s-martin commented Mar 24, 2024

pabera commented Mar 26, 2024

s-martin commented Mar 26, 2024

Fix Docker for pyzmq #2167

Fix Docker for pyzmq #2167

Conversation

s-martin commented Dec 17, 2023

s-martin commented Dec 20, 2023

Groovylein left a comment

Choose a reason for hiding this comment

Groovylein left a comment • edited Loading

Choose a reason for hiding this comment

AlvinSchiller commented Dec 21, 2023

Groovylein commented Dec 21, 2023 • edited Loading

AlvinSchiller commented Dec 21, 2023

s-martin commented Dec 30, 2023

s-martin commented Dec 31, 2023

s-martin commented Jan 2, 2024 • edited Loading

Groovylein commented Jan 22, 2024

Groovylein commented Jan 22, 2024

s-martin commented Feb 18, 2024

s-martin commented Feb 19, 2024

AlvinSchiller commented Feb 19, 2024 • edited Loading

pabera commented Feb 20, 2024 • edited Loading

AlvinSchiller commented Feb 20, 2024 • edited Loading

pabera commented Feb 21, 2024

AlvinSchiller commented Feb 21, 2024

AlvinSchiller commented Mar 21, 2024

pabera commented Mar 21, 2024

notapirate commented Mar 24, 2024 • edited Loading

s-martin commented Mar 24, 2024

pabera commented Mar 26, 2024

s-martin commented Mar 26, 2024

Groovylein left a comment •

edited

Loading

Groovylein commented Dec 21, 2023 •

edited

Loading

s-martin commented Jan 2, 2024 •

edited

Loading

AlvinSchiller commented Feb 19, 2024 •

edited

Loading

pabera commented Feb 20, 2024 •

edited

Loading

AlvinSchiller commented Feb 20, 2024 •

edited

Loading

notapirate commented Mar 24, 2024 •

edited

Loading