All-in-One Docker Setup

[codysnider]

What interest is there in an all-in-one docker setup (including reverse proxy and let's encrypt companion container)? I'm building one out for my own use and, if there is interest in having this integrated, I can work on keeping it as user-friendly and environment-agnostic as possible.

[krisgesling]

Hi Cody, thanks for the assistance. I've posted your question in the Backend channel on Mycroft Chat as it gets a lot more traffic than these issues threads.

[codysnider]

Thanks, buddy. I'll pop in there tomorrow night when I get back onto this project. I've actually got the full set of APIs running on separate containers and orchestrated under a single docker-compose file. Things are coming along very quickly.

[biziosan]

@codysnider Let me know if you want to test your container on another server. I have a server rack at home, and I am already running docker for few services.

[DigitalElise]

I am very interested, I absolutely need a solution to run mycroft without a internet connection. Would be nice to have a all in one available

[jimmykarily]

Since there was no follow up here I started working on an "offline" deployment here: https://github.com/jimmykarily/mycroft_offline . It's still work-in-progress. If you try it out you might have to play a bit with this "magic" sleep timeout: https://github.com/jimmykarily/mycroft_offline/blob/master/docker-compose.yaml#L26 because the postgres container restarts itself after init script have been run and there is nothing to wait on and be sure it won't be restarted after that. Also the bootstrap_mycroft_db.py doesn't recover after Postgres going away (it doesn't try to reconnect).

I'm currently stuck hooking the various components together. I'm using these nginx settings for reverse proxy but I get 502 responses from the APIs. If anyone knows how the APIs should be accessed from the applications your help is welcome!

The error I'm seeing:

2020/02/12 18:19:53 [error] 6#6: *1 upstream prematurely closed connection while reading response header from upstream, client: 172.19.0.1, server: _, request: "GET /api/account HTTP/1.1", upstream: "http://172.19.0.9:5000/api/account", host: "172.19.0.13", referrer: "http://172.19.0.13/"

this happens on the market api container so the nginx proxy seems to be working (it send traffic to the right container).

[chrisveilleux]

Sorry for the lack of follow up. The Mycroft team has very limited bandwidth right now and we are focusing on another project. I think a container setup for this repository is a great idea and would welcome any contributions to that effort.

[jimmykarily]

Hi @chrisveilleux as soon as we get it working, it would be a lot better if I could move all the file to some upstream repo. For the time being, I managed to connect various pieces together but I'm seeing frontend errors in Javascript that don't allow me to create accounts or login. The whole effort was part of my HackWeek at SUSE which is now done so I will have less time to work on it but if there is interest (and maybe some help) I could try to complete the work.

I'm also thinking that maybe I shouldn't focus too much on the various applications (account, marketplace etc) if mycroft-core can work without them. E.g. I started adding a container for precise to allow offline wake up word detection. Also it seems that the deepspeech container I've added is being used by mycroft-core and is quite accurate. So if we can limit the scope and skip the web apps maybe it will be easier to complete and maintain in the long run. What do you think? Is it possible to have a fully working (offline) mycroft setup without the web applications?

[chrisveilleux]

@jimmykarily Not sure I agree with storing the Dockerfile to a different repo. Seems like an anti-pattern to me. Ideally, the "official" Dockerfile would exist within the repository. It should be written in such a way that it can be extended for "non-official" use cases (such as an offline implementation, perhaps?). I can envision a Docker compose file in a separate repository that did something like build containers for mycroft-core, selene-backend and Postgres.

You can use the any API in this repo independently from a web application. If you choose to omit the account API and or SSO API, you would need some way to do device pairing so that the mycroft-core container can authenticate with the device API in Selene. You would also need a mechanism to change skill settings.

[NaveenPrasanth]

@jimmykarily hi, are you still working on this? I want this solution. I can either join in or work on my own.

[jimmykarily]

Hi @NaveenPrasanth unfortunately I haven't found the time to continue on this. I would recommend you to have a look at the state of things here: https://github.com/jimmykarily/mycroft_offline to avoid re-implementing it. Other than my comments above and the README on the mycroft_offline repo I don't remember much more. I think a good start is to play with the docker-compose on my repo and see where you get stuck.

If you like the codebase and want to continue, I can add you as a contributor and I will try to help as much as my free time allows.

[NaveenPrasanth]

@jimmykarily sounds like a plan! let me give it a shot

[jimmykarily]

@NaveenPrasanth this just landed: jimmykarily/mycroft_offline#1 (I didn't have time to play with it yet but maybe it's worth checking).

[cvvs]

I have a working compose with PostgreSQL in Docker that includes the UI. It's nice to spin up a clean and pristine dev environment. I had to make a change to the database bootstrap so that it uses a tempfile to create city.dump, otherwise, postgres can't create the dump file. By making this change, I can run the bootstrap when pg initializes the database. If I need to clean the db, I just have to docker-compose rm postgres.

Regarding using the existing Dockerfile, the current Dockerfile won't build because it depends on a private devops repo, and seems to be extremely focused on internal development. I don't think it's a good fit for just running a local setup.

I'll submit a PR with the bootstrap change to see if there's any appetite for it.

[djpbessems]

@cvvs Did you end up creating that PR (if so can you link to it?)

[cvvs]

@cvvs Did you end up creating that PR (if so can you link to it?)

I haven't submitted the PR, but here's my branch
https://github.com/cvvs/selene-backend/tree/use_tempfile_with_dump

#233