Bareos backup server #94 #395
Conversation
Multi-container Bareos server set Rock-on containing: - Director Service (overall controller). - Catalog Service (Postgres DB for Director). - Storage Service (Back-end Storage for Director and File (client) services). - Director-local File (client) Service (for Catalog backup capability). - Web-UI Service (Director Web-UI interface). Each of the above 5 services are hosted in their own dedicated containers. Uses custom-made docker images from Rockstor developers as no official docker images where available.
|
@Hooverdan96 & @FroggyFlox Draft and untested. Pushing to draft PR to aid collaboration in pending development. Plus I can more easily test end-to-end, using back-end mechanisms, when it's a PR, draft or otherwise. |
Initial install test onlyWe have remaining way too many 'fake' double entry passwords here as a hack to supply each container with its version of the passwords subsets require to communicate. Hoping to remove most of these via their mutual shares and each containers self-config via their docker-entrypoint.sh scripts. |
|
like on the paperless Rockon, how important/secure is it to have the user set passwords across all containers? Would it make sense for example to also hard-code a Postgres password and hide it in the options section of the Rockon? What about the others? |
|
@Hooverdan96 Re passwords. I'm still thinking on this one. We may, as you say, have some scope to set default passwords: and the rpm install do just this, but in this case the user needs to know the password as bareos client installs on other machines to be backed-up will have to enter said password. Initially I'm working towards having the minimum external/user-set passwords to actually enter. Folks always get exited about password stuff so we have to tread carefully here. But the initial plan is to try and stick to upstream as closely as possibly. But the non-shared OS between these abstracted containers is working against us. As per many technologies it kicks problems into another area. But there is hope I think re the share resource via my approach here of a shared volume. Experimentation ongoing and some work as yet as a proof of concept. Essential we need a secrets manager!! But again that is for another. My overall aim here is to get something that works - and is non-complex with same defaults and close to upstream. We can then all chip-in with refinements: such as supporting Bareos's supported binaries repo via credentials. But I'm still fathoming getting minimum rights right first. I.e. running everything with only as much access as is required. I'll welcome more input on that font once we have an actual working server set here. Note that this is only the server side. Folks will then install on their laptops directly from upstream the Client/File service, and connect to the server set to host the actual backup and Web-UI. |
|
A major concern with passwords here is that as this is a network back-up: it will host potentially all machines data. Ergo we have zero pre-known passwords. Even if the particular services are inaccessible. Tricky thought. As last resort I may use our custom config and set a random password in some places (lots of services here): but we then have to have folks do yet more cli to find that password!! Bit by bit. |
The File (client) Service, bareos-fd, must be able to name resolve the bareos-storage container, as the Director instructs the File service to seek out a specific Storage service. File and Storage then need to communicate directly on a name basis.
|
The same limitation encountered in a prior draft re #383 (comment) has come-up again in this PR with the commit proceeding this comment. The Bareos Web-UI reports the following when executing a build-in/pre-configured job:
Looking to @Hooverdan96 exposition in the comments following the above referenced comment, and their reproducer detailed in the following issue:
As this issue now represents a blocker bug re further development on this Rock-on. See also @FroggyFlox, author of our docker networks wrapper, #383 (comment) indicating that this limitation is not expected. |
|
So, a workaround (in absence of a quick solution) could be the manual creation of networks and attaching of all relevant containers with corresponding notes in the instructions of the Rockon? Or, do you "feel" that the root cause can and should be addressed before the next stable? |
|
@Hooverdan96 Re:
Yes that is a good point. It did occur to me: but only belatedly as I super keen on keeping the pre and post user intervention to a minimum on this one. But all-in, with the more recent notes on your linked issue re our current limitations. What you suggest may well be the only way though here. Another way could be to hard-wire a Re:
I've just brought up a discussion regarding this on the linked issue. With a call-out to you on your opinion. We should continue that discussion there I think. In short this is a way too deep change this late in testing. So I'll likely follow your work-around here for the Bareos back server approach. At least until we have our next generation of docker network support :). |
I'm afraid it could be a problem, yes. We indeed create networks and connect containers to it as the last step of the rockon install process: |
Work-around re
|
| Resource type | Internal Reference | External Reference / Configured value |
|---|---|---|
| Network | bareos-storage | BareosFileToStorage |
| Network | bareos-fd | BareosFileToStorage |
Confirm inter container name resolution
Do we fix Files container's ability to resolve the Storage container by name, and visa-versa:
(Required as File service is instructed to use Storage service directly under instruction from overall controller service of Director.
See prior failure detailed in: #395 (comment)
Storage from File
docker exec -it bareos-fd sh
sh-4.4# ping bareos-storage
PING bareos-storage (172.26.0.2) 56(84) bytes of data.
64 bytes from bareos-storage.BareosFileToStorage (172.26.0.2): icmp_seq=1 ttl=64 time=0.087 ms
64 bytes from bareos-storage.BareosFileToStorage (172.26.0.2): icmp_seq=2 ttl=64 time=0.070 ms
^C
File from Storage
docker exec -it bareos-storage sh
sh-4.4$ ping bareos-fd
PING bareos-fd (172.26.0.3) 56(84) bytes of data.
64 bytes from bareos-fd.BareosFileToStorage (172.26.0.3): icmp_seq=1 ttl=64 time=0.059 ms
64 bytes from bareos-fd.BareosFileToStorage (172.26.0.3): icmp_seq=2 ttl=64 time=0.080 ms
^C
|
Using the last comments work-around we have our first successful Job run, initiated via Bareos's Web-UI , of the build-in Job:
Which defaults to using: From small beginnings :). |
We intend to run as uid 105, make this explicity via the uid Rock-on directive.
Status updateAfter a tweak to the bareos-director docker image re adapting an upstream pg_dump command in 'RunBeforeJob' script (make_catalog_backup) as part of the BackupCatalog job, which has the following FileSet: we now have our first success for this 'self backup' job, which also includes the config for all but the Web-UI containers via the docker image set's use of a single shared Rockstor Share for all mutual config. There-after the files contained within related backups for this Director-local client can be viewed via the restore Web-UI capability:
N.B. upon uninstall and re-install it is required to re-establish the containers link to the by-hand created rock-net. |
As a server-set, with ordered install, we can inherit (via the shared config arrangement) all intra Bareos service passwords. File and Storage containers are now able to self-config when uses in this intended AIO setup. This is akin to the rpms default behaviour of reading each-others config in order to auto-setup the initial server set. Includes minor Description rewording.
Intended to enable access to Rockstor as an SMTP relay; enabling the default Bareos email notification with minimum setup.
Multi-container Bareos server set Rock-on containing:
Each of the above 5 services are hosted in their own dedicated containers.
Fixes #94
Information on docker image
Checklist
root.jsonin alphabetical order (for new rock-on only)"description"object lists and links to the docker image used"description"object provides information on the image's particularities (advantage over another existing rock-on for the same project, for instance)"website"object links to project's main website