-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Startup error #456
Comments
I used the following command to create a container, but it failed.
|
I want to persist my station ID, but I failed to start using the above command
|
Are you saying it failed because it doesn't produce any logs beside "v20.12.2"? |
Yes, Under normal circumstances, it should output Spark's work log, but I failed to create the container using the above command, and the container terminated after creation
|
I can reproduce that it exits after the version message when you pass a state root this way |
actually I don't know why it logs |
@bajtos you initially looked into mounting state_root, do you have an idea here? |
I'm also curious where |
This v20.12.2 looks to be the Node.js version. Did some poking around as well, as would like to persist the state between the upgrades (deleting and then installing the container again), however the issue looks to be, at least on this platform, the containers can't do any write operations at the container mount, meaning the modules can't create directories, nor any files on the host as that will result in permissions error.
My solution for now is to install the container, shell into it, cat station_id, create station_id at the host, mount the file, restart the container and since it will be R/O, it will run without any issues. |
AFAICT, this is caused by On fixing the permissions: Maybe we must explicitly tell Docker to mount the volume as read-write? The following command works for me on macOS and writes state files to
Can you please check whether the command above works on your machine too?
Persisting Persisting other files is not strictly required now. However, we may add more state files in the future that require to be persisted, like the recently introduced I highly recommend persisting the entire state directory. It's also important to NOT share the state between Station instances, we expect each Station to have its own exclusive state storage. |
I have made attempts in both the sudo user group and root user groups
Under the sudo user group
|
Under the root user
|
The above two attempts all failed to create a new station |
Thank you, @csmasterpath2023, for testing. Maybe this issue is specific to Linux? The following StackOverflow answer explains the problem with users & groups and permissions inside the Docker container: https://stackoverflow.com/a/29251160/69868 It looks too complicated to me, I would hope the situation has improved since 2015 when the answer was posted. But maybe it's a place where to start? |
Yes , I use the Ubuntu 23.10 (GNU/Linux 6.5.0-28-generic x86_64) |
@bajtos You are welcome, I haven't looked at the link you provided yet, but I think the threshold for the vast majority of users is so high that we may lose a lot of users |
Can you please run the following command to list directories & permissions and post the output?
This is the output I get on my macOS:
|
|
Ah, of course, Station Core was not able to create any state files/directories, therefore the second Could you please run the following two commands? Check permissions inside the container:
Check the permissions on your host computer:
|
This is really weird. Here is what I see on my computer: In the container:
On the host:
On the container, the root-owned
|
In the container
|
On the host
|
ok, I think I know what can be problem here.
I think you should be able to find that user by running According to https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04, the Docker service runs under the user in the group
Alternatively, if the Docker service is running under the
|
@bajtos thank you very much, it works now |
@csmasterpath2023 could you please describe what commands you executed? Other operators running on Linux can find that useful. |
As you said, if the Docker service is running under the docker user, then you can change the owning user instead of the owning group.
|
@bajtos But if I use /tmp/state , then I reboot the machine, the directory will be deleted automatically So, I modified the path to |
In my case I had to change the folder owner and group of the local folder to UID/GID 1000:1000. Since that's what is used internally for the "node" user in the container. After that it worked to keep the files persistent. |
Originally posted by @juliangruber in #444 (comment)
The text was updated successfully, but these errors were encountered: