Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Failed to start docker container #26

Open
quaff opened this issue Jul 7, 2023 · 1 comment
Open

[Bug]: Failed to start docker container #26

quaff opened this issue Jul 7, 2023 · 1 comment

Comments

@quaff
Copy link

quaff commented Jul 7, 2023

Environment

  • OS Version and CPU Arch(uname -a): Darwin 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:22 PDT 2023; root:xnu-8796.121.3~7/RELEASE_X86_64 x86_64

  • Docker Desktop Version 4.20.1 (110738)

Fast Reproduce Steps(Required)

Steps to reproduce the behavior:

  1. docker pull oceanbase/oceanbase-ce
  2. docker run -p 2881:2881 --name obstandalone -e MINI_MODE=1 -d oceanbase/oceanbase-ce

Expected behavior
Container obstandalone keep running

Actual Behavior
Container obstandalone exited with failure

Additional context
docker exec -it obstandalone obd display-trace a0cbb15e-1c7c-11ee-acab-0242ac110002

[2023-07-07 04:13:29.429] [DEBUG] - mkdir /root/.obd/lock/
[2023-07-07 04:13:29.430] [DEBUG] - unknown lock mode 
[2023-07-07 04:13:29.430] [DEBUG] - try to get share lock /root/.obd/lock/global
[2023-07-07 04:13:29.430] [DEBUG] - share lock `/root/.obd/lock/global`, count 1
[2023-07-07 04:13:29.430] [DEBUG] - cmd: ['obcluster']
[2023-07-07 04:13:29.430] [DEBUG] - opts: {'servers': None, 'components': None, 'force_delete': None, 'strict_check': None, 'without_parameter': None}
[2023-07-07 04:13:29.430] [DEBUG] - Get Deploy by name
[2023-07-07 04:13:29.430] [DEBUG] - mkdir /root/.obd/cluster/
[2023-07-07 04:13:29.431] [DEBUG] - mkdir /root/.obd/config_parser/
[2023-07-07 04:13:29.431] [DEBUG] - try to get exclusive lock /root/.obd/lock/deploy_obcluster
[2023-07-07 04:13:29.431] [DEBUG] - exclusive lock `/root/.obd/lock/deploy_obcluster`, count 1
[2023-07-07 04:13:29.436] [DEBUG] - Deploy status judge
[2023-07-07 04:13:29.438] [INFO] Get local repositories
[2023-07-07 04:13:29.438] [DEBUG] - mkdir /root/.obd/repository
[2023-07-07 04:13:29.438] [DEBUG] - Get local repository oceanbase-ce-4.1.0.0-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.438] [DEBUG] - Search repository oceanbase-ce version: 4.1.0.0, tag: 8439ecf8db5e0649bd49671b41ea9e8c85756b63, release: None, package_hash: None
[2023-07-07 04:13:29.438] [DEBUG] - try to get share lock /root/.obd/lock/mirror_and_repo
[2023-07-07 04:13:29.438] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 1
[2023-07-07 04:13:29.438] [DEBUG] - mkdir /root/.obd/repository/oceanbase-ce
[2023-07-07 04:13:29.441] [DEBUG] - Found repository oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.441] [DEBUG] - Get local repository obagent-1.3.0-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.441] [DEBUG] - Search repository obagent version: 1.3.0, tag: d57fbb4962b2fbecb6282358c59295fdfba4d6ac, release: None, package_hash: None
[2023-07-07 04:13:29.441] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 2
[2023-07-07 04:13:29.441] [DEBUG] - mkdir /root/.obd/repository/obagent
[2023-07-07 04:13:29.444] [DEBUG] - Found repository obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.444] [DEBUG] - Get deploy config
[2023-07-07 04:13:29.465] [INFO] Search plugins
[2023-07-07 04:13:29.466] [DEBUG] - Searching start_check plugin for components ...
[2023-07-07 04:13:29.466] [DEBUG] - Searching start_check plugin for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.466] [DEBUG] - mkdir /root/.obd/plugins
[2023-07-07 04:13:29.468] [DEBUG] - Found for oceanbase-ce-py_script_start_check-4.0.0.0 for oceanbase-ce-4.1.0.0
[2023-07-07 04:13:29.468] [DEBUG] - Searching start_check plugin for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.469] [DEBUG] - Found for obagent-py_script_start_check-1.3.0 for obagent-1.3.0
[2023-07-07 04:13:29.469] [DEBUG] - Searching create_tenant plugin for components ...
[2023-07-07 04:13:29.469] [DEBUG] - Searching create_tenant plugin for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.470] [DEBUG] - Found for oceanbase-ce-py_script_create_tenant-4.0.0.0 for oceanbase-ce-4.1.0.0
[2023-07-07 04:13:29.470] [DEBUG] - Searching create_tenant plugin for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.470] [DEBUG] - No such create_tenant plugin for obagent-1.3.0
[2023-07-07 04:13:29.470] [DEBUG] - Searching start plugin for components ...
[2023-07-07 04:13:29.470] [DEBUG] - Searching start plugin for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.470] [DEBUG] - Found for oceanbase-ce-py_script_start-4.0.0.0 for oceanbase-ce-4.1.0.0
[2023-07-07 04:13:29.470] [DEBUG] - Searching start plugin for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.471] [DEBUG] - Found for obagent-py_script_start-1.3.0 for obagent-1.3.0
[2023-07-07 04:13:29.471] [DEBUG] - Searching connect plugin for components ...
[2023-07-07 04:13:29.471] [DEBUG] - Searching connect plugin for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.471] [DEBUG] - Found for oceanbase-ce-py_script_connect-3.1.0 for oceanbase-ce-4.1.0.0
[2023-07-07 04:13:29.471] [DEBUG] - Searching connect plugin for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.471] [DEBUG] - Found for obagent-py_script_connect-1.3.0 for obagent-1.3.0
[2023-07-07 04:13:29.471] [DEBUG] - Searching bootstrap plugin for components ...
[2023-07-07 04:13:29.471] [DEBUG] - Searching bootstrap plugin for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.472] [DEBUG] - Found for oceanbase-ce-py_script_bootstrap-4.0.0.0 for oceanbase-ce-4.1.0.0
[2023-07-07 04:13:29.472] [DEBUG] - Searching bootstrap plugin for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.472] [DEBUG] - Found for obagent-py_script_bootstrap-0.1 for obagent-1.3.0
[2023-07-07 04:13:29.472] [DEBUG] - Searching display plugin for components ...
[2023-07-07 04:13:29.472] [DEBUG] - Searching display plugin for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:29.472] [DEBUG] - Found for oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.1.0.0
[2023-07-07 04:13:29.473] [DEBUG] - Searching display plugin for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:29.473] [DEBUG] - Found for obagent-py_script_display-1.3.0 for obagent-1.3.0
[2023-07-07 04:13:29.473] [INFO] Open ssh connection
[2023-07-07 04:13:29.474] [INFO] Load cluster param plugin
[2023-07-07 04:13:29.474] [DEBUG] - Searching param plugin for components ...
[2023-07-07 04:13:29.474] [DEBUG] - Search param plugin for oceanbase-ce
[2023-07-07 04:13:29.474] [DEBUG] - Found for oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.0
[2023-07-07 04:13:29.474] [DEBUG] - Applying oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:30.001] [DEBUG] - Search param plugin for obagent
[2023-07-07 04:13:30.001] [DEBUG] - Found for obagent-param-1.3.0 for obagent-1.3.0
[2023-07-07 04:13:30.001] [DEBUG] - Applying obagent-param-1.3.0 for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:30.059] [DEBUG] - Call oceanbase-ce-py_script_start_check-4.0.0.0 for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:30.059] [DEBUG] - import start_check
[2023-07-07 04:13:30.065] [DEBUG] - add start_check ref count to 1
[2023-07-07 04:13:30.066] [INFO] Check before start observer
[2023-07-07 04:13:30.066] [DEBUG] -- local execute: ls /root/ob/store/clog/tenant_1/ 
[2023-07-07 04:13:30.077] [DEBUG] -- exited code 2, error output:
[2023-07-07 04:13:30.077] [DEBUG] ls: cannot access /root/ob/store/clog/tenant_1/: No such file or directory
[2023-07-07 04:13:30.077] [DEBUG] 
[2023-07-07 04:13:30.077] [DEBUG] -- local execute: cat /root/ob/run/observer.pid 
[2023-07-07 04:13:30.081] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.082] [DEBUG] -- local execute: ls /proc/221 
[2023-07-07 04:13:30.086] [DEBUG] -- exited code 2, error output:
[2023-07-07 04:13:30.086] [DEBUG] ls: cannot access /proc/221: No such file or directory
[2023-07-07 04:13:30.087] [DEBUG] 
[2023-07-07 04:13:30.087] [DEBUG] -- 127.0.0.1 port check
[2023-07-07 04:13:30.087] [DEBUG] -- local execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:0B41' | awk -F' ' '{print $2}' | uniq 
[2023-07-07 04:13:30.093] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.094] [DEBUG] -- local execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:0B42' | awk -F' ' '{print $2}' | uniq 
[2023-07-07 04:13:30.101] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.102] [DEBUG] -- local execute: ls /root/ob/store/sstable/block_file 
[2023-07-07 04:13:30.107] [DEBUG] -- exited code 2, error output:
[2023-07-07 04:13:30.107] [DEBUG] ls: cannot access /root/ob/store/sstable/block_file: No such file or directory
[2023-07-07 04:13:30.107] [DEBUG] 
[2023-07-07 04:13:30.107] [DEBUG] -- local execute: grep -e '^ *lo:' /proc/net/dev 
[2023-07-07 04:13:30.110] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.110] [DEBUG] -- local execute: cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr 
[2023-07-07 04:13:30.113] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.114] [DEBUG] -- local execute: ulimit -a 
[2023-07-07 04:13:30.117] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.118] [DEBUG] -- local execute: cat /proc/meminfo 
[2023-07-07 04:13:30.120] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.121] [DEBUG] -- local execute: df --block-size=1024  
[2023-07-07 04:13:30.125] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.126] [DEBUG] -- get disk info for path /, total: 251650445312 avail: 210873688064
[2023-07-07 04:13:30.126] [DEBUG] -- get disk info for path /dev, total: 67108864 avail: 67108864
[2023-07-07 04:13:30.126] [DEBUG] -- get disk info for path /dev/shm, total: 67108864 avail: 67108864
[2023-07-07 04:13:30.126] [DEBUG] -- get disk info for path /etc/hosts, total: 251650445312 avail: 210873688064
[2023-07-07 04:13:30.126] [DEBUG] -- get disk info for path /proc/acpi, total: 6280445952 avail: 6280445952
[2023-07-07 04:13:30.126] [DEBUG] -- get disk info for path /sys/firmware, total: 6280445952 avail: 6280445952
[2023-07-07 04:13:30.126] [DEBUG] -- local execute: df --block-size=1024 /root/ob/store/clog 
[2023-07-07 04:13:30.129] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.129] [DEBUG] -- get disk info for path /, total: 251650445312 avail: 210873688064
[2023-07-07 04:13:30.129] [DEBUG] -- local execute: df --block-size=1024 /root/ob/store 
[2023-07-07 04:13:30.132] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.132] [DEBUG] -- get disk info for path /, total: 251650445312 avail: 210873688064
[2023-07-07 04:13:30.132] [DEBUG] -- disk: {'/': {'total': 251650445312, 'avail': 210873688064, 'need': 0}, '/dev': {'total': 67108864, 'avail': 67108864, 'need': 0}, '/dev/shm': {'total': 67108864, 'avail': 67108864, 'need': 0}, '/etc/hosts': {'total': 251650445312, 'avail': 210873688064, 'need': 0}, '/proc/acpi': {'total': 6280445952, 'avail': 6280445952, 'need': 0}, '/sys/firmware': {'total': 6280445952, 'avail': 6280445952, 'need': 0}}
[2023-07-07 04:13:30.133] [WARNING] OBD-1012: (127.0.0.1) clog and data use the same disk (/)
[2023-07-07 04:13:30.133] [DEBUG] -- local execute: date +%s%N 
[2023-07-07 04:13:30.136] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.136] [DEBUG] -- 127.0.0.1 time delta -0.441162109375
[2023-07-07 04:13:30.136] [INFO] [WARN] OBD-1012: (127.0.0.1) clog and data use the same disk (/)
[2023-07-07 04:13:30.136] [INFO] 
[2023-07-07 04:13:30.136] [DEBUG] - sub start_check ref count to 0
[2023-07-07 04:13:30.136] [DEBUG] - export start_check
[2023-07-07 04:13:30.137] [DEBUG] - Call obagent-py_script_start_check-1.3.0 for obagent-1.3.0-22.el7-d57fbb4962b2fbecb6282358c59295fdfba4d6ac
[2023-07-07 04:13:30.137] [DEBUG] - import start_check
[2023-07-07 04:13:30.139] [DEBUG] - add start_check ref count to 1
[2023-07-07 04:13:30.139] [INFO] Check before start obagent
[2023-07-07 04:13:30.141] [DEBUG] -- local execute: cat /root/obagent/run/ob_agentd.pid 
[2023-07-07 04:13:30.145] [DEBUG] -- exited code 1, error output:
[2023-07-07 04:13:30.145] [DEBUG] cat: /root/obagent/run/ob_agentd.pid: No such file or directory
[2023-07-07 04:13:30.145] [DEBUG] 
[2023-07-07 04:13:30.145] [DEBUG] -- 127.0.0.1 port check
[2023-07-07 04:13:30.145] [DEBUG] -- local execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F99' | awk -F' ' '{print $2}' | uniq 
[2023-07-07 04:13:30.152] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.152] [DEBUG] -- local execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F98' | awk -F' ' '{print $2}' | uniq 
[2023-07-07 04:13:30.158] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.159] [DEBUG] - sub start_check ref count to 0
[2023-07-07 04:13:30.159] [DEBUG] - export start_check
[2023-07-07 04:13:30.159] [DEBUG] - Call oceanbase-ce-py_script_start-4.0.0.0 for oceanbase-ce-4.1.0.0-100000192023032010.el7-8439ecf8db5e0649bd49671b41ea9e8c85756b63
[2023-07-07 04:13:30.159] [DEBUG] - import start
[2023-07-07 04:13:30.161] [DEBUG] - add start ref count to 1
[2023-07-07 04:13:30.162] [INFO] Start observer
[2023-07-07 04:13:30.162] [DEBUG] -- local execute: ls /root/ob/store/clog/tenant_1/ 
[2023-07-07 04:13:30.167] [DEBUG] -- exited code 2, error output:
[2023-07-07 04:13:30.167] [DEBUG] ls: cannot access /root/ob/store/clog/tenant_1/: No such file or directory
[2023-07-07 04:13:30.167] [DEBUG] 
[2023-07-07 04:13:30.167] [DEBUG] -- local execute: cat /root/ob/run/observer.pid 
[2023-07-07 04:13:30.171] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.171] [DEBUG] -- local execute: ls /proc/221 
[2023-07-07 04:13:30.176] [DEBUG] -- exited code 2, error output:
[2023-07-07 04:13:30.176] [DEBUG] ls: cannot access /proc/221: No such file or directory
[2023-07-07 04:13:30.176] [DEBUG] 
[2023-07-07 04:13:30.176] [DEBUG] -- 127.0.0.1 start command construction
[2023-07-07 04:13:30.176] [DEBUG] -- starting 127.0.0.1 observer
[2023-07-07 04:13:30.177] [DEBUG] -- [email protected] set env LD_LIBRARY_PATH to '/root/ob/lib:'
[2023-07-07 04:13:30.177] [DEBUG] -- local execute: cd /root/ob; /root/ob/bin/observer -r '127.0.0.1:2882:2881' -p 2881 -P 2882 -z 'zone1' -n 'obcluster' -c 1 -d '/root/ob/store' -i 'lo' -l 'INFO' -o __min_full_resource_pool_memory=2147483648,memory_limit='6G',system_memory='1G',datafile_size='5G',log_disk_size='15G',cpu_count=16,enable_syslog_wf=False,enable_syslog_recycle=True,max_syslog_file_count=4 
[2023-07-07 04:13:30.588] [DEBUG] -- exited code 0
[2023-07-07 04:13:30.589] [DEBUG] -- [email protected] set env LD_LIBRARY_PATH to '/tmp/_MEIFrdMcB'
[2023-07-07 04:13:30.589] [INFO] observer program health check
[2023-07-07 04:13:33.592] [DEBUG] -- 127.0.0.1 program health check
[2023-07-07 04:13:33.592] [DEBUG] -- local execute: cat /root/ob/run/observer.pid 
[2023-07-07 04:13:33.596] [DEBUG] -- exited code 0
[2023-07-07 04:13:33.596] [DEBUG] -- local execute: ls /proc/56 
[2023-07-07 04:13:33.600] [DEBUG] -- exited code 2, error output:
[2023-07-07 04:13:33.600] [DEBUG] ls: cannot access /proc/56: No such file or directory
[2023-07-07 04:13:33.601] [DEBUG] 
[2023-07-07 04:13:33.601] [WARNING] OBD-2002: Failed to start 127.0.0.1 observer
[2023-07-07 04:13:33.601] [DEBUG] - sub start ref count to 0
[2023-07-07 04:13:33.601] [DEBUG] - export start
[2023-07-07 04:13:33.601] [ERROR] oceanbase-ce start failed
[2023-07-07 04:13:33.601] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes .
[2023-07-07 04:13:33.601] [INFO] Trace ID: a0cbb15e-1c7c-11ee-acab-0242ac110002
[2023-07-07 04:13:33.602] [INFO] If you want to view detailed obd logs, please run: obd display-trace a0cbb15e-1c7c-11ee-acab-0242ac110002
[2023-07-07 04:13:33.602] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 1
[2023-07-07 04:13:33.602] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 0
[2023-07-07 04:13:33.602] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo
[2023-07-07 04:13:33.602] [DEBUG] - exclusive lock /root/.obd/lock/deploy_obcluster release, count 0
[2023-07-07 04:13:33.602] [DEBUG] - unlock /root/.obd/lock/deploy_obcluster
[2023-07-07 04:13:33.602] [DEBUG] - share lock /root/.obd/lock/global release, count 0
[2023-07-07 04:13:33.602] [DEBUG] - unlock /root/.obd/lock/global
@hnwyllmm
Copy link
Contributor

I think you were starting the oceanbase docker container on your mac.
But I cannot reproduce with your command on my mac.
The error message valuable is in the container's log(/root/ob/log/observer.log)

@hnwyllmm hnwyllmm transferred this issue from oceanbase/oceanbase Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@quaff @hnwyllmm