-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tokumx crash after restart #1237
Comments
Could you post the content of the log file ? |
I rollback disk to 5 days ago( from a disk snapshot 5 days ago) . It's still crashed. Thu Oct 29 21:56:11.440 [initandlisten] TokuMX starting : pid=1143 port=27017 dbpath=/data/tokulogdb 64-bit host=iZ23b4efmaeZ Thu Oct 29 21:56:11.469 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokuportability.so(Z27db_env_do_backtrace_errfuncPFvPKviPKczES0+0x1f) [0x7f37d227c55f] Thu Oct 29 21:56:11.470 Memory usage: Arena 0: Thu Oct 29 21:56:11.472 |
I create a new database . It happened again .. :( Sun Nov 8 16:07:32.290 [conn16247] end connection 10.165.101.94:53452 (5 connections now open) ***** SERVER RESTARTED ***** Sun Nov 8 16:09:11.647 [initandlisten] TokuMX starting : pid=7622 port=28005 dbpath=/data/tokulogdb 64-bit host=iZ23yhdv81qZ Sun Nov 8 16:09:11.666 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokuportability.so(Z27db_env_do_backtrace_errfuncPFvPKviPKczES0+0x1f) [0x7f9a0918c55f] Sun Nov 8 16:09:11.666 Memory usage: Arena 0: Sun Nov 8 16:09:11.668 |
I have a tokumx database(single server). Today this db started crashing after I restarted it.( press ctrl + c to shutdown database)
[root@xxxxx bin]# ./tokumx --port 28005 --dbpath /data/tokulogdb
Thu Oct 29 19:59:30.905 nssize is a deprecated parameter
Thu Oct 29 19:59:30.908 [initandlisten] TokuMX starting : pid=11554 port=28005 dbpath=/data/tokulogdb 64-bit host=xxxxx
Thu Oct 29 19:59:30.908 [initandlisten] TokuMX mongod server v2.0.2-mongodb-2.4.10, using TokuKV rev unknown
Thu Oct 29 19:59:30.908 [initandlisten] git version: unknown
Thu Oct 29 19:59:30.908 [initandlisten] build info: Linux vps-centos5-x64-05.ci.percona.com 2.6.32-042stab106.6 #1 SMP Mon Apr 20 14:48:47 MSK 2015 x86_64 x86_64 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49
Thu Oct 29 19:59:30.908 [initandlisten] options: { dbpath: "/data/tokulogdb", port: 28005 }
Thu Oct 29 19:59:30.910 [initandlisten] [tokumx] startup
Thu Oct 29 19:59:30.911
Thu Oct 29 19:59:30.911 ================================================================================
Thu Oct 29 19:59:30.911 Fatal error detected
Thu Oct 29 19:59:30.911 ================================================================================
Thu Oct 29 19:59:30.911
Thu Oct 29 19:59:30.911 About to gather debugging information, please include all of the following along
Thu Oct 29 19:59:30.912 with logs from other servers in the cluster in a bug report.
Thu Oct 29 19:59:30.912
Thu Oct 29 19:59:30.912 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.912 Version info:
Thu Oct 29 19:59:30.912
Thu Oct 29 19:59:30.912 tokumxVersion: 2.0.2
Thu Oct 29 19:59:30.912 gitVersion: unknown
Thu Oct 29 19:59:30.912 tokukvVersion: unknown
Thu Oct 29 19:59:30.912 tokumxAuditVersion: unknown
Thu Oct 29 19:59:30.913 sysInfo: Linux vps-centos5-x64-05.ci.percona.com 2.6.32-042stab106.6 #1 SMP Mon Apr 20 14:48:47 MSK 2015 x86_64 x86_64 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49
Thu Oct 29 19:59:30.913 loaderFlags:
Thu Oct 29 19:59:30.913 compilerFlags: -fPIC -fno-strict-aliasing -ggdb -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -pipe -Wnon-virtual-dtor -Woverloaded-virtual -Wno-unused-local-typedefs -fno-builtin-memcmp -O3
Thu Oct 29 19:59:30.913 debug: false
Thu Oct 29 19:59:30.913
Thu Oct 29 19:59:30.913 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.913 Simple stacktrace:
Thu Oct 29 19:59:30.914
0xb62596 0x847dd4 0x849eff 0x6f284e 0x7f5c0d9f4130 0x7f5c0e24e3ac 0x7f5c0e2dfae0 0x7865ef 0x6fa097 0x6fa73f 0x6da40c 0x7f5c0ca08af5 0x6f1fad
./tokumx(_ZN5mongo15printStackTraceERSo+0x26) [0xb62596]
./tokumx() [0x847dd4]
./tokumx(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x1f) [0x849eff]
./tokumx(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x24e) [0x6f284e]
/lib64/libpthread.so.0(+0xf130) [0x7f5c0d9f4130]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1a3ac) [0x7f5c0e24e3ac]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0xabae0) [0x7f5c0e2dfae0]
./tokumx(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x38f) [0x7865ef]
./tokumx(_ZN5mongo14_initAndListenEi+0x407) [0x6fa097]
./tokumx(_ZN5mongo13initAndListenEi+0xf) [0x6fa73f]
./tokumx(main+0x2bc) [0x6da40c]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5c0ca08af5]
./tokumx() [0x6f1fad]
Thu Oct 29 19:59:30.918
Thu Oct 29 19:59:30.918 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.919 TokuKV engine backtrace:
Thu Oct 29 19:59:30.919
Thu Oct 29 19:59:30.919 Backtrace: (Note: toku_do_assert=0x0x7f5c0e02cca0)
Thu Oct 29 19:59:30.922 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokuportability.so(Z27db_env_do_backtrace_errfuncPFvPKviPKczES0+0x1f) [0x7f5c0e02e55f]
Thu Oct 29 19:59:30.922 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1447d) [0x7f5c0e24847d]
Thu Oct 29 19:59:30.922 ./tokumx() [0x847eed]
Thu Oct 29 19:59:30.922 ./tokumx(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x25) [0x849f05]
Thu Oct 29 19:59:30.922 ./tokumx(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x24e) [0x6f284e]
Thu Oct 29 19:59:30.923 /lib64/libpthread.so.0(+0xf130) [0x7f5c0d9f4130]
Thu Oct 29 19:59:30.923 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1a3ac) [0x7f5c0e24e3ac]
Thu Oct 29 19:59:30.923 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0xabae0) [0x7f5c0e2dfae0]
Thu Oct 29 19:59:30.923 ./tokumx(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x38f) [0x7865ef]
Thu Oct 29 19:59:30.923 ./tokumx(_ZN5mongo14_initAndListenEi+0x407) [0x6fa097]
Thu Oct 29 19:59:30.923 ./tokumx(_ZN5mongo13initAndListenEi+0xf) [0x6fa73f]
Thu Oct 29 19:59:30.923 ./tokumx(main+0x2bc) [0x6da40c]
Thu Oct 29 19:59:30.923 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5c0ca08af5]
Thu Oct 29 19:59:30.924 ./tokumx() [0x6f1fad]
Thu Oct 29 19:59:30.924 Engine status function not available
Thu Oct 29 19:59:30.924 Memory usage:
Arena 0:
system bytes = 0
in use bytes = 0
Total (incl. mmap):
system bytes = 0
in use bytes = 0
max mmap regions = 0
max mmap bytes = 0
Backtrace: (Note: toku_do_assert=0x0x7f5c0e02cca0)
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokuportability.so(_Z19db_env_do_backtraceP8_IO_FILE+0x1b)[0x7f5c0e02c19b]
./tokumx[0x847eed]
./tokumx(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x25)[0x849f05]
./tokumx(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x24e)[0x6f284e]
/lib64/libpthread.so.0(+0xf130)[0x7f5c0d9f4130]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1a3ac)[0x7f5c0e24e3ac]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0xabae0)[0x7f5c0e2dfae0]
./tokumx(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x38f)[0x7865ef]
./tokumx(_ZN5mongo14_initAndListenEi+0x407)[0x6fa097]
./tokumx(_ZN5mongo13initAndListenEi+0xf)[0x6fa73f]
./tokumx(main+0x2bc)[0x6da40c]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5c0ca08af5]
./tokumx[0x6f1fad]
Engine status function not available
Memory usage:
Arena 0:
system bytes = 0
in use bytes = 0
Total (incl. mmap):
system bytes = 0
in use bytes = 0
max mmap regions = 0
max mmap bytes = 0
Thu Oct 29 19:59:30.931
Thu Oct 29 19:59:30.931 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.931 Crash reason:
Thu Oct 29 19:59:30.931
Thu Oct 29 19:59:30.932 Invalid access at address: 0x7f5c187d6073 from thread: initandlisten
Thu Oct 29 19:59:30.932
Thu Oct 29 19:59:30.932 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.932 Process info:
Thu Oct 29 19:59:30.932
Thu Oct 29 19:59:30.932 OS: Linux CentOS Linux release 7.1.1503 (Core) Kernel 3.10.0-229.7.2.el7.x86_64 x86_64
Thu Oct 29 19:59:30.932 NCPU: 4
Thu Oct 29 19:59:30.932 VIRT: 196 MB
Thu Oct 29 19:59:30.933 RES: 196 MB
Thu Oct 29 19:59:30.933 PHYS: 7566 MB
Thu Oct 29 19:59:30.933 RLIMIT_CORE: 0 (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_CPU: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_DATA: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_FSIZE: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_NOFILE: 65535 (soft), 65535 (hard)
Thu Oct 29 19:59:30.933 RLIMIT_STACK: 8388608 (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_AS: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 _SC_OPEN_MAX: 65535
Thu Oct 29 19:59:30.934 _SC_PAGESIZE: 4096
Thu Oct 29 19:59:30.934 _SC_PHYS_PAGES: 1937060
Thu Oct 29 19:59:30.934 _SC_AVPHYS_PAGES: 261335
Thu Oct 29 19:59:30.934 _SC_NPROCESSORS_CONF: 4
Thu Oct 29 19:59:30.934 _SC_NPROCESSORS_ONLN: 4
Thu Oct 29 19:59:30.934
Thu Oct 29 19:59:30.934 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.934 Parsed server options:
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 dbpath: "/data/tokulogdb"
Thu Oct 29 19:59:30.935 port: 28005
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.935 Filesystem information:
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 Information for dbpath "/data/tokulogdb":
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 type magic: 0x0000EF53
Thu Oct 29 19:59:30.935 type: unknown
Thu Oct 29 19:59:30.935 bsize: 4096
Thu Oct 29 19:59:30.935 blocks: 128982382
Thu Oct 29 19:59:30.935 bfree: 24651634
Thu Oct 29 19:59:30.935 bavail: 18093951
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.935 Current operations in progress:
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 { opid: 0, active: false, op: "none", ns: "", query: {}, desc: "initandlisten", threadId: "0x7f5c0e965840", rootTxnid: 0, waitingForLock: false, lockStats: { timeLockedMicros: {}, timeAcquiringMicros: {} } }
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.936 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.936 OpDebug info:
Thu Oct 29 19:59:30.936
Thu Oct 29 19:59:30.936 none keyUpdates:0 0ms
Thu Oct 29 19:59:30.936
Thu Oct 29 19:59:30.936 Got signal: 11 (Segmentation fault).
Segmentation fault
The text was updated successfully, but these errors were encountered: