Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mainnet fullnode not sync, validator-engine keep crashing #985

Open
dnjef9duo opened this issue May 9, 2024 · 1 comment
Open

mainnet fullnode not sync, validator-engine keep crashing #985

dnjef9duo opened this issue May 9, 2024 · 1 comment

Comments

@dnjef9duo
Copy link

mainnet fullnode cannot sync
validator-engine keep crashing.

it runs smooth in the past year.
but it crashed almost every mins recently and stop syncing.
status shows local validator working time always very short.

what can we do ?

Machine

12 Cores
128G ram
2T disk

OS info

Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.3 LTS
Release:	20.04
mytonctrl > status

Mytoncore status: working, 5 hours
Local validator status: working, 1 minutes
Local validator out of sync: 465052 s
Local validator database size: 1505.89 Gb, 74.9%
Version mytonctrl: b97d502 (master)
Version validator: 4cfe1d1 (master)
tail -f /var/ton-work/log

[ 1][t 0][2024-05-09 08:04:38.289379708][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:05:05.001387141][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:05:16.805744396][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:05:22.401140903][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:05:24.038765442][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:05:29.238843791][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:05:41.629979098][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:05:50.597547245][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[pid 3123580] [time 1715241967] Signal: 6
------- Stack Backtrace -------
/usr/bin/ton/validator-engine/validator-engine[0x53ec79]
/usr/bin/ton/validator-engine/validator-engine[0x53d216]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f518dd83420]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f518d86a00b]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f518d849859]
/usr/bin/ton/validator-engine/validator-engine[0x548c67]
/usr/bin/ton/validator-engine/validator-engine[0x55b50a]
/usr/bin/ton/validator-engine/validator-engine[0x548996]
/usr/bin/ton/validator-engine/validator-engine[0x46d972]
/usr/bin/ton/validator-engine/validator-engine[0xabf4fa]
/usr/bin/ton/validator-engine/validator-engine[0xac6c91]
/usr/bin/ton/validator-engine/validator-engine[0xadbb17]
/usr/bin/ton/validator-engine/validator-engine[0x55c4af]
/usr/bin/ton/validator-engine/validator-engine[0x562c42]
/usr/bin/ton/validator-engine/validator-engine[0x561100]
/usr/bin/ton/validator-engine/validator-engine[0x56115e]
/usr/bin/ton/validator-engine/validator-engine[0x4b8890]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7f518dd77609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7f518d946353]
-------------------------------
[ 1][t 0][2024-05-09 08:06:40.733403801][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:06:51.352765713][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:01.208000893][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:02.184830228][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:10.572986009][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:11.720746323][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:14.118491353][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:32.105529929][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:33.285767072][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:37.010698391][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:40.812644431][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:07:45.200289302][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:08:02.006068970][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[pid 3125029] [time 1715242085] Signal: 6
------- Stack Backtrace -------
/usr/bin/ton/validator-engine/validator-engine[0x53ec79]
/usr/bin/ton/validator-engine/validator-engine[0x53d216]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fabdd5a7420]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7fabdd08e00b]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7fabdd06d859]
/usr/bin/ton/validator-engine/validator-engine[0x548c67]
/usr/bin/ton/validator-engine/validator-engine[0x55b50a]
/usr/bin/ton/validator-engine/validator-engine[0x548996]
/usr/bin/ton/validator-engine/validator-engine[0x46d972]
/usr/bin/ton/validator-engine/validator-engine[0xabf4fa]
/usr/bin/ton/validator-engine/validator-engine[0xac6c91]
/usr/bin/ton/validator-engine/validator-engine[0xadbb17]
/usr/bin/ton/validator-engine/validator-engine[0x55c4af]
/usr/bin/ton/validator-engine/validator-engine[0x562c42]
/usr/bin/ton/validator-engine/validator-engine[0x561100]
/usr/bin/ton/validator-engine/validator-engine[0x56115e]
/usr/bin/ton/validator-engine/validator-engine[0x4b8890]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7fabdd59b609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7fabdd16a353]
-------------------------------
[ 1][t 0][2024-05-09 08:08:43.892442663][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:08:43.904639881][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:08:47.518034250][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:08:54.577694618][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:09:12.925566974][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:09:21.804740038][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:09:46.532957447][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:09:50.562824176][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:09:53.892134064][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[ 1][t 0][2024-05-09 08:09:57.961116471][TcpListener.cpp:64][!TcpListener[port:18886]]	Accept
[pid 3127186] [time 1715242203] Signal: 6
------- Stack Backtrace -------
/usr/bin/ton/validator-engine/validator-engine[0x53ec79]
/usr/bin/ton/validator-engine/validator-engine[0x53d216]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f08d4f3f420]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f08d4a2600b]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f08d4a05859]
/usr/bin/ton/validator-engine/validator-engine[0x548c67]
/usr/bin/ton/validator-engine/validator-engine[0x55b50a]
/usr/bin/ton/validator-engine/validator-engine[0x548996]
/usr/bin/ton/validator-engine/validator-engine[0x46d972]
/usr/bin/ton/validator-engine/validator-engine[0xabf4fa]
/usr/bin/ton/validator-engine/validator-engine[0xac6c91]
/usr/bin/ton/validator-engine/validator-engine[0xadbb17]
/usr/bin/ton/validator-engine/validator-engine[0x55c4af]
/usr/bin/ton/validator-engine/validator-engine[0x562c42]
/usr/bin/ton/validator-engine/validator-engine[0x561100]
/usr/bin/ton/validator-engine/validator-engine[0x56115e]
/usr/bin/ton/validator-engine/validator-engine[0x4b8890]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7f08d4f33609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7f08d4b02353]
-------------------------------
@NoFeeding
Copy link

Same here, always Stack Backtrace. Any way I can fix this?
Version is the newest 2024.08

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants