false positives with T0 #199
Comments
|
Here is another one: |
|
Another one: |
xairy
referenced
this issue
in google/ktsan
Aug 31, 2015
Nikolay has reported a hang when a memcg reclaim got stuck with the following backtrace: PID: 18308 TASK: ffff883d7c9b0a30 CPU: 1 COMMAND: "rsync" #0 __schedule at ffffffff815ab152 #1 schedule at ffffffff815ab76e #2 schedule_timeout at ffffffff815ae5e5 #3 io_schedule_timeout at ffffffff815aad6a #4 bit_wait_io at ffffffff815abfc6 #5 __wait_on_bit at ffffffff815abda5 #6 wait_on_page_bit at ffffffff8111fd4f #7 shrink_page_list at ffffffff81135445 #8 shrink_inactive_list at ffffffff81135845 #9 shrink_lruvec at ffffffff81135ead #10 shrink_zone at ffffffff811360c3 #11 shrink_zones at ffffffff81136eff #12 do_try_to_free_pages at ffffffff8113712f #13 try_to_free_mem_cgroup_pages at ffffffff811372be #14 try_charge at ffffffff81189423 #15 mem_cgroup_try_charge at ffffffff8118c6f5 #16 __add_to_page_cache_locked at ffffffff8112137d #17 add_to_page_cache_lru at ffffffff81121618 #18 pagecache_get_page at ffffffff8112170b #19 grow_dev_page at ffffffff811c8297 #20 __getblk_slow at ffffffff811c91d6 #21 __getblk_gfp at ffffffff811c92c1 #22 ext4_ext_grow_indepth at ffffffff8124565c #23 ext4_ext_create_new_leaf at ffffffff81246ca8 #24 ext4_ext_insert_extent at ffffffff81246f09 #25 ext4_ext_map_blocks at ffffffff8124a848 #26 ext4_map_blocks at ffffffff8121a5b7 #27 mpage_map_one_extent at ffffffff8121b1fa #28 mpage_map_and_submit_extent at ffffffff8121f07b #29 ext4_writepages at ffffffff8121f6d5 #30 do_writepages at ffffffff8112c490 #31 __filemap_fdatawrite_range at ffffffff81120199 #32 filemap_flush at ffffffff8112041c #33 ext4_alloc_da_blocks at ffffffff81219da1 #34 ext4_rename at ffffffff81229b91 #35 ext4_rename2 at ffffffff81229e32 #36 vfs_rename at ffffffff811a08a5 #37 SYSC_renameat2 at ffffffff811a3ffc #38 sys_renameat2 at ffffffff811a408e #39 sys_rename at ffffffff8119e51e #40 system_call_fastpath at ffffffff815afa89 Dave Chinner has properly pointed out that this is a deadlock in the reclaim code because ext4 doesn't submit pages which are marked by PG_writeback right away. The heuristic was introduced by commit e62e384 ("memcg: prevent OOM with too many dirty pages") and it was applied only when may_enter_fs was specified. The code has been changed by c3b94f4 ("memcg: further prevent OOM with too many dirty pages") which has removed the __GFP_FS restriction with a reasoning that we do not get into the fs code. But this is not sufficient apparently because the fs doesn't necessarily submit pages marked PG_writeback for IO right away. ext4_bio_write_page calls io_submit_add_bh but that doesn't necessarily submit the bio. Instead it tries to map more pages into the bio and mpage_map_one_extent might trigger memcg charge which might end up waiting on a page which is marked PG_writeback but hasn't been submitted yet so we would end up waiting for something that never finishes. Fix this issue by replacing __GFP_IO by may_enter_fs check (for case 2) before we go to wait on the writeback. The page fault path, which is the only path that triggers memcg oom killer since 3.12, shouldn't require GFP_NOFS and so we shouldn't reintroduce the premature OOM killer issue which was originally addressed by the heuristic. As per David Chinner the xfs is doing similar thing since 2.6.15 already so ext4 is not the only affected filesystem. Moreover he notes: : For example: IO completion might require unwritten extent conversion : which executes filesystem transactions and GFP_NOFS allocations. The : writeback flag on the pages can not be cleared until unwritten : extent conversion completes. Hence memory reclaim cannot wait on : page writeback to complete in GFP_NOFS context because it is not : safe to do so, memcg reclaim or otherwise. Cc: [email protected] # 3.9+ [[email protected]: corrected the control flow] Fixes: c3b94f4 ("memcg: further prevent OOM with too many dirty pages") Reported-by: Nikolay Borisov <[email protected]> Signed-off-by: Michal Hocko <[email protected]> Signed-off-by: Hugh Dickins <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>
|
The last report (handle_edge_irq) turned out to be a real race. |
melver
referenced
this issue
in google/ktsan
Nov 4, 2019
In nsim_fib_init(), if register_fib_notifier failed, nsim_fib_net_ops should be unregistered before return. In nsim_fib_exit(), unregister_fib_notifier should be called before nsim_fib_net_ops be unregistered, otherwise may cause use-after-free: BUG: KASAN: use-after-free in nsim_fib_event_nb+0x342/0x570 [netdevsim] Read of size 8 at addr ffff8881daaf4388 by task kworker/0:3/3499 CPU: 0 PID: 3499 Comm: kworker/0:3 Not tainted 5.3.0-rc7+ #30 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014 Workqueue: ipv6_addrconf addrconf_dad_work [ipv6] Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0xa9/0x10e lib/dump_stack.c:113 print_address_description+0x65/0x380 mm/kasan/report.c:351 __kasan_report+0x149/0x18d mm/kasan/report.c:482 kasan_report+0xe/0x20 mm/kasan/common.c:618 nsim_fib_event_nb+0x342/0x570 [netdevsim] notifier_call_chain+0x52/0xf0 kernel/notifier.c:95 __atomic_notifier_call_chain+0x78/0x140 kernel/notifier.c:185 call_fib_notifiers+0x30/0x60 net/core/fib_notifier.c:30 call_fib6_entry_notifiers+0xc1/0x100 [ipv6] fib6_add+0x92e/0x1b10 [ipv6] __ip6_ins_rt+0x40/0x60 [ipv6] ip6_ins_rt+0x84/0xb0 [ipv6] __ipv6_ifa_notify+0x4b6/0x550 [ipv6] ipv6_ifa_notify+0xa5/0x180 [ipv6] addrconf_dad_completed+0xca/0x640 [ipv6] addrconf_dad_work+0x296/0x960 [ipv6] process_one_work+0x5c0/0xc00 kernel/workqueue.c:2269 worker_thread+0x5c/0x670 kernel/workqueue.c:2415 kthread+0x1d7/0x200 kernel/kthread.c:255 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352 Allocated by task 3388: save_stack+0x19/0x80 mm/kasan/common.c:69 set_track mm/kasan/common.c:77 [inline] __kasan_kmalloc.constprop.3+0xa0/0xd0 mm/kasan/common.c:493 kmalloc include/linux/slab.h:557 [inline] kzalloc include/linux/slab.h:748 [inline] ops_init+0xa9/0x220 net/core/net_namespace.c:127 __register_pernet_operations net/core/net_namespace.c:1135 [inline] register_pernet_operations+0x1d4/0x420 net/core/net_namespace.c:1212 register_pernet_subsys+0x24/0x40 net/core/net_namespace.c:1253 nsim_fib_init+0x12/0x70 [netdevsim] veth_get_link_ksettings+0x2b/0x50 [veth] do_one_initcall+0xd4/0x454 init/main.c:939 do_init_module+0xe0/0x330 kernel/module.c:3490 load_module+0x3c2f/0x4620 kernel/module.c:3841 __do_sys_finit_module+0x163/0x190 kernel/module.c:3931 do_syscall_64+0x72/0x2e0 arch/x86/entry/common.c:296 entry_SYSCALL_64_after_hwframe+0x49/0xbe Freed by task 3534: save_stack+0x19/0x80 mm/kasan/common.c:69 set_track mm/kasan/common.c:77 [inline] __kasan_slab_free+0x130/0x180 mm/kasan/common.c:455 slab_free_hook mm/slub.c:1423 [inline] slab_free_freelist_hook mm/slub.c:1474 [inline] slab_free mm/slub.c:3016 [inline] kfree+0xe9/0x2d0 mm/slub.c:3957 ops_free net/core/net_namespace.c:151 [inline] ops_free_list.part.7+0x156/0x220 net/core/net_namespace.c:184 ops_free_list net/core/net_namespace.c:182 [inline] __unregister_pernet_operations net/core/net_namespace.c:1165 [inline] unregister_pernet_operations+0x221/0x2a0 net/core/net_namespace.c:1224 unregister_pernet_subsys+0x1d/0x30 net/core/net_namespace.c:1271 nsim_fib_exit+0x11/0x20 [netdevsim] nsim_module_exit+0x16/0x21 [netdevsim] __do_sys_delete_module kernel/module.c:1015 [inline] __se_sys_delete_module kernel/module.c:958 [inline] __x64_sys_delete_module+0x244/0x330 kernel/module.c:958 do_syscall_64+0x72/0x2e0 arch/x86/entry/common.c:296 entry_SYSCALL_64_after_hwframe+0x49/0xbe Reported-by: Hulk Robot <[email protected]> Fixes: 59c84b9 ("netdevsim: Restore per-network namespace accounting for fib entries") Signed-off-by: YueHaibing <[email protected]> Acked-by: Jakub Kicinski <[email protected]> Signed-off-by: David S. Miller <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I am seeing false positives with T0. Presumably both racy accesses are executed on the same CPU, but first accesses is executed on a thread context, while another on the scheduler context (T0).
The text was updated successfully, but these errors were encountered: