Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky data_iterator_test.go by adding locking #377

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

grodowski
Copy link
Member

@grodowski grodowski commented Dec 9, 2024

Fixes fatal error: concurrent map read and map write that can occur when listeners are called concurrently

Example stacktrace:

=== RUN   TestDataIterator/TestDoneListenerGetsNotifiedWhenDone
fatal error: concurrent map read and map write

goroutine 187 [running]:
runtime.throw({0xb21635?, 0x0?})
	/opt/hostedtoolcache/go/1.18.10/x64/src/runtime/panic.go:992 +0x71 fp=0xc000a0ece8 sp=0xc000a0ecb8 pc=0x437b91
runtime.mapaccess1_faststr(0x465bfb?, 0xa417e0?, {0xc00029ccc0, 0xc})
	/opt/hostedtoolcache/go/1.18.10/x64/src/runtime/map_faststr.go:22 +0x3a5 fp=0xc000a0ed50 sp=0xc000a0ece8 pc=0x4148c5
github.com/Shopify/ghostferry/test/go.(*DataIteratorTestSuite).SetupTest.func1(0x10158a0?)
	/home/runner/work/ghostferry/ghostferry/test/go/data_iterator_test.go:64 +0x56 fp=0xc000a0edc0 sp=0xc000a0ed50 pc=0x92b636
github.com/Shopify/ghostferry.(*DataIterator).Run.func1.1(0xc0006f81e0)
	/home/runner/work/ghostferry/ghostferry/data_iterator.go:138 +0x304 fp=0xc000a0f1d8 sp=0xc000a0edc0 pc=0x881bc4
github.com/Shopify/ghostferry.(*Cursor).Each(0xc000a0f9f8, 0xc000a0f878)
	/home/runner/work/ghostferry/ghostferry/cursor.go:145 +0x3b7 fp=0xc000a0f6d0 sp=0xc000a0f1d8 pc=0x87d7f7
github.com/Shopify/ghostferry.(*DataIterator).Run.func1()
	/home/runner/work/ghostferry/ghostferry/data_iterator.go:[107](https://github.com/Shopify/ghostferry/actions/runs/12238239633/job/34135851396#step:5:108) +0x6d8 fp=0xc000a0ffe0 sp=0xc000a0f6d0 pc=0x880f38
runtime.goexit()
	/opt/hostedtoolcache/go/1.18.10/x64/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc000a0ffe8 sp=0xc000a0ffe0 pc=0x46ab21
created by github.com/Shopify/ghostferry.(*DataIterator).Run
	/home/runner/work/ghostferry/ghostferry/data_iterator.go:71 +0x6ae
...

https://github.com/Shopify/ghostferry/actions/runs/12238239633/job/34135851396
https://github.com/Shopify/ghostferry/actions/runs/12239693187/job/34140746321

Fixes `fatal error: concurrent map read and map write` that can occur when listeners are called concurrently
@grodowski grodowski marked this pull request as ready for review December 9, 2024 20:43
@grodowski grodowski merged commit 24001d1 into main Dec 10, 2024
9 checks passed
@grodowski grodowski deleted the grodowski/fix-flaky-go-test branch December 10, 2024 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants