ccl/changefeedccl: TestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart failed

[github-actions]

ccl/changefeedccl.TestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart failed on master @ 24ac6d76b5c16eca9f29611779f808b2394188ed:

=== RUN   TestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart
    test_log_scope.go:165: test logs captured to: outputs.zip/logTestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart3313006657
    test_log_scope.go:76: use -show-logs to present logs inline
    helpers_test.go:962: making enterprise feed factory
    helpers_test.go:976: pgURL enterprise root
    changefeed_test.go:6572: 
        	Error Trace:	pkg/ccl/changefeedccl/changefeed_test.go:6572
        	Error:      	Received unexpected error:
        	            	still waiting for job status; current running
        	            	(1) attached stack trace
        	            	  -- stack trace:
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*jobFeed).WaitForStatus.func1
        	            	  | 	pkg/ccl/changefeedccl/testfeed_test.go:458
        	            	  | github.com/cockroachdb/cockroach/pkg/testutils.SucceedsWithinError.func1
        	            	  | 	pkg/testutils/soon.go:71
        	            	  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
        	            	  | 	pkg/util/retry/retry.go:213
        	            	  | github.com/cockroachdb/cockroach/pkg/testutils.SucceedsWithinError
        	            	  | 	pkg/testutils/soon.go:77
        	            	  | github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoonError
        	            	  | 	pkg/testutils/soon.go:47
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*jobFeed).WaitForStatus
        	            	  | 	pkg/ccl/changefeedccl/testfeed_test.go:449
        	            	  | github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.TestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart
        	            	  | 	pkg/ccl/changefeedccl/changefeed_test.go:6572
        	            	  | testing.tRunner
        	            	  | 	GOROOT/src/testing/testing.go:1689
        	            	  | runtime.goexit
        	            	  | 	src/runtime/asm_amd64.s:1695
        	            	Wraps: (2) still waiting for job status; current running
        	            	Error types: (1) *withstack.withStack (2) *errutil.leafError
        	Test:       	TestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart3313006657
--- FAIL: TestChangefeedTimelyResolvedTimestampUpdatePostRollingRestart (178.30s)

Parameters:

• attempt=1
• run=1
• shard=3

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/cdc

This test on roachdash | Improve this report!

Jira issue: CRDB-44286

[asg0451]

I managed to repro this locally after running it thousands of times and got the attached stack trace and test.log. it seems to indicate that the feed is still waiting on data from kv and that it hasnt yet hit its configured end time. can someone from kv take a look at it?

goros.txt
test.log

[blathers-crl]

Based on the specified backports for linked PR #135194, I applied the following new label(s) to this issue: branch-release-24.3. Please adjust the labels as needed to match the branches actually affected by this issue, including adding any known older branches.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}