Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terms don't match in node shutdown test #20

Open
studzien opened this issue Oct 17, 2019 · 0 comments
Open

Terms don't match in node shutdown test #20

studzien opened this issue Oct 17, 2019 · 0 comments

Comments

@studzien
Copy link

studzien commented Oct 17, 2019

What are we doing to trigger the bug:

while (mix test test/fuzzy/partitions_test.exs:36); do sleep 1; done

What we expect:

  • When running the fuzzy/partitions_test repeatedly, we expect it to pass every time.

What happens:

Some runs trigger the following test failure:

   1) test node shutdowns (Raft.Fuzzy.PartitionsTest)
     test/fuzzy/partitions_test.exs:36
     ** (RuntimeError) Terms don't match for  [s0: %{logs: [%Raft.Log.Entry{data: %Raft.Configuration{index: 0, new_servers: [], old_servers: [], state: :none}, index: 0, term: 0, type: :config}, %Raft.Log.Entry (...)

Additional info:

We have added the following debug output to try to understand what precisely
goes wrong during the failed test runs.

diff --git a/test/support/cluster.ex b/test/support/cluster.ex
index 25968bd..de65f99 100644
--- a/test/support/cluster.ex
+++ b/test/support/cluster.ex
@@ -87,6 +87,8 @@ defmodule Raft.Support.Cluster do
       {s, data}
     end

+    IO.inspect data, label: :data, limit: :infinity
+    IO.inspect commands, label: :commands, limit: :infinity
     verify_terms(data) && verify_logs(data, commands)
   end

We have also simplified the StreamData generator to make the command logs easier to read:

diff --git a/test/fuzzy/partitions_test.exs b/test/fuzzy/partitions_test.exs
index 6bee922..11e7bd4 100644
--- a/test/fuzzy/partitions_test.exs
+++ b/test/fuzzy/partitions_test.exs
@@ -10,7 +10,7 @@ defmodule Raft.Fuzzy.PartitionsTest do
   }

   def commands, do: one_of([
-    tuple({constant(:put), term()}),
+    tuple({constant(:put), integer()}),
     constant(:pop),
   ])

Attached is the full output of a failed test, run with the above changes in place.
terms-dont-match.log

/cc @pzel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant