Faulty transition for robustness test #29
Conversation
| @@ -163,6 +164,9 @@ func (ep *EtcdServerProcess) Start(ctx context.Context) error { | |||
| return err | |||
| } | |||
| } | |||
| if ep.cfg.FaultyTransitionEnabled { | |||
| ep.Failpoints().SetupEnv("faultyNetwork", "return") | |||
There was a problem hiding this comment.
This failpoint is specific for message transition, we should consider activate it through http in message_transition.go
There was a problem hiding this comment.
I haven't figured out a good way to enable this failpoint on the fly given that it would require to reconnect peers to make the new encoder effective. So far the easiest way is to set it up in the very beginning.
There was a problem hiding this comment.
How about keep using SetupEnv() but move it into Inject(). Kill all members, setup env for gofail and then restart members. It will minimize the unnecessary influence on other failpoints.
https://github.com/joshuazh-x/etcd/blob/55516234d324f64a97e1d49b7a8e3dba68de6ffd/tests/robustness/failpoint/gofail.go#L159-L178
| return err | ||
| } | ||
| } | ||
| return nil |
There was a problem hiding this comment.
Before exit, use latest gofail to get the execution count for this failpoint to check whether it was really executed.
$curl http://127.0.0.1:1234/SomeFuncString/count -XGET
| return fe | ||
| } | ||
|
|
||
| func (fe *faultyEncoder) encode(m *raftpb.Message) error { |
There was a problem hiding this comment.
In which part of the robustness test will this method be invoked? I only found it in unit tests.
This is a network transition fault test in message level. It covers various network faults that raft shall tolerate, including blocked transition, message lost, message duplication, message delay, and message reordering. Some of these faults cannot be easily applied via existing test framework.