-
Notifications
You must be signed in to change notification settings - Fork 0
/
output.txt
1804 lines (1771 loc) · 237 KB
/
output.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Scenario: Configure minikube # src/test/resources/rcnit/00_validateEnvironment.feature:3
16:23:54.185 [main] INFO rcnit.ValidateEnvironment - Starting minikube (this might take a while)...
16:26:43.490 [main] INFO rcnit.ValidateEnvironment - minikube runs v1.9.0:
* minikube v1.9.0 on Microsoft Windows 10 Pro 10.0.18363 Build 18363
* Using the virtualbox driver based on user configuration
* Creating virtualbox VM (CPUs=4, Memory=8192MB, Disk=20000MB) ...
! Unable to verify SSH connectivity: dial tcp 192.168.99.254:22: i/o timeout. Will retry...
! Unable to verify SSH connectivity: dial tcp 192.168.99.254:22: i/o timeout. Will retry...
* Preparing Kubernetes v1.18.0 on Docker 19.03.8 ...
* Enabling addons: default-storageclass, storage-provisioner
* Done! kubectl is now configured to use "minikube"
Given minikube runs "v1.9.0" # rcnit.ValidateEnvironment.minikube_runs(java.lang.String)
16:26:43.740 [main] INFO rcnit.ValidateEnvironment - minikube uses the VirtualBox VM driver
And minikube uses the VirtualBox VM driver # rcnit.ValidateEnvironment.minikube_uses_the_virtualbox_vm_driver()
16:26:44.161 [main] INFO rcnit.ValidateEnvironment - minikube runs k8s v1.18.0
And minikube runs the correct Kubernetes version # rcnit.ValidateEnvironment.minikube_runs_the_correct_kubernetes_version()
16:26:44.503 [main] INFO rcnit.ValidateEnvironment - minikube uses 4 CPU cores
And minikube uses the correct core count # rcnit.ValidateEnvironment.minikube_uses_the_correct_core_count()
16:26:44.761 [main] INFO rcnit.ValidateEnvironment - minikube uses 8 GBs of memory
And minikube uses the correct amount of memory # rcnit.ValidateEnvironment.minikube_uses_the_correct_amount_of_memory()
16:26:46.720 [main] INFO rcnit.ValidateEnvironment - The 'metrics-server' addon is enabled
And the metrics-server is enabled # rcnit.ValidateEnvironment.the_metrics_server_is_enabled()
16:26:46.722 [main] INFO rcnit.ValidateEnvironment - PASSED! Minikube is configured correctly!
Then minikube is configured correctly # rcnit.ValidateEnvironment.minikube_is_configured_correctly()
Scenario: Encrypt data at rest # src/test/resources/rcnit/01_testEtcd.feature:3
16:26:46.760 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-etcd-testing' namespace...
16:26:50.951 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-etcd-testing' namespace
16:26:51.194 [main] INFO rcnit.TestEtcd - Created 'jimmy-hoffa-location' secret
When I create a secret # rcnit.TestEtcd.i_create_a_secret()
16:26:51.399 [main] INFO rcnit.TestEtcd - PASSED! The 'jimmy-hoffa-location' secret was successfully encrypted by etcd!
16:26:51.399 [main] INFO rcnit.TestEtcd - Encrypted secret stored in etcd: YnVyaWVkSW5OZXZlcmxhbmRSYW5jaA==
16:26:51.399 [main] INFO rcnit.TestEtcd - Decrypted secret stored in etcd: buriedInNeverlandRanch
16:26:51.463 [main] INFO rcnit.TestEtcd - Deleted 'jimmy-hoffa-location' secret
16:26:51.465 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-etcd-testing' namespace...
16:27:14.417 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-etcd-testing' namespace
Then The etcd key should be encrypted # rcnit.TestEtcd.the_etcd_key_should_be_encrypted()
Scenario: Test the liveness probe # src/test/resources/rcnit/02_testKubelet.feature:3
16:27:14.446 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kubelet-testing' namespace...
16:27:14.592 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kubelet-testing' namespace
16:27:14.755 [main] INFO rcnit.TestKubelet - Created 'rcnit-kubelet-testing-svc-account' service account
16:27:15.203 [main] INFO rcnit.TestKubelet - Created 'liveness-probe-pod' pod
16:27:15.208 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for 'liveness-probe-pod' pod in 'rcnit-kubelet-testing' namespace to be ready...
16:27:18.426 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'liveness-probe-pod' is ready
Given A pod with a liveness probe of 5 periodSeconds and a "sleep 10" container command # rcnit.TestKubelet.a_pod_with_a_liveness_probe_of_periodSeconds_and_a_container_command(java.lang.Integer,java.lang.String)
16:27:18.932 [main] INFO rcnit.TestKubelet - PASSED! There are no anomalous events for 'liveness-probe-pod' pod!
16:27:19.108 [main] INFO rcnit.util.ExecKubernetesCmd - Events for 'liveness-probe-pod' object:
LAST SEEN TYPE REASON OBJECT MESSAGE
4s Normal Scheduled pod/liveness-probe-pod Successfully assigned rcnit-kubelet-testing/liveness-probe-pod to minikube
3s Normal Pulling pod/liveness-probe-pod Pulling image "k8s.gcr.io/busybox"
2s Normal Pulled pod/liveness-probe-pod Successfully pulled image "k8s.gcr.io/busybox"
2s Normal Created pod/liveness-probe-pod Created container liveness-probe-pod-container
1s Normal Started pod/liveness-probe-pod Started container liveness-probe-pod-container
Then There are no anomalous pod events for the first ten seconds # rcnit.TestKubelet.there_are_no_anomalous_pod_events_for_the_first_ten_seconds()
16:27:19.109 [main] INFO rcnit.TestKubelet - Sleeping for [10] seconds...
16:27:29.298 [main] INFO rcnit.TestKubelet - PASSED! There are anomalous events for 'liveness-probe-pod' pod:
LAST SEEN TYPE REASON OBJECT MESSAGE
2s Warning Unhealthy pod/liveness-probe-pod Liveness probe failed: OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused "exec: \"cat /tmp/healthy\": stat cat /tmp/healthy: no such file or directory": unknown
16:27:29.298 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'liveness-probe-pod' pod...
16:27:29.427 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'liveness-probe-pod' pod
16:27:29.429 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kubelet-testing' namespace...
16:28:13.139 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kubelet-testing' namespace
But There are anomalous pod events after 10 seconds # rcnit.TestKubelet.there_are_anomalous_pod_events_after_seconds(java.lang.Integer)
Scenario: Test the LimitRange admission controller # src/test/resources/rcnit/03_testKubeApiServer.feature:8
16:28:13.244 [main] INFO rcnit.TestKubeApiServer - The default admission controllers for Kubernetes v1.18 are enabled
Given The default admission plugins are enabled # rcnit.TestKubeApiServer.the_default_admission_plugins_are_enabled()
16:28:13.349 [main] INFO rcnit.TestKubeApiServer - The storage class 'standard' is the default storage class
And A default storage class # rcnit.TestKubeApiServer.a_default_storage_class()
16:28:13.350 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-apiserver-testing' namespace...
16:28:13.487 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-apiserver-testing' namespace
16:28:13.735 [main] INFO rcnit.TestKubeApiServer - Created 'persistent-volume-test' PersistentVolume
And A 20 GB PersistentVolume # rcnit.TestKubeApiServer.a_GB_PersistentVolume(java.lang.Integer)
16:28:13.850 [main] INFO rcnit.TestKubeApiServer - Created 'limit-range-test' LimitRange
Given A max 5 GB LimitRange # rcnit.TestKubeApiServer.a_max_GB_LimitRange(java.lang.Integer)
16:28:13.876 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'tolerated-claim-test-1' PersistentVolumeClaim...
Then A tolerated 3 GB PersistentVolumeClaim will get provisioned # rcnit.TestKubeApiServer.a_tolerated_GB_PersistentVolumeClaim_will_get_provisioned(java.lang.Integer)
16:28:14.203 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'prohibited-claim-test-1' PersistentVolumeClaim...
16:28:14.331 [main] INFO rcnit.TestKubeApiServer - PASSED! The 'prohibited-claim-test-1' PersistentVolumeClaim was rejected!
16:28:14.331 [main] INFO rcnit.TestKubeApiServer - Failure executing: POST at: https://192.168.99.254:8443/api/v1/namespaces/rcnit-kube-apiserver-testing/persistentvolumeclaims. Message: Forbidden! User minikube doesn't have permission. persistentvolumeclaims "prohibited-claim-test-1" is forbidden: maximum storage usage per PersistentVolumeClaim is 5Gi, but request is 6Gi.
16:28:14.332 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'limit-range-test' LimitRange...
16:28:15.076 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'limit-range-test' LimitRange
16:28:15.077 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'persistent-volume-test' PersistentVolume...
16:28:15.205 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'persistent-volume-test' PersistentVolume
16:28:15.206 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-apiserver-testing' namespace...
16:28:25.960 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-apiserver-testing' namespace
But A prohibited 6 GB PersistentVolumeClaim will get rejected # rcnit.TestKubeApiServer.a_prohibited_GB_PersistentVolumeClaim_will_get_rejected(java.lang.Integer)
Scenario: Test the ResourceQuota admission controller # src/test/resources/rcnit/03_testKubeApiServer.feature:13
16:28:26.063 [main] INFO rcnit.TestKubeApiServer - The default admission controllers for Kubernetes v1.18 are enabled
Given The default admission plugins are enabled # rcnit.TestKubeApiServer.the_default_admission_plugins_are_enabled()
16:28:26.142 [main] INFO rcnit.TestKubeApiServer - The storage class 'standard' is the default storage class
And A default storage class # rcnit.TestKubeApiServer.a_default_storage_class()
16:28:26.142 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-apiserver-testing' namespace...
16:28:26.278 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-apiserver-testing' namespace
16:28:26.423 [main] INFO rcnit.TestKubeApiServer - Created 'persistent-volume-test' PersistentVolume
And A 20 GB PersistentVolume # rcnit.TestKubeApiServer.a_GB_PersistentVolume(java.lang.Integer)
16:28:26.598 [main] INFO rcnit.TestKubeApiServer - Created 'resource-quota-test' ResourceQuota
Given A max 5 GB storage ResourceQuota # rcnit.TestKubeApiServer.a_max_GB_storage_ResourceQuota(java.lang.Integer)
16:28:26.619 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'tolerated-claim-test-2' PersistentVolumeClaim...
Then A tolerated 3 GB PersistentVolumeClaim will get accepted # rcnit.TestKubeApiServer.a_tolerated_GB_PersistentVolumeClaim_will_get_accepted(java.lang.Integer)
16:28:26.875 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'prohibited-claim-test-2' PersistentVolumeClaim...
16:28:26.975 [main] INFO rcnit.TestKubeApiServer - PASSED! The 'prohibited-claim-test-2' PersistentVolumeClaim was rejected!
16:28:26.976 [main] INFO rcnit.TestKubeApiServer - Failure executing: POST at: https://192.168.99.254:8443/api/v1/namespaces/rcnit-kube-apiserver-testing/persistentvolumeclaims. Message: Forbidden! User minikube doesn't have permission. persistentvolumeclaims "prohibited-claim-test-2" is forbidden: exceeded quota: resource-quota-test, requested: requests.storage=6Gi, used: requests.storage=3Gi, limited: requests.storage=5Gi.
16:28:26.976 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'resource-quota-test' ResourceQuota...
16:28:27.112 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'resource-quota-test' ResourceQuota
16:28:27.113 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'persistent-volume-test' PersistentVolume...
16:28:27.236 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'persistent-volume-test' PersistentVolume
16:28:27.238 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-apiserver-testing' namespace...
16:28:36.981 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-apiserver-testing' namespace
But A prohibited 6 GB PersistentVolumeClaim will be denied # rcnit.TestKubeApiServer.a_prohibited_GB_PersistentVolumeClaim_will_be_denied(java.lang.Integer)
Scenario: Test the DefaultStorageClass admission controller # src/test/resources/rcnit/03_testKubeApiServer.feature:18
16:28:37.093 [main] INFO rcnit.TestKubeApiServer - The default admission controllers for Kubernetes v1.18 are enabled
Given The default admission plugins are enabled # rcnit.TestKubeApiServer.the_default_admission_plugins_are_enabled()
16:28:37.186 [main] INFO rcnit.TestKubeApiServer - The storage class 'standard' is the default storage class
And A default storage class # rcnit.TestKubeApiServer.a_default_storage_class()
16:28:37.187 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-apiserver-testing' namespace...
16:28:37.350 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-apiserver-testing' namespace
16:28:37.787 [main] INFO rcnit.TestKubeApiServer - Created 'persistent-volume-test' PersistentVolume
And A 20 GB PersistentVolume # rcnit.TestKubeApiServer.a_GB_PersistentVolume(java.lang.Integer)
16:28:37.789 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'pvc-with-no-class-specification-test-1' PersistentVolumeClaim...
16:28:38.926 [main] INFO rcnit.util.ExecKubernetesCmd - There are no events for the 'pvc-with-no-class-specification-test-1' object
16:28:38.935 [main] INFO rcnit.TestKubeApiServer - PASSED! The 'pvc-with-no-class-specification-test-1' PersistentVolumeClaim was assigned the 'standard' default storage class!
Name: pvc-with-no-class-specification-test-1
Namespace: rcnit-kube-apiserver-testing
StorageClass: standard
Status: Bound
Volume: persistent-volume-test
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 20Gi
Access Modes: RWO
VolumeMode: Filesystem
Mounted By: <none>
Events: <none>
16:28:38.935 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'pvc-with-no-class-specification-test-1' PersistentVolumeClaim...
16:28:39.598 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'pvc-with-no-class-specification-test-1' PersistentVolumeClaim
Then A PersistentVolumeClaim with no storage specification will get the default one # rcnit.TestKubeApiServer.a_PersistentVolumeClaim_with_no_storage_class_specification_will_get_the_default_one()
16:28:39.599 [main] INFO rcnit.TestKubeApiServer - Making sure there is no default storage class in the 'rcnit-kube-apiserver-testing' namespace...
16:28:39.768 [main] INFO rcnit.TestKubeApiServer - Storage class 'standard' is now non-default
When I do not have a default storage class # rcnit.TestKubeApiServer.i_do_not_have_a_default_storage_class()
16:28:39.774 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'pvc-with-no-class-specification-test-2' PersistentVolumeClaim...
16:28:40.241 [main] INFO rcnit.TestKubeApiServer - PASSED! The 'pvc-with-no-class-specification-test-2' PersistentVolumeClaim was provisioned with no storage class!
Name: pvc-with-no-class-specification-test-2
Namespace: rcnit-kube-apiserver-testing
StorageClass:
Status: Pending
Volume:
Labels: <none>
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Mounted By: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 1s persistentvolume-controller no persistent volumes available for this claim and no storage class is set
16:28:40.241 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'pvc-with-no-class-specification-test-2' PersistentVolumeClaim...
16:28:40.888 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'pvc-with-no-class-specification-test-2' PersistentVolumeClaim
Then The DefaultStorageClass admission controller does not do anything # rcnit.TestKubeApiServer.the_DefaultStorageClass_admission_controller_does_not_do_anything()
16:28:40.889 [main] INFO rcnit.TestKubeApiServer - Creating 'rcnit-storage-class' storage class in the 'rcnit-kube-apiserver-testing' namespace...
16:28:40.974 [main] INFO rcnit.TestKubeApiServer - Making sure that all storage classes in the 'rcnit-kube-apiserver-testing' namespace are default...
16:28:41.003 [main] INFO rcnit.TestKubeApiServer - Storage class 'rcnit-storage-class' is now default
16:28:41.032 [main] INFO rcnit.TestKubeApiServer - Storage class 'standard' is now default
When I have multiple default storage classes # rcnit.TestKubeApiServer.i_have_multiple_default_storage_classes()
16:28:41.048 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'pvc-with-no-class-specification-test-3' PersistentVolumeClaim...
16:28:41.172 [main] INFO rcnit.TestKubeApiServer - PASSED! The 'pvc-with-no-class-specification-test-3' PersistentVolumeClaim was rejected!
16:28:41.173 [main] INFO rcnit.TestKubeApiServer - Failure executing: POST at: https://192.168.99.254:8443/api/v1/namespaces/rcnit-kube-apiserver-testing/persistentvolumeclaims. Message: Forbidden! User minikube doesn't have permission. persistentvolumeclaims "pvc-with-no-class-specification-test-3" is forbidden: Internal error occurred: 2 default StorageClasses were found.
16:28:41.174 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-storage-class' StorageClass...
16:28:41.309 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-storage-class' StorageClass
16:28:41.311 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'persistent-volume-test' PersistentVolume...
16:28:41.425 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'persistent-volume-test' PersistentVolume
16:28:41.426 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-apiserver-testing' namespace...
16:28:47.135 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-apiserver-testing' namespace
Then The DefaultStorageClass admission controller will reject the PersistentVolumeClaim # rcnit.TestKubeApiServer.the_DefaultStorageClass_admission_controller_will_reject_the_PersistentVolumeClaim()
Scenario: Test CPU request scheduling # src/test/resources/rcnit/04_testKubeScheduler.feature:3
16:28:47.154 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-scheduler-testing' namespace...
16:28:47.285 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-scheduler-testing' namespace
16:28:47.383 [main] INFO rcnit.TestKubeScheduler - Created 'rcnit-kube-scheduler-testing-svc-account' service account
16:28:47.549 [main] INFO rcnit.TestKubeScheduler - Created 'cpu-request-test' pod
When I create a pod with a CPU request that is too big for my node # rcnit.TestKubeScheduler.i_create_a_pod_with_a_CPU_request_that_is_too_big_for_my_node()
16:28:49.202 [main] INFO rcnit.util.ExecKubernetesCmd - There are FailedScheduling events:
16:28:49.202 [main] INFO rcnit.util.ExecKubernetesCmd - LAST SEEN TYPE REASON OBJECT MESSAGE
2s Warning FailedScheduling pod/cpu-request-test 0/1 nodes are available: 1 Insufficient cpu.
16:28:49.202 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'cpu-request-test' pod is in 'Pending' phase!
16:28:49.203 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-scheduler-testing' namespace...
16:28:54.873 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-scheduler-testing' namespace
Then The scheduler should not schedule the pod due to insufficient CPU # rcnit.TestKubeScheduler.the_scheduler_should_not_schedule_the_pod_due_to_insufficient_cpu()
Scenario: Test memory request scheduling # src/test/resources/rcnit/04_testKubeScheduler.feature:7
16:28:54.882 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-scheduler-testing' namespace...
16:28:55.010 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-scheduler-testing' namespace
16:28:55.155 [main] INFO rcnit.TestKubeScheduler - Created 'rcnit-kube-scheduler-testing-svc-account' service account
16:28:55.266 [main] INFO rcnit.TestKubeScheduler - Created 'memory-request-test' pod
When I create a pod with a memory request that is too big for my node # rcnit.TestKubeScheduler.i_create_a_pod_with_a_memory_request_that_is_too_big_for_my_node()
16:28:57.059 [main] INFO rcnit.util.ExecKubernetesCmd - There are FailedScheduling events:
16:28:57.059 [main] INFO rcnit.util.ExecKubernetesCmd - LAST SEEN TYPE REASON OBJECT MESSAGE
1s Warning FailedScheduling pod/memory-request-test 0/1 nodes are available: 1 Insufficient memory.
16:28:57.060 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'memory-request-test' pod is in 'Pending' phase!
16:28:57.060 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-scheduler-testing' namespace...
16:29:00.188 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-scheduler-testing' namespace
Then The scheduler should not schedule the pod due to insufficient memory # rcnit.TestKubeScheduler.the_scheduler_should_not_schedule_the_pod_due_to_insufficient_memory()
Scenario: CPU-test a pod # src/test/resources/rcnit/04_testKubeScheduler.feature:11
16:29:00.201 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-scheduler-testing' namespace...
16:29:00.333 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-scheduler-testing' namespace
16:29:00.482 [main] INFO rcnit.TestKubeScheduler - Created 'rcnit-kube-scheduler-testing-svc-account' service account
16:29:03.623 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for 'cpu-test-1' pod in 'rcnit-kube-scheduler-testing' namespace to be ready...
16:29:07.873 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'cpu-test-1' is ready
Log of 'cpu-test-1' pod in namespace 'rcnit-kube-scheduler-testing'
-------------------------------------------------------------------
stress: info: [1] dispatching hogs: 1 cpu, 0 io, 0 vm, 0 hdd
stress: dbug: [1] using backoff sleep of 3000us
stress: dbug: [1] setting timeout to 20s
stress: dbug: [1] --> hogcpu worker 1 [7] forked
stress: dbug: [1] <-- worker 7 signalled normally
stress: info: [1] successful run completed in 20s
When I use 1000 milliCPU from a pod with 2000 milliCPU request and limit # rcnit.TestKubeScheduler.i_use_milliCPU_from_a_pod_with_milliCPU_request_and_limit(java.lang.Integer,java.lang.Integer)
16:29:32.080 [main] INFO rcnit.util.ExecKubernetesCmd - The scheduler is healthy and there are no FailedScheduling events!
16:29:32.081 [main] INFO rcnit.TestKubeScheduler - PASSED! The pod did not get terminated!
16:29:32.081 [main] INFO rcnit.TestKubeScheduler - Pod is in 'Succeeded' phase with exit code [0].
16:29:32.081 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-scheduler-testing' namespace...
16:29:37.750 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-scheduler-testing' namespace
Then The CPU-test pod should be fine # rcnit.TestKubeScheduler.the_CPU_test_pod_should_be_fine()
16:29:37.750 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-scheduler-testing' namespace...
16:29:37.867 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-scheduler-testing' namespace
16:29:37.952 [main] INFO rcnit.TestKubeScheduler - Created 'rcnit-kube-scheduler-testing-svc-account' service account
16:29:39.513 [main] INFO rcnit.TestKubeScheduler - * The 'metrics-server' addon is enabled
16:29:40.650 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for 'cpu-test-2' pod in 'rcnit-kube-scheduler-testing' namespace to be ready...
16:29:42.250 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'cpu-test-2' is ready
Log of 'cpu-test-2' pod in namespace 'rcnit-kube-scheduler-testing'
-------------------------------------------------------------------
stress: info: [1] dispatching hogs: 10 cpu, 0 io, 0 vm, 0 hdd
stress: dbug: [1] using backoff sleep of 30000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 10 [6] forked
stress: dbug: [1] using backoff sleep of 27000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 9 [7] forked
stress: dbug: [1] using backoff sleep of 24000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 8 [8] forked
stress: dbug: [1] using backoff sleep of 21000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 7 [9] forked
stress: dbug: [1] using backoff sleep of 18000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 6 [10] forked
stress: dbug: [1] using backoff sleep of 15000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 5 [11] forked
stress: dbug: [1] using backoff sleep of 12000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 4 [12] forked
stress: dbug: [1] using backoff sleep of 9000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 3 [13] forked
stress: dbug: [1] using backoff sleep of 6000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 2 [14] forked
stress: dbug: [1] using backoff sleep of 3000us
stress: dbug: [1] setting timeout to 120s
stress: dbug: [1] --> hogcpu worker 1 [15] forked
stress: dbug: [1] <-- worker 10 signalled normally
stress: dbug: [1] <-- worker 6 signalled normally
stress: dbug: [1] <-- worker 12 signalled normally
stress: dbug: [1] <-- worker 13 signalled normally
stress: dbug: [1] <-- worker 15 signalled normally
stress: dbug: [1] <-- worker 7 signalled normally
stress: dbug: [1] <-- worker 14 signalled normally
stress: dbug: [1] <-- worker 9 signalled normally
stress: dbug: [1] <-- worker 8 signalled normally
stress: dbug: [1] <-- worker 11 signalled normally
stress: info: [1] successful run completed in 120s
16:31:42.635 [main] INFO rcnit.TestKubeScheduler - ==== Pod metrics in 'rcnit-kube-scheduler-testing' namespace ====
16:31:42.641 [main] INFO rcnit.TestKubeScheduler - cpu-test-2 cpu-test-2-container CPU: 1 Memory: 1604Ki
16:31:42.644 [main] INFO rcnit.TestKubeScheduler - ==== Node metrics ====
16:31:42.658 [main] INFO rcnit.TestKubeScheduler - minikube CPU: 1395m Memory: 1245120Ki
16:31:42.658 [main] INFO rcnit.TestKubeScheduler - PASSED! The pod got throttled!
16:31:42.659 [main] INFO rcnit.TestKubeScheduler - The 'cpu-test-2' pod was forced to use less than [10000] milliCPUs (+/- 100 milliCPUs)
16:31:42.659 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-scheduler-testing' namespace...
16:31:48.396 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-scheduler-testing' namespace
When I use 10000 milliCPU from a pod with 1000 milliCPU compute allocation the CPU-test pod should be throttled # rcnit.TestKubeScheduler.i_use_CPU_from_a_pod_with_CPU_compute_allocation_the_CPU_test_should_be_throttled(java.lang.Integer,java.lang.Integer)
Scenario: Memory-test a pod # src/test/resources/rcnit/04_testKubeScheduler.feature:16
16:31:48.407 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-scheduler-testing' namespace...
16:31:48.528 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-scheduler-testing' namespace
16:31:48.699 [main] INFO rcnit.TestKubeScheduler - Created 'rcnit-kube-scheduler-testing-svc-account' service account
16:31:49.794 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for 'memory-test-1' pod in 'rcnit-kube-scheduler-testing' namespace to be ready...
16:31:51.478 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'memory-test-1' is ready
Log of 'memory-test-1' pod in namespace 'rcnit-kube-scheduler-testing'
-------------------------------------------------------------------
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: dbug: [1] using backoff sleep of 3000us
stress: dbug: [1] setting timeout to 20s
stress: dbug: [1] --> hogvm worker 1 [6] forked
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [6] freed 104857600 bytes
stress: dbug: [6] allocating 104857600 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: dbug: [6] sleeping for 1s with allocated memory
stress: dbug: [1] <-- worker 6 signalled normally
stress: info: [1] successful run completed in 20s
When I use 100 MB of memory from a pod with 200 MB memory request and limit # rcnit.TestKubeScheduler.i_use_MB_of_memory_from_a_pod_with_MB_memory_request_and_limit(int,int)
6:32:14.144 [main] INFO rcnit.util.ExecKubernetesCmd - The scheduler is healthy and there are no FailedScheduling events!
16:32:14.145 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'memory-test-1' pod did not get terminated!
16:32:14.145 [main] INFO rcnit.TestKubeScheduler - The 'memory-test-1' pod is in 'Succeeded' phase with exit code [0].
16:32:14.146 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'memory-test-1' pod...
16:32:14.262 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'memory-test-1' pod
Then The memory-test pod should be fine # rcnit.TestKubeScheduler.the_memory_test_pod_should_be_fine()
Log of 'memory-test-2' pod in namespace 'rcnit-kube-scheduler-testing'
-------------------------------------------------------------------
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: dbug: [1] using backoff sleep of 3000us
stress: dbug: [1] setting timeout to 20s
stress: dbug: [1] --> hogvm worker 1 [6] forked
stress: dbug: [6] allocating 314572800 bytes ...
stress: dbug: [6] touching bytes in strides of 4096 bytes ...
stress: FAIL: [1] (415) <-- worker 6 got signal 9
stress: WARN: [1] (417) now reaping child worker processes
stress: FAIL: [1] (421) kill error: No such process
stress: FAIL: [1] (451) failed run completed in 2s
When I use 300 MB of memory from a pod with 200 MB memory allocation # rcnit.TestKubeScheduler.i_use_MB_of_memory_from_a_pod_with_MB_memory_allocation(int,int)
16:32:29.100 [main] INFO rcnit.util.ExecKubernetesCmd - The scheduler is healthy and there are no FailedScheduling events!
16:32:29.101 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'memory-test-2' pod got terminated!
16:32:29.101 [main] INFO rcnit.TestKubeScheduler - The 'memory-test-2' pod is in 'Failed' phase with exit code [1].
16:32:29.102 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-scheduler-testing' namespace...
16:32:34.780 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-scheduler-testing' namespace
Then The memory-test pod should be terminated # rcnit.TestKubeScheduler.the_memory_test_pod_should_be_terminated()
Scenario: Test killing deployment replicas # src/test/resources/rcnit/04_testKubeScheduler.feature:22
16:32:34.798 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-scheduler-testing' namespace...
16:32:34.903 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-scheduler-testing' namespace
16:32:35.124 [main] INFO rcnit.TestKubeScheduler - Created 'hello-world' deployment
16:32:35.124 [main] INFO rcnit.TestKubeScheduler - Waiting for all 'hello-world' deployment replicas to become available...
Given A deployment with 20 replicas # rcnit.TestKubeScheduler.a_deployment_with_replicas(java.lang.Integer)
16:33:25.618 [main] INFO rcnit.TestKubeScheduler - Killed 'hello-world-7cbf98bc54-46n2l' replica #0
16:33:25.658 [main] INFO rcnit.TestKubeScheduler - Killed 'hello-world-7cbf98bc54-6nzcp' replica #1
16:33:25.713 [main] INFO rcnit.TestKubeScheduler - Killed 'hello-world-7cbf98bc54-6pcg4' replica #2
16:33:25.804 [main] INFO rcnit.TestKubeScheduler - Killed 'hello-world-7cbf98bc54-99648' replica #3
16:33:25.834 [main] INFO rcnit.TestKubeScheduler - Killed 'hello-world-7cbf98bc54-9dvq5' replica #4
16:33:25.835 [main] INFO rcnit.TestKubeScheduler - Killed [5] replicas
When I kill 5 replicas # rcnit.TestKubeScheduler.i_kill_replicas(java.lang.Integer)
16:33:25.837 [main] INFO rcnit.TestKubeScheduler - Waiting for all 'hello-world' deployment replicas to become available...
16:33:34.424 [main] INFO rcnit.TestKubeScheduler - The deployment 'hello-world' created [5] more replicas
Then The deployment should create 5 more replicas to equal 20 again # rcnit.TestKubeScheduler.the_deployment_should_create_more_replicas(java.lang.Integer,java.lang.Integer)
16:33:35.927 [main] INFO rcnit.util.ExecKubernetesCmd - The scheduler is healthy and there are no FailedScheduling events!
16:33:35.927 [main] INFO rcnit.TestKubeScheduler - PASSED!
16:33:35.928 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-scheduler-testing' namespace...
16:34:08.048 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-scheduler-testing' namespace
And The scheduler should be healthy # rcnit.TestKubeScheduler.the_scheduler_should_be_healthy()
Scenario: Test node affinity # src/test/resources/rcnit/04_testKubeScheduler.feature:28
16:34:08.059 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-kube-scheduler-testing' namespace...
16:34:08.188 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-kube-scheduler-testing' namespace
16:34:08.382 [main] INFO rcnit.TestKubeScheduler - Applied '{isThisTheCoolestNodeLabel=true}' node label
Given A node label # rcnit.TestKubeScheduler.a_node_label()
16:34:08.500 [main] INFO rcnit.TestKubeScheduler - Created 'rcnit-kube-scheduler-testing-svc-account' service account
16:34:10.134 [main] INFO rcnit.util.ExecKubernetesCmd - Events for 'node-affinity-test-1' object:
LAST SEEN TYPE REASON OBJECT MESSAGE
2s Normal Scheduled pod/node-affinity-test-1 Successfully assigned rcnit-kube-scheduler-testing/node-affinity-test-1 to minikube
1s Normal Pulled pod/node-affinity-test-1 Container image "k8s.gcr.io/echoserver:1.4" already present on machine
1s Normal Created pod/node-affinity-test-1 Created container node-affinity-test-1-container
16:34:10.135 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'node-affinity-test-1' pod was scheduled!
16:34:10.135 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'node-affinity-test-1' pod...
16:34:10.254 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'node-affinity-test-1' pod
Then A pod with a correct requiredDuringSchedulingIgnoredDuringExecution node affinity will get scheduled # rcnit.TestKubeScheduler.a_pod_with_a_correct_requiredDuringSchedulingIgnoredDuringExecution_node_affinity_will_get_scheduled()
16:34:11.724 [main] INFO rcnit.util.ExecKubernetesCmd - Events for 'node-affinity-test-2' object:
LAST SEEN TYPE REASON OBJECT MESSAGE
1s Warning FailedScheduling pod/node-affinity-test-2 0/1 nodes are available: 1 node(s) didn't match node selector.
16:34:11.724 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'node-affinity-test-2' pod was not scheduled!
16:34:11.724 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'node-affinity-test-2' pod...
16:34:11.858 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'node-affinity-test-2' pod
But A pod with a wrong requiredDuringSchedulingIgnoredDuringExecution node affinity will get rejected # rcnit.TestKubeScheduler.a_pod_with_a_wrong_requiredDuringSchedulingIgnoredDuringExecution_node_affinity_will_get_rejected()
16:34:13.719 [main] INFO rcnit.util.ExecKubernetesCmd - Events for 'node-affinity-test-3' object:
LAST SEEN TYPE REASON OBJECT MESSAGE
1s Normal Scheduled pod/node-affinity-test-3 Successfully assigned rcnit-kube-scheduler-testing/node-affinity-test-3 to minikube
0s Normal Pulled pod/node-affinity-test-3 Container image "k8s.gcr.io/echoserver:1.4" already present on machine
0s Normal Created pod/node-affinity-test-3 Created container node-affinity-test-3-container
16:34:13.720 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'node-affinity-test-3' pod was scheduled!
16:34:13.720 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'node-affinity-test-3' pod...
16:34:13.859 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'node-affinity-test-3' pod
And A pod with a correct preferredDuringSchedulingIgnoredDuringExecution node affinity will get scheduled # rcnit.TestKubeScheduler.a_pod_with_a_correct_preferredDuringSchedulingIgnoredDuringExecution_node_affinity_will_get_scheduled()
16:34:15.518 [main] INFO rcnit.util.ExecKubernetesCmd - Events for 'node-affinity-test-4' object:
LAST SEEN TYPE REASON OBJECT MESSAGE
1s Normal Scheduled pod/node-affinity-test-4 Successfully assigned rcnit-kube-scheduler-testing/node-affinity-test-4 to minikube
0s Normal Pulled pod/node-affinity-test-4 Container image "k8s.gcr.io/echoserver:1.4" already present on machine
0s Normal Created pod/node-affinity-test-4 Created container node-affinity-test-4-container
16:34:15.519 [main] INFO rcnit.TestKubeScheduler - PASSED! The 'node-affinity-test-4' pod was scheduled!
16:34:15.519 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'node-affinity-test-4' pod...
16:34:15.678 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'node-affinity-test-4' pod
16:34:15.814 [main] INFO rcnit.TestKubeScheduler - Removed '{isThisTheCoolestNodeLabel=true}' NodeRestriction label
16:34:15.814 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-kube-scheduler-testing' namespace...
16:35:33.521 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-kube-scheduler-testing' namespace
And A pod with a wrong preferredDuringSchedulingIgnoredDuringExecution node affinity will get scheduled # rcnit.TestKubeScheduler.a_pod_with_a_wrong_preferredDuringSchedulingIgnoredDuringExecution_node_affinity_will_get_scheduled()
Scenario: Run k8s-testsuite # src/test/resources/rcnit/05_useKubernetesTestSuite.feature:3
16:35:33.530 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-k8s-testsuite-testing' namespace...
16:35:33.610 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-k8s-testsuite-testing' namespace
16:35:33.611 [main] INFO rcnit.UseKubernetesTestSuite - Installing 'load-test' Helm chart...
16:35:34.847 [main] INFO rcnit.UseKubernetesTestSuite - Installed 'load-test' Helm chart:
NAME: load-test
LAST DEPLOYED: Fri May 29 16:35:34 2020
NAMESPACE: rcnit-k8s-testsuite-testing
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Load test deployed. You can watch the test with:
kubectl --namespace rcnit-k8s-testsuite-testing logs -f aggregator
16:35:34.959 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for all pods in 'rcnit-k8s-testsuite-testing' namespace to be ready...
16:35:47.382 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'aggregator' is ready
16:35:50.953 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'loadbots-fq9jj' is ready
16:35:54.016 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'webserver-x74mc' is ready
When I deploy the k8s-testsuite # rcnit.UseKubernetesTestSuite.i_deploy_the_k8s_testsuite()
Log of 'aggregator' pod in namespace 'rcnit-k8s-testsuite-testing'
-------------------------------------------------------------------
I0529 14:35:46.410237 1 aggregator.go:100] Creating Kubernetes client
I0529 14:35:46.410394 1 aggregator.go:126] Creating in cluster config
I0529 14:35:46.431308 1 aggregator.go:144] Running in Kubernetes Cluster - version v1.18 (v1.18.0) - platform linux/amd64
I0529 14:35:46.431537 1 aggregator.go:168] Running aggregator in namespace rcnit-k8s-testsuite-testing
I0529 14:35:46.433128 1 aggregator.go:104] Running preflight checks
I0529 14:35:46.433150 1 aggregator.go:172] Waiting for initial loadbot and webserver pods to be Running...
I0529 14:35:46.454920 1 aggregator.go:402] Pods are not running. Waiting 1s then checking again.
I0529 14:35:47.471327 1 aggregator.go:402] Pods are not running. Waiting 2s then checking again.
I0529 14:35:49.485468 1 aggregator.go:402] Pods are not running. Waiting 4s then checking again.
I0529 14:35:53.500432 1 aggregator.go:402] Pods are not running. Waiting 8s then checking again.
I0529 14:36:01.513289 1 aggregator.go:106] Finished preflight checks
I0529 14:36:01.513354 1 aggregator.go:108] Running scale test with max replicas 10
I0529 14:36:01.513372 1 aggregator.go:205] Load scenario 'Idle': 1 Loadbots - 1 Webservers
I0529 14:36:15.549112 1 aggregator.go:251] Summary of load scenario 'Idle': QPS: 112 Success: 100.00 % Latency: 2.524805ms (mean) 3.56956ms (99th)
I0529 14:36:16.550693 1 aggregator.go:205] Load scenario 'Under load': 1 Loadbots - 1 Webservers
I0529 14:36:30.586956 1 aggregator.go:251] Summary of load scenario 'Under load': QPS: 111 Success: 100.00 % Latency: 2.299346ms (mean) 3.256175ms (99th)
I0529 14:36:32.588174 1 aggregator.go:205] Load scenario 'Equal load': 1 Loadbots - 1 Webservers
I0529 14:36:46.627255 1 aggregator.go:251] Summary of load scenario 'Equal load': QPS: 111 Success: 100.00 % Latency: 2.426962ms (mean) 3.334956ms (99th)
I0529 14:36:47.628189 1 aggregator.go:412] Scaling loadbots to 10 replicas
I0529 14:36:48.662755 1 aggregator.go:205] Load scenario 'Over load': 10 Loadbots - 1 Webservers
I0529 14:36:48.719058 1 aggregator.go:402] Pods are not running. Waiting 1s then checking again.
I0529 14:36:49.763882 1 aggregator.go:402] Pods are not running. Waiting 2s then checking again.
I0529 14:36:51.796558 1 aggregator.go:402] Pods are not running. Waiting 4s then checking again.
I0529 14:37:09.865258 1 aggregator.go:251] Summary of load scenario 'Over load': QPS: 1113 Success: 100.00 % Latency: 2.566323ms (mean) 3.65245ms (99th)
I0529 14:37:11.868237 1 aggregator.go:412] Scaling webserver to 10 replicas
I0529 14:37:11.896889 1 aggregator.go:205] Load scenario 'High load': 10 Loadbots - 10 Webservers
I0529 14:37:12.019645 1 aggregator.go:372] Pods status output too short. Waiting 1s then checking again.
I0529 14:37:13.097363 1 aggregator.go:402] Pods are not running. Waiting 2s then checking again.
I0529 14:37:15.173557 1 aggregator.go:402] Pods are not running. Waiting 4s then checking again.
I0529 14:37:19.240543 1 aggregator.go:402] Pods are not running. Waiting 8s then checking again.
I0529 14:37:41.316565 1 aggregator.go:251] Summary of load scenario 'High load': QPS: 1112 Success: 100.00 % Latency: 2.520805ms (mean) 3.710719ms (99th)
I0529 14:37:41.316588 1 aggregator.go:324] Summary of load scenarios:
I0529 14:37:41.316670 1 aggregator.go:325] GENERATING SUMMARY OUTPUT
I0529 14:37:41.316747 1 aggregator.go:327] 0. Idle : QPS: 112 Success: 100.00 % Latency: 2.524805ms (mean) 3.56956ms (99th)
I0529 14:37:41.316774 1 aggregator.go:327] 1. Under load: QPS: 111 Success: 100.00 % Latency: 2.299346ms (mean) 3.256175ms (99th)
I0529 14:37:41.316791 1 aggregator.go:327] 2. Equal load: QPS: 111 Success: 100.00 % Latency: 2.426962ms (mean) 3.334956ms (99th)
I0529 14:37:41.316904 1 aggregator.go:327] 3. Over load : QPS: 1113 Success: 100.00 % Latency: 2.566323ms (mean) 3.65245ms (99th)
I0529 14:37:41.316930 1 aggregator.go:327] 4. High load : QPS: 1112 Success: 100.00 % Latency: 2.520805ms (mean) 3.710719ms (99th)
I0529 14:37:41.316961 1 aggregator.go:329] END SUMMARY DATA
I0529 14:37:41.316977 1 aggregator.go:412] Scaling loadbots to 0 replicas
I0529 14:37:41.354168 1 aggregator.go:412] Scaling webserver to 0 replicas
16:37:46.112 [main] INFO rcnit.util.ExecKubernetesCmd - The scheduler is healthy and there are no FailedScheduling events!
16:37:46.112 [main] INFO rcnit.UseKubernetesTestSuite - PASSED! The load-test finished successfully!
16:37:46.113 [main] INFO rcnit.util.ExecKubernetesCmd - Uninstalling 'load-test' Helm chat...
16:37:48.699 [main] INFO rcnit.util.ExecKubernetesCmd - Uninstalled 'load-test' Helm chart
16:37:48.699 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-k8s-testsuite-testing' namespace...
16:38:06.009 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-k8s-testsuite-testing' namespace
Then The k8s-testsuite should be successful # rcnit.UseKubernetesTestSuite.the_k8s_testsuite_should_be_successful()
Scenario: Deploy Pumba tests # src/test/resources/rcnit/06_usePumba.feature:3
16:38:06.017 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-pumba-testing' namespace...
16:38:06.130 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-pumba-testing' namespace
16:38:06.272 [main] INFO rcnit.UsePumba - Created 'rcnit-pumba-testing-svc-account' service account
16:38:06.358 [main] INFO rcnit.UsePumba - Deployed 'test-kill' pod
16:38:06.397 [main] INFO rcnit.UsePumba - Deployed 'test-pause' pod
16:38:06.426 [main] INFO rcnit.UsePumba - Deployed 'test-remove' pod
16:38:06.468 [main] INFO rcnit.UsePumba - Deployed 'test-delay' pod
16:38:06.589 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for all pods in 'rcnit-pumba-testing' namespace to be ready...
16:38:11.216 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-delay' is ready
16:38:11.236 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-kill' is ready
16:38:11.240 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-pause' is ready
16:38:12.258 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-remove' is ready
16:38:12.259 [main] INFO rcnit.UsePumba - Deploying pumba actions...
16:38:13.214 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for all pods in 'rcnit-pumba-testing' namespace to be ready...
16:38:22.428 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'pumba-delay' is ready
16:38:22.439 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'pumba-kill' is ready
16:38:23.471 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'pumba-pause' is ready
16:38:25.011 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'pumba-remove' is ready
16:38:25.019 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-delay' is ready
16:38:25.025 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-kill' is ready
16:38:25.076 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-pause' is ready
16:38:26.093 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'test-remove' is ready
16:38:26.095 [main] INFO rcnit.UsePumba - Letting the pumba actions run for [3] minutes...
When I run all Pumba actions for 3 minutes # rcnit.UsePumba.i_run_all_Pumba_actions_for_minutes(java.lang.Integer)
'pumba-delay' logs
------------------
time="2020-05-29T14:38:20Z" level=debug msg="adding network delay to all matching containers"
time="2020-05-29T14:38:20Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-delay]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:38:20Z" level=debug msg="listing containers"
time="2020-05-29T14:38:20Z" level=debug msg="found container" id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:38:20Z" level=debug msg="found container" id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:38:21Z" level=debug msg="found container" id=0bf925f523a433a86101ffb90910fe87b640c6a4961784a81ed270837ab89e46 name=/k8s_test-delay-1_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:38:21Z" level=debug msg="found container" id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:38:21Z" level=debug msg="adding network delay for container" container="{{0xc000132c60 [{bind /var/lib/kubelet/pods/b4bc2dec-02ee-422f-8f9b-ebf334ba9128/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b4bc2dec-02ee-422f-8f9b-ebf334ba9128/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/b4bc2dec-02ee-422f-8f9b-ebf334ba9128/containers/test-delay-2/cd89a175 /dev/termination-log true rprivate}] 0xc0004b2140 0xc00016a500} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000512000 1.9.1 Prashanth B <[email protected]> 0xc000512140 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:38:21Z" level=debug msg="running netem command" duration=20s id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e iface=eth0 ips="[]" name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="[delay 3000ms 30ms 20.00 distribution normal]" pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:38:21Z" level=info msg="running netem on container" command="[delay 3000ms 30ms 20.00 distribution normal]" dryrun=false duration=20s id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e ips="[]" name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:38:21Z" level=debug msg="start netem for container" dryrun=false id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e iface=eth0 name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="delay 3000ms 30ms 20.00 distribution normal" pull=true tcimage=gaiadocker/iproute2
time="2020-05-29T14:38:21Z" level=debug msg="adding netem qdisc" netem="qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal"
time="2020-05-29T14:38:21Z" level=debug msg="executing tc command in a separate container joining target container network namespace" args="[qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal]" container=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:38:21Z" level=debug msg="network mode" network="container:e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e"
time="2020-05-29T14:38:21Z" level=debug msg="pulling tc-image" image=gaiadocker/iproute2
time="2020-05-29T14:38:22Z" level=debug msg="&{Pulling from gaiadocker/iproute2 {0 0}}"
time="2020-05-29T14:38:22Z" level=debug msg="&{Pulling fs layer {0 0}}"
time="2020-05-29T14:38:22Z" level=debug msg="&{Pulling fs layer {0 0}}"
time="2020-05-29T14:38:22Z" level=debug msg="&{Pulling fs layer {0 0}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Verifying Checksum [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Download complete [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [> ] 23.65kB/2.32MB {23654 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [> ] 8.596kB/735.7kB {8596 735651}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [===> ] 147kB/2.32MB {146984 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [================================> ] 474.1kB/735.7kB {474133 735651}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [======> ] 293.9kB/2.32MB {293908 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Download complete [======> ] 293.9kB/2.32MB {293908 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [=============> ] 608.7kB/2.32MB {608691 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Downloading [=======================> ] 1.112MB/2.32MB {1112499 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Download complete [=======================> ] 1.112MB/2.32MB {1112499 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Extracting [> ] 32.77kB/2.32MB {32768 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Extracting [==> ] 98.3kB/2.32MB {98304 2320188}}"
time="2020-05-29T14:38:23Z" level=debug msg="&{Extracting [===========> ] 524.3kB/2.32MB {524288 2320188}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [=======================> ] 1.081MB/2.32MB {1081344 2320188}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [======================================> ] 1.769MB/2.32MB {1769472 2320188}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [================================================> ] 2.228MB/2.32MB {2228224 2320188}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [==================================================>] 2.32MB/2.32MB {2320188 2320188}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [==================================================>] 2.32MB/2.32MB {2320188 2320188}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Pull complete [==================================================>] 2.32MB/2.32MB {2320188 2320188}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [==> ] 32.77kB/735.7kB {32768 735651}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [======> ] 98.3kB/735.7kB {98304 735651}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [=====================================> ] 557.1kB/735.7kB {557056 735651}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [============================================> ] 655.4kB/735.7kB {655360 735651}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [==================================================>] 735.7kB/735.7kB {735651 735651}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [==================================================>] 735.7kB/735.7kB {735651 735651}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Pull complete [==================================================>] 735.7kB/735.7kB {735651 735651}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:24Z" level=debug msg="&{Extracting [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:25Z" level=debug msg="&{Pull complete [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:25Z" level=debug msg="&{Digest: sha256:d0cd5819fbeaf2e7501dcb223bb5bad4fda7c1e76ff9877baba6e559c70b6d1e [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:25Z" level=debug msg="&{Status: Downloaded newer image for gaiadocker/iproute2:latest [==================================================>] 124B/124B {124 124}}"
time="2020-05-29T14:38:25Z" level=debug msg="creating tc-container" image=gaiadocker/iproute2
time="2020-05-29T14:38:25Z" level=debug msg="tc container created, starting it" id=0184e0d90b2b222017057837d59d2406cb250bfc911a96d35c6527c5ac2d1753
time="2020-05-29T14:38:46Z" level=debug msg="stopping netem command on timout" id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e iface=eth0 ips="[]" name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 tc-image=gaiadocker/iproute2
time="2020-05-29T14:38:46Z" level=info msg="stopping netem on container" IPs="[]" dryrun=false id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e iface=eth0 name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:38:46Z" level=debug msg="stop netem for container" IPs="[]" dryrun=false id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e iface=eth0 name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tcimage=gaiadocker/iproute2
time="2020-05-29T14:38:46Z" level=debug msg="deleting netem qdisc" netem="qdisc del dev eth0 root netem"
time="2020-05-29T14:38:46Z" level=debug msg="executing tc command in a separate container joining target container network namespace" args="[qdisc del dev eth0 root netem]" container=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:38:46Z" level=debug msg="network mode" network="container:e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e"
time="2020-05-29T14:38:46Z" level=debug msg="pulling tc-image" image=gaiadocker/iproute2
time="2020-05-29T14:38:47Z" level=debug msg="&{Pulling from gaiadocker/iproute2 {0 0}}"
time="2020-05-29T14:38:47Z" level=debug msg="&{Digest: sha256:d0cd5819fbeaf2e7501dcb223bb5bad4fda7c1e76ff9877baba6e559c70b6d1e {0 0}}"
time="2020-05-29T14:38:47Z" level=debug msg="&{Status: Image is up to date for gaiadocker/iproute2:latest {0 0}}"
time="2020-05-29T14:38:47Z" level=debug msg="creating tc-container" image=gaiadocker/iproute2
time="2020-05-29T14:38:47Z" level=debug msg="tc container created, starting it" id=d8755470bedab9d4bc49e57fffe48a348f4e17e4501054ade390283cbefa673a
time="2020-05-29T14:39:20Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:39:20Z" level=debug msg="adding network delay to all matching containers"
time="2020-05-29T14:39:20Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-delay]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:39:20Z" level=debug msg="listing containers"
time="2020-05-29T14:39:20Z" level=debug msg="found container" id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:39:20Z" level=debug msg="found container" id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:39:21Z" level=debug msg="found container" id=0bf925f523a433a86101ffb90910fe87b640c6a4961784a81ed270837ab89e46 name=/k8s_test-delay-1_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:39:21Z" level=debug msg="found container" id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:39:21Z" level=debug msg="adding network delay for container" container="{{0xc00058e2c0 [] 0xc00001e140 0xc00059e000} {sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c [k8s.gcr.io/pause:3.2] [k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f] buildkit.dockerfile.v0 2020-02-14T10:51:50.60182885-08:00 0xc00001e280 0xc00001e3c0 amd64 linux 682696 682696 {map[MergedDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/merged UpperDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/diff WorkDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/work] overlay2} {layers [sha256:ba0dae6243cc9fa2890df40a625721fdbea5c94ca6da897acdd814d710149770] } {{0 0 <nil>}}}}"
time="2020-05-29T14:39:21Z" level=debug msg="running netem command" duration=20s id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 iface=eth0 ips="[]" name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="[delay 3000ms 30ms 20.00 distribution normal]" pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:39:21Z" level=info msg="running netem on container" command="[delay 3000ms 30ms 20.00 distribution normal]" dryrun=false duration=20s id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 ips="[]" name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:39:21Z" level=debug msg="start netem for container" dryrun=false id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 iface=eth0 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="delay 3000ms 30ms 20.00 distribution normal" pull=true tcimage=gaiadocker/iproute2
time="2020-05-29T14:39:21Z" level=debug msg="adding netem qdisc" netem="qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal"
time="2020-05-29T14:39:21Z" level=debug msg="executing tc command in a separate container joining target container network namespace" args="[qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal]" container=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:39:21Z" level=debug msg="network mode" network="container:9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2"
time="2020-05-29T14:39:21Z" level=debug msg="pulling tc-image" image=gaiadocker/iproute2
time="2020-05-29T14:39:22Z" level=debug msg="&{Pulling from gaiadocker/iproute2 {0 0}}"
time="2020-05-29T14:39:22Z" level=debug msg="&{Digest: sha256:d0cd5819fbeaf2e7501dcb223bb5bad4fda7c1e76ff9877baba6e559c70b6d1e {0 0}}"
time="2020-05-29T14:39:22Z" level=debug msg="&{Status: Image is up to date for gaiadocker/iproute2:latest {0 0}}"
time="2020-05-29T14:39:22Z" level=debug msg="creating tc-container" image=gaiadocker/iproute2
time="2020-05-29T14:39:22Z" level=debug msg="tc container created, starting it" id=fca5a8f5aaa18c4e435ce290c77d588740f20fdbedbe9b61f0aa56eea5db6a7a
time="2020-05-29T14:39:43Z" level=debug msg="stopping netem command on timout" id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 iface=eth0 ips="[]" name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 tc-image=gaiadocker/iproute2
time="2020-05-29T14:39:43Z" level=info msg="stopping netem on container" IPs="[]" dryrun=false id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 iface=eth0 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:39:43Z" level=debug msg="stop netem for container" IPs="[]" dryrun=false id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 iface=eth0 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tcimage=gaiadocker/iproute2
time="2020-05-29T14:39:43Z" level=debug msg="deleting netem qdisc" netem="qdisc del dev eth0 root netem"
time="2020-05-29T14:39:43Z" level=debug msg="executing tc command in a separate container joining target container network namespace" args="[qdisc del dev eth0 root netem]" container=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:39:43Z" level=debug msg="network mode" network="container:9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2"
time="2020-05-29T14:39:43Z" level=debug msg="pulling tc-image" image=gaiadocker/iproute2
time="2020-05-29T14:39:45Z" level=debug msg="&{Pulling from gaiadocker/iproute2 {0 0}}"
time="2020-05-29T14:39:45Z" level=debug msg="&{Digest: sha256:d0cd5819fbeaf2e7501dcb223bb5bad4fda7c1e76ff9877baba6e559c70b6d1e {0 0}}"
time="2020-05-29T14:39:45Z" level=debug msg="&{Status: Image is up to date for gaiadocker/iproute2:latest {0 0}}"
time="2020-05-29T14:39:45Z" level=debug msg="creating tc-container" image=gaiadocker/iproute2
time="2020-05-29T14:39:45Z" level=debug msg="tc container created, starting it" id=e66df5e88f4854b6376ef6012759beddc6cc9254255aec737e02e94cefff2ce2
time="2020-05-29T14:40:20Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:40:20Z" level=debug msg="adding network delay to all matching containers"
time="2020-05-29T14:40:20Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-delay]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:40:20Z" level=debug msg="listing containers"
time="2020-05-29T14:40:20Z" level=debug msg="found container" id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:40:20Z" level=debug msg="found container" id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:40:21Z" level=debug msg="found container" id=0bf925f523a433a86101ffb90910fe87b640c6a4961784a81ed270837ab89e46 name=/k8s_test-delay-1_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:40:21Z" level=debug msg="found container" id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:40:21Z" level=debug msg="adding network delay for container" container="{{0xc00058edc0 [{bind /var/lib/kubelet/pods/b4bc2dec-02ee-422f-8f9b-ebf334ba9128/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b4bc2dec-02ee-422f-8f9b-ebf334ba9128/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/b4bc2dec-02ee-422f-8f9b-ebf334ba9128/containers/test-delay-3/1bddf838 /dev/termination-log true rprivate}] 0xc00001e640 0xc00059e500} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0003e6500 1.9.1 Prashanth B <[email protected]> 0xc0003e6640 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:40:21Z" level=debug msg="running netem command" duration=20s id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 iface=eth0 ips="[]" name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="[delay 3000ms 30ms 20.00 distribution normal]" pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:40:21Z" level=info msg="running netem on container" command="[delay 3000ms 30ms 20.00 distribution normal]" dryrun=false duration=20s id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 ips="[]" name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:40:21Z" level=debug msg="start netem for container" dryrun=false id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 iface=eth0 name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="delay 3000ms 30ms 20.00 distribution normal" pull=true tcimage=gaiadocker/iproute2
time="2020-05-29T14:40:21Z" level=debug msg="adding netem qdisc" netem="qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal"
time="2020-05-29T14:40:21Z" level=debug msg="executing tc command in a separate container joining target container network namespace" args="[qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal]" container=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:40:21Z" level=debug msg="network mode" network="container:904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8"
time="2020-05-29T14:40:21Z" level=debug msg="pulling tc-image" image=gaiadocker/iproute2
time="2020-05-29T14:40:22Z" level=debug msg="&{Pulling from gaiadocker/iproute2 {0 0}}"
time="2020-05-29T14:40:22Z" level=debug msg="&{Digest: sha256:d0cd5819fbeaf2e7501dcb223bb5bad4fda7c1e76ff9877baba6e559c70b6d1e {0 0}}"
time="2020-05-29T14:40:22Z" level=debug msg="&{Status: Image is up to date for gaiadocker/iproute2:latest {0 0}}"
time="2020-05-29T14:40:22Z" level=debug msg="creating tc-container" image=gaiadocker/iproute2
time="2020-05-29T14:40:22Z" level=debug msg="tc container created, starting it" id=ce59147df14af10dc3ed921ffdee878157085dd99d64054e3e1ded1f348a157d
time="2020-05-29T14:40:43Z" level=debug msg="stopping netem command on timout" id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 iface=eth0 ips="[]" name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 tc-image=gaiadocker/iproute2
time="2020-05-29T14:40:43Z" level=info msg="stopping netem on container" IPs="[]" dryrun=false id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 iface=eth0 name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:40:43Z" level=debug msg="stop netem for container" IPs="[]" dryrun=false id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 iface=eth0 name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tcimage=gaiadocker/iproute2
time="2020-05-29T14:40:43Z" level=debug msg="deleting netem qdisc" netem="qdisc del dev eth0 root netem"
time="2020-05-29T14:40:43Z" level=debug msg="executing tc command in a separate container joining target container network namespace" args="[qdisc del dev eth0 root netem]" container=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:40:43Z" level=debug msg="network mode" network="container:904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8"
time="2020-05-29T14:40:43Z" level=debug msg="pulling tc-image" image=gaiadocker/iproute2
time="2020-05-29T14:40:45Z" level=debug msg="&{Pulling from gaiadocker/iproute2 {0 0}}"
time="2020-05-29T14:40:45Z" level=debug msg="&{Digest: sha256:d0cd5819fbeaf2e7501dcb223bb5bad4fda7c1e76ff9877baba6e559c70b6d1e {0 0}}"
time="2020-05-29T14:40:45Z" level=debug msg="&{Status: Image is up to date for gaiadocker/iproute2:latest {0 0}}"
time="2020-05-29T14:40:45Z" level=debug msg="creating tc-container" image=gaiadocker/iproute2
time="2020-05-29T14:40:45Z" level=debug msg="tc container created, starting it" id=f655902bfdf1fcb90be4fe46e2e43d1c7db8e7b0a699298ecbb089ca8c090545
time="2020-05-29T14:41:20Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:41:20Z" level=debug msg="adding network delay to all matching containers"
time="2020-05-29T14:41:20Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-delay]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:41:20Z" level=debug msg="listing containers"
time="2020-05-29T14:41:20Z" level=debug msg="found container" id=904ec78cbbe3869451c38a6bf3c9081b508a913ae4cc90cd3f481851d774a0b8 name=/k8s_test-delay-3_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:41:20Z" level=debug msg="found container" id=e5c514496a5b2aa6e4e52895303069fa313f9e54ae7eb16441f7ca012e71f32e name=/k8s_test-delay-2_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:41:20Z" level=debug msg="found container" id=0bf925f523a433a86101ffb90910fe87b640c6a4961784a81ed270837ab89e46 name=/k8s_test-delay-1_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:41:20Z" level=debug msg="found container" id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0
time="2020-05-29T14:41:20Z" level=debug msg="adding network delay for container" container="{{0xc000568c60 [] 0xc0000c1040 0xc0003c9a00} {sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c [k8s.gcr.io/pause:3.2] [k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f] buildkit.dockerfile.v0 2020-02-14T10:51:50.60182885-08:00 0xc00001f180 0xc0000c1180 amd64 linux 682696 682696 {map[MergedDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/merged UpperDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/diff WorkDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/work] overlay2} {layers [sha256:ba0dae6243cc9fa2890df40a625721fdbea5c94ca6da897acdd814d710149770] } {{0 0 <nil>}}}}"
time="2020-05-29T14:41:20Z" level=debug msg="running netem command" duration=20s id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 iface=eth0 ips="[]" name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="[delay 3000ms 30ms 20.00 distribution normal]" pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:41:20Z" level=info msg="running netem on container" command="[delay 3000ms 30ms 20.00 distribution normal]" dryrun=false duration=20s id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 ips="[]" name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:41:20Z" level=debug msg="start netem for container" dryrun=false id=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 iface=eth0 name=/k8s_POD_test-delay_rcnit-pumba-testing_b4bc2dec-02ee-422f-8f9b-ebf334ba9128_0 netem="delay 3000ms 30ms 20.00 distribution normal" pull=true tcimage=gaiadocker/iproute2
time="2020-05-29T14:41:20Z" level=debug msg="adding netem qdisc" netem="qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal"
time="2020-05-29T14:41:20Z" level=debug msg="executing tc command in a separate container joining target container network namespace" args="[qdisc add dev eth0 root netem delay 3000ms 30ms 20.00 distribution normal]" container=9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2 pull=true tc-image=gaiadocker/iproute2
time="2020-05-29T14:41:20Z" level=debug msg="network mode" network="container:9c144fd573a5e8dc6e461c2625cb4f723bb11ff9a77bf63b40582a29ee93aae2"
time="2020-05-29T14:41:20Z" level=debug msg="pulling tc-image" image=gaiadocker/iproute2
time="2020-05-29T14:41:22Z" level=debug msg="&{Pulling from gaiadocker/iproute2 {0 0}}"
time="2020-05-29T14:41:22Z" level=debug msg="&{Digest: sha256:d0cd5819fbeaf2e7501dcb223bb5bad4fda7c1e76ff9877baba6e559c70b6d1e {0 0}}"
time="2020-05-29T14:41:22Z" level=debug msg="&{Status: Image is up to date for gaiadocker/iproute2:latest {0 0}}"
time="2020-05-29T14:41:22Z" level=debug msg="creating tc-container" image=gaiadocker/iproute2
time="2020-05-29T14:41:22Z" level=debug msg="tc container created, starting it" id=d4679ff65a030b401caf4a25162065236f126548b1e195825d9fb8f57a884b79
'pumba-kill' logs
------------------
time="2020-05-29T14:38:19Z" level=debug msg="killing all matching containers"
time="2020-05-29T14:38:19Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-kill]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:38:19Z" level=debug msg="listing containers"
time="2020-05-29T14:38:19Z" level=debug msg="found container" id=5bf09ebcaf52464605e19060ef1ae22e4be9655ec92144d473fd0b5906968b25 name=/k8s_test-kill-3_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0
time="2020-05-29T14:38:19Z" level=debug msg="found container" id=a03b394d4b01bba6431e53860452518058283964b04cbbc86546f26331eff301 name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0
time="2020-05-29T14:38:19Z" level=debug msg="found container" id=2e54ec77619e41fdc2e4f068d1a9e85f0783eeab5113aa1cb928a74b7a1b520a name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0
time="2020-05-29T14:38:19Z" level=debug msg="found container" id=195d797df215ba969262937b1a224f82c5854eb5a3e27fd465cc306de6300fc4 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0
time="2020-05-29T14:38:19Z" level=debug msg="killing container" container="{{0xc00017f600 [{bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/containers/test-kill-2/1ab38052 /dev/termination-log true rprivate}] 0xc000363400 0xc000157600} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000426280 1.9.1 Prashanth B <[email protected]> 0xc0004263c0 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" signal=SIGKILL
time="2020-05-29T14:38:19Z" level=info msg="killing container" dryrun=false id=a03b394d4b01bba6431e53860452518058283964b04cbbc86546f26331eff301 name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0 signal=SIGKILL
time="2020-05-29T14:38:49Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:38:49Z" level=debug msg="killing all matching containers"
time="2020-05-29T14:38:49Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-kill]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:38:49Z" level=debug msg="listing containers"
time="2020-05-29T14:38:49Z" level=debug msg="found container" id=fddbecd84d581376897fcac9daed171d9880f8ceebd68956a14ce31d55f8c654 name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:38:49Z" level=debug msg="found container" id=5bf09ebcaf52464605e19060ef1ae22e4be9655ec92144d473fd0b5906968b25 name=/k8s_test-kill-3_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0
time="2020-05-29T14:38:49Z" level=debug msg="found container" id=2e54ec77619e41fdc2e4f068d1a9e85f0783eeab5113aa1cb928a74b7a1b520a name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0
time="2020-05-29T14:38:49Z" level=debug msg="found container" id=195d797df215ba969262937b1a224f82c5854eb5a3e27fd465cc306de6300fc4 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0
time="2020-05-29T14:38:49Z" level=debug msg="killing container" container="{{0xc00038d1e0 [] 0xc0003a8780 0xc0003c6600} {sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c [k8s.gcr.io/pause:3.2] [k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f] buildkit.dockerfile.v0 2020-02-14T10:51:50.60182885-08:00 0xc0003a88c0 0xc0003a8a00 amd64 linux 682696 682696 {map[MergedDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/merged UpperDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/diff WorkDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/work] overlay2} {layers [sha256:ba0dae6243cc9fa2890df40a625721fdbea5c94ca6da897acdd814d710149770] } {{0 0 <nil>}}}}" signal=SIGKILL
time="2020-05-29T14:38:49Z" level=info msg="killing container" dryrun=false id=195d797df215ba969262937b1a224f82c5854eb5a3e27fd465cc306de6300fc4 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_0 signal=SIGKILL
time="2020-05-29T14:39:19Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:39:19Z" level=debug msg="killing all matching containers"
time="2020-05-29T14:39:19Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-kill]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:39:19Z" level=debug msg="listing containers"
time="2020-05-29T14:39:19Z" level=debug msg="found container" id=0b7f3dcecdbab26b58b92d2f215982dbe2eac12607078a8ed0f29c5a414e5c74 name=/k8s_test-kill-3_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:39:19Z" level=debug msg="found container" id=a7edc10a89cdd092acfa0b6573c6d63445bc7ea9992051b2c4528f4411687d9b name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_2
time="2020-05-29T14:39:19Z" level=debug msg="found container" id=aaea47112d4329a2176176c626c559be84dd198a399fd6e73170f622dccb4643 name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:39:19Z" level=debug msg="found container" id=3bcb44e20c360d6fa96bd2027825861cdcb38ff763e6c96df84172337bcef927 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:39:19Z" level=debug msg="killing container" container="{{0xc0000aa2c0 [{bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/containers/test-kill-2/35f31424 /dev/termination-log true rprivate}] 0xc0000dc000 0xc0000e0000} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014c000 1.9.1 Prashanth B <[email protected]> 0xc00014c140 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" signal=SIGKILL
time="2020-05-29T14:39:19Z" level=info msg="killing container" dryrun=false id=a7edc10a89cdd092acfa0b6573c6d63445bc7ea9992051b2c4528f4411687d9b name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_2 signal=SIGKILL
time="2020-05-29T14:39:49Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:39:49Z" level=debug msg="killing all matching containers"
time="2020-05-29T14:39:49Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-kill]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:39:49Z" level=debug msg="listing containers"
time="2020-05-29T14:39:49Z" level=debug msg="found container" id=29d497b0582fea782d54d10c4cdcd46df5acfb693fd8c938314c9f9a110088e1 name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_3
time="2020-05-29T14:39:49Z" level=debug msg="found container" id=0b7f3dcecdbab26b58b92d2f215982dbe2eac12607078a8ed0f29c5a414e5c74 name=/k8s_test-kill-3_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:39:49Z" level=debug msg="found container" id=aaea47112d4329a2176176c626c559be84dd198a399fd6e73170f622dccb4643 name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:39:49Z" level=debug msg="found container" id=3bcb44e20c360d6fa96bd2027825861cdcb38ff763e6c96df84172337bcef927 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:39:49Z" level=debug msg="killing container" container="{{0xc000516c60 [{bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/containers/test-kill-2/1303808c /dev/termination-log true rprivate}] 0xc0003a8140 0xc0004d2600} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0003623c0 1.9.1 Prashanth B <[email protected]> 0xc000362500 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" signal=SIGKILL
time="2020-05-29T14:39:49Z" level=info msg="killing container" dryrun=false id=29d497b0582fea782d54d10c4cdcd46df5acfb693fd8c938314c9f9a110088e1 name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_3 signal=SIGKILL
time="2020-05-29T14:40:19Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:40:19Z" level=debug msg="killing all matching containers"
time="2020-05-29T14:40:19Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-kill]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:40:19Z" level=debug msg="listing containers"
time="2020-05-29T14:40:19Z" level=debug msg="found container" id=0b7f3dcecdbab26b58b92d2f215982dbe2eac12607078a8ed0f29c5a414e5c74 name=/k8s_test-kill-3_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:40:19Z" level=debug msg="found container" id=aaea47112d4329a2176176c626c559be84dd198a399fd6e73170f622dccb4643 name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:40:19Z" level=debug msg="found container" id=3bcb44e20c360d6fa96bd2027825861cdcb38ff763e6c96df84172337bcef927 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:40:19Z" level=debug msg="killing container" container="{{0xc0000ab760 [{bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/containers/test-kill-1/daf96acf /dev/termination-log true rprivate}] 0xc000362dc0 0xc0000e1800} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0003a8780 1.9.1 Prashanth B <[email protected]> 0xc0003a88c0 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" signal=SIGKILL
time="2020-05-29T14:40:19Z" level=info msg="killing container" dryrun=false id=aaea47112d4329a2176176c626c559be84dd198a399fd6e73170f622dccb4643 name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1 signal=SIGKILL
time="2020-05-29T14:40:49Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:40:49Z" level=debug msg="killing all matching containers"
time="2020-05-29T14:40:49Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-kill]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:40:49Z" level=debug msg="listing containers"
time="2020-05-29T14:40:49Z" level=debug msg="found container" id=827dcd069124c1acc106a811325fe0373adaa9ddaf1c486b4a5b8eb43d9c75ee name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_4
time="2020-05-29T14:40:49Z" level=debug msg="found container" id=ad98fd5440d753d521b69c6e94c4a6e6e8bd31dd1e5e3b26f15383eff2522895 name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_2
time="2020-05-29T14:40:49Z" level=debug msg="found container" id=0b7f3dcecdbab26b58b92d2f215982dbe2eac12607078a8ed0f29c5a414e5c74 name=/k8s_test-kill-3_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:40:49Z" level=debug msg="found container" id=3bcb44e20c360d6fa96bd2027825861cdcb38ff763e6c96df84172337bcef927 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:40:49Z" level=debug msg="killing container" container="{{0xc00021d340 [{bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/containers/test-kill-2/da7a5134 /dev/termination-log true rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/etc-hosts /etc/hosts true rprivate}] 0xc0003a8dc0 0xc0004d3700} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014c780 1.9.1 Prashanth B <[email protected]> 0xc00014c8c0 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" signal=SIGKILL
time="2020-05-29T14:40:49Z" level=info msg="killing container" dryrun=false id=827dcd069124c1acc106a811325fe0373adaa9ddaf1c486b4a5b8eb43d9c75ee name=/k8s_test-kill-2_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_4 signal=SIGKILL
time="2020-05-29T14:41:19Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:41:19Z" level=debug msg="killing all matching containers"
time="2020-05-29T14:41:19Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-kill]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:41:19Z" level=debug msg="listing containers"
time="2020-05-29T14:41:19Z" level=debug msg="found container" id=ad98fd5440d753d521b69c6e94c4a6e6e8bd31dd1e5e3b26f15383eff2522895 name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_2
time="2020-05-29T14:41:19Z" level=debug msg="found container" id=0b7f3dcecdbab26b58b92d2f215982dbe2eac12607078a8ed0f29c5a414e5c74 name=/k8s_test-kill-3_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:41:19Z" level=debug msg="found container" id=3bcb44e20c360d6fa96bd2027825861cdcb38ff763e6c96df84172337bcef927 name=/k8s_POD_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_1
time="2020-05-29T14:41:19Z" level=debug msg="killing container" container="{{0xc0001c7b80 [{bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe/containers/test-kill-1/859acf69 /dev/termination-log true rprivate}] 0xc00014d180 0xc0003c7600} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000363180 1.9.1 Prashanth B <[email protected]> 0xc0003632c0 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" signal=SIGKILL
time="2020-05-29T14:41:19Z" level=info msg="killing container" dryrun=false id=ad98fd5440d753d521b69c6e94c4a6e6e8bd31dd1e5e3b26f15383eff2522895 name=/k8s_test-kill-1_test-kill_rcnit-pumba-testing_b0cef41d-6f1f-4af1-b1bd-43c4d31cdbbe_2 signal=SIGKILL
'pumba-pause' logs
------------------
time="2020-05-29T14:38:22Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:38:22Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:38:22Z" level=debug msg="listing containers"
time="2020-05-29T14:38:22Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:22Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:22Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:22Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:22Z" level=debug msg="pausing container for duration" container="{{0xc0001982c0 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate}] 0xc0001fa000 0xc0001fc000} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000343b80 1.9.1 Prashanth B <[email protected]> 0xc000343cc0 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:38:22Z" level=info msg="pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:32Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:38:32Z" level=debug msg="unpause container" container="{{0xc0001982c0 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate}] 0xc0001fa000 0xc0001fc000} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000343b80 1.9.1 Prashanth B <[email protected]> 0xc000343cc0 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:38:32Z" level=info msg="stop pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:42Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:38:42Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:38:42Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:38:42Z" level=debug msg="listing containers"
time="2020-05-29T14:38:42Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:42Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:42Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:42Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:42Z" level=debug msg="pausing container for duration" container="{{0xc000522840 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate}] 0xc000510280 0xc000155a00} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0001fab40 1.9.1 Prashanth B <[email protected]> 0xc0001fac80 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:38:42Z" level=info msg="pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:38:52Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:38:52Z" level=debug msg="unpause container" container="{{0xc000522840 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate}] 0xc000510280 0xc000155a00} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0001fab40 1.9.1 Prashanth B <[email protected]> 0xc0001fac80 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:38:52Z" level=info msg="stop pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:02Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:39:02Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:39:02Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:39:02Z" level=debug msg="listing containers"
time="2020-05-29T14:39:02Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:02Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:02Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:02Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:02Z" level=debug msg="pausing container for duration" container="{{0xc0004e2580 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-2/86e122b4 /dev/termination-log true rprivate}] 0xc0001fb180 0xc0004e8000} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0001fb2c0 1.9.1 Prashanth B <[email protected]> 0xc0001fb400 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:39:02Z" level=info msg="pausing container" dryrun=false id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:12Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:39:12Z" level=debug msg="unpause container" container="{{0xc0004e2580 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-2/86e122b4 /dev/termination-log true rprivate}] 0xc0001fb180 0xc0004e8000} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0001fb2c0 1.9.1 Prashanth B <[email protected]> 0xc0001fb400 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:39:12Z" level=info msg="stop pausing container" dryrun=false id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:22Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:39:22Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:39:22Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:39:22Z" level=debug msg="listing containers"
time="2020-05-29T14:39:22Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:22Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:22Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:22Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:22Z" level=debug msg="pausing container for duration" container="{{0xc00017cf20 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate}] 0xc00040c140 0xc000438700} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014a500 1.9.1 Prashanth B <[email protected]> 0xc00014a640 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:39:22Z" level=info msg="pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:32Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:39:32Z" level=debug msg="unpause container" container="{{0xc00017cf20 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate}] 0xc00040c140 0xc000438700} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014a500 1.9.1 Prashanth B <[email protected]> 0xc00014a640 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:39:32Z" level=info msg="stop pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:42Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:39:42Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:39:42Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:39:42Z" level=debug msg="listing containers"
time="2020-05-29T14:39:42Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:42Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:42Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:42Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:42Z" level=debug msg="pausing container for duration" container="{{0xc0001ea9a0 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate}] 0xc00040c780 0xc000439600} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014af00 1.9.1 Prashanth B <[email protected]> 0xc00014b040 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:39:42Z" level=info msg="pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:39:52Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:39:52Z" level=debug msg="unpause container" container="{{0xc0001ea9a0 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate}] 0xc00040c780 0xc000439600} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014af00 1.9.1 Prashanth B <[email protected]> 0xc00014b040 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:39:52Z" level=info msg="stop pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:02Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:40:02Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:40:02Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:40:02Z" level=debug msg="listing containers"
time="2020-05-29T14:40:02Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:02Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:02Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:02Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:02Z" level=debug msg="pausing container for duration" container="{{0xc00027ba20 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate}] 0xc00040d400 0xc000272d00} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014b400 1.9.1 Prashanth B <[email protected]> 0xc00014b540 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:40:02Z" level=info msg="pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:12Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:40:12Z" level=debug msg="unpause container" container="{{0xc00027ba20 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate}] 0xc00040d400 0xc000272d00} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00014b400 1.9.1 Prashanth B <[email protected]> 0xc00014b540 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:40:12Z" level=info msg="stop pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:22Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:40:22Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:40:22Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:40:22Z" level=debug msg="listing containers"
time="2020-05-29T14:40:22Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:22Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:22Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:22Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:22Z" level=debug msg="pausing container for duration" container="{{0xc0002b2dc0 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate}] 0xc00040d7c0 0xc000273600} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000560640 1.9.1 Prashanth B <[email protected]> 0xc000560780 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:40:22Z" level=info msg="pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:32Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:40:32Z" level=debug msg="unpause container" container="{{0xc0002b2dc0 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate}] 0xc00040d7c0 0xc000273600} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000560640 1.9.1 Prashanth B <[email protected]> 0xc000560780 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:40:32Z" level=info msg="stop pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:42Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:40:42Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:40:42Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:40:42Z" level=debug msg="listing containers"
time="2020-05-29T14:40:42Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:42Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:42Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:42Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:42Z" level=debug msg="pausing container for duration" container="{{0xc000332000 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate}] 0xc00014be00 0xc000324300} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0005612c0 1.9.1 Prashanth B <[email protected]> 0xc000561400 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:40:42Z" level=info msg="pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:40:52Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:40:52Z" level=debug msg="unpause container" container="{{0xc000332000 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate}] 0xc00014be00 0xc000324300} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0005612c0 1.9.1 Prashanth B <[email protected]> 0xc000561400 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:40:52Z" level=info msg="stop pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:02Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:41:02Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:41:02Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:41:02Z" level=debug msg="listing containers"
time="2020-05-29T14:41:02Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:02Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:02Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:02Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:02Z" level=debug msg="pausing container for duration" container="{{0xc000472b00 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate}] 0xc000561a40 0xc00046ab00} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0001fa140 1.9.1 Prashanth B <[email protected]> 0xc0001fa280 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:41:02Z" level=info msg="pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:12Z" level=debug msg="unpause containers after duration" duration=10s
time="2020-05-29T14:41:12Z" level=debug msg="unpause container" container="{{0xc000472b00 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-1/74bef7ec /dev/termination-log true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate}] 0xc000561a40 0xc00046ab00} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc0001fa140 1.9.1 Prashanth B <[email protected]> 0xc0001fa280 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}"
time="2020-05-29T14:41:12Z" level=info msg="stop pausing container" dryrun=false id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:22Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:41:22Z" level=debug msg="pausing all matching containers"
time="2020-05-29T14:41:22Z" level=debug msg="listing matching containers" duration=10s labels="[io.kubernetes.pod.name=test-pause]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:41:22Z" level=debug msg="listing containers"
time="2020-05-29T14:41:22Z" level=debug msg="found container" id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:22Z" level=debug msg="found container" id=5f5f1bcefab4d5b8a7e9ef9853bd6fad9a08677b5d6188d73e1a4e5c0196b084 name=/k8s_test-pause-2_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:22Z" level=debug msg="found container" id=cbabd61b624f34e4c27a1a8031038c1713602423aab4f343a92f207ea7fa378f name=/k8s_test-pause-1_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:22Z" level=debug msg="found container" id=42c053e3f914782b291b53198c05e51948386fbdd1529a244d64aa56f6f46359 name=/k8s_POD_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
time="2020-05-29T14:41:22Z" level=debug msg="pausing container for duration" container="{{0xc0002d5760 [{bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/a051bef1-6031-46cd-9ac6-19b2ad819f30/containers/test-pause-3/c202efbd /dev/termination-log true rprivate}] 0xc00014a3c0 0xc00046ac00} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc00040c000 1.9.1 Prashanth B <[email protected]> 0xc00040c140 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" duration=10s
time="2020-05-29T14:41:22Z" level=info msg="pausing container" dryrun=false id=9842ab2722832788fdd8ffe7a5628dbc03234cb4d542dc95632066e8d69d5f4b name=/k8s_test-pause-3_test-pause_rcnit-pumba-testing_a051bef1-6031-46cd-9ac6-19b2ad819f30_0
'pumba-remove' logs
------------------
time="2020-05-29T14:38:23Z" level=debug msg="removing all matching containers"
time="2020-05-29T14:38:23Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-remove]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:38:23Z" level=debug msg="listing containers"
time="2020-05-29T14:38:24Z" level=debug msg="found container" id=058625470cd1f26eb61f2aab7d3c8654658cc72e4bef7018bf70b0e9898d9b39 name=/k8s_test-remove-3_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:38:24Z" level=debug msg="found container" id=2c518a5c8c9904544787f6b9e5180f4b55d9f68abc38984f136ce14399c5fae7 name=/k8s_test-remove-2_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:38:24Z" level=debug msg="found container" id=9a7990d520bb14307148349a2a6fd2d0b08d7ce1ab1798857553d7ffeb335422 name=/k8s_test-remove-1_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:38:24Z" level=debug msg="found container" id=c53e9cdfb8038d94bc38ab8214abc45165080d7e01d87f3f114acc923a3e7ae0 name=/k8s_POD_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:38:24Z" level=debug msg="removing container" container="{{0xc00048a2c0 [{bind /var/lib/kubelet/pods/216d98be-2b12-4523-b3de-ce2e2309a4d6/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/216d98be-2b12-4523-b3de-ce2e2309a4d6/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/216d98be-2b12-4523-b3de-ce2e2309a4d6/containers/test-remove-2/ba6035df /dev/termination-log true rprivate}] 0xc0004b8000 0xc0004bc000} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000077680 1.9.1 Prashanth B <[email protected]> 0xc0000777c0 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" force=true links=false volumes=true
time="2020-05-29T14:38:24Z" level=info msg="removing container" dryrun=false force=true id=2c518a5c8c9904544787f6b9e5180f4b55d9f68abc38984f136ce14399c5fae7 links=false name=/k8s_test-remove-2_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0 volumes=true
time="2020-05-29T14:39:23Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:39:23Z" level=debug msg="removing all matching containers"
time="2020-05-29T14:39:23Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-remove]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:39:23Z" level=debug msg="listing containers"
time="2020-05-29T14:39:24Z" level=debug msg="found container" id=59e7ce8f7f9a029643596ced537da6a870d8bbc84169812e62def7b336d48bc5 name=/k8s_test-remove-2_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:39:24Z" level=debug msg="found container" id=058625470cd1f26eb61f2aab7d3c8654658cc72e4bef7018bf70b0e9898d9b39 name=/k8s_test-remove-3_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:39:24Z" level=debug msg="found container" id=9a7990d520bb14307148349a2a6fd2d0b08d7ce1ab1798857553d7ffeb335422 name=/k8s_test-remove-1_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:39:24Z" level=debug msg="found container" id=c53e9cdfb8038d94bc38ab8214abc45165080d7e01d87f3f114acc923a3e7ae0 name=/k8s_POD_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:39:24Z" level=debug msg="removing container" container="{{0xc000500580 [] 0xc000077e00 0xc00039ee00} {sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c [k8s.gcr.io/pause:3.2] [k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f] buildkit.dockerfile.v0 2020-02-14T10:51:50.60182885-08:00 0xc000440c80 0xc000440dc0 amd64 linux 682696 682696 {map[MergedDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/merged UpperDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/diff WorkDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/work] overlay2} {layers [sha256:ba0dae6243cc9fa2890df40a625721fdbea5c94ca6da897acdd814d710149770] } {{0 0 <nil>}}}}" force=true links=false volumes=true
time="2020-05-29T14:39:24Z" level=info msg="removing container" dryrun=false force=true id=c53e9cdfb8038d94bc38ab8214abc45165080d7e01d87f3f114acc923a3e7ae0 links=false name=/k8s_POD_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0 volumes=true
time="2020-05-29T14:40:23Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:40:23Z" level=debug msg="removing all matching containers"
time="2020-05-29T14:40:23Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-remove]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:40:23Z" level=debug msg="listing containers"
time="2020-05-29T14:40:24Z" level=debug msg="found container" id=05ec76db01f55700c9d12e5f464c27102a5190237e73c92f57984be56501d1e9 name=/k8s_test-remove-3_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_1
time="2020-05-29T14:40:24Z" level=debug msg="found container" id=e8e5b2a3b4035dc87fd1e9a269735ea2293aac930a4efcd34357abcf5f194da2 name=/k8s_test-remove-2_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_1
time="2020-05-29T14:40:24Z" level=debug msg="found container" id=90e40c5280d02563c1fd80908aa33a9c9c53e5bac34d20e93119d9e39f404c47 name=/k8s_test-remove-1_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_1
time="2020-05-29T14:40:24Z" level=debug msg="found container" id=20891d4d4bbbadcc1e7b86f010fed9cd169977f781db55105701bfd8ef093851 name=/k8s_POD_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:40:24Z" level=debug msg="removing container" container="{{0xc000500dc0 [{bind /var/lib/kubelet/pods/216d98be-2b12-4523-b3de-ce2e2309a4d6/volumes/kubernetes.io~secret/rcnit-pumba-testing-svc-account-token-vxtdk /var/run/secrets/kubernetes.io/serviceaccount ro false rprivate} {bind /var/lib/kubelet/pods/216d98be-2b12-4523-b3de-ce2e2309a4d6/etc-hosts /etc/hosts true rprivate} {bind /var/lib/kubelet/pods/216d98be-2b12-4523-b3de-ce2e2309a4d6/containers/test-remove-3/e662fa2e /dev/termination-log true rprivate}] 0xc000516000 0xc00039f300} {sha256:a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 [k8s.gcr.io/echoserver:1.4] [k8s.gcr.io/echoserver@sha256:5d99aa1120524c801bc8c1a7077e8f5ec122ba16b6dda1a5d3826057f67b9bcb] 2016-05-29T05:03:36.393304451Z 102588599d91103dd87be28cc3470c0d5813ac8b19b31437c602290a6bb7fb17 0xc000440f00 1.9.1 Prashanth B <[email protected]> 0xc000441040 amd64 linux 140393469 140393469 {map[LowerDir:/var/lib/docker/overlay2/7003c06e906197be067c9f082615fbbe7c4cf8d6b39ded773172f1a3c0ba96ec/diff:/var/lib/docker/overlay2/f461ac3cfcc0f039a973e21e416912dd7ec19ed9066b03ce59dc3fade57dd9db/diff:/var/lib/docker/overlay2/5a3ad980d40c053c5deeed78911d19794dcf56ce17c569651dcd9861f4e21fd1/diff:/var/lib/docker/overlay2/6d073af3063b7347ae99a86b0d8b9aad113b3bd60ef42eca3c35af955c8aa912/diff:/var/lib/docker/overlay2/276af1b65e8990a88a21a57dd95d68711b9b2026fa6a207f2f338767458ffbb6/diff:/var/lib/docker/overlay2/45324d58f7157724f3e4e8069c7484a5dade6eddcd131897cd24e57d4a97908a/diff:/var/lib/docker/overlay2/c2acb96e6f32836c04d4c9418924de368e93faef3e5151e8b9a17148b3911cca/diff:/var/lib/docker/overlay2/3a962169a93a06371fb1d7ca3343d0ce00b5c5c2b3c0b1073ff4c92959d9de3b/diff:/var/lib/docker/overlay2/a7b6307f787ff526459754d4dc0bac4b8ee081fd7666fbd65b819e80bebff9c7/diff:/var/lib/docker/overlay2/9552e75485eebef1a93bfded4db7f3ea274eaef3203346c1cfc597b17afeab1a/diff MergedDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/merged UpperDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/diff WorkDir:/var/lib/docker/overlay2/043a65c4ff07a6d91642729bc31622c66c004e349fad9b11f0ba2ef3b4ebdf2f/work] overlay2} {layers [sha256:6cc9890d69b6ce7756e6e0738bdcd4ce6486dd91587dbc50e5fa0a2382dff091 sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:e105cd2171638d95189a6250d82ef4497b509505d1495b298169a94986b6b807 sha256:9f9b8efa9a34969c7b4bc157aeddb4bb1e20f634b63fc8f7931c78a897072f23 sha256:4cc84b7b3abae12972fafa47ec56803838042584860836cfd9d1afd41082833f sha256:e2615e4925e2d7af41e75a0f40d5b301c2b4fe6a1533bf9ff7d3b8e90389cc9e sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef sha256:1787713d6d5d0881bfa9c17d6e49b07a20877bc09c2875a97546af6156c56741 sha256:67639a8a79165f594630b1aa398e714d603be6c3bfca35ece5f1c49f76118805] } {{0 0 <nil>}}}}" force=true links=false volumes=true
time="2020-05-29T14:40:24Z" level=info msg="removing container" dryrun=false force=true id=05ec76db01f55700c9d12e5f464c27102a5190237e73c92f57984be56501d1e9 links=false name=/k8s_test-remove-3_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_1 volumes=true
time="2020-05-29T14:41:23Z" level=debug msg="next chaos execution (tick) ..."
time="2020-05-29T14:41:23Z" level=debug msg="removing all matching containers"
time="2020-05-29T14:41:23Z" level=debug msg="listing matching containers" labels="[io.kubernetes.pod.name=test-remove]" limit=0 names="[]" pattern= random=true
time="2020-05-29T14:41:23Z" level=debug msg="listing containers"
time="2020-05-29T14:41:24Z" level=debug msg="found container" id=7b830a2b8c812009685f2ffb68d5e4cd62b42c5736479d88bb55ca742023f76c name=/k8s_test-remove-3_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_1
time="2020-05-29T14:41:24Z" level=debug msg="found container" id=e8e5b2a3b4035dc87fd1e9a269735ea2293aac930a4efcd34357abcf5f194da2 name=/k8s_test-remove-2_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_1
time="2020-05-29T14:41:24Z" level=debug msg="found container" id=90e40c5280d02563c1fd80908aa33a9c9c53e5bac34d20e93119d9e39f404c47 name=/k8s_test-remove-1_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_1
time="2020-05-29T14:41:24Z" level=debug msg="found container" id=20891d4d4bbbadcc1e7b86f010fed9cd169977f781db55105701bfd8ef093851 name=/k8s_POD_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0
time="2020-05-29T14:41:24Z" level=debug msg="removing container" container="{{0xc0004ee9a0 [] 0xc0000b03c0 0xc00039e200} {sha256:80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c [k8s.gcr.io/pause:3.2] [k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f] buildkit.dockerfile.v0 2020-02-14T10:51:50.60182885-08:00 0xc00015c640 0xc00015c780 amd64 linux 682696 682696 {map[MergedDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/merged UpperDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/diff WorkDir:/var/lib/docker/overlay2/5822a6eae07908c623dbb63d8fd6a55e70a6ac5ff68d33c876dde2f126b444cf/work] overlay2} {layers [sha256:ba0dae6243cc9fa2890df40a625721fdbea5c94ca6da897acdd814d710149770] } {{0 0 <nil>}}}}" force=true links=false volumes=true
time="2020-05-29T14:41:24Z" level=info msg="removing container" dryrun=false force=true id=20891d4d4bbbadcc1e7b86f010fed9cd169977f781db55105701bfd8ef093851 links=false name=/k8s_POD_test-remove_rcnit-pumba-testing_216d98be-2b12-4523-b3de-ce2e2309a4d6_0 volumes=true
16:41:27.377 [main] INFO rcnit.util.ExecKubernetesCmd - The scheduler is healthy and there are no FailedScheduling events!
16:41:27.377 [main] INFO rcnit.util.ExecKubernetesCmd - Deleting 'rcnit-pumba-testing' namespace...
16:41:44.259 [main] INFO rcnit.util.ExecKubernetesCmd - Deleted 'rcnit-pumba-testing' namespace
Then The actions should be successful # rcnit.UsePumba.the_actions_should_be_successful()
Scenario Outline: Inject chaos actions # src/test/resources/rcnit/07_useChaosMesh.feature:12
16:41:44.266 [main] INFO rcnit.util.ExecKubernetesCmd - Creating 'rcnit-chaos-mesh-testing' namespace...
16:41:44.406 [main] INFO rcnit.util.ExecKubernetesCmd - Created 'rcnit-chaos-mesh-testing' namespace
16:41:45.771 [main] INFO rcnit.UseChaosMesh - Created Chaos Mesh CRDs:
customresourcedefinition.apiextensions.k8s.io/iochaos.pingcap.com created
customresourcedefinition.apiextensions.k8s.io/kernelchaos.pingcap.com created
customresourcedefinition.apiextensions.k8s.io/networkchaos.pingcap.com created
customresourcedefinition.apiextensions.k8s.io/podchaos.pingcap.com created
customresourcedefinition.apiextensions.k8s.io/stresschaos.pingcap.com created
customresourcedefinition.apiextensions.k8s.io/timechaos.pingcap.com created
16:41:45.771 [main] INFO rcnit.UseChaosMesh - Installing 'chaos-mesh' Helm templates...
16:41:47.861 [main] INFO rcnit.UseChaosMesh - Installed 'chaos-mesh' Helm templates:
NAME: chaos-mesh
LAST DEPLOYED: Fri May 29 16:41:46 2020
NAMESPACE: rcnit-chaos-mesh-testing
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Make sure chaos-mesh components are running
kubectl get pods --namespace rcnit-chaos-mesh-testing -l app.kubernetes.io/instance=chaos-mesh
16:42:12.934 [main] INFO rcnit.util.ExecKubernetesCmd - Waiting for all pods in 'rcnit-chaos-mesh-testing' namespace to be ready...
16:42:12.939 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'chaos-controller-manager-649544f84-229rg' is ready
16:42:12.947 [main] INFO rcnit.util.ExecKubernetesCmd - Pod 'chaos-daemon-cfrsx' is ready
Given Chaos Mesh is running # rcnit.UseChaosMesh.chaos_mesh_is_running()
16:42:13.066 [main] INFO rcnit.util.DeployChaos - Created 'hello-world' deployment
16:42:13.066 [main] INFO rcnit.util.DeployChaos - Waiting for all 'hello-world' deployment replicas to become available...
16:42:22.788 [main] INFO rcnit.util.DeployChaos - All 'hello-world' deployment replicas are available
16:42:38.247 [main] INFO rcnit.util.DeployChaos - podchaos.pingcap.com/pod-failure created