-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(pod-experiments): Add volume-fill experiment (to target pvc) #458
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: Remi Ziolkowski <[email protected]>
Signed-off-by: Remi Ziolkowski <[email protected]>
} | ||
} | ||
|
||
//diskFill contains steps to inject disk-fill chaos |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can change diskfill -> volumefill at all places
volumes := container.VolumeMounts | ||
for _, volume := range volumes { | ||
if volume.Name == experimentsDetails.TargetVolume { | ||
mountPath = volume.MountPath |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can handle the case when mountPath is empty - invalid target volume case
<th> Documentation Link </th> | ||
</tr> | ||
<tr> | ||
<td> Disk Fill </td> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Volume Fill
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can change the description as well
} | ||
|
||
//PRE-CHAOS AUXILIARY APPLICATION STATUS CHECK | ||
if experimentsDetails.AuxiliaryAppInfo != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can remove AuxiliaryApps status checks(from both pre/post checks) as it needs cluster role permissions
value: '' | ||
|
||
- name: CONTAINER_PATH | ||
value: '/var/lib/docker/containers' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/var/lib/docker/volumes
experimentDetails.AuxiliaryAppInfo = types.Getenv("AUXILIARY_APPINFO", "") | ||
experimentDetails.TargetContainer = types.Getenv("TARGET_CONTAINER", "") | ||
experimentDetails.TargetVolume = types.Getenv("TARGET_VOLUME", "") | ||
experimentDetails.ContainerPath = types.Getenv("CONTAINER_PATH", "/var/lib/docker/containers") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/var/lib/docker/volumes/
|
||
fullPath := strings.TrimPrefix(volumePath, experimentsDetails.ContainerPath) | ||
|
||
du := fmt.Sprintf("sudo df /diskfill/%v |tail -n+2 |awk '{print $3}'", fullPath) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a comment on how this command is working?
} | ||
usedStorageSize, err := strconv.Atoi(strings.TrimSuffix(string(out), "\n")) | ||
|
||
du2 := fmt.Sprintf("sudo df /diskfill/%v |tail -n+2 |awk '{print $4}'", fullPath) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a comment on how this command is working?
return err | ||
} | ||
} else { | ||
log.Warn("No required free space found!, It's Housefull") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
log.Warn("No required free space found!, It's Housefull") | |
log.Warn("No required free space found!") |
return nil | ||
} | ||
|
||
// getEphemeralStorageAttributes derive the ephemeral storage attributes from the target pod |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// getEphemeralStorageAttributes derive the ephemeral storage attributes from the target pod | |
// getVolumePath derives the ephemeral storage attributes from the target pod |
return nil | ||
} | ||
|
||
//stopDockerContainer kill the application container |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
//stopDockerContainer kill the application container | |
// inspectDockerContainer returns the docker container mounts |
// creating the helper pod to perform disk-fill chaos | ||
for _, pod := range targetPodList.Items { | ||
runID := common.GetRunID() | ||
if err := createHelperPod(experimentsDetails, clients, chaosDetails, pod.Name, pod.Spec.NodeName, runID, labelSuffix); err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a log before this step?
} | ||
|
||
// creating the helper pod to perform disk-fill chaos | ||
for _, pod := range targetPodList.Items { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a log before this step?
<tr> | ||
<td> Disk Fill </td> | ||
<td> This experiment causes disk stress by filling up the ephemeral storage space of the pod and forces the pod to get evicted if the used space exceeds the set ephemeral storage limit. </td> | ||
<td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/disk-fill/"> Here </a> </td> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/disk-fill/"> Here </a> </td> | |
<td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/volume-fill/"> Here </a> </td> |
log.Infof("[PreReq]: Updating the chaos result of %v experiment (SOT)", experimentsDetails.ExperimentName) | ||
if err := result.ChaosResult(&chaosDetails, clients, &resultDetails, "SOT"); err != nil { | ||
log.Errorf("Unable to Create the Chaos Result, err: %v", err) | ||
failStep := "Updating the chaos result of disk-fill experiment (SOT)" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you update the failSteps so that the error string is also appended to them? You can check any other experiment file for reference, all the experiment files have been modified as such.
labels: | ||
app: litmus-experiment | ||
spec: | ||
serviceAccountName: disk-fill-sa |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
serviceAccountName: disk-fill-sa | |
serviceAccountName: volume-fill-sa |
Signed-off-by: Remi Ziolkowski [email protected]
What this PR does / why we need it:
It add a new experiment into litmus lib that permit to target pvc into container
Which issue this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)
format, will close that issue when PR gets merged): fixes NoSpecial notes for your reviewer:
No
Checklist:
breaking-changes
tagrequires-upgrade
tag