Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pod-experiments): Add volume-fill experiment (to target pvc) #458

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

Vr00mm
Copy link
Member

@Vr00mm Vr00mm commented Oct 19, 2021

Signed-off-by: Remi Ziolkowski [email protected]

What this PR does / why we need it:
It add a new experiment into litmus lib that permit to target pvc into container

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes No

Special notes for your reviewer:
No

Checklist:

  • Fixes #
  • PR messages has document related information
  • Labelled this PR & related issue with breaking-changes tag
  • PR messages has breaking changes related information
  • Labelled this PR & related issue with requires-upgrade tag
  • PR messages has upgrade related information
  • Commit has unit tests
  • Commit has integration tests
  • E2E run Required for the changes

Signed-off-by: Remi Ziolkowski <[email protected]>
@Vr00mm Vr00mm changed the title Add volume-fill experiment (to target pvc) feat(pod-experiments): Add volume-fill experiment (to target pvc) Oct 19, 2021
@neelanjan00 neelanjan00 added the enhancement New feature or request label May 25, 2022
}
}

//diskFill contains steps to inject disk-fill chaos
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can change diskfill -> volumefill at all places

volumes := container.VolumeMounts
for _, volume := range volumes {
if volume.Name == experimentsDetails.TargetVolume {
mountPath = volume.MountPath
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can handle the case when mountPath is empty - invalid target volume case

<th> Documentation Link </th>
</tr>
<tr>
<td> Disk Fill </td>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Volume Fill

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can change the description as well

}

//PRE-CHAOS AUXILIARY APPLICATION STATUS CHECK
if experimentsDetails.AuxiliaryAppInfo != "" {
Copy link
Member

@ispeakc0de ispeakc0de Oct 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can remove AuxiliaryApps status checks(from both pre/post checks) as it needs cluster role permissions

value: ''

- name: CONTAINER_PATH
value: '/var/lib/docker/containers'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/var/lib/docker/volumes

experimentDetails.AuxiliaryAppInfo = types.Getenv("AUXILIARY_APPINFO", "")
experimentDetails.TargetContainer = types.Getenv("TARGET_CONTAINER", "")
experimentDetails.TargetVolume = types.Getenv("TARGET_VOLUME", "")
experimentDetails.ContainerPath = types.Getenv("CONTAINER_PATH", "/var/lib/docker/containers")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/var/lib/docker/volumes/


fullPath := strings.TrimPrefix(volumePath, experimentsDetails.ContainerPath)

du := fmt.Sprintf("sudo df /diskfill/%v |tail -n+2 |awk '{print $3}'", fullPath)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment on how this command is working?

}
usedStorageSize, err := strconv.Atoi(strings.TrimSuffix(string(out), "\n"))

du2 := fmt.Sprintf("sudo df /diskfill/%v |tail -n+2 |awk '{print $4}'", fullPath)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment on how this command is working?

return err
}
} else {
log.Warn("No required free space found!, It's Housefull")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
log.Warn("No required free space found!, It's Housefull")
log.Warn("No required free space found!")

return nil
}

// getEphemeralStorageAttributes derive the ephemeral storage attributes from the target pod
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// getEphemeralStorageAttributes derive the ephemeral storage attributes from the target pod
// getVolumePath derives the ephemeral storage attributes from the target pod

return nil
}

//stopDockerContainer kill the application container
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//stopDockerContainer kill the application container
// inspectDockerContainer returns the docker container mounts

// creating the helper pod to perform disk-fill chaos
for _, pod := range targetPodList.Items {
runID := common.GetRunID()
if err := createHelperPod(experimentsDetails, clients, chaosDetails, pod.Name, pod.Spec.NodeName, runID, labelSuffix); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a log before this step?

}

// creating the helper pod to perform disk-fill chaos
for _, pod := range targetPodList.Items {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a log before this step?

<tr>
<td> Disk Fill </td>
<td> This experiment causes disk stress by filling up the ephemeral storage space of the pod and forces the pod to get evicted if the used space exceeds the set ephemeral storage limit. </td>
<td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/disk-fill/"> Here </a> </td>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/disk-fill/"> Here </a> </td>
<td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/pods/volume-fill/"> Here </a> </td>

log.Infof("[PreReq]: Updating the chaos result of %v experiment (SOT)", experimentsDetails.ExperimentName)
if err := result.ChaosResult(&chaosDetails, clients, &resultDetails, "SOT"); err != nil {
log.Errorf("Unable to Create the Chaos Result, err: %v", err)
failStep := "Updating the chaos result of disk-fill experiment (SOT)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the failSteps so that the error string is also appended to them? You can check any other experiment file for reference, all the experiment files have been modified as such.

labels:
app: litmus-experiment
spec:
serviceAccountName: disk-fill-sa
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
serviceAccountName: disk-fill-sa
serviceAccountName: volume-fill-sa

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants