Improved multi-numa alignment in Topology Manager

[klueska]

Enhancement Description

• One-line enhancement description (can be used as a release note):
  An enhancement to the TopologyManager that considers relative distances between NUMA nodes when calculating "best hints" that include more than one NUMA node.

• Kubernetes Enhancement Proposal:
  https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/3545-improved-multi-numa-alignment/README.md

• Discussion Link:
  Private slack messages

• Primary contact (assignee):
  @PiotrProkop

• Responsible SIGs:
  /sig node

• Enhancement target (which target equals to which milestone):

  • Alpha release target: 1.26
  • Beta release target: 1.28
  • Stable release target: 1.32

• Alpha

  • KEP (k/enhancements) update PR(s):
    KEP-3545: Improved multi-numa alignment in Topology Manager #3549
  • Code (k/k) update PR(s):
    node: topologymanager: Improved multi-numa alignment in Topology Manager kubernetes#112914
  • Docs (k/website) update PR(s):
    topologymanager: document topology manager policy options website#37668

• Beta

  • KEP (k/enhancements) update PR(s):
    Promote Improved multi-numa alignment in Topology Manager to beta #4079
  • Code (k/k) update PR(s):
    topologymanager: Promote support for improved multi-numa alignment in Topology Manager to beta kubernetes#118816
  • Docs (k/website) update(s):
    topologymanager: document topology manager policy options promotion to beta  website#41718

• Stable:

  • KEP (k/enhancements) update PR(s):
    KEP-3545: graduate to GA #4882
  • Code (k/k) update PR(s):
    add e2e tests for prefer-closest-numa-nodes TopologyManagerPolicyOption kubernetes#127922
    topologymanager: Promote support for improved multi-numa alignment in Topology Manager to GA kubernetes#128124
  • Docs (k/website) update(s):
    topology-manager: promoto TopologyManagerPolicyOptions to stable website#48339

[klueska]

/cc @fromanirh
/cc @swatisehgal

[ffromani]

thanks for the heads up, I'll review shortly

[derekwaynecarr]

/label lead-opted-in

[Atharva-Shinde]

Hey @klueska 👋, 1.26 Enhancements team here!

Just checking in as we approach Enhancements Freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage alpha for 1.26

Here's where this enhancement currently stands:

• KEP file using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable
• KEP has an updated detailed test plan section filled out
• KEP has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to:
-Get the PR #3549 merged before Enhancements Freeze to make this enhancement eligible for 1.26 release.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well.
Thank you :)

[Atharva-Shinde]

/milestone v1.26
/label tracked/yes

[k8s-ci-robot]

@Atharva-Shinde: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.26
/label tracked/yes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Atharva-Shinde]

/stage alpha

[Atharva-Shinde]

Hello @klueska 👋, just a quick check-in again, as we approach the 1.26 Enhancements freeze.

Please plan to get the action items mentioned in my comment above done before Enhancements freeze on 18:00 PDT on Thursday 6th October 2022 i.e tomorrow

For note, the current status of the enhancement is marked at-risk :)

[rhockenbury]

We have this marked as tracked for v1.26 with #3549 merged.

[Atharva-Shinde]

Hi @klueska 👋,

Checking in as we approach 1.26 code freeze at 17:00 PDT on Tuesday 8th November 2022.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes). If there are any other PRs beyond node: topologymanager: Improved multi-numa alignment in Topology Manager kubernetes#112914, please include them in the Issue Description too.
• All PRs are fully merged by the code freeze deadline.

As always, we are here to help should questions come up.
Thanks :)

[Atharva-Shinde]

Hey @klueska 👋, just a quick check-in again before 1.26 code freeze at 17:00 PDT Tuesday 8th November 2022 i.e tomorrow.
Looks like we would at least need to get the code PR/s: kubernetes/kubernetes#112914 (any more PRs?) merged before the code-freeze :)

[krol3]

Hello @klueska 👋, 1.26 Release Docs Lead here. This enhancement is marked as ‘Needs Docs’ for 1.26 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.26 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by November 9. Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Any doubt, reach us! Thank you!

[haircommander]

/milestone v1.32
/label lead-opted-in

[kannon92]

Looking at our e2e tests around Multi-NUMA alignment, it seems that we don't have the right hardware to actually test this features. If you look at TopologyManager tests, they are being skipped because the tests do not have multi numa alignment.

• https://testgrid.k8s.io/sig-node-cri-o#ci-crio-cgroupv2-node-e2e-resource-managers
• https://testgrid.k8s.io/sig-node-containerd#node-kubelet-containerd-resource-managers

I posted this concern on #sigs-k8s-infra and it seems that we should be able to request the nodes if we know what kind of one to ask for.

Maybe we should get these jobs passing and not skipping for GA so we can make sure that these features don't regress.

[PiotrProkop]

@ffromani created an issue to track hardware requirements for sig-node e2e tests here:
kubernetes/k8s.io#7339

[impact-maker]

Hello @haircommander @PiotrProkop 👋, v1.32 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024.

This enhancement is targeting for stage stable for v1.32 (correct me, if otherwise).

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.32.
• KEP readme has up-to-date graduation criteria.
• KEP has submitted a production readiness review request for approval and has a reviewer assigned.
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 3rd October 2024 so that the PRR team has enough time to review your KEP.

For this KEP, we would need to update the following:

• KEP has submitted a production readiness review request for approval and has a reviewer assigned.
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 3rd October 2024 so that the PRR team has enough time to review your KEP.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[PiotrProkop]

@impact-maker I've created a PR with KEP updates for stable release and submitted a PRR review on the PR:
#4882
Is there something I am still missing?

[PiotrProkop]

I don't have permissions to edit the description of the issue in-place, so I'll post update here:

• Stable:
  • KEP (k/enhancements) update PR(s): KEP-3545: graduate to GA #4882
  • PRR review PR: KEP-3545: graduate to GA #4882
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):

[haircommander]

thanks @PiotrProkop I have added the section! Now we just need the PR to be reviewed/approved, but you're all set deadline-wise until the enhancement freeze next week

[haircommander]

It looks like the enhancement PRs have merged, so I'm moving this to tracked on the SIG node KEP board

[dipesh-rawat]

Hello @PiotrProkop 👋, v1.32 Enhancements team here,

Now that PR #4882 has been merged, all the KEP requirements are in place and merged into k/enhancements.

The status of this enhancement is now marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[michellengnx]

Hello @PiotrProkop and @klueska 👋, 1.32 Docs Shadow here.
Does this enhancement work planned for 1.32 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday October 24th 2024 18:00 PDT.
Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[michellengnx]

Hello @PiotrProkop and @klueska 👋, 1.32 Docs Shadow here. Does this enhancement work planned for 1.32 require any new docs or modification to existing docs? If so, please follows the steps here to open a PR against dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday October 24th 2024 18:00 PDT. Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release. Thank you!

Hello @PiotrProkop 👋,

Just a reminder to open a placeholder PR against dev-1.32 branch in the k/website repo for this (steps available here). The deadline for this is 4 days away at Thursday October 24, 2024 18:00 PDT.

Thanks,
Michelle

[PiotrProkop]

Hello @PiotrProkop and @klueska 👋, 1.32 Docs Shadow here. Does this enhancement work planned for 1.32 require any new docs or modification to existing docs? If so, please follows the steps here to open a PR against dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday October 24th 2024 18:00 PDT. Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release. Thank you!

Hello @PiotrProkop 👋,

Just a reminder to open a placeholder PR against dev-1.32 branch in the k/website repo for this (steps available here). The deadline for this is 4 days away at Thursday October 24, 2024 18:00 PDT.

Thanks, Michelle

Hey @michellengnx , I've already opened docs PR -> kubernetes/website#48339

[wrkode]

👋 Hi there, William here from v1.32 Comms
We'd love for you to consider writing a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt-in, let us know and open a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

[tjons]

Hello @klueska 👋, Enhancements team here

With all the implementation(code related) PRs merged as per the issue description:

• topologymanager: Promote support for improved multi-numa alignment in Topology Manager to GA kubernetes#128124
• add e2e tests for prefer-closest-numa-nodes TopologyManagerPolicyOption kubernetes#127922

This enhancement is now marked as tracked for code freeze for the v1.32 Code Freeze!

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.

[PiotrProkop]

Hello @klueska 👋, Enhancements team here

With all the implementation(code related) PRs merged as per the issue description:

* [topologymanager: Promote support for improved multi-numa alignment in Topology Manager to GA kubernetes#128124](https://github.com/kubernetes/kubernetes/pull/128124)

* [add e2e tests for prefer-closest-numa-nodes TopologyManagerPolicyOption kubernetes#127922](https://github.com/kubernetes/kubernetes/pull/127922)

This enhancement is now marked as tracked for code freeze for the v1.32 Code Freeze!

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.

@tjons here is the PR to mark this KEP as implemented:

• KEP-3545: mark as implemented #4952