Memory Manager

[cezaryzukowski]

Enhancement Description

• One-line enhancement description (can be used as a release note): Memory Manager is a new component in kubelet ecosystem proposed to offer multi-NUMA guaranteed memory allocation for pods in Guaranteed QoS class.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/1769-memory-manager
• Primary contact (assignee): @Tal-or
• Responsible SIGs: sig-node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (1.21)
  • Beta release target (1.22)
  • Stable release target (1.32)
• Alpha
  • KEP (k/enhancements) update PR(s): Submitting a KEP for production readiness approval #2456
  • Code (k/k) update PR(s): Memory manager kubernetes#95479
  • Docs (k/website) update(s): The Memory Manager official documentation website#26730
• Beta
  • KEP (k/enhancements) update PR(s): Graduate memory manager to beta #2708
  • Code (k/k) update PR(s):
    • memory manager: move to beta kubernetes#101947
    • Extend pod resource API response to return the information from memory manager kubernetes#101030
  • Docs (k/website) update(s):
    • The Memory Manager graduates to Beta website#28851
    • Blog: Add memory manager moves to beta feature blog post 1.22 website#28958
• Stable
  • KEP (k/enhancements) update PR(s): KEP-1769: Graduate Memory Manager to GA #4251
  • Code (k/k) update PR(s):
    • kubelet: memorymanager: metrics: add metrics about static allocation  kubernetes#121778
    • memorymanager: avoid violating NUMA node memory allocation rule kubernetes#123443
    • memorymanager: fix the bug of numa node grouping change kubernetes#113090
    • memorymanager: fix checkpoint file comparison kubernetes#127074
    • node: kubelet: memmgr: Memory Manager to GA kubernetes#128517
  • Docs (k/website) update(s):
    • blog: Memory Manager goes GA website#48578
    • kubelet memory manager GA graduation website#48494

[k8s-ci-robot]

@cezaryzukowski: The label(s) sig/sig-node cannot be applied, because the repository doesn't have them

In response to this:

/sig sig-node

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[cezaryzukowski]

/sig node

[cezaryzukowski]

/kind kep
/kind feature

[cezaryzukowski]

/milestone v1.19

[k8s-ci-robot]

@cezaryzukowski: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[palnabarun]

Hey there @cezaryzukowski @bg-chun -- 1.19 Enhancements Lead here. I wanted to check in and see if you think this Enhancement will be graduating to Alpha in 1.19?

In order to have this part of the release:

1. The KEP PR must be merged in an implementable state
2. The KEP must have test plans
3. The KEP must have graduation criteria.

As an additional note, #1620 merged recently, adding production readiness review questions to the KEP template. We are not making it mandatory for the 1.19 release cycle, but it would be great if the PRR questionnaire is filled since the KEP PR is in flight.

If you do, I'll add it to the 1.19 tracking sheet (http://bit.ly/k8s-1-19-enhancements). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks! 🙂

---

The current release schedule is:

• Monday, April 13: Week 1 - Release cycle begins
• Tuesday, May 19: Week 6 - Enhancements Freeze
• Thursday, June 25: Week 11 - Code Freeze
• Thursday, July 9: Week 14 - Docs must be completed and reviewed
• Tuesday, August 4: Week 17 - Kubernetes v1.19.0 released

[palnabarun]

@cezaryzukowski @bg-chun -- Unfortunately, the deadline for the 1.19 Enhancement freeze has passed. For now, this is being removed from the milestone and 1.19 tracking sheet. If there is a need to get this in, please file an enhancement exception.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palnabarun]

/remove-lifecycle stale

[kikisdeliveryservice]

Hi @cezaryzukowski @bg-chun !

Enhancements Lead here, do you still intend to do any work on this (alpha/beta/stable) for 1.20?

Thanks!
Kirsten

[cezaryzukowski]

Hi @kikisdeliveryservice, yes w plan Alpha for v1.20 as a target.
Should we update the description above?

PS We planned Alpha for v1.19, but we couldn't make it.

[kikisdeliveryservice]

Thanks for the update @cezaryzukowski !

As a reminder the KEP PR should also include a kep.yaml file, please see here for the templates:
https://github.com/kubernetes/enhancements/tree/master/keps/NNNN-kep-template

If you could also update the description to reflect alpha in 1.20 that would be great.

Best,
Kirsten

/milestone v1.20

[kikisdeliveryservice]

Hi @cezaryzukowski @bg-chun

As a reminder Enhancements Freeze is next week ** Tuesday October 6th**. By that time your completed KEP PR will need to merge.

If you have any questions, please let me know

Best,
Kirsten

[kikisdeliveryservice]

Update: the PR merged! 👍

[cezaryzukowski]

@kikisdeliveryservice Many thanks for your support.

Do we need to do anything else beyond "KEP merged" to have this feature in v1.20?

[kikisdeliveryservice]

Hi @cezaryzukowski at the moment, no. 😄 But you do have to keep to all of the remaining milestone deadlines such as Code Freeze (Nov 12th) and Test Freeze(Nov 23rd) as well as the Docs Placeholder PR deadline of Nov 6th.

As you go forward, also please link all PRs related to this enhancement on this issue.

Lmk if you have any questions!!! I'll be here 😄 and will remind you of upcoming important dates.

[cezaryzukowski]

Hi @kikisdeliveryservice

That's grand, we know the exact schedule.

With respect to "Docs Placeholder PR", is there any template or guideline on how to prepare the docs?

[kikisdeliveryservice]

Hi @cezaryzukowski !

So the placeholder PR is very simple. See: https://kubernetes.io/docs/contribute/new-content/new-features/#open-a-placeholder-pr. This tells the docs team that you intend on adding feature documentation. Later on you'll add that full documentation and work with the docs team. You just need to do the placeholder by Nov. 6th but the sooner the better =)

Also, going forward, please link the placeholder PR and any k/k PRs back to this issue so the team can track them.

If you have any questions, let me know 😸

Just to recap:
Upcoming dates:
Friday, Nov 6th: Week 8 - Docs Placeholder PR deadline
Thursday, Nov 12th: Week 9 - Code Freeze
Monday, Nov 23rd: Week 11 - Test Freeze

Thanks!
Kirsten

[pacoxu]

/remove-lifecycle rotten

[rashansmith]

Hello @Tal-or 👋 from the v1.32 Communications Team!
We'd love for you to consider writing a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt-in, let us know and open a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

[Tal-or]

Hello @Tal-or 👋, 1.32 Docs Shadow here. Does this enhancement work planned for 1.32 require any new docs or modification to existing docs? If so, please follows the steps here to open a PR against dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday October 24th 2024 18:00 PDT. Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release. Thank you!

Hey @michellengnx,

Thanks for reaching out. there's no to update docs because no user-facing changes has been done.
The only change in the doc is maybe changing the feature maturity from beta -> GA

[Tal-or]

Hello @Tal-or 👋 from the v1.32 Communications Team! We'd love for you to consider writing a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.

To opt-in, let us know and open a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

Hey @rashansmith,

Thanks for reaching out. I would like to post a blog indeed.
I'll be back from vacation on 27th Oct 2024 and open a placehodler PR.
Could you please advice what is the deadline for the blog to be completed?

[rashansmith]

Hello @Tal-or 👋 from the v1.32 Communications Team! We'd love for you to consider writing a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating.
To opt-in, let us know and open a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

Hey @rashansmith,

Thanks for reaching out. I would like to post a blog indeed. I'll be back from vacation on 27th Oct 2024 and open a placehodler PR. Could you please advice what is the deadline for the blog to be completed?

Thanks for the update @Tal-or . Blog review will begin on November 10th, so it should be completed by then. Let me know if you have any more questions!

[ffromani]

Hello @Tal-or 👋, 1.32 Docs Shadow here. Does this enhancement work planned for 1.32 require any new docs or modification to existing docs? If so, please follows the steps here to open a PR against dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday October 24th 2024 18:00 PDT. Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release. Thank you!

Hey @michellengnx,

Thanks for reaching out. there's no to update docs because no user-facing changes has been done. The only change in the doc is maybe changing the feature maturity from beta -> GA

This is my fault, I misunderstood the requirements. We will still need a PR, albeit likely a trivial one for the reasons above. Filed: kubernetes/website#48494

[jenshu]

Hey again @Tal-or 👋, v1.32 Enhancements team here.

Just checking in as we approach code freeze at 02:00 UTC Friday 8th November 2024 / 19:00 PDT Thursday 7th November 2024.

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, it looks like the following PRs need to be merged before code freeze (and we need to update the Issue description to include all the related PRs of this KEP):

• kubelet: memorymanager: metrics: add metrics about static allocation  kubernetes#121778
• memorymanager: fix the bug of numa node grouping change kubernetes#113090
• memorymanager: fix checkpoint file comparison kubernetes#127074

If you anticipate missing code freeze, you can file an exception request in advance.

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.
As always, we are here to help if any questions come up. Thanks!

[rashansmith]

Hello @Tal-or !

Just reaching out to remind you to author your feature blog!

To opt-in, let us know and open a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

Please feel free to reach out if you have any questions.

[Tal-or]

Hello @rashansmith

I opened a placeholder PR kubernetes/website#48578
Will continue the work during this week.

[Tal-or]

Thank you @jenshu for the reminder.

For this enhancement, it looks like the following PRs need to be merged before code freeze (and we need to update the Issue description to include all the related PRs of this KEP):

• kubelet: memorymanager: metrics: add metrics about static allocation  kubernetes#121778
• memorymanager: fix the bug of numa node grouping change kubernetes#113090
• memorymanager: fix checkpoint file comparison kubernetes#127074

I do not have permission to edit the original issue description. we're tracking the recent work here:
#1769 (comment)

If you anticipate missing code freeze, you can file an exception request in advance.

@ffromani WDYT?

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.

At this point there's none.

[jenshu]

I do not have permission to edit the original issue description. we're tracking the recent work here:
#1769 (comment)

@Tal-or thanks, I've updated the issue description to add the missing PR

[ffromani]

If you anticipate missing code freeze, you can file an exception request in advance.

@ffromani WDYT?

Too early to tell. Let's reconvene after sig-node meeting 2024/Oct/29

[Tal-or]

Bump the open tasks here again for visibility since I cannot edit the issue's description:
The following issues need to be addressed in order to consider this work complete (GA-ready):

• Memory manager UnexpectedAdmissionError kubernetes#120733

  • Fix: memorymanager: avoid violating NUMA node memory allocation rule kubernetes#123443
  • Actively handled: Yes - Waiting for review

• kubelet: memorymanager static policy startup error kubernetes#113130

  • Fix: memorymanager: fix checkpoint file comparison kubernetes#127074
  • Actively handled: Yes

• kubelet: when memoryManager are enabled, kubelet fails to restart occasionally kubernetes#113035

  • Fix: memorymanager: fix the bug of numa node grouping change kubernetes#113090
  • Actively handled: No
    • Reason: Marked as duplicate of kubelet: memorymanager static policy startup error kubernetes#113130

• blog: Memory Manager goes GA website#48578 - A blog PR

• node: kubelet: memmgr: Memory Manager to GA kubernetes#128517 - A PR to finalize the changes for MemoryManager graduation

• kubelet memory manager GA graduation website#48494 - A docs PR

[Tal-or]

Hello @rashansmith,

The blog is ready for review: kubernetes/website#48578

[jenshu]

Hello @Tal-or 👋, v1.32 Enhancements team here.

With all the implementation (code related) PRs merged as per the issue description:

• kubelet: memorymanager: metrics: add metrics about static allocation  kubernetes#121778
• memorymanager: avoid violating NUMA node memory allocation rule kubernetes#123443
• memorymanager: fix checkpoint file comparison kubernetes#127074
• node: kubelet: memmgr: Memory Manager to GA kubernetes#128517

This enhancement is now marked as tracked for code freeze for the v1.32 Code Freeze!

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.

[Tal-or]

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.

Thank you @jenshu,

Filed a PR: #4959