HPCC-27610 Update Container Placements Documentation #17917

g-pan · 2023-10-18T14:05:23Z

Type of change:

This change is a bug fix (non-breaking change which fixes an issue).
This change is a new feature (non-breaking change which adds functionality).
This change improves the code (refactor or other change that does not change the functionality)
This change fixes warnings (the fix does not alter the functionality or the generated code)
This change is a breaking change (fix or feature that will cause existing behavior to change).
This change alters the query API (existing queries will have to be recompiled)

Checklist:

Smoketest:

Send notifications about my Pull Request position in Smoketest queue.
Test my draft Pull Request.

Testing:

Successful unit testing: http://10.224.20.18/view/Docs-gp/job/DocBuild-GPRepo3/

github-actions · 2023-10-18T15:44:39Z

https://track.hpccsystems.com/browse/HPCC-27610
Jira updated

JamesDeFabia

A few comments inline

JamesDeFabia · 2023-10-25T18:30:35Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+    needs.</para>
+
+    <para>You can deploy these values either using the values.yaml file or you
+    can place into an file and have Kubernetes instead read the values from


a file, not an file

JamesDeFabia · 2023-10-25T18:36:24Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

-          <orderedlist>
+        <para>The pods: [list] item can contain one of the following:</para>
+
+        <para><orderedlist>


This should not be a numbered list. A table would be a better representation.

JamesDeFabia · 2023-10-25T18:37:23Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

-        <emphasis>global.env</emphasis> portion of the provided HPCC Systems
-        values.yaml file. These values are specified as a list of name value
-        pairs as illustrated below.</para>
+      <para>In a Kubernetes container environment there are several ways to


needs a comma :
In a Kubernetes container environment, there...

JamesDeFabia · 2023-10-25T18:41:13Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+       nodeSelector:
+         group: "hpcc"</programlisting></para>
+
+        <para>Note:the label: group:hpcc matches the node pool label:


Note: s/b bold, then a space and capitalize the sentence. also no need for the quotes around hpcc.

Note: The label: group:hpcc matches the node pool label:hpcc.

JamesDeFabia · 2023-10-25T18:43:43Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

-                    <entry>JAVA_LIBRARY_PATH</entry>
-                  </row>
+        <para>Taints and Tolerations are types of Kubernetes node constraints
+        also referred to by Node Affinity. Only one "affinity" can be applied


inconsistent capitalization of Node Affinity. Previously, it was node Affinity.

JamesDeFabia · 2023-10-25T18:44:26Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

-                    <entry>JAVA_LIBRARY_PATH</entry>
-                  </row>
+        <para>Taints and Tolerations are types of Kubernetes node constraints
+        also referred to by Node Affinity. Only one "affinity" can be applied


no need for quotes around affinity (2 places)

JamesDeFabia · 2023-10-25T18:47:45Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+        respectively. The Roxie pods will be evenly scheduled on the two node
+        pools.</para>
+
+        <para>After deployment you can verify by issuing the following:</para>


After deployment you can verify by issuing the following command:

JamesDeFabia · 2023-10-25T18:49:22Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+
+          <para>There is no schema check for the content of affinity. Only one
+          affinity can be applied to a pod or job. If a pod/job matches
+          multiple placement 'pods' lists, then only the last affinity


no need for the quotes around 'pods'

JamesDeFabia · 2023-10-25T18:51:31Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+
+        <para>Only one "schedulerName" can be applied to any pod/job.</para>
+
+        <para>A SchedulerName example:</para>


Inconsistent capitalization of schedulerName

xwang2713

Looks good for me. Just Remove "Examples:" before "Taints and Tolerations" on p20

JamesDeFabia

Couple of minor comments inline

JamesDeFabia · 2023-10-30T19:45:09Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+      refer to Kubernetes scheduling as placements. Placements is a value in
+      an HPCC Systems configuration which is at a level above items, such as
+      the nodeSelector, Toleration, Affinity and Anti-Affinity, and
+      TopologySpreadConstratints.</para>


DId you mean TopologySpreadConstraints?

JamesDeFabia · 2023-10-30T19:46:16Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+      TopologySpreadConstratints.</para>
+
+      <para>The placement is responsible for finding the best node for a pod.
+      Most often placement are handled automatically by Kubernetes. You can


Subject/Verb agreement: S/B Most often placements are handled

JamesDeFabia · 2023-10-30T19:47:13Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+    <sect2 id="YAML_FileStruct_Placement">
+      <title>Placements</title>
+
+      <para>The Placements is a term used by HPCC Systems, which Kubernetes


remove word "The"

JamesDeFabia

Looks good from my POV

JamesDeFabia

Looks good from my POV

JamesDeFabia

Looks OK from my POV

jakesmith · 2023-11-02T16:56:14Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+
+                <entry>This is the "Deployment" metadata name from the name of
+                the array item of a type. For example, "eclwatch-", "mydali-",
+                "thor-thoragent" This can be a regular expression since


@xwang2713 - are regular expressions supported or just wildcards?

The regular expressions are preferred. But I think substrings are also OK . Helm docs (mustRegexMatch):
regexMatch, mustRegexMatch. Returns true if the input string contains any match of the regular expression.

jakesmith · 2023-11-02T16:57:41Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+                <entry>The job name is typically a regular expression as well,
+                since the job name is generated dynamically. For example, a
+                compile job compile-54eB67e567e, could use "compile-" or
+                "compile-*" or the exact match "^compile-.$"</entry>


@xwang2713 - I want to clarify if this supports regular expressions or just wildcards, "compile-*" is not an example of a regular expression (well it is, but would match 'compile----' , or 'compile', not 'compile-blah')

Either "compile-.*" or "compile-". "compile-*" is not valid regular expression or substring

Ok, then line 1308 needs changing to "compile-.*"
@g-pan

jakesmith · 2023-11-02T16:58:35Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+          <para>NodeSelector, taints and tolerations, and other values can all
+          be placed on the same pods: [list] both per zone and per node on
+          Azure <programlisting>placements:
+- pods: ["eclwatch","roxie-workunit","^compile-.*$","mydali"]


but 'compile-.$' is a regex, so is it that it does support regex's, but the example 'compile-' is wrong?

jakesmith

@g-pan - please see question re. regex/pattern match.

ghalliday · 2023-11-10T10:20:43Z

Assigned to @xwang2713 to answer Jake's questions.

stuartort · 2023-12-07T15:23:57Z

@g-pan, the jira is assigned to you. You have indicated that you have the information that you need

jakesmith · 2023-12-08T09:17:28Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

@@ -1305,7 +1305,7 @@ thor:
                <entry>The job name is typically a regular expression as well,
                since the job name is generated dynamically. For example, a
                compile job compile-54eB67e567e, could use "compile-" or
-                "compile-*" or the exact match "^compile-.$"</entry>
+                "compile-.*" or the exact match "^compile-.$"</entry>


minor: sorry, I did not notice this before:

For example, a compile job compile-54eB67e567e, could use "compile-" or "compile-.*" or the exact match "^compile-.$"

"exact match" in this sentence is misleading/inaccurate, and the regular expression "compile-.$" does not match.
I'd probably drop that qualifier and correct the regex :

For example, a compile job compile-54eB67e567e, could use "compile-", "compile-.*" or "^compile-.*$"

jakesmith

@g-pan - see follow-up comment.

g-pan · 2023-12-11T13:24:05Z

added the last change and squashed (since it was minor) should be ready to merge now.

jakesmith · 2023-12-11T14:33:21Z

docs/EN_US/ContainerizedHPCC/ContainerizedMods/ConfigureValues.xml

+                <entry>The job name is typically a regular expression as well,
+                since the job name is generated dynamically. For example, a
+                compile job compile-54eB67e567e, could use "compile-" or
+                "compile-.*" or "^compile-.$"</entry>


@g-pan - still not quite right, missing * before $, see previous comment

jakesmith

@g-pan - back to you.

Signed-off-by: g-pan <[email protected]>

g-pan · 2023-12-11T20:27:03Z

@g-pan - back to you.

good catch!

jakesmith

@g-pan - looks good now.

g-pan requested review from JamesDeFabia and xwang2713 October 18, 2023 14:05

JamesDeFabia requested changes Oct 25, 2023

View reviewed changes

hpcc-systems deleted a comment from JamesDeFabia Oct 25, 2023

g-pan force-pushed the H27610-Plmt branch from 31433aa to 7948bc1 Compare October 25, 2023 20:20

xwang2713 approved these changes Oct 30, 2023

View reviewed changes

g-pan requested a review from JamesDeFabia October 30, 2023 19:28

JamesDeFabia requested changes Oct 30, 2023

View reviewed changes

g-pan force-pushed the H27610-Plmt branch from 6e23ff4 to f2988af Compare October 31, 2023 14:19

JamesDeFabia approved these changes Oct 31, 2023

View reviewed changes

g-pan requested a review from JamesDeFabia October 31, 2023 14:59

JamesDeFabia approved these changes Oct 31, 2023

View reviewed changes

g-pan requested review from jakesmith and JamesDeFabia October 31, 2023 15:08

JamesDeFabia approved these changes Oct 31, 2023

View reviewed changes

jakesmith reviewed Nov 2, 2023

View reviewed changes

ghalliday assigned xwang2713 Nov 10, 2023

g-pan requested a review from jakesmith December 7, 2023 19:51

jakesmith reviewed Dec 8, 2023

View reviewed changes

g-pan force-pushed the H27610-Plmt branch from 47e8bcb to ef0efa3 Compare December 11, 2023 13:23

g-pan requested a review from jakesmith December 11, 2023 13:25

jakesmith reviewed Dec 11, 2023

View reviewed changes

HPCC-27610 Update Container Placements Documentation

9d70a3a

Signed-off-by: g-pan <[email protected]>

g-pan force-pushed the H27610-Plmt branch from ef0efa3 to 9d70a3a Compare December 11, 2023 20:26

g-pan requested a review from jakesmith December 11, 2023 20:27

jakesmith approved these changes Dec 12, 2023

View reviewed changes

ghalliday merged commit 547610b into hpcc-systems:candidate-9.2.x Dec 14, 2023
50 checks passed


		<para>Only one "schedulerName" can be applied to any pod/job.</para>

		<para>A SchedulerName example:</para>

HPCC-27610 Update Container Placements Documentation #17917

HPCC-27610 Update Container Placements Documentation #17917

Conversation

g-pan commented Oct 18, 2023

Type of change:

Checklist:

Smoketest:

Testing:

github-actions bot commented Oct 18, 2023

JamesDeFabia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xwang2713 left a comment

Choose a reason for hiding this comment

JamesDeFabia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JamesDeFabia Oct 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JamesDeFabia left a comment

Choose a reason for hiding this comment

JamesDeFabia left a comment

Choose a reason for hiding this comment

JamesDeFabia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xwang2713 Dec 6, 2023 • edited by jakesmith Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakesmith left a comment

Choose a reason for hiding this comment

ghalliday commented Nov 10, 2023

stuartort commented Dec 7, 2023

Choose a reason for hiding this comment

jakesmith left a comment

Choose a reason for hiding this comment

g-pan commented Dec 11, 2023

Choose a reason for hiding this comment

jakesmith left a comment

Choose a reason for hiding this comment

g-pan commented Dec 11, 2023

jakesmith left a comment

Choose a reason for hiding this comment

JamesDeFabia Oct 30, 2023 •

edited

Loading

xwang2713 Dec 6, 2023 •

edited by jakesmith

Loading