Merge pull request #3911 from EnterpriseDB/release/2023-04-11

Release: 2023-04-11

install_template/README.md

@@ -1,6 +1,6 @@
 # Install Docs Templates
 
-This script allows the generation of product instalation docs for many different product versions and platforms with a minimum of duplicated copy. [Nunjucks](https://mozilla.github.io/nunjucks/) is used as our templating engine.
+This script allows the generation of product installation docs for many different product versions and platforms with a minimum of duplicated copy. [Nunjucks](https://mozilla.github.io/nunjucks/) is used as our templating engine.
 
 ## Quick Start
 
@@ -28,6 +28,34 @@ When generating docs, this script will look for templates in `templates/products
 1. `{platform name}.njk`
    e.g. for EDB postgres advanced server 13 on centos 7 for x86_64, it would check for a file called `centos-7.njk`
 
+## Deploying generated files
+
+The `install-docs:build` command generates output into install_template/renders. To deploy them to their final location, run `npm run install-docs:deploy`. This command uses the same information from `config.yaml` to identify generated files, and then examines their frontmatter to determine their final resting place:
+
+```yaml
+deployPath: jdbc_connector/42.5.4.1/installing/linux_x86_64/jdbc_debian_10.mdx
+```
+
+You'll usually want to parameterize these when generating the file. A typical setup might look like this:
+
+```
+{% import "platformBase/_deploymentConstants.njk" as deploy %}
+{% block frontmatter %}
+{# 
+  If you modify deployment path here, please first copy the old expression
+  and add it to the list under "redirects:" below - this ensures we don't 
+  break any existing links.  
+#}
+deployPath: jdbc_connector/{{ product.version }}/installing/linux_{{platform.arch}}/jdbc_{{deploy.map_platform[platform.name]}}.mdx
+redirects:
+  - jdbc_connector/{{ product.version }}/04_installing_and_configuring_the_jdbc_connector/01_installing_the_connector_with_an_rpm_package/{{deploy.expand_arch[platform.arch]}}/jdbc42_{{deploy.map_platform_old[platform.name]}}_{{platform.arch | replace(r/_?64/g, "")}}.mdx
+{% endblock frontmatter %}
+```
+
+Note that the deployment script also rewrites the redirects values to allow deploy paths to be copied directly into that list when changed. It will:
+  - remove a trailing `.mdx` 
+  - add a leading `/` (but only when removing a trailing `.mdx`)
+
 ## Writing Templates
 
 We are using Nunjucks as the templating engine, which is a javascript implementation of Jinja.
@@ -60,7 +88,7 @@ After a template file is found, no rules are enforced on how that template shoul
 - These files are largely responsible for setting up the `prerequisites` and `installCommand` blocks
 - Currently, they rely on a `packageName` macro to be set by a child template. This macro is being used in the `installCommand` block.
 - You will notice there are no Ubuntu templates in the platformBase folder. This is because install instructions were the same as Debian 10, and so only the `debian-10.njk` file was created to reduce duplication. If Ubuntu specific instructions are needed, new template files could be created which inherit from `debian-10.njk`.
-- The `centos-7.njk` template contains a conditional to include ppc64le specific instructions. To display these instructions, add `{% set includePPC = true %}` to a child template
+
 
 #### `/templates/products/edb-postgres-advanced-server/base.njk`
 
@@ -69,6 +97,36 @@ After a template file is found, no rules are enforced on how that template shoul
 - the `packageName` macro used by the `platformBase` templates can be found in this template
 - a `postinstall` block containing copy shared by all of the EPAS docs can be found here. That said, this block is extended by some child templates.
 
+## Index files
+
+In addition to individual platform installer docs, you can also generate index files for each product-version combination. Two levels of index are supported:
+
+1. A top-level index covering all platforms. This template can take one of two forms:
+
+   1. `index.njk` - top level index for all product versions
+   1. `v{version}_index.njk` - top level index specific to one product version
+
+   The context provided for these templates includes the following values:
+
+   - `product.name` — the name of the product we are rendering a template for. e.g. "EDB Postgres Advanced Server"
+   - `product.version` — the version of the product we are rendering a template for. e.g. "13"
+   - `osArchitectures` — an object with keys for each supported CPU architecture, each containing an array of OS versions supported for that architecture. 
+      An OS version is represented by an object of the form,
+      ```javascript
+      {name: "Ubuntu 18.04", "Ubuntu", "Ubuntu", "18.04"}
+      ```
+      In addition, the array itself has several methods available:
+      - `hasOS(shortname, [osVersion])` returns true if the list contains an OS/version combination matching the OS shortname and optionally a specific OS version
+      - `hasFamily(family)` returns true if the list contains an OS from the specified family (e.g., `hasFamily("RHEL")` returns true if any of RHEL, Oracle Linux, Alma, Rocky or CentOS are listed)
+      - `filterFamily(family)` returns a subset of the list matching the specified family, sorted by version in descending order
+      - `filterOS(shortname, [osVersion])` returns a subset of the list matching the specified shortname and optionally a specific version, sorted by version in descending order
+
+1. An architecture-level index covering all platforms for a given CPU architecture. This template can take one of two forms:
+
+   1. `{arch}_index.njk` - top level index for all product versions
+   1. `v{version}_{arch}_index.njk` - top level index specific to one product version
+
+
 ### Template techniques
 
 Nunjucks offers many tools that we can use to minimize repetition and increase the readability of templates. Which to use depends on circumstances.
@@ -81,60 +139,89 @@ Note that the definition should be filled in at the most general template possib
 
 Placeholder blocks are convenient and self-documenting, but they do depend on having the right inheritance structure in place. It's sometimes necessary to add a level of inheritance to create the appropriate abstraction layers. If you find yourself repeating the same block customization, you should evaluate whether the inheritance structure is right. Alternatively, you might need to use another technique.
 
-#### Include other templates
+When adding a new placeholder, try to be mindful of folks who will need to add or modify overrides in the future; avoid placing them in contexts that require special consideration, and *never* use the same block in two different contexts. For example, avoid:
 
-When there are pieces of text that need to be repeated, but don't have a place in the existing structure, consider using [the `include` tag](https://mozilla.github.io/nunjucks/templating.html#include). For instance, we might want to includes a disclaimer for platforms nearing the end of our support cycle. Rather than repeat that warning in each affected template, we could put the text in a new template that we include when needed.
+- Using {% block install_commands %} inside a code block for one product, and inside a list for another.
+- Using {% block additional_prerequisites %} inside a code block that is never closed (and thus requires it to be overridden for no other reason than to close the block!)
 
-In Nunjucks, `include` doesn't work like a pre-processor (inserting code before rendering). Instead, included templates are rendered and the results are inserted. That means you can't use `include` to override a block in the calling template and blocks defined in the calling template aren't visible to the included template. Included templates should be considered self-contained snippets.
+If a particular block requires content to be formatted in a specific way, such as indented to match the indentation of a list... Consider using a `{% filter ... %}` block to transform the content as needed. For example:
 
-#### Macros
+```
+- This list
+- Includes
+- A block
+{% filter indent(2) %}
+{% block subitem %}
+{% endblock subitem %}
+- But derived templates
+- Don't need to know this
+```
 
-Sometimes a chunk of text varies only by a few discrete variables. In that case, a [macro](https://mozilla.github.io/nunjucks/templating.html#macro) might be in order. We currently have two macros in `/templates/platformBase/_shared.njk`:
+This example uses [`indent`](https://mozilla.github.io/nunjucks/templating.html#indent) to ensure the block's content will not break out of the list item. Other useful filters include [`replace(...)`](https://mozilla.github.io/nunjucks/templating.html#replace) and [`trim(...)`](https://mozilla.github.io/nunjucks/templating.html#trim) - more can be added if the need arises.
 
-1. `centosRepositoryConfiguration(packageManager, epelRepo)`
-2. `centosInstallCommand(packageManager, packageName)`
+#### Building on base blocks
 
-When that file is [imported](https://mozilla.github.io/nunjucks/templating.html#import) (not included) in another template, those macros can be used to insert the proper instructions for configuring the repository and installing the package. By using a few variables (`packageManager`, `epelRepo` and `packageName`), we can generate many different combinations of instructions from a single source. 
+Of special note is the ability to *integrate* parent blocks in derived templates via the `super()` function. This can greatly reduce the proliferation of placeholders by allowing you to build on existing blocks in derived templates:
 
-Macros can be self-documenting and avoid [action-at-a-distance problems](https://en.wikipedia.org/wiki/Action_at_a_distance_(computer_programming)). If you call a macro, it's easier to know what you are going to get in the render.
+```
+{% block prerequisites %}
+Some base content
+{% endblock prerequisites %}
+```
 
-#### Conditionals
+```
+{% block prerequisites %}
+{{ super() }}
+Some derived content that appears after the base content
+{% endblock prerequisites %}
+```
 
-We also use global variables to trigger conditionals:
+```
+{% block prerequisites %}
+Some derived content that appears before...
+{{ super() }}
+...AND after the base content
+{% endblock prerequisites %}
+```
 
-1. `templates/platformBase/centos-7.njk` uses the value of `includePPC` to determine whether to install [Advance Toolchain](https://github.com/advancetoolchain/advance-toolchain).
-2. `templates/platformBase/rhel-7-or-ol-7.njk` uses the value of `includeLOCAL` to trigger an option to create a local repository.
+```
+{% block prerequisites %}
+Some derived content that makes the base content look like it was SHOUTED
+{{ super() | upper }}
+{% endblock prerequisites %}
+```
 
-For small template systems, this system works well enough. But as the number of templates increases, this becomes harder to understand. In this case, we need to search through the template files to find out where the variables are being used. Fortunately, they are used just once, but it's not hard to imagine multiple (and exclusive) conditionals that are hard to read, modify and debug.
+#### Include other templates
 
-It's almost always better to use one of the other techniques than fall back on conditionals triggered by global variables.
+When there are pieces of text that need to be repeated, but don't have a place in the existing structure, consider using [the `include` tag](https://mozilla.github.io/nunjucks/templating.html#include). For instance, we might want to includes a disclaimer for platforms nearing the end of our support cycle. Rather than repeat that warning in each affected template, we could put the text in a new template that we include when needed.
 
-## Deployment rules
+In Nunjucks, `include` doesn't work like a pre-processor (inserting code before rendering). Instead, included templates are rendered and the results are inserted. That means you can't use `include` to override a block in the calling template and blocks defined in the calling template aren't visible to the included template. Included templates should be considered self-contained snippets.
 
-The deployment script (`npm run install-docs:deploy`) can use two sources of information on where to deploy the final MDX files:
+#### Macros
 
-1. A rule defined within the deployment script itself (`deploy.mjs`). 
-2. (RECOMMENDED) A frontmatter key (`deployPath`) written to the generated MDX file itself. Ex:
+Sometimes a chunk of text varies only by a few discrete variables. In that case, a [macro](https://mozilla.github.io/nunjucks/templating.html#macro) might be in order. We currently have two macros in `/templates/platformBase/_shared.njk`:
 
-   ```yaml
-   deployPath: mongo_data_adapter/5/installing/linux_ppc64le/mongo_sles_12.mdx
-   ```
+1. `centosRepositoryConfiguration(packageManager, epelRepo)`
+2. `centosInstallCommand(packageManager, packageName)`
+
+When that file is [imported](https://mozilla.github.io/nunjucks/templating.html#import) (not included) in another template, those macros can be used to insert the proper instructions for configuring the repository and installing the package. By using a few variables (`packageManager`, `epelRepo` and `packageName`), we can generate many different combinations of instructions from a single source. 
 
-   This is best defined within the product template heirarchy, where context variables can be used:
+Macros can be self-documenting and avoid [action-at-a-distance problems](https://en.wikipedia.org/wiki/Action_at_a_distance_(computer_programming)). If you call a macro, it's easier to know what you are going to get in the render.
 
-   ```
-   {% block frontmatter %}
-   deployPath: mongo_data_adapter/{{ product.version }}/installing/linux_{{platform.arch}}/mongo_{{deploy.map_platform[platform.name]}}.mdx
-   {% endblock frontmatter %}
-   ```
+#### Conditionals
 
-The 2nd technique above lends itself well to writing redirect rules that prevent broken links when deployment paths change:
+In a handful of situations it is useful to employ conditionals to include or modify output based on a test or comparison:
 
 ```
-{% block frontmatter %}
-redirects:
-  - mongo_data_adapter/{{ product.version }}/04_installing_the_mongo_data_adapter/{{deploy.expand_arch[platform.arch]}}/mongo_{{deploy.map_platform_old[platform.name]}}_{{platform.arch | replace("_64", "")}}.mdx
-{% endblock frontmatter %}
+{% if osArchitectures["ppc64le"].hasFamily("RHEL") %}
+...
+{% endif %}
+
+{% if platform.arch == "x86_64" %}
+...
+{% endif %}
 ```
 
-**Note:** the path generated from that pattern wouldn't normally be a valid redirect rule - it should start with `/` and *not* end with `.mdx`. For convenience, the deploy script will automatically rewrite redirects written in this form to the proper format, thus allowing you to just copy the old `deployPath` value to a redirect before modifying it. 
+Be wary of over-using these; prefer inheritance structures that push this content down to sufficiently-specific "leaf" templates instead. As a general rule, use conditionals only when the test is much, much simpler than the alternative inheritance structure, and be willing to abandon it when (over time) that simplicity is lost.
+
+In particular, avoid the trap of setting a flag in a leaf template and then checking it in a base template: this separates the context from the decision, making it extremely difficult to judge when the test has lost its value.

install_template/config.yaml

@@ -651,90 +651,90 @@ products:
     platforms:
       - name: CentOS 7
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: AlmaLinux 8 or Rocky Linux 8
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: RHEL 7 or OL 7
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: RHEL 8 or OL 8
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: RHEL 8
         arch: ppc64le
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Debian 11
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Debian 10
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Ubuntu 22.04
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Ubuntu 20.04
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Ubuntu 18.04
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 12
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 15
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 12
         arch: ppc64le
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 15
         arch: ppc64le
-        supported versions: [9]
+        supported versions: [8, 9]
   - name: Postgres Enterprise Manager agent
     platforms:
       - name: CentOS 7
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: AlmaLinux 8 or Rocky Linux 8
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: RHEL 7 or OL 7
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: RHEL 8 or OL 8
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: RHEL 8
         arch: ppc64le
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Debian 11
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Debian 10
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Ubuntu 22.04
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Ubuntu 20.04
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: Ubuntu 18.04
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 12
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 15
         arch: x86_64
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 12
         arch: ppc64le
-        supported versions: [9]
+        supported versions: [8, 9]
       - name: SLES 15
         arch: ppc64le
-        supported versions: [9]
+        supported versions: [8, 9]
   - name: Replication Server
     platforms:
       - name: CentOS 7