Merge pull request #4 from IBM/Release-Sep-2021

Tuning guide release 30th September 2021

CHANGELOG.md

@@ -2,6 +2,18 @@
 
 All notable changes to this project will be documented in this file
 
+## v21.9.0
+
+### Added
+
+* Foreign key constraints recommendations for production databases
+* Db2: Set `HIGHMEM` bufferpool size to `AUTOMATIC`
+* Transaction Isolation Level settings for JMS producer / consumer pod data sources
+
+### Changed
+
+* Updated IBM Documentation link to SPM V8.
+
 ## v21.7.1
 
 ### Changed

package.json

@@ -1,7 +1,7 @@
 {
   "name": "spm-performance-tuning",
   "private": true,
-  "version": "21.7.0",
+  "version": "21.9.0",
   "license": "Apache 2.0",
   "scripts": {
     "dev": "gatsby develop -H 0.0.0.0",
@@ -15,9 +15,9 @@
     "test:prefix": "gatsby build:prefix && gatsby serve:prefix"
   },
   "dependencies": {
-    "gatsby": "3.10.2",
-    "gatsby-cli": "3.10.0",
-    "gatsby-theme-carbon": "2.0.0",
+    "gatsby": "3.14.1",
+    "gatsby-cli": "3.14.0",
+    "gatsby-theme-carbon": "2.1.0",
     "markdownlint-cli": "0.28.1",
     "react": "17.0.2",
     "react-dom": "17.0.2"

src/data/nav-items.yaml

@@ -8,7 +8,7 @@
 - title: Common tuning tasks
   pages:
     - title: Database Configuration
-      path: /common_task/dbconfig/db2
+      path: /common_task/dbconfig/common
     - title: Tuning the application
       path: /common_task/spm-application
     - title: Tuning batch processes

src/gatsby-theme-carbon/components/LeftNav/ResourceLinks.js

@@ -8,7 +8,7 @@ const links = [
   },
   {
     title: 'IBM Documentation',
-    href: 'https://www.ibm.com/docs/en/spm/7.0.11?topic=product-overview',
+    href: 'https://www.ibm.com/docs/en/spm/8.0.0?topic=product-overview',
   },
   {
     title: 'Change Log',

src/pages/common_task/batch-processes.mdx

@@ -16,7 +16,7 @@ Use the following guidelines to tune the performance of batch processes.
 For streamed batch jobs, to offer good balance between scalability and error management,
 tune the chunk size for a batch job so that it takes between one to two minutes to process one chunk.
 
-For more information see the SPM [Tuning batch streaming performance](https://www.ibm.com/docs/en/spm/7.0.11?topic=application-tuning-batch-streaming-performance) topic.
+For more information see the SPM [Tuning batch streaming performance](https://www.ibm.com/docs/en/spm/8.0.0?topic=application-tuning-batch-streaming-performance) topic.
 
 ## Db2®
 

src/pages/common_task/dbconfig/common.mdx

@@ -0,0 +1,31 @@
+---
+title: Common DB configuration and tuning
+description: Common DB configuration and tuning
+tabs: ['Common','Db2','Oracle']
+---
+
+<AnchorLinks small>
+  <AnchorLink>Foreign key constraints</AnchorLink>
+</AnchorLinks>
+
+The following maintenance and database design guidelines apply to tuning across all supported databases.
+
+<InlineNotification kind="warning">
+
+Database tuning provided in this guidance is not comprehensive.
+
+However, the following values are a starting point for how to configure a performant database for IBM® Cúram Social Program Management.
+It is our recommendation that project teams verify their own tuning, and where necessary adapt and expand the configuration based on their own requirements prior to going in to production.
+</InlineNotification>
+
+## Foreign key constraints
+
+As outlined in the IBM Cúram Social Program Management documentation [Creating the Production Database in the Application](https://www.ibm.com/docs/en/spm/8.0.0?topic=setup-creating-production-database-in-application):
+
+* Omit foreign key constraints as they are not supported in a production environment
+
+ <InlineNotification>
+
+ **Note:** The Cúram Reference Application enforces referential integrity (RI) in the application and as such using database-enforced RI will degrade performance. It is strongly recommended that in a production environment you do not create foreign key constraints on your database.
+
+ </InlineNotification>

src/pages/common_task/dbconfig/db2.mdx

@@ -1,7 +1,7 @@
 ---
 title: Db2
 description: Db2
-tabs: ['Db2','Oracle']
+tabs: ['Common','Db2','Oracle']
 ---
 
 <AnchorLinks small>
@@ -35,6 +35,19 @@ In addition to physical database design done as part of the project, the followi
 * Enable compression on tables if it is wanted. Compression was tested by IBM with no issues.
 * Review the use of dedicated table spaces. Consider moving the largest tables (for example, `DYNAMICEVIDENCEDATAATTRIBUTE` and `DEPENDENCY`) to dedicated table spaces. However, from experience, with good I/O layouts, the only reason for dedicated table spaces is maintenance.
 * Review the use of partitioning. From experience, with good I/O layouts, the only reason for partitioning is maintenance (similar to dedicated table spaces).
+* After creating the database, see [Configuring the IBM Db2 database](https://www.ibm.com/docs/en/spm/8.0.0?topic=configuration-configuring-db2-database), consider setting the `highmem` buffer pool size to `AUTOMATIC` e.g.,
+
+ ```sql
+  db2 alter bufferpool HIGHMEM size AUTOMATIC
+ ```
+
+  see [ALTER BUFFERPOOL](https://www.ibm.com/support/producthub/db2/docs/content/SSEPGG_11.5.0/com.ibm.db2.luw.sql.ref.doc/doc/r0000885.html)
+
+ <InlineNotification kind="warning">
+
+  **Note:** The above will cause the Ant `configtest` target to fail, see [Testing the configuration](https://www.ibm.com/docs/en/spm/8.0.0?topic=configuration-testing). Therefore, run the `build cofigtest` procedure prior to altering the bufferpool.
+
+ </InlineNotification>
 
 ## Maintenance
 

src/pages/common_task/dbconfig/oracle.mdx

@@ -1,7 +1,7 @@
 ---
 title: Oracle
 description: Oracle
-tabs: ['Db2','Oracle']
+tabs: ['Common','Db2','Oracle']
 ---
 
 <AnchorLinks small>

src/pages/common_task/xmlservertuning.mdx

@@ -20,16 +20,16 @@ These are the basic high-level steps for tuning the XML server:
 
 Your main source of XML server performance information comes from its `stats/ThreadWorkerPool-*` files. For more information see these XML server topics in IBM Documentation:
 
-* [Statistics](https://www.ibm.com/docs/en/spm/7.0.11?topic=server-statistics) - outlines the information available in these files.
-* [Configuration options](https://www.ibm.com/docs/en/spm/7.0.11?topic=server-configuration-options) - identifies the configuration necessary to produce these files.
+* [Statistics](https://www.ibm.com/docs/en/spm/8.0.0?topic=server-statistics) - outlines the information available in these files.
+* [Configuration options](https://www.ibm.com/docs/en/spm/8.0.0?topic=server-configuration-options) - identifies the configuration necessary to produce these files.
 
 ## Identifying causes of poor performance
 
 If the the `stats/ThreadWorkerPool-*` files indicate a performance problem--for example: showing increasing processing times--these are some things to check:
 
 * Check the `$CURAMSDEJ/xmlserver/tmp/xmlserver.log` file for unexpected errors:
   * Look for any indications of socket timeouts or dropped connections, which could be indicators that the `<THREAD_POOL_QUEUE_SIZE>` setting is too small.
-    As per the [IBM Documentation](https://www.ibm.com/docs/en/spm/7.0.11?topic=server-configuration-options),
+    As per the [IBM Documentation](https://www.ibm.com/docs/en/spm/8.0.0?topic=server-configuration-options),
     that setting determines the point at which requests are held inside the XML server rather than in the TCP backlog queue.
     How the TCP backlog queue behaves when it is exhausted depends on the underlying operating system and in some cases may retry or timeout the connection.
     Some indicators of socket connection issues in `xmlserver.log` might include:
@@ -45,14 +45,14 @@ If the the `stats/ThreadWorkerPool-*` files indicate a performance problem--for
 
 These are the areas available within, or closely related to, the XML server for tuning:
 
-* These XML server settings in `xmlserverconfigconfig.xml`, which are described in the related [Configuration options](https://www.ibm.com/docs/en/spm/7.0.11?topic=server-configuration-options) topic IBM Documentation:
+* These XML server settings in `xmlserverconfigconfig.xml`, which are described in the related [Configuration options](https://www.ibm.com/docs/en/spm/8.0.0?topic=server-configuration-options) topic IBM Documentation:
   * `<THREAD_POOL_QUEUE_SIZE>`
   * `<THREAD_POOL_SIZE>`
   * `<USE_TEMPLATE_CACHE>`
   * `<SO_TIMEOUT>`
 * These Java JVM settings:
   * Thread stack size can be set when starting the XML server. For example: `ant -f xmlserver.xml -Djava.thread.stack.size="-Xss8m"`.
-    See the related [Overriding the Java thread stack size](https://www.ibm.com/docs/en/spm/7.0.11?topic=server-overriding-java-thread-stack-size) topic in IBM Documentation for more information.
+    See the related [Overriding the Java thread stack size](https://www.ibm.com/docs/en/spm/8.0.0?topic=server-overriding-java-thread-stack-size) topic in IBM Documentation for more information.
   * Java heap is set in `xmlserver.xml` as an Ant property, `java.maxmemory`, specifying the `maxmemory` attribute of the Ant `<java>` task.
     For example: `ant -f xmlserver.xml -Djava.maxmemory="1024m"`.
-* XML server load balancing is described in the [Load balancing and failover](https://www.ibm.com/docs/en/spm/7.0.11?topic=xml-load-balancing-failover) topic in IBM Documentation.
+* XML server load balancing is described in the [Load balancing and failover](https://www.ibm.com/docs/en/spm/8.0.0?topic=xml-load-balancing-failover) topic in IBM Documentation.

src/pages/openshift/introduction.mdx

@@ -44,7 +44,7 @@ The Application/EAR responsible for processing JMS initiated transactions is cal
 
 This resolution is part of a multi-faceted solution, with client HTTP initiated transactions and JMS initiated transactions isolated from each other, and the implementation of local interfaces.
 
-For more information please review the SPM [Transaction isolation](https://www.ibm.com/docs/en/spm/7.0.11?topic=architecture-transaction-isolation) topic.
+For more information please review the SPM [Transaction isolation](https://www.ibm.com/docs/en/spm/8.0.0?topic=architecture-transaction-isolation) topic.
 
 ## Helm Charts
 

src/pages/openshift/tune_jms_consumer/tune_jms_consumer.mdx

@@ -31,7 +31,7 @@ For more information see:
 The term **JMS consumer** refers to the functional structure of SPM applications (for example, `curam`, `rest`, etc.)
 as deployed in OpenShift such that the JMS reading and processing portion of the application is isolated from its client side functionality.
 That is, SPM clients produce JMS messages that perform asynchronous operations, which are consumed by  **JMS consumer** pods.
-For more information please review [Transaction isolation](https://www.ibm.com/docs/en/spm/7.0.11?topic=architecture-transaction-isolation).
+For more information please review [Transaction isolation](https://www.ibm.com/docs/en/spm/8.0.0?topic=architecture-transaction-isolation).
 
 Benefits of this separation include:
 
@@ -227,17 +227,26 @@ In that case, it is recommended to review the SQLStats from the JMX Stats, from
 Then, use a simple heuristic based on the distribution of SQL executions to find a smaller cache size that covers around
 90% of SQL executions from the application and gives a better balance between system performance and Java™ heap utilization.
 However, monitor prepared statement cache discards, system performance, and heap utilization, and adjust the cache size further as needed.
-
 </InlineNotification>
 
+### Transaction Isolation Level
+
+As outlined in the IBM Cúram Social Program Management documentation [Transaction control/Underlying design](https://www.ibm.com/docs/en/spm/8.0.0?topic=control-underlying-design),
+the `isolationLevel` for the application producer and consumer pod data sources must be set in the Helm configmaps
+[configmap-dbconn.yaml](https://github.com/IBM/spm-kubernetes/blob/main/helm-charts/apps/templates/configmaps/configmap-dbconn.yaml) and
+[configmap-sessions.yaml](https://github.com/IBM/spm-kubernetes/blob/main/helm-charts/apps/templates/configmaps/configmap-sessions.yaml) to:
+
+* `isolationLevel="TRANSACTION_REPEATABLE_READ"` for Db2 (default)
+* `isolationLevel="TRANSACTION_READ_COMMITTED"` for Oracle
+
 ### JDBC configuration tuning
 
 The WebSphere Liberty [JDBC configuration](https://www.ibm.com/docs/en/was-liberty/base?topic=configuration-datasource) is tunable for each of these database definitions:
 
 |Database definitions | Description |
 |:- | :- |
 |`curamdb`|Used by SPM applications|
-|`curamtimerdb` | Used by the [SPM timer infrastructure](https://www.ibm.com/docs/en/spm/7.0.11?topic=behavior-cram-timer) |
+|`curamtimerdb` | Used by the [SPM timer infrastructure](https://www.ibm.com/docs/en/spm/8.0.0?topic=behavior-cram-timer) |
 |`curamsessdb`| Used for WebSphere Liberty's [HTTP session replication](/openshift/tune_jms_producer/tune_jms_producer#liberty-http-session-replication)|
 
 The following yaml keys are provided for tuning the WebSphere Liberty JDBC configuration:
@@ -248,9 +257,9 @@ The following yaml keys are provided for tuning the WebSphere Liberty JDBC confi
   * `curamsessdb_maxPoolSize`= `( max_threads + 1 )`
 * _numConnectionsPerThreadLocal_ - Number of connections to the database to be cached for each thread:
   <InlineNotification>
-  
+
   **Note:** For SPM _numConnectionsPerThreadLocal_ should always be set to `2`. `1` connection for the `keyserver` and `1` connection for the SPM database
-  
+
   </InlineNotification>
 
   * `curamdb_numConnectionsPerThreadLocal` = `2`

src/pages/openshift/tune_jms_producer/tune_jms_producer.mdx

@@ -32,7 +32,7 @@ For more information see:
 The term **JMS producer** refers to the functional structure of SPM applications (for example, `curam`, `rest`, etc.)
 as deployed in OpenShift such that the client portion of the application is isolated from its server side functionality.
 That is, SPM clients produce JMS messages that perform asynchronous operations, which are consumed by corresponding **JMS consumer** pods.
-For more information please review [Transaction isolation](https://www.ibm.com/docs/en/spm/7.0.11?topic=architecture-transaction-isolation).
+For more information please review [Transaction isolation](https://www.ibm.com/docs/en/spm/8.0.0?topic=architecture-transaction-isolation).
 
 Benefits of this separation include:
 
@@ -74,7 +74,7 @@ The OpenShift and Kubernetes documentation provide more information about replic
 
 * OpenShift: [Replication controllers](https://docs.openshift.com/container-platform/4.7/applications/deployments/what-deployments-are.html#deployments-replicationcontrollers_what-deployments-are)
 * Kubernetes: [ReplicaSet](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/)
-  
+
 The number of replicas in a deployment can be specified globally via `apps.replicaCount` or via `global.apps.config.<applicationID>.replicaCount`: where `<applicationID>` is replaced by the lower-case EAR file basename (e.g., `curam`, `citizenportal`, `rest`, etc).
 
 The default is a single replica.
@@ -227,17 +227,26 @@ In that case, it is recommended to review the SQLStats from the JMX Stats, from
 Then, use a simple heuristic based on the distribution of SQL executions to find a smaller cache size that covers around
 90% of SQL executions from the application and gives a better balance between system performance and Java™ heap utilization.
 However, monitor prepared statement cache discards, system performance, and heap utilization, and adjust the cache size further as needed.
-
 </InlineNotification>
 
+### Transaction Isolation Level
+
+As outlined in the IBM Cúram Social Program Management documentation [Transaction control/Underlying design](https://www.ibm.com/docs/en/spm/8.0.0?topic=control-underlying-design),
+the `isolationLevel` for the application producer and consumer pod data sources must be set in the Helm configmaps
+[configmap-dbconn.yaml](https://github.com/IBM/spm-kubernetes/blob/main/helm-charts/apps/templates/configmaps/configmap-dbconn.yaml) and
+[configmap-sessions.yaml](https://github.com/IBM/spm-kubernetes/blob/main/helm-charts/apps/templates/configmaps/configmap-sessions.yaml) to:
+
+* `isolationLevel="TRANSACTION_REPEATABLE_READ"` for Db2 (default)
+* `isolationLevel="TRANSACTION_READ_COMMITTED"` for Oracle
+
 ### JDBC configuration tuning
 
 The WebSphere Liberty [JDBC configuration](https://www.ibm.com/docs/en/was-liberty/base?topic=configuration-datasource) is tunable for each of these database definitions:
 
 |Database definitions | Description |
 |:- | :- |
 |`curamdb`|Used by SPM applications|
-|`curamtimerdb` | Used by the [SPM timer infrastructure](https://www.ibm.com/docs/en/spm/7.0.11?topic=behavior-cram-timer) |
+|`curamtimerdb` | Used by the [SPM timer infrastructure](https://www.ibm.com/docs/en/spm/8.0.0?topic=behavior-cram-timer) |
 |`curamsessdb`| Used for WebSphere Liberty's [HTTP session replication](/openshift/tune_jms_producer/tune_jms_producer#liberty-http-session-replication)|
 
 The following yaml keys are provided for tuning the WebSphere Liberty JDBC configuration:
@@ -249,9 +258,9 @@ The following yaml keys are provided for tuning the WebSphere Liberty JDBC confi
 * _numConnectionsPerThreadLocal_ - Number of connections to the database to be cached for each thread; Helm chart default: `2`
 
   <InlineNotification>
-  
+
   **Note:** For SPM _numConnectionsPerThreadLocal_ should always be set to `2`. `1` connection for the `keyserver` and `1` connection for the SPM database
-  
+
   </InlineNotification>
 
   * `curamdb_numConnectionsPerThreadLocal` = `2`

src/pages/overview/Assumptions.mdx

@@ -12,7 +12,7 @@ To ensure you are using supported software with the correct versions, refer to t
 [IBM Cúram Social Program Management Supported Prerequisites](https://www.ibm.com/support/pages/ibm-cúram-social-program-management-supported-prerequisites). In addition to the product prerequisites, consult the system requirement documents for each of the required/optional software components.
 
 For more information about IBM Cúram Social Program Management, see
-[IBM Cúram Social Program Management](https://www.ibm.com/docs/en/spm/7.0.11).
+[IBM Cúram Social Program Management](https://www.ibm.com/docs/en/spm/8.0.0).
 
 ## System configuration