Documentation for Participant Config Includes (#21)

SilKit/source/config/ParticipantConfiguration.schema.json

@@ -709,6 +709,28 @@
       },
       "additionalProperties": false
     },
+    "Includes": {
+      "type": "object",
+      "description": "Optional configuration to include other participant configurations files",
+      "properties": {
+        "SearchPathHints": {
+          "type": "array",
+          "items": {
+            "type": "string",
+            "description": "A filesystem path to additionally search for files to be included",
+            "examples": [ "/urs/etc/sil-kit-configs/", "C:\\Temp\\sil-kit-configs\\" ]
+          }
+        },
+        "Files": {
+          "type": "array",
+          "items": {
+            "type": "string",
+            "description": "Participant configuration files to be included",
+            "examples": [ "relative/path/to/included.silkit.yaml", "logging.silkit.yaml" ]
+          }
+        }
+      }
+    },
     "Middleware": {
       "type": "object",
       "description": "Optional configuration of the SIL Kit middleware",
@@ -740,9 +762,9 @@
           "default": true
         },
         "ConnectTimeoutSeconds": {
-            "type": "number",
-            "minimum": 0.0,
-            "default": 5.0
+          "type": "number",
+          "minimum": 0.0,
+          "default": 5.0
         }
       },
       "additionalProperties": false

docs/configuration/configuration.rst

@@ -30,14 +30,28 @@ However, the following scenarios are conceivable in which some flexibility is us
 - Developers or users want to temporarily enable debugging features without the need or the ability to recompile the sources, for example logging, tracing, and health checking.
 
 To cover these scenarios, |ProductName| participants can be modified by *participant configuration files*. 
-This is done by creating a participant configuration object from a given file and passing the object as an argument when a participant is created:
+This is done by creating a participant configuration object from a given file in YAML or JSON format and passing the object as an argument when a participant is created:
 
 .. code-block:: c++
 
   auto participantConfiguration = SilKit::Config::ParticipantConfigurationFromFile(participantConfigurationFilename);
   auto participant = SilKit::CreateParticipant(participantConfiguration, participantName, registryUri);
 
-Participant configuration files allow to override a subset of parameters which are configurable via the |ProductName| API.
+Alternatively, the configuration object can be created directly from string:
+
+.. code-block:: c++
+
+  const std::string participantConfigText = R"(
+  Description: My participant configuration
+  Logging:
+      Sinks:
+      - Type: Stdout
+        Level: Info
+  )";
+  auto participantConfiguration = SilKit::Config::ParticipantConfigurationFromString(participantConfigText);
+  auto participant = SilKit::CreateParticipant(participantConfiguration, participantName, registryUri);
+
+Participant configurations allow to override a subset of parameters which are configurable via the |ProductName| API.
 Configuration parameters that are specified within the participant configuration override corresponding programmatically defined values. 
 For example, the ``ParticipantName`` field overrides the participant name that is provided through the API when a participant is created, namely :cpp:func:`CreateParticipant(..., const std::string& participantName, ...)<SilKit::CreateParticipant()>`. 
 This gives users the ability to run a simulation with multiple instances of a participant from a single implementation.
@@ -55,7 +69,7 @@ This gives users the ability to run a simulation with multiple instances of a pa
 
 A participant configuration file is written in YAML syntax according to a specified schema. 
 It starts with the ``SchemaVersion``, the ``Description`` for the configuration and the ``ParticipantName``. 
-This is followed by further sections for ``Middleware``, ``Logging``, ``HealthCheck``, ``Tracing``, ``Extentions`` and sections for the different services of the |ProductName|.
+This is followed by further sections for ``Includes``, ``Middleware``, ``Logging``, ``HealthCheck``, ``Tracing``, ``Extentions`` and sections for the different services of the |ProductName|.
 The outline of a participant configuration file is as follows:
 
 .. code-block:: yaml
@@ -64,6 +78,8 @@ The outline of a participant configuration file is as follows:
     SchemaVersion: 1
     Description: Sample configuration with all root nodes
     ParticipantName: Participant1
+    Includes: 
+      ...
     Middleware: 
       ...
     Logging: 
@@ -72,7 +88,7 @@ The outline of a participant configuration file is as follows:
       ...
     Tracing:
       ...
-    Extensions:
+    Extensions: 
       ...
     CanControllers:
       - ...
@@ -103,11 +119,11 @@ Overview
    * - Setting Name
      - Description
 
-   * - $schema
+   * - ``"$schema"``
      - File path to the participant configuration schema. 
        The ``ParticipantConfiguration.schema.json`` is part of the |ProductName| sources and can be found in the folder ``./SilKit/source/config/``.
 
-   * - schemaVersion
+   * - ``SchemaVersion``
      - The version of the used participant configuration schema. Current version is 1.
 
    * - ``Description``
@@ -117,6 +133,9 @@ Overview
      - The name of the simulation participant that joins the |ProductName| simulation. 
        Overrides a programmatically defined participant name.
 
+   * - :ref:`Includes<sec:cfg-participant-includes>`
+     - This can be used to include other participant configuration files.
+
    * - :ref:`Logging<sec:cfg-participant-logging>`
      - The logger configuration for this participant.
 
@@ -163,6 +182,7 @@ Configuration Options
    :maxdepth: 1
 
    services-configuration
+   includes-configuration
    logging-configuration
    healthcheck-configuration
    tracing-configuration

docs/configuration/includes-configuration.rst

@@ -0,0 +1,168 @@
+.. _sec:cfg-participant-includes:
+
+======================
+Includes Configuration
+======================
+
+.. |ProductName| replace:: SIL Kit
+
+.. |ParticipantConfigurationFromFile| replace:: :cpp:func:`ParticipantConfigurationFromFile()<SilKit::Config::ParticipantConfigurationFromFile()>`
+.. |ParticipantConfigurationFromString| replace:: :cpp:func:`ParticipantConfigurationFromString()<SilKit::Config::ParticipantConfigurationFromString()>`
+
+.. contents:: :local:
+   :depth: 3
+
+Overview
+========
+
+The ``Includes`` section allows to reference other participant configuration files.
+This can be used to share common parts of a participant configuration or to include dynamically generated configurations into a static configuration base.
+
+.. code-block:: yaml
+
+   Includes:
+     Files:
+       - LoggingIncludes.silkit.yaml
+       - generated/MiddlewareInclude.silkit.yaml
+
+To deal with different locations of the included files depending on the execution scenario, a list of search paths pointing to the locations of the included files can be specified in the ``SearchPathHints``:
+
+.. code-block:: yaml
+
+   Includes:
+     SearchPathHints:
+       - ConfigSnippets/Logging     # relative paths are valid (same as ./ConfigSnippets/Logging/) 
+       - /urs/etc/sil-kit-configs/  # absolute paths are valid
+       - C:\Temp\ConfigSnippets     # for Windows, backslash as path separator is valid
+       - C:\Temp\Path with spaces\  # for Windows, paths with spaces are valid
+     Files:
+       - LoggingIncludes.silkit.yaml
+       - generated/MiddlewareInclude.silkit.yaml
+
+When using |ParticipantConfigurationFromFile|, the current working directory and the path of the including participant configuration file are considered as additional search paths.
+When using |ParticipantConfigurationFromString|, the default search path is the current working directory.
+
+Merge rules
+===========
+
+The included configurations are merged according to the following rules:
+
+* All properties of the ``Middleware`` section can only be defined once, otherwise a ``SilKit::ConfigurationError`` occurs:
+
+  .. code-block:: yaml
+
+     # root.silkit.yaml 
+     Includes:
+        Files:
+           - included.silkit.yaml
+     Middleware:
+        RegistryUri: silkit://localhost:8500 
+  
+  .. code-block:: yaml
+
+     # included.silkit.yaml 
+     Middleware:
+        RegistryUri: silkit://0.0.0.0:8501 # Already specified in root.silkit.yaml, ConfigurationError!
+
+* Multiple inclusions of the same file are automatically prevented.
+  This also applies for nested includes of the same file:
+
+  .. code-block:: yaml
+
+     # root.silkit.yaml 
+     Includes:
+        Files:
+           - included_1.silkit.yaml
+           - included_2.silkit.yaml
+  
+  .. code-block:: yaml
+
+     # included_1.silkit.yaml 
+     Includes:
+        Files:
+           - included_2.silkit.yaml # Ignored (already appeared in root.silkit.yaml)
+
+* List items of top-level properties (e.g. ``CanControllers``, ``DataPublishers``) are combined:
+
+  .. code-block:: yaml
+
+     # root.silkit.yaml 
+     Includes:
+        Files:
+           - included.silkit.yaml
+     DataPublishers:
+     - Name: DataPublisher1 # Will be used
+       Topic: SomeTopic
+  
+  .. code-block:: yaml
+
+     # included.silkit.yaml 
+     DataPublishers:
+     - Name: DataPublisher2 # Will also be used
+       Topic: SomeTopic
+
+* *Named* items with the same name cannot be merged in a meaningful way and result in a ``SilKit::ConfigurationError``:
+
+  .. code-block:: yaml
+
+     # root.silkit.yaml 
+     Includes:
+        Files:
+           - included.silkit.yaml
+     CanControllers:
+     - Name: CAN1 # Name "CAN1" set here
+       Network: CAN1
+  
+  .. code-block:: yaml
+
+     # included.silkit.yaml 
+     CanControllers:
+     - Name: CAN1  # SilKit::ConfigurationError: Conflicting name "CAN1"
+       Network: CAN2
+
+* *Named* items where all properties match (i.e., duplicates of *named* items) are permitted.
+
+* The list items of ``Sinks`` in the ``Logging`` section are merged as follows:
+
+  * Only a single sink of type ``Type: Stdout`` can be defined, otherwise a ``SilKit::ConfigurationError`` occurs.
+    The same applies to the sink type ``Type: Remote``.
+  * Sinks of ``Type: File`` are combined. 
+    However their ``LogName`` must be unique, otherwise a ``SilKit::ConfigurationError`` occurs. 
+
+* List items of the ``SearchPathHints`` in the sections ``Includes`` or ``Extensions`` are merged and all entries are retained.
+  Possible duplicates here are uncritical.
+
+* All properties of the ``HealthCheck`` section can only be defined once, otherwise a ``SilKit::ConfigurationError`` occurs.
+
+Dynamic port generation
+=======================
+
+An important use-case is to include a configuration with a dynamically generated ``RegistryUri`` of the ``Middleware`` section:
+In a CI environment, it is unfavorable to setup a static port in the ``listen-uri`` of the  :ref:`SIL Kit registry<sec:util-registry>`.
+Instead, a port ``0`` advises the |ProductName| registry to let the operating system choose a random free port.
+This URI then has to be used by the participants in the Middleware property ``RegistryUri``.
+For this purpose, the ``--generate-configuration`` CLI parameter of the |ProductName| registry creates a participant configuration file containing the dynamic ``RegistryUri`` in the Middleware section.
+By referencing this generated participant configuration in the ``Includes`` section, the static part of the configuration (e.g., network names, logging) can be combined with the dynamic ``RegistryUri``.
+
+Configuration
+=============
+
+.. code-block:: yaml
+
+    Includes:
+       SearchPathHints:
+          - ./ConfigSnippets/Generated/
+       Files:
+          - generated-uri.silkit.yaml
+          - ../common-logging.silkit.yaml
+
+.. list-table:: Includes Configuration
+   :widths: 15 85
+   :header-rows: 1
+
+   * - Property Name
+     - Description
+   * - SearchPathHints
+     - A list of paths that are used to search for included configuration files.
+   * - Files
+     - A list of configuration files to be included.