Merge branch 'fix/record-aditional-info' of https://github.com/IBM/co…

…mpliance-trestle into fix/record-aditional-info

.gitignore

@@ -49,3 +49,5 @@ site
 tmp_bin_test
 .mypy_cache
 
+/venv.trestle/
+

CHANGELOG.md

@@ -3,6 +3,19 @@
 
 <!--next-version-placeholder-->
 
+## v2.4.0 (2023-10-26)
+### Feature
+* Adding validate template type to author docs command ([#1465](https://github.com/IBM/compliance-trestle/issues/1465)) ([`5289f51`](https://github.com/IBM/compliance-trestle/commit/5289f516e9710361e0dc391cefd979b5e2d46ed0))
+
+### Fix
+* Upgrade urllib version to fix vulnerability ([#1472](https://github.com/IBM/compliance-trestle/issues/1472)) ([`e9d4175`](https://github.com/IBM/compliance-trestle/commit/e9d4175fabd015ada6e8cdd26450c454ad83fbe8))
+* Improve bad property error message by including csv row number ([#1466](https://github.com/IBM/compliance-trestle/issues/1466)) ([`ab97beb`](https://github.com/IBM/compliance-trestle/commit/ab97beb2367112e9e68fb258af6dc2c75d909279))
+* Cryptic error message + feat: # indicates comment column ([#1459](https://github.com/IBM/compliance-trestle/issues/1459)) ([`45eda01`](https://github.com/IBM/compliance-trestle/commit/45eda015751d2f9121e14fe609b14acd890440fd))
+* Update community call information ([#1444](https://github.com/IBM/compliance-trestle/issues/1444)) ([`5a03d06`](https://github.com/IBM/compliance-trestle/commit/5a03d06783fff8db4bf402b1e21acb99fd485454))
+
+### Documentation
+* Updating vtt documentation for trestle author docs ([#1471](https://github.com/IBM/compliance-trestle/issues/1471)) ([`63d436a`](https://github.com/IBM/compliance-trestle/commit/63d436a7752e50ef0c52c93cbab36f4c1fc16748))
+
 ## v2.3.1 (2023-09-20)
 ### Fix
 * Improper indentation structure validation not working ([#1451](https://github.com/IBM/compliance-trestle/issues/1451)) ([`2e6936a`](https://github.com/IBM/compliance-trestle/commit/2e6936a4705251fd8412fd67163a7cd9d801a4b8))

docs/api_reference/trestle.core.crm.bycomp_interface.md

@@ -0,0 +1,2 @@
+::: trestle.core.crm.bycomp_interface
+handler: python

docs/api_reference/trestle.core.crm.export_reader.md

@@ -0,0 +1,2 @@
+::: trestle.core.crm.export_reader
+handler: python

docs/api_reference/trestle.core.crm.export_writer.md

@@ -0,0 +1,2 @@
+::: trestle.core.crm.export_writer
+handler: python

docs/api_reference/trestle.core.crm.leveraged_statements.md

@@ -0,0 +1,2 @@
+::: trestle.core.crm.leveraged_statements
+handler: python

docs/api_reference/trestle.core.crm.ssp_inheritance_api.md

@@ -0,0 +1,2 @@
+::: trestle.core.crm.ssp_inheritance_api
+handler: python

docs/trestle_author.md

@@ -296,6 +296,61 @@ Running `trestle author docs validate -tn docs_task -gh="Governed section"` will
 - If `--template-version 1.0.0` (`-tv`) is passed the header field `x-trestle-template-version` will be ignored and document will be forcefully validated against template of version `1.0.0`.
   Use this for testing purposes _only_ when you need to validate the document against a specific template. By default the template version will be determined based on `x-trestle-template-version` in the document.
 
+### Validating the documents against different templates
+
+Validation against multiple templates can be done when there is a scenario where you have multiple templates that will have multiple instances. In this particular case you can have a 1:1 relationship between the template and the instance document you are creating out of it, so validation can be performed based on template type and version of that particular template defined in headers.
+
+For that to happen you will need to provide your template with the following parameter at the yaml header level, matching the type of template to be implemented so that the validation can occur:
+
+> x-trestle-template-type: insert_template_type_here
+
+Please, take into consideration that for the validation to happen you will also need to provide each instance document in the task folder a field called `x-trestle-template-type: insert_template_type_here` in the yaml header matching with the template name.
+
+```yaml
+---
+authors: tmp
+owner: tmp
+valid:
+  from: null
+  to: null
+x-trestle-template-type: insert_template_type_here
+---
+```
+
+With that, you will be able to create more than 1 instance document per template and give the instance the desired name.
+
+For instance, let´s consider the next folder structure:
+
+```text
+trestle_root
+┣ .trestle
+┃ ┣ author
+┃ ┃ ┣ my_task_2
+┃ ┃ ┃ ┣ 0.0.1
+┃ ┃ ┃ ┃ ┣ a_template.md
+┃ ┃ ┃ ┃ ┣ another_template.md
+┃ ┃ ┃ ┃ ┗ arhitecture.drawio
+┃ ┗ config.ini
+
+trestle_root
+ ┣ .trestle
+ ┣ my_task_2
+ ┃ ┣ sample_folder_0
+ ┃ ┃ ┣ a_template_1.md
+ ┃ ┃ ┣ a_template_2.md
+ ┃ ┃ ┣ arhitecture_1.drawio
+ ┃ ┃ ┗ another_template_123.md
+
+```
+
+If you noticed, names are no longer needed to match with exact template names, and that´s because validation will run through `x-trestle-template-type` field defined at the instance header, not through the name.
+
+To validate the documents against their respective templates using `x-trestle-template-type`, run:
+
+> trestle author docs validate -tn my_task_name -vtt
+
+Now, `-vtt` stands for validate template type. Validate template type option will provide you the ability to have more than 1 instance per template validated.
+
 </details>
 
 <details markdown>

docs/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring.md

@@ -1050,6 +1050,131 @@ As with all the `assemble` tools, you may optionally specify a `--name` for a co
 
 If you do not specify component-defintions during assembly, the markdown should not refer to any components other than `This System`.  Thus you may first generate markdown with `ssp-generate` and no component-definitions specified - and then you may assemble that ssp with `ssp-assemble` and no component-definitions specified - but only if there are no components other than `This System` referenced in the markdown.  You may add new component implementation details to the markdown later, but any new components must be defined in a component-defintion file, and that file must be specified when `ssp-assemble` is run.
 
+## Inheritance view
+
+The inheritance view is generated by setting the `--leveraged-ssp` flag with `trestle author ssp-generate`. It contains information relating to exported information such as inherited capabilities and customer responsibilities that can be used to populate the inheritance information in the assembled SSP. When used, a directory named "inheritance" is created within the markdown directory. This directory serves as a designated space for  mapping inherited capabilities and responsibilities onto components in the assemble SSP and authoring satisfied statements for responsibilities.
+
+Example usage for creation of the markdown:
+
+`trestle author ssp-generate --profile my_prof --compdefs "compdef_a,compdef_b" --yaml /my_yaml_dir/header.yaml --leveraged-ssp my_provider_ssp --output my_ssp`
+
+In this example the leveraged ssp has previously been imported into the trestle directory, but it can be fetched from remote location.
+
+The generated markdown output with the inheritance view will be placed in the trestle subdirectory `my_ssp/inheritance` with a subdirectory for each component in the leveraged ssp with directories separated by control and statement id below.
+
+An example of this directory structure is below.
+
+```text
+.
+├── Application
+│   ├── ac-1_stmt.a
+│   │   └── 11111111-0000-4000-9009-001001002006.md
+│   ├── ac-2.1
+│   │   └── 11111111-0000-4000-9009-001001002004.md
+│   └── ac-2_stmt.a
+│       └── 11111111-0000-4000-9009-002001002001_11111111-0000-4000-9009-002001002002.md
+└── This System
+    ├── ac-1_stmt.a
+    │   └── 11111111-0000-4000-9009-001002002001.md
+    ├── ac-2.1
+    │   └── 11111111-0000-4000-9009-001001002001.md
+    └── ac-2_stmt.a
+        └── 11111111-0000-4000-9009-002001001001.md
+```
+
+The leveraged components are used as the top level directory to allow any non-leveraged components to be easily skipped or removed. Each markdown file is named in accordance with the uuid of the exported statement to ensure statement description updates can be applied.
+
+There are three types of markdown files that can be generated from this process.
+
+The examples below demonstrate these types:
+
+<details markdown>
+
+<summary>Example of inheritance provided only markdown after ssp-generate</summary>
+
+```markdown
+---
+x-trestle-statement:
+  # Add or modify leveraged SSP Statements here.
+  provided-uuid: 11111111-0000-4000-9009-001002002001
+x-trestle-leveraging-comp:
+  # Leveraged statements can be optionally associated with components in this system.
+  # Associate leveraged statements to Components of this system here:
+  - name: REPLACE_ME
+---
+
+# Provided Statement Description
+
+Customer_appropriate description of what may be inherited.
+```
+
+</details>
+
+<details markdown>
+
+<summary>Example of inheritance customer responsibility only markdown after ssp-generate</summary>
+
+```markdown
+---
+x-trestle-statement:
+  # Add or modify leveraged SSP Statements here.
+  responsibility-uuid: 11111111-0000-4000-9009-002001001001
+x-trestle-leveraging-comp:
+  # Leveraged statements can be optionally associated with components in this system.
+  # Associate leveraged statements to Components of this system here:
+  - name: REPLACE_ME
+---
+
+# Responsibility Statement Description
+
+Leveraging system's responsibilities with respect to inheriting this capability.
+
+# Satisfied Statement Description
+
+<!-- Use this section to explain how the inherited responsibility is being satisfied. -->
+```
+
+</details>
+
+<details markdown>
+
+<summary>Example of inheritance shared responsibility markdown after ssp-generate</summary>
+
+```markdown
+---
+x-trestle-statement:
+  # Add or modify leveraged SSP Statements here.
+  provided-uuid: 11111111-0000-4000-9009-002001002001
+  responsibility-uuid: 11111111-0000-4000-9009-002001002002
+x-trestle-leveraging-comp:
+  # Leveraged statements can be optionally associated with components in this system.
+  # Associate leveraged statements to Components of this system here:
+  - name: REPLACE_ME
+---
+
+# Provided Statement Description
+
+Consumer-appropriate description of what may be inherited.
+
+# Responsibility Statement Description
+
+Leveraging system's responsibilities with respect to inheriting this capability.
+
+# Satisfied Statement Description
+
+<!-- Use this section to explain how the inherited responsibility is being satisfied. -->
+```
+
+</details>
+
+Some additional information and tips about this markdown are below:
+
+- Do not change the statement UUIDs in the YAML header. This is used in the assembled JSON to link the statements in the leveraged SSP to the components in the leveraging SSP.
+- When mapping components in the YAML header, use the component title. If you do not wish to map a component to a particular inherited capability or responsibility, just leave the file as is. Files without mapped components or that contain the default "REPLACE ME" entry will be skipped.
+- If the file exists, just the editable information will be preserved when regenerating existing inheritance view markdown. This includes the information under `Satisfied Statement Description` and the mapped components in the YAML header.
+
+After manually editing the inheritance view markdown, the `trestle author ssp-assemble` command can be run without modifications for the inheritance view use case. During assemble, the inheritance directory is detected and the information will be assembled into the SSP. The by-component assemblies will be updated or added under existing implemented requirement or statement sections with the information from the markdown.
+
 </details>
 
 <details markdown>

mkdocs.yml

@@ -113,6 +113,12 @@ nav:
       - control_interface: api_reference/trestle.core.control_interface.md
       - control_reader: api_reference/trestle.core.control_reader.md
       - control_writer: api_reference/trestle.core.control_writer.md
+      - crm:
+        - bycomp_interface: api_reference/trestle.core.crm.bycomp_interface.md
+        - export_reader: api_reference/trestle.core.crm.export_reader.md
+        - export_writer: api_reference/trestle.core.crm.export_writer.md
+        - leveraged_statements: api_reference/trestle.core.crm.leveraged_statements.md
+        - ssp_inheritance_api: api_reference/trestle.core.crm.ssp_inheritance_api.md
       - docs_control_writer: api_reference/trestle.core.docs_control_writer.md
       - draw_io: api_reference/trestle.core.draw_io.md
       - duplicates_validator: api_reference/trestle.core.duplicates_validator.md

setup.cfg

@@ -64,6 +64,7 @@ dev =
     pytest-xdist
     pre-commit>=2.4.0
     setuptools
+    urllib3==1.26.17
     wheel
     yapf
     python-semantic-release==7.33.2

tests/data/author/docs/a_folder/folder_level_1/folder_level_2-1/a_v1.md

@@ -0,0 +1,14 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from: null
+  to: null
+x-trestle-template-type: a
+x-trestle-template-version: 1.0.0
+---
+# System architecture
+
+## Overview
+
+## Security model

tests/data/author/docs/a_folder/folder_level_1/folder_level_2-1/a_v2.md

@@ -0,0 +1,14 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from: null
+  to: null
+x-trestle-template-type: a
+x-trestle-template-version: 2.0.0
+---
+# System architecture
+
+## Overview
+
+## Security model

tests/data/author/docs/a_folder/folder_level_1/folder_level_2-1/b.md

@@ -0,0 +1,17 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from:
+  to:
+x-trestle-template-type: b
+x-trestle-template-version: 1.1.1
+---
+
+# Network architectures
+
+## External interconnections
+
+## Corporate interconnections
+
+## Out of scope interconnections

tests/data/author/docs/a_folder/folder_level_1/folder_level_2-2/a_v3.md

@@ -0,0 +1,14 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from: null
+  to: null
+x-trestle-template-type: a
+x-trestle-template-version: 3.0.0
+---
+# System architecture
+
+## Overview
+
+## Security model

tests/data/author/docs/a_folder/folder_level_1/folder_level_2-2/b.md

@@ -0,0 +1,17 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from:
+  to:
+x-trestle-template-type: b
+x-trestle-template-version: 1.0.0
+---
+
+# Network architecture
+
+## External interconnections
+
+## Corporate interconnections
+
+## Out of scope interconnections

tests/data/author/docs/a_folder_template/1.0.0/a.md

@@ -0,0 +1,14 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from: null
+  to: null
+x-trestle-template-type: a
+x-trestle-template-version: 1.0.0
+---
+# System architecture
+
+## Overview
+
+## Security model

tests/data/author/docs/a_folder_template/1.0.0/b.md

@@ -0,0 +1,17 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from:
+  to:
+x-trestle-template-type: b
+x-trestle-template-version: 1.0.0
+---
+
+# Network architecture
+
+## External interconnections
+
+## Corporate interconnections
+
+## Out of scope interconnections

tests/data/author/docs/a_folder_template/1.1.1/b.md

@@ -0,0 +1,17 @@
+---
+authors: tmp
+owner: tmp
+valid:
+  from:
+  to:
+x-trestle-template-type: b
+x-trestle-template-version: 1.1.1
+---
+
+# Network architectures
+
+## External interconnections
+
+## Corporate interconnections
+
+## Out of scope interconnections