Errors evaluating expressions with item of sequence but work in messages of metaschema-meta-constraints

[aj-stein-gsa]

Describe the bug

When trying to use metaschema-java through oscal-cli for FedRAMP testing, I encounter errors I cannot debug where trying to compare values that I can debug in an embedded <message/> for a given constraint, but using that same expression in a test fails.

Who is the bug affecting

Given OSCAL v1.1.2 models and the constraints such as those in this branch (fedramp-allowed-values.xml; fedramp-external-constraints), I add the following constraint to the fedramp-external-constraints file.

I could not get this to original version below to work:

 <context>
        <metapath target="/system-security-plan/metadata"/>
        <constraints>
            <let var="person-only-party-uuids" expression="party[@type='person']/@uuid"/>
            <expect id="responsible-party-is-person" target="responsible-party/party-uuid" test=". = $person-only-party-uuids[1]" level="ERROR">
                <message>{.} is not {$person-only-party-uuids[1]}</message>
            </expect>
            <remarks>
                <p>For roles 'system-owner', 'authorizing-official', 'authorizing-official-poc', 'system-poc-management', 'system-poc-technical', 'system-poc-other', and 'information-system-security-officer', the responsible-role party must be a party of type 'person'.</p>
            </remarks>
        </constraints>
    </context>

I get the following Metapath syntax errors.

oscal-cli validate '/home/me/computer/fedramp-automation/src/validations/constraints/content/ssp-all-VALID.xml' -c '/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-allowed-values.xml' -c '/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-constraints.xml'
Loading 'file:/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-allowed-values.xml'
Loading 'file:/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-constraints.xml'
The path '//part' is not properly contextualized using '.'. Using './/part' instead.
The path '//prop' is not properly contextualized using '.'. Using './/prop' instead.
The path '//(control|group|part)' is not properly contextualized using '.'. Using './/(control|group|part)' instead.
The path '//control' is not properly contextualized using '.'. Using './/control' instead.
The path '//param' is not properly contextualized using '.'. Using './/param' instead.
The path '//group' is not properly contextualized using '.'. Using './/group' instead.
Validating 'file:/home/me/computer/fedramp-automation/src/validations/constraints/content/ssp-all-VALID.xml' as XML.
Validation identified the following issues:
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[1]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[1]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[2]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[2]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[3]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[3]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[4]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[4]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[5]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[5]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[6]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[6]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[7]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[7]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[8]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[8]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[9]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[9]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids[1]'.

Strange, I thought that is valid. So I tried the following approach for debugging, pulling out the one item in the sequence I know with [1] notation.

 <context>
        <metapath target="/system-security-plan/metadata"/>
        <constraints>
            <let var="person-only-party-uuids" expression="party[@type='person']/@uuid"/>
            <expect id="responsible-party-is-person" target="responsible-party/party-uuid" test=". = $person-only-party-uuids[1]" level="ERROR">
                <message>{.} is not {$person-only-party-uuids[1]}</message>
            </expect>
            <remarks>
                <p>For roles 'system-owner', 'authorizing-official', 'authorizing-official-poc', 'system-poc-management', 'system-poc-technical', 'system-poc-other', and 'information-system-security-officer', the responsible-role party must be a party of type 'person'.</p>
            </remarks>
        </constraints>
    </context>

I apply this to a valid SSP valid and get the following errors.

oscal-cli validate '/home/me/computer/fedramp-automation/src/validations/constraints/content/ssp-all-VALID.xml' -c '/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-allowed-values.xml' -c '/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-constraints.xml'
Loading 'file:/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-allowed-values.xml'
Loading 'file:/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-constraints.xml'
The path '//part' is not properly contextualized using '.'. Using './/part' instead.
The path '//prop' is not properly contextualized using '.'. Using './/prop' instead.
The path '//(control|group|part)' is not properly contextualized using '.'. Using './/(control|group|part)' instead.
The path '//control' is not properly contextualized using '.'. Using './/control' instead.
The path '//param' is not properly contextualized using '.'. Using './/param' instead.
The path '//group' is not properly contextualized using '.'. Using './/group' instead.
Validating 'file:/home/me/computer/fedramp-automation/src/validations/constraints/content/ssp-all-VALID.xml' as XML.
Validation identified the following issues:
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[1]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[1]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[2]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[2]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[3]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[3]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[4]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[4]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[5]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[5]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[6]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[6]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[7]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[7]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[8]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[8]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.
FATAL: [CRITICAL] [/system-security-plan/metadata[1]/responsible-party[9]/party-uuid[1]] A gov.nist.secauto.metaschema.core.model.constraint.impl.DefaultExpectConstraint constraint with id 'responsible-party-is-person', matching the item at path '/system-security-plan/metadata[1]/responsible-party[9]/party-uuid[1]', resulted in an unexpected error. The error was: An error occurred while evaluating the expression '. = $person-only-party-uuids'.

When debugging and removing just the same expression from test, the error goes away, and the debug message prints out the UUID in the single sequence result as intended (designed it to intentionally fail).

 <context>
        <metapath target="/system-security-plan/metadata"/>
        <constraints>
            <let var="person-only-party-uuids" expression="party[@type='person']/@uuid"/>
            <expect id="responsible-party-is-person" target="responsible-party/party-uuid" test=". = ''" level="ERROR">
                <message>{.} is not {$person-only-party-uuids[1]}</message>
            </expect>
            <remarks>
                <p>For roles 'system-owner', 'authorizing-official', 'authorizing-official-poc', 'system-poc-management', 'system-poc-technical', 'system-poc-other', and 'information-system-security-officer', the responsible-role party must be a party of type 'person'.</p>
            </remarks>
        </constraints>
    </context>

oscal-cli validate '/home/me/computer/fedramp-automation/src/validations/constraints/content/ssp-all-VALID.xml' -c '/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-allowed-values.xml' -c '/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-constraints.xml'
Loading 'file:/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-allowed-values.xml'
Loading 'file:/home/me/computer/fedramp-automation/src/validations/constraints/fedramp-external-constraints.xml'
The path '//part' is not properly contextualized using '.'. Using './/part' instead.
The path '//prop' is not properly contextualized using '.'. Using './/prop' instead.
The path '//(control|group|part)' is not properly contextualized using '.'. Using './/(control|group|part)' instead.
The path '//control' is not properly contextualized using '.'. Using './/control' instead.
The path '//param' is not properly contextualized using '.'. Using './/param' instead.
The path '//group' is not properly contextualized using '.'. Using './/group' instead.
Validating 'file:/home/me/computer/fedramp-automation/src/validations/constraints/content/ssp-all-VALID.xml' as XML.
Validation identified the following issues:
[ERROR] [/system-security-plan/metadata[1]/responsible-party[1]/party-uuid[1]] 11111111-0000-4000-9000-000000000001 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[2]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[3]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[4]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[5]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[6]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[7]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[8]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
[ERROR] [/system-security-plan/metadata[1]/responsible-party[9]/party-uuid[1]] 22222222-0000-4000-9000-000000000002 is not 22222222-0000-4000-9000-000000000002
The file 'file:/home/me/computer/fedramp-automation/src/validations/constraints/content/ssp-all-VALID.xml' is invalid.

How do we replicate this issue

1. Run the command snippets and evaluate the Metapath above.

Expected behavior (i.e. solution)

1. Sequence evaluation is successfully completed.
2. Error messages may more specifically indicate what the error is (if obvious, I know this may be a fall-through error case).

Other comments

Details about version in use:

oscal-cli --version
oscal-cli 2.0.2 built at 2024-08-20 13:19 from branch 7df9154fb123ed6515ea51bacc65b7489da82717 (7df9154) at https://github.com/metaschema-framework/oscal-cli
liboscal-java  built at 2024-08-20 04:25 from branch 827cfe5b63f0f1f4905fc82c76d7e68733fcd271 (827cfe5) at https://github.com/metaschema-framework/liboscal-java
oscal v1.1.2 built at 2024-08-20 04:25 from branch 4f02dac6f698efda387cc5f55bc99581eaf494b6 (4f02dac) at https://github.com/usnistgov/OSCAL.git
metaschema-java 1.0.2 built at 2024-08-19T18:05:06+0000 from branch 76fe3c9dcebe45223d83034f4836510e3a347ba7 (76fe3c9) at https://github.com/metaschema-framework/metaschema-java
metaschema  built at 2024-08-19T18:05:06+0000 from branch 894b2238764c8732623a3894f0c236625ca5a686 (894b223) at https://github.com/metaschema-framework/metaschema.git

So this error seems very specific and minor, but I cannot seem to use sequences, in @test or debug, to work around that fn:contains is not yet implemented. I have been attempting to help developers in the FedRAMP Team but I am not sure of an alternative approach to work around this error or if it is in fact a bug.

[aj-stein-gsa]

Per our quick discussion this morning about the above, @david-waltermire, I think all the information above is enough to repro and debug with the latest stable release. I will need to look into SNAPSHOT builds later. FedRAMP Team members have alternate attempts at a solution for this constraint, but they use non-extant functions so I had to rewrite. They have moved to a sequence-oriended target = $sequence-of-items approach like me since we last spoke in this WIP PR and supporting branch.

GSA/fedramp-automation#652

Feel free to ask questions.