Convert and validate some READMEs to use the new YAML format. (openco…

…nfig#2903) * Convert and validate some READMEs to use the new YAML format. * Change YAML format of platform_type to list * Improve error messages * fix telemetry_basic_check_test (openconfig#2904) * fix telemetry_basic_check_test * enable workflow * fix basic check add int packet * enable workflow * revise yaml format * fix hw port path --------- Co-authored-by: Wen Bo Li <[email protected]> * Make exempted readmes populatable via CLI flag * fix * Enable all validations after making fixes and adding TODOs for READMEs that require domain expertise or longer code perusing * fix typo * add component type for a few paths and fix static analysis errors * fix typo --------- Co-authored-by: Darren Loher <[email protected]>
open-traffic-generator · Jul 10, 2024 · 02b4901 · 02b4901
1 parent 9fb156f
commit 02b4901
Show file tree

Hide file tree

Showing 43 changed files with 934 additions and 535 deletions.
diff --git a/.github/workflows/readme_oc_path_and_rpc.yml b/.github/workflows/readme_oc_path_and_rpc.yml
@@ -27,10 +27,40 @@ jobs:
     - name: Validate Template README
       run: |
         go install ./tools/validate_readme_spec
-        validate_readme_spec -alsologtostderr doc/test-requirements-template.md
+        validate_readme_spec --alsologtostderr doc/test-requirements-template.md
 
     - name: Validate Test READMEs
       run: |
         go install ./tools/validate_readme_spec
-        # TODO: Remove -feature-dir argument after all READMEs have converted to the new format.
-        validate_readme_spec -alsologtostderr -feature-dir feature/bgp/policybase/otg_tests/import_export_multi
+
+        exemption_flags=(
+          --non-test-readme security/gnsi/certz/test_data/README.md
+          --non-test-readme experimental/p4rt/README.md
+          --non-test-readme feature/security/gnsi/acctz/README.md
+        )
+
+        # TODO: Just use this one line after all READMEs have converted to the new format.
+        # validate_readme_spec --alsologtostderr "${exemption_flags[@]}"
+
+        function validate() {
+          validate_readme_spec --feature-dir "$1" --alsologtostderr "${exemption_flags[@]}"
+        }
+
+        validate feature/gnoi/os
+        validate feature/networkinstance
+        validate feature/security
+        validate feature/bgp/policybase/otg_tests/import_export_multi
+        validate feature/aft/aft_summary/otg_tests/route_summary_counters_test
+        validate feature/gnmi/otg_tests/telemetry_basic_check_test
+        validate feature/gnmi/otg_tests/telemetry_interface_packet_counters_test
+        validate feature/gnmi/otg_tests/telemetry_port_speed_test
+        validate feature/gnmi/subscribe/tests/gnmi_sample_mode_test
+        validate feature/gnmi/subscribe/tests/gnmi_subscriptionlist_test
+        validate feature/gnoi/system/tests/chassis_reboot_status_and_cancel_test
+        validate feature/gnoi/system/tests/ping_test
+        validate feature/gnoi/system/tests/supervisor_switchover_test
+        validate feature/isis/otg_tests/weighted_ecmp_test
+        validate feature/isis/static_route_isis_redistribution
+        validate feature/mtu/largeippacket/otg_tests/large_ip_packet_transmission
+        validate feature/staticroute/otg_tests/basic_static_route_support_test
+        validate feature/staticroute/tests
diff --git a/doc/test-requirements-template.md b/doc/test-requirements-template.md
@@ -49,9 +49,9 @@ paths:
   # interface configuration
   /interfaces/interface/config/description:
   /interfaces/interface/config/enabled:
-  # name of chassis component
+  # name of chassis and linecard components
   /components/component/state/name:
-    platform_type: "CHASSIS"
+    platform_type: ["CHASSIS", "LINECARD"]
 
 rpcs:
   gnmi:

diff --git a/feature/aft/aft_summary/otg_tests/route_summary_counters_test/README.md b/feature/aft/aft_summary/otg_tests/route_summary_counters_test/README.md
@@ -14,20 +14,28 @@ Establish eBGP sessions between ATE:port1 and DUT:port1 and another between ATE:
 *   Advertise prefixes from ATE port-1, observe received prefixes at ATE port-2 for IPv4 and IPv6
 *   Validate total number of entries of AFT for IPv4 and IPv6
 
-## Config Parameter Coverage
+## OpenConfig Path and RPC Coverage
 
+The below yaml defines the OC paths intended to be covered by this test.  OC paths used for test setup are not listed here.
 
-## Telemetry Parameter Coverage
+```yaml
+paths:
+  ## Config Paths ##
 
-/network-instances/network-instance/afts/aft-summaries/ipv4-unicast/protocols/protocol/state/counters/aft-entries
-/network-instances/network-instance/afts/aft-summaries/ipv6-unicast/protocols/protocol/state/counters/aft-entries
+  ## State Paths ##
+  /network-instances/network-instance/afts/aft-summaries/ipv4-unicast/protocols/protocol/state/counters/aft-entries:
+  /network-instances/network-instance/afts/aft-summaries/ipv6-unicast/protocols/protocol/state/counters/aft-entries:
 
-## Protocol/RPC Parameter Coverage
+rpcs:
+  gnmi:
+    gNMI.Subscribe:
+```
+
+## Control Protocol Coverage
 
 BGP
 IS-IS
 
 ## Minimum DUT Platform Requirement
 
 vRX
-
diff --git a/feature/gnmi/otg_tests/telemetry_basic_check_test/README.md b/feature/gnmi/otg_tests/telemetry_basic_check_test/README.md
@@ -12,10 +12,10 @@ following features:
 *   Ethernet interface
 
     *   Check the telemetry port-speed exists with correct speed.
-        *   /interfaces/interfaces/interface/ethernet/state/port-speed 
+        *   /interfaces/interfaces/interface/ethernet/state/port-speed
     *   Check the telemetry mac-address with correct format.
         *   /interfaces/interfaces/interface/ethernet/state/mac-address
-    
+
 
 *   Interface status
 
@@ -114,42 +114,65 @@ following features:
     *   Check the following path exists with correct node ID.
         *   /components/component/integrated-circuit/state/node-id
 
-## Config Parameter coverage
-
-No configuration coverage.
-
-## Telemetry Parameter coverage
-
-*   /interfaces/interface/state/admin-status
-*   /lacp/interfaces/interface/members/member
-*   /interfaces/interface/ethernet/state/mac-address
-*   /interfaces/interface/state/hardware-port /interfaces/interface/state/id
-*   /interfaces/interface/state/oper-status
-*   /interfaces/interface/ethernet/state/port-speed
-*   /interfaces/interface/state/physical-channel
-*   /components/component/integrated-circuit/state/node-id
-*   /components/component/state/parent
-*   /interfaces/interface/state/counters/in-octets
-*   /interfaces/interface/state/counters/in-unicast-pkts
-*   /interfaces/interface/state/counters/in-broadcast-pkts
-*   /interfaces/interface/state/counters/in-multicast-pkts
-*   /interfaces/interface/state/counters/in-discards
-*   /interfaces/interface/state/counters/in-errors
-*   /interfaces/interface/state/counters/in-fcs-errors
-*   /interfaces/interface/state/counters/out-unicast-pkts
-*   /interfaces/interface/state/counters/out-broadcast-pkts
-*   /interfaces/interface/state/counters/out-multicast-pkts
-*   /interfaces/interface/state/counters/out-octets
-*   /interfaces/interface/state/counters/out-discards
-*   /interfaces/interface/state/counters/out-errors
-*   /qos/interfaces/interface/output/queues/queue/state/transmit-pkts
-*   /qos/interfaces/interface/output/queues/queue/state/transmit-octets
-*   /qos/interfaces/interface/output/queues/queue/state/dropped-pkts
-*   /qos/interfaces/interface/output/queues/queue/state/dropped-octets
-
-## Protocol/RPC Parameter coverage
+## OpenConfig Path and RPC Coverage
+
+The below yaml defines the OC paths intended to be covered by this test.  OC paths used for test setup are not listed here.
+
+```yaml
+paths:
+  ## Config Paths ##
+  # None
+
+  ## State Paths ##
+  /interfaces/interface/state/admin-status:
+  /lacp/interfaces/interface/members/member/state/interface:
+  /lacp/interfaces/interface/members/member/state/counters/lacp-in-pkts:
+  /lacp/interfaces/interface/members/member/state/counters/lacp-out-pkts:
+  /lacp/interfaces/interface/members/member/state/aggregatable:
+  /lacp/interfaces/interface/members/member/state/collecting:
+  /lacp/interfaces/interface/members/member/state/distributing:
+  /lacp/interfaces/interface/members/member/state/partner-id:
+  /lacp/interfaces/interface/members/member/state/partner-key:
+  /lacp/interfaces/interface/members/member/state/partner-port-num:
+  /interfaces/interface/ethernet/state/mac-address:
+  /interfaces/interface/state/hardware-port:
+  /interfaces/interface/state/id:
+  /interfaces/interface/state/oper-status:
+  /interfaces/interface/ethernet/state/port-speed:
+  /interfaces/interface/state/physical-channel:
+  /components/component/integrated-circuit/state/node-id:
+    platform_type: [ "INTEGRATED_CIRCUIT" ]
+  /components/component/state/parent:
+    platform_type: [
+        "CONTROLLER_CARD",
+        "LINECARD",
+        "FABRIC",
+        "POWER_SUPPLY",
+        "INTEGRATED_CIRCUIT"
+    ]
+  /interfaces/interface/state/counters/in-octets:
+  /interfaces/interface/state/counters/in-unicast-pkts:
+  /interfaces/interface/state/counters/in-broadcast-pkts:
+  /interfaces/interface/state/counters/in-multicast-pkts:
+  /interfaces/interface/state/counters/in-discards:
+  /interfaces/interface/state/counters/in-errors:
+  /interfaces/interface/state/counters/in-fcs-errors:
+  /interfaces/interface/state/counters/out-unicast-pkts:
+  /interfaces/interface/state/counters/out-broadcast-pkts:
+  /interfaces/interface/state/counters/out-multicast-pkts:
+  /interfaces/interface/state/counters/out-octets:
+  /interfaces/interface/state/counters/out-discards:
+  /interfaces/interface/state/counters/out-errors:
+  /qos/interfaces/interface/output/queues/queue/state/transmit-pkts:
+  /qos/interfaces/interface/output/queues/queue/state/transmit-octets:
+  /qos/interfaces/interface/output/queues/queue/state/dropped-pkts:
+  /qos/interfaces/interface/output/queues/queue/state/dropped-octets:
+
+rpcs:
+  gnmi:
+    gNMI.Subscribe:
+```
 
-N/A
 
 ## Minimum DUT platform requirement
 

diff --git a/feature/gnmi/otg_tests/telemetry_interface_packet_counters_test/README.md b/feature/gnmi/otg_tests/telemetry_interface_packet_counters_test/README.md
@@ -59,49 +59,41 @@ following features:
     *   TODO: /interfaces/interface/state/cpu
     *   TODO: /interfaces/interface/state/management
 
-## Config Parameter coverage
-
-*   /interfaces/interface/config/enabled
-*   /interfaces/interface/subinterfaces/subinterface/config/enabled
-*   /interfaces/interface/subinterfaces/subinterface/ipv4/config/enabled
-*   /interfaces/interface/subinterfaces/subinterface/ipv6/config/enabled
-
-## Telemetry Parameter coverage
-
-*   /interfaces/interface/state/counters/in-pkts
-*   /interfaces/interface/state/counters/out-pkts
-
-*   /interfaces/interface/subinterfaces/subinterface]/ipv4/state/counters/in-pkts
-
-*   /interfaces/interface/subinterfaces/subinterface]/ipv4/state/counters/out-pkts
-
-*   /interfaces/interface/subinterfaces/subinterface]/ipv6/state/counters/in-pkts
-
-*   /interfaces/interface/subinterfaces/subinterface]/ipv6/state/counters/out-pkts
-
-*   /interfaces/interface/subinterfaces/subinterface]/ipv6/state/counters/in-discarded-pkts
-
-*   /interfaces/interface/subinterfaces/subinterface]/ipv6/state/counters/out-discarded-pkts
-
-*   /interfaces/interface/ethernet/state/counters/in-maxsize-exceeded
-
-*   /interfaces/interface/ethernet/state/counters/in-mac-pause-frames
-
-*   /interfaces/interface/ethernet/state/counters/out-mac-pause-frames
-
-*   /interfaces/interface/ethernet/state/counters/in-crc-errors
-
-*   /interfaces/interface/ethernet/state/counters/in-fragment-frames
-
-*   /interfaces/interface/ethernet/state/counters/in-jabber-frames
-
-*   /interfaces/interface/state/cpu
-
-*   /interfaces/interface/state/management
-
-## Protocol/RPC Parameter coverage
-
-No coverage
+## OpenConfig Path and RPC Coverage
+
+The below yaml defines the OC paths intended to be covered by this test.  OC paths used for test setup are not listed here.
+
+```yaml
+paths:
+  ## Config Paths ##
+  /interfaces/interface/config/enabled:
+  /interfaces/interface/subinterfaces/subinterface/config/enabled:
+  /interfaces/interface/subinterfaces/subinterface/ipv4/config/enabled:
+  /interfaces/interface/subinterfaces/subinterface/ipv6/config/enabled:
+
+  ## State Paths ##
+  /interfaces/interface/state/counters/in-pkts:
+  /interfaces/interface/state/counters/out-pkts:
+  /interfaces/interface/subinterfaces/subinterface/ipv4/state/counters/in-pkts:
+  /interfaces/interface/subinterfaces/subinterface/ipv4/state/counters/out-pkts:
+  /interfaces/interface/subinterfaces/subinterface/ipv6/state/counters/in-pkts:
+  /interfaces/interface/subinterfaces/subinterface/ipv6/state/counters/out-pkts:
+  /interfaces/interface/subinterfaces/subinterface/ipv6/state/counters/in-discarded-pkts:
+  /interfaces/interface/subinterfaces/subinterface/ipv6/state/counters/out-discarded-pkts:
+  /interfaces/interface/ethernet/state/counters/in-maxsize-exceeded:
+  /interfaces/interface/ethernet/state/counters/in-mac-pause-frames:
+  /interfaces/interface/ethernet/state/counters/out-mac-pause-frames:
+  /interfaces/interface/ethernet/state/counters/in-crc-errors:
+  /interfaces/interface/ethernet/state/counters/in-fragment-frames:
+  /interfaces/interface/ethernet/state/counters/in-jabber-frames:
+  /interfaces/interface/state/cpu:
+  /interfaces/interface/state/management:
+
+rpcs:
+  gnmi:
+    gNMI.Subscribe:
+    gNMI.Set:
+```
 
 ## Minimum DUT platform requirement
 

diff --git a/feature/gnmi/otg_tests/telemetry_port_speed_test/README.md b/feature/gnmi/otg_tests/telemetry_port_speed_test/README.md
@@ -20,19 +20,27 @@ Validate port speed telemetry used by controller infrastructure.
     *   Turn ports sequentially up at the ATE, and determine that the effective
         speed is increased as expected.
 
-## Config Parameter Coverage
+## OpenConfig Path and RPC Coverage
 
-TBD
+The below yaml defines the OC paths intended to be covered by this test.  OC paths used for test setup are not listed here.
 
-## Telemetry Parameter Coverage
+TODO(OCPATHS): Config paths TBD
 
-/interfaces/interface/state/oper-status
-/interfaces/interface/ethernet/state/port-speed
-/interfaces/interface/aggregation/state/lag-speed
+```yaml
+paths:
+  ## Config Paths ##
+  # TBD
 
-## Protocol/RPC Parameter Coverage
+  ## State Paths ##
+  /interfaces/interface/state/oper-status:
+  /interfaces/interface/ethernet/state/port-speed:
+  /interfaces/interface/aggregation/state/lag-speed:
+
+rpcs:
+  gnmi:
+    gNMI.Subscribe:
+```
 
-No new protocol coverage.
 
 ## Minimum DUT platform requirement
 

diff --git a/feature/gnmi/subscribe/tests/gnmi_sample_mode_test/README.md b/feature/gnmi/subscribe/tests/gnmi_sample_mode_test/README.md
@@ -39,3 +39,18 @@ Test to validate basic gNMI streaming telemetry works with `SAMPLE` mode.
 *   Configure ISIS on Port 1 in Default Network Instance.
 
 *   Verify that ISIS telemetry is streamed within the next 5 samples.
+
+## OpenConfig Path and RPC Coverage
+
+The below yaml defines the OC paths intended to be covered by this test.  OC paths used for test setup are not listed here.
+
+TODO(OCPATHS): Add paths
+
+```yaml
+paths:
+
+rpcs:
+  gnmi:
+    gNMI.Subscribe:
+      SAMPLE: true
+```