Resolving conflicts in spec and proto only

Merge remote-tracking branch 'p4lang/generic-table-dev' into generic_table_2

Signed-off-by: Bandyopadhyay, Sayan <[email protected]>

.github/workflows/ci-build-proto.yml

@@ -40,7 +40,7 @@ jobs:
         sudo mv $BAZEL /usr/local/bin/bazel
 
     - name: Build proto/
-      run: cd proto && bazel build //...
+      run: cd proto && bazel build //... && bazel test //...
 
     - name: Build bazel/example/
       run: cd bazel/example/ && bazel build //...

docs/v1/P4Runtime-Spec.mdk

@@ -230,8 +230,24 @@ format and available at
 P4Runtime is designed to be implemented in conjunction with the P4~16~ language
 version or later. P4~14~ programs should be translated into P4~16~ to be made
 compatible with P4Runtime. This version of P4Runtime utilizes features which are
-not in P4~16~ 1.0, but were introduced in P4~16~ 1.1.0 [@P4Revisions110]. For
-this version of P4Runtime, we recommend using P4~16~ 1.2.1 [@P4Spec].
+not in P4~16~ 1.0, but were introduced in P4~16~ 1.2.4 [@P4Revisions124]. For
+this version of P4Runtime, we recommend using P4~16~ 1.2.4 [@P4Revisions124].
+
+This version of the P4Runtime specification does not yet explicitly
+address compatibility with the following P4~16~ language features
+introduced in versions 1.2.2 or 1.2.4 of the language specification:
+
+* Added support for generic structures [@P4Revisions122].
+* Added support for additional enumeration types [@P4Revisions122].
+* Added support for 0-width bitstrings and varbits [@P4Revisions122].
+* Clarified restrictions for parameters with default values
+  [@P4Revisions124].
+* Allow ranges to be specified by serializable enums
+  [@P4Revisions124].
+* Added `list` type [@P4Revisions124].
+* Clarified behavior of table with no `key` property, or if its list
+  of keys is empty [@P4Revisions124].
+
 
 ## In Scope
 
@@ -1524,6 +1540,12 @@ control plane. This message contains the following fields:
 * `is_const_table`, a boolean flag indicating that the table is filled with
   static entries and cannot be modified by the control plane at runtime.
 
+* `has_initial_entries`, a boolean flag indicating that the table has
+  entries populated into it when the P4 program is loaded, which is
+  true for tables in the P4 source code with either the `entries` or
+  `const entries` properties, and there is at least one entry in the
+  list.
+
 * `other_properties`, an `Any` Protobuf message [@ProtoAny] to embed
   architecture-specific table properties [@P4TableProperties] which are not part
   of the core P4 language or of the PSA architecture.
@@ -3022,6 +3044,9 @@ the `TableEntry` entity, which has the following fields:
   implement idle-timeout support for the table, if applicable. See
   [Idle-timeout](#sec-idle-timeout) section for more information.
 
+* `is_const`, a boolean value that is `true` if and only if the entry
+  cannot be modified or deleted by the client.
+
 * `resources`, a `repeated GenericTableEntry` to contain a list of
   target-specific direct resources. For example, if a target supports special
   direct resources like x-meter and y-counter, then the P4info will contain
@@ -3063,6 +3088,10 @@ limitation. It is recommended that, for the sake of portability, P4Runtime
 clients do not try to insert additional entries once the size indicated in
 P4Info has been reached.
 
+The `is_const` field must be `false` in any `INSERT`, `MODIFY`, or
+`DELETE` write request of a table entry.  If it is true, the server
+must reject the operation and return an `INVALID_ARGUMENT` error.
+
 ### Match Format { #sec-match-format}
 
 The bytes fields in the `FieldMatch` message follow the format described in
@@ -3290,10 +3319,16 @@ indirect tables --- tables with an ActionProfile or ActionSelector
 `implementation` property --- to a constant `NoAction` action entry, with the
 hope that it would simplify the implementation of the P4Runtime service.
 
-### Constant Tables
+### Constant Tables { #sec-constant-tables }
 
-Constant tables are defined as tables whose match entries are immutable. They
-are identified by the `is_const_table` flag in P4Info.
+Constant tables are defined as tables whose match entries are
+immutable. They are identified by the table property `const entries`
+in the P4~16~ source code.  In the P4Info, such tables have
+`is_const_table` equal to true, and if the list of entries in the
+source code has at least one entry in it, they also have
+`has_initial_entries` flag equal to true.  For tables declared with
+the `entries` property, without `const` before `entries` see Section
+[#sec-preinitialized-tables].
 
 The only write updates which are allowed for constant tables are `MODIFY`
 operations on direct resources, and the default action (assuming the default
@@ -3310,13 +3345,74 @@ The contents of const tables can be queried by the client through a
 the following fields must be set by the server: `table_id`, `match`, `action`,
 `is_default_action`, and `priority` (if required). This is in addition to any
 direct resources that are being queried.  Idle timeouts are not supported for
-static entries. If the table requires a priority value for entries, the server
-must populate the `priority` field appropriately, starting at 1 for the lowest
-priority entry and incrementing the value by 1 for each successive entry. Note
-that P4~16~ does not support assigning explicit priorities to static
-entries. When a priority value is required (⪚ for tables including `RANGE`,
-`TERNARY` or `OPTIONAL` matches), it is inferred based on the order in which
-entries appear in the table declaration.
+static entries.
+
+When a priority value is required (⪚ for tables including `RANGE`,
+`TERNARY` or `OPTIONAL` matches), it is inferred based on the order in
+which entries appear in the table declaration.  As of August 2023, the
+open source `p4c` compiler always assigns entry priority values in a
+constant table with `N` entries starting at `N` for the first entry
+and decrementing the value by 1 for each successive entry.  The P4~16~
+language specification does not preclude the P4 developer from
+explicitly specifying priorities for entries in constant tables, but
+`p4c` does not yet support this.
+
+### Preinitialized tables { #sec-preinitialized-tables }
+
+Preinitialized tables are those defined with an `entries` table
+property in the P4~16~ source code, with no `const` qualifier before
+`entries`, and at least one entry in that list.  In the P4Info, such
+tables have `has_initial_entries` flag equal to true, but
+`is_const_table` is false.  For tables declared with `const entries`,
+see Section [#sec-constant-tables].
+
+Every P4 table falls into one of three categories:
+
+* *Normal table*: Neither `entries` nor `const entries` are declared
+  in the source code, and thus `is_const_table` and
+  `has_initial_entries` will both be false.  A corner case is that if
+  it has `entries = { }` with no `const` before `entries`, &ie; an
+  empty list of entries, that is also a normal table.
+* *Constant table*: The table has `const entries` declared, and thus a
+  separate `entries` property is not permitted by the language.  Such
+  a table will have `is_const_table` true.  Such a table will have
+  `has_initial_entries` true if there is at least one entry in the
+  source code, or false if the list is empty.
+* *Preinitialized table*: The table has `entries` declared, and thus a
+  separate `const entries` property is not permitted by the language.
+  It also has at least one entry in the list.  Such a table will have
+  `is_const_table` false and `has_initial_entries` true.
+
+A preinitialized table is allowed to have a mix of some entries marked
+`const`, and other entries not marked `const`.
+
+Entries not marked `const` may be modified or deleted, just as a
+client may do for any entry in a normal table.
+
+Entries marked `const` behave like entries in a constant table,
+&ie; only `MODIFY` operations on direct resources are allowed.
+
+Unlike a table with `is_const_table = true`, a client may insert
+entries into a table with `has_initial_entries = true` and
+`is_const_table = false`, subject to capacity constraints on the
+number of entries supported by the target for the table.
+
+The contents of preinitialized tables can be queried by the client
+through a `ReadRequest`.  The server fills in the same fields in the
+response as it does for constant tables, as described in Section
+[#sec-constant-tables], and with the same restrictions on table
+features supported.
+
+If the table requires a priority value for entries, the priorities of
+the initial entries are determined according to the P4~16~ language
+specification.  After the P4 program is initially loaded, the entries
+not marked `const` can be modified at run time just as table entries
+in a normal table can.
+
+The contents of all table entries within the `entries` table
+properties in a P4 program can be written to a separate output file by
+the open source `p4c` compiler.  See Section [#sec-entries-files] for
+details.
 
 ### Wildcard Reads { #sec-table-wildcard-reads}
 
@@ -4383,14 +4479,18 @@ semantics.
   server must return an `INVALID_ARGUMENT` error. The egress port (`port` field)
   must be an SDN port and must refer to a singleton port. No two replicas may
   have identical values of *both* `port` and `instance`, or the server must
-  return `INVALID_ARGUMENT`.
-* `MODIFY`: Modify the set of replicas for a given multicast group entry,
-  indexed by the given `multicast_group_id`. Same restrictions as `INSERT` apply
-  here.
+  return `INVALID_ARGUMENT`. The `metadata` field is an arbitrary `bytes` value
+  which is opaque to the target. There is no requirement of where this is
+  stored, but it must be returned by the server along with the rest of the entry
+  when the client performs a read on the entry.
+* `MODIFY`: Modify the set of replicas and metadata for a given multicast group
+  entry, indexed by the given `multicast_group_id`. Same restrictions as
+  `INSERT` apply here.
 * `DELETE`: Delete the multicast group indexed by the given
-  `multicast_group_id`. The replicas need not be provided for this
-  operation. Any packets with their `multicast_group` metadata in the data plane
-  set to the deleted `multicast_group_id` will be dropped.
+  `multicast_group_id`. The `replicas` and `metadata` fields need not be
+  provided for this operation. Any packets with their `multicast_group`
+  metadata in the data plane set to the deleted `multicast_group_id` will be
+  dropped.
 
 When reading a multicast group, only `multicast_group_id` is considered. All
 other fields in `MulticastGroupEntry` are ignored. To perform a *wildcard*
@@ -6787,6 +6887,7 @@ compiler.
 
 ### Changes in v1.4.0
 
+* Add a `metadata` field to the `MulticastGroupEntry` message.
 * Clarify that the limitation on supported types for `FieldMatch`, action
   `Param`, and Packet IO metadata fields (no support for signed integers, with
   type `int<W>`) apply to all minor revisions of P4Runtime v1, not just to
@@ -7042,4 +7143,51 @@ clients are aware that the server is using a larger maximum receive message
 size. The gRPC server running the P4Runtime service must not set the maximum
 receive message size to a value smaller than the default (4MB).
 
+## P4Runtime Entries files { #sec-entries-files }
+
+The open source P4 compiler `p4c` [@p4c] implements an option to
+generate an "entries file", &ie; a file that contains all table
+entries declared via the `entries` table property within the program.
+
+An example P4~16~ program that can be used to demonstrate this
+capability is `table-entries-ternary-bmv2.p4` [@p4cTestProgramForConstEntries]:
+
+    git clone https://github.com/p4lang/p4c
+    cd p4c/testdata/p4_16_samples
+    mkdir tmp
+    p4test --arch v1model \
+        --p4runtime-files tmp/p4info.txt \
+        --p4runtime-entries-files tmp/entries.txt \
+        table-entries-ternary-bmv2.p4
+
+You can replace the `.txt` suffix of the file name `tmp/entries.txt`
+in the example command above with `.json` or `.bin`.  The `.bin`
+format is a binary P4Runtime API protobuf message format.  The `.txt`
+format is the text encoding of the same Protobuf messages.
+
+Target devices are *not* required to use this file.  For example, if a
+target has a P4 compiler back end that encodes all of the necessary
+details from the P4 source program, including the `entries` of tables,
+in a target-specific binary format, then that target might have no
+reason to generate these entries files.
+
+Some target devices might choose to generate entries files, and also
+to require doing so in order to have a correct implementation.  For
+example, a target runtime implementation might take a target-specific
+binary format for the compiled P4 program that does *not* contain any
+data describing the `entries` of tables, plus the entries file
+generated by `p4c`, and use the entries file to load the initial
+entries of tables into the device.
+
+The format of the entries file is a single `WriteRequest` message
+containing one `Update` sub-message per entry in the P4 source program
+defined via an `entries` table property.  All `Update` sub-messages
+have `type` equal to `INSERT`, and `entity` is a `TableEntry` message
+containing the data for one table entry.
+
+Note that if a P4Runtime client attempted to send a `WriteRequest` to
+a P4Runtime server with the contents of the entries file, the server
+must return an error for each entry that has `is_const` true, as
+described in Section [#sec-table-entry].
+
 [BIB]

docs/v1/README.md

@@ -7,7 +7,8 @@ specification document.
 
 The markup version uses Madoko (https://www.madoko.net) to produce
 HTML and PDF versions of the documentation. Pre-built versions of the
-documentation are available at **[TODO]**
+documentation are available on the [P4.org specifications
+page](https://p4.org/specs).
 
 
 Files:
@@ -63,13 +64,3 @@ Note that to build the PDF you need a functional TeX version installed.
 You need to install miktex [http://miktex.org/], madoko
 [https://www.madoko.net/] and node.js [https://nodejs.org/en/].  To
 build you can invoke the make.bat script.
-
-
-# TODO
-## Formating Fixups TODO
-
-## Content TODO
-Following are major content items which are missing or incomplete:
-*  Section 17. P4Runtime Versioning
-*  Section 18. Extending P4Runtime for non-PSA architectures
-*  Section 19. Lifetime of a session

docs/v1/references.bib

@@ -65,6 +65,16 @@ @ONLINE { P4Revisions110
     url = "https://p4.org/p4-spec/docs/P4-16-v1.2.1.html#sec-summary-of-changes-made-in-version-110"
 }
 
+@ONLINE { P4Revisions122,
+    title = "Summary of changes made in $P4_{16}$ version 1.2.2",
+    url = "https://p4.org/p4-spec/docs/P4-16-v1.2.4.html#sec-summary-of-changes-made-in-version-122-released-may-17-2021"
+}
+
+@ONLINE { P4Revisions124,
+    title = "Summary of changes made in $P4_{16}$ version 1.2.4",
+    url = "https://p4.org/p4-spec/docs/P4-16-v1.2.4.html#sec-summary-of-changes-made-in-version-124"
+}
+
 @ONLINE { P4Spec,
     title = "$P4_{16}$ 1.2.1 specification",
     url = "https://p4.org/p4-spec/docs/P4-16-v1.2.1.html"
@@ -210,3 +220,13 @@ @ONLINE { ArenaAllocation
     title = "C++ Arena Allocation Guide",
     url = "https://developers.google.com/protocol-buffers/docs/reference/arenas"
 }
+
+@ONLINE { p4c,
+    title = "P4_16 reference compiler",
+    url = "https://github.com/p4lang/p4c"
+}
+
+@ONLINE { p4cTestProgramForConstEntries,
+    title = "P4_16 reference compiler test program table-entries-ternary-bmv2.p4",
+    url = "https://github.com/p4lang/p4c/blob/main/testdata/p4_16_samples/table-entries-ternary-bmv2.p4"
+}

go.mod

@@ -4,13 +4,13 @@ go 1.20
 
 require (
 	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1
-	google.golang.org/grpc v1.56.1
+	google.golang.org/grpc v1.56.3
 	google.golang.org/protobuf v1.31.0
 )
 
 require (
 	github.com/golang/protobuf v1.5.3 // indirect
-	golang.org/x/net v0.9.0 // indirect
-	golang.org/x/sys v0.7.0 // indirect
-	golang.org/x/text v0.9.0 // indirect
+	golang.org/x/net v0.17.0 // indirect
+	golang.org/x/sys v0.13.0 // indirect
+	golang.org/x/text v0.13.0 // indirect
 )

go.sum

@@ -3,17 +3,17 @@ github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg
 github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
 github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
-golang.org/x/net v0.9.0 h1:aWJ/m6xSmxWBx+V0XRHTlrYrPG56jKsLdTFmsSsCzOM=
-golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
-golang.org/x/sys v0.7.0 h1:3jlCCIQZPdOYu1h8BkNvLz8Kgwtae2cagcG/VamtZRU=
-golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE=
-golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
+golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
+golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
+golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
+golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
+golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 h1:KpwkzHKEF7B9Zxg18WzOa7djJ+Ha5DzthMyZYQfEn2A=
 google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1/go.mod h1:nKE/iIaLqn2bQwXBg8f1g2Ylh6r5MN5CmZvuzZCgsCU=
-google.golang.org/grpc v1.56.1 h1:z0dNfjIl0VpaZ9iSVjA6daGatAYwPGstTjt5vkRMFkQ=
-google.golang.org/grpc v1.56.1/go.mod h1:I9bI3vqKfayGqPUAwGdOSu7kt6oIJLixfffKrpXqQ9s=
+google.golang.org/grpc v1.56.3 h1:8I4C0Yq1EjstUzUJzpcRVbuYA2mODtEmpWiQoN/b2nc=
+google.golang.org/grpc v1.56.3/go.mod h1:I9bI3vqKfayGqPUAwGdOSu7kt6oIJLixfffKrpXqQ9s=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
 google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
 google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=