From 6767223a55235138dacd5ef7ee1e685ed6051bca Mon Sep 17 00:00:00 2001 From: vincentgoat Date: Mon, 26 Dec 2022 21:30:10 +0800 Subject: [PATCH 1/2] KubeEdge Conformance Tests Proposal Signed-off-by: vincentgoat --- conformance/README.md | 9 ++ conformance/instructions.md | 107 ++++++++++++++ .../Certified_KubeEdge_Form.md | 94 +++++++++++++ .../Certified_KubeEdge_Terms.md | 131 ++++++++++++++++++ 4 files changed, 341 insertions(+) create mode 100644 conformance/README.md create mode 100644 conformance/instructions.md create mode 100644 conformance/participation-form/Certified_KubeEdge_Form.md create mode 100644 conformance/terms-conditions/Certified_KubeEdge_Terms.md diff --git a/conformance/README.md b/conformance/README.md new file mode 100644 index 0000000..4903e6d --- /dev/null +++ b/conformance/README.md @@ -0,0 +1,9 @@ +## KubeEdge Conformance Tests + +With the iteration of the KubeEdge version, the KubeEdge conformance test cases will continue to increase, and the release document of the conformance test cases will be released synchronously. + +### How to test + +As prerequisites, please read the document `Certified_KubeEdge_Terms.md`. + +Please refer to the instructions.md document to run the tests, and before starting the test. \ No newline at end of file diff --git a/conformance/instructions.md b/conformance/instructions.md new file mode 100644 index 0000000..a485099 --- /dev/null +++ b/conformance/instructions.md @@ -0,0 +1,107 @@ +# How to submit conformance results + +## About the KubeEdge conformance tests + +The standard set of conformance tests of KubeEdge is currently those defined by the `[Conformance]` tag in the [KubeEdge e2e](https://github.com/kubeedge/kubeedge/tree/master/tests/e2e) suite, plus a part of kubernetes conformance tests included at [here](https://github.com/kubeedge/kubeedge/blob/master/build/conformance/kubernetes/kube_conformance_test.go). + +## Running + +KubeEdge has provided the docker image of the conformance test, which contains the scripts and related files of the conformance test. Follow these steps to perform a conformance test. + +### Prerequisite + +- At least one master node and one edge node exist on different VMS or physical machines. +- Enable the CloudStream and EdgeStream modules. + +### Launch the conformance test container + +Pull conformance test image `kubeedge/conformance-test` + +``` +$ docker pull kubeedge/conformance-test +``` + +Or build the image locally. + +``` +$ git clone https://github.com/kubeedge/kubeedge.git +$ cd kubeedge +$ docker build -t {image_name}:{tag_name} -f build/conformance/Dockerfile . +``` + +Examples of running the conformance test containers: + +``` +docker run --env KUBECONFIG=/root/.kube/config --env RESULTS_DIR=/tmp/results -v /root/.kube/config:/root/.kube/config -v /tmp/results:/tmp/results --network host -it kubeedge/conformance-test +``` + +Description of container environment variables: + +| Environment variables | The corresponding ginkgo parameters | Parameters description | +| --------------------- | ----------------------------------- | ------------------------------------------------------------ | +| E2E_DRYRUN | dryRun | If set, ginkgo will walk the test hierarchy without actually running anything. Best paired with -v. | +| E2E_SKIP | skip | If set, ginkgo will only run specs that do not match this regular expression. Can be specified multiple times, values are ORed. | +| E2E_FOCUS | focus | If set, ginkgo will only run specs that match this regular expression. Can be specified multiple times, values are ORed. | +| RESULTS_DIR | NA | Output report path, default /tmp/results | +| REPORT_PREFIX | NA | Report file prefix, optional | +| IMAGE_URL | NA | The name of the image used by the test case | +| TEST_WITH_DEVICE | NA | Whether to test device. The default value is false | +| GINKGO_BIN | NA | `ginkgo` binary path，default /usr/local/bin/ginkgo | +| E2E_EXTRA_ARGS | NA | Extra arguments | +| KUBECONFIG | NA | `kubeconfig` file path | +| TEST_BIN | NA | `e2e.test` binary path，default /usr/local/bin/e2e.test | + +## Uploading + +Prepare a PR to https://github.com/kubeedge/community to upload the report files to the directory `conformance`. In the descriptions below, `X.Y` refers to the KubeEdge major and minor version, and `$dir` is a short subdirectory name to hold the results for your product. + +Description: `Conformance results for vX.Y/$dir` + +### Contents of the PR + +For simplicity you can submit the tarball or extract the relevant information from the tarball to compose your submission. + +If submitting test results for multiple versions, submit a PR for each product, ie. one PR for vX.Y results and a second PR for vX.Z + +``` +vX.Y/$dir/README.md: A script or human-readable description of how to reproduce +your results. +vX.Y/$dir/e2e.log: Test log output (from the container kubeedge/conformance). +vX.Y/$dir/junit_conformance.xml: Machine-readable test log (from the container kubeedge/conformance). +vX.Y/$dir/PRODUCT.yaml: See below. +``` + +#### PRODUCT.yaml + +This file describes your product. It is YAML formatted with the following root-level fields. Please fill in as appropriate. + +| Field | Description | +| ----------------------- | ------------------------------------------------------------ | +| `vendor` | Name of the legal entity that is certifying. | +| `name` | Name of the product being certified. | +| `version` | The version of the product being certified (not the version of KubeEdge it runs). | +| `website_url` | URL to the product information website | +| `repo_url` | If your product is open source, this field is necessary to point to the primary GitHub repo containing the source. It's OK if this is a mirror. OPTIONAL | +| `documentation_url` | URL to the product documentation | +| `product_logo_url` | URL to the product's logo, (must be in SVG, AI or EPS format -- not a PNG -- and include the product name). OPTIONAL. If not supplied, we'll use your company logo. Please see logo [guidelines](https://github.com/cncf/landscape#logos) | +| `type` | Is your product a distribution, hosted platform, or installer. | +| `description` | One sentence description of your offering | +| `contact_email_address` | An email address which can be used to contact maintainers regarding the product submitted and updates to the submission process | + +Examples: + +``` +vendor: Huawei Cloud +name: IEF +version: v1.x.x +website_url: https://xxx +repo_url: https://xxx +documentation_url: https://xxx/docs +product_logo_url: https://xxx.svg +type: hosted platform +description: "Based on the KubeEdge and kubernetes ecosystems, IEF applies cloud native technologies to edge computing." +``` + +## Contact + +If you have any problems about certifying or conformance testing, please file an issue in the [kubeedge/community](https://github.com/kubeedge/community). Questions and comments can also be sent to the working group's [slack channel](https://kubeedge.slack.com/archives/CKVUCM5ED). SIG Testing is the change controller of the conformance definition. \ No newline at end of file diff --git a/conformance/participation-form/Certified_KubeEdge_Form.md b/conformance/participation-form/Certified_KubeEdge_Form.md new file mode 100644 index 0000000..f9bcec7 --- /dev/null +++ b/conformance/participation-form/Certified_KubeEdge_Form.md @@ -0,0 +1,94 @@ +### Certified KubeEdge Conformance Program +#### Participation Form + +Complete this form for each Participant (company or other entity) that desires to participate in the Certified KubeEdge Conformance Program and to use the Certified KubeEdge Marks or Participant KubeEdge Combinations. KubeEdge and the Certified KubeEdge Marks are trademarks of The Linux Foundation. Capitalized terms used herein and not otherwise defined shall have the same meanings set forth in the Program Terms. + +By signing below and submitting this form to The Linux Foundation (by [DocuSign](#TO-BE-CREATED) or by emailing the [PDF](https://github.com/kubeedge/community/blob/master/sig-testing/conformance/Certified_KubeEdge_Form.pdf) to [conformance@cncf.io](mailto:conformance@cncf.io)): + +1. The Participant agrees to the Terms and Conditions of the Certified KubeEdge Conformance Program (the "**Program Terms**"), available at https://github.com/kubeedge/community/blob/master/sig-testing/conformance/Certified_KubeEdge_Terms.pdf>. +2. The Participant confirms that the products and services identified below as Qualifying Offerings have passed all of the self-tests described in the Certification Guide, and are Qualifying Offerings under the Program Terms. +3. The Participant confirms that it has submitted to the Cloud Native Computing Foundation ("**CNCF**") the results of the self-tests prior to its first public use of the Certified KubeEdge Marks associated with the corresponding version of KubeEdge. +4. The Participant confirms that it will either (a) maintain conformance of the Qualifying Offerings with later versions of KubeEdge, or (b) cease use of the Certified KubeEdge Marks and Participant KubeEdge Combinations at the end of the applicable conformance time period described in the Program Terms. +5. The Participant confirms that it has listed below all Participant KubeEdge Combinations that it intends to use with the Qualifying Offerings. +6. The Participant confirms that it will promptly submit an updated Participant Form to The Linux Foundation prior to (a) using the Certified KubeEdge Marks with Qualifying Offerings not listed here, or (b) using Participant KubeEdge Combinations not listed here. +7. I confirm that I am authorized to make the above statements and to submit this form on behalf of the Participant. + + +#### Participant Information + + +Company / entity name: + +\___________________________________________________ + +Contact address: + +\___________________________________________________ + +\___________________________________________________ + +\___________________________________________________ + +\___________________________________________________ + +Contact telephone: + +\___________________________________________________ + +Contact email: + +\___________________________________________________ + + +Select one: + + - [ ] Participant is a member of CNCF. + - [ ] Participant is a non-profit organization. + - [ ] Neither of the above. **Please contact CNCF to discuss fees for participation in the Conformance Program.** + +#### Qualifying Offerings + +Name, brief description and URLs for more information: + +\___________________________________________________________________________ + +\___________________________________________________________________________ + +\___________________________________________________________________________ + +\___________________________________________________________________________ + + +#### Participant KubeEdge Combinations + +List all Participant KubeEdge Combinations to be used with the Qualifying Offerings, if any: + +(for example, "XYZ KubeEdge" or "XYZ KubeEdge Platform") + +\___________________________________________________________________________ + +\___________________________________________________________________________ + +\___________________________________________________________________________ + +\___________________________________________________________________________ + + +#### Conformance Details + +Initial Version of KubeEdge for Conformance (e.g., v1.11): _______ + +Conformance Date: __________________ + + +#### Signed on behalf of Participant by: + +``` +Signature: __________________________________ + +Name: __________________________________ + +Title: __________________________________ + +Date: __________________________________ +``` \ No newline at end of file diff --git a/conformance/terms-conditions/Certified_KubeEdge_Terms.md b/conformance/terms-conditions/Certified_KubeEdge_Terms.md new file mode 100644 index 0000000..09ff893 --- /dev/null +++ b/conformance/terms-conditions/Certified_KubeEdge_Terms.md @@ -0,0 +1,131 @@ +## Certified KubeEdge Conformance Program – Terms and Conditions + +This document sets forth the terms and conditions that apply to the Conformance Program and use of the Certified KubeEdge Marks. + +In this document, these terms have the following meanings: + +- "**Certified KubeEdge Marks**" means (1) the mark "Certified KubeEdge" and (2) all other logos and marks described in the Branding Guide, as defined herein; +- "**KubeEdge Word Mark**" means the trademark "KubeEdge"; +- "**Participant KubeEdge Combination**" means a combination that includes both the Participant’s mark(s) and the KubeEdge Word Mark as elements (for example, "XYZ KubeEdge" or "XYZ KubeEdge Platform"); +- "**Conformance Program**" means the program described in this document; +- "**Participant**" means a company or other entity that is permitted to use one or more of the Certified KubeEdge Marks in association with their goods and services, pursuant to the Conformance Program; and +- "**Qualifying Offering(s)**" means a Participant’s goods or services that have passed all of the self-tests described in the Certification Guide, where the self-test results have been submitted to and accepted by CNCF as described below. + +#### Related Documents + +Together with these terms and conditions, the following documents are part of the Conformance Program, each as updated from time to time: + +- "**Certification Guide**": the conformance testing instructions and requirements for the Conformance Program, available at https://github.com/kubeedge/community/blob/master/sig-testing/conformance/instructions.md; +- "**Branding Guide**": the Certified KubeEdge Brand Guidelines, to be designed (Once completed, it will be uploaded to https://github.com/cncf/artwork/tree/master/projects/kubeedge); +- "**LF Trademark Usage Guidelines**": The Linux Foundation Trademark Usage Guidelines, available at https://www.linuxfoundation.org/trademark-usage; and +- "**Participation Form**": the Certified KubeEdge Participation Form, available at https://github.com/kubeedge/community/blob/master/sig-testing/conformance/Certified_KubeEdge_Form.md. + +#### Introduction + +The Certified KubeEdge Marks and the KubeEdge Word Mark are trademarks of The Linux Foundation. The Linux Foundation has established the Conformance Program to achieve two objectives: 1) The Linux Foundation needs to ensure that the Certified KubeEdge Marks and the KubeEdge Word Mark remain reliable indicators of the qualities that they have been created to preserve, and 2) The Linux Foundation needs to ensure that community members are able to accurately describe their Qualifying Offerings. + +To achieve these objectives, The Linux Foundation requires that third parties who use the Certified KubeEdge Marks or Participant KubeEdge Combinations in association with their goods and services may do so only as Participants pursuant to the Conformance Program described in this document. By participating in the Conformance Program and by using the Certified KubeEdge Marks or Participant KubeEdge Combinations, a Participant acknowledges that third parties will be relying on the accuracy of its self-testing results and on its compliance with the terms of the Conformance Program. + +#### Participant Requirements + +In order to be a Participant in the Conformance Program and to use the Certified KubeEdge Marks or Participant KubeEdge Combinations, a Participant must do the following: + +1. determine those of its goods and services with which it intends to use with the Certified KubeEdge Marks and Participant KubeEdge Combinations; +2. confirm through self-testing that those goods and services are Qualifying Offerings, as described herein; +3. submit to The Linux Foundation a signed Participation Form (by email to [conformance@cncf.io](mailto:conformance@cncf.io), by [DocuSign](#TO-BE-CREATED) or other method made available by The Linux Foundation), designating the applicable Qualifying Offerings and Participant KubeEdge Combinations and accepting the terms and conditions of the Conformance Program; +4. submit to the Cloud Native Computing Foundation ("CNCF") the successfully-passing self-testing results for the Qualifying Offerings, and have the results accepted by CNCF; +5. pay fees (if any) required for its participation in the Conformance Program; and +6. abide by the terms and conditions of the Conformance Program, including subsequent re-testing as described herein, throughout the period of the Participant’s use of the Certified KubeEdge Marks and Participant KubeEdge Combinations. + +Members of CNCF and non-profit organizations will not be charged fees to participate in the Conformance Program. Other potential Participants should contact CNCF for more information about participation fees. + +#### Qualifying Offerings and Self-Testing + +*Eligible Versions*. A Qualifying Offering may be initially self-certified for (1) the then-current minor (x.y) official release of KubeEdge, and (2) the two immediately-preceding minor (x.y-1 and x.y-2) official releases of KubeEdge; provided, in each case, that CNCF has made available a testing evaluation program for that version of KubeEdge. + +Example: Following the release of KubeEdge v1.11, a new Qualifying Offering may be initially self-certified for KubeEdge v1.11, v1.10 or v1.9, but not for v1.8. + +*Self-Testing Process*. The Certified KubeEdge Marks and Participant KubeEdge Combinations are intended to be used with Participants’ goods and services that are Qualifying Offerings. The Certification Guide contains details about the self-testing process and the specific tests that must be passed for a Participant’s offering to be a Qualifying Offering. A Participant confirms that its goods and services are Qualifying Offerings by demonstrating that its offerings have successfully passed all of the self-tests set forth in the Certification Guide. CNCF will make available a testing evaluation program that Participants will use to determine whether the self-tests are passed or failed. + +*Submission of Self-Testing Results*. After passing the designated self-tests, a Participant must submit to CNCF (in the manner described in the Certification Guide) a copy of the self-testing results generated by the testing evaluation program no more than 90 days prior to its first public use of the Certified KubeEdge Marks or Participant KubeEdge Combinations. These results will be made available to the public on a CNCF website or public repository if Participant makes public use of the Certified KubeEdge Marks or Participant KubeEdge Combinations. However, if the Participant decides not to proceed with any such public use following submission of self-testing results to CNCF, then upon Participant’s written request to CNCF the test results will not be publicly disclosed. + +*Acceptance of Self-Testing Results*. The Certified KubeEdge Marks and Participant KubeEdge Combinations may only be used with Qualifying Offerings whose self-tests have been both submitted to and accepted by CNCF (in the manner described in the Certification Guide). + +*Profiles*. The Certification Guide may also specify, for certain versions of KubeEdge, one or more "Profiles" consisting of groups of different or additional functionality. These Profiles will be associated with additional sets of self-tests. If a Participant’s Qualifying Offering also passes the set of self-tests associated with a Profile, then the Participant may additionally use with the Qualifying Offering any Certified KubeEdge Marks designated for that Profile. + +*End User Reproducibility*. One objective of the Conformance Program is that the self-tests set forth in the Certification Guide should be verifiable by end users of each Qualifying Offering. A Participant may not prohibit or prevent end users of its Qualifying Offerings from running the corresponding self-tests on a Qualifying Offering, in order to reproduce the submitted results and verify their accuracy. If The Linux Foundation determines or is informed that (1) end users of a Qualifying Offering are prevented from running the tests, or (2) that end users’ results from running the tests demonstrate that a Qualifying Offering does not successfully pass the tests, then upon written notice from The Linux Foundation the Participant must address any non-conformance concerns raised by The Linux Foundation; successfully pass the corresponding self-tests; and submit to CNCF a copy of the new self-testing results generated by the testing evaluation program within 30 days following such notice. Otherwise, the Participant will be subject to the requirements to remove the Certified KubeEdge Marks and Participant KubeEdge Combinations set forth herein. + +*Changes to Qualifying Offerings*. The self-testing process and permission to use the Certified KubeEdge Marks and Participant KubeEdge Combinations are intended for a specific version of a Qualifying Offering. If a Participant subsequently modifies a Qualifying Offering, and The Linux Foundation determines or is informed that the Qualifying Offering no longer conforms to the compatibility tests set forth in the Certification Guide or to users’ expectations for Qualifying Offerings, then upon written notice from The Linux Foundation the Participant must again successfully pass the corresponding self-tests and submit to CNCF a copy of the new self-testing results generated by the testing evaluation program within 30 days following such notice. Otherwise, the Participant’s permitted usage time period will be deemed expired and will be subject to the requirements to remove the Certified KubeEdge Marks and Participant KubeEdge Combinations set forth herein. + +*Conformance Time Period; Later Versions of KubeEdge*. A Participant’s permission to use the Certified KubeEdge Marks and Participant KubeEdge Combinations with its Qualifying Offerings is time-limited. The Certified KubeEdge Marks and Participant KubeEdge Combinations may be used with a Qualifying Offering for a particular version of KubeEdge (e.g., KubeEdge x.y) until the later of: + +- 12 months after the release date of KubeEdge x.y, or +- 9 months after the release date of the next minor release (e.g., KubeEdge x.y+1). + +However, notwithstanding the time limits above, the Participant may also continue to use the Certified KubeEdge Marks and Participant KubeEdge Combinations for any prior version of KubeEdge for which the offering was a Qualifying Offering, for so long as (1) the offering continues to be a Qualifying Offering for each subsequent version of KubeEdge; (2) the Qualifying Offering with the subsequent version of KubeEdge is made generally available to users of the prior version; and (3) the Participant otherwise continues to abide by the terms of the Conformance Program. + +Example: A Participant’s Qualifying Offering passed the self-tests for KubeEdge v1.9. Over the next two years, the Participant continued to ensure that passing self-tests for the Qualifying Offering were submitted to and accepted by CNCF for subsequent KubeEdge versions through the then-current version, KubeEdge v1.12. The Participant may continue to use the applicable Certified KubeEdge Marks with the Qualifying Offering for KubeEdge 1.9, even though the time limit described above for v1.9 has expired. + +*Removal of Certified KubeEdge Marks and Participant KubeEdge Combinations at End of Participation*. After the expiration of the permitted usage time period (if the Participant does not continue to ensure that it passes the self-tests for subsequent versions of KubeEdge) or other termination of participation in the Conformance Program, a Participant may no longer use the Certified KubeEdge Marks or Participant KubeEdge Combinations in new materials, and must remove the Certified KubeEdge Marks and Participant KubeEdge Combinations from existing marketing materials and websites within 30 days. + +#### Use of the Certified KubeEdge Marks and Participant KubeEdge Combinations + +Use of the Certified KubeEdge Marks and Participant KubeEdge Combinations is only permitted for Participants who have satisfied the requirements of the Conformance Program; is only permitted in connection with its Qualifying Offerings; and is only permitted for the usage time periods specified above. No other rights to use the Certified KubeEdge Marks or the KubeEdge Word Mark are permitted hereunder, except to the extent permitted under "fair use" or other applicable law. + +All use of the Certified KubeEdge Marks and Participant KubeEdge Combinations is subject to the LF Trademark Usage Guidelines, which are incorporated herein. + +Additionally, the following specific rules also apply to use of the Certified KubeEdge Marks and Participant KubeEdge Combinations: + +- **Do** follow the requirements set forth in the Branding Guide when using the Certified KubeEdge Marks. +- **Do** specify the version of KubeEdge for which a Qualifying Offering has been self-tested, in a location near the Participant’s use of the Certified KubeEdge Marks. +- **Do** include appropriate notices of The Linux Foundation’s ownership of the Certified KubeEdge Marks and the KubeEdge Word Mark. +- **Do not** use the Certified KubeEdge Marks or Participant KubeEdge Combinations except as permitted pursuant to the Conformance Program’s terms. For example, do not use the Certified KubeEdge Marks or Participant KubeEdge Combinations with a service offering that is not itself a Qualifying Offering, or with a formerly Qualifying Offering whose period of use has expired. +- **Do not** be misleading about the nature or scope of certification or conformance. **Do not** state or imply that the Participant’s participation in the Conformance Program constitutes The Linux Foundation’s or CNCF’s endorsement of a Participant or its goods or services. For example, do not say that a Participant or a Qualifying Offering has been "certified by CNCF" or "certified by The Linux Foundation," or that "The Linux Foundation has certified Product XYZ." +- **Do not** state or imply that there are different degrees of certification or conformance. For example, do not say that a Qualifying Offering is "more certified" than another, or that it was "certified before" another Participant. However, this does not prohibit use of Certified KubeEdge Marks designated for Profiles that a Qualifying Offering has passed, as described above. + +#### Additional Terms for Use of Participant KubeEdge Combinations + +In addition to the rules set forth above and in the LF Trademark Usage Guidelines, the following specific rules also apply to use of Participant KubeEdge Combinations: + +- **Do** use the "circle R" symbol adjacent to the end of the KubeEdge Word Mark within the first prominent appearance of the Participant KubeEdge Combination, for example: XYZ KubeEdge® Engine. +- **Do** include the following attribution statement in a conspicuous location near the first prominent appearance of the Participant KubeEdge Combination: KubeEdge® is a registered trademark of The Linux Foundation in the United States and other countries, and is used pursuant to a license from The Linux Foundation. +- **Do** use the Participant KubeEdge Combination only in plain text form, in a similar font to the surrounding text (e.g., in the same font and style used for other header or body text on a web page). +- **Do not** use the Participant KubeEdge Combination in a stylized form, or incorporate the Participant KubeEdge Combination or KubeEdge Word Mark into a custom logo. + +During and after the term of Participant’s participation in the Conformance Program, Participant will not, anywhere in the world, (1) register or attempt to register a mark that contains or consists of the KubeEdge Word Mark or Participant KubeEdge Combination; or (2) challenge, oppose, petition to cancel or otherwise attack or limit the KubeEdge Word Mark (whether its validity, enforceability, scope of coverage or otherwise) or The Linux Foundation’s exclusive ownership thereof. Performing either of the preceding actions shall constitute a material breach of the terms of the Conformance Program, resulting in termination of Participant’s participation in the Conformance Program without further notice and without opportunity to cure. + +Participant will promptly cooperate with The Linux Foundation, and comply with its requests and instructions, regarding use of the Participant KubeEdge Combination. The Linux Foundation shall have a reasonable right to request copies of and to inspect products and advertising distributed by Participant as an aid to enforcing the KubeEdge Word Mark and its rights hereunder, as a part of policing the KubeEdge Word Mark. Participant will promptly cease any use of the Participant KubeEdge Combination that The Linux Foundation determines may negatively impact the validity, enforceability or scope of coverage of the KubeEdge Word Mark. + +Participant will not otherwise combine the KubeEdge Word Mark with its own marks, other than Participant KubeEdge Combinations used in full compliance with the terms of the Conformance Program. + +#### Participant Marks + +Each Participant hereby grants to The Linux Foundation a royalty-free, worldwide license to use and display the names and logos of Participant and its Qualifying Offerings (the "Participant Marks") in connection with The Linux Foundation’s operation of the Conformance Program and its marketing and promotion of Participant’s participation in the Conformance Program. Each Participant represents and warrants that it possesses all rights necessary to make this license grant. The Linux Foundation will comply with any reasonable requirements regarding use of the Participant Marks about which the Participant notifies The Linux Foundation, and will correct any identified misuse within a reasonable time following Participant’s notification. Nothing in this section shall require The Linux Foundation to make any use of the Participant Marks. As between The Linux Foundation and Participant, any goodwill associated with the foregoing use of the Participant Marks shall inure to the benefit of the Participant. + +#### Term and Termination + +*Term*. The term of a Participant’s participation in the Conformance Program shall begin upon the Participant’s submission (by email to [conformance@cncf.io](mailto:conformance@cncf.io) or other method made available by The Linux Foundation), and The Linux Foundation’s receipt and acceptance, of a fully-completed Participation Form signed by the Participant. + +*Termination*. Either The Linux Foundation or the Participant may terminate Participant’s participation in the Conformance Program upon written notice to the other. Additionally, Participant’s participation in the Conformance Program shall automatically terminate (unless otherwise agreed by The Linux Foundation in writing) upon the occurrence of: (1) the Participant’s breach of the terms of the Conformance Program, if such breach is incurable or, if curable, is not cured within 15 days thereafter; or (2) the expiration of the usage time period for all of the Participant’s Qualifying Offerings, if the Participant does not confirm any new Qualifying Offering within 30 days thereafter. + +*Effect of Termination; Survival*. Upon the termination of Participant’s participation in the Conformance Program, the Participant may no longer use the Certified KubeEdge Marks or Participant KubeEdge Combinations and must remove them from existing marketing materials and websites. The sections of this agreement entitled "Participant Marks", "Disclaimer of Warranties", "Limitation of Liability", "Indemnification" and "Miscellaneous" shall survive any such termination. + +#### Disclaimer of Warranties + +THE LINUX FOUNDATION MAKES NO EXPRESS OR IMPLIED WARRANTIES WITH RESPECT TO THE CONFORMANCE PROGRAM, THE CERTIFIED KUBEEDGE MARKS OR PARTICIPANT KUBEEDGE COMBINATIONS, EITHER TO PARTICIPANT OR TO ANY THIRD PARTY, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, OR NON-INFRINGEMENT OF ANY THIRD PARTY INTELLECTUAL PROPERTY RIGHTS. THE DISCLAIMERS AND INDEMNITIES SET FORTH IN THE LF TRADEMARK USAGE GUIDELINES ARE ALSO DEEMED TO BE INCORPORATED HEREIN. A PARTICIPANT’S PARTICIPATION IN THE CONFORMANCE PROGRAM DOES NOT CONSTITUTE ANY FORM OF ENDORSEMENT BY THE LINUX FOUNDATION OF ANY PARTICIPANT OR ITS QUALIFYING OFFERINGS, AND THE LINUX FOUNDATION MAKES NO EXPRESS OR IMPLIED WARRANTIES WITH RESPECT THERETO. THE CONFORMANCE PROGRAM IS OFFERED "AS-IS", "AS-AVAILABLE" AND "WITH ALL FAULTS." + +#### Limitation of Liability + +TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, THE LINUX FOUNDATION SHALL NOT BE LIABLE FOR ANY DAMAGES, INCLUDING DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL, EXEMPLARY OR PUNITIVE DAMAGES, ARISING OUT OF OR RELATING TO THE CONFORMANCE PROGRAM OR ANY PARTICIPANT’S PARTICIPATION THEREIN. IF ANY DAMAGES ARE NOT EXCLUDED UNDER APPLICABLE LAW PURSUANT TO THE PRECEDING SENTENCE, THEN IN NO EVENT SHALL THE LINUX FOUNDATION’S LIABILITY FOR ANY SUCH DAMAGES EXCEED THE FEES PAID TO THE LINUX FOUNDATION BY THE PARTICIPANT SOLELY FOR PARTICIPATION IN THE CONFORMANCE PROGRAM. THE FOREGOING LIMITATIONS OF LIABILITY ARE AN ESSENTIAL BASIS OF THE LINUX FOUNDATION’S DECISION TO OFFER THE CONFORMANCE PROGRAM AND THE PARTICIPANT’S PARTICIPATION THEREIN, AND SHALL APPLY REGARDLESS OF THE LEGAL THEORY UPON WHICH DAMAGES MAY BE CLAIMED; REGARDLESS OF WHETHER A PARTY KNEW OR SHOULD HAVE KNOWN OF THE POSSIBILITY OF SUCH DAMAGES; AND REGARDLESS OF WHETHER THE FOREGOING LIMITATIONS OF LIABILITY CAUSE ANY REMEDY TO FAIL IN ITS ESSENTIAL PURPOSE. + +#### Indemnification + +By electing to participate in the Conformance Program or to make any use of the Certified KubeEdge Marks or Participant KubeEdge Combinations, each Participant agrees to indemnify, defend and hold harmless The Linux Foundation, together with its affiliates and its and their respective employees, officers and directors (collectively, the "Indemnified Parties") from and against any and all losses, liabilities, damages and penalties, and all related costs and expenses (including reasonable attorneys' fees) arising from (1) the Participant’s breach of the terms of the Conformance Program, (2) any third party claims that may arise in any manner by reason of Participant’s use of the Certified KubeEdge Marks or the Participant KubeEdge Combination with the Participant’s Qualifying Offerings or the Participant’s other goods and services, and (3) any third-party claim relating to the Participant’s Qualifying Offerings or the Participant’s other goods and services, including without limitation any product liability claim. + +#### Entire Agreement; Modifications + +These Terms and Conditions, together with the other documents referenced herein, constitute the entire agreement between any Participant and The Linux Foundation regarding Participant’s participation in the Conformance Program. These Terms and Conditions and the other documents referenced herein may be modified from time to time by The Linux Foundation, and updated versions will be placed on the CNCF website. Participant’s continued participation in the Conformance Program or continued use of the Certified KubeEdge Marks or Participant KubeEdge Combinations constitutes Participant’s acceptance of all such modifications. + +#### Miscellaneous + +All notices to be sent to The Linux Foundation hereunder should be sent to [conformance@cncf.io](mailto:conformance@cncf.io), unless otherwise specified herein. The Participant and The Linux Foundation agree that the Conformance Program does not create a partnership or joint venture between them or any other Participant. \ No newline at end of file From 3e60307cb9621ca52e7841c37f85f73b95b84eee Mon Sep 17 00:00:00 2001 From: Shelley-BaoYue Date: Wed, 28 Jun 2023 20:38:31 +0800 Subject: [PATCH 2/2] add node conformance Signed-off-by: Shelley-BaoYue --- conformance/README.md | 4 +- conformance/instructions.md | 55 ++++++++--------- nodeconformance/README.md | 9 +++ nodeconformance/instructions.md | 106 ++++++++++++++++++++++++++++++++ 4 files changed, 144 insertions(+), 30 deletions(-) create mode 100644 nodeconformance/README.md create mode 100644 nodeconformance/instructions.md diff --git a/conformance/README.md b/conformance/README.md index 4903e6d..38ecdc2 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -4,6 +4,6 @@ With the iteration of the KubeEdge version, the KubeEdge conformance test cases ### How to test -As prerequisites, please read the document `Certified_KubeEdge_Terms.md`. +As prerequisites, please read the document [Certified_KubeEdge_Terms.md](terms-conditions/Certified_KubeEdge_Terms.md). -Please refer to the instructions.md document to run the tests, and before starting the test. \ No newline at end of file +Please refer to the `instructions.md` document to run the tests, and before starting the test. \ No newline at end of file diff --git a/conformance/instructions.md b/conformance/instructions.md index a485099..9d111c0 100644 --- a/conformance/instructions.md +++ b/conformance/instructions.md @@ -15,10 +15,10 @@ KubeEdge has provided the docker image of the conformance test, which contains t ### Launch the conformance test container -Pull conformance test image `kubeedge/conformance-test` +Pull conformance test image `kubeedge/conformance` ``` -$ docker pull kubeedge/conformance-test +$ docker pull kubeedge/conformance ``` Or build the image locally. @@ -37,23 +37,23 @@ docker run --env KUBECONFIG=/root/.kube/config --env RESULTS_DIR=/tmp/results - Description of container environment variables: -| Environment variables | The corresponding ginkgo parameters | Parameters description | -| --------------------- | ----------------------------------- | ------------------------------------------------------------ | -| E2E_DRYRUN | dryRun | If set, ginkgo will walk the test hierarchy without actually running anything. Best paired with -v. | +| Environment variables | The corresponding ginkgo parameters | Parameters description | +|-----------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------------------------| +| E2E_DRYRUN | dryRun | If set, ginkgo will walk the test hierarchy without actually running anything. Best paired with -v. | | E2E_SKIP | skip | If set, ginkgo will only run specs that do not match this regular expression. Can be specified multiple times, values are ORed. | -| E2E_FOCUS | focus | If set, ginkgo will only run specs that match this regular expression. Can be specified multiple times, values are ORed. | -| RESULTS_DIR | NA | Output report path, default /tmp/results | -| REPORT_PREFIX | NA | Report file prefix, optional | -| IMAGE_URL | NA | The name of the image used by the test case | -| TEST_WITH_DEVICE | NA | Whether to test device. The default value is false | -| GINKGO_BIN | NA | `ginkgo` binary path，default /usr/local/bin/ginkgo | -| E2E_EXTRA_ARGS | NA | Extra arguments | -| KUBECONFIG | NA | `kubeconfig` file path | -| TEST_BIN | NA | `e2e.test` binary path，default /usr/local/bin/e2e.test | +| E2E_FOCUS | focus | If set, ginkgo will only run specs that match this regular expression. Can be specified multiple times, values are ORed. | +| RESULTS_DIR | NA | Output report path, default /tmp/results | +| REPORT_PREFIX | NA | Report file prefix, optional | +| IMAGE_URL | NA | The name of the image used by the test case | +| TEST_WITH_DEVICE | NA | Whether to test device. The default value is false | +| GINKGO_BIN | NA | `ginkgo` binary path，default /usr/local/bin/ginkgo | +| E2E_EXTRA_ARGS | NA | Extra arguments | +| KUBECONFIG | NA | `kubeconfig` file path | +| TEST_BIN | NA | `e2e.test` binary path，default /usr/local/bin/e2e.test | ## Uploading -Prepare a PR to https://github.com/kubeedge/community to upload the report files to the directory `conformance`. In the descriptions below, `X.Y` refers to the KubeEdge major and minor version, and `$dir` is a short subdirectory name to hold the results for your product. +Prepare a PR to [https://github.com/kubeedge/community](https://github.com/kubeedge/community) to upload the report files to the directory `conformance`. In the descriptions below, `X.Y` refers to the KubeEdge major and minor version, and `$dir` is a short subdirectory name to hold the results for your product. Description: `Conformance results for vX.Y/$dir` @@ -64,8 +64,7 @@ For simplicity you can submit the tarball or extract the relevant information fr If submitting test results for multiple versions, submit a PR for each product, ie. one PR for vX.Y results and a second PR for vX.Z ``` -vX.Y/$dir/README.md: A script or human-readable description of how to reproduce -your results. +vX.Y/$dir/README.md: A script or human-readable description of how to reproduce your results. vX.Y/$dir/e2e.log: Test log output (from the container kubeedge/conformance). vX.Y/$dir/junit_conformance.xml: Machine-readable test log (from the container kubeedge/conformance). vX.Y/$dir/PRODUCT.yaml: See below. @@ -75,18 +74,18 @@ vX.Y/$dir/PRODUCT.yaml: See below. This file describes your product. It is YAML formatted with the following root-level fields. Please fill in as appropriate. -| Field | Description | -| ----------------------- | ------------------------------------------------------------ | -| `vendor` | Name of the legal entity that is certifying. | -| `name` | Name of the product being certified. | -| `version` | The version of the product being certified (not the version of KubeEdge it runs). | -| `website_url` | URL to the product information website | -| `repo_url` | If your product is open source, this field is necessary to point to the primary GitHub repo containing the source. It's OK if this is a mirror. OPTIONAL | -| `documentation_url` | URL to the product documentation | +| Field | Description | +|-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `vendor` | Name of the legal entity that is certifying. | +| `name` | Name of the product being certified. | +| `version` | The version of the product being certified (not the version of KubeEdge it runs). | +| `website_url` | URL to the product information website | +| `repo_url` | If your product is open source, this field is necessary to point to the primary GitHub repo containing the source. It's OK if this is a mirror. OPTIONAL | +| `documentation_url` | URL to the product documentation | | `product_logo_url` | URL to the product's logo, (must be in SVG, AI or EPS format -- not a PNG -- and include the product name). OPTIONAL. If not supplied, we'll use your company logo. Please see logo [guidelines](https://github.com/cncf/landscape#logos) | -| `type` | Is your product a distribution, hosted platform, or installer. | -| `description` | One sentence description of your offering | -| `contact_email_address` | An email address which can be used to contact maintainers regarding the product submitted and updates to the submission process | +| `type` | Is your product a distribution, hosted platform, or installer. | +| `description` | One sentence description of your offering | +| `contact_email_address` | An email address which can be used to contact maintainers regarding the product submitted and updates to the submission process | Examples: diff --git a/nodeconformance/README.md b/nodeconformance/README.md new file mode 100644 index 0000000..90f5d3d --- /dev/null +++ b/nodeconformance/README.md @@ -0,0 +1,9 @@ +## KubeEdge Node Conformance Tests + +With the iteration of the KubeEdge version, the KubeEdge node conformance test cases will continue to increase, and the release document of the node conformance test cases will be released synchronously. + +### How to test + +As prerequisites, please read the document [Certified_KubeEdge_Terms.md](../conformance/terms-conditions/Certified_KubeEdge_Terms.md). + +Please refer to the `instructions.md` document to run the tests, and before starting the test. \ No newline at end of file diff --git a/nodeconformance/instructions.md b/nodeconformance/instructions.md new file mode 100644 index 0000000..2a8bfda --- /dev/null +++ b/nodeconformance/instructions.md @@ -0,0 +1,106 @@ +# How to submit node conformance results + +## About the KubeEdge node conformance tests + +The standard set of node conformance tests of KubeEdge is currently those defined by the `[sig-node] * [Conformance]` tag in the [KubeEdge e2e](https://github.com/kubeedge/kubeedge/tree/master/tests/e2e) suite, plus a part of kubernetes node conformance tests included at [here](https://github.com/kubeedge/kubeedge/blob/master/build/conformance/kubernetes/kube_node_conformance_test.go). + +## Running + +KubeEdge has provided the docker image of the node conformance test, which contains the scripts and related files of the node conformance test. Follow these steps to perform a node conformance test. + +### Prerequisite + +- At least one master node and one edge node exist on different VMS or physical machines. +- Enable the CloudStream and EdgeStream modules. + +### Launch the conformance test container + +Pull node conformance test image `kubeedge/nodeconformance` + +``` +$ docker pull kubeedge/nodeconformance +``` + +Or build the image locally. + +``` +$ git clone https://github.com/kubeedge/kubeedge.git +$ cd kubeedge +$ docker build -t {image_name}:{tag_name} -f build/conformance/nodeconformance.Dockerfile . +``` + +Examples of running the conformance test containers: + +``` +docker run --env KUBECONFIG=/root/.kube/config --env RESULTS_DIR=/tmp/results -v /root/.kube/config:/root/.kube/config -v /tmp/results:/tmp/results --network host -it kubeedge/nodeconformance +``` + +Description of container environment variables: + +| Environment variables | The corresponding ginkgo parameters | Parameters description | +|-----------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------------------------| +| E2E_DRYRUN | dryRun | If set, ginkgo will walk the test hierarchy without actually running anything. Best paired with -v. | +| E2E_SKIP | skip | If set, ginkgo will only run specs that do not match this regular expression. Can be specified multiple times, values are ORed. | +| E2E_FOCUS | focus | If set, ginkgo will only run specs that match this regular expression. Can be specified multiple times, values are ORed. | +| RESULTS_DIR | NA | Output report path, default /tmp/results | +| REPORT_PREFIX | NA | Report file prefix, optional | +| IMAGE_URL | NA | The name of the image used by the test case | +| TEST_WITH_DEVICE | NA | Whether to test device. The default value is false | +| GINKGO_BIN | NA | `ginkgo` binary path，default /usr/local/bin/ginkgo | +| E2E_EXTRA_ARGS | NA | Extra arguments | +| KUBECONFIG | NA | `kubeconfig` file path | +| TEST_BIN | NA | `e2e.test` binary path，default /usr/local/bin/e2e.test | + +## Uploading + +Prepare a PR to [https://github.com/kubeedge/community](https://github.com/kubeedge/community) to upload the report files to the directory `nodeconformance`. In the descriptions below, `X.Y` refers to the KubeEdge major and minor version, and `$dir` is a short subdirectory name to hold the results for your product. + +Description: `Node Conformance results for vX.Y/$dir` + +### Contents of the PR + +For simplicity you can submit the tarball or extract the relevant information from the tarball to compose your submission. + +If submitting test results for multiple versions, submit a PR for each product, i.e. one PR for vX.Y results and a second PR for vX.Z + +``` +vX.Y/$dir/README.md: A script or human-readable description of how to reproduce your results. +vX.Y/$dir/e2e.log: Test log output (from the container kubeedge/nodeconformance). +vX.Y/$dir/junit_conformance.xml: Machine-readable test log (from the container kubeedge/nodeconformance). +vX.Y/$dir/PRODUCT.yaml: See below. +``` + +#### PRODUCT.yaml + +This file describes your product. It is YAML formatted with the following root-level fields. Please fill in as appropriate. + +| Field | Description | +|-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `vendor` | Name of the legal entity that is certifying. | +| `name` | Name of the product being certified. | +| `version` | The version of the product being certified (not the version of KubeEdge it runs). | +| `website_url` | URL to the product information website | +| `repo_url` | If your product is open source, this field is necessary to point to the primary GitHub repo containing the source. It's OK if this is a mirror. OPTIONAL | +| `documentation_url` | URL to the product documentation | +| `product_logo_url` | URL to the product's logo, (must be in SVG, AI or EPS format -- not a PNG -- and include the product name). OPTIONAL. If not supplied, we'll use your company logo. Please see logo [guidelines](https://github.com/cncf/landscape#logos) | +| `type` | Is your product a distribution, hosted platform, or installer. | +| `description` | One sentence description of your offering | +| `contact_email_address` | An email address which can be used to contact maintainers regarding the product submitted and updates to the submission process | + +Examples: + +``` +vendor: Huawei Cloud +name: IEF +version: v1.x.x +website_url: https://xxx +repo_url: https://xxx +documentation_url: https://xxx/docs +product_logo_url: https://xxx.svg +type: hosted platform +description: "Based on the KubeEdge and kubernetes ecosystems, IEF applies cloud native technologies to edge computing." +``` + +## Contact + +If you have any problems about certifying or conformance testing, please file an issue in the [kubeedge/community](https://github.com/kubeedge/community). Questions and comments can also be sent to the working group's [slack channel](https://kubeedge.slack.com/archives/CKVUCM5ED). SIG Testing is the change controller of the conformance definition. \ No newline at end of file