Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conformance & Node Conformance test proposal #144

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conformance & Node Conformance test proposal #144

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6767223
KubeEdge Conformance Tests Proposal
vincentgoat Dec 26, 2022
3e60307
add node conformance
Shelley-BaoYue Jun 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions conformance/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
## KubeEdge Conformance Tests

With the iteration of the KubeEdge version, the KubeEdge conformance test cases will continue to increase, and the release document of the conformance test cases will be released synchronously.

### How to test

As prerequisites, please read the document [Certified_KubeEdge_Terms.md](terms-conditions/Certified_KubeEdge_Terms.md).

Please refer to the `instructions.md` document to run the tests, and before starting the test.
106 changes: 106 additions & 0 deletions conformance/instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# How to submit conformance results

## About the KubeEdge conformance tests

The standard set of conformance tests of KubeEdge is currently those defined by the `[Conformance]` tag in the [KubeEdge e2e](https://github.com/kubeedge/kubeedge/tree/master/tests/e2e) suite, plus a part of kubernetes conformance tests included at [here](https://github.com/kubeedge/kubeedge/blob/master/build/conformance/kubernetes/kube_conformance_test.go).

## Running

KubeEdge has provided the docker image of the conformance test, which contains the scripts and related files of the conformance test. Follow these steps to perform a conformance test.

### Prerequisite

- At least one master node and one edge node exist on different VMS or physical machines.
- Enable the CloudStream and EdgeStream modules.

### Launch the conformance test container

Pull conformance test image `kubeedge/conformance`

```
$ docker pull kubeedge/conformance
```

Or build the image locally.

```
$ git clone https://github.com/kubeedge/kubeedge.git
$ cd kubeedge
$ docker build -t {image_name}:{tag_name} -f build/conformance/Dockerfile .
```

Examples of running the conformance test containers:

```
docker run --env KUBECONFIG=/root/.kube/config --env RESULTS_DIR=/tmp/results -v /root/.kube/config:/root/.kube/config -v /tmp/results:/tmp/results --network host -it kubeedge/conformance-test
```

Description of container environment variables:

| Environment variables | The corresponding ginkgo parameters | Parameters description |
|-----------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|
| E2E_DRYRUN | dryRun | If set, ginkgo will walk the test hierarchy without actually running anything. Best paired with -v. |
| E2E_SKIP | skip | If set, ginkgo will only run specs that do not match this regular expression. Can be specified multiple times, values are ORed. |
| E2E_FOCUS | focus | If set, ginkgo will only run specs that match this regular expression. Can be specified multiple times, values are ORed. |
| RESULTS_DIR | NA | Output report path, default /tmp/results |
| REPORT_PREFIX | NA | Report file prefix, optional |
| IMAGE_URL | NA | The name of the image used by the test case |
| TEST_WITH_DEVICE | NA | Whether to test device. The default value is false |
| GINKGO_BIN | NA | `ginkgo` binary path，default /usr/local/bin/ginkgo |
| E2E_EXTRA_ARGS | NA | Extra arguments |
| KUBECONFIG | NA | `kubeconfig` file path |
| TEST_BIN | NA | `e2e.test` binary path，default /usr/local/bin/e2e.test |

## Uploading

Prepare a PR to [https://github.com/kubeedge/community](https://github.com/kubeedge/community) to upload the report files to the directory `conformance`. In the descriptions below, `X.Y` refers to the KubeEdge major and minor version, and `$dir` is a short subdirectory name to hold the results for your product.

Description: `Conformance results for vX.Y/$dir`

### Contents of the PR

For simplicity you can submit the tarball or extract the relevant information from the tarball to compose your submission.

If submitting test results for multiple versions, submit a PR for each product, ie. one PR for vX.Y results and a second PR for vX.Z

```
vX.Y/$dir/README.md: A script or human-readable description of how to reproduce your results.
vX.Y/$dir/e2e.log: Test log output (from the container kubeedge/conformance).
vX.Y/$dir/junit_conformance.xml: Machine-readable test log (from the container kubeedge/conformance).
vX.Y/$dir/PRODUCT.yaml: See below.
```

#### PRODUCT.yaml

This file describes your product. It is YAML formatted with the following root-level fields. Please fill in as appropriate.

| Field | Description |
|-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `vendor` | Name of the legal entity that is certifying. |
| `name` | Name of the product being certified. |
| `version` | The version of the product being certified (not the version of KubeEdge it runs). |
| `website_url` | URL to the product information website |
| `repo_url` | If your product is open source, this field is necessary to point to the primary GitHub repo containing the source. It's OK if this is a mirror. OPTIONAL |
| `documentation_url` | URL to the product documentation |
| `product_logo_url` | URL to the product's logo, (must be in SVG, AI or EPS format -- not a PNG -- and include the product name). OPTIONAL. If not supplied, we'll use your company logo. Please see logo [guidelines](https://github.com/cncf/landscape#logos) |
| `type` | Is your product a distribution, hosted platform, or installer. |
| `description` | One sentence description of your offering |
| `contact_email_address` | An email address which can be used to contact maintainers regarding the product submitted and updates to the submission process |

Examples:

```
vendor: Huawei Cloud
name: IEF
version: v1.x.x
website_url: https://xxx
repo_url: https://xxx
documentation_url: https://xxx/docs
product_logo_url: https://xxx.svg
type: hosted platform
description: "Based on the KubeEdge and kubernetes ecosystems, IEF applies cloud native technologies to edge computing."
```

## Contact

If you have any problems about certifying or conformance testing, please file an issue in the [kubeedge/community](https://github.com/kubeedge/community). Questions and comments can also be sent to the working group's [slack channel](https://kubeedge.slack.com/archives/CKVUCM5ED). SIG Testing is the change controller of the conformance definition.
94 changes: 94 additions & 0 deletions conformance/participation-form/Certified_KubeEdge_Form.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
### Certified KubeEdge Conformance Program
#### Participation Form

Complete this form for each Participant (company or other entity) that desires to participate in the Certified KubeEdge Conformance Program and to use the Certified KubeEdge Marks or Participant KubeEdge Combinations. KubeEdge and the Certified KubeEdge Marks are trademarks of The Linux Foundation. Capitalized terms used herein and not otherwise defined shall have the same meanings set forth in the Program Terms.

By signing below and submitting this form to The Linux Foundation (by [DocuSign](#TO-BE-CREATED) or by emailing the [PDF](https://github.com/kubeedge/community/blob/master/sig-testing/conformance/Certified_KubeEdge_Form.pdf) to [[email protected]](mailto:[email protected])):

1. The Participant agrees to the Terms and Conditions of the Certified KubeEdge Conformance Program (the "**Program Terms**"), available at https://github.com/kubeedge/community/blob/master/sig-testing/conformance/Certified_KubeEdge_Terms.pdf>.
2. The Participant confirms that the products and services identified below as Qualifying Offerings have passed all of the self-tests described in the Certification Guide, and are Qualifying Offerings under the Program Terms.
3. The Participant confirms that it has submitted to the Cloud Native Computing Foundation ("**CNCF**") the results of the self-tests prior to its first public use of the Certified KubeEdge Marks associated with the corresponding version of KubeEdge.
4. The Participant confirms that it will either (a) maintain conformance of the Qualifying Offerings with later versions of KubeEdge, or (b) cease use of the Certified KubeEdge Marks and Participant KubeEdge Combinations at the end of the applicable conformance time period described in the Program Terms.
5. The Participant confirms that it has listed below all Participant KubeEdge Combinations that it intends to use with the Qualifying Offerings.
6. The Participant confirms that it will promptly submit an updated Participant Form to The Linux Foundation prior to (a) using the Certified KubeEdge Marks with Qualifying Offerings not listed here, or (b) using Participant KubeEdge Combinations not listed here.
7. I confirm that I am authorized to make the above statements and to submit this form on behalf of the Participant.


#### Participant Information


Company / entity name:

\___________________________________________________

Contact address:

\___________________________________________________

\___________________________________________________

\___________________________________________________

\___________________________________________________

Contact telephone:

\___________________________________________________

Contact email:

\___________________________________________________


Select one:

- [ ] Participant is a member of CNCF.
- [ ] Participant is a non-profit organization.
- [ ] Neither of the above. **Please contact CNCF to discuss fees for participation in the Conformance Program.**

#### Qualifying Offerings

Name, brief description and URLs for more information:

\___________________________________________________________________________

\___________________________________________________________________________

\___________________________________________________________________________

\___________________________________________________________________________


#### Participant KubeEdge Combinations

List all Participant KubeEdge Combinations to be used with the Qualifying Offerings, if any:

(for example, "XYZ KubeEdge" or "XYZ KubeEdge Platform")

\___________________________________________________________________________

\___________________________________________________________________________

\___________________________________________________________________________

\___________________________________________________________________________


#### Conformance Details

Initial Version of KubeEdge for Conformance (e.g., v1.11): _______

Conformance Date: __________________


#### Signed on behalf of Participant by:

```
Signature: __________________________________

Name: __________________________________

Title: __________________________________

Date: __________________________________
```
Loading