[Feature] KubeEdge SIG AI: Benchmarks for Edge-cloud Collaborative Lifelong Learning

[MooreZheng]

Background:
The KubeEdge SIG AI is chartered to facilitate Edge AI applications with KubeEdge. An overview of SIG AI activities can be found in this charter.

KubeEdge-Sedna has released the first edge-cloud collaborative lifelong learning on June 2021, together with a hands-on example and a free playground on Katacoda. The sceme learn coming tasks from the edge based on previously learned tasks on cloud knowledgebase.

Why is this project needed:
According to the survey on edge ai landing challenges, the top 1 advice goes to "providing public datasets, pre-processing and baseline codes to build benchmarks". This advice gets 82.18%, 92.98%, 87.10% and 86.67% votes among all paticipants, the industial, acadamic and student community on edge ai, respectively.

Our Effort starts from the edge-cloud collaborative benchmarks. This project focuses on measuring and validating the desired behaviors for an epoch-making Edge AI scheme, i.e., Edge-cloud Collaborative Lifelong Learning.

What contents are to be added/modified:
This project will help all Edge AI application developers to validate and select the best-matched algorithm of lifelong learning. Parts of the effort are developing test cases on the existing scheme of Edge-cloud Collaborative Lifelong Learning on KubeEdge-Sedna, including interfaces for benchmark specification, algorithms like preprocessing and metrics, and even baselines.

Other information:
Recommended Skills: TensorFlow/Pytorch, Python
Sedna lifelong-learning introduction
Sedna guide
Sedna lifelong-learning proposal
How to contribute Sedna

Related issue for reference (if any):
#58 Add an example of implementation Federated Learning to ReID
#96 Edge AI Benchmark review
#118 Edge AI Benchmark: Consolidate/Prioritize metrics
#119 Edge AI Benchmark: add step by step instructions for users to benchmark their works
#120 Edge AI Benchmark: add leaderboard/forum
#274 KubeEdge SIG AI: Benchmarks for Edge-cloud Joint Inference

---

Several breath-taking Edge-AI scenarios have been prepared for benchmarking: looking forward to seeing your codes involved in real-world robots, outer-space satellites, and industrial production lines!

[MooreZheng]

KubeEdge has launched a new sub-project KubeEdge-Ianvs for distributed collaborative AI benchmarking to handle this issue.

@iszhyang owns a great osop project working on a simulator for this issue! Pls refer to:

• Proposal Pull Request: [OSPP] Proposal for Edge Intelligence Benchmark for Edge-Cloud Collaborative Lifelong Detection ianvs#35
• Code Pull Request: [OSPP] The codes of Edge Intelligence Benchmark for Edge-Cloud Collaborative Lifelong Detection ianvs#39
• Demo Video（in Chinese）on KEAW'22: https://www.bilibili.com/video/BV1nD4y1Y7cJ

A related issue on cloud robotics is available at KubeEdge-Ianvs: kubeedge/ianvs#48