Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report results of the weekly scenario tests publicly as dashboard #4727

Open
mitsudome-r opened this issue May 15, 2024 · 0 comments
Open

Report results of the weekly scenario tests publicly as dashboard #4727

mitsudome-r opened this issue May 15, 2024 · 0 comments
Labels
type:ci Continuous Integration (CI) processes and testing.

Comments

@mitsudome-r
Copy link
Member

mitsudome-r commented May 15, 2024

Description

We should have some public notification about the results of scenario tests without the need of logging in to TIER IV's evaluator console.

Background

Currently, we are doing weekly scenario tests within ODD WG using TIER IV's evaluator functions, and the results are visible under TIER IV's evaluator console. However, you need to create an account and ask admins add give them access to view the results. We should make the result of the weekly scenario tests publicly visible and reported to Autoware GitHub so that developers could know any regression quicker.

Possible Approach

There is WebAutoCLI which can retrieve the results from command line. We can have AWS host which use these commands that retrieves the result of scenario tests periodically and publish the result as dashboard.
We could also create an issue in Autoware GitHub for failing issue

Image

@mitsudome-r mitsudome-r converted this from a draft issue May 15, 2024
@mitsudome-r mitsudome-r added the type:ci Continuous Integration (CI) processes and testing. label May 15, 2024
@xmfcx xmfcx moved this to Todo in Autoware Labs May 30, 2024
@xmfcx xmfcx removed this from Autoware Labs Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:ci Continuous Integration (CI) processes and testing.
Projects
Status: Todo
Development

No branches or pull requests

1 participant