[tests][ci] Refactor ci

[GOODBOY008]

No description provided.

[yuxiqian]

Thanks for @GOODBOY008's great work!

A little suggestion: is it possible to split jobs into sub-workflow files to organize them by categories? Current there are a little too much jobs, and most of them has only one sub-task compile_and_test.

[yuxiqian]

Seems Flink version specification isn't working as expected. Flink 1.20 test job is still running tests on Flink 1.17 (and all other versions).

[GOODBOY008]

@yuxiqian The pr just for test ci, I will add my commit to your pr.We usually check the failed CI jobs in the CI job view on the PR page, so I don't think it's necessary.

[yuxiqian]

Looks good.

Categorized folders aren't that important. Could be done later if it takes too much efforts to do now.

[yuxiqian]

Somehow Elasticsearch connector tests have never been added into CI...

[Jiabao-Sun]

Thanks @GOODBOY008 for this great work.
How about using github action's matrix to control flink's version that can improve the coverage of the flink versions and simplify the Parameterized Tests.

[GOODBOY008]

@Jiabao-Sun
Do you mean specifying the Flink version and passing it to run tests using a custom Maven parameter, while the default Flink version corresponds to the project’s dependency version?

I think we can discuss later.

[Jiabao-Sun]

@Jiabao-Sun Do you mean specifying the Flink version and passing it to run tests using a custom Maven parameter, while the default Flink version corresponds to the project’s dependency version?

I think we can discuss later.

Yes, but we can do it later.
cc @leonardBang