-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Save the subresults for tmt-report-result #3200
base: main
Are you sure you want to change the base?
Conversation
e927a18
to
3c0f414
Compare
@@ -135,6 +139,12 @@ items: | |||
duration: | |||
$ref: "#/definitions/duration" | |||
|
|||
# TODO: Fixme? e.g. using ref. otherwise I am getting the error: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@happz Aren't we missing the data-path
in the result
and subresult
schemas? Otherwise, I'm still getting the "result format violation" errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Result schema is definitely missing data-path
key, subresults shouldn't have one, IIUIC, data path is a path in which tmt stores test data and info and stuff, so there is no dedicated subpath for a subresult.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you sure about the data-path
in the subresult? I do not have much insight, but it seems the subresults generated by tmt-report-result
are using this field when the tmt-report-results.yaml
gets created:
echo " data-path: ${TMT_TEST_DATA/$TMT_TEST_EXE}" >> "$REPORT_RESULT_OUTPUTFILE"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmm, that does not look right. TMT_TEST_EXE
is ${TMT_TREE/tree/execute}
, TMT_TEST_DATA
is said to be "Path to the directory where test can store logs and other artifacts generated during its execution." and I can't imagine that ${TMT_TEST_DATA/$TMT_TREE/tree/execute}
could be anything sane.
@psss WDYT? There is "test data" for the test, TMT_TEST_DATA
points to it, but data paths for Beakerlib phases? tmt does not manage those, in custom results we do not even have any idea whether the test used any subresult data paths, so we don't touch them, this seems like a mistake to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, do you think we should somehow limit the data-path
using a pattern
in the schema?
For a parent result, the value is always a relative path (data/guest/default-0/tests/example-1/data
). For subresults, the data-path
seems to always start with pattern: "^/"
- it's not relative, but absolute where root starts at 'execute' directory, e.g. /data/guest/default-0/tests/example-1/data
.
Isn't it a bug? Also, the subresult's data-path
seems to be always the same as the parent data-path
(without /
prefix).
3c0f414
to
21f4b42
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've tried to change the method docstrings, releases.rst
and report-result.fmf
story to reflect the subresults feature. Are there any other places where I should document this feature?
Let me know if you need any further adjustments / tests. Thanks.
918ef90
to
70d2fcf
Compare
/packit build |
794f4fb
to
701b4de
Compare
387636a
to
821c8e1
Compare
821c8e1
to
39181f9
Compare
428d10b
to
9dff2f8
Compare
All the results generated by tmt-report-results become objects of `tmt.result.SubResult`.
7381161
to
fe4b1df
Compare
All the results generated by
tmt-report-results
become objects oftmt.result.SubResult
and they get assigned under a parenttmt.Result
. Also, set theTESTID
andBEAKERLIB_COMMAND_REPORT_RESULT
test environment variables, so every call of beakerlib'srlPhaseEnd
macro becomes a subresult.Related to:
TODOs:
Pull Request Checklist