Skip failure on task to install XCTestHTMLReport and only publish result to artifact when the task succeeded #2620
Conversation
KangxuanYe
changed the title
| @@ -18,7 +18,10 @@ steps: | |||
| targetType: inline | |||
| script: | | |||
| brew install xctesthtmlreport | |||
There was a problem hiding this comment.
If this fails (I assume this is the command that is failing intermittently?) is there a way to get more verbose logs about what the failure is?
There was a problem hiding this comment.
I think they typically hide logs for us when we install this. Basically the error log it throws to console is the detailed one. Not sure if we can get more detailed log on VM.
| @@ -148,7 +151,7 @@ steps: | |||
| echo "ls -lR ${output}" | |||
| ls -lR "${output}" | |||
| displayName: Preparations for publishing results | |||
| condition: always() | |||
| condition: eq(variables['xctestHtmlReportInstalled'], 'Yes') | |||
There was a problem hiding this comment.
What effect will this have? Will the test results not be published if installation failed? Does that mean that any test failures will be overlooked if the html reporter didn't get installed?
There was a problem hiding this comment.
This condition is to say, only when xctestHtmlReportInstall task successfully be executed, this task will then be executed to publish results to the artifact.
Any test failures will NOT be overlooked. It just affects devs to figure out why test cases fail on CI pipeline with detailed screenshots and logs.
There was a problem hiding this comment.
Can you add a comment explaining that? Thanks!
There was a problem hiding this comment.
just wanted to make sure we are on the same page:
This does look like improvement, i.e, if we don't have xctestHtmlReport tool installed correctly, we shouldn't try to use the tool in later phase. The PR is not fixing existing environment issue, but rather than a pipeline task improvement. is my understanding correct ? @KangxuanYe
| @@ -18,7 +18,10 @@ steps: | |||
| targetType: inline | |||
| script: | | |||
| brew install xctesthtmlreport | |||
|
|
|||
| echo "##vso[task.setvariable variable=xctestHtmlReportInstalled]Yes" | |||
There was a problem hiding this comment.
If you wonder what it does, please check this link: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops#variables-created-in-a-step-used-in-subsequent-step-conditions
Example:
steps:
# This step creates a new pipeline variable: doThing. This variable is available to subsequent steps.
- bash: |
echo "##vso[task.setvariable variable=doThing]Yes"
displayName: Step 1
# This step is able to use doThing, so it uses doThing in its condition
- script: |
# Access the variable from Step 1 as an environment variable.
echo "Value of doThing (as DOTHING env var): $DOTHING."
displayName: Step 2
condition: and(succeeded(), eq(variables['doThing'], 'Yes')) # or and(succeeded(), eq(variables.doThing, 'Yes'))
| @@ -18,7 +18,10 @@ steps: | |||
| targetType: inline | |||
| script: | | |||
| brew install xctesthtmlreport | |||
There was a problem hiding this comment.
I think they typically hide logs for us when we install this. Basically the error log it throws to console is the detailed one. Not sure if we can get more detailed log on VM.
| @@ -148,7 +151,7 @@ steps: | |||
| echo "ls -lR ${output}" | |||
| ls -lR "${output}" | |||
| displayName: Preparations for publishing results | |||
| condition: always() | |||
| condition: eq(variables['xctestHtmlReportInstalled'], 'Yes') | |||
There was a problem hiding this comment.
This condition is to say, only when xctestHtmlReportInstall task successfully be executed, this task will then be executed to publish results to the artifact.
Any test failures will NOT be overlooked. It just affects devs to figure out why test cases fail on CI pipeline with detailed screenshots and logs.
| @@ -18,7 +18,10 @@ steps: | |||
| targetType: inline | |||
| script: | | |||
| brew install xctesthtmlreport | |||
|
|
|||
| echo "##vso[task.setvariable variable=xctestHtmlReportInstalled]Yes" | |||
There was a problem hiding this comment.
Can you add a comment or two explaining what this does and why it is here?
Description
As title says, this PR is to skip failure on task to install XCTestHTMLReport and only publish result to artifact when the task succeeded, considering we encounter the error as below every day.
Main changes in the PR: YML file in iOS
Validation
Hopefully it works when next time we encounter the same issue.
Unit Tests added: No
End-to-end tests added: No
Additional Requirements
Change file added: No