Nightly fast dispatch tests #5770
tt-rkimfast-dispatch-full-regressions-and-models.yaml
on: schedule
build-artifact
/
...
/
build-docker-image
19s
Matrix: build-artifact / build-artifact
Matrix: fd-nightly / fd-nightly
Matrix: fd-nightly / nightly-wh-models
Annotations
17 errors, 11 warnings, and 148 notices
|
fd-nightly / Nightly N150 mistral7b_eth
Process completed with exit code 1.
|
|
fd-nightly / Nightly N300 mistral7b
Process completed with exit code 1.
|
|
fd-nightly / Nightly N300 yolov4
Process completed with exit code 1.
|
|
fd-nightly / Nightly N150 llama3.2-1B
The self-hosted runner: tt-metal-ci-vm-68 lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.
|
|
fd-nightly / FD GS ttnn nightly grayskull
Process completed with exit code 1.
|
|
fd-nightly / FD Common models GS grayskull
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
|
|
fd-nightly / FD Common models GS grayskull
Process completed with exit code 1.
|
|
unsuccessful-reset-cleanup
Unable to reset board successfully, rebooting
|
|
fd-nightly / FD Common models GS grayskull
The operation was canceled.
|
|
fd-nightly / Nightly N300 resnet50
Process completed with exit code 1.
|
|
fd-nightly / FD [Unstable] N300 models wormhole_b0
Process completed with exit code 1.
|
|
fd-nightly / Nightly N150 mamba
The self-hosted runner: tt-metal-ci-vm-80 lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.
|
|
fd-nightly / Nightly N150 mamba
The action 'Run frequent reg tests scripts' has timed out after 50 minutes.
|
|
pcie-cards-are-being-used-cleanup
Tenstorrent cards seem to be in use. Killing PIDs and exiting unsuccessfully. This can happen if a test hung and is normally an issue with the test, rather than infra.
|
|
fd-nightly / Nightly N300 mamba
The self-hosted runner: tt-metal-ci-vm-44 lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.
|
|
fd-nightly / Nightly N300 mamba
The action 'Run frequent reg tests scripts' has timed out after 50 minutes.
|
|
pcie-cards-are-being-used-cleanup
Tenstorrent cards seem to be in use. Killing PIDs and exiting unsuccessfully. This can happen if a test hung and is normally an issue with the test, rather than infra.
|
|
fd-nightly / FD WH N300 pgm dispatch nightly wormhole_b0
No files were found with the provided path: generated/test_reports/. No artifacts will be uploaded.
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 86 %
|
|
disk-usage-after-startup
Disk usage is 86 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 84 %
|
|
disk-usage-after-startup
Disk usage is 84 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 85 %
|
|
disk-usage-after-startup
Disk usage is 85 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 90 %
|
|
disk-usage-after-startup
Disk usage is 90 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 89 %
|
|
disk-usage-after-startup
Disk usage is 89 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 85 %
|
|
disk-usage-after-startup
Disk usage is 85 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 80 %
|
|
disk-usage-after-startup
Disk usage is 80 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 80 %
|
|
disk-usage-after-startup
Disk usage is 80 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 83 %
|
|
disk-usage-after-startup
Disk usage is 83 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 83 %
|
|
disk-usage-after-startup
Disk usage is 83 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 86 %
|
|
disk-usage-after-startup
Disk usage is 86 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 83 %
|
|
disk-usage-after-startup
Disk usage is 83 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 85 %
|
|
disk-usage-after-startup
Disk usage is 85 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 86 %
|
|
disk-usage-after-startup
Disk usage is 86 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 85 %
|
|
disk-usage-after-startup
Disk usage is 85 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 82 %
|
|
disk-usage-after-startup
Disk usage is 82 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 84 %
|
|
disk-usage-after-startup
Disk usage is 84 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
attempting-reset-cleanup
Attempting to reset card(s).
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 85 %
|
|
disk-usage-after-startup
Disk usage is 85 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 87 %
|
|
disk-usage-after-startup
Disk usage is 87 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 73 %
|
|
disk-usage-after-startup
Disk usage is 73 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
disk-usage-before-startup
Disk usage is 85 %
|
|
disk-usage-after-startup
Disk usage is 85 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
disk-usage-before-startup
Disk usage is 76 %
|
|
disk-usage-after-startup
Disk usage is 76 %
|
|
printing-smi-info-startup
Touching and printing out SMI info
|
|
reset-successful-startup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
Artifacts
Produced during runtime
| Name | Size | |
|---|---|---|
|
TTMetal_build_grayskull
|
230 MB |
|
|
TTMetal_build_wormhole_b0
|
230 MB |
|
|
test_reports_0b31dc5d-8569-45af-a44e-3d77cbb746f6
|
643 Bytes |
|
|
test_reports_362d01d8-5c7b-42df-8eaa-5f942ab2b28c
|
1.37 KB |
|
|
test_reports_3f77eff4-0355-4514-92d7-4dbd343db75b
|
5.91 KB |
|
|
test_reports_439193c4-53c8-4c48-acce-b75b180f616a
|
613 Bytes |
|
|
test_reports_4c3e52c8-305a-455e-8534-4298aa0f9838
|
1.13 KB |
|
|
test_reports_4e38af68-f759-4686-8333-39188a372194
|
713 Bytes |
|
|
test_reports_623740cb-35d2-45e0-a7ca-a19fff58f12e
|
5.92 KB |
|
|
test_reports_6d471a7e-d41f-4eb3-91a8-867e64e74027
|
1.02 KB |
|
|
test_reports_700336b8-2586-4b06-b4d0-e149f082576c
|
623 Bytes |
|
|
test_reports_74e9d394-c5b0-4354-8241-2b131e6901f6
|
746 Bytes |
|
|
test_reports_8fcc0bee-2d6c-4e7b-b2bc-0ce714bf1ace
|
4.65 KB |
|
|
test_reports_aad4a583-54ab-4a9d-ac2b-8c9c649ac70e
|
1.37 KB |
|
|
test_reports_ae03bd54-859e-453f-97ca-e44718e67ad3
|
479 Bytes |
|
|
test_reports_b94cbdf3-8b58-4fab-b6ba-c3b00059c4ee
|
4.79 KB |
|
|
test_reports_bd2fa603-e520-4062-910f-6650b8b73983
|
430 Bytes |
|
|
test_reports_d4639eb5-acc6-49cf-890e-4e98a00b7c8a
|
1.84 KB |
|
|
test_reports_e2064211-6bfe-414b-8570-ccc6a31e1c46
|
555 Bytes |
|
|
test_reports_f07d5f7c-3fb9-4c1f-849f-f2eeb74437f9
|
6.61 KB |
|
|
test_reports_f8f2196d-023e-4e32-bd23-7e3dc7bd3f26
|
483 Bytes |
|
|
test_reports_fce20282-c916-441b-9bc9-5dff835962a4
|
571 Bytes |
|