-
Notifications
You must be signed in to change notification settings - Fork 92
Release Schedule
Community 2.10 Tagging build testing https://jenkins.openbmc.org/job/release-tag/9/
OpenBMC test repository will also tag a branch and release once this activity is completed.
Build: https://jenkins.openbmc.org/job/release-tag/9/
BMC Driver Version: 2.10.0-rc1-dirty
(Linux witherspoon 5.10.36-fa5949e #1 Thu May 20 16:34:31 UTC 2021 armv6l GNU/Linux)
System: Witherspoon
Test Report:
1. Code update from recent upstream master to candidate tag 2.10.0-rc1-dirty: PASS
Test case: 1
Failed : 0
Redfish Bmc Code Update :: Update the BMC code on a target BMC via... | [32mPASS[0m |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
2. HW CI test(Refer test_lists/HW_CI) : PASS
Test Case: 20
Failed : 0
==============================================================================
Ipmi & Tests & Redfish | PASS |
20 critical tests, 20 passed, 0 failed
20 tests total, 20 passed, 0 failed
==============================================================================
3. Boot Test with Partition: PASS
Test Case: 13
Failed : 0
Boot Type Total Pass Fail
---------------------------------------- ----- ---- ----
Redfish Power On 2 2 0
IPMI Power On 1 1 0
Redfish Power Off 2 2 0
IPMI Power Off 1 1 0
IPMI Power Soft 1 1 0
OBMC Reboot (run) 1 1 0
REST OBMC Reboot (run) 1 1 0
OBMC Reboot (off) 1 1 0
REST OBMC Reboot (off) 1 1 0
IPMI Power Cycle 1 1 0
IPMI Power Reset 1 1 0
Auto Reboot 1 1 0
RF SYS GracefulRestart 1 1 0
========================================================
Totals 15 15 0
4. Redfish validator: PASS
Test Case: 1
Failed : 0
Redfish Service Validator :: Test BMC using https://github.com/DMT... | [32mPASS[0m |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
5. Regression Test bucket (Refer test_lists/witherspoon/regression_bucket_master): FAIL
Test Case: 72
Failed : 14
==============================================================================
Redfish & Ipmi | FAIL |
72 critical tests, 58 passed, 14 failed
72 tests total, 58 passed, 14 failed
==============================================================================
Note: Bug opened for the failure. Not a blocker since LDAP services are removed from Witherspoon, but should be working in others.
- Configure_Hostname_And_Verify [New interface to use, need to check automation]
- Add_Valid_IPv4_Addres_And_Verify [Working fine, need to check automation]
- Verify_Client_Certificate_Replace
- Verify_CA_Certificate_Replace
- Verify_Client_Certificate_Install
- Verify_CA_Certificate_Install
- Verify_Maximum_CA_Certificate_Install
- Verify_Error_While_Uploading_Same_CA_Certificate
- Verify_CSR_Generation_For_Client_Certificate
- Verify_Expired_Certificate_Install
- Verify_Not_Yet_Valid_Certificate_Install
- Verify_Not_Yet_Valid_Certificate_Replace
- Verify_Certificates_Location_Via_Redfish
- Login_And_Verify_HTTP_Response_Header
bug:
- 1. https://github.com/openbmc/phosphor-certificate-manager/issues/19
- 2. https://github.com/openbmc/phosphor-certificate-manager/issues/20
6. IPMI Test bucket: FAIL
Test Case: 86
Failed : 21
==============================================================================
Ipmi | FAIL |
86 critical tests, 65 passed, 21 failed
86 tests total, 65 passed, 21 failed
==============================================================================
Note: To debug the following failure
- Test_FRU_Device_Name
- Test_Activate_Payload
- Test_Deactivate_Payload
- Test_Get_Payload_Instance_Info
- Test_TPM_Enable_SDR_Info
- Test_Sensor_Threshold_Via_IPMI
- Set_SOL_Enabled
- Set_SOL_Disabled
- Set_Valid_SOL_Privilege_Level
- Set_Invalid_SOL_Privilege_Level
- Set_Invalid_SOL_Retry_Count
- Set_Invalid_SOL_Retry_Interval
- Set_Invalid_SOL_Character_Accumulate_Level
- Set_Invalid_SOL_Character_Send_Threshold
- Verify_SOL_During_Boot
- Verify_Deactivate_Non_Existing_SOL
- Set_Valid_SOL_Retry_Count
- Set_Valid_SOL_Retry_Interval
- Set_Valid_SOL_Character_Accumulate_Level
- Set_Valid_SOL_Character_Send_Threshold
- Verify_Continuous_Activation_And_Deactivation_Of_SOL
7. BMC Reset test (Loop 20 iteration at Host standby and check for errors): PASS
Test Case: 1
Failed : 0
Boot Type Total Pass Fail
---------------------------------------- ----- ---- ----
Redfish OBMC Reboot (off) 20 20 0
========================================================
Totals 20 20 0
Run Multiple BMC Reset Via Redfish :: Execute multiple reboots via... | [32mPASS[0m |
------------------------------------------------------------------------------
Test Bmc Reset Loop :: Power cycle loop. This is to test where net... | [32mPASS[0m |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
8. SNMP Test: FAIL
Test Case: 13
Failed : 6
Configure SNMP Manager With Less Octet IP And Verify :: Configure ... | FAIL |
Allowing the configuration of an invalid SNMP.: 200 != 400
------------------------------------------------------------------------------
Test Bmc Snmp Config :: This testing requires special setup where ... | FAIL |
13 critical tests, 7 passed, 6 failed
13 tests total, 7 passed, 6 failed
==============================================================================
Note: Bug opened for the failure. Not a blocker but negative use cases test failure.
- Configure_SNMP_Manager_With_Less_Octet_IP_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Out_Of_Range_IP_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Empty_Port_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Negative_Port_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Alpha_Port_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Out_Of_Range_Port_And_Verify
bug:
9. BMC web GUI Test: PASS
Manually checked options and pages and they look intact and working.
Community 2.8 Tagging build testing report https://lists.ozlabs.org/pipermail/openbmc/2020-June/021961.html
From Kurt Taylor IRC note:
There isn't an official yocto 2.8 (they are jumping to 3.0) and we won't release openbmc until May 2020 to line up with > the 3.1 yocto release since November was too soon after the August release. > https://github.com/openbmc/openbmc/wiki/Releases
Note: So we won't be releasing 2.8 bellow schedule to align with OpenBMC release.
Community,
Its time for the 2.8 release of OpenBMC test automation which is scheduled on Dec 6th, 2019.
The testing is progressing on the latest master and we are accepting patches only to get a stable release on the following test
- Local User (Redfish / IPMI )
- LDAP
- Firmware update
- Certificate
- Network
- BMC web GUI
- Others for back compatible
Note:
There is no support planned for earlier versions of Python (<3.x) and Robot CLI package releases after OpenBMC test automation 2.8 release.