Skip to content

Release Schedule

Jump to bottom
George Keishing edited this page May 24, 2021 · 47 revisions

May 21 2021

Community 2.10 Tagging build testing https://jenkins.openbmc.org/job/release-tag/9/

OpenBMC test repository will also tag a branch and release once this activity is completed.

Status: IN PROGRESS

Build: https://jenkins.openbmc.org/job/release-tag/9/
BMC Driver Version: 2.10.0-rc1-dirty    
(Linux witherspoon 5.10.36-fa5949e #1 Thu May 20 16:34:31 UTC 2021 armv6l GNU/Linux)
System: Witherspoon

Defect List:

  1. https://github.com/openbmc/phosphor-certificate-manager/issues/19
  2. https://github.com/openbmc/phosphor-certificate-manager/issues/20
  3. https://github.com/openbmc/phosphor-host-ipmid/issues/171
  4. https://github.com/openbmc/phosphor-host-ipmid/issues/172
  5. https://github.com/openbmc/phosphor-snmp/issues/3

Test Report:

1. Code update from recent upstream master to candidate tag 2.10.0-rc1-dirty: PASS

Test case: 1
Failed   : 0

Redfish Bmc Code Update :: Update the BMC code on a target BMC via... | [32mPASS[0m |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

2. HW CI test(Refer test_lists/HW_CI) : PASS

Test Case: 20
Failed   : 0

==============================================================================
Ipmi & Tests & Redfish                                                | PASS |
20 critical tests, 20 passed, 0 failed
20 tests total, 20 passed, 0 failed
==============================================================================

3. Boot Test with Partition: PASS

Test Case: 13
Failed   : 0

Boot Type                                Total Pass Fail
---------------------------------------- ----- ---- ----
Redfish Power On                             2    2    0
IPMI Power On                                1    1    0
Redfish Power Off                            2    2    0
IPMI Power Off                               1    1    0
IPMI Power Soft                              1    1    0
OBMC Reboot (run)                            1    1    0
REST OBMC Reboot (run)                       1    1    0
OBMC Reboot (off)                            1    1    0
REST OBMC Reboot (off)                       1    1    0
IPMI Power Cycle                             1    1    0
IPMI Power Reset                             1    1    0
Auto Reboot                                  1    1    0
RF SYS GracefulRestart                       1    1    0
========================================================
Totals                                      15   15    0

4. Redfish validator: PASS

Test Case: 1
Failed   : 0

Redfish Service Validator :: Test BMC using https://github.com/DMT... | [32mPASS[0m |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

5. Regression Test bucket (Refer test_lists/witherspoon/regression_bucket_master): FAIL

Test Case: 72
Failed   : 14

==============================================================================
Redfish & Ipmi                                                        | FAIL |
72 critical tests, 58 passed, 14 failed
72 tests total, 58 passed, 14 failed
==============================================================================

Note: Bug opened for the failure. Not a blocker since LDAP services are removed from Witherspoon, but should be working in others.

- Configure_Hostname_And_Verify      [New interface to use, need to check automation]
- Add_Valid_IPv4_Addres_And_Verify   [Working fine, need to check automation]
- Verify_Client_Certificate_Replace
- Verify_CA_Certificate_Replace
- Verify_Client_Certificate_Install
- Verify_CA_Certificate_Install
- Verify_Maximum_CA_Certificate_Install
- Verify_Error_While_Uploading_Same_CA_Certificate
- Verify_CSR_Generation_For_Client_Certificate
- Verify_Expired_Certificate_Install
- Verify_Not_Yet_Valid_Certificate_Install
- Verify_Not_Yet_Valid_Certificate_Replace
- Verify_Certificates_Location_Via_Redfish
- Login_And_Verify_HTTP_Response_Header

bug:

6. IPMI Test bucket: FAIL

Test Case: 86
Failed   : 21

==============================================================================
Ipmi                                                                  | FAIL |
86 critical tests, 65 passed, 21 failed
86 tests total, 65 passed, 21 failed
==============================================================================

Note: Bug opened for the failure. Not a blocker.

- Test_FRU_Device_Name
- Test_Activate_Payload
- Test_Deactivate_Payload
- Test_Get_Payload_Instance_Info
- Test_TPM_Enable_SDR_Info
- Test_Sensor_Threshold_Via_IPMI
- Set_SOL_Enabled
- Set_SOL_Disabled
- Set_Valid_SOL_Privilege_Level
- Set_Invalid_SOL_Privilege_Level
- Set_Invalid_SOL_Retry_Count
- Set_Invalid_SOL_Retry_Interval
- Set_Invalid_SOL_Character_Accumulate_Level
- Set_Invalid_SOL_Character_Send_Threshold
- Verify_SOL_During_Boot
- Verify_Deactivate_Non_Existing_SOL
- Set_Valid_SOL_Retry_Count
- Set_Valid_SOL_Retry_Interval
- Set_Valid_SOL_Character_Accumulate_Level
- Set_Valid_SOL_Character_Send_Threshold
- Verify_Continuous_Activation_And_Deactivation_Of_SOL

bug:

7. BMC Reset test (Loop 20 iteration at Host standby and check for errors): PASS

Test Case: 1
Failed   : 0

Boot Type                                Total Pass Fail
---------------------------------------- ----- ---- ----
Redfish OBMC Reboot (off)                   20   20    0
========================================================
Totals                                      20   20    0


Run Multiple BMC Reset Via Redfish :: Execute multiple reboots via... | [32mPASS[0m |
------------------------------------------------------------------------------
Test Bmc Reset Loop :: Power cycle loop. This is to test where net... | [32mPASS[0m |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================

8. SNMP Test: FAIL

Test Case: 13
Failed   : 6

Configure SNMP Manager With Less Octet IP And Verify :: Configure ... | FAIL |
Allowing the configuration of an invalid SNMP.: 200 != 400
------------------------------------------------------------------------------
Test Bmc Snmp Config :: This testing requires special setup where ... | FAIL |
13 critical tests, 7 passed, 6 failed
13 tests total, 7 passed, 6 failed
==============================================================================

Note: Bug opened for the failure. Not a blocker but negative use cases test failure.

- Configure_SNMP_Manager_With_Less_Octet_IP_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Out_Of_Range_IP_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Empty_Port_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Negative_Port_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Alpha_Port_And_Verify
- Configure_SNMP_Manager_On_BMC_With_Out_Of_Range_Port_And_Verify

bug:

9. BMC web GUI Test: PASS

Manually checked pages and they look intact. If required more operation based function can be checked.

June 9 2020

Community 2.8 Tagging build testing report https://lists.ozlabs.org/pipermail/openbmc/2020-June/021961.html

Nov 20 2019

From Kurt Taylor IRC note:

There isn't an official yocto 2.8 (they are jumping to 3.0) and we won't release openbmc until May 2020 to line up with > the 3.1 yocto release since November was too soon after the August release. > https://github.com/openbmc/openbmc/wiki/Releases

Note: So we won't be releasing 2.8 bellow schedule to align with OpenBMC release.

Nov 08 2019

Community,

Its time for the 2.8 release of OpenBMC test automation which is scheduled on Dec 6th, 2019.

The testing is progressing on the latest master and we are accepting patches only to get a stable release on the following test

  • Local User (Redfish / IPMI )
  • LDAP
  • Firmware update
  • Certificate
  • Network
  • BMC web GUI
  • Others for back compatible

Note: There is no support planned for earlier versions of Python (<3.x) and Robot CLI package releases after OpenBMC test automation 2.8 release.

Clone this wiki locally