[RELEASE] Release version 2.16.0

[github-actions]

Release OpenSearch and OpenSearch Dashboards 2.16.0

I noticed that a manifest was automatically created in manifests/2.16.0. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the Release Process OpenSearch Guide.

Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.

Entrance Criteria

Criteria	Status	Description	Comments
Documentation draft PRs are up and in tech review for all component changes	🟢
Sanity testing is done for all components	🟢
Code coverage has not decreased (all new code has tests)	🟢
Release notes are ready and available for all components	🟢	Missing Release Notes OS and OSD components
Roadmap is up-to-date (information is available to create release highlights)	🟢	2.16.0 Roadmap
Release ticket is cut, and there's a forum post announcing the start of the window	🟢
Any necessary security reviews are complete	🟢

OpenSearch 2.16.0 exit criteria status:

Criteria	Status	Description	Comments
Performance tests are run, results are posted to the release ticket and there no unexpected regressions	🟢
No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days	🟢	Coming from PA-RCA Fix in RC6, scan_docker_image.txt
Documentation has been fully reviewed and signed off by the documentation community.	🟢
All integration tests are passing	🟢	Current Status: #4771 (comment) as of 2024/08/07
Release blog is ready	🟢	Related PR's opensearch-project/project-website#3140

OpenSearch-Dashboards 2.16.0 exit criteria status:

Criteria	Status	Description	Comments
Documentation has been fully reviewed and signed off by the documentation community	🟢
No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days	🟢	scan_docker_image.txt
All integration tests are passing	🟢	Current Status: #4771 (comment) of 2024/08/07
Release blog is ready	🟢	Related PR's opensearch-project/project-website#3140

Preparation

• Release manager assigned.
  • prudhvigodithi
  • Divyaasm
  • bshien
• Update release page on the website with release manager and release issue details. Related PR: [Release 2.16.0] Update the website with 2.16.0 release information project-website#3073
• Existence of label in each component repo. For more information check the release-label section.
• Increase the build frequency. Related PR [Release 2.16.0] Increase the build frequency and onboard the test manifests. #4853
• Release Issue.
• New plugins:
  * Query Insights plugin: [On-boarding]: Query Insights plugin #4826

Campaigns

• Component Release Issue.
• Release Campaigns.

Release Branch and Version Increment - Starts 2024/07/19

• Core Release Branch.
• Core Version Increment.
  * [Forwardport] Bump 2.x to 2.16.0 (#14216) OpenSearch#14227
  * Bump 2.x to 2.16.0 OpenSearch#14216
  * Bump 2.x to 2.16.0 OpenSearch-Dashboards#7012
• Components Release Branch.
• Components Version Increment.
  * OpenSearch Plugins
  * OpenSearch Dashboards Plugins

Feature Freeze - Ends 2024/07/22

• OpenSearch / OpenSearch-Dashboards core and components teams finalize their features.

Code Complete - Ends 2024/07/22 Ends 2024/07/23

• Mark this as done once the Code Complete is reviewed.
• Create/Verify pull requests to add each component to relase input manifests/2.16.0/opensearch-2.16.0.yml and manifests/2.16.0/opensearch-dashboards-2.16.0.yml.

Release Candidate Creation and Testing - Starts 2024/07/23 Starts 2024/07/24 after 4PM PDT

• Generate Release Candidate.
  • RC1 Generate on 2024/07/24: [RELEASE] Release version 2.16.0 #4771 (comment)
  • RC2 Generate on 2024/07/25: [RELEASE] Release version 2.16.0 #4771 (comment)
  • RC3 2024/07/26: [RELEASE] Release version 2.16.0 #4771 (comment)
  • RC4 2024/07/30: [RELEASE] Release version 2.16.0 #4771 (comment)
  • RC5 2024/07/30 Started at 5PM PDT: [RELEASE] Release version 2.16.0 #4771 (comment).
  • RC6 2024/08/1 Started at 7:30PM PDT [RELEASE] Release version 2.16.0 #4771 (comment).
  • RC7 2024/08/2 Started at 8:15 PM PST [RELEASE] Release version 2.16.0 #4771 (comment).
  • RC8 2024/08/5 Started at 12:30 PM PST [RELEASE] Release version 2.16.0 #4771 (comment).
  • RC9 Started at 2024/08/06 5:00 PM PST [RELEASE] Release version 2.16.0 #4771 (comment)
• Integ Test TAR.
• Integ Test RPM.
• Docker Build and Scan.
• Backwards Compatibility Tests.
• Windows Integration Test.
• Broadcast and Communication.
• Release Candidate Lock.

Performance testing validation - Starts 2024/07/23

• Post the benchmark-tests
• Longevity tests do not show any issues.

Pre Release - 2024/08/05 2024/08/07

• Release Labeled Issues.
• Go or No-Go.
• Promote Repos. Part of the central promotion workflow, Related build link
• Promote artifacts. Part of the central promotion workflow, Related build link
• Release Notes. Related PR for consolidated release notes [Release 2.16.0] Consolidated release notes #4917

Release - 2024/08/05 2024/08/07

• Maven Promotion. https://build.ci.opensearch.org/job/publish-to-maven/37/console
• Docker Promotion.
• Release Validation.
• Collaboration with the Project Management Team.

Release Checklist.

Release Checklist

Pre-Release activities

Part of the central promotion workflow, Related build link

• Promote Repos.
  • • OS
  • • OSD
• Promote Artifacts.
  • • Windows
  • • Linux Debian
  • • Linux RPM
  • • Linux TAR
• Consolidated Release Notes.

Release activities

• Docker Promotion.
• Release Validation part 1.
  • • OpenSearch and OpenSearch Dashboard Validation.
  • • Validate the native plugin installation.
• Merge consolidated release notes PR. Related PR: [Release 2.16.0] Consolidated release notes #4917
• Website and Documentation Changes. Related PR: [Release] 2.16 Release artifact volley project-website#3140
  • • Merge staging website PR. ([Release] 2.16 Release artifact volley project-website#3140)
  • • Promote the website changes to prod. ([prod] 2.16 release project-website#3149 (review), Add 2.16 to version selector on main documentation-website#7941)
  • • Add website alert.
• Release Validation part 2. (Part of the central promotion workflow, Related build link)
  • • Validate the artifact download URL's and signatures.
• Release Validation part 3. (Part of the central promotion workflow, Related build link)
  • • Trigger the validation build (Search for Completed validation for <> in the logs).
• Maven Promotion. (Part of the central promotion workflow, Related build link)
• Publish blog posts.
• Advertise on Social Media.
• Post on public slack and Github Release issue.

Post-Release activities

• Release Tags.
  OpenSearch: https://build.ci.opensearch.org/job/distribution-release-tag-creation/115/console
  OpenSearch Dashboards: https://build.ci.opensearch.org/job/distribution-release-tag-creation/116/console
• Input Manifest Update.
  Related Workflow: https://build.ci.opensearch.org/job/release-manifest-commit-lock/49/
  Related PR: [2.16.0] Manifest Commit Lock with action UPDATE_TO_TAGS #4928
• Decrease the Build Frequency.
• OpenSearch Build Release notes. [Release 2.16.0] Consolidated release notes #4917
• Retrospective Issue. [Retrospective] Release Version 2.16.0 #4847
• Helm and Ansible Playbook release.
  Helm: [Release 2.16.0] Helm charts update helm-charts#569
  Ansible: [Release 2.16.0] Update the OS and OSD version to 2.16.0 ansible-playbook#165
• Upcoming Release Preparation.

Post Release

• Release Tags.
  OpenSearch: https://build.ci.opensearch.org/job/distribution-release-tag-creation/115/console
  OpenSearch Dashboards: https://build.ci.opensearch.org/job/distribution-release-tag-creation/116/console
• Input Manifest Update.
  Related Workflow: https://build.ci.opensearch.org/job/release-manifest-commit-lock/49/
  Related PR: [2.16.0] Manifest Commit Lock with action UPDATE_TO_TAGS #4928
• OpenSearch Build Release notes. [Release 2.16.0] Consolidated release notes #4917
• Decrease the Build Frequency.
• Retrospective Issue. [Retrospective] Release Version 2.16.0 #4847
• Helm and Ansible Playbook release.
  Helm: [Release 2.16.0] Helm charts update helm-charts#569
  Ansible: [Release 2.16.0] Update the OS and OSD version to 2.16.0 ansible-playbook#165
• Upcoming Release Preparation.

Components

OpenSearch

Component	Issues tagged with 2.16.0
opensearch-project/OpenSearch#13547
opensearch-project/common-utils#652
opensearch-project/job-scheduler#627
opensearch-project/security#4318
opensearch-project/k-NN#1690
opensearch-project/geospatial#656
opensearch-project/cross-cluster-replication#1384
opensearch-project/ml-commons#2406
opensearch-project/neural-search#732
opensearch-project/notifications#919
opensearch-project/observability#1827
opensearch-project/reporting#997
opensearch-project/sql#2656
opensearch-project/asynchronous-search#570
opensearch-project/anomaly-detection#1204
opensearch-project/alerting#1545
opensearch-project/security-analytics#1018
opensearch-project/index-management#1170
opensearch-project/performance-analyzer#653
opensearch-project/performance-analyzer-rca#550
opensearch-project/custom-codecs#143
opensearch-project/flow-framework#706
opensearch-project/skills#312

OpenSearch Dashboards

Component	Issues tagged with 2.16.0
opensearch-project/OpenSearch-Dashboards#6718
opensearch-project/opensearch-dashboards-functional-test#1269
opensearch-project/dashboards-observability#1820
opensearch-project/dashboards-reporting#352
opensearch-project/dashboards-visualizations#366
opensearch-project/dashboards-query-workbench#322
opensearch-project/dashboards-maps#619
opensearch-project/anomaly-detection-dashboards-plugin#756
opensearch-project/ml-commons-dashboards#327
opensearch-project/index-management-dashboards-plugin#1059
opensearch-project/dashboards-notifications#204
opensearch-project/alerting-dashboards-plugin#948
opensearch-project/security-analytics-dashboards-plugin#1005
opensearch-project/security-dashboards-plugin#1938
opensearch-project/dashboards-search-relevance#393
opensearch-project/dashboards-assistant#189

Documentation

Component	Issues tagged with 2.16.0
https://github.com/opensearch-project/documentation-website/

Legend

Symbol	Meaning
🟢	On track with overall release
🟡	Missed last milestone
🔴	Missed multiple milestones

Related Issues and PR's

• Remove dependency on opensearch build repo libs from custom build.sh security-dashboards-plugin#2033
• Switch 2.16.0 manifest to use al2 docker images #4858
• [Deprecation] Properly deprecate CentOS7 as CI build image/Supported OS and switch to Almalinux8 #4379
• Adding OS matrix changelog and remove CentOS7 as it is approaching EOL documentation-website#7287
• [BUG] Incremental build copy the entire distribution folder from S3 without cleaning up component zips that requires rebuild opensearch-build-libraries#455
• [BUG] Maven failed publishPluginZipPublicationToZipStagingRepository alerting#1599
• [3.0.0][Bug] Jarhell when installing the plugin zip on opensearch core security-analytics#1165
• add task resource tracking service to cluster service OpenSearch#14681
• sync bug fixes from core to the plugin repo query-insights#13
• Move query categorization changes to plugin query-insights#16
• Fix build failure due to patch conflict k-NN#1851
• [Backport 2.x] Ioc upload integ tests and fix update security-analytics#1169
• [BUG] Maven failed publishPluginZipPublicationToZipStagingRepository alerting#1599
• [3.0.0][Bug] Jarhell when installing the plugin zip on opensearch core security-analytics#1165
• remove redundant dependencies alerting#1571
• [Backport 2.x] Fixing build script to only publish alerting zip alerting#1609 (comment)
• [AUTO] Increment version to 2.16.0-SNAPSHOT performance-analyzer#669
• [AUTO] Increment version to 2.16.0.0 index-management-dashboards-plugin#1089
• [AUTO] Increment version to 2.16.0-SNAPSHOT performance-analyzer-rca#559
• [AUTO] Increment version to 2.16.0.0 dashboards-notifications#224 (comment)
• Upgrading org.bouncycastle:bcprov-jdk15to18 to 1.78 performance-analyzer#686
• Add SortResponseProcessor to Search Pipelines OpenSearch#14785
• Add SplitResponseProcessor to Search Pipelines OpenSearch#14800
• Add support for Lucene inbuilt Scalar Quantizer k-NN#1848
• Create listener to refresh search thread resource usage OpenSearch#14832
• [Remote Routing Table] Implement write and read flow for shard diff file. OpenSearch#14684
• Writable warm relocation recovery OpenSearch#14670
• add initial MLInferenceSearchResponseProcessor ml-commons#2688
• [RFC] ML Inference Processors  ml-commons#2173
• Add initial search request inference processor ml-commons#2616
• [Remote Routing Table] Implement write and read flow for shard diff file. OpenSearch#14684
• [Backport 2.16][Backport 2.x] Refactor remote-routing-table service inline with remote state interfaces (#14668) OpenSearch#14886
• [Discover-next] add query enhancements plugin as a core plugin OpenSearch-Dashboards#7212
• [Discover-next] toggle enhancements OpenSearch-Dashboards#7314
• [Discover-next] remove plugin config toggle for data enhancements OpenSearch-Dashboards#7291
• [Discover-Next] Adding datasources support for dataframes OpenSearch-Dashboards#7106
• [WIP] Dataset navigator OpenSearch-Dashboards#7348
• New Query Enhancement UI OpenSearch-Dashboards#7309
• Fix add filter for numeric scripted field OpenSearch-Dashboards#7022
• [Backport 2.x] [Discover-next] (QueryEditorExtensions) change isEnabled to an observable  OpenSearch-Dashboards#7193
• [Backport 2.x] [Discover-next] add datasource container OpenSearch-Dashboards#7164
• Move TopNavLinks to new Nav Bar Discover OpenSearch-Dashboards#7326
• [WIP] Async Queries Followup OpenSearch-Dashboards#6966
• Track progress updates in async querying interceptors OpenSearch-Dashboards#7337
• Add Scripted Field Support in Vis Builder OpenSearch-Dashboards#6974
• [VisBuilder][BUG] Flat render structure in Metric and Table Vis OpenSearch-Dashboards#6674
• [VisBuilder] Add Capability to generate dynamic vega OpenSearch-Dashboards#7288
• [Look&Feel] apply missing pattern guidance to Alerting experience alerting-dashboards-plugin#1004
• Make reroute iteration time-bound for large shard allocations OpenSearch#14848
• Update nextUrl validation to incorporate serverBasePath security-dashboards-plugin#2048
• Make reroute iteration time-bound for large shard allocations OpenSearch#14848
• [Remote Routing Table] Implement write and read flow for shard diff file. OpenSearch#14684
• [Backport 2.x] Star tree mapping changes OpenSearch#14655
• Updating Cluster Stats Optimisation Versions to 2.16 OpenSearch#14914
• [Backport 2.x] Use Lucene provided Persian stem OpenSearch#14894
• [Backport 2.16] Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14920
• Fix IngestServiceTests.testBulkRequestExecutionWithFailures OpenSearch#14918
• [Backport 2.16] Bump braces from 3.0.2 to 3.0.3 dashboards-query-workbench#361
• [Backport 2.16] Fix IngestServiceTests.testBulkRequestExecutionWithFailures OpenSearch#14927
• [Backport 2.16] Optmising AwarenessAllocationDecider for hashmap.get call (#14761) OpenSearch#14925
• [Backport 2.16] Create listener to refresh search thread resource usage (#14832) OpenSearch#14923
• [Backport 2.16] Implement write and read flow for shard diff file. OpenSearch#14922
• Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14761
• [Backport 2.16] Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14920
• [Backport 2.x] Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14919
• [Backport 2.x] Implement write and read flow for shard diff file. OpenSearch#14915
• [MDS] Adds datasource filter for version decoupling security-dashboards-plugin#2051
• Add initial search request inference processor ml-commons#2616
• [Backport 2.16] [MDS] Adds datasource filter for version decoupling (#2051) security-dashboards-plugin#2053
• [Backport 2.16] Make reroute iteration time-bound for large shard allocations OpenSearch#14954
• [Backport 2.16] Create new IndexInput for multi part upload OpenSearch#14937

[prudhvigodithi]

Current Release Status: 2024/08/06

• The RC8 (OpenSearch 10150 / OpenSearch Dashboards 7858 ) can be a final release candidate for 2.16.0 release. We can finalize in todays GO/NO-GO voting call.

• Coming from past 2.15.0 cross-cluster-replication will not run on deb/rpm as it needs multi cluster due to [Bug]: CCR plugin remoteIntegTest are failing in deb and rpm distribution due to multiple clusters are not forming #4610. But cross-cluster-replication is green for for other platform and distributions. Refer the Metrics Dashboard Link.

• Coming from the comment alerting is green for integration testing. As per comment the issue is created to track the bug with SampleRemoteMonitorIT tests.

• Integration tests for all other components are green for all platform, architecture and distributions.Refer the Metrics Dashboard Link.
  

Thank you

[prudhvigodithi]

Release Update (2024/08/06)

Based on the votes from GO/NO-GO voting call it is a NO-GO for 2.16 today. The current blocker identified is the Manual Sign-off for the Alerting Plugin.The decision from the meeting was to delay the release by one day, setting the new target release date for OS 2.16 to August 7th 2024.
Thank you

[prudhvigodithi]

Issues/PR's (2024/08/06)

• fix distribution builds alerting#1640
• [Backport 2.16] fix distribution builds (#1640) alerting#1642
• Alerting commit update: [2.16.0] Manifest Commit Lock with action UPDATE_TO_RECENT_COMMITS #4920

[prudhvigodithi]

OpenSearch RC9 2024/08/6 Started at 1:25 PM PST

OS (10154):

Triggered the OpenSearch build with a commit change from alerting, Related PR.
https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch/detail/distribution-build-opensearch/10154/pipeline/

OSD (7858):

Using the same RC7 build for OpenSearch Dashboards as there is no change with components.

Thank you

[prudhvigodithi]

RC9 Build and Integration Test Results (Started at 2024/08/06 5:00 PM PST)

Coming from RC9

RC9 OpenSearch 10154 / OpenSearch Dashboards 7858 is ready for your test.

OpenSearch - Build 10154
OpenSearch Dashboards - Build 7858

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.16.0.10154
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.16.0.10154
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.16.0.7858
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.16.0.10154

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.16.0.10154 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.16.0.10154
    • Connect command curl http://localhost:9200/
  • Start with security
    • Docker command

    docker pull opensearchstaging/opensearch:2.16.0.10154 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e 'OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!' opensearchstaging/opensearch:2.16.0.10154
    

    • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'

• OpenSearch-Dashboards docker 2.16.0.7858

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.16.0.7858 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.16.0.7858
    • URL http://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.16.0.7858 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.16.0.7858
    • URL http://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 10154 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 7858

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Integration Test Results

• For the detailed integration test failures for RC7, OpenSearch Dashboards (7858) Use this Metrics Dashboard Link.

• For thethe detailed integration test failures for RC9, OpenSearch (10154) Use this Metrics Dashboard Link.

• Find the list of the created AUTOCUT Integration test failure issues here https://github.com/issues?q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.16.0+label%3Aautocut+%5BAUTOCUT%5D+integration+in%3Atitle+

Thank you

[prudhvigodithi]

Current Release Status: 2024/08/07

• The RC9 (OpenSearch 10154 / OpenSearch Dashboards 7858 ) can be a final release candidate for 2.16.0 release. We can finalize in todays GO/NO-GO voting call Wednesday, August 7, 2024 at 8:30 in the morning PDT.

• Coming from past 2.15.0 cross-cluster-replication will not run on deb/rpm as it needs multi cluster due to [Bug]: CCR plugin remoteIntegTest are failing in deb and rpm distribution due to multiple clusters are not forming #4610. But cross-cluster-replication is green for for other platform and distributions and this was the same case in the yesterday status before RC9. Refer the Metrics Dashboard Link.

• For the alerting which was a no-go based on yesterdays status, a new OpenSearch RC9 was generated which has the fix for SampleRemoteMonitorIT tests. The Integration tests also look green for alerting, refer the Metrics Dashboard link. Refer the the related AUTOCUT integration test failure issue for more information.

• Integration tests for all other components are green for all platform, architecture and distributions.Refer the Metrics Dashboard Link.

• Since all the components pass and the AUTOCUT issues does not have any platform, distribution and architecture specific labels (existence of these labels indicates the components have failed in one of platform, distribution and architecture) I have closed all the created Integration failure AUTOCUT issues except for cross-cluster-replication.

Thank you

[prudhvigodithi]

Release Update (2024/08/07)

Thanks everyone for joining the call, based on the votes from GO/NO-GO voting call it is a GO for 2.16 release today. During the meeting we discussed the concerns on CCR Plugin integration tests due to current hard limitation on how rpm/deb runs, posted the same details on the status comment. More details on this topic opensearch-project/cross-cluster-replication#1414 (comment).

The following report is download from Metrics Dashboard .

Thank you

[prudhvigodithi]

Release Update (2024/08/07)

Started the Central Promotion Workflow (Started at ~2:00 PM PST):

https://build.ci.opensearch.org/blue/organizations/jenkins/central-release-promotion/detail/central-release-promotion/11/pipeline/

Docker and ECR validation results

`Pulling public.ecr.aws/opensearchproject/opensearch:2.16.0 from the ecr
2024-08-07 15:35:26 INFO     Image is pulled at local : sha256:85073dd56eee37daea3fcca9b73760320bb937545877883067ecfe97d6433003
 : public.ecr.aws/opensearchproject/opensearch : 2.16.0
2024-08-07 15:35:26 INFO     Image does not exist at local, proceed with Pull from the ecr
2024-08-07 15:35:26 INFO     Pulling public.ecr.aws/opensearchproject/opensearch-dashboards:2.16.0 from the ecr
2024-08-07 15:35:26 INFO     Image is pulled at local : sha256:c8825d47a3634040043f5a95cb30a97d326d37cce0dd586234f3f244e2906d4e`

`Pulling opensearchproject/opensearch:2.16.0 from the dockerhub
2024-08-07 15:31:39 INFO     Image is pulled at local : sha256:85073dd56eee37daea3fcca9b73760320bb937545877883067ecfe97d6433003
 : opensearchproject/opensearch : 2.16.0
2024-08-07 15:31:39 INFO     Image does not exist at local, proceed with Pull from the dockerhub
2024-08-07 15:31:39 INFO     Pulling opensearchproject/opensearch-dashboards:2.16.0 from the dockerhub
2024-08-07 15:33:24 INFO     Image is pulled at local : sha256:c8825d47a3634040043f5a95cb30a97d326d37cce0dd586234f3f244e2906d4e`

` Fetching mainfest from DockerHub
2024-08-07 15:35:27 INFO     "sha256:bd8c6fae35754f0fbb8fd83f30b7aa6e15b3a3a80d413f9ec9ca8abd3a4166e4" <-- DockerHub image repon digest opensearchstaging/opensearch:2.16.0
2024-08-07 15:35:27 INFO     Fetching mainfest from local
2024-08-07 15:35:27 INFO     "sha256:bd8c6fae35754f0fbb8fd83f30b7aa6e15b3a3a80d413f9ec9ca8abd3a4166e4" <-- local image repo digest opensearchproject/opensearch:2.16.0
2024-08-07 15:35:27 INFO     Fetching access token from dockerHub
2024-08-07 15:35:27 INFO     Fetching mainfest from DockerHub
2024-08-07 15:35:28 INFO     "sha256:bc7732427d5988e819be7329e26ba222fcfda1edeae54412724ab9e2b09b2b64" <-- DockerHub image repon digest opensearchstaging/opensearch-dashboards:2.16.0
2024-08-07 15:35:28 INFO     Fetching mainfest from local
2024-08-07 15:35:28 INFO     "sha256:bc7732427d5988e819be7329e26ba222fcfda1edeae54412724ab9e2b09b2b64" <-- local image repo digest opensearchproject/opensearch-dashboards:2.16.0
2024-08-07 15:35:28 INFO     Image digest is validated.

Maven Promotion (Started at ~3:10 PM PST):

https://build.ci.opensearch.org/job/publish-to-maven/37/console

Release Tag Creation (Started at ~3:10 PM PST):

OpenSearch: https://build.ci.opensearch.org/job/distribution-release-tag-creation/115/console
OpenSearch Dashboards: https://build.ci.opensearch.org/job/distribution-release-tag-creation/116/console

Manifests Update to Tags: (Started at ~3:15 PM PST):

Related Workflow: https://build.ci.opensearch.org/job/release-manifest-commit-lock/49/
Related PR: #4928

[prudhvigodithi]

The release process is now complete and the OpenSearch 2.16.0 has been released to the public! 🎉
A big thank you to all the maintainers, contributors, and everyone who participated in this release and made it successful!

[prudhvigodithi]

Hi All,
Thank you for all your support on the 2.16.0 release. We have a retrospective meeting today (August 13th 2024 - 9:00AM PDT). I’ve added some discussion points to the retro board, which can be accessed here. Please feel free to review and contribute additional items based on the release.

Thanks!