Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] Release version 1.2.0 #567

Closed
42 tasks done
github-actions bot opened this issue Sep 23, 2021 · 36 comments
Closed
42 tasks done

[RELEASE] Release version 1.2.0 #567

github-actions bot opened this issue Sep 23, 2021 · 36 comments
Assignees
Labels
release v1.2.0 OpenSearch 1.2 version

Comments

@github-actions
Copy link
Contributor

github-actions bot commented Sep 23, 2021

Release OpenSearch and OpenSearch Dashboards 1.2.0

I noticed that a manifest was automatically created in manifests/1.2.0. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee is responsible for driving the release. Please contact them or add a comment on this issue for help. There are linked issues on components of the release where individual components can be tracked.

Release Steps

There are several steps to the release process, these steps are completed as the whole release and components that are behind present risk to the release. The release owner completes the tasks in this ticket, whereas component owners resolve tasks on their ticket.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

Component List

To aid in understanding the state of the release there is a table with status indicating each component state. This is manually updated based on the status of the component issues.

Preparation - Ends Oct 28th

  • Assign this issue to a release owner.
  • Declare a pencils down date for new features to be merged. Nov 2nd is Pencils down
  • Document any new quality requirements or changes.
  • Finalize scope and feature set and update the Public Roadmap.
  • Create a version label in each component repo.
  • Create a release issue in every component repo that links back to this issue.
  • Ensure that all release issues created above are assigned to an owner in the component team.

Engine and Dashboard Development - Ends Nov 2nd

  • Create Jenkins workflows that run daily snapshot builds for OpenSearch.
  • Create Jenkins workflows that run daily snapshot builds for OpenSearch Dashboards.
  • Increment each component version to 1.2.0 and ensure working CI in component repositories.
  • Complete all engine and dashboard features for release on or before Nov 2nd.

All Component Development - Ends Nov 8th

Release testing - Ends Nov 16th

Release - Ends Nov 18th Ends Nov 23

Post Release

Components

Overall status: Post Release 1.2.0 has been released, all post release tasks should be worked on and component release tasks closed out.
Next Step: Closing this release task, ETA Dec 13th.

Component On track Notes
opensearch-project/asynchronous-search#42 🟢 Released
opensearch-project/job-scheduler#66 🟢 Released
opensearch-project/alerting#182 🟢 Released
opensearch-project/cross-cluster-replication#218 🟢 Released
opensearch-project/sql#214 🟢 Released
opensearch-project/security#1459 🟢 Released
opensearch-project/observability#118 🟢 Released
opensearch-project/security-dashboards-plugin#832 🟢 Released
opensearch-project/dashboards-visualizations#26 🟢 Released
opensearch-project/reporting#167 🟢 Released
opensearch-project/OpenSearch-Dashboards#833 🟢 Released
opensearch-project/common-utils#89 🟢 Released
opensearch-project/index-management-dashboards-plugin#98 🟢 Released
opensearch-project/OpenSearch#1317 🟢 All Tasks Complete
opensearch-project/performance-analyzer#65 🟢 All Tasks Complete
opensearch-project/alerting-dashboards-plugin#118 🟢 All Tasks Complete
opensearch-project/k-NN#102 🟢 All Tasks Complete
opensearch-project/index-management#154 🟢 All Tasks Complete
opensearch-project/anomaly-detection-dashboards-plugin#97 🟢 All Tasks Complete
opensearch-project/anomaly-detection#229 🟢 All Tasks Complete
opensearch-project/dashboards-notebooks#81 Not In Release
opensearch-project/notifications#332 Not In Release
Legend

| Symbol | Meaning | | -------- | ---------- | | 🟢 | On track with overall release | | 🟡 | Missed last milestone | | 🔴 | Missed multiple milestones |

Issue tracking

State Total Bug Enhancement
Open 18 0 1
Closed 278 57 46

Updated on 11/16 9:18a CST Query are accessible in the table headers

Campaigns

Release testing

To aid in release testing we are providing all teams with access to docker images of OpenSearch to aid in the rapid setup/tear down for testing, for additional discussion about test documentation see #933


  • Use Docker-Compose to setup a cluster

    • Download this gist as docker-compose.yml on your machine
    • Get latest image versions docker-compose pull
    • Start the cluster docker-compose up
  • OpenSearch docker

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch:1.2.0 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:1.2.0
      • Connect command curl https://localhost:9200
    • Start with security
      • Docker command docker pull opensearchstaging/opensearch:1.2.0 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:1.2.0
      • Connect command curl --insecure https://admin:admin@localhost:9200
  • OpenSearch-Dashboards docker

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:1.2.0 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:1.2.0
      • URL https://localhost:5601
    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:1.2.0 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:1.2.0
      • URL https://localhost:5601
  • Use TARs to deploy OpenSearch Manually (Updated 2021/11/22 11:00 PST)

From Docker Release

@github-actions github-actions bot added release untriaged Issues that have not yet been triaged v{{ env.VERSION }} labels Sep 23, 2021
@dblock dblock added v1.2.0 OpenSearch 1.2 version and removed v{{ env.VERSION }} untriaged Issues that have not yet been triaged labels Sep 23, 2021
@dblock dblock pinned this issue Oct 11, 2021
@dblock
Copy link
Member

dblock commented Oct 14, 2021

Created/updated a 1.2.0 label using meta from https://github.com/opensearch-project/opensearch-plugins

meta exec "ghi label 'v1.2.0' -c '#bfdadc'"

@peternied
Copy link
Member

peternied commented Nov 10, 2021

  • Use Docker-Compose to setup a cluster

    • Download this gist as docker-compose.yml on your machine
    • Get latest image versions docker-compose pull
    • Start the cluster docker-compose up
  • OpenSearch docker

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch:1.2.0 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:1.2.0
      • Connect command curl https://localhost:9200
    • Start with security
      • Docker command docker pull opensearchstaging/opensearch:1.2.0 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:1.2.0
      • Connect command curl --insecure https://admin:admin@localhost:9200
  • OpenSearch-Dashboards docker

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:1.2.0 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:1.2.0
      • URL https://localhost:5601
    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:1.2.0 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:1.2.0
      • URL https://localhost:5601
  • Use TARs to deploy OpenSearch Manually

From Docker Release

@peternied
Copy link
Member

peternied commented Nov 10, 2021

Note; we are tracking a OpenSearch-Dashboards blocker on #934

@peternied
Copy link
Member

Nov 11th, End of day today all tests should have executed on build 964 or 364+ so we can be sure that all issues have been flushed out. Then we will begin taking a closer look at outstanding 1.2.0 issues and make sure they are burning down so we are on target for the release on Nov 16th.

@peternied
Copy link
Member

We have a validation blocking bug, tracking #943 as the core issue. At this point validation will not be complete tonight as integration tests have not fully run. Thanks for your focus on this @mch2 @gaiksaya

@peternied
Copy link
Member

Release testing has reset

We have found a validation blocking bug and all testing needs to be reset unless you know distribution downloader is not used, #953. The distribution download was pulling snapshot artifacts that were 3 weeks old - before feature complete. The artifacts have been updated, but all testing needs to be reset. I will be updating build images shortly and also creating an office hours today for anyone to join with questions.

This will slip the release by 2 days to give all teams time to restart testing, new release testing completion date is scheduled for Nov 15th.

@peternied
Copy link
Member

Release date - update

Since the testing process has slipped, we are pushing the release date out to Thursday, Nov18th. I'll be following day-by-day to make sure that everything is in alignment and be reaching out to release tickets.

@peternied
Copy link
Member

Release 1.2.0 is scheduled for 18th. Today all testing must be complete, we will be pushing to have all remaining issues marked for the release resolved as soon as possible. I am still waiting for confirmation that tests were rerun and completed for:

All release drivers, please drive down all open issues and check off all the completed work in the component specific issues.

@peternied
Copy link
Member

Current issues, from https://github.com/opensearch-project/opensearch-plugins via meta exec 'ghi list -L v1.2.0', filtered out release issues

opensearch-project/opensearch-plugins open issues, labeled v1.2.0

64: Lower CI Java Version to Java 11 for all plugins and run integration tests on 8 enhancement meta v1.2.0 1
45: Use AWS OSS Maven -SNAPSHOT for plugin dependencies enhancement v1.2.0 19 @abbashus
opensearch-plugins ✓

opensearch-project/alerting open issues, labeled v1.2.0

82: Alerts needs stricter ownership enhancement backlog v1.2.0 5
alerting ✓

opensearch-project/asynchronous-search open issues, labeled v1.2.0

49: Typed Keys Response param added in Get Asynchronous Search API v1.2.0 ↑
46: Support for typed_keys in GET asynchronous search API... bug enhancement v1.2.0 3 @Bukhtawar
18: Standardize branching to match OpenSearch v1.0.0 v1.2.0 1
asynchronous-search ✓

opensearch-project/index-management open issues, labeled v1.2.0

197: [Distribution Build] Integ-Test Failure RestDeleteTransformActionIT v1.2.0 1 @downsrob
index-management ✓

opensearch-project/job-scheduler open issues, labeled v1.2.0

101: [BundleBuild] opensearch-job-scheduler-spi maven artifacts are not copied to the correct location bug v1.2.0 2
job-scheduler ✓

opensearch-project/k-nn open issues, labeled v1.2.0

214: [IntegTest] knn.plugin.action.RestLegacyKNNStatsHandlerIT on distribution build v1.2.0 3
213: [IntegTest] knn.index.FaissIT.testEndToEnd_fromMethod on distribution build v1.2.0
k-nn ✓

opensearch-project/sql open issues, labeled v1.2.0

262: Release SQL Drivers and CLI OpenSearch 1.2 v1.2.0 @chloe-zh
sql ✓

@stockholmux
Copy link
Member

stockholmux commented Nov 17, 2021

@peternied / @peterzhuamazon - can you confirm these artifacts are correct for 1.2.0?

(this includes compatible but independently released OpenSearch stuff like cli, data-prepper. I assume no changes from 1.1.0, so just a glance would be good on those; Mostly concerned with the core artifacts/URLs)

opensearch 1.2.0
    arm64
        https://hub.docker.com/r/opensearchproject/opensearch/tags?page=1&ordering=last_updated&name=1.2.0
        https://artifacts.opensearch.org/releases/bundle/opensearch/1.2.0/opensearch-1.2.0-linux-arm64.tar.gz
            (sig: https://artifacts.opensearch.org/releases/bundle/opensearch/1.2.0/opensearch-1.2.0-linux-arm64.tar.gz.sig)
    x64
        https://hub.docker.com/r/opensearchproject/opensearch/tags?page=1&ordering=last_updated&name=1.2.0
        https://artifacts.opensearch.org/releases/bundle/opensearch/1.2.0/opensearch-1.2.0-linux-x64.tar.gz 
            (sig: https://artifacts.opensearch.org/releases/bundle/opensearch/1.2.0/opensearch-1.2.0-linux-x64.tar.gz.sig)

opensearch-dashboards 1.2.0
    arm64
        https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/1.2.0/opensearch-dashboards-1.2.0-linux-arm64.tar.gz
            (sig: https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/1.2.0/opensearch-dashboards-1.2.0-linux-arm64.tar.gz.sig)
        https://hub.docker.com/r/opensearchproject/opensearch-dashboards/tags?page=1&ordering=last_updated&name=1.2.0
    x64
        https://hub.docker.com/r/opensearchproject/opensearch-dashboards/tags?page=1&ordering=last_updated&name=1.2.0
        https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/1.2.0/opensearch-dashboards-1.2.0-linux-x64.tar.gz
            (sig: https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/1.2.0/opensearch-dashboards-1.2.0-linux-x64.tar.gz.sig)

opensearch-cli 1.0.0
    arm64
        https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-linux-arm64.zip
            (sig: https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-linux-arm64.zip.sig)
        https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-macos-arm64.pkg
    x64
        https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-linux-x64.zip
            (sig: https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-linux-x64.zip.sig)
        https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-macos-x64.pkg
        https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-windows-x64.zip
    x86
        https://artifacts.opensearch.org/opensearch-clients/opensearch-cli/opensearch-cli-1.0.0-windows-x86.zip

logstash-oss-with-opensearch-output-plugin 7.13.2
    arm64
        https://hub.docker.com/r/opensearchproject/logstash-oss-with-opensearch-output-plugin
        https://artifacts.opensearch.org/logstash/logstash-oss-with-opensearch-output-plugin-7.13.2-linux-arm64.tar.gz
            (sig: https://artifacts.opensearch.org/logstash/logstash-oss-with-opensearch-output-plugin-7.13.2-linux-arm64.tar.gz.sig)
    x64
        https://hub.docker.com/r/opensearchproject/logstash-oss-with-opensearch-output-plugin
        https://artifacts.opensearch.org/logstash/logstash-oss-with-opensearch-output-plugin-7.13.2-linux-x64.tar.gz
            (sig: https://artifacts.opensearch.org/logstash/logstash-oss-with-opensearch-output-plugin-7.13.2-linux-x64.tar.gz.sig)
        https://artifacts.opensearch.org/logstash/logstash-oss-with-opensearch-output-plugin-7.13.2-macos-x64.tar.gz
            (sig: https://artifacts.opensearch.org/logstash/logstash-oss-with-opensearch-output-plugin-7.13.2-macos-x64.tar.gz.sig)

data-prepper-1.1.0
    x64
        https://hub.docker.com/r/opensearchproject/data-prepper/tags?page=1&ordering=last_updated&name=1.1.0

opensearch-min 1.2.0
    arm64
        https://artifacts.opensearch.org/releases/core/opensearch/1.2.0/opensearch-min-1.2.0-linux-arm64.tar.gz
            (sig: https://artifacts.opensearch.org/releases/core/opensearch/1.2.0/opensearch-min-1.2.0-linux-arm64.tar.gz.sig)
    x64
        https://artifacts.opensearch.org/releases/core/opensearch/1.2.0/opensearch-min-1.2.0-linux-x64.tar.gz
            (sig: https://artifacts.opensearch.org/releases/core/opensearch/1.2.0/opensearch-min-1.2.0-linux-x64.tar.gz.sig)

opensearch-dashboards-min 1.2.0
    arm64
        https://artifacts.opensearch.org/releases/core/opensearch-dashboards/1.2.0/opensearch-dashboards-min-1.2.0-linux-arm64.tar.gz
            (sig: https://artifacts.opensearch.org/releases/core/opensearch-dashboards/1.2.0/opensearch-dashboards-min-1.2.0-linux-arm64.tar.gz.sig)
    x64
        https://artifacts.opensearch.org/releases/core/opensearch-dashboards/1.2.0/opensearch-dashboards-min-1.2.0-linux-x64.tar.gz
            (sig: https://artifacts.opensearch.org/releases/core/opensearch-dashboards/1.2.0/opensearch-dashboards-min-1.2.0-linux-x64.tar.gz.sig)

@peternied
Copy link
Member

@stockholmux That is really hard for my brain to process seems like a good opportunity for a tooling, do we have the current list for 1.1.0 that I could diff this against?

@stockholmux
Copy link
Member

@peternied It's just a updated text version of this: https://opensearch.org/artifacts/by-version/#v1-1-0

Otherwise, you can view my branch to see the new files. stockholmux/project-website@5451bba

I'm not sure we can tool in an effective way since there is more fluidity than you would imagine between versions (e.g. 1.1.0 has FreeBSD but not yet for 1.2.0, we'll be adding additional artifacts soon, etc.)

@peternied
Copy link
Member

@stockholmux Beyond Compare + Gist + https://gistpreview.github.io/ -> [Diff], it looks good. I am not familiar with the FreeBSD deployment process, do we need a tracking ticket for that?

@stockholmux
Copy link
Member

@peternied FreeBSD folks will take our released version and make the changes needed to FreshPorts (@smortex is a key contact). I've already pinged them in their discord.

@peternied
Copy link
Member

Tomorrow is the scheduled OpenSearch 1.2.0 release, if you were waiting for the last minute here it is! Track on #567

Urgent follow up needed

For all release owners, clean up any issues on this list [Query] and mark all Pre-Release and Release tasks complete by EOD otherwise you might hold up the release.

As soon as that issues list has 0 items, I will kick off a build and I'll run the integration tests one last time and declare a Release Candidate. I'll post new links for everything just like what we did at feature freeze for sanity testing.

As always please use github release issues for questions.

@dblock
Copy link
Member

dblock commented Nov 17, 2021

Current issues, from https://github.com/opensearch-project/opensearch-plugins via meta exec 'ghi list -L v1.2.0', filtered out release issues

opensearch-project/opensearch-plugins open issues, labeled v1.2.0

64: Lower CI Java Version to Java 11 for all plugins and run integration tests on 8 enhancement

Moved to 1.3.

45: Use AWS OSS Maven -SNAPSHOT for plugin dependencies enhancement v1.2.0

opensearch-project/opensearch-plugins#45 (comment)

@peternied
Copy link
Member

For the release process today the team is focusing in on OpenSearch 1.2.0 Performance regression of 10% compared to 1.1.0 opensearch-project/OpenSearch#1560

@peternied
Copy link
Member

1.2 Update

For investigating the removal of ShardIndexingPressure, we switched from using a CI build to both builds from the same dev box and the results were very close to identical. We have created a build removing ShardIndexingPressure via the CI system, to confirm the ~20% drop was configuration related. Those tests have been kicked off and will be validated tomorrow morning.

Distribution testing showed more index latency degradation on arm builds, we will re-run tests from the 7 key dates on ARM to see if we can identify a smoking gun - long shot but doing our due diligence.

Next steps: Review the test results after they come back, if we cannot reproduce the perf drop, we will start the final release process on Monday.

Details: opensearch-project/OpenSearch#1560

@peterzhuamazon
Copy link
Member

Talked to @peternied will use the latest version built on Monday morning for integTest.

@peterzhuamazon peterzhuamazon self-assigned this Nov 22, 2021
@gaiksaya gaiksaya self-assigned this Nov 22, 2021
@gaiksaya
Copy link
Member

We are using build# 1127 for opensearch and build# 512 for opensearch-dashboards as release candidates.
Thanks!

@peternied
Copy link
Member

Retrospective is scheduled for Friday Dec 10th, published action items should be made available the following week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release v1.2.0 OpenSearch 1.2 version
Projects
None yet
Development

No branches or pull requests

6 participants