Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NR-329534 Automate release of nri-jmx #158

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
+83 −354
Open

NR-329534 Automate release of nri-jmx #158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
53faa1e
updated workflows to use coreint-automation reusable workflows
sairaj18 Nov 21, 2024
aa8a106
added fixes for push-pr jobs
sairaj18 Nov 21, 2024
ef24626
moved NRJMX_VERSION from env to jobs declaration
sairaj18 Nov 21, 2024
9662fb7
disabled integration_nix test in reusable_push_pr and running it in p…
sairaj18 Nov 21, 2024
f93ab75
fix parse error in changelog.md
sairaj18 Nov 22, 2024
6793903
fix parse error in changelog.md
sairaj18 Nov 22, 2024
a5e5c92
fix parse error in changelog.md
sairaj18 Nov 22, 2024
17ccf85
fix parse error in changelog.md and added unreleased changes
sairaj18 Nov 22, 2024
86ae414
typo fix
sairaj18 Nov 22, 2024
a50cb41
updated changelog
sairaj18 Nov 22, 2024
84fe994
updated the prerelease.yml file
sairaj18 Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/automated_release.yaml
View file
Open in desktop
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'd keep newlines at the end of files. (See this rationale)

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: Automated release creation

on:
workflow_dispatch:
schedule:
- cron: "0 19 * * 4"

jobs:
release_management:
uses: newrelic/coreint-automation/.github/workflows/reusable_release_automation.yaml@v3
secrets: inherit
191 changes: 10 additions & 181 deletions .github/workflows/prerelease.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,65 +8,9 @@ on:
- 'v*'

env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NRJMX_VERSION: '2.6.0'
INTEGRATION: "jmx"
ORIGINAL_REPO_NAME: 'newrelic/nri-jmx'
REPO_FULL_NAME: ${{ github.event.repository.full_name }}
TAG: ${{ github.event.release.tag_name }}
NRJMX_VERSION: '2.6.0' # TODO check if we should update it to latest release version

jobs:
snyk:
name: Run security checks via snyk
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Run Snyk to check for vulnerabilities
env:
SNYK_TOKEN: ${{ secrets.CAOS_SNYK_TOKEN }}
run: make ci/snyk-test

test-nix:
name: Run unit tests on *Nix
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Unit tests
run: make ci/test

test-windows:
name: Run unit tests on Windows
runs-on: windows-2019
env:
GOPATH: ${{ github.workspace }}
defaults:
run:
working-directory: src/github.com/${{env.ORIGINAL_REPO_NAME}}
steps:
- name: Checkout
uses: actions/checkout@v2
with:
path: src/github.com/${{env.ORIGINAL_REPO_NAME}}
- name: Install Go
uses: actions/setup-go@v5
with:
go-version-file: 'src/github.com/${{env.ORIGINAL_REPO_NAME}}/go.mod'
- name: Running unit tests
shell: pwsh
run: |
.\build\windows\unit_tests.ps1

# can't run this step inside of container because of tests specific
test-integration-nix:
name: Run integration tests on *Nix
runs-on: ubuntu-20.04
Expand All @@ -88,127 +32,12 @@ jobs:
GOPATH: ${{ github.workspace }}
run: make integration-test

prerelease:
name: Build binary for *Nix/Win, create archives for *Nix/Win, create packages for *Nix, upload all artifacts into GH Release assets
runs-on: ubuntu-20.04
needs: [test-nix, test-windows, snyk, test-integration-nix]
env:
GPG_MAIL: '[email protected]'
GPG_PASSPHRASE: ${{ secrets.OHAI_GPG_PASSPHRASE }}
GPG_PRIVATE_KEY_BASE64: ${{ secrets.OHAI_GPG_PRIVATE_KEY_BASE64 }} # base64 encoded
steps:
- uses: actions/checkout@v2
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Pre release
run: make ci/prerelease
- name: Notify failure via Slack
if: ${{ failure() }}
uses: archive/github-actions-slack@master
with:
slack-bot-user-oauth-access-token: ${{ secrets.COREINT_SLACK_TOKEN }}
slack-channel: ${{ secrets.COREINT_SLACK_CHANNEL }}
slack-text: "❌ `${{ env.REPO_FULL_NAME }}`: prerelease pipeline failed."

package-win:
name: Create MSI & Upload into GH Release assets
runs-on: windows-2019
needs: [prerelease]
env:
GOPATH: ${{ github.workspace }}
PFX_CERTIFICATE_BASE64: ${{ secrets.OHAI_PFX_CERTIFICATE_BASE64 }} # base64 encoded
PFX_CERTIFICATE_DESCRIPTION: 'New Relic'
PFX_PASSPHRASE: ${{ secrets.OHAI_PFX_PASSPHRASE }}
defaults:
run:
working-directory: src/github.com/${{env.ORIGINAL_REPO_NAME}}
strategy:
matrix:
goarch: [ amd64 ] # 386 not supported in jmx integrations
test-upgrade: [true,false]
steps:
- name: Checkout
uses: actions/checkout@v2
with:
path: src/github.com/${{env.ORIGINAL_REPO_NAME}}
- name: Get PFX certificate from GH secrets
shell: bash
run: printf "%s" "$PFX_CERTIFICATE_BASE64" | base64 -d - > wincert.pfx
- name: Download zip from GH Release assets and extract .exe
shell: pwsh
run: |
build\windows\download_zip_extract_exe.ps1 "$env:INTEGRATION" ${{ matrix.goarch }} "$env:TAG" "$env:REPO_FULL_NAME"
- name: Download nrjmx
shell: bash
run: |
build/windows/download_nrjmx.sh
- name: Create MSI
shell: pwsh
run: |
build\windows\package_msi.ps1 -integration "$env:INTEGRATION" -arch ${{ matrix.goarch }} -tag "$env:TAG" -pfx_passphrase "$env:PFX_PASSPHRASE" -pfx_certificate_description "$env:PFX_CERTIFICATE_DESCRIPTION"
- name: Test win packages installation
uses: newrelic/integrations-pkg-test-action/windows@v1
with:
tag: ${{ env.TAG }}
integration: nri-${{ env.INTEGRATION }}
arch: ${{ matrix.goarch }}
upgrade: ${{ matrix.test-upgrade }}
pkgType: "exe" # JMX integrations are shipped in .exe format
- name: Upload MSI to GH
if: startsWith(matrix.test-upgrade, 'false')
shell: bash
run: |
build/windows/upload_msi.sh ${INTEGRATION} ${{ matrix.goarch }} ${TAG}
- name: Notify failure via Slack
if: ${{ failure() }}
uses: archive/github-actions-slack@master
with:
slack-bot-user-oauth-access-token: ${{ secrets.COREINT_SLACK_TOKEN }}
slack-channel: ${{ secrets.COREINT_SLACK_CHANNEL }}
slack-text: "❌ `${{ env.REPO_FULL_NAME }}`: prerelease pipeline failed."

publish-to-s3:
name: Send release assets to S3
runs-on: ubuntu-20.04
needs: [package-win]
steps:
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Publish to S3 action
uses: newrelic/infrastructure-publish-action@v1
env:
AWS_S3_BUCKET_NAME: "nr-downloads-ohai-staging"
AWS_S3_LOCK_BUCKET_NAME: "onhost-ci-lock-staging"
with:
disable_lock: false
run_id: ${{ github.run_id }}
tag: ${{env.TAG}}
app_name: "nri-${{env.INTEGRATION}}"
repo_name: ${{ env.ORIGINAL_REPO_NAME }}
access_point_host: "staging"
schema: "ohi-jmx" # ohi-jmx for integrations that bundle JMX on windows installers
aws_region: "us-east-1"
aws_role_arn: ${{ secrets.OHAI_AWS_ROLE_ARN_STAGING }}
aws_role_session_name: ${{ secrets.OHAI_AWS_ROLE_SESSION_NAME_STAGING }}
aws_access_key_id: ${{ secrets.OHAI_AWS_ACCESS_KEY_ID_STAGING }}
aws_secret_access_key: ${{ secrets.OHAI_AWS_SECRET_ACCESS_KEY_STAGING }}
aws_s3_bucket_name: ${{ env.AWS_S3_BUCKET_NAME }}
# used for locking in case of concurrent releases
aws_s3_lock_bucket_name: ${{ env.AWS_S3_LOCK_BUCKET_NAME }}
# used for signing package stuff
gpg_passphrase: ${{ secrets.OHAI_GPG_PASSPHRASE }}
gpg_private_key_base64: ${{ secrets.OHAI_GPG_PRIVATE_KEY_BASE64 }}
- name: Test uploaded package installability
uses: newrelic/integrations-pkg-test-action/linux@v1
with:
tag: ${{ env.TAG }}
integration: nri-${{ env.INTEGRATION }}
packageLocation: repo
stagingRepo: true
upgrade: false
pre-release:
uses: newrelic/coreint-automation/.github/workflows/reusable_pre_release.yaml@v3
with:
tag: ${{ github.event.release.tag_name }}
integration: "jmx"
windows_goarch_matrix: ["amd64"] # 386 not supported in jmx integrations
win_package_type: exe # JMX integrations are shipped in .exe format
publish_schema: "ohi-jmx" # ohi-jmx for integrations that bundle JMX on windows installers
secrets: inherit
126 changes: 30 additions & 96 deletions .github/workflows/push_pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,87 +8,34 @@ on:
pull_request:

env:
TAG: "v0.0.0" # needed for goreleaser windows builds
REPO_FULL_NAME: ${{ github.event.repository.full_name }}
ORIGINAL_REPO_NAME: "newrelic/nri-jmx"
NRJMX_VERSION: '2.6.0'
DOCKER_LOGIN_AVAILABLE: ${{ secrets.OHAI_DOCKER_HUB_ID }}
NRJMX_VERSION: '2.6.0' # TODO check if we should update it to latest release version

jobs:
static-analysis:
name: Run all static analysis checks
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- uses: newrelic/newrelic-infra-checkers@v1
- name: Semgrep
uses: returntocorp/semgrep-action@v1
with:
auditOn: push

- name: Install Go
uses: actions/setup-go@v5
with:
go-version-file: 'go.mod'
# reusable_push_pr contains static-analysis but it does not contain the Semgrep step
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we managed to use the reusable static-analysis we we could get rid of the tools module.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep we can remove the commented static-analysis job as it is managed by reusable workflow.

# static-analysis job in reusable_push_pr cannot be disabled
# uncommenting the below leads to the following error "creating validator: parsing markdown: parsing markdown headers: unexpected additional L1 header \"2.4.7 (2021-06-10)\" found, only a single L1 header is allowed"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the error due to the changelog format or is it something else?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it was due to the changelog format. Now we are not getting that error I made changes to the changelog.md file to fix the parsing error

# TODO check if we can remove the below code and ignore the Semgrep step
# static-analysis:
# name: Run all static analysis checks
# runs-on: ubuntu-20.04
# steps:
# - uses: actions/checkout@v2
# - uses: newrelic/newrelic-infra-checkers@v1
# - name: Semgrep
# uses: returntocorp/semgrep-action@v1
# with:
# auditOn: push
Comment on lines +17 to +27
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was removed from other repositories. Ex nri-postgresql. I'd double-check with the team if it is required here for some reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it was removed in nri-postgresql then it makes sense to remove the static-analysis step as the remaining part is present in reusable workflow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @paologallinaharbur can you confirm if we can ignore the Semgrep step in the static-analysis job, similar to the above mentioned nri-postgresql


- name: golangci-lint
uses: golangci/golangci-lint-action@v6
continue-on-error: ${{ github.event_name != 'pull_request' }}
with:
only-new-issues: true

snyk:
name: Run security checks via snyk
runs-on: ubuntu-20.04
env:
SNYK_TOKEN: ${{ secrets.CAOS_SNYK_TOKEN }}
steps:
- uses: actions/checkout@v2
- name: Login to DockerHub
if: ${{env.DOCKER_LOGIN_AVAILABLE}}
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Scan code for vulnerabilities
if: ${{env.SNYK_TOKEN}}
run: make ci/snyk-test

test-nix:
name: Run unit tests on *Nix
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Login to DockerHub
if: ${{env.DOCKER_LOGIN_AVAILABLE}}
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Unit tests
run: make ci/test
# - name: Install Go
# uses: actions/setup-go@v5
# with:
# go-version-file: 'go.mod'

test-windows:
name: Run unit tests on Windows
runs-on: windows-2019
env:
GOPATH: ${{ github.workspace }}
defaults:
run:
working-directory: src/github.com/${{ env.ORIGINAL_REPO_NAME }}
steps:
- name: Checkout
uses: actions/checkout@v2
with:
path: src/github.com/${{env.ORIGINAL_REPO_NAME}}
- name: Install Go
uses: actions/setup-go@v5
with:
go-version-file: 'src/github.com/${{env.ORIGINAL_REPO_NAME}}/go.mod'
- name: Running unit tests
shell: pwsh
run: |
.\build\windows\unit_tests.ps1
# - name: golangci-lint
# uses: golangci/golangci-lint-action@v6
# continue-on-error: ${{ github.event_name != 'pull_request' }}
# with:
# only-new-issues: true

# can't run this step inside of container because of tests specific
test-integration-nix:
Expand All @@ -107,27 +54,14 @@ jobs:
uses: actions/setup-go@v5
with:
go-version-file: 'src/github.com/${{env.ORIGINAL_REPO_NAME}}/go.mod'
- name: Login to DockerHub
if: ${{env.DOCKER_LOGIN_AVAILABLE}}
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Integration test
env:
GOPATH: ${{ github.workspace }}
run: make integration-test

test-build:
name: Test binary compilation for all platforms:arch
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Login to DockerHub
if: ${{env.DOCKER_LOGIN_AVAILABLE}}
uses: docker/login-action@v3
with:
username: ${{ secrets.OHAI_DOCKER_HUB_ID }}
password: ${{ secrets.OHAI_DOCKER_HUB_PASSWORD }}
- name: Build all platforms:arch
run: make ci/build
push-pr:
uses: newrelic/coreint-automation/.github/workflows/reusable_push_pr.yaml@v3
with:
integration: jmx
run_integration_nix: false
secrets: inherit
Loading
Loading