Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Start improving local deployments #136

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Start improving local deployments #136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6be9f04
Try to improve local deployments by reusing locally what shuriv has s…
pierotibou Jul 21, 2022
802a5e5
Update build-packages-dev.sh
pierotibou Jul 28, 2022
4881df4
Merge branch 'master' into pierre/ease-local-deploy
pierotibou Jul 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 7 additions & 27 deletions .github/workflows/update_aas_extension.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,48 +32,28 @@ jobs:
- name: "Modify build-packages-dev"
id: "versions"
run: |
versionRegex="[0-9]*.[0-9]*.[0-9]*"
splitVersionRegex="([0-9]+).([0-9]+).([0-9]+)"

CURRENT_DEV_VERSION="$(grep -o -e DEVELOPMENT_VERSION\=\"$versionRegex dotnet/build-packages-dev.sh | sed 's/DEVELOPMENT_VERSION="//')"
echo Current dev version is: $CURRENT_DEV_VERSION

major=0
minor=0
build=${{ github.run_id }}
build="${build:1}"

if [[ $CURRENT_DEV_VERSION =~ $splitVersionRegex ]]; then
major="${BASH_REMATCH[1]}"
minor="${BASH_REMATCH[2]}"
fi

DEV_VERSION=$major.$minor.$(echo $build | bc)
DEV_VERSION=0.2.$(echo $build | bc)
echo New dev version is $DEV_VERSION
echo "::set-output name=dev_version::$DEV_VERSION"

sed -i -e "s/DEVELOPMENT_VERSION=\"$CURRENT_DEV_VERSION/DEVELOPMENT_VERSION=\"$DEV_VERSION/g" dotnet/build-packages-dev.sh
echo Replaced dev version in file.

sha=${{ steps.set_variables.outputs.sha }}
echo Setting install sha to $sha
sed -i -e "s/INSTALL_SHA/$sha/g" dotnet/build-packages-dev.sh
echo Replaced install sha in file.

- uses: actions/setup-dotnet@v1
with:
dotnet-version: '6.0.100'

- name: "Build nuget package"
run: |
bash dotnet/build-packages-dev.sh
sha=${{ steps.set_variables.outputs.sha }}
dev_version=${{steps.versions.outputs.dev_version}}

bash dotnet/build-packages-dev.sh $dev_version $sha

- name: "Upload dev nuget"
run: |
dotnet nuget add source https://pkgs.dev.azure.com/datadoghq/dd-trace-dotnet/_packaging/Public_Feed/nuget/v3/index.json --name Public_Feed --username any_string --password ${{ secrets.AZDO_PAT }} --store-password-in-clear-text

dev_version="${{steps.versions.outputs.dev_version}}-prerelease"
dotnet nuget push package/DevelopmentVerification.DdDotNet.Apm.$dev_version.nupkg --source Public_Feed --api-key any_string
dev_version=${{steps.versions.outputs.dev_version}}
bash dotnet/upload-dev-nuget.sh $dev_version ${{ secrets.AZDO_PAT }}

- name: Azure CLI script
uses: azure/CLI@v1
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@ content.zip
obj/
/set-versions-log.txt
/bin/*
/dotnet/content/Tracer/*
/dotnet-agent-extract
/dotnet/content/Agent/dogstatsd.exe
/dotnet/content/Agent/datadog-trace-agent.exe
/dotnet/content/v*
/dotnet/content/Tracer/*
/package/*
*.zip
.idea
4 changes: 4 additions & 0 deletions dotnet/README.MD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
## Development information

You can build locally the extension by using ./dotnet/build-packages-local.sh
It will both build the application and upload to a dev nuget feed that you can use on your app. This feed can be used by setting the variable `SCM_SITEEXTENSIONS_FEED_URL` to `https://pkgs.dev.azure.com/datadoghq/dd-trace-dotnet/_packaging/Public_Feed/nuget/v2` in your app.
14 changes: 10 additions & 4 deletions dotnet/build-packages-dev.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,27 @@
DEVELOPMENT_VERSION="0.2.114-prerelease"
DEVELOPMENT_VERSION="$1"
TRACER_SHA="$2"

AGENT_DOWNLOAD_URL="http://s3.amazonaws.com/dsd6-staging/windows/agent7/buildpack/agent-binaries-7.35.2-1-x86_64.zip"
TRACER_DOWNLOAD_URL="https://apmdotnetci.blob.core.windows.net/apm-dotnet-ci-artifacts-master/INSTALL_SHA/windows-tracer-home.zip"
TRACER_DOWNLOAD_URL="https://apmdotnetci.blob.core.windows.net/apm-dotnet-ci-artifacts-master/$TRACER_SHA/windows-tracer-home.zip"

echo "Downloading tracer from ${TRACER_DOWNLOAD_URL}"
echo "Downloading tracer from $TRACER_DOWNLOAD_URL"
wget -O tracer.zip $TRACER_DOWNLOAD_URL

echo "Unzipping tracer"
unzip tracer.zip -d dotnet/content/Tracer

DEVELOPMENT_VERSION_FILE=$( echo ${DEVELOPMENT_VERSION} | tr '.' '_' )
DEVELOPMENT_DIR=dotnet/content/v${DEVELOPMENT_VERSION_FILE}

echo "Clean previous runs"
rm -r dotnet/content/v*

echo "Downloading agent from ${AGENT_DOWNLOAD_URL}"
wget -O agent.zip $AGENT_DOWNLOAD_URL
unzip agent.zip -d dotnet-agent-extract

echo "Moving agent executables"
mv dotnet-agent-extract/bin/agent/dogstatsd.exe dotnet/content/Agent
mv dotnet-agent-extract/bin/agent/dogstatsd.exe dotnet/content/Agent/dogstatsd.exe
mv dotnet-agent-extract/bin/agent/trace-agent.exe dotnet/content/Agent/datadog-trace-agent.exe

echo "Versioning development files"
Expand Down
2 changes: 1 addition & 1 deletion dotnet/build-packages.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ unzip agent.zip -d dotnet-agent-extract

echo "Moving agent executables"
mkdir dotnet/content/Agent
mv dotnet-agent-extract/bin/agent/dogstatsd.exe dotnet/content/Agent
mv dotnet-agent-extract/bin/agent/dogstatsd.exe dotnet/content/Agent/dogstatsd.exe
mv dotnet-agent-extract/bin/agent/trace-agent.exe dotnet/content/Agent/datadog-trace-agent.exe

echo "Copying files for development version"
Expand Down
7 changes: 7 additions & 0 deletions dotnet/deploy-packages-local.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
INSTALL_SHA="38787d2e9cadd408f541675e95fab7b9a687d559"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get how the SHA is set for the GitHub Action, but is this also supposed to work right with a local dev build or is that out of scope?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's out of scope. As said in the description, 2 things are not yet handled: use a local tracer, and modify where the files are modified to avoid having to revert modified files (as the build process do a lot of sed to change the version)

PACKAGE_BUILD_VERSION="0.2.659679454"

# Handle later the possibility to use a local commit
./dotnet/build-packages-dev.sh $PACKAGE_BUILD_VERSION $INSTALL_SHA
# SET AZDO_PAT in external env variable as it's a secret
./dotnet/upload-dev-nuget.sh $PACKAGE_BUILD_VERSION $AZDO_PAT
5 changes: 5 additions & 0 deletions dotnet/upload-dev-nuget.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
dev_version="$1"
AZDO_PAT=$2

dotnet nuget add source https://pkgs.dev.azure.com/datadoghq/dd-trace-dotnet/_packaging/Public_Feed/nuget/v3/index.json --name Public_Feed --username any_string --password $AZDO_PAT --store-password-in-clear-text
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wary of using this ever since I've been burnt in the past as artifiacts go in, are never removed, use up your allowance, and kill all CI... so on your head be-it 😛

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 , Thanks for raising this, I didn't see this as a potential issue.
Here, we will do this only for testing changing on the extension itself and its setup, which is quite rare. So I assume it lowers the risk.
That said, we planned to do that on nightly builds, where the impact would be greater. I'll discuss it with @shurivich as well as they used to do that in Ozcode.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may just be scare-mongering, it's just something I avoid in personal projects now 🙂 Oh wait... maybe that was GitHub... pfft, I forget 😂

dotnet nuget push package/DevelopmentVerification.DdDotNet.Apm.$dev_version.nupkg --source Public_Feed --api-key any_string