Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1360070 upgrade docker container to Ubuntu 20.04 dotnet 8 #940

Merged
merged 3 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions .github/workflows/linter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,6 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
DOTNET_VERSION: 6.0
DOTNET_LEGACY_VERSION: 4.7.1

jobs:
run-linter:
name: Run linter
Expand All @@ -33,7 +29,7 @@ jobs:
- name: Set up .NET
uses: actions/setup-dotnet@v1
with:
dotnet-version: "6.0.x"
dotnet-version: '8.0.x'
dotnet-quality: 'ga'
- name: Run linters
uses: wearerequired/lint-action@v2
Expand Down
29 changes: 16 additions & 13 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
DOTNET_VERSION: 6.0
DOTNET_LEGACY_VERSION: 4.7.1
# uncomment to run the tests sequentially
#SEQUENTIAL_ENV: SEQUENTIAL_TEST_RUN

Expand All @@ -34,15 +31,17 @@ jobs:
strategy:
fail-fast: false
matrix:
dotnet: ['net6.0', 'net472', 'net471']
dotnet: ['net8.0','net6.0', 'net472', 'net471']
cloud_env: ['AZURE', 'GCP', 'AWS']
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Setup Dotnet
uses: actions/setup-dotnet@v3
uses: actions/setup-dotnet@v4
with:
dotnet-version: '6.0.x'
dotnet-version: |
6.0.x
8.0.x
dotnet-quality: 'ga'
- name: Setup Python
uses: actions/setup-python@v4
Expand Down Expand Up @@ -93,14 +92,16 @@ jobs:
strategy:
fail-fast: false
matrix:
dotnet: ['net6.0']
dotnet: ['net6.0', 'net8.0']
cloud_env: ['AZURE', 'GCP', 'AWS']
steps:
- uses: actions/checkout@v3
- name: Setup Dotnet
uses: actions/setup-dotnet@v3
uses: actions/setup-dotnet@v4
with:
dotnet-version: '6.0.x'
dotnet-version: |
6.0.x
8.0.x
dotnet-quality: 'ga'
- name: Setup Python
uses: actions/setup-python@v4
Expand Down Expand Up @@ -150,14 +151,16 @@ jobs:
strategy:
fail-fast: false
matrix:
dotnet: ['net6.0']
dotnet: ['net6.0', 'net8.0']
cloud_env: ['AZURE', 'GCP', 'AWS']
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Setup Dotnet
uses: actions/setup-dotnet@v3
uses: actions/setup-dotnet@v4
with:
dotnet-version: '6.0.x'
dotnet-version: |
6.0.x
8.0.x
dotnet-quality: 'ga'
- name: Setup Python
uses: actions/setup-python@v4
Expand Down
2 changes: 1 addition & 1 deletion Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ timestamps {
string(name: 'branch', value: 'main'),
string(name: 'client_git_commit', value: scmInfo.GIT_COMMIT),
string(name: 'client_git_branch', value: scmInfo.GIT_BRANCH),
string(name: 'TARGET_DOCKER_TEST_IMAGE', value: 'dotnet-centos7-net6'),
string(name: 'TARGET_DOCKER_TEST_IMAGE', value: 'dotnet-ubuntu204-net8'),
string(name: 'parent_job', value: env.JOB_NAME),
string(name: 'parent_build_number', value: env.BUILD_NUMBER)
]
Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ The Snowflake .NET connector supports the the following .NET framework and libra
- .NET Framework 4.7.1
- .NET Framework 4.7.2
- .NET 6.0
- .NET 8.0

Please refer to the Notice section below for information about safe usage of the .NET Driver

Expand Down Expand Up @@ -999,7 +1000,7 @@ dotnet-coverage collect "dotnet test --framework net472 --no-build -l console;ve

## Notice

1. CVE-2019-0820 -
1. CVE-2019-0820 -
This CVE has been reported in systems.text.regularexpressions.dll which is used by the regular expressions packages - systems.text.regularexpressions.4.3.1.nupkg. This vulnerability manifests itself ONLY when the following .NET runtime environments are being used:

* v1.0 branch: 1.0 - 1.0.16 (exclusive)
Expand All @@ -1009,7 +1010,7 @@ dotnet-coverage collect "dotnet test --framework net472 --no-build -l console;ve

In order to mitigate this vulnerability, we recommend to update to higher Runtime versions. If you're already running on a .NET Runtime version higher than the ones listed above, you're not going to be affected by this vulnerability.

2. Logging -
2. Logging -
Snowflake has identified an issue on Feb 20, 2020, with our logging code for the .NET drivers in which we write Master and Session tokens in the clear to the debug logs. The debug logs are collected locally on the drive where your programs are running. This issue impacts only those instances where the programs are run with debug flags enabled, i.e. setting the log level value= "Debug” or “All" in the log4Net config

Under normal conditions, the Master and Session tokens captured in the log files are short-lived for about 4 and 1 hours, respectively. They will expire after the 4-hour window unless explicitly refreshed, in which case they could be refreshed indefinitely.
Expand All @@ -1022,8 +1023,8 @@ dotnet-coverage collect "dotnet test --framework net472 --no-build -l console;ve
- If you cannot upgrade for any reason, please ensure all debugging is disabled
- If you are concerned about a potential compromise, contact Snowflake Customer Support for assistance with invalidating all active sessions/tokens.

3. Global HTTP connection settings -
Snowflake has identified an issue where the driver is globally enforcing TLS 1.2 and certificate revocation checks with the .NET Driver v1.2.1 and earlier versions.
3. Global HTTP connection settings -
Snowflake has identified an issue where the driver is globally enforcing TLS 1.2 and certificate revocation checks with the .NET Driver v1.2.1 and earlier versions.
Starting with v2.0.0, the driver will set these locally.

Note that the driver is now targeting .NET 6.0. When upgrading, you might also need to run “Update-Package -reinstall” to update the dependencies.
11 changes: 5 additions & 6 deletions Snowflake.Data.Tests/Snowflake.Data.Tests.csproj
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net6.0;net471;net472</TargetFrameworks>
<TargetFrameworks Condition="'$(OS)' != 'Windows_NT'">net6.0</TargetFrameworks>
<TargetFrameworks>net8.0;net6.0;net471;net472</TargetFrameworks>
<TargetFrameworks Condition="'$(OS)' != 'Windows_NT'">net6.0;net8.0</TargetFrameworks>
<RuntimeFrameworkVersion>6.0.0</RuntimeFrameworkVersion>
<RuntimeFrameworkVersion Condition="'$(TargetFramework)' == 'net8.0'">8.0.0</RuntimeFrameworkVersion>
<Title>Snowflake.Data.Tests</Title>
<Description>Snowflake Connector for .NET</Description>
<Company>Snowflake Computing, Inc</Company>
<Product>Snowflake Connector for .NET</Product>
<Copyright>Copyright (c) 2012-2023 Snowflake Computing Inc. All rights reserved.</Copyright>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<LangVersion>9</LangVersion>
<DefineConstants>$(SEQUENTIAL_ENV)</DefineConstants>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="JunitXml.TestLogger" Version="3.1.12" />
<PackageReference Include="Moq" Version="4.18.4" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.7.1" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit" Version="3.14.0" />
<PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
<PackageReference Include="RichardSzalay.MockHttp" Version="6.0.0" />
<PackageReference Include="System.Net.Http" Version="4.3.4" />
Expand All @@ -37,7 +36,7 @@
<Reference Include="System.Web" />
<Reference Include="System.Net.Http.WebRequest" />
</ItemGroup>

<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DebugType>full</DebugType>
<DebugSymbols>True</DebugSymbols>
Expand Down
17 changes: 8 additions & 9 deletions Snowflake.Data/Snowflake.Data.csproj
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net6.0;net471;net472</TargetFrameworks>
<TargetFrameworks Condition="'$(OS)' != 'Windows_NT'">net6.0</TargetFrameworks>
<TargetFrameworks>net8.0;net6.0;net471;net472</TargetFrameworks>
<TargetFrameworks Condition="'$(OS)' != 'Windows_NT'">net6.0;net8.0</TargetFrameworks>
<Title>Snowflake.Data</Title>
<PackageId>Snowflake.Data</PackageId>
<PackageLicenseUrl>https://github.com/snowflakedb/snowflake-connector-net/blob/master/LICENSE</PackageLicenseUrl>
Expand All @@ -11,13 +11,12 @@
<Description>Snowflake Connector for .NET</Description>
<Company>Snowflake Computing, Inc</Company>
<Product>Snowflake Connector for .NET</Product>
<Authors>howryu, tchen</Authors>
<Copyright>Copyright (c) 2012-2023 Snowflake Computing Inc. All rights reserved.</Copyright>
<Authors>Snowflake</Authors>
<Version>3.1.0</Version>
<DebugType>Full</DebugType>
<LangVersion>7.3</LangVersion>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Apache.Arrow" Version="14.0.2" />
<PackageReference Include="AWSSDK.S3" Version="3.7.0.4" />
Expand All @@ -31,7 +30,7 @@
<PackageReference Include="System.IdentityModel.Tokens.Jwt" Version="6.34.0" />
<PackageReference Include="System.Text.RegularExpressions" Version="4.3.1" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFramework)' == 'net471' Or '$(TargetFramework)' == 'net472'">
<Reference Include="System.Net.Http" />
</ItemGroup>
Expand All @@ -45,7 +44,7 @@
<!--needed by Moq to be able to mock internal interfaces-->
<InternalsVisibleTo Include="DynamicProxyGenAssembly2" />
</ItemGroup>

<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<DebugType>full</DebugType>
<DebugSymbols>True</DebugSymbols>
Expand All @@ -55,13 +54,13 @@
<DebugType>full</DebugType>
<DebugSymbols>True</DebugSymbols>
</PropertyGroup>

<PropertyGroup>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
<AssemblyVersion>$(Version)</AssemblyVersion>
</PropertyGroup>

<ItemGroup>
<Folder Include="Properties\" />
</ItemGroup>
Expand Down
4 changes: 2 additions & 2 deletions ci/_init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ BUILD_IMAGE_VERSION=1
TEST_IMAGE_VERSION=1

declare -A BUILD_IMAGE_NAMES=(
[$DRIVER_NAME-centos7-net6]=$DOCKER_REGISTRY_NAME/client-$DRIVER_NAME-centos7-net6-build:$BUILD_IMAGE_VERSION
[$DRIVER_NAME-ubuntu204-net8]=$DOCKER_REGISTRY_NAME/client-$DRIVER_NAME-ubuntu204-net8-build:$BUILD_IMAGE_VERSION
)
export BUILD_IMAGE_NAMES

declare -A TEST_IMAGE_NAMES=(
[$DRIVER_NAME-centos7-net6]=$DOCKER_REGISTRY_NAME/client-$DRIVER_NAME-centos7-net6-test:$TEST_IMAGE_VERSION
[$DRIVER_NAME-ubuntu204-net8]=$DOCKER_REGISTRY_NAME/client-$DRIVER_NAME-ubuntu204-net8-test:$TEST_IMAGE_VERSION
)
export TEST_IMAGE_NAMES
2 changes: 1 addition & 1 deletion ci/image/Dockerfile.dotnet-centos7-net6-build
sfc-gh-erojaslizano marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ RUN yum -y update && \
yum -y install which && \
yum -y install zstd && \
yum -y install jq \

# python
RUN yum -y install python36
RUN python3 -V
Expand Down
56 changes: 56 additions & 0 deletions ci/image/Dockerfile.dotnet-ubuntu204-net8-build
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Use the official Ubuntu base image from Docker Hub
FROM ubuntu:20.04

USER root
WORKDIR /

ENV DEBIAN_FRONTEND noninteractive

# create dir for workspace
RUN mkdir -p /home/user
RUN chmod 777 /home/user

# Update packages and install any desired dependencies
RUN apt-get update
RUN apt-get install -y
RUN apt-get install -y wget
RUN apt-get install -y apt-transport-https
RUN apt-get install -y build-essential
RUN apt-get install -y libpng-dev
RUN apt-get install -y libtiff-dev
RUN apt-get install -y libjpeg-dev
RUN apt-get install -y libx11-dev
RUN apt-get install -y libgl1-mesa-dev
RUN apt-get install -y libglu1-mesa-dev
RUN apt-get install -y gcc
RUN wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
RUN dpkg -i packages-microsoft-prod.deb

# install basic tools
RUN apt-get install -y git
RUN apt-get install -y zstd
RUN apt-get install -y jq

# Install .NET SDK 8.0
RUN apt-get update
RUN apt-get install -y
RUN apt-get install -y dotnet-sdk-6.0
RUN apt-get install -y dotnet-sdk-8.0

# gosu
RUN wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/1.14/gosu-$(dpkg --print-architecture)" && \
chmod +x /usr/local/bin/gosu && \

# clean up
RUN apt-get clean
RUN rm -rf /var/lib/apt/lists/*

# workspace
RUN mkdir -p /home/user && \
chmod 777 /home/user
WORKDIR /home/user

# entry point
COPY scripts/entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
Original file line number Diff line number Diff line change
@@ -1 +1 @@
FROM nexus.int.snowflakecomputing.com:8086/docker/client-dotnet-centos7-net6-build:1
FROM nexus.int.snowflakecomputing.com:8086/docker/client-dotnet-ubuntu204-net8-build:1
2 changes: 2 additions & 0 deletions ci/image/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ source $THIS_DIR/../_init.sh

for name in "${!BUILD_IMAGE_NAMES[@]}"; do
docker build \
--platform=linux/amd64 \
--file $THIS_DIR/Dockerfile.$name-build \
--label snowflake \
--label $DRIVER_NAME \
Expand All @@ -16,6 +17,7 @@ done

for name in "${!TEST_IMAGE_NAMES[@]}"; do
docker build \
--platform=linux/amd64 \
--file $THIS_DIR/Dockerfile.$name-test \
--label snowflake \
--label $DRIVER_NAME \
Expand Down
Loading