Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a new pipeline for publishing to Python Cuda 12 packages. #18712

Merged
merged 39 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
9195fc4
Update the template files to correct stage
jchen351 Nov 30, 2023
7bedbf1
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
cc57082
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
0d47bbe
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
8dd81af
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
d4d922f
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
9d4fe7e
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
e6d44d2
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
53ecbaa
Adding publishing stage to python cuda-12 pipeline
jchen351 Dec 5, 2023
fea8b34
change to use python -m twine
jchen351 Dec 5, 2023
495fc0a
remove artifact_feed_url variable
jchen351 Dec 5, 2023
816cccc
Update artifact name
jchen351 Dec 6, 2023
716cd5e
Update machine pool
jchen351 Dec 6, 2023
81cfc74
Adding new pipeline for publishing cuda package to specific feed
jchen351 Dec 7, 2023
8218cc7
Adding new pipeline for publishing cuda package to specific feed
jchen351 Dec 8, 2023
9962055
Adding new pipeline for publishing cuda package to specific feed
jchen351 Dec 8, 2023
a5b2e25
Adding new pipeline for publishing cuda package to specific feed
jchen351 Dec 8, 2023
aa656ee
Adding new pipeline for publishing cuda package to specific feed
jchen351 Dec 8, 2023
2c51a16
Change eq to ne
jchen351 Dec 8, 2023
1c4a997
Update parameteres
jchen351 Dec 8, 2023
41c921e
Update DownloadPipelineArtifact
jchen351 Dec 8, 2023
6c190f6
Update DownloadPipelineArtifact
jchen351 Dec 8, 2023
618356a
Update DownloadPipelineArtifact
jchen351 Dec 8, 2023
37b73d1
Update DownloadPipelineArtifact
jchen351 Dec 8, 2023
b83db92
Update keyring artifacts-keyring
jchen351 Dec 8, 2023
31978b0
Update keyring artifacts-keyring
jchen351 Dec 8, 2023
640bf49
Update --config-file $(PYPIRC_PATH)
jchen351 Dec 8, 2023
b6187b9
Allow env:
jchen351 Dec 8, 2023
69c9dcb
Allow env:
jchen351 Dec 8, 2023
d9dbb12
Allow env:
jchen351 Dec 8, 2023
6ee946d
Allow env:
jchen351 Dec 8, 2023
e14b758
Allow env:
jchen351 Dec 8, 2023
1489b12
Allow env:
jchen351 Dec 8, 2023
b107d4c
--config-file $(PYPIRC_PATH)
jchen351 Dec 8, 2023
e34be18
Merge branch 'main' into Cjian/python-cuda12
jchen351 Dec 8, 2023
342cb09
Delete file
jchen351 Dec 8, 2023
c7367de
Update file with main
jchen351 Dec 8, 2023
520041d
Remove repo-url
jchen351 Dec 11, 2023
63c411d
Remove typo
jchen351 Dec 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,10 @@ resources:
ref: 5eda9aded5462201e6310105728d33016e637ea7

stages:
- template: stages/py-nuget-combine-cuda-stage.yml
- template: stages/py-cuda-packaging-stage.yml
parameters:
enable_linux_gpu: ${{ parameters.enable_linux_gpu }}
enable_windows_gpu: ${{ parameters.enable_windows_gpu }}
cmake_build_type: ${{ parameters.cmake_build_type }}
cuda_version: ${{ parameters.cuda_version }}
cuda_version: ${{ parameters.cuda_version }}

Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
parameters:
- name: nightly
type: string
default: '1'
- name: build_id
type: string
default: 'latest'
- name: project
type: string
default: 'Lotus'
- name: pipeline
type: string
default: 'Python-CUDA-Packaging-Pipeline'

stages:
- template: stages/py-cuda-publishing-stage.yml
parameters:
build_id: ${{ parameters.build_id }}
project: ${{ parameters.project }}
pipeline: ${{ parameters.pipeline }}
${{ if ne(parameters.nightly, '1') }}:
artifact_feed: onnxruntime-cuda-12
${{ else }}:
artifact_feed: ort-cuda-12-nightly
jchen351 marked this conversation as resolved.
Show resolved Hide resolved
Empty file.
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
parameters:
- name: build_id
type: string
- name: project
type: string
- name: pipeline
type: string
- name: artifact_feed
type: string
default: 'onnxruntime-cuda-12'
- name: depends_on
type: string
default: 'none'

stages:
- stage: Python_Publishing
# ${{ if ne(parameters.depends_on, 'none') }}:
# dependsOn: [ ${{ parameters.depends_on }} ]
# ${{ else }}:
# dependsOn:
# - Linux_Packaging_combined_GPU
variables:
${{ if eq(parameters.artifact_feed, 'ort-cuda-12-nightly') }}:
repository-url: https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/upload/
${{ if eq(parameters.artifact_feed, 'onnxruntime-cuda-12') }}:
repository-url: https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/upload/
jobs:
- job:
pool: 'onnxruntime-Ubuntu2004-AMD-CPU'
steps:
- checkout: none
- script: |
echo "Build ID: ${{ parameters.build_id }}"
echo "Project: ${{ parameters.project }}"
echo "Pipeline: ${{ parameters.pipeline }}"
echo "Artifact Feed: ${{ parameters.artifact_feed }}"
displayName: 'Print Parameters'
- task: DownloadPipelineArtifact@2
inputs:
artifact: 'onnxruntime_gpu'
targetPath: '$(Build.SourcesDirectory)/onnxruntime-gpu'
${{ if ne(parameters.build_id, 'latest') }}:
buildType: 'specific'
project: '${{ parameters.project }}'
pipeline: '${{ parameters.pipeline }}'
buildVersionToDownload: 'specific'
buildId: '${{ parameters.build_id }}'
displayName: 'Download Build Artifacts - onnxruntime-gpu'
- task: UsePythonVersion@0
displayName: 'Use Python 3.x'
- script: 'pip install twine==3.4.2'
displayName: 'Install Twine'
- task: TwineAuthenticate@1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like this:

steps:
- task: TwineAuthenticate@1
  displayName: 'Twine Authenticate '
  inputs:
    artifactFeed: 'PublicPackages/ORT-Nightly'

- script: ' find . -type f -name ''*.whl'' | xargs -I % python -m twine upload --skip-existing % -r ORT-Nightly --config-file $(PYPIRC_PATH)'
  workingDirectory: '$(System.DefaultWorkingDirectory)/training cuda wheels'
  displayName: 'Uploading wheels'

displayName: 'Twine Authenticate '
inputs:
artifactFeed: PublicPackages/${{ parameters.artifact_feed }}
- script: 'python -m twine upload -r ${{ parameters.artifact_feed }} --config-file $(PYPIRC_PATH) --non-interactive --skip-existing *.whl'
workingDirectory: '$(Build.SourcesDirectory)/onnxruntime-gpu'
displayName: 'Uploading wheels to ${{ parameters.artifact_feed }}'
retryCountOnTaskFailure: 3
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)

Loading