Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Windows ARM64 building & packaging #687

Merged
merged 77 commits into from
Jul 24, 2024
Merged

Improve Windows ARM64 building & packaging #687

merged 77 commits into from
Jul 24, 2024

Conversation

skyline75489
Copy link
Contributor

@skyline75489 skyline75489 commented Jul 10, 2024 •
edited
Loading

A major overhaul of the NuGet packaging pipeline, including:

  • Restructure capi & NuGet packaging pipeline to make multi-platform nupkg possible. The NuGet artifacts are separated by EP(CPU, Cuda and DirectML). A single nupkg contains native libraries for different os & arch.
  • Make win-arm64 CPU packages use DLLs from NuGet package, just like win-x64.
  • Enable win-arm64 DirectML NuGet build & packaging (Internal work item: https://task.ms/aii/40037)
  • Enable linux-x64 CPU & CUDA NuGet build & packaging (Internal work item: https://task.ms/aii/34234)
  • Continue the work of Initialize DotNet package validation pipeline #683 to make it work with ARM64.

Run: https://dev.azure.com/aiinfra/ONNX%20Runtime/_build/results?buildId=507547&view=results

Prior work: #498
Supports: #637

Future work: Needs #693 to fix Linux NuGet.

@skyline75489 skyline75489 marked this pull request as ready for review July 12, 2024 00:20
@skyline75489 skyline75489 requested a review from a team as a code owner July 12, 2024 00:20
@skyline75489 skyline75489 changed the title [WIP] Improve ARM64 building & packaging Improve Windows ARM64 building & packaging Jul 12, 2024
@skyline75489 skyline75489 changed the title Improve Windows ARM64 building & packaging [Don't merge yet] Improve Windows ARM64 building & packaging Jul 12, 2024
@skyline75489 skyline75489 mentioned this pull request Jul 12, 2024
Closed
skyline75489 and others added 6 commits July 15, 2024 13:58
@skyline75489
Merge branch 'main' into jialli/for-arm64
5e03340
@skyline75489
Oga global functions should be inline (#702)
061a0db
@baijumeswani @skyline75489
Load onnxruntime shared lib from onnxruntime python package (#662)
b54c1e0
@baijumeswani @skyline75489
Fix windows arm64 pipeline (#704)
518ed1b
@kunal-vaishnavi @skyline75489
Add example to build Phi-3 vision ONNX models (#705)
72a3c41 
### Description

This PR open-sources the scripts used to generate the Phi-3 vision ONNX
models that run with ONNX Runtime GenAI. The extra files needed for
generating the Phi-3 vision ONNX models have been uploaded to the
Hugging Face repos.

-
[`microsoft/Phi-3-vision-128k-instruct-onnx-cpu`](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct-onnx-cpu/tree/main/onnx)
-
[`microsoft/Phi-3-vision-128k-instruct-onnx-cuda`](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct-onnx-cuda/tree/main/onnx)
-
[`microsoft/Phi-3-vision-128k-instruct-onnx-directml`](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct-onnx-directml/tree/main/onnx)

### Motivation and Context

This PR allows users to build the ONNX models needed for Phi-3 vision.
It also helps the following issues.

- #571
- #685
@skyline75489
Merge branch 'main' into jialli/for-arm64
c4348a7
@skyline75489 skyline75489 changed the title [Don't merge yet] Improve Windows ARM64 building & packaging Improve Windows ARM64 building & packaging Jul 17, 2024
baijumeswani
baijumeswani approved these changes Jul 23, 2024
View reviewed changes
Copy link
Collaborator

@baijumeswani baijumeswani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for making this work.

@skyline75489 skyline75489 merged commit 1f96323 into main Jul 24, 2024
13 checks passed
@skyline75489 skyline75489 deleted the jialli/for-arm64 branch July 24, 2024 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snnn snnn snnn left review comments

@baijumeswani baijumeswani baijumeswani approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@skyline75489 @snnn @baijumeswani @kunal-vaishnavi