Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add in Dockerfile.hpu.ubi #602

Open
wants to merge 1 commit into
base: v1.18.0
Choose a base branch
from
Open

Add in Dockerfile.hpu.ubi #602

wants to merge 1 commit into from

Conversation

Xaenalt
Copy link

@Xaenalt Xaenalt commented Dec 9, 2024

Creates the 1.18.0 version of the UBI dockerfile for use with Red Hat OpenShift AI

@Xaenalt Xaenalt mentioned this pull request Dec 10, 2024
Closed
uMartinXu
uMartinXu reviewed Dec 12, 2024
View reviewed changes
Dockerfile.hpu.ubi Show resolved Hide resolved
Dockerfile.hpu.ubi Outdated
DISABLE_LOGPROBS_DURING_SPEC_DECODING=false

USER 2000
ENTRYPOINT ["python3", "-m", "vllm_tgis_adapter", "--uvicorn-log-level=warning"]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR looks good.
My question is how could 1.19.0 and the future releases also has the feature. Should the PR be submitted to main branch?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the main part that needs updating release-to-release is L7: ENV HABANA_SOFTWARE_VERSION="1.18.0-524"

@uMartinXu uMartinXu mentioned this pull request Dec 16, 2024
Merged
vbedida79 added a commit to vbedida79/intel-technology-enabling-for-openshift that referenced this pull request Dec 16, 2024
@vbedida79
tests_gaudi: Added L2 vllm workload
5f35eab 
vllm gaudi ubi image based on PR HabanaAI/vllm-fork#602

Signed-off-by: vbedida79 <[email protected]>
vbedida79 added a commit to vbedida79/intel-technology-enabling-for-openshift that referenced this pull request Dec 16, 2024
@vbedida79
tests_gaudi: Added L2 vllm workload
4e68146 
vllm gaudi ubi image based on PR HabanaAI/vllm-fork#602

Signed-off-by: vbedida79 <[email protected]>
vbedida79 added a commit to vbedida79/intel-technology-enabling-for-openshift that referenced this pull request Dec 16, 2024
@vbedida79
tests_gaudi: Added L2 vllm workload
dd2a16c 
vllm gaudi ubi image based on PR HabanaAI/vllm-fork#602

Signed-off-by: vbedida79 <[email protected]>
vbedida79 added a commit to vbedida79/intel-technology-enabling-for-openshift that referenced this pull request Dec 16, 2024
@vbedida79
tests_gaudi: Added L2 vllm workload
9a963b3 
vllm gaudi ubi image based on PR HabanaAI/vllm-fork#602

Signed-off-by: vbedida79 <[email protected]>
@uMartinXu
Copy link

we have verified the PR in https://github.com/intel/intel-technology-enabling-for-openshift/tree/main/tests/gaudi/l2#vllm
It works fine, I think with some modification in the comments in the PR the PR is good to merge.
Thanks!

@Xaenalt
Copy link
Author

Xaenalt commented Dec 17, 2024

What other changes are needed?

@uMartinXu
Copy link

uMartinXu commented Dec 17, 2024 •
edited
Loading

What other changes are needed?

The PR itself is good.
I think the PR should be merged into main branch so the following 1.19.0,1.20.0 releases can have the openShift and RHEL support.
Besides the related README file also needs to be updated. see https://github.com/HabanaAI/vllm-fork/blob/habana_main/README_GAUDI.md
So users can know how to build and run vLLM Gaudi support on RHEL and OpenShift.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@uMartinXu uMartinXu uMartinXu left review comments

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Xaenalt @uMartinXu @vaibhavjainwiz