Authorization for InferenceGraph (Serverless) #345

israel-hdez · 2025-01-08T23:20:59Z

Description

This adds a new controller for KServe InferenceGraph resources. This new controller will have the responsibility of creating Authorino AuthConfig resources (similarly to InferenceServices case), when authorization is available in ODH platform.

InferenceGraphs can now be annotated with security.opendatahub.io/enable-auth: "true" to secure InferenceGraphs and only serve requests that are authorized.

Requires PRs:

Fixes: https://issues.redhat.com/browse/RHOAIENG-13449

How Has This Been Tested?

Deploy (since odh-operator is changed, it would be easier to use custom builds of everything):
- odh-model-controller from this PR
- KServe using code of dependent PR
- operator using code of dependent PR
Deploy InferenceGraph example of KServe website: https://kserve.github.io/website/latest/modelserving/inference_graph/image_pipeline/
Check that the InferenceGraph is accessible
Annotate the InferenceGraph with security.opendatahub.io/enable-auth: "true"
Check that the InferenceGraph is accessible only to authorized requests

Merge criteria:

The commits are squashed in a cohesive manner and have meaningful messages.
Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
The developer has manually tested the changes and verified that the changes work

openshift-ci · 2025-01-08T23:21:06Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

config/crd/external/authorino.kuadrant.io_authconfigs.yaml

internal/controller/serving/inferencegraph_controller_test.go

internal/controller/serving/inferencegraph_controller.go

This adds a new controller for KServe InferenceGraph resources. This new controller will have the responsibility of creating Authorino AuthConfig resources (similarly to InferenceServices case), when authorization is available in ODH platform. InferenceGraphs can now be annotated with `security.opendatahub.io/enable-auth: "true"` to secure InferenceGraphs and only serve requests that are authorized. Signed-off-by: Edgar Hernández <[email protected]>

Signed-off-by: Edgar Hernández <[email protected]>

openshift-ci · 2025-01-10T19:56:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: israel-hdez, spolti

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [israel-hdez,spolti]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

spolti · 2025-01-10T21:08:55Z

/lgtm

zdtsw · 2025-01-13T16:12:41Z

i assume we need to add "InferenceService" with rbac into Operator?

zdtsw · 2025-01-13T16:45:25Z

i assume we need to add "InferenceService" with rbac into Operator?

nvm, it should be fine. we already have inferencegraphs, finalizers might not needed.

* update global ca bundle logic and storage-config logic to follow up odh operator pr(1339) (#308) Signed-off-by: jooho lee <[email protected]> * disable dashboard and fix servingruntime display name Signed-off-by: jooho lee <[email protected]> * Use the main branch to build stable image tags, incubating for latest image tags (#316) Signed-off-by: Hannah DeFazio <[email protected]> * [RHOAIENG-13638] - Do not allow isvc creation in protected isvc (#311) * [RHOAIENG-13638] - Do not allow isvc creation in protected namespace chore: Fixes [RHOAIENG-13638] - Kserve model is not Ready after a kserve model is created and deleted from istio-system namespace Signed-off-by: Spolti <[email protected]> * review suggestions Signed-off-by: Spolti <[email protected]> * Update controllers/webhook/isvc_validator.go Co-authored-by: Edgar Hernández <[email protected]> Signed-off-by: Spolti <[email protected]> --------- Signed-off-by: Spolti <[email protected]> Co-authored-by: Edgar Hernández <[email protected]> * update gitaction based on branch strategy change (#322) Signed-off-by: jooho lee <[email protected]> * feat: added performance metric grpahs config for nvidia nim (#320) * feat: added performance metric grpahs config for nvidia nim Signed-off-by: Tomer Figenblat <[email protected]> * chore: modifyed the runtime id annotation Co-authored-by: Edgar Hernández <[email protected]> Signed-off-by: Tomer Figenblat <[email protected]> --------- Signed-off-by: Tomer Figenblat <[email protected]> Co-authored-by: Edgar Hernández <[email protected]> * Add NIM flag logic (#312) Signed-off-by: mtrujillo <[email protected]> * Grab the old release tag based on creation date Signed-off-by: Hannah DeFazio <[email protected]> * Updated the checkout code command Signed-off-by: Mariah Holder <[email protected]> * Updated the checkout code command (#329) Signed-off-by: Mariah Holder <[email protected]> Co-authored-by: Mariah Holder <[email protected]> * Add reconciliation for Kserve Raw (#274) Signed-off-by: Vedant Mahabaleshwarkar <[email protected]> * chore: added pagination support for nim catalog response (#332) Signed-off-by: Tomer Figenblat <[email protected]> * feat(mr): enable model registry inference reconcile (#326) Signed-off-by: Alessio Pragliola <[email protected]> * add upstream release metadata (#333) Signed-off-by: heyselbi <[email protected]> * Migration to kubebuilder v4 (#324) * Migration to kubebuilder v4 Signed-off-by: Edgar Hernández <[email protected]> * Restore MR E2Es Signed-off-by: Edgar Hernández <[email protected]> * Restore top-level files Signed-off-by: Edgar Hernández <[email protected]> * Cleaning Signed-off-by: Edgar Hernández <[email protected]> * Fixing Makefile and Containerfile Signed-off-by: Edgar Hernández <[email protected]> * Linter fixes Signed-off-by: Edgar Hernández <[email protected]> * Initial rework of manifests Signed-off-by: Edgar Hernández <[email protected]> * Fix manifests Signed-off-by: Edgar Hernández <[email protected]> * Fix lint issues Signed-off-by: Edgar Hernández <[email protected]> * Deactivate E2Es Because setup is not automated, yet. Signed-off-by: Edgar Hernández <[email protected]> * Feedback: Filippe Signed-off-by: Edgar Hernández <[email protected]> * Feedback: Filippe Test differences after `go mod tidy` Signed-off-by: Edgar Hernández <[email protected]> * Apply suggestions from code review: Filippe Co-authored-by: Filippe Spolti <[email protected]> Signed-off-by: Edgar Hernández <[email protected]> * Feedback: Filippe * Pin go-toolset base image in Containerfile. * Add `gosec` linter Signed-off-by: Edgar Hernández <[email protected]> * Update config/prometheus/monitor.yaml Co-authored-by: Filippe Spolti <[email protected]> Signed-off-by: Edgar Hernández <[email protected]> * Feedback: Filippe * Small change to comments in Makefile, to make the text clearer. * Remove (again) `gosec` linter Signed-off-by: Edgar Hernández <[email protected]> * Fix panic on controller startup Signed-off-by: Edgar Hernández <[email protected]> --------- Signed-off-by: Edgar Hernández <[email protected]> Co-authored-by: Filippe Spolti <[email protected]> * chore: use naming convention for resources created by nim (#340) * chore: use naming convention for resources created by nim Signed-off-by: Tomer Figenblat <[email protected]> * test: added assertions for dyamic nim resources name Signed-off-by: Tomer Figenblat <[email protected]> --------- Signed-off-by: Tomer Figenblat <[email protected]> * chore: set nim runtime api call page size to 1000 (#344) Signed-off-by: Tomer Figenblat <[email protected]> * Nim enablement change default to managed and add clean up job (#342) * initial commit for clean up of nim and managed set as default Signed-off-by: mtrujillo <[email protected]> * remove space Signed-off-by: mtrujillo <[email protected]> * fix code length for linting Signed-off-by: mtrujillo <[email protected]> * fixed comments / adjusted import Signed-off-by: mtrujillo <[email protected]> --------- Signed-off-by: mtrujillo <[email protected]> * chore: added new graph object for nim runtimes (#334) * chore: added new graph object for nim runtimes Signed-off-by: Tomer Figenblat <[email protected]> * chore: added REQUEST_OUTCOMES nim graph Signed-off-by: Tomer Figenblat <[email protected]> * chore: added fixed typo in nim query object Signed-off-by: Tomer Figenblat <[email protected]> * chore: fixed typo in nim query object Signed-off-by: Tomer Figenblat <[email protected]> * chore: added initial query for nim gpu cache usage Signed-off-by: Tomer Figenblat <[email protected]> * chore: rewrite queries for nim new graphs Signed-off-by: Tomer Figenblat <[email protected]> --------- Signed-off-by: Tomer Figenblat <[email protected]> * Update ovms to current build (#343) Signed-off-by: Steve Grubb <[email protected]> Co-authored-by: Steve Grubb <[email protected]> * Automatically inject expected ODH annotations to InferenceGraph and InferenceServices (#339) * Implementation of ODH defaulters for InferenceGraph and InferenceService On creation of InferenceGraph or InferenceService resources, the following default annotations will be added: * `serving.knative.openshift.io/enablePassthrough: true` * `sidecar.istio.io/inject: true` * `sidecar.istio.io/rewriteAppHTTPProbers: true` The annotations are added only for Serverless mode, and only if they are missing. Signed-off-by: Edgar Hernández <[email protected]> * Feedback: Filippe Extract "ENABLE_WEBHOOKS" string to constant Signed-off-by: Edgar Hernández <[email protected]> --------- Signed-off-by: Edgar Hernández <[email protected]> * Authorization for InferenceGraph (Serverless) (#345) * Authorization for InferenceGraph (Serverless) This adds a new controller for KServe InferenceGraph resources. This new controller will have the responsibility of creating Authorino AuthConfig resources (similarly to InferenceServices case), when authorization is available in ODH platform. InferenceGraphs can now be annotated with `security.opendatahub.io/enable-auth: "true"` to secure InferenceGraphs and only serve requests that are authorized. Signed-off-by: Edgar Hernández <[email protected]> * Feedback: Filippe - Event when auth is not available Signed-off-by: Edgar Hernández <[email protected]> --------- Signed-off-by: Edgar Hernández <[email protected]> * [RHOAIENG-10293] add metrics resources for rawdeployment (#347) * [RHOAIENG-10293] add metrics resources for rawdeployment Signed-off-by: Vedant Mahabaleshwarkar <[email protected]> * [RHOAIENG-10293] address feedback Signed-off-by: Vedant Mahabaleshwarkar <[email protected]> --------- Signed-off-by: Vedant Mahabaleshwarkar <[email protected]> * [RHOAIENG-16851] rawdeployment route bug fixes (#341) Signed-off-by: Vedant Mahabaleshwarkar <[email protected]> * fix null pointer error (RHOAIENG-18228) (#349) Signed-off-by: jooho lee <[email protected]> * remove old file Signed-off-by: jooho lee <[email protected]> update go.mod Signed-off-by: jooho lee <[email protected]> --------- Signed-off-by: jooho lee <[email protected]> Signed-off-by: Hannah DeFazio <[email protected]> Signed-off-by: Spolti <[email protected]> Signed-off-by: Tomer Figenblat <[email protected]> Signed-off-by: mtrujillo <[email protected]> Signed-off-by: Mariah Holder <[email protected]> Signed-off-by: Vedant Mahabaleshwarkar <[email protected]> Signed-off-by: Alessio Pragliola <[email protected]> Signed-off-by: heyselbi <[email protected]> Signed-off-by: Edgar Hernández <[email protected]> Signed-off-by: Steve Grubb <[email protected]> Co-authored-by: Hannah DeFazio <[email protected]> Co-authored-by: Filippe Spolti <[email protected]> Co-authored-by: Edgar Hernández <[email protected]> Co-authored-by: Tomer Figenblat <[email protected]> Co-authored-by: Marcus Trujillo <[email protected]> Co-authored-by: Mariah Holder <[email protected]> Co-authored-by: Mariah Holder <[email protected]> Co-authored-by: Vedant Mahabaleshwarkar <[email protected]> Co-authored-by: Tomer Figenblat <[email protected]> Co-authored-by: Alessio Pragliola <[email protected]> Co-authored-by: Selbi Nuryyeva <[email protected]> Co-authored-by: Steven Grubb <[email protected]> Co-authored-by: Steve Grubb <[email protected]>

israel-hdez requested review from spolti and Jooho January 8, 2025 23:20

openshift-ci bot added the do-not-merge/work-in-progress label Jan 8, 2025

openshift-ci bot added the approved label Jan 8, 2025

This was referenced Jan 9, 2025

feat: KServe InferenceGraph auth configurations opendatahub-io/opendatahub-operator#1483

Closed

feat: KServe InferenceGraph auth configurations [main] opendatahub-io/opendatahub-operator#1491

Merged

israel-hdez force-pushed the j13449-ig-auth branch from 4572519 to e82444d Compare January 10, 2025 00:38

israel-hdez marked this pull request as ready for review January 10, 2025 00:38

openshift-ci bot removed the do-not-merge/work-in-progress label Jan 10, 2025

openshift-ci bot requested review from mwaykole and rnetser January 10, 2025 00:38

israel-hdez removed request for mwaykole and rnetser January 10, 2025 00:39

spolti reviewed Jan 10, 2025

View reviewed changes

config/crd/external/authorino.kuadrant.io_authconfigs.yaml Show resolved Hide resolved

spolti reviewed Jan 10, 2025

View reviewed changes

internal/controller/serving/inferencegraph_controller_test.go Show resolved Hide resolved

spolti reviewed Jan 10, 2025

View reviewed changes

internal/controller/serving/inferencegraph_controller_test.go Show resolved Hide resolved

spolti reviewed Jan 10, 2025

View reviewed changes

internal/controller/serving/inferencegraph_controller.go Outdated Show resolved Hide resolved

israel-hdez added 2 commits January 10, 2025 13:37

Feedback: Filippe - Event when auth is not available

dc60d27

Signed-off-by: Edgar Hernández <[email protected]>

israel-hdez force-pushed the j13449-ig-auth branch from e82444d to dc60d27 Compare January 10, 2025 19:40

spolti approved these changes Jan 10, 2025

View reviewed changes

openshift-ci bot assigned spolti Jan 10, 2025

openshift-ci bot added the lgtm label Jan 10, 2025

openshift-merge-bot bot merged commit b13e9e4 into opendatahub-io:incubating Jan 10, 2025
7 checks passed

israel-hdez deleted the j13449-ig-auth branch January 10, 2025 21:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Authorization for InferenceGraph (Serverless) #345

Authorization for InferenceGraph (Serverless) #345

israel-hdez commented Jan 8, 2025 •

edited

Loading

openshift-ci bot commented Jan 8, 2025

openshift-ci bot commented Jan 10, 2025

spolti commented Jan 10, 2025

zdtsw commented Jan 13, 2025

zdtsw commented Jan 13, 2025

Authorization for InferenceGraph (Serverless) #345

Authorization for InferenceGraph (Serverless) #345

Conversation

israel-hdez commented Jan 8, 2025 • edited Loading

Description

How Has This Been Tested?

Merge criteria:

openshift-ci bot commented Jan 8, 2025

openshift-ci bot commented Jan 10, 2025

spolti commented Jan 10, 2025

zdtsw commented Jan 13, 2025

zdtsw commented Jan 13, 2025

israel-hdez commented Jan 8, 2025 •

edited

Loading