Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark client mode doesn’t support PodGroup Template? #3744

Open
xuanlang opened this issue Sep 23, 2024 · 1 comment
Open

Spark client mode doesn’t support PodGroup Template? #3744

xuanlang opened this issue Sep 23, 2024 · 1 comment
Labels
kind/question Categorizes issue related to a new question

Comments

@xuanlang
Copy link

Please describe your problem in detail

spark-version:3.5.2
volcano-version:1.9.0

When I submit a job using spark-sql, I always get the following error:

Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://*.*.*.*:10000/api/v1/namespaces/ns-hdf/pods. Message: admission webhook "validatepod.volcano.sh" denied the request: failed to get PodGroup for pod <ns-hdf/tpcds-10-face4e921daafccd-exec-1>: podgroups.scheduling.volcano.sh "spark-10d04fd77c9545059297dae304568d93-podgroup" not found. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "validatepod.volcano.sh" denied the request: failed to get PodGroup for pod <ns-hdf/tpcds-10-face4e921daafccd-exec-1>: podgroups.scheduling.volcano.sh "spark-10d04fd77c9545059297dae304568d93-podgroup" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=null, status=Failure, additionalProperties={}).

Any other relevant information

No response

@xuanlang xuanlang added the kind/question Categorizes issue related to a new question label Sep 23, 2024
@xuanlang xuanlang changed the title Spark SQL doesn’t support PodGroup Template? Spark client mode doesn’t support PodGroup Template? Sep 23, 2024
@Monokaix
Copy link
Member

You can refer to https://spark.apache.org/docs/latest/running-on-kubernetes.html#using-volcano-as-customized-scheduler-for-spark-on-kubernetes and set --conf spark.kubernetes.scheduler.volcano.podGroupTemplateFile=/path/to/podgroup-template.yaml flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/question Categorizes issue related to a new question
Projects
None yet
Development

No branches or pull requests

2 participants