Skip to content

Commit

Permalink
[SPARK-50211][PYTHON][CONNECT][TESTS] Skip FeatureTests if torch is n…
Browse files Browse the repository at this point in the history
…ot installed

### What changes were proposed in this pull request?

This PR proposes to skip FeatureTests if torch is not installed

### Why are the changes needed?

If tourch isn't available, we should run the tests `FeatureTests `. That's an optional dependency.

To fix Python 3.13 build
https://github.com/apache/spark/actions/runs/11654255828/job/32447601706

### Does this PR introduce _any_ user-facing change?

No, test-only.

### How was this patch tested?

Manually tested.

### Was this patch authored or co-authored using generative AI tooling?

No

Closes apache#48743 from HyukjinKwon/SPARK-50211.

Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
HyukjinKwon authored and dongjoon-hyun committed Nov 4, 2024
1 parent a0b4205 commit c53dac0
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions python/pyspark/ml/tests/connect/test_legacy_mode_feature.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,10 @@
from pyspark.util import is_remote_only
from pyspark.sql import SparkSession
from pyspark.testing.connectutils import should_test_connect, connect_requirement_message

from pyspark.ml.tests.connect.test_connect_classification import (
have_torch,
torch_requirement_message,
)

if should_test_connect:
from pyspark.ml.connect.feature import (
Expand Down Expand Up @@ -196,8 +199,10 @@ def test_array_assembler(self):


@unittest.skipIf(
not should_test_connect or is_remote_only(),
connect_requirement_message or "pyspark-connect cannot test classic Spark",
not should_test_connect or not have_torch or is_remote_only(),
connect_requirement_message
or torch_requirement_message
or "pyspark-connect cannot test classic Spark",
)
class FeatureTests(FeatureTestsMixin, unittest.TestCase):
def setUp(self) -> None:
Expand Down

0 comments on commit c53dac0

Please sign in to comment.