Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Test runner(Python SDK) fails when generic type is specified as a type for pardo element #33189

Open
1 of 17 tasks
m-morozov-kw opened this issue Nov 22, 2024 · 3 comments

Comments

@m-morozov-kw
Copy link

m-morozov-kw commented Nov 22, 2024

What happened?

Test env details:
Python 3.11.3
Beam SDK version: 2.60.0

Hello! We want to use generic types as a type annotation for elements in Pardo in Python SDK to improve code readability, but it seems like Test Runner doesn't support that. Are there any options for how we can work around that?
Please find the reproduction snippet:
Create a file like /tmp/test_pardo.py

import apache_beam as beam
from typing import TypeVar, Generic

from apache_beam.testing.util import assert_that, equal_to
from apache_beam.testing.test_pipeline import TestPipeline

T = TypeVar("T")


class ResultContainer(Generic[T]):
    def __init__(self, payload: T) -> None:
        self.payload = payload


class SomeDoFn(beam.DoFn):
    def process(self, data: ResultContainer[int]):
        yield data.payload + 1


def test_pardo():
    with TestPipeline() as p:
        output = p | beam.Create([ResultContainer(1)]) | beam.ParDo(SomeDoFn())
        assert_that(
            label="check result",
            actual=output,
            matcher=equal_to([2]),
        )

Run it as a test

pytest /tmp/test_pardo.py

Expected: test passes
Actual behavior:

    def __subclasscheck__(self, cls):
>       raise TypeError("Subscripted generics cannot be used with"
                        " class and instance checks")
E       TypeError: Subscripted generics cannot be used with class and instance checks

Thank you

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@m-morozov-kw
Copy link
Author

m-morozov-kw commented Nov 22, 2024

.add-labels python

Copy link
Contributor

Label generic type annotations cannot be managed because it does not exist in the repo. Please check your spelling.

@m-morozov-kw m-morozov-kw changed the title [Bug]: Test runner(Oython SDK) fails when generic type is specified as a type for pardo element [Bug]: Test runner(Python SDK) fails when generic type is specified as a type for pardo element Nov 22, 2024
@liferoad
Copy link
Collaborator

@jrmccluskey do we support this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants