Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type instability in DataLoader #168

Open
Vilin97 opened this issue Oct 18, 2023 · 1 comment
Open

Type instability in DataLoader #168

Vilin97 opened this issue Oct 18, 2023 · 1 comment

Comments

@Vilin97
Copy link

Vilin97 commented Oct 18, 2023

Is DataLoader type-unstable? Seems so. Does it have to be type-unstable?

using Flux
function test_data_loader(data_loader)
    for (x,y) in data_loader
        continue
    end
end
X = rand(Float32, 2, 10)
Y = rand(Float32, 2, 10)
data_loader = Flux.DataLoader((data=X, label=Y))
@code_warntype test_data_loader(data_loader) # lots of red

If it is type-unstable and it's the intended behavior, it would be great to add a sentence This function is type-unstable to the docstring.

@ToucheSir
Copy link
Contributor

I don't think it would be impossible to make iterate(::DataLoader) type stable, but it would be tricky since there are so many conditionally-applied wrappers here. @lorenzoh and @CarloLucibello thoughts? Do you recall why we moved away from creating the wrappers in the DataLoader constructor?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants