-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rewrite: removing dimensionality and allow non-bijective transformations #183
Conversation
Let's perhaps make a push to finish the work here. @torfjelde |
@@ -0,0 +1,144 @@ | |||
# Bijectors.jl |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@devmotion I'm curious what you think of this design:) You can ignore the code for now, the main thing is just whether or not you're happy with the idea of doing
forward(b, x) = forward_single(b, x)
# Non-batched version
forward_single(b, x) = ...
# Batched version
forward_multiple(b, x) = ...
We can then introduce a Batch
type which generalizes ColVecs
etc., and do
forward(b, x::AbstractBatch) = forward_multiple(b, x)
My plan is to split this into two PRs:
- Remove all "official" support for batched computation, thus always assuming that the input given represents a single input (some bijectors might still support it, but there's not going to be a "official" support for it). In this PR there is no
forward_single
, etc., justforward
. - Add support for batching and overload
forward_single
instead offorward
.
But ideally we'd fully adopt the ChangesOfVariables.jl interface, i.e. replace forward
with with_logabsdet_jacobian
. The issue here is that we of course can no longer do
with_logabsdet_jacobian(b, x) = with_logabsdet_jacobian_single(b, x)
etc. which makes me want to keep forward
or ChangesOfVariables decides to take on a similar interface and encourage people to instead implement with_logabsdet_jacobian_single
.
Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it will be a huge improvement if batching is declared more explicitly. Is there a specific reason why one needs forward_single
and forward_multiple
instead of just forward(::MyBijector, x)
and forward(::MyBijector, x::AbstractBatch)
? More functions means more entry points and hence more possible confusion for developers. If there's no dedicated _single
function anymore, it would also work better with the CoV API, I assume?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue is method ambiguity 😕 If only forward
or with_logabsdet_jacobian
is the entry-point, then we cannot provide sane defaults, e.g. forward(b, x::AbstractBatch)
without every other implementation of forward
being very explicit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah this is annoying - but there are multiple different options I think. One would be e.g. to implement (possibly different) default implementations but not as forward(...)
but some forward_batch_style1(..)
(whatever) and then to "activate" it by defining forward(b::MyBijector, x::AbstractBatch) = forward_batch_style1(b, x)
etc. I.e., batch support would be enabled manually but without much effort. And one could reduce the amount of code even more with some helper macro.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah that works of course, but I think this will be easy to forget vs. always just implemented *_single
and batching is guaranteed to at least work.
Both approaches feel sub-optimal 😕
@torfjelde do we still need this after #214? |
No this can be closed:) |
This is a draft for a proper overhaul of Bijectors.jl
Goals with this PR are:
rv
fromforward(b::Bijector, x)
result #41transform!
,logabsdetjac!
.Also, this is likely to result end up including a lot of changes, so we might end up splitting this into multiple PRs once it becomes more than a draft. But for now it's all here.