Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix upsampling misalignments after resizing #63

Closed
wants to merge 2 commits into from

Conversation

nandometzger
Copy link
Member

In the current implementation, we have the following problem: If the user input an image of odd shape (either not divible by 8 or aspect ratio which don't allow to resize to a multiple of 8) the code does something unexpected: It (or the stable diffusion backbone) cut's of the boarders to make it divisible by 8. After that we just resample it to original resolution. This can cause a misalignment (depht vs RGB) of up to 7 pixels in the extrem case at the bottom right corner. I propose to modify the resize_max_resolution function to slightly change the aspect ratio (which will be at max 768/7~=1% off from the true ratio), but this will in turn fix the misalignment when upsampling to match the resolution.

For now, I set the div8 boolean to false, so changes will not have any effects. Does any of the test-datasets have edges that are not divisible by 8? because that could mess with the evaluation protocol and the results are not exactly reproduceable anymore. If not, I propose to change div8 to True

In the current implementation, we have the following problem:
If the user input an image of odd shape (either not divible by 8 or aspect ratio which don't allow to resize to a multiple of 8) the code does something unexpected: It (or the stable diffusion backbone) cut's of the boarders to make it divisible by 8. After that we just resample it to original resolution. This can cause a misalignment (depht vs RGB) of up to 7 pixels in the extrem case at the bottom right corner.
I propose to modify the resize_max_resolution function to slightly change the aspect ratio (which will be at max 768/7~=1% off from the true ratio), but this will in turn fix the misalignment when upsampling to match the resolution.
@nandometzger
Copy link
Member Author

code quality check failed :'(

@nandometzger nandometzger requested a review from markkua May 13, 2024 07:57
@toshas
Copy link
Collaborator

toshas commented Jun 11, 2024

Maybe it is better to take the padding route from diffusers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants