Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CPU/GPU image diff testing #668

Draft
wants to merge 4 commits into
base: develop
Choose a base branch
from

Conversation

adroitwhiz
Copy link
Contributor

Resolves

Resolves #593

Proposed Changes

This PR adds a framework for testing pixel differences between the CPU and GPU rendering pipelines, and adds tests that ensure that those differences remain below a certain amount.

The diff tests work much like the integration tests--they run as Scratch projects that control the test framework via "say" blocks. At any point, the project can ask the framework to measure the difference in terms of either percentage or absolute number of pixels, and then assert that difference is below a certain value.

One pixel's worth of difference is measured by a pixel differing from another by 255, the maximum color value (so e.g. a difference of 127 would be half a pixel of difference). The difference per pixel is averaged across all 3 color channels.

If the assertions fail, then an image will be saved to the test/integration/diff-tests/fails directory that contains the CPU rendering, GPU rendering, and comparison of the two. This will allow developers to more easily visually inspect rendering errors.

One thing I'm not sure how to account for is the fact that the CPU pipeline always uses nearest-neighbor sampling while the GPU pipeline uses linear interpolation for vector sprites. So far I've been getting around this by avoiding cases in my tests where linear interpolation would be used on the GPU. Perhaps a good way to solve this would be to introduce a "touching color mode" for GPU rendering that always uses nearest-neighbor interpolation--this would also have the effect of making the two pipelines more consistent with each other outside of testing as well.

Reason for Changes

See #593 for rationale

Test Coverage

Included are several diff tests:

  • basic.sb3
    • This is a single Scratch cat rendered at 100% size in the middle of the screen.
  • blending.sb3
    • This is a few sprites with semitransparent gradients layered on top of each other which tests alpha blending.
  • effects-*.sb3
    • These each test one specific effect.
  • effects-all.sb3.skip
    • This tests all combinations of effects together. It's being skipped because there are currently 2^7 such combinations and they all fail, each generating an image.
  • rotation-90deg.sb3
    • This tests a sprite that rotates in 90-degree increments.
  • rotation-30deg.sb3
    • This tests a sprite that rotates in 30-degree increments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test the software rendering pipeline's results by comparing them to the GPU
1 participant