Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves
Resolves #593
Proposed Changes
This PR adds a framework for testing pixel differences between the CPU and GPU rendering pipelines, and adds tests that ensure that those differences remain below a certain amount.
The diff tests work much like the integration tests--they run as Scratch projects that control the test framework via "say" blocks. At any point, the project can ask the framework to measure the difference in terms of either percentage or absolute number of pixels, and then assert that difference is below a certain value.
One pixel's worth of difference is measured by a pixel differing from another by 255, the maximum color value (so e.g. a difference of 127 would be half a pixel of difference). The difference per pixel is averaged across all 3 color channels.
If the assertions fail, then an image will be saved to the
test/integration/diff-tests/fails
directory that contains the CPU rendering, GPU rendering, and comparison of the two. This will allow developers to more easily visually inspect rendering errors.One thing I'm not sure how to account for is the fact that the CPU pipeline always uses nearest-neighbor sampling while the GPU pipeline uses linear interpolation for vector sprites. So far I've been getting around this by avoiding cases in my tests where linear interpolation would be used on the GPU. Perhaps a good way to solve this would be to introduce a "touching color mode" for GPU rendering that always uses nearest-neighbor interpolation--this would also have the effect of making the two pipelines more consistent with each other outside of testing as well.
Reason for Changes
See #593 for rationale
Test Coverage
Included are several diff tests: