basic API test suite #1974
basic API test suite #1974
Conversation
- Added tests for all obj creation, transformation, window + system info graphics methods
- Added half the state methods for graphics + added placeholders for missing drawing methods
- Added TestMethod:assertNotNil() for quick nil checking
- Added time total to the end of each module summary in console log to match file output
- Removed a bunch of unessecary nil checks
- Removed :release() from test methods, collectgarbage("collect") is called between methods instead
- Renamed /output to /examples to avoid confusion
- Replaced love.filesystem.newFile with love.filesystem.openFile
- Replaced love.math.noise with love.math.perlinNoise / love.math.simplexNoise
- Fixed newGearJoint throwing an error in 12 as body needs to be dynamic not static now
- Some general cleanup, incl. better comments and time format in file output
dorny/test-reporter@v1 will fail to parse <failure> tags that are empty as it's expected a message both in the attr and inside
added md support to work with love-test-report, a basic action made to just dump md files into repo checks
|
I think the place I'd expect to have object tests is inside the file for the module that owns the object, e.g. |
|
Sure! Can do that - do you still want them accessible under the love.test.objects "module" or would you prefer having them directly under the module, i.e. love.test.graphics.Rasterizer()? |
|
Directly under the module like |
- added mostly all graphics draw tests - added a basic set of "expected" images (generated from successful tests) - HTML output now shows the expected vs actual generated images for each of the graphics tests applicable
- finished audio module - finished data module - finished filesystem module - finished font module (note: glphy test fails, but seems to be a 12.0 issue) - finished image module - finished maths module - finished sound module - finished thread module - finished video module - fixed pcall error not showing in results for invalid test code
- added graphics.Canvas, graphics.Font, graphics.Image, graphics.Quad, graphics.Shader and graphics.Text - removed rogue resource img - removed event wait test placeholder - updated todo list
- added obj tests for graphics.SpriteBatch, and graphics.Video - added obj tests for physics.World - added test for setStencilMode and removed deprecated stencil+setStencilTest - added graphics.drawInstanced - fixed deprecated physic shape calls (body now added as first param)
- added an automatic check for expected vs actual with the images already shown in the report - used a 1px tolerance for now for each of the pixels checked
- added physics.Contact, physics.Body, and physics.Shape obj tests - added graphics.Mesh and graphics.ParticleSystem obj tests - fixed some rgba tolerance needed on a couple methods - added test conclusion to zip artifact name for quick checking on PRs - ignored testsuite on compat mode windows builds
- spaced out object class tests and added more defined comment groups - changed reusing vars in case it causes unexpected asserts in future - fixed some shape:point test having wrong params - added assertTrue + assertFalse for basic checks - used assertRange more for some of the physics + audio tests
There was a problem hiding this comment.
I left a few comments but I don't think any of them need to be addressed before this version is merged. This was a huge amount of effort, thanks so much for taking it on! I like that it's already found several legit bugs in love 12 even while it was being fleshed out.
|
I might have more comments once I start adding to the tests myself, but for now I'll press the big green button and see what happens |
Initial commit of a basic test framework, see readme.md in /testing for more
Most modules are covered with basic unit tests, and there's an example test for a graphics draw (rectangle) and object (File) - for object tests doing more scenario based so we can check multiple things together, if style is fine will carry on working through the other graphics methods + object types (which is the bulk of the work tbh)