-
-
Notifications
You must be signed in to change notification settings - Fork 44
Pull Request Validation
Currently the Vega Strike project sticks to the Pristine Master branching strategy (see #48-comment for more details). Due to that, we want to make sure that everything that enters the master passes proper validation first.
- Local validation (as describe below).
- Pull Request approval at least by one core developer.
- CI pass.
As part of validation it is important to demonstrate that the issue is fixed or the feature exercised. This can take shape in several forms:
- specific testing where a clear, present, demonstrable test can be provided
- unit test showing the exercise of the explicit code
- play test (see below) that demonstrates the feature/issue
- build test when things directly impact how the build functions or some part of the build is generated
In many cases these can be very specific to a given function or ability, and currently there are no unit tests available. As unit tests become more prevalent then other forms of testing will be less necessary.
At this time, however, we primarily depend on the Play Test method. Deviations from this are acceptable but must be outlined in the PR so others can reproduce what was done so that it can be verified independently of the PR submitter. If you are unable to do this test yourself, but feel that a Play Test won't be able to demonstrate what the PR is about then please outline the kind of test necessary for someone to validate it.
Play Test are a form of integration test to show that the code is working properly. There can be many forms of play testing. For instance, if one is working on Jump Points, then the play tests should involve lots of usage of jump points. The below outlines our standardized play test that should over most use cases. If the change requires something that deviates from this then (1) please do this basic test, and (2) please document additional steps that are required to test the change.
NOTE: If there are common patterns in the additional steps then the standardized play test will be updated to reflect those additional requirements.
The author of the changes should run the game and do the actions described below. Starting a new game, and completing these steps in this order should take around 10-15 minutes.
- From the Main Menu, view the Introduction and the Credits, and verify the Vega Strike version number(s) displayed on each.
- From the Main Menu, start a new Campaign.
- Basic flight.
- Docking to a planet.
- On planet actions:
- Save the game.
- Read news.
- Buy/Sell some goods.
- Buy/Sell a ship upgrade.
- Accept a mission from the bulletin board.
- Talk to someone in the bar.
- Primary and secondary weapon usage.
- Fight (you can fight with Oswald).
- SPEC flight.
- A jump to a different system.
- Docking to a non-planet location (mining base, etc).
- Buy a ship. (If necessary, edit a save file to give yourself enough credits to do so.)
- Save the game again.
- Upgrade your new ship.
- Save once more.
- Purposely die, and then make sure you can respawn and continue playing without the game crashing or anything.
- Quit Vega Strike. Make sure it doesn't segfault on exit.
- Launch Vega Strike again. This time, from the Main Menu, choose Load, and load a previously saved game. Continue where you left off.
While completing those steps, the tester should look for oddities, especially in the areas that he changed. In general the following aspects should be looked at:
- Graphics (textures, light, geometry, etc)
- Audio (noise, volume, synchronisation, etc)
- Physics (momentum, gravity, collision, etc)
For any questions on the matter, feel free to ask questions on Gitter.