0.1.2 Testing Release Plan

[aaron97neu]

Github's milestone description does not allow for anything more than a paragraph without an formatting. A issue provides a better place to list and discuss the features planned for a milestone

Once the 0.1.2 milestone is reached, this issue will be closed and 0.1.2 will be considered released

For this release we will not release any new features, rather integrate automated testing to assist in future development.
This will include:

• Health checks for docker containers
• Docker-compose files that run a test suite for each container
• Virtualized testing of the ansible scripts?
• On board diagnostics via LED lights?

Others?

[aaron97neu]

During 11/2/2020 meeting we had an extended discussion on how we intend on doing testing. Our plan involves the following:

• Create a "data source simulator" for each data source that we would like to provide for containers within Avena. This is the red and grey boxes on the diagram below. An example for this would be a python container (red "GPSD" box) that reads a gpsd log (grey "GPSD log" box) that feeds gps data to DBUS or provides a GPSD like IP interface.
• Create a "verifier" for each service we write. An example would be a python container (green "GPS verifier" box) that reads gpsd logs (grey "GPSD log" box) and CAN logs (grey "CAN log" box) and compares it to the output that the GPS container (blue "GPS" box), in this case the postgres database (blue "postgres" box). It would expect to see all gpsd messages and all CAN NEMA GPS messages in the proper form in the postgres database and would output and pass/fail accordingly

All of these containers would be orchestrated with a set of docker-compose files. The tester (script or human) can then run any combination of tests or the entire test using docker-compose. For example, docker-compose start gps-verify would start the gps-verification container and all the containers that depend on gps-verify to verify the gps logger functionality (GPS logger, GPSD, CAN, and postgres in the example below). docker-compose up would boot the entire system and test everything.

@abalmos @facastiblancor Please chime in for things that I missed

[aaron97neu]

Significant bug fixes have been identified since work on testing has begun. 0.1.1 has been changed to these bug fixes and the testing features have been pushed to 0.1.2