Staging and Tests against Dojos

[ConnorNelson]

No seriously.

This affects our development speed and quality of our platform significantly. As things currently stand, this might even be more important than ripping out our NFS bottleneck for multi-node (and if not, it is just behind that).

I should be able to know if changes to this repository will affect the solvability of dojos.

We need an interface for dojos to declare testcases. This will need a mechanism for those testcases to be private, because probably most of the tests will be "does my solution script work?". And then we need infrastructure that runs those testscases against production.

• Phase 1: Button that can be pressed to run CI
• Phase 2: Run all the tests every night / after every update
• Phase 3: Staging server to get a before-it-goes-live understanding

Picture this ultra fancy system:

• Push to master
• That immediately deploys to https://staging.pwn.college/
• Testcases run
• Drop some poor unsuspecting users randomly into that instance and see if anyone complains (the real testcase)
• Push to stable branch after 24h without issue
• Deploy stable branch to https://pwn.college/

Make it all automatic and delete @zardus's keys. Bumo.

Seriously, we should at least get a system that supports phase 1.