[WIP] ATC, ApplicationTestCase

[mesalu]

This pull request tracks completion progress for a new tool to aid in unit testing.
ATC aims to make creating unit tests that require a full running application quick and easy with minimal compromise.

This is the list of required features and standards that must be fulfilled before a formal Pull Request will be made:

• Allow full application unit testing after developing a single widget
• Allow multiple test case classes per test module
• Provide all unittest features: decorators, test methods, etc
• Do not muck up pytest standards (ex: 1 internal test is one test as seen by unittest / pytest)
• Allow tests to span several events
• Allow any widget to be tested, not just Frames
• Ensure useful traceback is printed when a test fails
• Ensure tests exit in test friendly way
• Add a watchdog to cover any accidental mistakes.

This is a general TODO list for changes that need to be completed as well

• Add proper documentation
• Ensure ATC plays nice with custom TestSuites
• Change launch approach away from timer
• Automatically wrap testwidget test_ methods with TestCritical
• Create decorator to minimize usage of test(Passed/Failed)
• Ensure ATC cooperates with certain widget types, such as Dialogs
• Re-implement some test cases that would benefit from atc
• Polish! (Well this was just a bad goal... Cleaning up is never done!)

Please feel free to add any new requirements as seen fit.

[mesalu]

@RobinD42 Any suggestions for addressing the current launch protocol? As mentioned in the second item in TODO. Details can be found in unittests/atc.py TestWidget.__init__ and TestWidget.OnCommence

[RobinD42]

This is a very cool idea and I like how it is shaping up. Definitely looking forward to it becoming more generally useful.

A couple quick suggestions after my first pass over the code and example:

• TestDone(bool) seems a little unintuitive to me. Maybe having 2 methods would be more readable, like TestPassed() and TestFailed() or something. Then if there is ever a 3rd state we won't have to refactor the bool into something else.

• I know it's on your list, but I just want to emphasize it. Docs, and docstrings! This is something completely new and we want people to be able to use it without needing to dig too deeply or just blindly coping something without understanding it.

• Since a ATC and TestWidget are stand-ins for unittest.TestCase then it might be better to follow its naming convention of leadingLowerMixedCase for methods and such, at least for those that are intended to be used by people writing tests.

[RobinD42]

Any suggestions for addressing the current launch protocol?

On Windows and OSX you can probably start things with just a wx.CallAfter to OnCommence in __init__ as everything should be ready to roll as soon as the MainLoop starts. On Linux you can wait until the TestWidget gets a EVT_WINDOW_CREATE event, and use the wx.CallAfter from there.

The difference is due to the X Window not being created as soon as the client-side GUI APIs are called. It needs to wait until the Xlib commands have been sent to the server and it responds back, which is when wx sends the EVT_WINDOW_CREATE event. At that point the widget is fully created and APIs that need to directly interact with it are able to do so.

[RobinD42]

When this PR is finished go ahead and remove the wip label and add the needs-review label. Thanks.

[mesalu]

I'm dropping TestSuite support as a goal for a couple reasons:
Firstly, its a bit outside the scope of ATC's use.
Secondly, its appearing to be more trouble than its worth. Running a custom TestSuite will require the test case to implement runTest, which essentially boils a multi-test testcase down to one test.

Right now I'm not sure how useful additional decorators will be. testCritical covers unexpected failures, but an endsTest decorator could be misleading and cause false failures. I personally believe it would be better to have ATC obviously misbehave (hang) rather than falsify tests. If some kind of demand arises for them later I'll take another look.