Proposal: add tooling for memory load tests

[fabiobrz]

This is just a proposal for the subject, built on code I've developed taking inspiration from internal QE tooling used in acceptance jobs (using MBeans).
I am not sure if it can be actually useful for all of us - although it would be cool - so feel free to let me know your thoughts.

The idea is to have some key components that stress test classes can use in an easy way and so that the test developer can focus on providing the code needed to create race conditions and evaluate results.

The tooling-performance module has been created to provide the following components:

• Tester: executes test sessions and perform measurements, collecting results for the ongoing test session
• Gauge: the tool that a Tester uses to perform measurements
• Protocol: a component that is responsible for executing the proper actions needed during a test session and can communicate with the Tester that runs it
• Evaluator: provides results evaluation business logic, returning an outcome

Usage is demonstrated by the class MultipleSubsequentDeploymentMemoryStressTest which has been added to the microprofile-open-api module.
It defines one memory stress tests that runs a multiple redeploy protocol, aiming at detecting memory leaks.
The variable that is probed is Heap Used space and the evaluation is based on increase over allowed tolerance of final value compared to the initial one.

The test case has been able to detect the leak reported at https://issues.redhat.com/browse/WFLY-10991

As said, this is just a proposal which can be considered with lower priority than the PRs that are actually providing tests for TP coverage, just let me know what you think.

P.S.: I had to add manual mode to arquillian.xml to control container start and stop. I think this is not wanted so any idea is welcome (another container definition?).

Please make sure your PR meets the following requirements:

• Pull Request contains a description of the changes
• Pull Request does not include fixes for multiple issues/topics
• Code is formatted, imports ordered, code compiles and tests are passing
• N/A Link to the passing job is provided: no job can execute the test
• Code is self-descriptive and/or documented
• Description of the tests scenarios is included (see Update PR template to include TPG stuff #46)

[marekkopecky]

@mnovak1 FYI see this perf/memory tests, feel free to share this with rjanik. I can't notice him directly, because (probably) he is not a member of this project.

[rsvoboda]

Is this relevant @marekkopecky @fabiobrz ?

[fabiobrz]

Is this relevant @marekkopecky @fabiobrz ?

Hi @rsvoboda ! The initial idea was to have some internal (with respect to the TS) tooling to be used to improve CPU load tests in https://github.com/jboss-eap-qe/eap-microprofile-test-suite/tree/master/tooling-cpu-load.
I got inspiration from some bash acceptance tests, and prepared this to target the memory leak use case, specifically.
Then, I had a quick chat with @mnovak1 back in the days, who suggested for us to use one of the existing frameworks for the goal, so I am not sure. I'd like to have more feedback. WDYT?