[WFCORE-6728] Ability to reload tests to a different stability level

server/WFCORE-6728-reload-to-stability-level.adoc

@@ -0,0 +1,155 @@
+---
+categories:
+  - core
+# Add any category for this proposal
+# if missing, add it to _data/widfly-categories and use its id
+---
+= Ability to reload a server to a different stability level
+:author:            Kabir Khan
+:email:             [email protected]
+:toc:               left
+:icons:             font
+:idprefix:
+:idseparator:       -
+
+== Overview
+
+WildFly's (and WildFly Core's) testsuite runs at a particular stability level by default. At the time of writing this
+is set to the 'community' level.
+
+Since features may be introduced to WildFly at a lower stability level than this (e.g. 'experimental' and 'preview'), those features will not be available at the stability level the tests are currently run. Thus, we need a way to run tests at a chosen stability level.
+
+This proposal attempts to solve that by providing a set of ServerSetupTasks that will reload the server to the desired stability level before running the test. To support this a new management operation based on the existing `reload` operation will be introduced.
+
+== Issue Metadata
+
+=== Issue
+
+* https://issues.redhat.com/browse/WFCORE[WFCORE-6728]
+
+=== Related Issues
+
+//* https://issues.redhat.com/browse/WFLY[WFLY-XXXX]
+
+=== Stability Level
+// Choose the planned stability level for the proposed functionality
+* [ ] Experimental
+
+* [ ] Preview
+
+* [x] Community
+
+* [ ] default
+
+=== Dev Contacts
+
+* mailto:{email}[{author}]
+
+=== QE Contacts
+
+=== Testing By
+// Put an x in the relevant field to indicate if testing will be done by Engineering or QE. 
+// Discuss with QE during the Kickoff state to decide this
+* [x] Engineering
+
+* [ ] QE
+
+=== Affected Projects or Components
+
+WildFly Core. The main changes will be in the server, testsuite/shared and testsuite/testsuite-runner modules.
+
+=== Other Interested Projects
+
+=== Relevant Installation Types
+// Remove the x next to the relevant field if the feature in question is not relevant
+// to that kind of WildFly installation
+* [x] Traditional standalone server (unzipped or provisioned by Galleon)
+
+* [] Managed domain
+
+* [x] OpenShift s2i
+
+* [x] Bootable jar
+
+== Requirements
+
+=== Hard Requirements
+
+* The WildFly Core test runner is enhanced to handle JUnit `Assume.assumeXXX()` calls from ServerSetupTasks the same way as in normal test code. Prior to this RFE the resulting `AssumptionViolatedException` would result in a stacktrace and test error, rather than the test being marked as skipped.
+* There will be a new operation based on the current reload operation introduced for standalone servers.
+- The operation will be registered at stability level 'community' at present
+- The operation is based on the reload operation, and adds a parameter called `stability`. This parameter takes the stability level we want to reload the server to
+- The operation also supports all the parameters supported by the standard reload operation
+- The operation has its own RBAC setting to narrow down who can call it
+* A set of ServerSetupTasks for the WildFly Core test runner will be introduced. These use the above operation to reload the server to the desired stability level before running the tests, and to reload the server back to the original stability level after the tests have completed.
+** The ServerSetupTasks perform the following checks, and skip the test using JUnit `Assume.assumeXXX()` if the condition is not met:
+*** The desired stability level is one of the ones supported by the server
+*** The enhanced reload operation exists at our current stability level
+*** We don't reload the server to a higher stability level than the one at which the enhanced reload operation is registered. This is because we need to reload back to the original stability level after the test has completed, and need the operation to be available. So with the operation currently registered at 'community' level, we cannot reload to default.
+
+=== Nice-to-Have Requirements
+// Requirements in this section do not have to be met to merge the proposed functionality.
+// Note: Nice-to-have requirements that don't end up being implemented as part of
+// the work covered by this proposal should be moved to the 'Future Work' section.
+
+
+=== Non-Requirements
+// Use this section to explicitly discuss things that readers might think are required
+// but which are not required.
+
+=== Future Work
+// Use this section to discuss requirements that are not addressed by this proposal
+// but which may be addressed in later proposals.
+This is currently only implemented for standalone servers, so there is no domain mode support. In the future it could be nice to be able to reload the host controllers and possibly the managed servers to different stability levels.
+
+Also, as this is currently mainly for the testsuite to be able to reload, there is no special support in the CLI's reload command for the enhanced reload operation. If this is seen usable for wider use, adding handling to the CLI command to expose the stability parameter could be interesting. The CLI command would need to take into account both the stability level of the server and the user's RBAC role in order to determine whether the enhanced reload operation is visible to the user. If the operation is visible, and the user wants to use the stability paramter, the enhanced reload operation will be called instead of the current reload operation.
+
+== Backwards Compatibility
+
+// Does this enhancement affect backwards compatibility with previously released
+// versions of WildFly?
+// Can the identified incompatibility be avoided?
+No, we are adding a new operation, which is mainly for testing as described above. There are no configuration changes.
+
+=== Default Configuration
+
+No impact
+
+=== Importing Existing Configuration
+
+No impact
+
+=== Deployments
+
+No impact
+
+=== Interoperability
+
+No impact
+
+== Security Considerations
+
+////
+Identification if any security implications that may need to be considered with this feature
+or a confirmation that there are no security implications to consider.
+////
+RBAC constraints will be configured on the enhanced reload operation to narrow down who can call it.
+
+== Test Plan
+
+Tests will be added to the testsuite/standalone module to ensure that the ServerSetupTasks mentioned in the requirements section reload properly to the desired stability levels.
+
+== Community Documentation
+
+The only user facing functionality here is the enhanced reload operation. The current intent is to keep this a bit hidden, so no community documentation will be added for this.
+
+== Release Note Content
+////
+Draft verbiage for up to a few sentences on the feature for inclusion in the
+Release Note blog article for the release that first includes this feature. 
+Example article: http://wildfly.org/news/2018/08/30/WildFly14-Final-Released/.
+This content will be edited, so there is no need to make it perfect or discuss
+what release it appears in.  "See Overview" is acceptable if the overview is
+suitable. For simple features best covered as an item in a bullet-point list 
+of features containing a few words on each, use "Bullet point: <The few words>" 
+////