Skip to content

Latest commit

 

History

History
91 lines (69 loc) · 5.34 KB

testcase_tool_driver.md

File metadata and controls

91 lines (69 loc) · 5.34 KB

Writing drivers for testcase tools

The following steps must be followed to implement a new driver. The final driver may be pushed to the muteria's github repository, to be used by others.

  1. The driver's code must be located in the folder muteria/drivers/testgeneration/tools_by_languages/<Language>/<Driver-Folder>.

  2. An __init__.py file must be created in the driver folder (<Driver-Folder>) and assign the driver's class name to either StaticCriteriaTool or DynamicTestcaseTool as following: StaticCriteriaTool = <Driver-Class> or DynamicTestcaseTool = <Driver-Class>. If the testcase tool uses existing tests to generate new tests, then assign the class name to DynamicTestcaseTool and None to StaticTestcaseTool, and vice versa.See for example the case of KLEE's driver in muteria/drivers/testgeneration/tools_by_languages/c/klee/__init__.py

  3. The driver's class must extend the class BaseTestcaseTool, located in muteria/drivers/testgeneration/base_testcasetool.py, and implement all the abstract methods (see for example KLEE's driver in muteria/drivers/testgeneration/tools_by_languages/c/klee/klee.py). The abstract methods that must be implemented are listed bellow.

    @classmethod
    def installed(cls, custom_binary_dir=None)

This method checks whether the tool is installed. It returns a boolean representing whether the tool is installed (executable is accessible on the path) or not. True means that the tool is installed, else False is returned.

Paramter Type Description
custom_binary_dir string custom directory to look for the relevant executable files for the corresponding tool. None means that the file are on the PATH. 
    def get_testcase_info_object(self)

This method returns the info object for all tests managed/generated by the tool. The info object for testcases is an object of the class TestcasesInfoObject, defined in the file muteria/drivers/testgeneration/testcases_info.py

    def _prepare_executable (self, exe_path_map, env_vars, \
                                                        collect_output=False):

This method is called before every test execution. Put any setup here.

Paramter Type Description
exe_path_map dict map each repository file to the corresponding instrumented file intended location.
env_vars dict env vars that are set for the execution.
collect_output bool map each repository file to the corresponding instrumented file intended location. 
    def restore_default_executable (self, exe_path_map, env_vars, \
                                                        collect_output=False):

This method is called after every test execution. Put any teardown here.

Paramter Type Description
exe_path_map dict map each repository file to the corresponding instrumented file intended location.
env_vars dict env vars that are set for the execution.
collect_output bool map each repository file to the corresponding instrumented file intended location. 
    def _execute_a_test (self, testcase, exe_path_map, env_vars, \
                    callback_object=None, timeout=None, collect_output=None)

This method is called after every test execution. Put any teardown here.

Paramter Type Description
testcase string name of test case to execute.
exe_path_map dict map each repository file to the corresponding instrumented file intended location.
env_vars dict env vars that are set for the execution.
callback_object BaseCallbackObject instace of implementation of Callback object (defined in muteria/repositoryandcode/callback_object.py). This is particularly useful when the test execution is performed in the repository dir and called through run_dev_test of the Repository manager (see for instance muteria/drivers/testgeneration/custom_dev_testcase/custom_dev_testcase.py).
timeout float timeout for the test execution.
collect_output bool map each repository file to the corresponding instrumented file intended location. 
    def _do_generate_tests (self, exe_path_map, code_builds_factory, \
                                        meta_criteria_tool_obj=None, \
                                        max_time=None)

This method generates the test cases by calling the underlying tool. For reference, see muteria/drivers/testgeneration/tools_by_languages/c/klee/klee.py)

Paramter Type Description
exe_path_map dict map each repository file to the corresponding instrumented file intended location.
code_builds_factory CodeBuildsFactory Class that defines methods to handle repository tasks, such as building code, ... (see file muteria/repositoryandcode/code_builds_factory.py).
meta_criteria_tool_obj MetaCriteriaTool MetaCriteriaTool object that managing the criterion tool with intrumented code on which the test generation should be conducted (example mutants to target). The MetaCriteriaTool is defined in muteria/drivers/criteria/meta_testcriteriatool.py
max_time float timeout for the test generation.