Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add python tests model classes #19

Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
f50fa34
Created model classes for SDK python tests
rquidute Nov 6, 2023
6a77265
Added parser implementation to python test
rquidute Nov 8, 2023
d54d375
Changed init from sdk_tests
rquidute Nov 8, 2023
790c528
Merge branch 'main' into feature/74_phase3-add_python_tests_model_cla…
rquidute Nov 8, 2023
1985230
Minor changes
rquidute Nov 8, 2023
91fdbf9
Disable support to list python test cases
rquidute Nov 8, 2023
df0e129
Reverted minor changes
rquidute Nov 8, 2023
fe24421
Added Unit tests
rquidute Nov 8, 2023
e8c6ef7
Minor change
rquidute Nov 8, 2023
e6871e0
Merge with main
rquidute Nov 8, 2023
b682e0d
Fix merge conflict
rquidute Nov 8, 2023
1b8e180
Fix CI issues
rquidute Nov 8, 2023
6b1bc86
Fix mypy and flake violations
rquidute Nov 8, 2023
ba061c2
Fix mypy and flake violations
rquidute Nov 9, 2023
817f72a
Fix mypy and flake violations
rquidute Nov 9, 2023
beb46b3
Fixed unit tests
rquidute Nov 9, 2023
8da8227
Add iniial code for Python Test runner
rquidute Nov 10, 2023
879b7de
Fix code violation and add code to chip_tool for python runner
rquidute Nov 10, 2023
5872265
Fix code violation
rquidute Nov 10, 2023
8a89c18
Changes after code review
rquidute Nov 13, 2023
118cc89
Removed unsued code
rquidute Nov 13, 2023
b914e19
Few tweeks based on observation running end to end.
gladystonfranca Nov 13, 2023
939bf15
Merge branch 'main' into feature/74_phase3-add_python_tests_model_cla…
rquidute Nov 14, 2023
ba9c21e
Created top level classes for test models
rquidute Nov 14, 2023
8f5f77c
Fix code violation
rquidute Nov 14, 2023
e0dc0da
Code changes after review
rquidute Nov 16, 2023
bfa0725
Merge branch 'project-chip:main' into feature/74_phase3-add_python_te…
rquidute Nov 17, 2023
352f80b
Code changes after code review
rquidute Nov 22, 2023
7f34cbf
Putting back .version_information
rquidute Nov 22, 2023
1502f2e
Putting back .version_information
rquidute Nov 22, 2023
10ac46b
Fixed mypy violations
rquidute Nov 22, 2023
17ec11a
Removed unused imports
rquidute Nov 22, 2023
77cf574
adding rpc client code and mounting new volumes on sdk container
gladystonfranca Nov 27, 2023
63e39e2
Merge branch 'project-chip:main' into feature/74_phase3-add_python_te…
rquidute Nov 27, 2023
2135ffd
Fixing sdk rpc file path
gladystonfranca Nov 27, 2023
cee194f
Code changed after code review
rquidute Nov 29, 2023
01d9d57
Update app/tests/python_tests/test_python_test_suite.py
rquidute Nov 29, 2023
030da70
Code changes after review
rquidute Nov 29, 2023
de0c0c4
Remove whitespace from a blank line
rquidute Nov 29, 2023
e78a09d
Merge branch 'project-chip:main' into feature/74_phase3-add_python_te…
rquidute Nov 30, 2023
0a769fb
Add support for PICS in python tests
ccruzagralopes Nov 30, 2023
c7ad322
Update some unit tests
ccruzagralopes Nov 30, 2023
edcf2cd
Merge pull request #1 from ccruzagralopes/add_support_for_PICS_in_pyt…
rquidute Nov 30, 2023
5904c9d
Get python test parameters from project config
rquidute Dec 1, 2023
3198ecd
Fix Unit tests
rquidute Dec 1, 2023
f35d043
Updated unit tests
rquidute Dec 4, 2023
9bb424f
Removed unused test files
rquidute Dec 4, 2023
e5dfd63
Code updated due to code review
rquidute Dec 5, 2023
1e1ae77
Make PICS optional for python tests
rquidute Dec 6, 2023
890cf2e
Added SDKPythonTestResult class in order to handle the python test re…
rquidute Dec 6, 2023
8a116d1
Create SDKPythonTestResultBase and make subclass for each result
rquidute Dec 6, 2023
eceb15b
Make default attribute type set in SDKPythonTestResultBase subclasses
rquidute Dec 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion app/chip_tool/chip_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ class ChipToolUnknownTestType(Exception):
class ChipToolTestType(str, Enum):
CHIP_TOOL = "chip-tool"
CHIP_APP = "chip-app"
PYTHON_TEST = "python-test"
rquidute marked this conversation as resolved.
Show resolved Hide resolved


class ChipTool(metaclass=Singleton):
Expand Down Expand Up @@ -265,6 +266,8 @@ async def start_chip_server(
elif test_type == ChipToolTestType.CHIP_APP:
prefix = CHIP_APP_EXE
command = ["--interactive", "--port 9002"]
elif test_type == ChipToolTestType.PYTHON_TEST:
pass
ccruzagralopes marked this conversation as resolved.
Show resolved Hide resolved
else:
raise ChipToolUnknownTestType(f"Unsupported Test Type: {test_type}")

Expand Down Expand Up @@ -586,7 +589,7 @@ async def run_test(

if test_type == ChipToolTestType.CHIP_TOOL:
test_path = f"{YAML_TESTS_PATH}/{test_id}.yaml"
else:
elif test_type == ChipToolTestType.CHIP_APP:
rquidute marked this conversation as resolved.
Show resolved Hide resolved
test_path = f"{YAML_TESTS_PATH}/{test_id}_Simulated.yaml"

parser_config = TestParserConfig(pics_path, self.specifications, test_options)
Expand All @@ -598,6 +601,9 @@ async def run_test(
adapter = ChipToolAdapter.Adapter(parser_config.definitions)
elif test_type == ChipToolTestType.CHIP_APP:
adapter = ChipAppAdapter.Adapter(parser_config.definitions)
elif test_type == ChipToolTestType.PYTHON_TEST:
# TODO - run_test() for python_test to be implemented
pass
else:
raise ChipToolUnknownTestType(f"Unsupported Test Type: {test_type}")

Expand Down
64 changes: 64 additions & 0 deletions app/tests/python_tests/test_python_folder.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#
# Copyright (c) 2023 Project CHIP Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# flake8: noqa
# Ignore flake8 check for this file
from pathlib import Path
from unittest import mock

from test_collections.sdk_tests.support.python_testing.models.python_test_folder import (
PythonTestFolder,
)

test_python_path = Path("/test/python")


def test_python_folder_version() -> None:
version_file_content = "python_test_version"

# We mock open to read version_file_content and Path exists to ignore that we're
# testing with a fake path
with mock.patch(
"test_collections.sdk_tests.support.python_testing.models."
"python_test_folder.open",
new=mock.mock_open(read_data=version_file_content),
), mock.patch.object(target=Path, attribute="exists", return_value=True) as _:
python_test_folder = PythonTestFolder(test_python_path)

assert python_test_folder.version == version_file_content


def test_python_folder_version_missing() -> None:
expected_version = "Unknown"
with mock.patch.object(target=Path, attribute="exists", return_value=False) as _:
python_folder = PythonTestFolder(test_python_path)
assert python_folder.version == expected_version


def test_python_folder_filename_pattern() -> None:
"""Test PythonTestFolder will search for files with filename pattern."""
with mock.patch.object(target=Path, attribute="glob") as path_glob:
# Default file_name_patter: *
ccruzagralopes marked this conversation as resolved.
Show resolved Hide resolved
python_folder = PythonTestFolder(test_python_path)
_ = python_folder.python_file_paths()
path_glob.assert_called_once_with("*.py")

path_glob.reset_mock()
pattern = "TC_*"
python_test_folder = PythonTestFolder(
test_python_path, filename_pattern=pattern
)
_ = python_test_folder.python_file_paths()
path_glob.assert_called_once_with(f"{pattern}.py")
93 changes: 93 additions & 0 deletions app/tests/python_tests/test_python_parser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
#
# Copyright (c) 2023 Project CHIP Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# flake8: noqa
# Ignore flake8 check for this file
from pathlib import Path
from unittest import mock

from test_collections.sdk_tests.support.python_testing.models.python_test_parser import (
PythonParserException,
parse_python_test,
)

sample_invalid_python_file_content = """
class TC_Sample(MatterBaseTest):

def steps_TC_Sample(self) -> list[TestStep]:
steps = [
TestStep(1, "Commissioning, already done", is_commissioning=True),
TestStep(2, "Second step"),
TestStep(3, "Third step"),
]
return steps

@async_test_body
async def test_steps_TC_Sample(self):
print("Test execution")

"""

sample_python_file_content = """
class TC_Sample(MatterBaseTest):

def desc_TC_Sample(self) -> str:
return "Sample TC Description"

def steps_TC_Sample(self) -> list[TestStep]:
steps = [
TestStep(1, "Commissioning, already done", is_commissioning=True),
TestStep(2, "Second step"),
TestStep(3, "Third step"),
]
return steps

@async_test_body
async def test_steps_TC_Sample(self):
print("Test execution")

"""


def test_python_file_parser_throws_pythonparserexception() -> None:
file_path = Path("/test/file.py")

with mock.patch(
"test_collections.sdk_tests.support.python_testing.models.python_test_parser."
"open",
new=mock.mock_open(read_data=sample_invalid_python_file_content),
):
try:
parse_python_test(file_path)
except PythonParserException as e:
assert (
"Test Case file does not have methods desc_file or steps_file" == str(e)
)


def test_python_file_parser() -> None:
file_path = Path("/test/file.py")

# We mock builtin `open` method to read sample python file content,
# to avoid having to load a real file.
with mock.patch(
"test_collections.sdk_tests.support.python_testing.models.python_test_parser."
"open",
new=mock.mock_open(read_data=sample_python_file_content),
) as file_open:
test = parse_python_test(file_path)

file_open.assert_called_once_with(file_path, "r")
assert test.path == file_path
1 change: 1 addition & 0 deletions app/tests/python_tests/test_python_script/.version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
unit-test-python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#
# Copyright (c) 2023 Project CHIP Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Copyright (c) 2021 Project CHIP Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# type: ignore
# Ignore mypy type check for this file
# flake8: noqa
# Ignore flake8 check for this file
"""
This is just a sample test case that should come from SDK.
It should not compile or run.
"""


class TC_Sample(MatterBaseTest):
def desc_TC_Sample(self) -> str:
return "Sample TC Description"

def steps_TC_Sample(self) -> list[TestStep]:
steps = [
TestStep(1, "Commissioning, already done", is_commissioning=True),
TestStep(2, "Second step"),
TestStep(3, "Third step"),
]
return steps

@async_test_body
async def test_steps_TC_Sample(self):
print("Test execution")
Loading
Loading