Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Actor revalidation #289

Merged
merged 84 commits into from
Apr 15, 2024
Merged
Show file tree
Hide file tree
Changes from 58 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
7d104f2
validation changed
pseusys Nov 22, 2023
d1e2c1f
some errors fixed
pseusys Nov 22, 2023
6664120
tests fixed
pseusys Nov 24, 2023
748befc
Merge branch 'dev' into feat/actor_revalidation
pseusys Nov 24, 2023
4c2792b
lint applied
pseusys Nov 27, 2023
4d15bd6
Merge branch 'feat/actor_revalidation' of https://github.com/deeppavl…
pseusys Nov 27, 2023
2f59d24
blank line removed
pseusys Nov 27, 2023
ed21272
format applied
pseusys Nov 27, 2023
bcaa5cc
function docs provided
pseusys Nov 27, 2023
e214350
lint applied
pseusys Nov 27, 2023
5ddc70c
reviews fixed
pseusys Dec 1, 2023
e2b04f7
actor updated
pseusys Feb 6, 2024
fc588bb
Merge branch 'dev' into feat/actor_revalidation
pseusys Feb 6, 2024
5f9ff22
aftermerge
pseusys Feb 6, 2024
809fc45
inspect moved out of scope
pseusys Feb 6, 2024
cab2d88
user functions types mocked
pseusys Feb 6, 2024
42d9112
any import added
pseusys Feb 6, 2024
e7eaacb
most of the errors fixed
pseusys Feb 7, 2024
79ae0b1
last failing test fixed
pseusys Feb 7, 2024
913887e
lint applied
pseusys Feb 7, 2024
cdf5348
Merge branch 'dev' into feat/actor_revalidation
pseusys Mar 18, 2024
9c848dc
repeat import added
pseusys Mar 18, 2024
d19ea74
in-script validation first attempt
pseusys Mar 26, 2024
0f0edc8
Merge branch 'feat/actor_revalidation' of https://github.com/deeppavl…
pseusys Mar 26, 2024
6a67f34
Merge branch 'dev' into feat/actor_revalidation
pseusys Mar 27, 2024
48cbc9e
validate_script reference removed
pseusys Mar 27, 2024
4d81d31
runtime imports added instead of typechecking imports
pseusys Mar 27, 2024
c3d5d25
lint fixed
pseusys Mar 27, 2024
aab8020
script creation test changed
pseusys Mar 27, 2024
247bac3
global import removed, test function fixed
pseusys Mar 27, 2024
fe55f02
make `normalize_label` raise on TypeError
RLKRo Mar 28, 2024
bd64bc8
remove irrelevant docs
RLKRo Mar 28, 2024
1c761d0
processing returns None
pseusys Mar 28, 2024
303b0f3
names replaced with enum, type_checking removed
pseusys Mar 28, 2024
9244921
strenum removed
pseusys Mar 28, 2024
0278ffc
local flow label fixed
pseusys Mar 28, 2024
db37dd6
type comparison fixed in case of strings
pseusys Mar 28, 2024
5c129b3
type description added
pseusys Mar 28, 2024
e4591fa
condition fixed
pseusys Mar 28, 2024
00b9a67
none matching improved
pseusys Mar 29, 2024
2632c7a
label normalization returned
pseusys Mar 29, 2024
a343b1a
label functions signature changed
pseusys Mar 29, 2024
040b84e
separate errors with new lines
RLKRo Mar 29, 2024
2b10cfd
return label representation instead of their str
RLKRo Mar 29, 2024
238359e
lable typing reverted
pseusys Mar 29, 2024
be648c1
Merge branch 'feat/actor_revalidation' of https://github.com/deeppavl…
pseusys Mar 29, 2024
4897859
types replaced with strings
pseusys Mar 29, 2024
5be4b86
function names changed + lint fixed
pseusys Mar 29, 2024
2fe6dc4
docs validated, validation tests added
pseusys Apr 2, 2024
d68fa74
testing function fix
pseusys Apr 2, 2024
b231433
execution mode changed to "before"
pseusys Apr 2, 2024
bffd43f
error expecting fixed
pseusys Apr 2, 2024
78df45a
testing errors fixed, excessive validation removed
pseusys Apr 2, 2024
af4bd33
lint applied
pseusys Apr 2, 2024
ba47769
few tutorials fixed
pseusys Apr 2, 2024
3ad859e
type annotations correct hanfling added
pseusys Apr 2, 2024
c5e3560
test functions -> test classes
pseusys Apr 3, 2024
271469f
lint applied (again)
pseusys Apr 3, 2024
8ccbfbb
remove unnecessary comments
RLKRo Apr 4, 2024
fc74d76
group user function for validation tests
RLKRo Apr 4, 2024
c75d0ba
assert number of found errors
RLKRo Apr 4, 2024
c3ce08a
logging function removed
pseusys Apr 8, 2024
067283a
multiple label type options added
pseusys Apr 8, 2024
dce18ac
fix error message addition
RLKRo Apr 9, 2024
4c82749
fix return type validation
RLKRo Apr 9, 2024
5206ce0
update tests
RLKRo Apr 9, 2024
f928ac4
improve error messages
RLKRo Apr 9, 2024
e01f459
lint
RLKRo Apr 9, 2024
aa7cf4f
add more possible return type annotations for labels
RLKRo Apr 9, 2024
5d1736c
reformat
RLKRo Apr 9, 2024
5cb48fe
signature name lowered
pseusys Apr 9, 2024
d9f6b25
Merge branch 'feat/actor_revalidation' of https://github.com/deeppavl…
pseusys Apr 11, 2024
d0718a2
types fix
pseusys Apr 11, 2024
90d071d
types renamed
pseusys Apr 13, 2024
c59d248
type names fixed
pseusys Apr 15, 2024
f353fed
node label type changed
pseusys Apr 15, 2024
66372c4
constlabel reverted
pseusys Apr 15, 2024
2e19912
test regex fixed
pseusys Apr 15, 2024
597a82e
regex attuned pnce again
pseusys Apr 15, 2024
3fa6d14
last regex error fixed
pseusys Apr 15, 2024
1e81d12
fprmatting applied
pseusys Apr 15, 2024
0522752
doc improvements
RLKRo Apr 15, 2024
50b354a
add some todos
RLKRo Apr 15, 2024
20717a6
ignore unused import
RLKRo Apr 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dff/messengers/telegram/messenger.py
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ def telegram_condition(
**kwargs,
)

def condition(ctx: Context, _: Pipeline, *__, **___): # pragma: no cover
def condition(ctx: Context, _: Pipeline) -> bool: # pragma: no cover
last_request = ctx.last_request
if last_request is None:
return False
Expand Down
50 changes: 2 additions & 48 deletions dff/pipeline/pipeline/actor.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,22 +45,6 @@
from dff.pipeline.pipeline.pipeline import Pipeline


def error_handler(error_msgs: list, msg: str, exception: Optional[Exception] = None, logging_flag: bool = True):
"""
This function handles errors during :py:class:`~dff.script.Script` validation.

:param error_msgs: List that contains error messages. :py:func:`~dff.script.error_handler`
adds every next error message to that list.
:param msg: Error message which is to be added into `error_msgs`.
:param exception: Invoked exception. If it has been set, it is used to obtain logging traceback.
Defaults to `None`.
:param logging_flag: The flag which defines whether logging is necessary. Defaults to `True`.
"""
error_msgs.append(msg)
if logging_flag:
logger.error(msg, exc_info=exception)


class Actor:
"""
The class which is used to process :py:class:`~dff.script.Context`
Expand Down Expand Up @@ -92,11 +76,11 @@ def __init__(
condition_handler: Optional[Callable] = None,
handlers: Optional[Dict[ActorStage, List[Callable]]] = None,
):
# script validation
# script evaluation
self.script = script if isinstance(script, Script) else Script(script=script)
self.label_priority = label_priority

# node labels validation
# node labels evaluation
self.start_label = normalize_label(start_label)
if self.script.get(self.start_label[0], {}).get(self.start_label[1]) is None:
raise ValueError(f"Unknown start_label={self.start_label}")
Expand Down Expand Up @@ -389,36 +373,6 @@ def _choose_label(
chosen_label = self.fallback_label
return chosen_label

def validate_script(self, pipeline: Pipeline, verbose: bool = True):
# TODO: script has to not contain priority == -inf, because it uses for miss values
flow_labels = []
node_labels = []
labels = []
conditions = []
for flow_name, flow in self.script.items():
for node_name, node in flow.items():
flow_labels += [flow_name] * len(node.transitions)
node_labels += [node_name] * len(node.transitions)
labels += list(node.transitions.keys())
conditions += list(node.transitions.values())

error_msgs = []
for flow_label, node_label, label, condition in zip(flow_labels, node_labels, labels, conditions):
if not callable(label):
label = normalize_label(label, flow_label)

# validate labeling
try:
node = self.script[label[0]][label[1]]
except Exception as exc:
msg = (
f"Could not find node with label={label}, "
f"error was found in (flow_label, node_label)={(flow_label, node_label)}"
)
error_handler(error_msgs, msg, exc, verbose)
break
return error_msgs


async def default_condition_handler(
condition: Callable, ctx: Context, pipeline: Pipeline
Expand Down
18 changes: 0 additions & 18 deletions dff/pipeline/pipeline/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,6 @@ class Pipeline:
:param fallback_label: Actor fallback label.
:param label_priority: Default priority value for all actor :py:const:`labels <dff.script.NodeLabel3Type>`
where there is no priority. Defaults to `1.0`.
:param validation_stage: This flag sets whether the validation stage is executed after actor creation.
It is executed by default. Defaults to `None`.
:param condition_handler: Handler that processes a call of actor condition functions. Defaults to `None`.
:param verbose: If it is `True`, logging is used in actor. Defaults to `True`.
:param handlers: This variable is responsible for the usage of external handlers on
Expand Down Expand Up @@ -88,7 +86,6 @@ def __init__(
start_label: NodeLabel2Type,
fallback_label: Optional[NodeLabel2Type] = None,
label_priority: float = 1.0,
validation_stage: Optional[bool] = None,
condition_handler: Optional[Callable] = None,
verbose: bool = True,
handlers: Optional[Dict[ActorStage, List[Callable]]] = None,
Expand Down Expand Up @@ -121,7 +118,6 @@ def __init__(
start_label,
fallback_label,
label_priority,
validation_stage,
condition_handler,
verbose,
handlers,
Expand Down Expand Up @@ -211,7 +207,6 @@ def from_script(
start_label: NodeLabel2Type,
fallback_label: Optional[NodeLabel2Type] = None,
label_priority: float = 1.0,
validation_stage: Optional[bool] = None,
condition_handler: Optional[Callable] = None,
verbose: bool = True,
parallelize_processing: bool = False,
Expand All @@ -234,8 +229,6 @@ def from_script(
:param fallback_label: Actor fallback label.
:param label_priority: Default priority value for all actor :py:const:`labels <dff.script.NodeLabel3Type>`
where there is no priority. Defaults to `1.0`.
:param validation_stage: This flag sets whether the validation stage is executed after actor creation.
It is executed by default. Defaults to `None`.
:param condition_handler: Handler that processes a call of actor condition functions. Defaults to `None`.
:param verbose: If it is `True`, logging is used in actor. Defaults to `True`.
:param parallelize_processing: This flag determines whether or not the functions
Expand Down Expand Up @@ -265,7 +258,6 @@ def from_script(
start_label=start_label,
fallback_label=fallback_label,
label_priority=label_priority,
validation_stage=validation_stage,
condition_handler=condition_handler,
verbose=verbose,
parallelize_processing=parallelize_processing,
Expand All @@ -281,7 +273,6 @@ def set_actor(
start_label: NodeLabel2Type,
fallback_label: Optional[NodeLabel2Type] = None,
label_priority: float = 1.0,
validation_stage: Optional[bool] = None,
condition_handler: Optional[Callable] = None,
verbose: bool = True,
handlers: Optional[Dict[ActorStage, List[Callable]]] = None,
Expand All @@ -298,8 +289,6 @@ def set_actor(
or there was an error while executing the scenario.
:param label_priority: Default priority value for all actor :py:const:`labels <dff.script.NodeLabel3Type>`
where there is no priority. Defaults to `1.0`.
:param validation_stage: This flag sets whether the validation stage is executed in actor.
It is executed by default. Defaults to `None`.
:param condition_handler: Handler that processes a call of actor condition functions. Defaults to `None`.
:param verbose: If it is `True`, logging is used in actor. Defaults to `True`.
:param handlers: This variable is responsible for the usage of external handlers on
Expand All @@ -308,14 +297,7 @@ def set_actor(
- key :py:class:`~dff.script.ActorStage` - Stage in which the handler is called.
- value List[Callable] - The list of called handlers for each stage. Defaults to an empty `dict`.
"""
old_actor = self.actor
self.actor = Actor(script, start_label, fallback_label, label_priority, condition_handler, handlers)
errors = self.actor.validate_script(self, verbose) if validation_stage is not False else []
if errors:
self.actor = old_actor
raise ValueError(
f"Found {len(errors)} errors: " + " ".join([f"{i}) {er}" for i, er in enumerate(errors, 1)])
)

@classmethod
def from_dict(cls, dictionary: PipelineBuilder) -> "Pipeline":
Expand Down
1 change: 0 additions & 1 deletion dff/pipeline/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,6 @@ class ExtraHandlerRuntimeInfo(BaseModel):
"start_label": NodeLabel2Type,
"fallback_label": NotRequired[Optional[NodeLabel2Type]],
"label_priority": NotRequired[float],
"validation_stage": NotRequired[Optional[bool]],
"condition_handler": NotRequired[Optional[Callable]],
"verbose": NotRequired[bool],
"handlers": NotRequired[Optional[Dict[ActorStage, List[Callable]]]],
Expand Down
10 changes: 1 addition & 9 deletions dff/script/core/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,14 +88,6 @@ class Context(BaseModel):
- key - Arbitrary data name.
- value - Arbitrary data.
"""
validation: bool = False
"""
`validation` is a flag that signals that :py:class:`~dff.pipeline.pipeline.pipeline.Pipeline`,
while being initialized, checks the :py:class:`~dff.script.core.script.Script`.
The functions that can give not valid data
while being validated must use this flag to take the validation mode into account.
Otherwise the validation will not be passed.
"""
framework_states: Dict[ModuleName, Dict[str, Any]] = {}
"""
`framework_states` is used for addons states or for
Expand Down Expand Up @@ -142,7 +134,7 @@ def cast(cls, ctx: Optional[Union[Context, dict, str]] = None, *args, **kwargs)
ctx = Context.model_validate(ctx)
elif isinstance(ctx, str):
ctx = Context.model_validate_json(ctx)
elif not issubclass(type(ctx), Context):
elif not isinstance(ctx, Context):
raise ValueError(
f"Context expected to be an instance of the Context class "
f"or an instance of the dict/str(json) type. Got: {type(ctx)}"
Expand Down
7 changes: 5 additions & 2 deletions dff/script/core/normalization.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,7 @@ def normalize_label(
:param label: If label is Callable the function is wrapped into try/except
and normalization is used on the result of the function call with the name label.
:param default_flow_label: flow_label is used if label does not contain flow_label.
:return: Result of the label normalization,
if Callable is returned, the normalized result is returned.
:return: Result of the label normalization
"""
if callable(label):

Expand Down Expand Up @@ -62,6 +61,10 @@ def get_label_handler(ctx: Context, pipeline: Pipeline) -> NodeLabel3Type:
elif isinstance(label, tuple) and len(label) == 3:
flow_label = label[0] or default_flow_label
return (flow_label, label[1], label[2])
else:
raise TypeError(
f"Label '{label!r}' is of incorrect type. It has to follow the `NodeLabelType`:\n" f"{NodeLabelType!r}"
)


def normalize_condition(condition: ConditionType) -> Callable[[Context, Pipeline], bool]:
Expand Down
Loading
Loading