Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flyte deck] Streaming Decks #2779

Draft
wants to merge 45 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
01182b4
[Flyte Decl] Streaming Decks
Future-Outlier Oct 1, 2024
9616fc3
print
Future-Outlier Oct 1, 2024
98ae7c7
sleep more
Future-Outlier Oct 1, 2024
4e92bb0
add dummy deck
Future-Outlier Oct 2, 2024
4df18b5
nit
Future-Outlier Oct 2, 2024
ebb4d4e
dummy deck
Future-Outlier Oct 2, 2024
99522d9
update
Future-Outlier Oct 2, 2024
c19d67d
nit
Future-Outlier Oct 2, 2024
67cd829
test
Future-Outlier Oct 2, 2024
06da3df
return html
Future-Outlier Oct 2, 2024
6d99d69
Change Deck
Future-Outlier Oct 2, 2024
b805cd7
fix
Future-Outlier Oct 2, 2024
4c97758
fix recursion error
Future-Outlier Oct 2, 2024
7b3574a
remove redundant code
Future-Outlier Oct 2, 2024
9b60564
add dummy deck to deck init
Future-Outlier Oct 2, 2024
18c994f
Better Dummy Deck Logic
Future-Outlier Oct 2, 2024
39f39d1
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Oct 7, 2024
aabcbbb
Deck Publish
Future-Outlier Oct 7, 2024
9ca43f3
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Nov 25, 2024
ed56352
litn
Future-Outlier Nov 27, 2024
b559fc9
remove dummy deck
Future-Outlier Nov 27, 2024
fc5578f
nit
Future-Outlier Nov 27, 2024
7139468
use auto refresh tab, 5 seconds as interval
Future-Outlier Dec 2, 2024
e0aee9e
revert
Future-Outlier Dec 2, 2024
3727588
test setDynamicTabs
Future-Outlier Dec 2, 2024
ce3ee15
change interval time
Future-Outlier Dec 2, 2024
d066231
test
Future-Outlier Dec 2, 2024
f9387ce
revert
Future-Outlier Dec 2, 2024
f14c3fa
test
Future-Outlier Dec 2, 2024
c33a909
nit
Future-Outlier Dec 2, 2024
8666c60
try dynamic containers
Future-Outlier Dec 2, 2024
93580d6
try dynamic containers v2
Future-Outlier Dec 2, 2024
bcaaabd
try dynamic containers v3
Future-Outlier Dec 2, 2024
a321700
debug
Future-Outlier Dec 2, 2024
6464fae
update
Future-Outlier Dec 2, 2024
884943c
nit
Future-Outlier Dec 3, 2024
d4b5b96
Refresh Botton
Future-Outlier Dec 3, 2024
406227c
fix
Future-Outlier Dec 3, 2024
1e77f54
lint
Future-Outlier Dec 3, 2024
d70a2d5
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Dec 3, 2024
7fc6393
test new refresh
Future-Outlier Dec 3, 2024
6980140
lint
Future-Outlier Dec 3, 2024
c87a342
Revert back html code, collaborating with Lyon
Future-Outlier Dec 5, 2024
f609760
lint
Future-Outlier Dec 5, 2024
473ae11
nit
Future-Outlier Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion flytekit/bin/entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ def _dispatch_execute(
logger.info(f"Engine folder written successfully to the output prefix {output_prefix}")

if task_def is not None and not getattr(task_def, "disable_deck", True):
_output_deck(task_def.name.split(".")[-1], ctx.user_space_params)
_output_deck(task_name=task_def.name.split(".")[-1], new_user_params=ctx.user_space_params)

logger.debug("Finished _dispatch_execute")

Expand Down
12 changes: 10 additions & 2 deletions flytekit/deck/deck.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,12 @@
def html(self) -> str:
return self._html

@classmethod
def publish(cls):
task_name = FlyteContextManager.current_context().user_space_params.task_id.name
new_user_params = FlyteContextManager.current_context().user_space_params
_output_deck(task_name=task_name, new_user_params=new_user_params)

Check warning on line 93 in flytekit/deck/deck.py

View check run for this annotation

Codecov / codecov/patch

flytekit/deck/deck.py#L91-L93

Added lines #L91 - L93 were not covered by tests


class TimeLineDeck(Deck):
"""
Expand Down Expand Up @@ -148,7 +154,8 @@


def _get_deck(
new_user_params: ExecutionParameters, ignore_jupyter: bool = False
new_user_params: ExecutionParameters,
ignore_jupyter: bool = False,
) -> typing.Union[str, "IPython.core.display.HTML"]: # type:ignore
"""
Get flyte deck html string
Expand Down Expand Up @@ -180,7 +187,7 @@
local_path = f"{local_dir}{os.sep}{DECK_FILE_NAME}"
try:
with open(local_path, "w", encoding="utf-8") as f:
f.write(_get_deck(new_user_params, ignore_jupyter=True))
f.write(_get_deck(new_user_params=new_user_params, ignore_jupyter=True))
logger.info(f"{task_name} task creates flyte deck html to file://{local_path}")
if ctx.execution_state.mode == ExecutionState.Mode.TASK_EXECUTION:
fs = ctx.file_access.get_filesystem_for_path(new_user_params.output_metadata_prefix)
Expand All @@ -197,6 +204,7 @@
def get_deck_template() -> Template:
root = os.path.dirname(os.path.abspath(__file__))
templates_dir = os.path.join(root, "html", "template.html")

with open(templates_dir, "r") as f:
template_content = f.read()
return Template(template_content)
Loading