Skip to content

Commit

Permalink
Fix kedro viz --load-file to run from any directory without requiri…
Browse files Browse the repository at this point in the history
…ng a Kedro project (#2206)

This PR fixes #2045.

Previously, kedro viz --load-file checked if the command was run from within a Kedro project directory, which is not required. The fix ensures that kedro viz --load-file can run from anywhere as long as it has the correct path the API data directory .

The key behavior is now as follows:

If the specified filepath does not exist, a ValueError is raised:
ValueError: The provided filepath 'hellod' does not exist.
If the filepath exists but does not contain the expected directory structure (api/main, api/pipelines/, and api/nodes/), a FileNotFoundError is raised:
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/xyz/api/main'

If the filepath exists and has the required api directory structure and conent, kedro viz --load-file runs successfully using the API JSON data.
  • Loading branch information
rashidakanchwala authored Nov 28, 2024
1 parent 6c6e8f1 commit 4d26f2f
Show file tree
Hide file tree
Showing 8 changed files with 79 additions and 44 deletions.
10 changes: 10 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ Please follow the established format:
- Include the ID number for the related PR (or PRs) in parentheses
-->

# Release 10.2.0


## Major features and improvements

## Bug fixes and other changes

- Fix kedro viz `--load-file` to run from any directory without requiring a Kedro project. (#2206)

# Release 10.1.0

## Major features and improvements
Expand All @@ -17,6 +26,7 @@ Please follow the established format:


## Bug fixes and other changes

- Fix tag being undefined when pipeline are ordered differently (#2162, #2146)
- Fix unserializable parameters value. (#2122)
- Update kedro-viz lite banner icon and message. (#2196)
Expand Down
23 changes: 21 additions & 2 deletions docs/source/cli-docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,10 @@ kedro viz run [OPTIONS]
- Whether to open the Kedro Viz interface in the default browser. The browser will open if the host is `localhost`. Defaults to `True`.

- `--load-file <path>`
- Path to load Kedro Viz data from a directory. If provided, Kedro Viz will load the visualisation data from this path instead of generating it from the pipeline.
- Path to load Kedro Viz data from a [directory](#kedro-viz-directory-structure-when-you-save-it-as-a-file). If provided, Kedro Viz will load the visualisation data from this path instead of generating it from the pipeline

- `--save-file <path>`
- Path to save Kedro Viz data to a directory. If provided, the visualisation data will be saved to this path for later use.
- Path to save Kedro Viz data to a [directory](#kedro-viz-directory-structure-when-you-save-it-as-a-file). If provided, the visualisation data will be saved to this path for later use.

- `--pipeline, -p <pipeline>`
- Name of the registered pipeline to visualise. If not set, the default pipeline is visualised.
Expand Down Expand Up @@ -162,4 +162,23 @@ kedro viz build --include-previews
```


### Kedro-viz directory structure when you save it as a file

When you use the `--save-file` option, Kedro Viz generates a directory structure to save the visualization data. This directory can later be used with the `--load-file` to reload the visualization.

The generated directory structure looks like this:

```bash
api/
├── main # Main file containing pipeline structure
├── nodes/
│ ├── node1 # JSON files for individual nodes
│ ├── node2
│ └── ...
├── pipelines/
│ ├── pipeline1 # JSON files for individual pipelines
│ ├── pipeline2
│ └── ...
```


2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
exclude_patterns = []
source_suffix = {".rst": "restructuredtext", ".md": "markdown"}

myst_heading_anchors = 2
myst_heading_anchors = 7

intersphinx_mapping = {
"kedro": ("https://docs.kedro.org/en/stable/", None),
Expand Down
4 changes: 2 additions & 2 deletions docs/source/kedro-viz_visualisation.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,8 +279,8 @@ The `%run_viz` command supports various optional arguments found in `kedro viz r

* `--host=<host>`: Specify the server host.
* `--port=<port>`: Set the server port.
* `--load-file=<file>`: Load a specific pipeline visualisation file.
* `--save-file=<file>`: Save the current pipeline visualisation to a file.
* `--load-file=<file>`: Load a specific pipeline visualisation from a [directory](./cli-docs.md#kedro-viz-directory-structure-when-you-save-it-as-a-file).
* `--save-file=<file>`: Save the current pipeline visualisation to a [directory](./cli-docs.md#kedro-viz-directory-structure-when-you-save-it-as-a-file).
* `--pipeline=<name>`: Visualise a specific pipeline.
* `--env=<name>`: Set the environment for the visualisation.
* `--autoreload`: Enable automatic reloading of the visualisation when source code changes.
Expand Down
25 changes: 15 additions & 10 deletions package/kedro_viz/launchers/cli/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,16 +123,21 @@ def run(
)
from kedro_viz.server import run_server

kedro_project_path = _find_kedro_project(Path.cwd())

if kedro_project_path is None:
display_cli_message(
"ERROR: Failed to start Kedro-Viz : "
"Could not find the project configuration "
f"file '{_PYPROJECT}' at '{Path.cwd()}'. ",
"red",
)
return
kedro_project_path = None

if load_file:
if not Path(load_file).exists():
raise ValueError(f"The provided filepath '{load_file}' does not exist.")
else:
kedro_project_path = _find_kedro_project(Path.cwd())
if kedro_project_path is None:
display_cli_message(
"ERROR: Failed to start Kedro-Viz : "
"Could not find the project configuration "
f"file '{_PYPROJECT}' at '{Path.cwd()}'. ",
"red",
)
return

installed_version = parse(__version__)
latest_version = get_latest_version()
Expand Down
3 changes: 0 additions & 3 deletions package/kedro_viz/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,9 +132,6 @@ def run_server(

app = apps.create_api_app_from_project(path, autoreload)
else:
if not Path(load_file).exists():
raise ValueError(f"The provided filepath '{load_file}' does not exist.")

app = apps.create_api_app_from_file(f"{path}/{load_file}/api")

uvicorn.run(app, host=host, port=port, log_config=None)
Expand Down
21 changes: 21 additions & 0 deletions package/tests/test_launchers/test_cli/test_run.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from pathlib import Path
from unittest.mock import call

import pytest
Expand Down Expand Up @@ -411,3 +412,23 @@ def test_find_available_port_with_occupied_ports(self, mocker):
assert (
available_port == 4143
), "Expected port 4143 to be returned as the available port"


def test_invalid_load_file_directory(mocker):
"""
Test that Kedro-Viz raises a ValueError when an invalid filepath
is provided to the `--load-file` argument.
"""
runner = CliRunner()

# Mock the existence of the file path to always return False (invalid path)
mocker.patch.object(Path, "exists", return_value=False)

# Invoke the CLI with an invalid `--load-file` path
result = runner.invoke(
main.viz_cli, ["viz", "run", "--load-file", "nonexistent_path.json"]
)

assert "The provided filepath 'nonexistent_path.json' does not exist." == str(
result.exception
)
35 changes: 9 additions & 26 deletions package/tests/test_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,32 +121,15 @@ def test_specific_pipeline(
{"data_science": example_pipelines["data_science"]}
)

@pytest.mark.parametrize(
"file_path, expected_exception",
[
("test.json", ValueError), # File does not exist, expect ValueError
("test.json", None), # File exists, expect no ValueError
],
)
def test_load_file(
self, file_path, expected_exception, patched_create_api_app_from_file, tmp_path
):
if expected_exception is not None:
with pytest.raises(expected_exception) as exc_info:
run_server(load_file=file_path)

# Check if the error message contains the expected message
assert "The provided filepath" in str(exc_info.value)
assert "does not exist." in str(exc_info.value)
else:
json_file_path = tmp_path / file_path

# File exists, no exception expected
with json_file_path.open("w") as file:
json.dump({"name": "John", "age": 30}, file)

run_server(load_file=json_file_path)
patched_create_api_app_from_file.assert_called_once()
def test_load_file(self, patched_create_api_app_from_file, tmp_path):
file_path = "test.json"
json_file_path = tmp_path / file_path

with json_file_path.open("w") as file:
json.dump({"name": "John", "age": 30}, file)

run_server(load_file=json_file_path)
patched_create_api_app_from_file.assert_called_once()

def test_save_file(self, tmp_path, mocker):
mock_filesystem = mocker.patch("fsspec.filesystem")
Expand Down

0 comments on commit 4d26f2f

Please sign in to comment.