[BUG]: Trying to run `zenml go` in a docker container results in errors #1742

jlopezpena · 2023-08-15T15:58:47Z

Contact Details [Optional]

No response

System Information

ZENML_LOCAL_VERSION: 0.43.0
ZENML_SERVER_VERSION: 0.43.0
ZENML_SERVER_DATABASE: sqlite
ZENML_SERVER_DEPLOYMENT_TYPE: other
ZENML_CONFIG_DIR: /root/.config/zenml
ZENML_LOCAL_STORE_DIR: /root/.config/zenml/local_stores
ZENML_SERVER_URL: sqlite:////root/.config/zenml/local_stores/default_zen_store/zenml.db
ZENML_ACTIVE_REPOSITORY_ROOT: None
PYTHON_VERSION: 3.11.4
ENVIRONMENT: native
SYSTEM_INFO: {'os': 'linux', 'linux_distro': 'ubuntu', 'linux_distro_like': 'debian', 'linux_distro_version':
'23.04'}
ACTIVE_WORKSPACE: default
ACTIVE_STACK: default
ACTIVE_USER: default
TELEMETRY_STATUS: disabled
ANALYTICS_CLIENT_ID: 38971716-74f6-47aa-9525-332d6258675b
ANALYTICS_USER_ID: e18d86b1-d795-476a-8c9a-7cd265254991
ANALYTICS_SERVER_ID: 38971716-74f6-47aa-9525-332d6258675b
INTEGRATIONS: ['kaniko']
PACKAGES: {'babel': '2.12.1', 'gitpython': '3.1.32', 'jinja2': '3.1.2', 'mako': '1.2.4', 'markupsafe': '2.1.3',
'pymysql': '1.0.3', 'pyyaml': '6.0.1', 'pygments': '2.16.1', 'sqlalchemy': '1.4.41', 'sqlalchemy-utils': '0.38.3',
'send2trash': '1.8.2', 'alabaster': '0.7.13', 'alembic': '1.8.1', 'analytics-python': '1.4.post1', 'anyio':
'3.7.1', 'argon2-cffi': '23.1.0', 'argon2-cffi-bindings': '21.2.0', 'arrow': '1.2.3', 'asttokens': '2.2.1',
'async-lru': '2.0.4', 'attrs': '23.1.0', 'azure-common': '1.1.28', 'azure-core': '1.29.2', 'azure-mgmt-core':
'1.4.0', 'azure-mgmt-resource': '23.0.1', 'backcall': '0.2.0', 'backoff': '1.10.0', 'bcrypt': '4.0.1',
'beautifulsoup4': '4.12.2', 'black': '23.7.0', 'bleach': '6.0.0', 'boto3': '1.28.26', 'botocore': '1.31.26',
'certifi': '2023.7.22', 'cffi': '1.15.1', 'charset-normalizer': '3.2.0', 'click': '8.1.3', 'click-params': '0.3.0',
'cloudpickle': '2.2.1', 'colorama': '0.4.6', 'comm': '0.1.4', 'commonmark': '0.9.1', 'coverage': '7.3.0',
'debugpy': '1.6.7.post1', 'decorator': '5.1.1', 'defusedxml': '0.7.1', 'distro': '1.8.0', 'docker': '6.1.3',
'docutils': '0.20.1', 'executing': '1.2.0', 'fastjsonschema': '2.18.0', 'fqdn': '1.5.1', 'furo': '2023.7.26',
'gitdb': '4.0.10', 'greenlet': '2.0.2', 'httplib2': '0.19.1', 'idna': '3.4', 'imagesize': '1.4.1', 'iniconfig':
'2.0.0', 'ipykernel': '6.25.1', 'ipython': '8.14.0', 'ipython-genutils': '0.2.0', 'ipywidgets': '7.8.0', 'isodate':
'0.6.1', 'isoduration': '20.11.0', 'jedi': '0.19.0', 'jmespath': '1.0.1', 'json5': '0.9.14', 'jsonpointer': '2.4',
'jsonschema': '4.19.0', 'jsonschema-specifications': '2023.7.1', 'jupyter-client': '8.3.0', 'jupyter-core':
'5.3.1', 'jupyter-events': '0.7.0', 'jupyter-lsp': '2.2.0', 'jupyter-server': '2.7.1', 'jupyter-server-terminals':
'0.4.4', 'jupyterlab': '4.0.5', 'jupyterlab-pygments': '0.2.2', 'jupyterlab-server': '2.24.0',
'jupyterlab-widgets': '1.1.5', 'livereload': '2.6.3', 'matplotlib-inline': '0.1.6', 'mistune': '3.0.1',
'monotonic': '1.6', 'mypy': '1.5.0', 'mypy-extensions': '1.0.0', 'nbclient': '0.8.0', 'nbconvert': '7.7.3',
'nbformat': '5.9.2', 'nest-asyncio': '1.5.7', 'notebook': '7.0.2', 'notebook-shim': '0.2.3', 'numpy': '1.25.2',
'overrides': '7.4.0', 'packaging': '22.0', 'pandas': '2.0.3', 'pandocfilters': '1.5.0', 'parso': '0.8.3',
'passlib': '1.7.4', 'pathspec': '0.11.2', 'pexpect': '4.8.0', 'pickleshare': '0.7.5', 'pip': '23.2.1',
'platformdirs': '3.10.0', 'pluggy': '1.2.0', 'prometheus-client': '0.17.1', 'prompt-toolkit': '3.0.39', 'psutil':
'5.9.5', 'ptyprocess': '0.7.0', 'pure-eval': '0.2.2', 'pycparser': '2.21', 'pydantic': '1.10.12', 'pyparsing':
'2.4.7', 'pytest': '7.4.0', 'python-dateutil': '2.8.2', 'python-json-logger': '2.0.7', 'python-terraform':
'0.10.1', 'pytz': '2023.3', 'pyzmq': '25.1.1', 'referencing': '0.30.2', 'requests': '2.31.0', 'rfc3339-validator':
'0.1.4', 'rfc3986-validator': '0.1.1', 'rich': '12.6.0', 'rpds-py': '0.9.2', 'ruff': '0.0.284', 's3transfer':
'0.6.1', 'setuptools': '68.0.0', 'six': '1.16.0', 'smmap': '5.0.0', 'sniffio': '1.3.0', 'snowballstemmer': '2.2.0',
'soupsieve': '2.4.1', 'sphinx': '7.1.2', 'sphinx-autobuild': '2021.3.14', 'sphinx-basic-ng': '1.0.0b2',
'sphinxcontrib-applehelp': '1.0.7', 'sphinxcontrib-devhelp': '1.0.5', 'sphinxcontrib-htmlhelp': '2.0.4',
'sphinxcontrib-jsmath': '1.0.1', 'sphinxcontrib-qthelp': '1.0.6', 'sphinxcontrib-serializinghtml': '1.1.8',
'sqlalchemy2-stubs': '0.0.2a35', 'sqlmodel': '0.0.8', 'stack-data': '0.6.2', 'terminado': '0.17.1', 'tinycss2':
'1.2.1', 'tornado': '6.3.3', 'traitlets': '5.9.0', 'typing-extensions': '4.7.1', 'tzdata': '2023.3',
'uri-template': '1.3.0', 'urllib3': '1.26.16', 'validators': '0.18.2', 'wcwidth': '0.2.6', 'webcolors': '1.13',
'webencodings': '0.5.1', 'websocket-client': '1.6.1', 'wf-aws': '1.12.2', 'wf-blackbox-zero': '1.1.0', 'wf-cli':
'1.25.0', 'wf-cli-plugin-aws': '1.5.6', 'wf-cli-plugin-docs': '1.2.6', 'wf-cli-plugin-login': '1.0.5',
'wf-cli-plugin-py': '2.3.4', 'wf-zenml-pipelines': '0.0.0', 'widgetsnbextension': '3.6.5', 'zenml': '0.43.0'}

CURRENT STACK

Name: default
ID: c5036656-d46b-4988-8b85-5cc6221eb0eb
Shared: No
User: default / e18d86b1-d795-476a-8c9a-7cd265254991
Workspace: default / 6d5238df-adb2-4d00-8804-856fbecfcfbf

ORCHESTRATOR: default

Name: default
ID: cf9ab169-8724-4602-b56f-9bbbb830d0ca
Type: orchestrator
Flavor: local
Configuration: {}
Shared: No
User: default / e18d86b1-d795-476a-8c9a-7cd265254991
Workspace: default / 6d5238df-adb2-4d00-8804-856fbecfcfbf

ARTIFACT_STORE: default

Name: default
ID: 22fab2ea-f53a-4d49-95c6-7800c532b994
Type: artifact_store
Flavor: local
Configuration: {'path': ''}
Shared: No
User: default / e18d86b1-d795-476a-8c9a-7cd265254991
Workspace: default / 6d5238df-adb2-4d00-8804-856fbecfcfbf

What happened?

Trying to run the zenml tutorial on a dev container (docker), when it gets to the point of running the jupyter notebook server the process fails because commands in the docker container are run as root.

This can be solved in a variety of ways:

Manually running the jupyter notebook server and opening the tutorial notebooks from there
Allowing zenml to pass the --allow-root flag when instantiating the jupyter notebook server (for instance, if some env variable is present)
Not launching the jupyter notebook server at all, and instead using vscode notebook extension to explore the tutorial notebooks

However, the zenml go path does not suggest any ways of fixing this, instead crashes with the very cryptic error message

CalledProcessError: Command '['jupyter', 'notebook']' returned non-zero exit status 1.

and a really long non-informative stack trace

Reproduction steps

Make a docker container that contains zenml as a dependency (eg a dev container)
In the container shell, run zenml go ad go through the steps

This error will come up:

I 2023-08-15 15:46:20.910 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-08-15 15:46:20.911 ServerApp] notebook | extension was successfully loaded.
[C 2023-08-15 15:46:20.912 ServerApp] Running as root is not recommended. Use --allow-root to bypass.

Relevant log output

🧑‍🏫 Get started with ZenML                           

The ZenML tutorial repository was cloned to your current working directory.     
Within the repository you can get started on one of these notebooks:            

 • /workspaces/zenml_pipelines/zenml_tutorial/notebooks/quickstart-zenml-cloud.i
   pynb                                                                         
 • /workspaces/zenml_pipelines/zenml_tutorial/notebooks/quickstart.ipynb        

Next we will start a Jupyter notebook server. Feel free to try your hand at our 
tutorial notebooks. If your browser does not open automatically click one of the
links below.                                                                    
Press ENTER to continue...
[I 2023-08-15 15:46:20.494 ServerApp] Package notebook took 0.0000s to import
[I 2023-08-15 15:46:20.508 ServerApp] Package jupyter_lsp took 0.0135s to import
[W 2023-08-15 15:46:20.508 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-08-15 15:46:20.515 ServerApp] Package jupyter_server_terminals took 0.0070s to import
[I 2023-08-15 15:46:20.515 ServerApp] Package jupyterlab took 0.0000s to import
[I 2023-08-15 15:46:20.771 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-08-15 15:46:20.771 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-08-15 15:46:20.772 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2023-08-15 15:46:20.774 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-08-15 15:46:20.776 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-08-15 15:46:20.778 ServerApp] notebook | extension was successfully linked.
[I 2023-08-15 15:46:20.779 ServerApp] Writing Jupyter server cookie secret to /opt/wf-state/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2023-08-15 15:46:20.886 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-08-15 15:46:20.906 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-08-15 15:46:20.907 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-08-15 15:46:20.908 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-08-15 15:46:20.909 LabApp] JupyterLab extension loaded from /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/jupyterlab
[I 2023-08-15 15:46:20.909 LabApp] JupyterLab application directory is /workspaces/zenml_pipelines/.venv/share/jupyter/lab
[I 2023-08-15 15:46:20.909 LabApp] Extension Manager is 'pypi'.
[I 2023-08-15 15:46:20.910 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-08-15 15:46:20.911 ServerApp] notebook | extension was successfully loaded.
[C 2023-08-15 15:46:20.912 ServerApp] Running as root is not recommended. Use --allow-root to bypass.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /workspaces/zenml_pipelines/.venv/bin/zenml:8 in <module>                                        │
│                                                                                                  │
│   5 from zenml.cli.cli import cli                                                                │
│   6 if __name__ == "__main__":                                                                   │
│   7 │   sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1130 in __call__    │
│                                                                                                  │
│   1127 │                                                                                         │
│   1128 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1129 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1130 │   │   return self.main(*args, **kwargs)                                                 │
│   1131                                                                                           │
│   1132                                                                                           │
│   1133 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1055 in main        │
│                                                                                                  │
│   1052 │   │   try:                                                                              │
│   1053 │   │   │   try:                                                                          │
│   1054 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                  │
│ ❱ 1055 │   │   │   │   │   rv = self.invoke(ctx)                                                 │
│   1056 │   │   │   │   │   if not standalone_mode:                                               │
│   1057 │   │   │   │   │   │   return rv                                                         │
│   1058 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                               │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1657 in invoke      │
│                                                                                                  │
│   1654 │   │   │   │   super().invoke(ctx)                                                       │
│   1655 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658 │   │                                                                                     │
│   1659 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1660 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:1404 in invoke      │
│                                                                                                  │
│   1401 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1402 │   │                                                                                     │
│   1403 │   │   if self.callback is not None:                                                     │
│ ❱ 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1405 │                                                                                         │
│   1406 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:  │
│   1407 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/click/core.py:760 in invoke       │
│                                                                                                  │
│    757 │   │                                                                                     │
│    758 │   │   with augment_usage_errors(__self):                                                │
│    759 │   │   │   with ctx:                                                                     │
│ ❱  760 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    761 │                                                                                         │
│    762 │   def forward(                                                                          │
│    763 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ /workspaces/zenml_pipelines/.venv/lib/python3.11/site-packages/zenml/cli/base.py:405 in go       │
│                                                                                                  │
│   402 │   │   )                                                                                  │
│   403 │   │   input("Press ENTER to continue...")                                                │
│   404 │   notebook_path = os.path.join(zenml_tutorial_path, "notebooks")                         │
│ ❱ 405 │   subprocess.check_call(["jupyter", "notebook"], cwd=notebook_path)                      │
│   406                                                                                            │
│   407                                                                                            │
│   408 def _prompt_email(event_source: AnalyticsEventSource) -> bool:                             │
│                                                                                                  │
│ /usr/lib/python3.11/subprocess.py:413 in check_call                                              │
│                                                                                                  │
│    410 │   │   cmd = kwargs.get("args")                                                          │
│    411 │   │   if cmd is None:                                                                   │
│    412 │   │   │   cmd = popenargs[0]                                                            │
│ ❱  413 │   │   raise CalledProcessError(retcode, cmd)                                            │
│    414 │   return 0                                                                              │
│    415                                                                                           │
│    416                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CalledProcessError: Command '['jupyter', 'notebook']' returned non-zero exit status 1.

Code of Conduct

I agree to follow this project's Code of Conduct

The text was updated successfully, but these errors were encountered:

strickvl · 2024-02-05T16:54:13Z

Maybe you could add a non-root user to the Dockerfile as a way round this?

# Create a new user "zenmluser" and set a working directory
RUN useradd -m zenmluser
WORKDIR /home/zenmluser

# Copy the application to the container and adjust permissions
COPY . /home/zenmluser
RUN chown -R zenmluser:zenmluser /home/zenmluser

# Switch to the new user
USER zenmluser

htahir1 · 2024-08-26T16:55:32Z

@jlopezpena Did this end up working for you? If yes we can close it !

jlopezpena added the bug Something isn't working label Aug 15, 2023

strickvl self-assigned this Feb 5, 2024

coderabbitai bot mentioned this issue Mar 29, 2024

Catch missing jupyter installation for zenml go #2571

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG]: Trying to run `zenml go` in a docker container results in errors #1742

[BUG]: Trying to run `zenml go` in a docker container results in errors #1742

jlopezpena commented Aug 15, 2023

strickvl commented Feb 5, 2024

htahir1 commented Aug 26, 2024 •

edited

Loading

[BUG]: Trying to run zenml go in a docker container results in errors #1742

[BUG]: Trying to run zenml go in a docker container results in errors #1742

Comments

jlopezpena commented Aug 15, 2023

Contact Details [Optional]

System Information

What happened?

Reproduction steps

Relevant log output

Code of Conduct

strickvl commented Feb 5, 2024

htahir1 commented Aug 26, 2024 • edited Loading

[BUG]: Trying to run `zenml go` in a docker container results in errors #1742

[BUG]: Trying to run `zenml go` in a docker container results in errors #1742

htahir1 commented Aug 26, 2024 •

edited

Loading