Skip to content

Commit

Permalink
Merge pull request #69 from kubeflow/main
Browse files Browse the repository at this point in the history
[pull] main from kubeflow:main
  • Loading branch information
openshift-merge-bot[bot] authored Jun 10, 2024
2 parents 878d35f + f38a86e commit b2d3397
Show file tree
Hide file tree
Showing 29 changed files with 646 additions and 169 deletions.
40 changes: 40 additions & 0 deletions .github/workflows/ui-bff-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: UI - BFF - Build
on:
push:
branches:
- 'main'
pull_request:
paths-ignore:
- 'LICENSE*'
- 'DOCKERFILE*'
- '**.gitignore'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.22.2'

- name: Clean
working-directory: clients/ui/bff
run: make clean

- name: Build
working-directory: clients/ui/bff
run: make build

- name: Check if there are uncommitted file changes
working-directory: clients/ui/bff
run: |
clean=$(git status --porcelain)
if [[ -z "$clean" ]]; then
echo "Empty git status --porcelain: $clean"
else
echo "Uncommitted file changes detected: $clean"
git diff
exit 1
fi
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
# Test binary, built with `go test -c`
*.test

__debug*

# Output of the go coverage tool, specifically when used with LiteIDE
*.out

Expand Down
41 changes: 41 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,19 @@ and now you can substitute `gmake` every time the make command is mentioned in g
Several options of docker engines are available for Mac.
Having Docker installed is also helpful for Testcontainers.

### Podman

[Podman](https://podman.io/) also supports Rosetta (Apple specific) emulation which is handy since the Google MLMD project dependency is x86 specific.

We recommend setting up the Podman machine with root privileges,
and setting the environment variable

```sh
export TESTCONTAINERS_RYUK_PRIVILEGED=true
```

when running TestContainer-based Model Registry Python tests (for more information, see [here](https://pypi.org/project/testcontainers/#:~:text=TESTCONTAINERS_RYUK_PRIVILEGED)).

### Colima

Colima offers Rosetta (Apple specific) emulation which is handy since the Google MLMD project dependency is x86 specific.
Expand Down Expand Up @@ -202,3 +215,31 @@ Then with the given setup MLMD is already installed inside the DevContainer:
At this point Poetry is already installed as well and can be used to build and run test of the Model Registry Python client.

<!-- to be continued with explanation of this "hack": https://github.com/tarilabs/ml-metadata-remote#readme -->

# FAQ

## Error `docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/containers ...`

This happens on Mac OSX when running Testcontainers-based Python tests, and [Ryuk](https://github.com/testcontainers/moby-ryuk) does not have the correct access priviledges.

```
-------------------------------------------- Captured stderr setup ---------------------------------------------
Pulling image testcontainers/ryuk:0.7.0
Container started: 4af385c2d670
---------------------------------------------- Captured log setup ----------------------------------------------
WARNING root:config.py:59 DOCKER_AUTH_CONFIG is experimental, see testcontainers/testcontainers-python#566
INFO testcontainers.core.container:container.py:88 Pulling image testcontainers/ryuk:0.7.0
INFO testcontainers.core.container:container.py:101 Container started: 4af385c2d670
=========================================== short test summary info ============================================
ERROR tests/store/test_wrapper.py::test_get_undefined_artifact_type_id - docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/containers/4af385c2d67067875f6f0...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================== 1 error in 0.50s ===============================================
```

Solution:
- Use `export TESTCONTAINERS_RYUK_PRIVILEGED=true` (we recommend this option)
- Use `export TESTCONTAINERS_RYUK_DISABLED=true` to disable Ryuk entirely; the tests are configured to close all container resources, but we do not recommend this option

For more information:
- section [Docker engine](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#docker-engine) in this document
- https://pypi.org/project/testcontainers/#:~:text=the%20database%20version.-,Configuration,-Env%20Variable
27 changes: 26 additions & 1 deletion clients/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,30 @@ There are caveats to be noted when using this method:
)
```

## Advanced use-cases

### Using Model Registry Python Client with newer Python versions (>=3.11)

> [!CAUTION]
> The mechanism described in this section is a temporary workaround and likely will never be supported.
> This workaround is ONLY applicable if your Python/Notebook project does NOT make use of MLMD directly or indirectly.

<!-- a longer-term plan to address this ties to the investigations to rebase this client on top of MR REST api directly,
so to avoid having to wrap the MLMD Wheel. See more: https://github.com/kubeflow/model-registry/pull/59 -->

This project _currently_ depends for internal implementations on the Google's [MLMD Python library](https://pypi.org/project/ml-metadata/).
Due to this dependency, this project supports [only the Python versions](https://github.com/kubeflow/model-registry/blob/8d77c13100c6cc5a9465d4293403114a3576fdd7/clients/python/pyproject.toml#L14) which are also available for the MLMD library (see more [here](https://pypi.org/project/ml-metadata/#files)).

As a workaround, **only IF your Python/Notebook project does NOT make use of MLMD directly or indirectly**,
you could opt-in to make use of a non-constrained variant of the MLMD dependency supporting _only_ remote gRPC calls (and not constrained by specific Python versions or architectures):

```
!pip install "https://github.com/opendatahub-io/ml-metadata/releases/download/v1.14.0%2Bremote.1/ml_metadata-1.14.0+remote.1-py3-none-any.whl" # need a Python 3.11 compatible version
!pip install --no-deps --ignore-requires-python --pre "model-registry" # ignore dependencies because of the above override
```

You can read more about this use-case, in the [Remote-only packaging of MLMD Python lib](https://github.com/kubeflow/model-registry/blob/main/docs/remote_only_packaging_of_MLMD_Python_lib.md) document.

## Development

Common tasks, such as building documentation and running tests, can be executed using [`nox`](https://github.com/wntrblm/nox) sessions.
Expand All @@ -110,13 +134,14 @@ Use `nox -l` to list sessions and execute them using `nox -s [session]`.

### Running Locally on Mac M1 or M2 (arm64 architecture)

If you want run tests locally you will need to set up a colima develeopment environment using the instructions [here](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#colima)
If you want run tests locally you will need to set up a development environment, including docker engine; we recommend following the instructions [here](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#docker-engine).

You will also have to change the package source to one compatible with ARM64 architecture. This can be actioned by uncommenting lines 14 or 15 in the pyproject.toml file. Run the following command after you have uncommented the line.

```sh
poetry lock
```

Use the following commands to directly run the tests with individual test output. Alternatively you can use the nox session commands above.

```sh
Expand Down
Loading

0 comments on commit b2d3397

Please sign in to comment.