[Epic] Contribute to successful usage of Thoth Service in the OS-Climate project

[codificat]

Problem statement

The aicoe-osc-demo repository is, at the moment, not taking full advantage of Thoth services.

High-level Goals

We would like that Thoth's guidance and pipeline services are successfully used by the OS-Climate reference project, so that further collaboration with the project continues to use Thoth services.

This implies that the aicoe-osc-demo repository should be able to:

• get Thoth advise to manage their notebooks and software stacks dependencies, manually (e.g. via thamos and/or the jupiterlab extension)
• have Kebechet automatically update dependencies in the repository, based on Thoth advise
• automatically build images from the repository content for data scientists to use in the operate first Open Data Hub environment

Proposal description

The work here involves working together with the team that maintains the repo to make sure that:

• the repository's dependencies can be advised by Thoth
• container images can be successfully built and maintained
• users of the repository understand the workflow involved, how to use the service, and what are the benefits of doing so

Alternatives

Leave as is

Additional context

This is part of this quarter's OKRs

Acceptance Criteria

• thamos advise works correctly with the repo's Pipfile
  • Application cannot be managed by Kebechet due to it containing an unsupported package location. os-climate/aicoe-osc-demo#43
  • Application cannot be managed by Kebechet due to it containing an unsupported package location. aicoe-aiops/project-template#48
• beyond the aicoe-ci-demo repo: explore options to have kebechet / thamos work on repos that use local files and/or file dependencies.
  • [3pt] Document Thoth's requirement for dependencies to be in an index #411
  • An option to ignore local dependencies #412
• Kebechet is correctly installed on the repo, with the advise manager enabled
  • Remove pipfile-requirements manager from .thoth.yaml os-climate/aicoe-osc-demo#207
• Pipelines correctly build and publish container images on demand
• the repo's documentation contains enough information (links, references) for users to understand the service
• the demos in the repo include references to the service (how to maintain dependencies, how to build/deploy, etc) where appropriate
• The repo uses multiple overlays to simplify each stack
  • Use overlays for building images os-climate/aicoe-osc-demo#121
• Kebechet functionality meets the requirements of the OS-C repos:
  • [2pt] Add signed-off-by in commit messages kebechet#1067
  • [5pt] An option to configure different managers per overlay kebechet#1126
  • [3pt] Allow local and git package refereces in update manager kebechet#1125
• Packages used by OS-C are known to Thoth adviser
  • Include package osc-ingest-tools in recommendations support#220
  • Include Torchvision support#245
  • Include pdf2image, tabula-py, mmdet, farm, pyspellchecker support#273
• Ongoing verification that Thoth services work on the repo: Add OS-Climate demo git repo to integration tests integration-tests#340

[codificat]

/sig user-experience
/triage accepted

Planning to work on this next, so
/assign
/priority critical-urgent

[codificat]

/milestone OKR review Q2 2022

[codificat]

/lifecycle active

[mayaCostantini]

/assign

[codificat]

The OSC https://github.com/os-climate/data-platform-demo repo has a simpler Pipfile without path/file based deps.

Maybe start with that one first?

Currently, the only thing preventing a thoth advise for that repo is that one of the dependencies is not known: osc-ingest-tools (https://github.com/os-climate/osc-ingest-tools).

[codificat]

The data-platform-repo now meets all the requirements to receive advice, and a thamos advise request succeeds.

However, PRs from the bot's advise manager would fail checks due to missing DCO - created thoth-station/kebechet#1067 for that

[goern]

any update on this?

[codificat]

Description updated with more related issues/tasks