kubeflow · google-oss-prow · Oct 10, 2024 · Oct 9, 2024
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,6 +1,6 @@
 <!-- Thanks for sending a pull request! Here are some tips for you:
 1. If this is your first time, check our contributor guidelines https://www.kubeflow.org/docs/about/contributing
-2. To know more about Katib components, check developer guide https://github.com/kubeflow/katib/blob/master/docs/developer-guide.md
+2. To know more about Katib components, check developer guide https://github.com/kubeflow/katib/blob/master/CONTRIBUTING.md
 3. If you want *faster* PR reviews, check how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
 -->
 

diff --git a/docs/developer-guide.md → CONTRIBUTING.md b/docs/developer-guide.md → CONTRIBUTING.md
@@ -2,13 +2,12 @@
 
 This developer guide is for people who want to contribute to the Katib project.
 If you're interesting in using Katib in your machine learning project,
-see the following user guides:
+see the following guides:
 
-- [Concepts](https://www.kubeflow.org/docs/components/katib/overview/)
- in Katib, hyperparameter tuning, and neural architecture search.
 - [Getting started with Katib](https://kubeflow.org/docs/components/katib/hyperparameter/).
-- Detailed guide to [configuring and running a Katib
- experiment](https://kubeflow.org/docs/components/katib/experiment/).
+- [How to configure Katib Experiment](https://kubeflow.org/docs/components/katib/experiment/).
+- [Katib architecture and concepts](https://www.kubeflow.org/docs/components/katib/reference/architecture/)
+ for hyperparameter tuning and neural architecture search.
 
 ## Requirements
 
@@ -93,10 +92,6 @@ Below is a list of command-line flags accepted by Katib DB Manager:
 | --------------- | ------------- | ------- | ------------------------------------------------------- |
 | connect-timeout | time.Duration | 60s | Timeout before calling error during database connection |
 
-## Workflow design
-
-Please see [workflow-design.md](./workflow-design.md).
-
 ## Katib admission webhooks
 
 Katib uses three [Kubernetes admission webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/).
@@ -113,7 +108,7 @@ Katib uses three [Kubernetes admission webhooks](https://kubernetes.io/docs/refe
 1. `mutator.pod.katib.kubeflow.org` - Mutating admission webhook to inject the metrics
  collector sidecar container to the training pod. Learn more about the Katib's
  metrics collector in the
- [Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/experiment/#metrics-collector).
+ [Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/user-guides/metrics-collector/).
 
 You can find the YAMLs for the Katib webhooks
 [here](../manifests/v1beta1/components/webhook/webhooks.yaml).
@@ -168,4 +163,4 @@ they'll be executed against every file in the repository.
 
 Specific programmatically generated files listed in the `exclude` field in
 [.pre-commit-config.yaml](../.pre-commit-config.yaml) are deliberately excluded
-from the hooks. 
+from the hooks.
diff --git a/README.md b/README.md
@@ -29,13 +29,13 @@ and many more.
 
 Katib stands for `secretary` in Arabic.
 
-# Search Algorithms
+## Search Algorithms
 
 Katib supports several search algorithms. Follow the
-[Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/experiment/#search-algorithms-in-detail)
+[Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#hp-tuning-algorithms)
 to know more about each algorithm and check the
-[Suggestion service guide](/docs/new-algorithm-service.md) to implement your
-custom algorithm.
+[this guide](https://www.kubeflow.org/docs/components/katib/user-guides/hp-tuning/configure-algorithm/#use-custom-algorithm-in-katib)
+to implement your custom algorithm.
 
 <table>
  <tbody>
@@ -137,141 +137,68 @@ custom algorithm.
  </tbody>
 </table>
 
-To perform above algorithms Katib supports the following frameworks:
+To perform the above algorithms Katib supports the following frameworks:
 
 - [Goptuna](https://github.com/c-bata/goptuna)
 - [Hyperopt](https://github.com/hyperopt/hyperopt)
 - [Optuna](https://github.com/optuna/optuna)
 - [Scikit Optimize](https://github.com/scikit-optimize/scikit-optimize)
 
-# Installation
-
-For the various Katib installs check the
-[Kubeflow guide](https://www.kubeflow.org/docs/components/katib/hyperparameter/#katib-setup).
-Follow the next steps to install Katib standalone.
-
 ## Prerequisites
 
-This is the minimal requirements to install Katib:
-
-- Kubernetes >= 1.27
-- `kubectl` >= 1.27
+Please check [the official Kubeflow documentation](https://www.kubeflow.org/docs/components/katib/installation/#prerequisites)
+for prerequisites to install Katib.
 
-## Latest Version
+## Installation
 
-For the latest Katib version run this command:
-
-```
-kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=master"
-```
+Please follow [the Kubeflow Katib guide](https://www.kubeflow.org/docs/components/katib/installation/#installing-katib)
+for the detailed instructions on how to install Katib.
 
-## Release Version
+### Installing the Control Plane
 
-For the specific Katib release (for example `v0.14.0`) run this command:
+Run the following command to install the latest stable release of Katib control plane:
 
 ```
-kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=v0.14.0"
+kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=v0.17.0"
 ```
 
-Make sure that all Katib components are running:
+Run the following command to install the latest changes of Katib control plane:
 
 ```
-$ kubectl get pods -n kubeflow
-
-NAME READY STATUS RESTARTS AGE
-katib-controller-566595bdd8-hbxgf 1/1 Running 0 36s
-katib-db-manager-57cd769cdb-4g99m 1/1 Running 0 36s
-katib-mysql-7894994f88-5d4s5 1/1 Running 0 36s
-katib-ui-5767cfccdc-pwg2x 1/1 Running 0 36s
+kubectl apply -k "github.com/kubeflow/katib.git/manifests/v1beta1/installs/katib-standalone?ref=master"
 ```
 
 For the Katib Experiments check the [complete examples list](./examples/v1beta1).
 
-# Quickstart
+### Installing the Python SDK
 
-You can run your first HyperParameter Tuning Experiment using [Katib Python SDK](./sdk/python/v1beta1).
+Katib implements [a Python SDK](https://pypi.org/project/kubeflow-katib/) to simplify creation of
+hyperparameter tuning jobs for Data Scientists.
 
-In the following example we are going to maximize a simple objective function:
-$F(a,b) = 4a - b^2$. The bigger $a$ and the lesser $b$ value, the bigger the function value $F$.
+Run the following command to install the latest stable release of Katib SDK:
 
-```python
-import kubeflow.katib as katib
-
-# Step 1. Create an objective function.
-def objective(parameters):
- # Import required packages.
- import time
- time.sleep(5)
- # Calculate objective function.
- result = 4 * int(parameters["a"]) - float(parameters["b"]) ** 2
- # Katib parses metrics in this format: <metric-name>=<metric-value>.
- print(f"result={result}")
-
-# Step 2. Create HyperParameter search space.
-parameters = {
- "a": katib.search.int(min=10, max=20),
- "b": katib.search.double(min=0.1, max=0.2)
-}
-
-# Step 3. Create Katib Experiment.
-katib_client = katib.KatibClient()
-name = "tune-experiment"
-katib_client.tune(
- name=name,
- objective=objective,
- parameters=parameters,
- objective_metric_name="result",
- max_trial_count=12
-)
-
-# Step 4. Get the best HyperParameters.
-print(katib_client.get_optimal_hyperparameters(name))
+```sh
+pip install -U kubeflow-katib
 ```
 
-# Documentation
-
-- Check
- [the Katib getting started guide](https://www.kubeflow.org/docs/components/katib/hyperparameter/#example-using-random-search-algorithm).
-
-- Learn about Katib **Concepts** in this
- [guide](https://www.kubeflow.org/docs/components/katib/overview/#katib-concepts).
-
-- Learn about Katib **Interfaces** in this
- [guide](https://www.kubeflow.org/docs/components/katib/overview/#katib-interfaces).
-
-- Learn about Katib **Components** in this
- [guide](https://www.kubeflow.org/docs/components/katib/hyperparameter/#katib-components).
-
-- Know more about Katib in the [presentations and demos list](./docs/presentations.md).
+## Getting Started
 
-# Community
+Please refer to [the getting started guide](https://www.kubeflow.org/docs/components/katib/getting-started/#getting-started-with-katib-python-sdk)
+to quickly create your first hyperparameter tuning Experiment using the Python SDK.
 
-We are always growing our community and invite new users and AutoML enthusiasts
-to contribute to the Katib project. The following links provide information
-about getting involved in the community:
+## Community
 
-- Subscribe to the
- [AutoML calendar](https://calendar.google.com/calendar/u/0/r?cid=ZDQ5bnNpZWZzbmZna2Y5MW8wdThoMmpoazRAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ)
- to attend Working Group bi-weekly community meetings.
+The following links provide information on how to get involved in the community:
 
-- Check the
- [AutoML and Training Working Group meeting notes](https://docs.google.com/document/d/1MChKfzrKAeFRtYqypFbMXL6ZIc_OgijjkvbqmwRV-64/edit).
-
-- If you use Katib, please update [the adopters list](ADOPTERS.md).
+- Attend [the bi-weekly AutoML and Training Working Group](https://bit.ly/2PWVCkV)
+ community meeting.
+- Join our [`#kubeflow-katib`](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels)
+ Slack channel.
+- Check out [who is using Katib](ADOPTERS.md) and [presentations about Katib project](docs/presentations.md).
 
 ## Contributing
 
-Please feel free to test the system! [Developer guide](./docs/developer-guide.md)
-is a good starting point for our developers.
-
-## Blog posts
-
-- [Kubeflow Katib: Scalable, Portable and Cloud Native System for AutoML](https://blog.kubeflow.org/katib/)
- (by Andrey Velichkevich)
-
-## Events
-
-- [AutoML and Training WG Summit. 16th of July 2021](https://docs.google.com/document/d/1vGluSPHmAqEr8k9Dmm82RcQ-MVnqbYYSfnjMGB-aPuo/edit?usp=sharing)
+Please refer to the [CONTRIBUTING guide](CONTRIBUTING.md).
 
 ## Citation
 

diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,5 @@
+# Katib Documentation
+
+Welcome to Kubeflow Katib!
+
+The Katib documentation is available on [kubeflow.org](https://www.kubeflow.org/docs/components/katib/).
diff --git a/docs/images-location.md b/docs/images-location.md
@@ -5,7 +5,7 @@ Here you can find the location for images that are used in Katib.
 ## Katib Components Images
 
 The following table shows images for the
-[Katib components](https://www.kubeflow.org/docs/components/katib/hyperparameter/#katib-components).
+[Katib components](https://www.kubeflow.org/docs/components/katib/reference/architecture/#katib-control-plane-components).
 
 <table>
  <tbody>
@@ -70,7 +70,7 @@ The following table shows images for the
 ## Katib Metrics Collectors Images
 
 The following table shows images for the
-[Katib Metrics Collectors](https://www.kubeflow.org/docs/components/katib/experiment/#metrics-collector).
+[Katib Metrics Collectors](https://www.kubeflow.org/docs/components/katib/user-guides/metrics-collector/).
 
 <table>
  <tbody>
@@ -113,8 +113,8 @@ The following table shows images for the
 ## Katib Suggestions and Early Stopping Images
 
 The following table shows images for the
-[Katib Suggestions](https://www.kubeflow.org/docs/components/katib/experiment/#search-algorithms-in-detail)
-and the [Katib Early Stopping algorithms](https://www.kubeflow.org/docs/components/katib/early-stopping/).
+[Katib Suggestion services](https://www.kubeflow.org/docs/components/katib/reference/architecture/#suggestion)
+and the [Katib Early Stopping algorithms](https://www.kubeflow.org/docs/components/katib/user-guides/early-stopping/#early-stopping-algorithms).
 
 <table>
  <tbody>
@@ -223,7 +223,7 @@ and the [Katib Early Stopping algorithms](https://www.kubeflow.org/docs/componen
 ## Training Containers Images
 
 The following table shows images for training containers which are used in the
-[Katib Trials](https://www.kubeflow.org/docs/components/katib/experiment/#packaging-your-training-code-in-a-container-image).
+[Katib Trials](https://www.kubeflow.org/docs/components/katib/reference/architecture/#trial).
 
 <table>
  <tbody>

diff --git a/docs/images/SystemFlow.png b/docs/images/SystemFlow.png