Metastore Summary

japila-books · Oct 17, 2024 · c074790 · c074790
1 parent dc0f01a
commit c074790
Show file tree

Hide file tree

Showing 8 changed files with 115 additions and 7 deletions.
diff --git a/docs/cli/MetastoreCli.md b/docs/cli/MetastoreCli.md
@@ -0,0 +1,32 @@
+# MetastoreCli
+
+`MetastoreCli` is used by [UnityCatalogCli](UnityCatalogCli.md) to [handle `metastore` sub-commands](#handle).
+
+``` console
+❯ ./bin/uc metastore --help
+Please provide a valid sub-command for metastore.
+Valid sub-commands for metastore are: get
+For detailed help on metastore sub-commands, use bin/uc metastore <sub-command> --help
+```
+
+## Handle Command Line { #handle }
+
+``` java
+void handle(
+  CommandLine cmd,
+  ApiClient apiClient)
+```
+
+`handle` handles the given `cmd` that can currently by `get` only.
+
+`handle` creates a `MetastoresApi` (with the given [ApiClient](../client/ApiClient.md)).
+
+Subcommand | Handler | API Handlers
+-|-|-
+ `get` | &nbsp; | `MetastoresApi`
+
+---
+
+`handle` is used when:
+
+* `UnityCatalogCli` is [launched on command line](UnityCatalogCli.md#main) with `metastore` command
diff --git a/docs/cli/UnityCatalogCli.md b/docs/cli/UnityCatalogCli.md
@@ -27,6 +27,7 @@ Command | Handler
  `auth` | [AuthCli](AuthCli.md)
  `catalog` | [CatalogCli](CatalogCli.md)
  `function` | [FunctionCli](FunctionCli.md)
+ `metastore` | [MetastoreCli](MetastoreCli.md)
  `model_version` | [ModelVersionCli](ModelVersionCli.md)
  `permission` | [PermissionCli](PermissionCli.md)
  `registered_model` | [ModelCli](ModelCli.md)

diff --git a/docs/cli/index.md b/docs/cli/index.md
@@ -3,10 +3,10 @@
 **Unity Catalog CLI** can be started using `./bin/uc` command-line utility that runs [UnityCatalogCli](UnityCatalogCli.md).
 
 ```text
-$ ./bin/uc --help
+❯ ./bin/uc --help
 
 Usage: bin/uc <entity> <operation> [options]
-Entities: schema, volume, model_version, auth, catalog, function, registered_model, user, table
+Entities: schema, volume, model_version, metastore, auth, catalog, function, permission, registered_model, user, table
 
 By default, the client will connect to UC running locally at http://localhost:8080
 
@@ -24,6 +24,7 @@ Unity Catalog CLI supports the following entities (commands):
 * [auth](AuthCli.md)
 * [catalog](CatalogCli.md)
 * [function](FunctionCli.md)
+* [metastore](MetastoreCli.md)
 * [model_version](ModelVersionCli.md)
 * [registered_model](ModelCli.md)
 * [schema](SchemaCli.md)

diff --git a/docs/features/index.md b/docs/features/index.md
@@ -66,7 +66,7 @@ OpenJDK 64-Bit Server VM Zulu17.52+17-CA (build 17.0.12+7-LTS, mixed mode, shari
 Please provide a entity.
 
 Usage: bin/uc <entity> <operation> [options]
-Entities: schema, volume, model_version, auth, catalog, function, permission, registered_model, user, table
+Entities: schema, volume, model_version, metastore, auth, catalog, function, permission, registered_model, user, table
 
 By default, the client will connect to UC running locally at http://localhost:8080
 

diff --git a/docs/server/MetastoreService.md b/docs/server/MetastoreService.md
@@ -0,0 +1,26 @@
+# MetastoreService
+
+`MetastoreService` is a Unity Catalog API service that [UnityCatalogServer](UnityCatalogServer.md) registers to handle HTTP requests to the base `/api/2.1/unity-catalog/` URL.
+
+Method | URL | Handler | Params
+-|-|-|-
+ GET | `/metastore_summary` | [getMetastoreSummary](#getMetastoreSummary) | &nbsp;
+
+```console
+$ http http://localhost:8081/api/2.1/unity-catalog/metastore_summary | jq '.metastore_id'
+"92418bf6-e62c-4d4c-bb5d-e5eda5c82688"
+```
+
+## MetastoreRepository { #METASTORE_REPOSITORY }
+
+`MetastoreService` requests the system-wide [MetastoreRepository](../persistent-storage/MetastoreRepository.md) when created.
+
+This `MetastoreRepository` is used to [getMetastoreSummary](../persistent-storage/MetastoreRepository.md#getMetastoreSummary) to handle `GET /metastore_summary` requests.
+
+## Get Metastore Summary { #getMetastoreSummary }
+
+```java
+HttpResponse getMetastoreSummary()
+```
+
+`getMetastoreSummary` requests the system-wide [MetastoreRepository](#METASTORE_REPOSITORY) instance for the [metastore summary](../persistent-storage/MetastoreRepository.md#getMetastoreSummary).
diff --git a/docs/server/URLTranscoderVerticle.md b/docs/server/URLTranscoderVerticle.md
@@ -0,0 +1,3 @@
+# URLTranscoderVerticle
+
+`URLTranscoderVerticle` is...FIXME
diff --git a/docs/server/UnityCatalogServer.md b/docs/server/UnityCatalogServer.md
@@ -2,18 +2,28 @@
 title: UnityCatalogServer
 ---
 
-# UnityCatalogServer &mdash; Unity Catalog's Localhost Reference Server
+# UnityCatalogServer &mdash; Localhost Reference Server
 
 `UnityCatalogServer` is Unity Catalog's **Localhost Reference Server**.
 
 `UnityCatalogServer` can be started on command line to [handle REST API requests](#addServices) at the [port](#port).
 
 ```console
-./bin/start-uc-server [-p|--port port]
+❯ ./bin/start-uc-server --help
+usage: bin/start-uc-server
+ -h,--help         Print help message.
+ -p,--port <arg>   Port number to run the server on. Default is 8080.
+ -v,--version      Display the version of the Unity Catalog server
 ```
 
 `UnityCatalogServer` starts Armeria documentation service at http://localhost:8081/docs and the other [Unity Catalog API services](#addServices).
 
+## Metastore
+
+`UnityCatalogServer` [runs](#start) with a single metastore only that [can be created unless available already](../persistent-storage/MetastoreRepository.md#initMetastoreIfNeeded).
+
+The summary of the single metastore is available through [MetastoreService](MetastoreService.md) at `/api/2.1/unity-catalog/` URL.
+
 ## Configuration Files
 
 `UnityCatalogServer` uses the following configuration files:
@@ -77,6 +87,7 @@ URL | Service
  `/` | Returns `Hello, Unity Catalog!` message
  `/api/1.0/unity-control/auth` |  [AuthService](AuthService.md)
  `/api/1.0/unity-control/scim2/Users` |  [Scim2UserService](Scim2UserService.md)
+ `/api/2.1/unity-catalog/` | [MetastoreService](MetastoreService.md)
  `/api/2.1/unity-catalog/catalogs` | [CatalogService](CatalogService.md)
  `/api/2.1/unity-catalog/functions` | [FunctionService](FunctionService.md)
  `/api/2.1/unity-catalog/iceberg` | [IcebergRestCatalogService](../iceberg/IcebergRestCatalogService.md)
@@ -118,7 +129,38 @@ void main(
 
 `main` requests the `UnityCatalogServer` to [start](#start).
 
-`main`...FIXME
+`main` creates a [URLTranscoderVerticle](URLTranscoderVerticle.md) on the following ports:
+
+Port | Description
+-|-
+8080 or `--port` | Transcode Port
+8081 | Service Port
+
+`main` deploys the `URLTranscoderVerticle` on a non-clustered Vert.x instance.
+
+## Start Server { #start }
+
+```java
+void start()
+```
+
+`start` prints out the following INFO message to the logs:
+
+```text
+Starting server...
+```
+
+`start` requests the [MetastoreRepository](../persistent-storage/MetastoreRepository.md) to [initMetastoreIfNeeded](../persistent-storage/MetastoreRepository.md#initMetastoreIfNeeded).
+
+`start` requests this [Server](#server) to start and listen to the defined ports.
+
+`start` waits until this `Server` is fully started up.
+
+---
+
+`start` is used when:
+
+* `UnityCatalogServer` is [launched on command line](#main)
 
 ## Logging
 

diff --git a/docs/server/index.md b/docs/server/index.md
@@ -1,6 +1,6 @@
 # Localhost Reference Server
 
-Unity Catalog comes with **Localhost Reference Server** that can be launched on command line using `./bin/start-uc-server` utility.
+Unity Catalog comes with [Localhost Reference Server](UnityCatalogServer.md) that can be launched on command line using `./bin/start-uc-server` utility.
 
 ``` console
 ❯ ./bin/start-uc-server -h
@@ -11,3 +11,6 @@ usage: bin/start-uc-server
 ```
 
 `start-uc-server` runs [UnityCatalogServer](UnityCatalogServer.md) to [handle REST API requests](UnityCatalogServer.md#addServices) at a [port](UnityCatalogServer.md#port) (specified using `-p` or `--port` option or defaults to `8080`).
+
+!!! note
+    The REST API requests can come from any Unity Catalog API-compatible client, incl. [Unity Catalog CLI](../cli/index.md).