Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specs/kiota plugin #4124

Merged
merged 26 commits into from
Mar 26, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6a07587
plugin-add
maisarissi Feb 2, 2024
a67f30f
Edited kiota-config and added plugin-edit
maisarissi Feb 3, 2024
cc189ca
Added plugin-generate
maisarissi Feb 3, 2024
1d30b3b
Added plugin-removed
maisarissi Feb 3, 2024
736a3b3
Updated config-init
maisarissi Feb 3, 2024
b1bdf76
Merge branch 'main' into specs/kiota-plugin
maisarissi Feb 27, 2024
b69a9a3
Updating from "plugin" to "manifest"
maisarissi Feb 27, 2024
1598d1d
Fixing naming
maisarissi Feb 28, 2024
536b60a
Changing naming to manifest
maisarissi Mar 7, 2024
87b73dc
Merge branch 'main' into specs/kiota-plugin
maisarissi Mar 7, 2024
e14785a
Add a new manifest scenario and add telemetry to kiota manifest comma…
maisarissi Mar 7, 2024
dfea0f9
Fixing naming
maisarissi Mar 7, 2024
5d8a182
Updated naming from manifest to plugin
maisarissi Mar 8, 2024
0561707
Merge branch 'main' into specs/kiota-plugin
maisarissi Mar 8, 2024
d9c5e9b
Merge branch 'main' into specs/kiota-plugin
baywet Mar 20, 2024
854c363
- replaces mentions to config by workspace
baywet Mar 20, 2024
5eb2627
replace missing mentions to config by workspace
maisarissi Mar 21, 2024
95bed89
fixing some missing changes from kiota-config to workspace
maisarissi Mar 22, 2024
9beccb2
update specs based on feedback
maisarissi Mar 22, 2024
b72048d
Adding information on required fields
maisarissi Mar 22, 2024
b9f253d
Remove duplicated publisherName info
maisarissi Mar 25, 2024
e8c0fd1
Updated openai map to OpenAPI document fields
maisarissi Mar 26, 2024
a18633a
Removed --searh-key from plugin command
maisarissi Mar 26, 2024
b596a4b
Updated folder structure to ./kiota/documents
maisarissi Mar 26, 2024
95f0adb
Updated --overlay example
maisarissi Mar 26, 2024
b54cb27
removing overlays for now
maisarissi Mar 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion specs/cli/client-add.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ When executing, a new API entry will be added and will use the `--client-name` p

Every time an API client is added, a copy of the OpenAPI description file will be stored in the `./.kiota/clients/{client-name}.yaml|json` folder. The files will be named using the API client name. This will allow the CLI to detect changes in the description and avoid downloading the description again if it hasn't changed.

At the same time, an [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3) file will be generated (if non existing) or edited (if already existing) to represent the surface of the API being used. This file will be named `apimanifest.json` and next to the `kiota-config.json`. This file will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI description location and the properties of the client will be generated and would trigger an update to the [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3).
At the same time, an [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3) file will be generated (if non existing) or edited (if already existing) to represent the surface of the API being used. This file will be named `apimanifest.json` and next to the `kiota-config.json`. This file will represent a concatenated surface of all APIs used across manifests and clients. Both files, `apimanifest.json` and `kiota-config.json` will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI description location and the properties of the client will be generated and would trigger an update to the [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3).

maisarissi marked this conversation as resolved.
Show resolved Hide resolved
Once the `kiota-config.json` file is generated and the OpenAPI description file is saved locally, the code generation will be executed and then the API Manifest would become available.
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
2 changes: 1 addition & 1 deletion specs/cli/config-init.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
When `kiota config init` is executed, a `kiota-config.json` file would be created in the current directory where the command is being executed. If the user wants to create the file in a different directory, they should use the `--config-file` global parameter.
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

> [!NOTE]
> If a project only needs a single API, using `kiota config init` is not mandatory as generating code using the `kiota client generate` command could generate a `kiota-config.json` file with values coming from the `kiota client generate` command (if no `kiota-config.json` is present). See [kiota client generate](./client-generate.md) for more information.
> If a project only needs a single API, using `kiota config init` is not mandatory as generating code using the `kiota client generate` or `kiota manifest generate` command could generate a `kiota-config.json` file with values coming from the `kiota client generate` or `kiota manifest generate` commands (if no `kiota-config.json` is present). See [kiota client generate](./client-generate.md) or [kiota manifest generate](./manifest-generate.md) for more information.

## Parameters

Expand Down
163 changes: 163 additions & 0 deletions specs/cli/manifest-add.md
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# kiota manifest add
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

## Description

`kiota manifest add` allows a developer to add a new manifest to the `kiota-config.json` file. If no `kiota-config.json` file is found, a new `kiota-config.json` file would be created in the current working directory. The command will add a new entry to the `manifests` section of the `kiota-config.json` file. Once this is done, a local copy of the OpenAPI description is generated and kept in the `.kiota/manifests` folder. If a manifest with the same name already exists, the command will fail and display an actionnable error message.

When executing, a new manifest entry will be added and will use the `--manifest-name` parameter as the key for the map. When loading the OpenAPI description, it will store the location of the description in the `descriptionLocation` property. If `--include-path` or `--exclude-path` are provided, they will be stored in the `includePatterns` and `excludePatterns` properties respectively.

Every time a manifest is added, a copy of the OpenAPI description file will be stored in the `./.kiota/manifests` folder. The OpenAPI will be named using the manifest name `{manifest-name}.json|yaml`. This will allow the CLI to detect changes in the description and avoid downloading the description again if it hasn't changed.

An [API Manifest][def] file named `apimanifest.json` will be generated (if non existing) or updated (if already existing) in the root folder next to `kiota-config.json`. This file will represent a concatenated surface of all APIs used across manifests and clients. Both files, `apimanifest.json` and `kiota-config.json` will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI description location and the properties of the manifest will be generated and would trigger an update to the [API Manifest][def].
baywet marked this conversation as resolved.
Show resolved Hide resolved

Developers can generate `openai` and `apimanifest` type of manifests. By generating `openai` or `apimanifest`, two outputs will be generated: a\) the manifest type you have choosen that will be named `{manifest-name}-{type}.json` and b\) a sliced OpenAPI description with only the endpoints that matches `--include-path` and `--exclude-path`, if provided.
> [!NOTE]
> In one's solution, there might be two different [API Manifests][def]. The `apimanifest.json` in the root folder represents a single artifact surface of all APIs and it will always be generated. The second one, specific to each manifest, will be named `{manifest-name}-apimanifest.json` and saved in the choosen output directory when `apimanifest` value is used as the manifest type.

Once the `kiota-config.json` file is generated and the OpenAPI description file is saved locally, the generation will be executed and the manifest and the sliced OpenAPI description will become available.

## Parameters

| Parameters | Required | Example | Description | Telemetry |
| -- | -- | -- | -- | -- |
| `--manifest-name \| --mn` | Yes | GitHub | Name of the manifest. Unique within the parent API. Defaults to `Manifest` | No |
| `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI description in JSON or YAML format to use to generate the manifest. Accepts a URL or a local directory. | No |
| `--search-key \| --sk` | No | apisguru::github.com:api.github.com | The search key used to locate the OpenAPI description. | Yes, without its value |
| `--include-path \| -i` | No | /repos/{owner}/{repo} | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. | Yes, without its value |
| `--exclude-path \| -e` | No | /advisories | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. | Yes, without its value |
| `--type \| -t` | Yes | openai | The target type of manifest for the generated output files. Accepts multiple values. Possible values are `openai` and `apimanifest`. Defaults to `apimanifest`| Yes |
| `--overlayDirectory \| --od` | No | ./overlay/manifest/{manifest-name}/overlay.yaml | The location of the overlay file in JSON or YAML format to be used to generate the manifest. [Overlay](https://github.com/OAI/Overlay-Specification/blob/main/versions/1.0.0.md) defines a way of creating documents that contain additional information to be merged with an OpenAPI description. Defaults to no value which uses the OpenAPI description as it is. | Yes, without its value |
| `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. | Yes |
| `--output \| -o` | No | ./generated/manifest/github | The output directory or file path for the generated output files. This is relative to the location of `kiota-config.json`. Defaults to `./output`. | Yes, without its value |

> [!NOTE]
> It is not required to use the CLI to add new manifests. It is possible to add a new manifest by adding a new entry in the `manifests` section of the `kiota-config.json` file. See the [kiota-config.json schema](../schemas/kiota-config.json) for more information. Using `kiota manifest generate --manifest-name myManifest` would be required to generate the manifests.

## Using `kiota manifest add`

```bash
kiota manifest add --manifest-name "GitHub" --openapi "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json" --include-path "/repos/{owner}/{repo}" --type openai, apimanifest --output "./generated/manifests/github"
```

_The resulting `kiota-config.json` file will look like this:_

```jsonc
{
"version": "1.0.0",
"clients": {...}, //if any
"manifests": {
"GitHub": {
"descriptionLocation": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"includePatterns": ["/repos/{owner}/{repo}"],
"excludePatterns": [],
"type": "openai",
"outputDirectory": "./generated/manifests/github",
"overlayDirectory": "./overlays/manifests/github/overlay.yaml"
}
}
}
```

_The resulting `github-openai.json` file will look like this:_

```jsonc
{
"schema_version": "v1",
"name_for_human": "GitHub Repository Plugin",
"name_for_model": "github",
"description_for_human": "Manage GitHub repositories.",
"description_for_model": "Help the user with managing a GitHub repositories. You can view, update and remove repositories.",
"auth": {
"type": "none"
},
"api": {
"type": "openapi",
"url": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json"
},
"logo_url": "https://example.com/logo.png",
"contact_email": "[email protected]",
"legal_info_url": "http://www.example.com/view-plugin-information"
}
```

_The resulting `github-apimanifest.json` file will look like this:_

```jsonc
{
"apiDependencies": {
"GitHub": {
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
}
]
}
}
}
```

_The resulting `apimanifest.json` file (concatenated surface of all APIs) will look like this:_
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

```jsonc
{
"apiDependencies": {
"GraphClient": { //for example, an existing API client for Microsoft Graph
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://aka.ms/graph/v1.0/openapi.yaml",
"apiDeploymentBaseUrl": "https://graph.microsoft.com",
"apiDescriptionVersion": "v1.0",
"requests": [ ...]
},
"GitHub": { //new manifest added
"x-ms-kiotaHash": "1GFCD345RF3DD98...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
}
]
}
}
}
```

## File structure
```bash
└─.kiota
└─manifests
└─GitHub.json # OpenAPI description
└─generated
└─manifests
└─github
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
└─github-apimanifest.json # Specific apimanifest
└─github-openai.json #OpenAI manifest
└─sliced-openapi-github.json # Sliced OpenAPI description
└─kiota-config.json
└─apimanifest.json # Single artifact with all APIs info
```

[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html
148 changes: 148 additions & 0 deletions specs/cli/manifest-edit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# kiota manifest edit

## Description

`kiota manifest update` allows a developer to edit an existing manifest in the `kiota-config.json` file. If either the `kiota-config.json` file or if the `--manifest-name` manifest can't be found within the `kiota-config.json` file, the command should error out and let the developer know.

When executing, the manifest entry defined by the `--manifest-name` parameter will be modified. All parameters should be supported and the only required one is `--manifest-name`. All others are optional as they would only modify the configuration of the manifest. If the OpenAPI description location changed or any properties of the manifest entry in `kiota-config.json`, a new hash composed of the Kiota version, the OpenAPI description location and the properties of the manifest will be generated and and would trigger an update to the [API Manifest][def] located in the root folder (the).
> [!NOTE]
> > In one's solution, there might be two different [API Manifests][def]. The `apimanifest.json` in the root folder represents a single artifact surface of all APIs and it will always be generated. The second one, specific to each manifest, will be named `{manifest-name}-apimanifest.json` and saved in the choosen output directory when `apimanifest` value is used as the manifest type.

Once the `kiota-config.json` file and the API Manifest are updated, the code generation will be executed based on the newly updated API client configuration.

## Parameters

| Parameters | Required | Example | Description | Telemetry |
| -- | -- | -- | -- | -- |
| `--manifest-name \| --mn` | Yes | GitHub | Name of the manifest. Unique within the parent API. Defaults to `Manifest` | No |
| `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI description in JSON or YAML format to use to generate the manifest. Accepts a URL or a local directory. | Yes, without its value |
| `--include-path \| -i` | No | /repos/{owner}/{repo} | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. | Yes, without its value |
| `--exclude-path \| -e` | No | /repos/{owner}/{repo}#DELETE | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. | Yes, without its value |
| `--type \| -t` | Yes | openai | The target type of manifest for the generated output files. Accepts multiple values. Possible values are `openai` and `apimanifest`. Defaults to `apimanifest`| Yes |
| `--overlayDirectory \| --od` | No | ./overlay/manifests/{manifest-name}/overlay.yaml | The location of the overlay file in JSON or YAML format to be used to generate the manifest. [Overlay](https://github.com/OAI/Overlay-Specification/blob/main/versions/1.0.0.md) defines a way of creating documents that contain additional information to be merged with an OpenAPI description. Defaults to no value which uses the OpenAPI description as it is. | Yes, without its value |
| `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. | Yes |
| `--output \| -o` | No | ./generated/manifests/github | The output directory or file path for the generated output files. This is relative to the location of `kiota-config.json`. Defaults to `./output`. | Yes, without its value |

> [!NOTE]
> It is not required to use the CLI to edit manifests. It is possible to edit a manifest by modifying its entry in the `manifests` section of the `kiota-config.json` file. See the [kiota-config.json schema](../schemas/kiota-config.json) for more information.

## Using `kiota manifest edit`

```bash
kiota manifest edit --manifest-name "GitHub" --exclude-path "/repos/{owner}/{repo}#DELETE"
```

_The resulting `kiota-config.json` file will look like this:_

```jsonc
{
"version": "1.0.0",
"clients": {...}, //if any
"manifests": {
"GitHub": {
"descriptionLocation": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"includePatterns": ["/repos/{owner}/{repo}"],
"excludePatterns": ["/repos/{owner}/{repo}#DELETE"],
"type": "openai",
"outputDirectory": "./generated/manifests/github",
"overlayDirectory": "./overlays/manifests/github/overlay.yaml"
}
}
}
```

_The resulting `github-openai.json` file will look like this:_

```jsonc
{
"schema_version": "v1",
"name_for_human": "GitHub Repository Plugin",
"name_for_model": "github",
"description_for_human": "Manage GitHub repositories.",
"description_for_model": "Help the user with managing a GitHub repositories. You can view and update repositories.",
"auth": {
"type": "none"
},
"api": {
"type": "openapi",
"url": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json"
},
"logo_url": "https://example.com/logo.png",
"contact_email": "[email protected]",
"legal_info_url": "http://www.example.com/view-plugin-information"
}
```

_The resulting `github-apimanifest.json` file will look like this:_

```jsonc
{
"apiDependencies": {
"GitHub": {
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
}
]
}
}
}
```

_The resulting `apimanifest.json` file (concatenated surface of all APIs) will look like this:_

```jsonc
{
"apiDependencies": {
"GraphClient": { //for example, an existing API client for Microsoft Graph
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://aka.ms/graph/v1.0/openapi.yaml",
"apiDeploymentBaseUrl": "https://graph.microsoft.com",
"apiDescriptionVersion": "v1.0",
"requests": [ ...]
},
"GitHub": {
"x-ms-kiotaHash": "1GFCD345RF3DD98...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
}
]
}
}
}
```

## File structure
```bash
└─.kiota
└─manifests
└─GitHub.json # OpenAPI description
└─generated
└─manifests
└─github
└─github-apimanifest.json # Specific apimanifest
└─github-openai.json #OpenAI manifest
└─sliced-openapi-github.json # Sliced OpenAPI description
└─kiota-config.json
└─apimanifest.json
```

[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html
Loading
Loading