Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specs/kiota plugin #4124

Merged
merged 26 commits into from
Mar 26, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6a07587
plugin-add
maisarissi Feb 2, 2024
a67f30f
Edited kiota-config and added plugin-edit
maisarissi Feb 3, 2024
cc189ca
Added plugin-generate
maisarissi Feb 3, 2024
1d30b3b
Added plugin-removed
maisarissi Feb 3, 2024
736a3b3
Updated config-init
maisarissi Feb 3, 2024
b1bdf76
Merge branch 'main' into specs/kiota-plugin
maisarissi Feb 27, 2024
b69a9a3
Updating from "plugin" to "manifest"
maisarissi Feb 27, 2024
1598d1d
Fixing naming
maisarissi Feb 28, 2024
536b60a
Changing naming to manifest
maisarissi Mar 7, 2024
87b73dc
Merge branch 'main' into specs/kiota-plugin
maisarissi Mar 7, 2024
e14785a
Add a new manifest scenario and add telemetry to kiota manifest comma…
maisarissi Mar 7, 2024
dfea0f9
Fixing naming
maisarissi Mar 7, 2024
5d8a182
Updated naming from manifest to plugin
maisarissi Mar 8, 2024
0561707
Merge branch 'main' into specs/kiota-plugin
maisarissi Mar 8, 2024
d9c5e9b
Merge branch 'main' into specs/kiota-plugin
baywet Mar 20, 2024
854c363
- replaces mentions to config by workspace
baywet Mar 20, 2024
5eb2627
replace missing mentions to config by workspace
maisarissi Mar 21, 2024
95bed89
fixing some missing changes from kiota-config to workspace
maisarissi Mar 22, 2024
9beccb2
update specs based on feedback
maisarissi Mar 22, 2024
b72048d
Adding information on required fields
maisarissi Mar 22, 2024
b9f253d
Remove duplicated publisherName info
maisarissi Mar 25, 2024
e8c0fd1
Updated openai map to OpenAPI document fields
maisarissi Mar 26, 2024
a18633a
Removed --searh-key from plugin command
maisarissi Mar 26, 2024
b596a4b
Updated folder structure to ./kiota/documents
maisarissi Mar 26, 2024
95f0adb
Updated --overlay example
maisarissi Mar 26, 2024
b54cb27
removing overlays for now
maisarissi Mar 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion specs/cli/config-init.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
When `kiota config init` is executed, a `kiota-config.json` file would be created in the current directory where the command is being executed. If the user wants to create the file in a different directory, they should use the `--config-file` global parameter.
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

> [!NOTE]
> If a project only needs a single API, using `kiota config init` is not mandatory as generating code using the `kiota client generate` command could generate a `kiota-config.json` file with values coming from the `kiota client generate` command (if no `kiota-config.json` is present). See [kiota client generate](./client-generate.md) for more information.
> If a project only needs a single API, using `kiota config init` is not mandatory as generating code using the `kiota client generate` or `kiota manifest generate` command could generate a `kiota-config.json` file with values coming from the `kiota client generate` or `kiota manifest generate` commands (if no `kiota-config.json` is present). See [kiota client generate](./client-generate.md) or [kiota manifest generate](./manifest-generate.md) for more information.

## Parameters

Expand Down
180 changes: 180 additions & 0 deletions specs/cli/manifest-add.md
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# kiota manifest add
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

## Description

`kiota manifest add` allows a developer to add a new plugin manifest to the `kiota-config.json` file. If no `kiota-config.json` file is found, a new `kiota-config.json` file would be created in the current working directory. The command will add a new entry to the `plugins` section of the `kiota-config.json` file. Once this is done, a local copy of the OpenAPI description is generated and kept in the `.kiota/plugins` folder. If a plugin manifest with the same name already exists, the command will fail and display an actionnable error message.
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

When executing, a new plugin manifest entry will be added and will use the `--plugin-name` parameter as the key for the map. When loading the OpenAPI description, it will store the location of the description in the `descriptionLocation` property. If `--include-path` or `--exclude-path` are provided, they will be stored in the `includePatterns` and `excludePatterns` properties respectively.

Every time a plugin manifest is added, a copy of the OpenAPI description file will be stored in the `./.kiota/plugins` folder. The OpenAPI will be named using the plugin name `{plugin-name}.json|yaml`. This will allow the CLI to detect changes in the description and avoid downloading the description again if it hasn't changed.

An [API Manifest][def] file named `apimanifest-plugins` will be generated (if non existing) or updated (if already existing) in the root folder next to `kiota-config.json`. This file will represent a concatenated surface of all APIs used across plugins. A new hash composed of the Kiota version, the OpenAPI description location and the properties of the plugins will be generated and would trigger an update to the [API Manifest][def].
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

Developers can generate `openai` and `apimanifest` plugin manifests. By generating `openai`, two outputs will be generated: a\) the manifest type you have choosen that will be named `{plugin-name}-{type}.json` and b\) an [API Manifest][def] which will be named `{plugin-name}-apimanifest.json` that will include only the information for this specific plugin. `apimanifest` will generate only an API Manifest.
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
> [!NOTE]
> There will be two different [API Manifests][def]. One in the root folder representing a concatenated surface of all APIs and a second one specific to each plugin saved in the choosen output directory.

Once the `kiota-config.json` file is generated and the OpenAPI description file is saved locally, the generation will be executed and the manifest will become available.

## Parameters

| Parameters | Required | Example | Description |
| -- | -- | -- | -- |
| `--plugin-name \| --pn` | Yes | GitHub | Name of the plugin. Unique within the parent API. Defaults to `PluginManifest` |
| `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI description in JSON or YAML format to use to generate the manifest. Accepts a URL or a local directory. |
| `--include-path \| -i` | No | /repos/\*\* | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. |
| `--exclude-path \| -e` | No | /advisories | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. |
| `--type \| -t` | Yes | openai | The target type of manifest for the generated output files. Possible values are `openai` and `apimanifest`. Defaults to `apimanifest`|
| `--overlayDirectory \| --od` | No | ./overlay/plugins/{plugin-name}/overlay.yaml | The location of the overlay file in JSON or YAML format to be used to generate the plugin manifest. [Overlay](https://github.com/OAI/Overlay-Specification/blob/main/versions/1.0.0.md) defines a way of creating documents that contain additional information to be merged with an OpenAPI description. Defaults to no value which uses the OpenAPI description as it is. |
| `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. |
| `--output \| -o` | No | ./generated/plugins/github | The output directory or file path for the generated output files. This is relative to the location of `kiota-config.json`. Defaults to `./output`. |

> [!NOTE]
> It is not required to use the CLI to add new manifests. It is possible to add a new plugin manifest by adding a new entry in the `plugins` section of the `kiota-config.json` file. See the [kiota-config.json schema](../schemas/kiota-plugin-config.json) for more information. Using `kiota manifest generate --plugin-name myPluginManifest` would be required to generate the manifests.

## Using `kiota manifest add`

```bash
kiota manifest add --plugin-name "GitHub" --openapi "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json" --include-path "/repos/**" --type openai --output "./generated/plugins/github"
```

_The resulting `kiota-config.json` file will look like this:_

```jsonc
{
"version": "1.0.0",
"clients": {...}, //if any
"plugins": {
"GitHub": {
"descriptionLocation": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"includePatterns": ["/repos/**"],
"excludePatterns": [],
"type": "openai",
"outputDirectory": "./generated/plugins/github",
"overlayDirectory": "./overlays/plugins/github/overlay.yaml"
}
}
}
```

_The resulting `github-openai.json` file will look like this:_

```jsonc

```
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

_The resulting `github-apimanifest.json` file will look like this:_

```jsonc
{
"apiDependencies": {
"GitHub": {
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},,
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/cache/usage"
}
]
}
}
}
```

_The resulting `apimanifest.json` file (concatenated surface of all APIs) will look like this:_
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

```jsonc
{
"apiDependencies": {
"GraphPlugin": {
"x-ms-kiotaHash": "1GFCD345RF3DD98...",
"apiDescriptionUrl": "https://aka.ms/graph/v1.0/openapi.yaml",
"apiDeploymentBaseUrl": "https://graph.microsoft.com",
"apiDescriptionVersion": "v1.0",
"requests": [ ... ]
},
"GitHub": {
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},,
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/cache/usage"
}
]
}
}
}
```

## File structure
```bash
└─.kiota
└─plugins
└─GraphPlugin.yaml
└─GitHub.json
└─generated
└─plugins
└─graphplugin
└─graphplugin-apimanifest.json
└─github
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
└─github-openai.json
└─github-apimanifest.json
└─kiota-config.json
└─apimanifest.json
```

[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3
166 changes: 166 additions & 0 deletions specs/cli/manifest-edit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# kiota manifest edit

## Description

`kiota manifest update` allows a developer to edit an existing plugin manifest in the `kiota-config.json` file. If either the `kiota-config.json` file or if the `--plugin-name` plugin can't be found within the `kiota-config.json` file, the command should error out and let the developer know.

When executing, the plugin entry defined by the `--plugin-name` parameter will be modified. All parameters should be supported and the only required one is `--plugin-name`. All others are optional as they would only modify the configuration of the plugin manifest. If the OpenAPI description location changed or any properties of the plugin entry in `kiota-config.json`, a new hash composed of the Kiota version, the OpenAPI description location and the properties of the plugin manifest will be generated and and would trigger an update to the [API Manifest][def] located in the root folder.
> [!NOTE]
> There will be two different [API Manifests][def]. One in the root folder representing a concatenated surface of all APIs and a second one specific to each plugin manifest saved in the choosen output directory.

Once the `kiota-config.json` file and the API Manifest are updated, the code generation will be executed based on the newly updated API client configuration.

## Parameters

| Parameters | Required | Example | Description |
| -- | -- | -- | -- |
| `--plugin-name \| --pn` | Yes | GitHub | Name of the plugin. Unique within the parent API. Defaults to `PluginManifest` |
| `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI description in JSON or YAML format to use to generate the manifest. Accepts a URL or a local directory. |
| `--include-path \| -i` | No | /repos/\*\* | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. |
| `--exclude-path \| -e` | No | /repos/{owner}/{repo}/actions/cache/usage | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. |
| `--type \| -t` | Yes | openai | The target type of manifest for the generated output files. Possible values are `openai` and `apimanifest`. Defaults to `apimanifest`|
| `--overlayDirectory \| --od` | No | ./overlay/plugins/{plugin-name}/overlay.yaml | The location of the overlay file in JSON or YAML format to be used to generate the plugin manifest. [Overlay](https://github.com/OAI/Overlay-Specification/blob/main/versions/1.0.0.md) defines a way of creating documents that contain additional information to be merged with an OpenAPI description. Defaults to no value which uses the OpenAPI description as it is. |
| `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. |
| `--output \| -o` | No | ./generated/plugins/github | The output directory or file path for the generated output files. This is relative to the location of `kiota-config.json`. Defaults to `./output`. |

> [!NOTE]
> It is not required to use the CLI to edit plugin manifests. It is possible to edit a plugins by modifying its entry in the `plugins` section of the `kiota-config.json` file. See the [kiota-config.json schema](../schemas/kiota-plugin-config.json) for more information.

## Using `kiota manifest edit`

```bash
kiota manifest edit --plugin-name "GitHub" --exclude-path "/repos/{owner}/{repo}/actions/cache/usage"
```

_The resulting `kiota-config.json` file will look like this:_

```jsonc
{
"version": "1.0.0",
"clients": {...}, //if any
"plugins": {
"GitHub": {
"descriptionLocation": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"includePatterns": ["/repos/**"],
"excludePatterns": ["/repos/{owner}/{repo}/actions/cache/usage"],
"type": "openai",
"outputDirectory": "./generated/plugins/github",
"overlayDirectory": "./overlays/plugins/github/overlay.yaml"
}
}
}
```

_The resulting `github-openai.json` file will look like this:_

```jsonc

```
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

_The resulting `github-apimanifest.json` file will look like this:_

```jsonc
{
"apiDependencies": {
"GitHub": {
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},,
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
}
]
}
}
}
```

_The resulting `apimanifest.json` file (concatenated surface of all APIs) will look like this:_

```jsonc
{
"apiDependencies": {
"GraphPlugin": {
"x-ms-kiotaHash": "1GFCD345RF3DD98...",
"apiDescriptionUrl": "https://aka.ms/graph/v1.0/openapi.yaml",
"apiDeploymentBaseUrl": "https://graph.microsoft.com",
"apiDescriptionVersion": "v1.0",
"requests": [ ... ]
},
"GitHub": {
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},,
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts"
},
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}/actions/artifacts/{artifact_id}"
}
]
}
}
}
```

## File structure
```bash
└─.kiota
└─plugins
└─GraphPlugin.yaml
└─GitHub.json
└─generated
└─plugins
└─graphplugin
└─graphplugin-apimanifest.json
└─github
└─github-openai.json
└─github-apimanifest.json
└─kiota-config.json
└─apimanifest.json
```

[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3
30 changes: 30 additions & 0 deletions specs/cli/manifest-generate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# kiota manifest generate

## Description

Now that we have a `kiota-config.json` file, all the parameters required to generate the plugin manifests are stored in the file. The `kiota manifest generate` command will read the `kiota-config.json` file and generate the output files for each of the available plugins.

It's also possible to specify for which plugin manifest the output files should be generated. This is useful when there are multiple plugin manifests. The `kiota manifest generate --plugin-name "MyPluginManifest"` command will read the `kiota-config.json` file and generate the code for the specified plugin manifest. If it can't find the specified plugin manifest, it will throw an error.

In general cases, the `kiota manifest generate` command will generate the outfiles for all the plugin manifests in the `kiota-config.json` file based on the cached OpenAPI description. If the `--refresh` parameter is provided, the command will refresh the cached OpenAPI description(s), update the different `x-ms-kiotaHash` in the API Manifests and then generate the output files for the specified plugins.

## Parameters

| Parameters | Required | Example | Description |
| -- | -- | -- | -- |
| `--plugin-name \| --cn` | No | GitHub | Name of the plugin. Unique within the parent API. |
| `--refresh \| -r` | No | true | Provided when refreshing the description(s) is required. |

## Usage

### Using `kiota manifest generate` for a single plugin manifest

```bash
kiota manifest generate --plugin-name "GitHub"
```

### Using `kiota manifest generate` for all plugin manifests

```bash
kiota manifest generate
```
Loading
Loading