Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specs/kiota plugin #4124

Merged
merged 26 commits into from
Mar 26, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6a07587
plugin-add
maisarissi Feb 2, 2024
a67f30f
Edited kiota-config and added plugin-edit
maisarissi Feb 3, 2024
cc189ca
Added plugin-generate
maisarissi Feb 3, 2024
1d30b3b
Added plugin-removed
maisarissi Feb 3, 2024
736a3b3
Updated config-init
maisarissi Feb 3, 2024
b1bdf76
Merge branch 'main' into specs/kiota-plugin
maisarissi Feb 27, 2024
b69a9a3
Updating from "plugin" to "manifest"
maisarissi Feb 27, 2024
1598d1d
Fixing naming
maisarissi Feb 28, 2024
536b60a
Changing naming to manifest
maisarissi Mar 7, 2024
87b73dc
Merge branch 'main' into specs/kiota-plugin
maisarissi Mar 7, 2024
e14785a
Add a new manifest scenario and add telemetry to kiota manifest comma…
maisarissi Mar 7, 2024
dfea0f9
Fixing naming
maisarissi Mar 7, 2024
5d8a182
Updated naming from manifest to plugin
maisarissi Mar 8, 2024
0561707
Merge branch 'main' into specs/kiota-plugin
maisarissi Mar 8, 2024
d9c5e9b
Merge branch 'main' into specs/kiota-plugin
baywet Mar 20, 2024
854c363
- replaces mentions to config by workspace
baywet Mar 20, 2024
5eb2627
replace missing mentions to config by workspace
maisarissi Mar 21, 2024
95bed89
fixing some missing changes from kiota-config to workspace
maisarissi Mar 22, 2024
9beccb2
update specs based on feedback
maisarissi Mar 22, 2024
b72048d
Adding information on required fields
maisarissi Mar 22, 2024
b9f253d
Remove duplicated publisherName info
maisarissi Mar 25, 2024
e8c0fd1
Updated openai map to OpenAPI document fields
maisarissi Mar 26, 2024
a18633a
Removed --searh-key from plugin command
maisarissi Mar 26, 2024
b596a4b
Updated folder structure to ./kiota/documents
maisarissi Mar 26, 2024
95f0adb
Updated --overlay example
maisarissi Mar 26, 2024
b54cb27
removing overlays for now
maisarissi Mar 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion specs/cli/client-add.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ When executing, a new API entry will be added and will use the `--client-name` p

Every time an API client is added, a copy of the OpenAPI description file will be stored in the `./.kiota/clients/{client-name}.yaml|json` folder. The files will be named using the API client name. This will allow the CLI to detect changes in the description and avoid downloading the description again if it hasn't changed.

At the same time, an [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3) file will be generated (if non existing) or edited (if already existing) to represent the surface of the API being used. This file will be named `apimanifest.json` and next to the `kiota-config.json`. This file will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI description location and the properties of the client will be generated and would trigger an update to the [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3).
At the same time, an [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html) file will be generated (if non existing) or edited (if already existing) in the root folder next to `kiota-config.json`. API Manifest represents a snapshot of API dependencies and permissions required to access those APIs. This file will represent a concatenated surface of all APIs used across plugins and clients. Both files, `apimanifest.json` and `kiota-config.json` will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI description location and the properties of the manifest will be generated and would trigger an update to the [API Manifest][https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html].
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

maisarissi marked this conversation as resolved.
Show resolved Hide resolved
Once the `kiota-config.json` file is generated and the OpenAPI description file is saved locally, the code generation will be executed and then the API Manifest would become available.
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
2 changes: 1 addition & 1 deletion specs/cli/client-edit.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

`kiota client update` allows a developer to edit an existing API client int the `kiota-config.json` file. If either the `kiota-config.json` file or if the `--client-name` client can't be found within the `kiota-config.json` file, the command should error out and let the developer know.

When executing, the API entry defined by the `--client-name` parameter will be modified. All parameters should be supported and the only required one is `--client-name`. All others are optional as they would only modify the configuration of the API client. If the OpenAPI description location changed or any properties of the client entry in `kiota-config.json`, a new hash composed of the Kiota version, the OpenAPI description location and the properties of the client will be generated and and would trigger an update to the [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html#section-2.5-3).
When executing, the API entry defined by the `--client-name` parameter will be modified. All parameters should be supported and the only required one is `--client-name`. All others are optional as they would only modify the configuration of the API client. If the OpenAPI description location changed or any properties of the client entry in `kiota-config.json`, a new hash composed of the Kiota version, the OpenAPI description location and the properties of the client will be generated and and would trigger an update to the [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html).

Once the `kiota-config.json` file and the API Manifest are updated, the code generation will be executed based on the newly updated API client configuration.

Expand Down
2 changes: 1 addition & 1 deletion specs/cli/config-init.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
When `kiota config init` is executed, a `kiota-config.json` file would be created in the current directory where the command is being executed. If the user wants to create the file in a different directory, they should use the `--config-file` global parameter.
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

> [!NOTE]
> If a project only needs a single API, using `kiota config init` is not mandatory as generating code using the `kiota client generate` command could generate a `kiota-config.json` file with values coming from the `kiota client generate` command (if no `kiota-config.json` is present). See [kiota client generate](./client-generate.md) for more information.
> If a project only needs a single API, using `kiota config init` is not mandatory as generating code using the `kiota client generate` or `kiota plugin generate` command could generate a `kiota-config.json` file with values coming from the `kiota client generate` or `kiota plugin generate` commands (if no `kiota-config.json` is present). See [kiota client generate](./client-generate.md) or [kiota plugin generate](./plugin-generate.md) for more information.

## Parameters

Expand Down
163 changes: 163 additions & 0 deletions specs/cli/plugin-add.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# kiota plugin add

## Description

`kiota plugin add` allows a developer to add a new plugin to the `kiota-config.json` file. If no `kiota-config.json` file is found, a new `kiota-config.json` file would be created in the current working directory. The command will add a new entry to the `plugins` section of the `kiota-config.json` file. Once this is done, a local copy of the OpenAPI description is generated and kept in the `.kiota/plugins` folder. If a plugin with the same name already exists, the command will fail and display an actionnable error message.

When executing, a new plugin entry will be added and will use the `--plugin-name` parameter as the key for the map. When loading the OpenAPI description, it will store the location of the description in the `descriptionLocation` property. If `--include-path` or `--exclude-path` are provided, they will be stored in the `includePatterns` and `excludePatterns` properties respectively.

Every time a plugin is added, a copy of the OpenAPI description file will be stored in the `./.kiota/plugins` folder. The OpenAPI will be named using the plugin name `{plugin-name}.json|yaml`. This will allow the CLI to detect changes in the description and avoid downloading the description again if it hasn't changed.
baywet marked this conversation as resolved.
Show resolved Hide resolved
baywet marked this conversation as resolved.
Show resolved Hide resolved

An [API Manifest][def] file named `apimanifest.json` will be generated (if non existing) or updated (if already existing) in the root folder `./kiota` next to `kiota-config.json`. API Manifest represents a snapshot of API dependencies and permissions required to access those APIs. This file will represent a concatenated surface of all APIs used across plugins and clients. Both files, `apimanifest.json` and `kiota-config.json` will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI description location and the properties of the manifest will be generated and would trigger an update to the [API Manifest][def].

Developers can generate `openai` and `apimanifest` type of plugins. By generating `openai` or `apimanifest`, two outputs will be generated: a\) the plugin type you have choosen that will be named `{plugin-name}-{type}.json` and b\) a sliced OpenAPI description with only the endpoints that matches `--include-path` and `--exclude-path`, if provided.
> [!NOTE]
> In one's solution, there might be two different [API Manifests][def]. The `apimanifest.json` in the root folder represents a single artifact surface of all APIs and it will always be generated. The second one, specific to each plugin, will be named `{plugin-name}-apimanifest.json` and saved in the choosen output directory when `apimanifest` value is used as the plugin type.

Once the `kiota-config.json` file is generated and the OpenAPI description file is saved locally, the generation will be executed and the plugin and the sliced OpenAPI description will become available.

## Parameters

| Parameters | Required | Example | Description | Telemetry |
| -- | -- | -- | -- | -- |
| `--plugin-name \| --mn` | Yes | GitHub | Name of the plugin. Unique within the parent API. Defaults to `Plugin` | No |
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
| `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI description in JSON or YAML format to use to generate the plugin. Accepts a URL or a local directory. | No |
| `--search-key \| --sk` | No | apisguru::github.com:api.github.com | The search key used to locate the OpenAPI description. | Yes, without its value |
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
| `--include-path \| -i` | No | /repos/{owner}/{repo} | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. | Yes, without its value |
| `--exclude-path \| -e` | No | /advisories | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. | Yes, without its value |
| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `openai` and `apimanifest`. Defaults to `apimanifest`| Yes |
baywet marked this conversation as resolved.
Show resolved Hide resolved
| `--overlayDirectory \| --od` | No | ./overlays/plugins/{plugin-name}/overlay.yaml | The location of the overlay file in JSON or YAML format to be used to generate the plugin. [Overlay](https://github.com/OAI/Overlay-Specification/blob/main/versions/1.0.0.md) defines a way of creating documents that contain additional information to be merged with an OpenAPI description. Defaults to no value which uses the OpenAPI description as it is. | Yes, without its value |
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
| `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. | Yes |
| `--output \| -o` | No | ./generated/plugins/github | The output directory or file path for the generated output files. This is relative to the location of `kiota-config.json`. Defaults to `./output`. | Yes, without its value |

> [!NOTE]
> It is not required to use the CLI to add new plugins. It is possible to add a new plugin by adding a new entry in the `plugins` section of the `kiota-config.json` file. See the [kiota-config.json schema](../schemas/kiota-config.json) for more information. Using `kiota plugin generate --plugin-name myPlugin` would be required to generate the plugins.

## Using `kiota plugin add`

```bash
kiota plugin add --plugins-name "GitHub" --openapi "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json" --include-path "/repos/{owner}/{repo}" --type openai, apimanifest --output "./generated/plugins/github"
```

_The resulting `kiota-config.json` file will look like this:_

```jsonc
{
"version": "1.0.0",
"clients": {...}, //if any
"plugins": {
"GitHub": {
"descriptionLocation": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"includePatterns": ["/repos/{owner}/{repo}"],
"excludePatterns": [],
"type": "openai",
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
"outputDirectory": "./generated/plugins/github",
"overlayDirectory": "./overlays/plugins/github/overlay.yaml"
}
}
}
```

_The resulting `github-openai.json` file will look like this:_

```jsonc
{
"schema_version": "v1",
"name_for_human": "GitHub Repository Plugin",
"name_for_model": "github",
"description_for_human": "Manage GitHub repositories.",
"description_for_model": "Help the user with managing a GitHub repositories. You can view, update and remove repositories.",
"auth": {
"type": "none"
},
"api": {
"type": "openapi",
"url": "./sliced-openapi-github.json"
},
"logo_url": "https://example.com/logo.png",
"contact_email": "[email protected]",
"legal_info_url": "http://www.example.com/view-plugin-information"
maisarissi marked this conversation as resolved.
Show resolved Hide resolved
}
```

_The resulting `github-apimanifest.json` file will look like this:_

```jsonc
{
"apiDependencies": {
"GitHub": {
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
}
]
}
}
}
```

_The resulting `apimanifest.json` file (concatenated surface of all APIs dependencies) will look like this:_

```jsonc
{
"apiDependencies": {
"GraphClient": { //for example, an existing API client for Microsoft Graph
"x-ms-kiotaHash": "9EDF8506CB74FE44...",
"apiDescriptionUrl": "https://aka.ms/graph/v1.0/openapi.yaml",
"apiDeploymentBaseUrl": "https://graph.microsoft.com",
"apiDescriptionVersion": "v1.0",
"requests": [ ...]
},
"GitHub": { //new plugin added
"x-ms-kiotaHash": "1GFCD345RF3DD98...",
"apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"apiDeploymentBaseUrl": "https://api.github.com/",
"apiDescriptionVersion": "v1.0",
"requests": [
{
"method": "GET",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "PATCH",
"uriTemplate": "/repos/{owner}/{repo}"
},
{
"method": "DELETE",
"uriTemplate": "/repos/{owner}/{repo}"
}
]
}
}
}
```

## File structure
```bash
└─.kiota
└─plugins
└─GitHub.json # OpenAPI description
└─generated
└─plugins
└─github
└─github-apimanifest.json # Specific API Manifest
└─github-openai.json #OpenAI Plugin
└─sliced-openapi-github.json # Sliced OpenAPI description
└─kiota-config.json
└─apimanifest.json # Single artifact with all APIs dependencies info
```

[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html
Loading
Loading