Document new experimental ingestion streaming APIs

[reta]

Description

Document new experimental ingestion streaming APIs

Issues Resolved

Closes #537

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[nhtruong]

The running linter will throw an error on the operationId and the x-operation-group.
I think you want to do bulk_stream for x-operation-group and bulk_stream.0 for operationId. Then on the generated client, there will be a client.bulk_stream() method, which is different from client.bulk()

[reta]

Thanks a lot for the hints, @nhtruong !

[nhtruong]

Please add tests to these operations as well.

[github-actions]

Changes Analysis

Commit SHA: 8043e59
Comparing To SHA: 7d05664

API Changes

Summary

├─┬Paths
│ ├──[➕] path (340:3)
│ └──[➕] path (9703:3)
└─┬Components
  ├──[➕] requestBodies (25199:7)
  ├──[➕] responses (26808:7)
  ├──[➕] parameters (14377:7)
  ├──[➕] parameters (14433:7)
  ├──[➕] parameters (14426:7)
  ├──[➕] parameters (14385:7)
  ├──[➕] parameters (14370:7)
  ├──[➕] parameters (14404:7)
  ├──[➕] parameters (14349:7)
  ├──[➕] parameters (14356:7)
  ├──[➕] parameters (14363:7)
  ├──[➕] parameters (14341:7)
  ├──[➕] parameters (14395:7)
  ├──[➕] parameters (14412:7)
  ├──[➕] parameters (14419:7)
  └──[➕] schemas (29888:7)

Document Element	Total Changes	Breaking Changes
paths	2	0
components	16	0

• Total Changes: 18
• Additions: 18

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/11108235686/artifacts/1996253002

API Coverage

	Before	After	Δ
Covered (%)	588 (57.59 %)	588 (57.59 %)	0 (0 %)
Uncovered (%)	433 (42.41 %)	433 (42.41 %)	0 (0 %)
Unknown	25	29	4

[github-actions]

Spec Test Coverage Analysis

Total	Tested
504	303 (60.12 %)

[dblock]

Good start. Needs tests, please!

[dblock]

You'll need to update the 2.17 version used to the released one, and switch the next version to 2.18. Also see validator failures, etc.

[dblock]

File should be _core/bulk/stream.yaml.

[dblock]

Same, bulk/stream.yaml.

[dblock]

So does it actually stream data? 🤔

[reta]

So does it actually stream data? 🤔

It does, I am not sure how we could express that in OpenAPI

[dblock]

So does it actually stream data? 🤔

It does, I am not sure how we could express that in OpenAPI

I don't think we need to, but do clients automatically do it? AFAIK we don't have code in our tooling that does anything special.

[reta]

ERROR   _core/bulk/stream.yaml (tests/default/_core/bulk/stream.yaml)
    ERROR   CHAPTERS
        ERROR   Create an index.
            PASSED  REQUEST BODY
            ERROR   RESPONSE STATUS (Expected status 200, but received 500: application/json. The engine does not support HTTP streaming, unable to serve uri [/_bulk/stream] and method [POST])
            SKIPPED RESPONSE PAYLOAD BODY
            SKIPPED RESPONSE PAYLOAD SCHEMA
            SKIPPED RESPONSE OUTPUT VALUES
        SKIPPED Delete document in an index.
        SKIPPED Bulk document CRUD.

@dblock need your advice here: streaming (as of today) requires dedicated transport plugin. Here are 2 options we could make it work:

• ignore the test if particular plugin is not installed, or
• install (and make default) the streaming transport

[reta]

I don't think we need to, but do clients automatically do it? AFAIK we don't have code in our tooling that does anything special.

Oh, we don't need to actually, no special treatment required

[dblock]

• install (and make default) the streaming transport

I think this is the right answer. The tooling in this repo should support streaming just like we support various content types.

[dblock]

You'll have to move tests in the plugins folder as well from tests/default.