-
Notifications
You must be signed in to change notification settings - Fork 141
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add README to async-query-core (#2766)
Signed-off-by: Tomoyuki Morita <[email protected]>
- Loading branch information
Showing
1 changed file
with
32 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# async-query-core library | ||
|
||
This directory contains async-query-core library, which implements the core logic of async-query and provide extension points to allow plugin different implementation of data storage, etc. | ||
`async-query` module provides implementations for OpenSearch index based implementation. | ||
|
||
## Type of queries | ||
There are following types of queries, and the type is automatically identified by analysing the query. | ||
- BatchQuery: Execute single query in Spark | ||
- InteractiveQuery: Establish session and execute queries in single Spark session | ||
- IndexDMLQuery: Handles DROP/ALTER/VACUUM operation for Flint indices | ||
- RefreshQuery: One time query request to refresh(update) Flint index | ||
- StreamingQuery: Continuously update flint index in single Spark session | ||
|
||
## Extension points | ||
Following is the list of extension points where the consumer of the library needs to provide their own implementation. | ||
|
||
- Data store interface | ||
- [AsyncQueryJobMetadataStorageService](src/main/java/org/opensearch/sql/spark/asyncquery/AsyncQueryJobMetadataStorageService.java) | ||
- [SessionStorageService](java/org/opensearch/sql/spark/execution/statestore/SessionStorageService.java) | ||
- [StatementStorageService](src/main/java/org/opensearch/sql/spark/execution/statestore/StatementStorageService.java) | ||
- [FlintIndexMetadataService](src/main/java/org/opensearch/sql/spark/flint/FlintIndexMetadataService.java) | ||
- [FlintIndexStateModelService](src/main/java/org/opensearch/sql/spark/flint/FlintIndexStateModelService.java) | ||
- [IndexDMLResultStorageService](src/main/java/org/opensearch/sql/spark/flint/IndexDMLResultStorageService.java) | ||
- Other | ||
- [LeaseManager](src/main/java/org/opensearch/sql/spark/leasemanager/LeaseManager.java) | ||
- [JobExecutionResponseReader](src/main/java/org/opensearch/sql/spark/response/JobExecutionResponseReader.java) | ||
- [QueryIdProvider](src/main/java/org/opensearch/sql/spark/dispatcher/QueryIdProvider.java) | ||
- [SessionIdProvider](src/main/java/org/opensearch/sql/spark/execution/session/SessionIdProvider.java) | ||
- [SessionConfigSupplier](src/main/java/org/opensearch/sql/spark/execution/session/SessionConfigSupplier.java) | ||
- [SparkExecutionEngineConfigSupplier](src/main/java/org/opensearch/sql/spark/config/SparkExecutionEngineConfigSupplier.java) | ||
- [SparkSubmitParameterModifier](src/main/java/org/opensearch/sql/spark/config/SparkSubmitParameterModifier.java) | ||
- [EMRServerlessClientFactory](src/main/java/org/opensearch/sql/spark/client/EMRServerlessClientFactory.java) |