Skip to content

Commit

Permalink
Reword user manual
Browse files Browse the repository at this point in the history
Signed-off-by: Chen Dai <[email protected]>
  • Loading branch information
dai-chen committed Nov 22, 2023
1 parent f1a02c2 commit 26d6e27
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 7 deletions.
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,9 +358,9 @@ In the index mapping, the `_meta` and `properties`field stores meta and schema i
(false), IMMEDIATE(true), WAIT_UNTIL(wait_for)]
- `spark.datasource.flint.read.scroll_size`: default value is 100.
- `spark.datasource.flint.read.scroll_duration`: default value is 5 minutes. scroll context keep alive duration.
- `spark.datasource.flint.retry.max_retries`: default value is 3. max retries on failed HTTP request. 0 means retry is disabled.
- `spark.datasource.flint.retry.http_status_codes`: default value is "429,502". retryable HTTP response status code list
- `spark.datasource.flint.retry.exception_class_names`: by default no retry on any exception thrown. retryable exception class name list.
- `spark.datasource.flint.retry.max_retries`: max retries on failed HTTP request. default value is 3. Use 0 to disable retry.
- `spark.datasource.flint.retry.http_status_codes`: retryable HTTP response status code list. default value is "429,502" (429 Too Many Request and 502 Bad Gateway).
- `spark.datasource.flint.retry.exception_class_names`: retryable exception class name list. by default no retry on any exception thrown.
- `spark.flint.optimizer.enabled`: default is true.
- `spark.flint.index.hybridscan.enabled`: default is false.
- `spark.flint.index.checkpoint.mandatory`: default is true.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,12 @@ public <T> Future<T> execute(HttpAsyncRequestProducer requestProducer,
HttpContext context,
FutureCallback<T> callback) {
return new Future<>() {
/** Delegate future object created per execution */
/**
* Delegated future object created per doExecuteAndFutureGetWithRetry() call which creates initial object too.
* In this way, we avoid the duplicate logic of first call and subsequent retry calls.
* Here the assumption is cancel, isCancelled and isDone never called before get().
* (OpenSearch RestClient seems only call get() API)
*/
private Future<T> delegate;

@Override
Expand All @@ -85,15 +90,15 @@ public boolean isDone() {

@Override
public T get() throws InterruptedException, ExecutionException {
return doGetWithRetry(() -> delegate.get());
return doExecuteAndFutureGetWithRetry(() -> delegate.get());
}

@Override
public T get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException {
return doGetWithRetry(() -> delegate.get(timeout, unit));
return doExecuteAndFutureGetWithRetry(() -> delegate.get(timeout, unit));
}

private T doGetWithRetry(Callable<T> futureGet) throws InterruptedException, ExecutionException {
private T doExecuteAndFutureGetWithRetry(Callable<T> futureGet) throws InterruptedException, ExecutionException {
try {
// Retry by creating a new Future object (as new delegate) and get its result again
return Failsafe
Expand Down

0 comments on commit 26d6e27

Please sign in to comment.