Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OE] Support compression for requests #6366

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[OE] Support compression for requests #6366

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e3b9e8f
[OE] Support compression for requests
kavilla Apr 8, 2024
0b166e8
address comments
kavilla May 2, 2024
840f186
Changeset file for PR #6366 created/updated
opensearch-changeset-bot[bot] May 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/cypress_workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ env:
TEST_REPO: ${{ inputs.test_repo != '' && inputs.test_repo || 'opensearch-project/opensearch-dashboards-functional-test' }}
TEST_BRANCH: "${{ inputs.test_branch != '' && inputs.test_branch || github.base_ref }}"
FTR_PATH: 'ftr'
START_CMD: 'node ../scripts/opensearch_dashboards --dev --no-base-path --no-watch --savedObjects.maxImportPayloadBytes=10485760 --server.maxPayloadBytes=1759977 --logging.json=false --data.search.aggs.shardDelay.enabled=true --csp.warnLegacyBrowsers=false'
START_CMD: 'node ../scripts/opensearch_dashboards --dev --no-base-path --no-watch --savedObjects.maxImportPayloadBytes=10485760 --server.maxPayloadBytes=1759977 --logging.json=false --data.search.aggs.shardDelay.enabled=true --csp.warnLegacyBrowsers=false --server.compression.enabled=true --opensearch.compression=true'
OPENSEARCH_SNAPSHOT_CMD: 'node ../scripts/opensearch snapshot -E cluster.routing.allocation.disk.threshold_enabled=false'
CYPRESS_BROWSER: 'chromium'
CYPRESS_VISBUILDER_ENABLED: true
Expand Down
2 changes: 2 additions & 0 deletions changelogs/fragments/6366.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- Support compressing traffic with `opensearch.compression: true` or `opensearch.compress: 'gzip'` ([#6366](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6366))
5 changes: 5 additions & 0 deletions src/core/server/opensearch/client/client_config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
| 'username'
| 'password'
| 'disablePrototypePoisoningProtection'
| 'compression'
> & {
memoryCircuitBreaker?:
| OpenSearchConfig['memoryCircuitBreaker']
Expand Down Expand Up @@ -120,6 +121,10 @@
clientOptions.disablePrototypePoisoningProtection = config.disablePrototypePoisoningProtection;
}

if (config.compression) {
clientOptions.compression = config.compression;

Check warning on line 125 in src/core/server/opensearch/client/client_config.ts

View check run for this annotation

Codecov / codecov/patch

src/core/server/opensearch/client/client_config.ts#L125 

Added line #L125 was not covered by tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to add a test for this given we updated the test to not do this by default?

}

return clientOptions;
}

Expand Down
1 change: 1 addition & 0 deletions src/core/server/opensearch/opensearch_config.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ test('set correct defaults', () => {
expect(configValue).toMatchInlineSnapshot(`
OpenSearchConfig {
"apiVersion": "7.x",
"compression": undefined,
"customHeaders": Object {},
"disablePrototypePoisoningProtection": undefined,
"healthCheckDelay": "PT2.5S",
Expand Down
17 changes: 17 additions & 0 deletions src/core/server/opensearch/opensearch_config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,14 @@ import { schema, TypeOf } from '@osd/config-schema';
import { Duration } from 'moment';
import { readFileSync } from 'fs';
import { ConfigDeprecationProvider } from 'src/core/server';
import { ClientOptions } from '@opensearch-project/opensearch';
import { readPkcs12Keystore, readPkcs12Truststore } from '../utils';
import { ServiceConfigDescriptor } from '../internal_types';

const hostURISchema = schema.uri({ scheme: ['http', 'https'] });

export const DEFAULT_API_VERSION = '7.x';
export const DEFAULT_COMPRESSION = 'gzip';

export type OpenSearchConfigType = TypeOf<typeof configSchema>;
type SslConfigSchema = OpenSearchConfigType['ssl'];
Expand Down Expand Up @@ -131,6 +133,9 @@ export const configSchema = schema.object({
healthCheck: schema.object({ delay: schema.duration({ defaultValue: 2500 }) }),
ignoreVersionMismatch: schema.boolean({ defaultValue: false }),
disablePrototypePoisoningProtection: schema.maybe(schema.boolean({ defaultValue: false })),
compression: schema.maybe(
schema.oneOf([schema.literal(DEFAULT_COMPRESSION), schema.boolean({ defaultValue: false })])
),
});

const deprecations: ConfigDeprecationProvider = ({ renameFromRoot, renameFromRootWithoutMap }) => [
Expand Down Expand Up @@ -313,6 +318,12 @@ export class OpenSearchConfig {
*/
public readonly disablePrototypePoisoningProtection?: boolean;

/**
* Specifies whether the client should use compression to engine
* or not.
*/
public readonly compression?: ClientOptions['compression'];

constructor(rawConfig: OpenSearchConfigType) {
this.ignoreVersionMismatch = rawConfig.ignoreVersionMismatch;
this.apiVersion = rawConfig.apiVersion;
Expand All @@ -334,6 +345,12 @@ export class OpenSearchConfig {
this.password = rawConfig.password;
this.customHeaders = rawConfig.customHeaders;
this.disablePrototypePoisoningProtection = rawConfig.disablePrototypePoisoningProtection;
this.compression =
typeof rawConfig.compression === 'boolean' && rawConfig.compression
? DEFAULT_COMPRESSION
: typeof rawConfig.compression === 'string'
? rawConfig.compression
: undefined;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An expression of rawConfig.compression = false gets out of this as undefined which is the wrong message out of this function (because it was defined). For the sanity of future devs, we should:

Suggested change
: undefined;
: false;

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, for the sanity of future devs, can we not use nested ternaries, especially without parens? calling a function with readable if statements isn't the worst thing imo :)


const { alwaysPresentCertificate, verificationMode } = rawConfig.ssl;
const { key, keyPassphrase, certificate, certificateAuthorities } = readKeyAndCerts(rawConfig);
Expand Down
Loading