Introduce aws sigv4a request signer

[noCharger]

Description

• Introduce AWSRequestSigV4ASigningApacheInterceptor to sign metadata access with sigV4a
• Refactor ResourceBasedAWSRequestSigningApacheInterceptor interface

Current signer:

24/04/04 20:53:48 INFO AWSRequestSigningApacheInterceptor: unsigned request: GET /query_results2?master_timeout=30s&ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false HTTP/1.1 [Content-Length: 0, Host: xxx, Connection: Keep-Alive, User-Agent: Apache-HttpAsyncClient/4.1.5 (Java/17.0.9)]
24/04/04 20:53:48 INFO AWSRequestSigningApacheInterceptor: signed request: GET /query_results2?master_timeout=30s&ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false HTTP/1.1 [Authorization: AWS4-HMAC-SHA256 Credential=xxx, SignedHeaders=host;user-agent;x-amz-date;x-amz-security-token, Signature=xxx, Connection: Keep-Alive, Host: xxx, User-Agent: Apache-HttpAsyncClient/4.1.5 (Java/17.0.9), X-Amz-Date: xxx, X-Amz-Security-Token: xxx]

New sigV4a signer

24/04/04 20:02:46 INFO AWSRequestSigV4ASigningApacheInterceptor: before: HEAD /flint_ql_sessions?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false HTTP/1.1 [Host: xxx, Connection: Keep-Alive, User-Agent: Apache-HttpAsyncClient/4.1.5 (Java/17.0.9)]
24/04/04 20:02:46 INFO AWSRequestSigV4ASigningApacheInterceptor: after: HEAD /flint_ql_sessions?ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=false HTTP/1.1 [Authorization: AWS4-ECDSA-P256-SHA256 Credential=xxx, SignedHeaders=connection;host;x-amz-date;x-amz-region-set;x-amz-security-token, Signature=xxx, Connection: Keep-Alive, Host: xxx, User-Agent: Apache-HttpAsyncClient/4.1.5 (Java/17.0.9), X-Amz-Date: 20240404T200246Z, X-Amz-Region-Set: eu-west-1, X-Amz-Security-Token: xxx]

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[shikharj05]

Thanks for the changes! I have left some comments.

[vmmusings]

Why is there a builder for AwsCrtV4asigner? Waht are the parameters available? Are we good with defaults. I see
plain constructor for AWS4Signer signer = new AWS4Signer();

Also when should we use AwsCrtV4asigner insread of AWS4Signer. what are the downsides of making AwsCrtV4asigner default.

[noCharger]

Why is there a builder for AwsCrtV4asigner? Waht are the parameters available? Are we good with defaults. I see plain constructor for AWS4Signer signer = new AWS4Signer();

Also when should we use AwsCrtV4asigner insread of AWS4Signer. what are the downsides of making AwsCrtV4asigner default.

AwsCrtV4aSigner is an interface that employs the builder pattern to instantiate. However, the AWS4Signer class has its own getter and setter.

@SdkPublicApi
@Immutable
@ThreadSafe
public interface AwsCrtV4aSigner extends Signer, Presigner {

    /**
     * Create a default Aws4aSigner.
     */
    static AwsCrtV4aSigner create() {
        return DefaultAwsCrtV4aSigner.create();
    }

    static Builder builder() {
        return DefaultAwsCrtV4aSigner.builder();
    }

    interface Builder {
        /**
         * The region scope that this signer will default to if not provided explicitly when the signer is invoked.
         *
         * @param defaultRegionScope The default region scope.
         * @return This builder for method chaining.
         */
        Builder defaultRegionScope(RegionScope defaultRegionScope);

        AwsCrtV4aSigner build();
    }
}

The main difference is that AWS4Signer utilizes AWS4-HMAC-SHA256, whereas AWSCrtV4asigner uses AWS4-ECDSA-P256-SHA256:

• Algorithm: The primary difference is the algorithm used for signing requests: HMAC with SHA-256 in the former, and ECDSA with P-256 and SHA-256 in the latter.
• Security: Both are considered secure, but ECDSA might offer better performance with equivalent levels of security due to the efficiency of elliptic curve cryptography.
• Implementation: AWS4Signer is from aws-java-sdk V1, whereas AwsCrtV4asigner is from V2. They are incompatible with one another.

For the long run, we could use AwsCrtV4asigner as the default. However, in this PR, I would like to confine the blast radius to only metadata access using a specified credential provider.

[penghuo]

long term plan is replace signer with signer4a?, and deprecated AWSRequestSigningApacheInterceptor? In case the major difference of singer vs singer 4a is With AWS Signature Version 4A, the signature does not include Region-specific information and is calculated using the AWS4-ECDSA-P256-SHA256 algorithm.
I think we should use one Interceptor instead of two.

if it is correct, could u create a issue to track future maintain works.

[noCharger]

long term plan is replace signer with signer4a?, and deprecated AWSRequestSigningApacheInterceptor? In case the major difference of singer vs singer 4a is With AWS Signature Version 4A, the signature does not include Region-specific information and is calculated using the AWS4-ECDSA-P256-SHA256 algorithm. I think we should use one Interceptor instead of two.

if it is correct, could u create a issue to track future maintain works.

#321

[shikharj05]

nit: better to check if header is against a set of headers

[opensearch-trigger-bot]

The backport to 0.3 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/opensearch-spark/backport-0.3 0.3
# Navigate to the new working tree
pushd ../.worktrees/opensearch-spark/backport-0.3
# Create a new branch
git switch --create backport/backport-303-to-0.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c877e09d0cbc45569b7e963382d93904a5801e9e
# Push it to GitHub
git push --set-upstream origin backport/backport-303-to-0.3
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/opensearch-spark/backport-0.3

Then, create a pull request where the base branch is 0.3 and the compare/head branch is backport/backport-303-to-0.3.