Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add FlintJob integration test with EMR serverless #449

Merged
merged 2 commits into from
Jul 25, 2024
Merged

Add FlintJob integration test with EMR serverless #449

merged 2 commits into from
Jul 25, 2024

Conversation

penghuo
Copy link
Collaborator

@penghuo penghuo commented Jul 24, 2024 •
edited
Loading

Description

  • add flintjob integration test with EMR serverless.
export AWS_OPENSEARCH_HOST=search-xxx.us-west-2.on.aws
export AWS_REGION=us-west-2
export AWS_EMRS_APPID=xxx
export AWS_EMRS_EXECUTION_ROLE=xxx
export AWS_S3_CODE_BUCKET=xxx
export AWS_S3_CODE_PREFIX=xxx
export AWS_OPENSEARCH_RESULT_INDEX=query_execution_result_glue
  • enforce jackson version to comptaible with spark jackson version.

Issues Resolved

#442

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@penghuo penghuo marked this pull request as ready for review July 24, 2024 21:28
dai-chen
dai-chen approved these changes Jul 24, 2024
View reviewed changes
Copy link
Collaborator

@dai-chen dai-chen left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes!

There is code style CI failure. Do we need to update dev doc here? https://github.com/opensearch-project/opensearch-spark/blob/main/DEVELOPER_GUIDE.md#aws-integration-test

@penghuo
Fix fmt
2e83d2b 
Signed-off-by: Peng Huo <[email protected]>
@penghuo penghuo merged commit 538dd54 into opensearch-project:main Jul 25, 2024
4 checks passed
@dai-chen dai-chen added infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc. 0.5 labels Jul 26, 2024
ykmr1224 pushed a commit to ykmr1224/opensearch-spark that referenced this pull request Jul 29, 2024
@penghuo @ykmr1224
Add FlintJob integration test with EMR serverless (opensearch-project…
8741862 
…#449)

Signed-off-by: Peng Huo <[email protected]>
@penghuo penghuo self-assigned this Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dai-chen dai-chen dai-chen approved these changes

@rupal-bq rupal-bq Awaiting requested review from rupal-bq

@vmmusings vmmusings Awaiting requested review from vmmusings

@seankao-az seankao-az Awaiting requested review from seankao-az seankao-az is a code owner

@anirudha anirudha Awaiting requested review from anirudha anirudha is a code owner

@kaituo kaituo Awaiting requested review from kaituo kaituo is a code owner

@YANG-DB YANG-DB Awaiting requested review from YANG-DB YANG-DB is a code owner

Assignees

@penghuo penghuo

Labels
0.5 infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@penghuo @dai-chen