Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HPCC4J-581 WsFS Client should add path delim only if needed #686

Closed
wants to merge 69 commits into from
Closed

HPCC4J-581 WsFS Client should add path delim only if needed #686

wants to merge 69 commits into from

Conversation

rpastrana
Copy link
Member

@rpastrana rpastrana commented Mar 1, 2024 •
edited
Loading

Type of change:

  • This change is a bug fix (non-breaking change which fixes an issue).
  • This change is a new feature (non-breaking change which adds functionality).
  • This change is a breaking change (fix or feature that will cause existing behavior to change).

Checklist:

  • I have created a corresponding JIRA ticket for this submission
  • My code follows the code style of this project.
    • I have applied the Eclipse code-format template provided.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly, or...
    • I have created a JIRA ticket to update the documentation.
    • Any new interfaces or exported functions are appropriately commented.
  • I have read the HPCC Systems CONTRIBUTORS document (https://github.com/hpcc-systems/HPCC-Platform/wiki/Guide-for-contributors).
  • The change has been fully tested:
    • This change does not cause any existing JUnits to fail.
    • I have include JUnit coverage to test this change
    • I have performed system test and covered possible regressions and side effects.
  • I have given due consideration to all of the following potential concerns:
    • Scalability
    • Performance
    • Security
    • Thread-safety
    • Premature optimization
    • This change fixes the problem, not just the symptom

Testing:

ghalliday and others added 30 commits June 15, 2023 17:47
@ghalliday
Split off 9.2.0
65a2856 
Signed-off-by: Gavin Halliday <[email protected]>
@rpastrana
HPCC4J-514 Ensure WUQueryTest creates multiple WUs (hpcc-systems#618)
48d067c 
* HPCC4J-334 Automate WsClient Interface (hpcc-systems#616)

- Adds python to generate new wsdls, stub, and wrappers
- Adds vscode launch profile
- Updates filespray wsdl reference in pom
- Targets source code wrapper utility rather than jared version

Signed-off-by: Rodrigo Pastrana <[email protected]>

* HPCC4J-514 Ensure WUQueryTest creates multiple WUs

- Ensures WUQueryTest creates multiple WUs

Signed-off-by: Rodrigo Pastrana <[email protected]>

---------

Signed-off-by: Rodrigo Pastrana <[email protected]>
@rpastrana
Update publish-snapshots-on-merge.yml (hpcc-systems#617)
d378484 
extend action to on branch creations as well
@jakesmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
8ce998f 
Signed-off-by: Jake Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@rpastrana
HPCC4J-522 GithutAction targets java8 (hpcc-systems#620)
8532695 
Signed-off-by: Rodrigo Pastrana <[email protected]>
Co-authored-by: Pastrana <[email protected]>
@GordonSmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
815b405 
Signed-off-by: Gordon Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@rpastrana
Create Jirabot.yml
678afb5 
Signed-off-by: Rodrigo <[email protected]>
@GordonSmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
3af0028 
Signed-off-by: Gordon Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@jakesmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
9aa9529 
Signed-off-by: Jake Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@jakesmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
6174408 
Signed-off-by: Jake Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@jpmcmu
HPCC4J-532 Jirabot should translate hpcc4j prefix (hpcc-systems#633)
a5611b8 
- Added logic to translate hpcc4j prefix

Signed-off-by: James McMullan [email protected]

Signed-off-by: James McMullan [email protected]
@GordonSmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
a20ff7c 
Signed-off-by: Gordon Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@jakesmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
26c3f39 
Signed-off-by: Jake Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@GordonSmith
Merge remote-tracking branch 'origin/candidate-9.2.x'
962a2f7 
Signed-off-by: Gordon Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@GordonSmith
Split off 9.4.0
50acafd 
Signed-off-by: Gordon Smith <[email protected]>
@jakesmith
Merge remote-tracking branch 'origin/candidate-9.2.x' into candidate-…
b2e45c9 
…9.4.x

Signed-off-by: Jake Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@GordonSmith
Merge remote-tracking branch 'origin/candidate-9.2.x' into candidate-…
15e4128 
…9.4.x

Signed-off-by: Gordon Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@ghalliday
Merge remote-tracking branch 'origin/candidate-9.2.x' into candidate-…
2646614 
…9.4.x

Signed-off-by: Gavin Halliday <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@jakesmith
Split off 9.4.2
acc0ecb 
Signed-off-by: Jake Smith <[email protected]>
@jpmcmu
HPCC4J-542 DFSClient: Create JUnit for read retry (hpcc-systems#648)
905b095 
* HPCC4J-542 DFSClient: Create JUnit for read retry

- Added file part failure retry test
- Fixed retry issue on initial connection

Signed-off-by: James McMullan [email protected]

* Code review changes

---------

Signed-off-by: James McMullan [email protected]
@jpmcmu
HPCC4J-545 Resume read test causes OOM error (hpcc-systems#649)
80d27a8 
- Updated resume read test to periodically run gc

Signed-off-by: James McMullan [email protected]

Signed-off-by: James McMullan [email protected]
@ghalliday
Split off 9.4.4
28183cf 
Signed-off-by: Gavin Halliday <[email protected]>
@GordonSmith
Split off 9.4.6
b5e89dd 
Signed-off-by: Gordon Smith <[email protected]>
@jpmcmu
HPCC4J-551 HPCCFile Make TLK Use Optional (hpcc-systems#653)
2dc8279 
- Added an option to HPCCFile to make TLK reading optional
- Fixed Javadoc issue

Signed-off-by: James McMullan [email protected]

Signed-off-by: James McMullan [email protected]
@jpmcmu
HPCC4J-551 HPCCFile Make TLK Use Optional
f0ec51e 
- Added getter & setter for the useTLK option

Signed-off-by: James McMullan [email protected]
@jpmcmu
Merge pull request hpcc-systems#655 from jpmcmu/HPCC4J-551
99601b0 
HPCC4J-551 HPCCFile Make TLK Use Optional
@ghalliday
Split off 9.4.8
5f8d0b5 
Signed-off-by: Gavin Halliday <[email protected]>
@ghalliday
Merge remote-tracking branch 'origin/candidate-9.2.x' into candidate-…
787373a 
…9.4.x

Signed-off-by: Gavin Halliday <[email protected]>
@jpmcmu
HPCC4J-532 Jirabot debugging improvements (hpcc-systems#656)
15e295f 
- Added exception to error output
- Fixed transition array output
- Added python, pip, jira version output

Signed-off-by: James McMullan [email protected]

Signed-off-by: James McMullan [email protected]
@rpastrana @dependabot
Bump org.json:json from 20230227 to 20231013 (hpcc-systems#650) (hpcc…
16e3bf6 
…-systems#657)

Bumps [org.json:json](https://github.com/douglascrockford/JSON-java) from 20230227 to 20231013.
- [Release notes](https://github.com/douglascrockford/JSON-java/releases)
- [Changelog](https://github.com/stleary/JSON-java/blob/master/docs/RELEASES.md)
- [Commits](https://github.com/douglascrockford/JSON-java/commits)

---
updated-dependencies:
- dependency-name: org.json:json
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
jpmcmu and others added 14 commits January 19, 2024 09:58
@jpmcmu
HPCC4J-553 Github Actions: Add baremetal test workflow (hpcc-systems#669
deca046 
)

* HPCC4J-553 Github Actions: Add baremetal test workflow

- Renamed existing k8s workflow
- Added baremetal workflow

Signed-off-by: James McMullan [email protected]

* Code review changes

---------

Signed-off-by: James McMullan [email protected]
@ghalliday
Split off 9.4.28
9f73bd1 
Signed-off-by: Gavin Halliday <[email protected]>
@jakesmith
Split off 9.4.30
87c36bf 
Signed-off-by: Jake Smith <[email protected]>
@jpmcmu
HCCP4J-571 Github Action: Jirabot Merge fails to find tag (hpcc-syste…
f743445 
…ms#673)

- Corrected checkout step

Signed-off-by: James McMullan [email protected]

Signed-off-by: James McMullan [email protected]
@jpmcmu
HPCC4J-571 Github Action: Jirabot Merge fails to find correct tag (hp…
7253313 
…cc-systems#677)

- Added debugging information for action vars
- Add a default Jira URL as a fallback

Signed-off-by: James McMullan [email protected]

Signed-off-by: James McMullan [email protected]
@jpmcmu
HPCC4J-553 Github Actions: Add baremetal test workflow (hpcc-systems#674
9b112b3 
)

- Fixed issue with version being returned as a string for master
- Fixed logic when determining version to test against for master

Signed-off-by: James McMullan [email protected]

Signed-off-by: James McMullan [email protected]
@ghalliday
Merge remote-tracking branch 'origin/candidate-9.2.x' into candidate-…
e705165 
…9.4.x

Signed-off-by: Gavin Halliday <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@ghalliday
Split off 9.4.32
365369b 
Signed-off-by: Gavin Halliday <[email protected]>
@GordonSmith
Split off 9.4.34
494517c 
Signed-off-by: Gordon Smith <[email protected]>
@GordonSmith
Split off 9.4.36
961b1e1 
Signed-off-by: Gordon Smith <[email protected]>
@jakesmith
Merge remote-tracking branch 'origin/candidate-9.2.x' into candidate-…
cd7229b 
…9.4.x

Signed-off-by: Jake Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@jakesmith
Split off 9.4.38
5634319 
Signed-off-by: Jake Smith <[email protected]>
@jakesmith
Merge remote-tracking branch 'origin/candidate-9.2.x' into candidate-…
71c82a3 
…9.4.x

Signed-off-by: Jake Smith <[email protected]>

# Conflicts:
#	commons-hpcc/pom.xml
#	dfsclient/pom.xml
#	pom.xml
#	wsclient/pom.xml
@jakesmith
Split off 9.4.40
4c55669 
Signed-off-by: Jake Smith <[email protected]>
@rpastrana
Copy link
Member Author

@jpmcmu this is currently targeting 9.4.x, but considering back-porting, unless you can think of a reason why we shouldn't

@rpastrana rpastrana requested a review from jpmcmu March 1, 2024 21:25
jpmcmu
jpmcmu approved these changes Mar 4, 2024
View reviewed changes
Copy link
Contributor

@jpmcmu jpmcmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, a minor question about needing trim()

wsclient/src/main/java/org/hpccsystems/ws/client/HPCCFileSprayClient.java
request.setSourceIP(targetDropZone.getNetAddress());
request.setSourcePath(targetDropZone.getPath() + "/" + sourceFileName);
request.setSourcePath(Utils.ensureTrailingPathSlash(targetDropZone.getPath()) + sourceFileName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these calls to ensureTrailingPathSlash also need the call to trim()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's been the user's responsibility to do so, added a java11 instruction to strip all trailing white space chars

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed offline, we will continue to target java8 for now.
I added a custom trimRight function instead.

@rpastrana rpastrana force-pushed the HPCC4J-581-HandleFilePathDelimiters branch from 1654257 to 5aa25be Compare March 4, 2024 21:01
@rpastrana rpastrana requested a review from jpmcmu March 4, 2024 21:01
@rpastrana
Copy link
Member Author

@jpmcmu please review

@rpastrana rpastrana force-pushed the HPCC4J-581-HandleFilePathDelimiters branch 3 times, most recently from dce9574 to 7587136 Compare March 5, 2024 16:33
jpmcmu
jpmcmu approved these changes Mar 5, 2024
View reviewed changes
Copy link
Contributor

@jpmcmu jpmcmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rpastrana Looks good to me

@rpastrana rpastrana requested a review from jpmcmu March 7, 2024 16:04
jpmcmu
jpmcmu approved these changes Mar 7, 2024
View reviewed changes
Copy link
Contributor

@jpmcmu jpmcmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rpastrana looks good, are you still planning on backporting this to 9.0x?

@rpastrana
HPCC4J-581 WsFS Client should add path delim only if needed
be37a87 
- Introduces functionality to strip trailing white space
- Adds trimright utility functionality
- Adds new test cases
- Adjusts pre-existing test cases to expected | actual format
- Do not trim original string parameter
- Do not duplicate Path URL param

Signed-off-by: Rodrigo Pastrana <[email protected]>
@rpastrana rpastrana force-pushed the HPCC4J-581-HandleFilePathDelimiters branch from 922afe2 to be37a87 Compare March 7, 2024 16:29
@rpastrana rpastrana changed the base branch from candidate-9.4.x to candidate-9.2.x March 8, 2024 14:25
@rpastrana rpastrana closed this Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jpmcmu jpmcmu jpmcmu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@rpastrana @jpmcmu @ghalliday @jakesmith @GordonSmith