Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preventing connection uri to show in logging #605

Open
grtjn opened this issue Dec 15, 2021 · 6 comments
Open

Preventing connection uri to show in logging #605

grtjn opened this issue Dec 15, 2021 · 6 comments

Comments

@grtjn
Copy link

grtjn commented Dec 15, 2021

I noticed that when running a CorbTask with gradle -i, Gradle will print a 'Starting process' log line that contains the connection uri in full detail:

> Task :deleteDocs
Caching disabled for task ':deleteDocs' because:
  Build cache is disabled
Task ':deleteDocs' is not up-to-date because:
  Task.upToDateWhen is false.
Starting process 'command 'C:\Java\jdk-11\bin\java.exe''. Working directory: D:\Projects\datahub Command: C:\Java\jdk-11\bin\java.exe -DEXPORT-FILE-NAME=deleteDocs.csv -DMODULES-DATABASE=datahub-modules -DOPTIONS-FILE=src/main/ml-modules/root/corb/options.properties -DPROCESS-MODULE=src/main/ml-modules/root/corb/delete-docs-xform.xqy|ADHOC -DPROCESS-TASK=com.marklogic.developer.corb.ExportBatchToFileTask -DTHREAD-COUNT=10 -DURIS-MODULE=src/main/ml-modules/root/corb/delete-docs-uris.xqy|ADHOC -DXCC-CONNECTION-URI=xcc://xxx:yyy@zzz:8140

I have been looking for ways to suppress that line, but seems like JavaExec (which is used under the covers) does not allow suppressing that line when info level logging is enabled: https://github.com/gradle/gradle/blob/b74338910359a112212adb7d0de40b156d4c9c31/subprojects/core/src/main/java/org/gradle/process/internal/DefaultExecHandle.java#L257

I could of course provide the connection string in an options file, but that would negate the flexibility of providing connection details via gradle properties.

So, main question: is there another way to specify the connection string, without it showing up in JavaExec logging?

@hansenmc
Copy link
Contributor

hansenmc commented Dec 15, 2021

If you set a system property for XCC-CONNECTION-URI (or the more specific XCC-* options) outside of the task, rather than setting them with -D switches, then the CoRB task should pick them up and apply them:
https://github.com/marklogic-community/ml-gradle/blob/master/src/main/groovy/com/marklogic/gradle/task/CorbTask.groovy#L205

@hansenmc
Copy link
Contributor

You could also have your deleteDocs task write the options to a temp file in a pre task, and then delete in a post task dependency, and use that temp options file.

@grtjn
Copy link
Author

grtjn commented Dec 16, 2021

It would be nicer if temp files could be avoided. To try your other suggestion, I made this change:

task deleteDocs(type: com.marklogic.gradle.task.CorbTask, group: project.name) {
    //xccConnectionUri = "${CorbFromStaging}"
    System.setProperty("XCC-CONNECTION-URI", "${CorbFromStaging}")

But the effect is the same, not seeing any difference in console:

Starting process 'command 'C:\Java\jdk-11\bin\java.exe''. Working directory: D:\Projects\datahub Command: C:\Java\jdk-11\bin\java.exe -DEXPORT-FILE-NAME=deleteDocs.csv -DMODULES-DATABASE=datahub-modules -DOPTIONS-FILE=src/main/ml-modules/root/corb/options.properties -DPROCESS-MODULE=src/main/ml-modules/root/corb/delete-docs-xform.xqy|ADHOC -DPROCESS-TASK=com.marklogic.developer.corb.ExportBatchToFileTask -DTHREAD-COUNT=8 -DURIS-MODULE=src/main/ml-modules/root/corb/delete-docs-uris.xqy|ADHOC -DXCC-CONNECTION-URI=xcc://xxx:yyy@zzz:10140

I also tried:

task deleteDocs(type: com.marklogic.gradle.task.CorbTask, group: project.name) {
    //xccConnectionUri = "${CorbFromStaging}"
    //System.setProperty("XCC-CONNECTION-URI", "${CorbFromStaging}")
    environment "XCC-CONNECTION-URI", "${CorbFromStaging}"

But looks like Corb2 is not reading settings from environment. It would provide a safe way to pass through sensitive data though.

@rjrudin
Copy link
Contributor

rjrudin commented Aug 8, 2022

@grtjn I think you need to do the following to set a system property in the JVM that is forked off by Gradle when running CorbTask/JavaExec (this is copy/pasted from a DHF Gradle file):

task bootstrap(type: JavaExec) {
    classpath = sourceSets.test.runtimeClasspath
    main = 'com.marklogic.bootstrap.TestAppInstaller'
    systemProperty "mlHost", mlHost
}

I believe that should do the trick (this is one of those Gradle techniques that I can never remember how to do and always have to google for it).

I'm closing this in anticipation of the above working, and also that I think any other improvement will likely involve an enhancement to Corb and not ml-gradle (though I could envision an update to the Corb example project in this repo).

@rjrudin rjrudin closed this as completed Aug 8, 2022
@grtjn
Copy link
Author

grtjn commented Oct 5, 2022

systemProperty or System.setProperty doesn't make a difference, it is still printed in the full java.exe command. Probably, because it isn't forked, but run with some system execute.

I think this should really be using environment, rather than system properties. Then again, not sure if Corb2 itself supports that (yet), so likely needs a change there as well..

@rjrudin
Copy link
Contributor

rjrudin commented Oct 5, 2022

Ick. Reopening this as it's not resolved. @hansenmc Does the feedback from @grtjn sound correct to you - i.e. that Corb2 may need a change to accept environment properties in addition to system properties?

@rjrudin rjrudin reopened this Oct 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants