Using a GPG secret key without a private key included causes a NullPointerException when uploading an artifact

[wdschei]

Thanks for creating an issue! Please fill out this form so we can be
sure to have all the information we need, and to minimize back and forth.

• What are you trying to do?
  Upload an artifact to a new NXRM.

• What feature or behavior is this required for?
  Using the plugin.

• How could we solve this issue? (Not knowing is okay!)
  I have no idea what is actually wrong.

• Anything else?

I'm standing up a new server hosting a NXRM to serve Debian Apt artifacts and have installed/built everything so far using the basic installs.

I'm using:

• nexus-3.14.0-04
• nexus-repository-apt-1.0.9

Located at:

• /opt/nexus/nexus-3.14.0-04/nexus-repository-apt/target/nexus-repository-apt-1.0.9.jar

I have confirmed the NXRM process is using OpenJDK:

• /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java

I have confirmed the Unlimited JCE is in effect using the methods described here.

Specifically the Gist here

I have confirmed my Secret Key and Passphrase are correct using the method described here.

I received the log messages below after creating an Apt repository named debian and trying to upload a file using the UI at:

• https://nexus_url/#browse/upload:debian

2019-01-21 19:29:08,630+0000 INFO  [qtp2143167841-48]  admin org.sonatype.nexus.repository.manager.internal.RepositoryManagerImpl - Creating repository: debian -> Configuration{repositoryName='debian', recipeName='apt-hosted', attributes={apt={distribution=stable}, aptHosted={assetHistoryLimit=null}, aptSigning={keypair=-----BEGIN PGP PRIVATE KEY BLOCK-----
Version: GnuPG v1

REDACTED
-----END PGP PRIVATE KEY BLOCK-----, passphrase=REDACTED}, storage={blobStoreName=default, strictContentTypeValidation=true, writePolicy=ALLOW_ONCE}}}
2019-01-21 19:30:40,830+0000 INFO  [qtp2143167841-48]  admin org.sonatype.nexus.coreui.internal.UploadService - Uploading component with parameters: repository="debian" format="apt"
2019-01-21 19:30:40,830+0000 INFO  [qtp2143167841-48]  admin org.sonatype.nexus.coreui.internal.UploadService - Asset with parameters: file="valve-8.10.0.0.deb"
2019-01-21 19:30:48,272+0000 ERROR [qtp2143167841-48]  admin org.sonatype.nexus.extdirect.internal.ExtDirectServlet - Failed to invoke action method: coreui_Upload.doUpload, java-method: org.sonatype.nexus.coreui.UploadComponentComponent.doUpload
java.lang.NullPointerException: null
	at org.bouncycastle.openpgp.operator.jcajce.JcaPGPContentSignerBuilder.build(Unknown Source)
	at org.bouncycastle.openpgp.PGPSignatureGenerator.init(Unknown Source)
	at net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet.signInline(AptSigningFacet.java:112)
	at net.staticsnow.nexus.repository.apt.internal.hosted.AptHostedFacet.rebuildIndexesInTransaction(AptHostedFacet.java:201)
	at net.staticsnow.nexus.repository.apt.internal.hosted.AptHostedFacet.ingestAsset(AptHostedFacet.java:158)
	at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:45)
	at net.staticsnow.nexus.repository.apt.internal.hosted.AptHostedFacet.ingestAsset(AptHostedFacet.java:124)
	at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:45)
	at net.staticsnow.nexus.repository.apt.AptUploadHandler.lambda$handle$3(AptUploadHandler.java:77)
	at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64)
	at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56)
	at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:200)
	at org.sonatype.nexus.transaction.Operations.call(Operations.java:146)
	at net.staticsnow.nexus.repository.apt.AptUploadHandler.handle(AptUploadHandler.java:64)
	at org.sonatype.nexus.repository.upload.internal.UploadManagerImpl.handle(UploadManagerImpl.java:71)
	at org.sonatype.nexus.coreui.internal.UploadService.upload(UploadService.java:103)
	at org.sonatype.nexus.coreui.internal.UploadService$upload$0.call(Unknown Source)
	at org.sonatype.nexus.coreui.UploadComponentComponent.doUpload(UploadComponentComponent.groovy:73)
	at com.palominolabs.metrics.guice.ExceptionMeteredInterceptor.invoke(ExceptionMeteredInterceptor.java:49)
	at com.palominolabs.metrics.guice.TimedInterceptor.invoke(TimedInterceptor.java:47)
	at org.sonatype.nexus.validation.internal.ValidationInterceptor.invoke(ValidationInterceptor.java:53)
	at org.apache.shiro.guice.aop.AopAllianceMethodInvocationAdapter.proceed(AopAllianceMethodInvocationAdapter.java:49)
	at org.apache.shiro.authz.aop.AuthorizingAnnotationMethodInterceptor.invoke(AuthorizingAnnotationMethodInterceptor.java:68)
	at org.apache.shiro.guice.aop.AopAllianceMethodInterceptorAdapter.invoke(AopAllianceMethodInterceptorAdapter.java:36)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.invokeJavaMethod(DispatcherBase.java:142)
	at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.invokeMethod(DispatcherBase.java:133)
	at org.sonatype.nexus.extdirect.internal.ExtDirectServlet$3.invokeMethod(ExtDirectServlet.java:281)
	at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.dispatch(DispatcherBase.java:63)
	at com.softwarementors.extjs.djn.router.processor.standard.StandardRequestProcessorBase.dispatchStandardMethod(StandardRequestProcessorBase.java:73)
	at com.softwarementors.extjs.djn.router.processor.standard.form.FormPostRequestProcessorBase.processRequest(FormPostRequestProcessorBase.java:125)
	at com.softwarementors.extjs.djn.router.processor.standard.form.FormPostRequestProcessorBase.process(FormPostRequestProcessorBase.java:64)
	at com.softwarementors.extjs.djn.router.processor.standard.form.upload.UploadFormPostRequestProcessor.process(UploadFormPostRequestProcessor.java:76)
	at com.softwarementors.extjs.djn.router.RequestRouter.processUploadFormPostRequest(RequestRouter.java:79)
	at com.softwarementors.extjs.djn.servlet.DirectJNgineServlet.processUploadFormPost(DirectJNgineServlet.java:662)
	at com.softwarementors.extjs.djn.servlet.DirectJNgineServlet.processRequest(DirectJNgineServlet.java:628)
	at com.softwarementors.extjs.djn.servlet.DirectJNgineServlet.doPost(DirectJNgineServlet.java:595)
	at org.sonatype.nexus.extdirect.internal.ExtDirectServlet.doPost(ExtDirectServlet.java:155)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
	at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
	at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
	at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
	at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
	at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
	at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
	at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
	at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
	at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
	at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:531)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
	at java.lang.Thread.run(Thread.java:748)

Just to ensure this isn't a UI issue, I used the curl example in the README and received:

2019-01-21 19:38:35,177+0000 WARN  [qtp2143167841-50]  admin org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Failure servicing: POST /repository/debian/
java.lang.NullPointerException: null
	at org.bouncycastle.openpgp.operator.jcajce.JcaPGPContentSignerBuilder.build(Unknown Source)
	at org.bouncycastle.openpgp.PGPSignatureGenerator.init(Unknown Source)
	at net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet.signInline(AptSigningFacet.java:112)
	...

Digging through the stack, I see that the following were hit:

• https://github.com/sonatype-nexus-community/nexus-repository-apt/blob/1.0.9/src/main/java/net/staticsnow/nexus/repository/apt/internal/gpg/AptSigningFacet.java#L112
• https://github.com/bcgit/bc-java/blob/r1v60/pg/src/main/java/org/bouncycastle/openpgp/PGPSignatureGenerator.java#L59
• https://github.com/bcgit/bc-java/blob/r1v60/pg/src/main/java/org/bouncycastle/openpgp/operator/jcajce/JcaPGPContentSignerBuilder.java#L77

This leads me to believe the private key extracted here was null:

Which leads me to believe this was returned here.

To confirm this, I applied this patch:

diff --git a/src/main/java/net/staticsnow/nexus/repository/apt/internal/gpg/AptSigningFacet.java b/src/main/java/net/staticsnow/nexus/repository/apt/internal/gpg/AptSigningFacet.java
index daf8edf..dde3382 100644
--- a/src/main/java/net/staticsnow/nexus/repository/apt/internal/gpg/AptSigningFacet.java
+++ b/src/main/java/net/staticsnow/nexus/repository/apt/internal/gpg/AptSigningFacet.java
@@ -105,8 +105,24 @@ public class AptSigningFacet
 
   public byte[] signInline(String input) throws IOException, PGPException {
     PGPSecretKey signKey = readSecretKey();
+    if (signKey.isPrivateKeyEmpty()) {
+      log.warn("The Secret Key is empty?!?!");
+      log.warn(" - isSigningKey\t\t\t: {}", signKey.isSigningKey());
+      log.warn(" - isMasterKey\t\t\t: {}", signKey.isMasterKey());
+      log.warn(" - getKeyEncryptionAlgorithm\t: {}", signKey.getKeyEncryptionAlgorithm());
+      log.warn(" - getKeyID\t\t\t: {}", signKey.getKeyID());
+      log.warn(" - getS2KUsage\t\t\t: {}", signKey.getS2KUsage());
+      Iterator<String> userIds = signKey.getUserIDs();
+      while(userIds.hasNext())
+      {
+        log.warn(" - userId\t\t\t: {}", userIds.next());
+      }
+    }
     PGPPrivateKey privKey = signKey.extractPrivateKey(
         new JcePBESecretKeyDecryptorBuilder().setProvider("BC").build(config.passphrase.toCharArray()));
+    if (privKey == null) {
+      log.warn("Failed to extract a private key from the secret key using configured passphrase.");
+    }
     PGPSignatureGenerator sigGenerator = new PGPSignatureGenerator(
         new JcaPGPContentSignerBuilder(signKey.getPublicKey().getAlgorithm(), PGPUtil.SHA256).setProvider("BC"));
     sigGenerator.init(PGPSignature.CANONICAL_TEXT_DOCUMENT, privKey);

and then received this in the log:

2019-01-21 20:33:52,371+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet - The Secret Key is empty?!?!
2019-01-21 20:33:52,372+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet -  - isSigningKey                 : true
2019-01-21 20:33:52,372+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet -  - isMasterKey                  : true
2019-01-21 20:33:52,372+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet -  - getKeyEncryptionAlgorithm    : 3
2019-01-21 20:33:52,373+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet -  - getKeyID                     : -2872208579159300776
2019-01-21 20:33:52,373+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet -  - getS2KUsage                  : 254
2019-01-21 20:33:52,373+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet -  - userId                       : Repose Dev (http://openrepose.org) <[email protected]>
2019-01-21 20:33:52,378+0000 WARN  [qtp2143167841-49]  admin net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet - Failed to extract a private key from the secret key using configured passphrase.
2019-01-21 20:33:52,440+0000 ERROR [qtp2143167841-49]  admin org.sonatype.nexus.extdirect.internal.ExtDirectServlet - Failed to invoke action method: coreui_Upload.doUpload, java-method: org.sonatype.nexus.coreui.UploadComponentComponent.doUpload
java.lang.NullPointerException: null
        at org.bouncycastle.openpgp.operator.jcajce.JcaPGPContentSignerBuilder.build(Unknown Source)
        at org.bouncycastle.openpgp.PGPSignatureGenerator.init(Unknown Source)
        at net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningFacet.signInline(AptSigningFacet.java:128)
	...

What would cause the extracted private key to be empty?

[wdschei]

Update. This appears to be an issue with GPG keys with subkeys.
When a key without a subkey was configured, then everything worked great.
I am still really curious about why the original key, which was valid, didn't work.
I am also concerned that I was presented with a NullPointerException in the UI instead of a meaningful message that my configured signing key had an issue.

[mpoindexter]

Thanks for the detailed info!

I wonder if at some point you removed the master private key from your keyring (like is suggested here: https://wiki.debian.org/Subkeys). Or perhaps gpg requires additional incantations to export the private portion of a master key.

As far as why the NPE instead of a meaningful message: this is simply not a case that I've encountered before and didn't realize was possible. This plugin is not a Sonatype product, it's a volunteer effort, so it can definitely lack some of the polish that a commercial product might have. Personally I don't have a great amount of time to devote to this plugin, so I probably won't get around to writing a fix to handle this case in a less confusing way any time soon, but I'm happy to review and merge patches!

[wdschei]

@mpoindexter I'll take a look at the subkeys thing and see what is going on there. We found that we actually didn't need to use it, but I would like to know what is going on there.

I fully understand the lack of polish and how only what is currently painful is what bubbles to the top. My team is currently maintaining a couple of Gradle plugins ourselves.

I'm going through the hoops on my end to get approval to sign the Sonatype Contributor License Agreement and submit a PR. It should be inbound this week.

[macalinao]

@wdschei do you have any updates on this?

[wdschei]

@macalinao I finally got approval and will be submitting a PR to provide a better message other than the NPE shortly.

[szhem]

Just passing by ...
I've recently submitted a PR that fixes a similar issue in Gradle.
Here it is: gradle/gradle#10366.
The main issue was that the keyring (multiple keys) was treated as a single key.

[bhamail]

APT is now part of Nexus Repository Manager. Version 3.17.0 includes the APT plugin by default.
If this is still an issue if using 3.17.0 or later please file an issue at https://issues.sonatype.org/.
Links to the new source code location are in the top level README.md