Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jakarta EE 10.0.0-RC1 can not be feched from Staging repo #1

Open
JanWesterkamp-iJUG opened this issue Feb 9, 2022 · 13 comments
Open

Jakarta EE 10.0.0-RC1 can not be feched from Staging repo #1

JanWesterkamp-iJUG opened this issue Feb 9, 2022 · 13 comments

Comments

@JanWesterkamp-iJUG
Copy link
Contributor

When the platform.artifact=jakarta.platform:jakarta.jakartaee-api:jar:10.0.0-RC1 (Jakarta EE 10 RC1) from the repo.url=https://jakarta.oss.sonatype.org/content/repositories/staging/ (Staging) is configured an empty result will be generated for the reports.

The Maven Artifact is available in the repo (https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/), but not fetched at all according to the search in the Maven log.

Further analysis using version 9.1.0-RC1 verifies this is not caused because of a root RC[1] version.
When not cleaning the data, multiple runs for different repos are possible to be aggregated - but this works only with the same Maven Artifact version (!) - when the version deviates, then additional data is not attached (this might be a separate issue).

There is a bug in the Nexus too, that might interfere here:
When opening the page https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/ the Last Modified column contains wrong timestamps for some of the meta data files!

Example:
Site (https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/) shows lines:
...
Name | Last Modified | Size | Description
...
maven-metadata.xml Mon Nov 02 17:03:41 UTC 2020 297
...

File maven-matedata.xml (https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/maven-metadata.xml) contains element:
...
20220209030723
...
@JanWesterkamp-iJUG
Copy link
Contributor Author

This issue was first reported by Scott in the mainling list here:
https://www.eclipse.org/lists/jakartaee-platform-dev/msg03246.html

@DirkMahler do you have any ideas where the root cause is located, in jQA or Nexus?
If Nexus has the problem, is there a possibility to have a (simple) workaround for this (i.e. parsing the meta data file element instead of the site) or do you have observations like this with the jQA Maven Plugin before?

@JanWesterkamp-iJUG JanWesterkamp-iJUG changed the title Jakarta EE 10.0.0-RC1 can not be feched form Staging repo Jakarta EE 10.0.0-RC1 can not be feched from Staging repo Feb 9, 2022
@DirkMahler
Copy link
Contributor

The issue most likely is related to an outdated index in the repository:

[INFO] Updating repository index, this may take a while...
...
[INFO] Updated Maven index timestamp: Tue Dec 21 20:44:22 CET 2021

So the artifact itself is present but cannot be found by the jQAssistant scanner.

@JanWesterkamp-iJUG
Copy link
Contributor Author

Thanks, I created an issue at Sonatype, where the Jakarta Staging is hosted, to get this fixed there: https://issues.sonatype.org/browse/OSSRH-78084

@starksm64
Copy link

It looks like the jakarta.oss.sonatype.org repo index has been updated, so I updated my clone with the latest changes and reran verify with 10.0.0-RC1 as the platform.version. I'm still seeing empty dependency diagrams produced:

[INFO] Using local repository '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/cache/jqassistant/m2repo/repo1.maven.org/maven2' for URL 'https://repo1.maven.org/maven2'
[INFO] Updating repository index, this may take a while...
[INFO] Received a full update.
[INFO] Updated Maven index timestamp: Tue Feb 15 13:15:26 CST 2022
[INFO] Executing query for artifacts that have been updated since Wed Dec 31 18:00:00 CST 1969.
[INFO] Artifact query returned 26683939 hits (total).
[INFO] Starting scan.
...
[INFO] Scanning model 'jakarta.authorization:jakarta.authorization-api:pom:2.1.0-RC1'.
[INFO] Processed 2000/26683939 artifacts (duration: PT10M34.338S).
[INFO] Processed 2500/26683939 artifacts (duration: PT10M34.62S).
[INFO] Processed 3000/26683939 artifacts (duration: PT10M34.896S).
[INFO] Finished scan: 3171 artifacts (duration: PT10M40.55S).
[INFO] --- jqassistant-maven-plugin:1.11.1:analyze (default-cli) @ jakarta-ee-dependencies ---
[INFO] Executing analysis for 'jakarta-ee-dependencies'.
[INFO] Will warn on violations starting from severity 'MINOR'
[INFO] Will fail on violations starting from severity 'MAJOR'.
[INFO] Reading rules from directory '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/jqassistant'.
[INFO] Executing group 'jakarta-ee-dependencies'
[INFO] Applying concept 'jakarta-ee-dependencies:ArtifactDependencies' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:ArtifactLastModifiedAsString' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:PlatformArtifact' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyDiagram' with severity: 'MINOR'.
[INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_PlatformDependencyDiagram.svg' 
[INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:Specification' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:SpecificationDependencyDiagram' with severity: 'MINOR'.
[INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_SpecificationDependencyDiagram.svg' 
[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:PlatformArtifact
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyDiagram
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates a CSV report about dependency versions.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:SpecificationDependencyDiagram
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates reports about dependencies between specifications as GraphML file and PlantUML component diagram.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  30:40 min
[INFO] Finished at: 2022-02-22T22:28:30-06:00
[INFO] ------------------------------------------------------------------------
[INFO] Closing connection to store 'file:/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/store/'.

@DirkMahler
Copy link
Contributor

DirkMahler commented Feb 23, 2022 via email

@JanWesterkamp-iJUG
Copy link
Contributor Author

JanWesterkamp-iJUG commented Feb 23, 2022 via email

@starksm64
Copy link

That was the issue, I had not uncommitted out the repo.url for the jakarta.oss.sonatype.org repo. I have generated the graph for EE10.

@JanWesterkamp-iJUG
Copy link
Contributor Author

It happend again! Instead of May 4th, the index points to April 14th...

[INFO] Current Maven index timestamp: Thu Apr 14 17:06:36 CEST 2022
[INFO] Updating repository index, this may take a while...
[INFO] Received an incremental update.
[INFO] Updated Maven index timestamp: Thu Apr 14 17:06:36 CEST 2022

I will have to knock Sonartype to rebuild the index.

@JanWesterkamp-iJUG
Copy link
Contributor Author

At the moment the 10.0.0 release will not be visible to jQA.

@JanWesterkamp-iJUG
Copy link
Contributor Author

Sonatype triggered the index creation manually (I created a 2nd ticket there https://issues.sonatype.org/browse/OSSRH-80749)

@JanWesterkamp-iJUG
Copy link
Contributor Author

I asked Sonatype to trigger the index creation manually. I created a new ticket there https://issues.sonatype.org/browse/OSSRH-80987 to speed up things.

@JanWesterkamp-iJUG
Copy link
Contributor Author

@DirkMahler Sonatype did a manual refesh of the index. It tryed it again to generate the report for version 10.0.0 of the (new) Core Profile (jakarta.platform:jakarta.jakartaee-core-api:10.0.0) to see the latest changes and saw, that the index still points to 2022-05-11 (!) and the reports are outdated. Then I deleted my local cache with a rerun showing this:

...
[INFO] Using local repository '/Users/jan/git/jakarta-ee-dependencies/cache/jqassistant/m2repo/jakarta.oss.sonatype.org/content/repositories/staging' for URL 'https://jakarta.oss.sonatype.org/content/repositories/staging/'
[INFO] Updating repository index, this may take a while...
[INFO] Received a full update.
[INFO] Updated Maven index timestamp: Wed May 11 22:08:26 CEST 2022
[INFO] Executing query for artifacts that have been updated since Thu Jan 01 01:00:00 CET 1970.
[INFO] Artifact query returned 87553 hits (total).
[INFO] Starting scan.
...
And now updated reports! I do not understand this, because the changes where done after the 11th of May for sure ;-)

Any ideas?

@DirkMahler
Copy link
Contributor

Just checked, current behavior is as follows:

  • the scanner always stores the timestamp of the last executed scan.
  • a subsequent scan executes queries the index for artifacts which are newer than that timestamp
  • after scanning those new artifacts the timestamp is update with the at-that-time "current" timestamp

Seems that is not an appropriate solution, taking the last index timestamp looks like being a better criteria.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@starksm64 @DirkMahler @JanWesterkamp-iJUG