Documentation not accessible via public-repo by "doc" abbrev during build

[ghost]

@joewiz commented on Oct 5, 2018, 9:39 AM UTC:

Where is the problem

In the generated package's expath-pkg.xml file.

What is the problem

The same issue I fixed in eXist-db/shared-resources#37 affects the documentation app. Its @abbrev (in expath-pkg.xml under package/@abbrev) used to be "doc", but it is now "exist-documentation".

As a result, the build step during which xars are fetched for the autodeploy directory (https://github.com/eXist-db/exist/blob/develop/build.properties#L33), or calling ./build.sh download-xars (https://github.com/eXist-db/exist/blob/develop/installer/apps.properties#L2), will fail to download the documentation app. Here's the relevant portion of the log from ./build.sh download-xars:

download-xar:
    [fetch] Getting: http://demo.exist-db.org/exist/apps/public-repo/pkg.zip?abbrev=doc&zip=yes&processor=4.5.0-SNAPSHOT
    [fetch] To: /var/folders/ky/gdx8y19x3gj7xxjm1zr3wr_r0000gn/T/FetchTask8738793445563451693tmp
    [fetch] Error opening connection java.io.FileNotFoundException: http://demo.exist-db.org/exist/apps/public-repo/pkg.zip?abbrev=doc&zip=yes&processor=4.5.0-SNAPSHOT
    [fetch] Can't get http://demo.exist-db.org/exist/apps/public-repo/pkg.zip?abbrev=doc&zip=yes&processor=4.5.0-SNAPSHOT to /var/folders/ky/gdx8y19x3gj7xxjm1zr3wr_r0000gn/T/FetchTask8738793445563451693tmp

The public repo responds to this request by build script with a 404 error and <p>Package file not found!</p>.

Unfortunately I'm not sure how we should address this in the mavenized build process now used for the documentation app... But you might see how I distinguished between the package "app" name and "abbrev" name in build.properties, build.xml, and expath-pkg.xml.tmpl in my PR that fixed this (again) in the shared-resources app: https://github.com/eXist-db/shared-resources/pull/37/files.

Please provide the following

• exist-db version: 4.x.x-develop
• documentation version: 4.0.3

This issue was moved by duncdrum from eXist-db/documentation#269.

[ghost]

@duncdrum commented on Oct 5, 2018, 9:48 AM UTC:

so to keep this inline with the maven world which might come to exist in the near future, i would suggest fixing build.sh download-xars instead of changing the artifact-id in @abbbrev

While, I remember dizzzz having strong feelings about not wanting the docs on his exist-instance. Given that we haven't fixed the update cycle of exist-db.org and keep trailing I vote to revisit his objections once the exist-db.org update woes have been addressed.

/move eXist-db/exist

[joewiz]

We'd need 2 copies of the documentation app in the public-repo then, one an old one with the old abbrev, and a new one with the new abbrev. I think the public-repo uses package/@name as its key, though, not abbrev, so the newer abbrev will essentially prevent anyone from requesting the older one. This is a mess...

[duncdrum]

Yes it's a mess. See eXist-db/documentation#181

I m not following, we haven't shipped with an autodeployed docs app since moving to maven around exist 3.5.1, with the next release we start again. changing the artifact-id will just mean the mess spreads to the nexus maven repo.

Since moving to versioned releases I really can't think why anyone would want to request an old version via public-repo, there are other ways for getting the xars though.

[joewiz]

Current situation:

• App name: doc (i.e., the app is installed at /exist/apps/doc)
• Package abbrev name: exist-documentation
• Repo name: documentation

The public repo allows lookup of apps by (a) package abbrev name or (b) package URI:

• abbrev:
  • http://demo.exist-db.org/exist/apps/public-repo/packages/exist-documentation.html?eXist-db-min-version=4.3.0
  • http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql?abbrev=exist-documentation&processor=4.5.0-SNAPSHOT
  • http://demo.exist-db.org/exist/apps/public-repo/pkg.zip?abbrev=exist-documentation&zip=yes&processor=4.5.0-SNAPSHOT
• URI:
  • repo:install-and-deploy("http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql", "http://exist-db.org/apps/doc")

In my testing of the public-repo, the newest version of an app sets the abbrev value for it and all previous versions of the app. For example, notice that at http://demo.exist-db.org/exist/apps/public-repo/packages/exist-documentation.html?eXist-db-min-version=4.3.0, version 0.5.0 is called "doc-0.5.0.xar", whereas 0.6.0 is called "exist-documentation-0.6.0.xar". All "doc" packages are now only available via "exist-documentation" lookups. They cannot be accessed by "doc" though. So http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql?abbrev=exist-documentation works, but http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql?abbrev=doc returns a 404.

So until we either change the abbrev for this package back to "doc" or change the public repo to allow "legacy" abbrev names, nearly all abbrev-based build scripts will fail to fetch the documentation.

[duncdrum]

@joewiz Actually i think if we simply fix the abbrev value here, we should be fine, since the artifact-id stays the same.

https://github.com/eXist-db/exist-apps-parent/blob/ea5fe0d0c003975796bde1288410e1809a54b02d/pom.xml#L57

but we need to test if links between e.g. funcdoc and docs still work with a changes value as these are also abbrev based ifrc

[joewiz]

@duncdrum Oh good, I was hoping something like that would be possible. Changing the documentation packages' abbrev is definitely the shortest path to a fix.

If we ever decide to change any package's abbrev, we'll need to remember to update the app URI, to avoid the public-repo problem described above.

[joewiz]

Fixed by eXist-db/public-repo#32.