Prospero user documentation

dist/docs/guide/channels/index.adoc

@@ -0,0 +1,194 @@
+## Working with channels
+
+More information on the Wildfly Channels can be found in https://github.com/wildfly-extras/wildfly-channel/blob/main/doc/spec.adoc. This chapter is intended to explain some concepts behind different channel types and how they can be used to provide software updates.
+
+A channel contains a collections of artifact streams used by a server. Each stream should contain only backwards-compatible artifact updates with new versions of artifacts replacing previous ones.
+
+There are two ways for a stream to determine the versions of its artifact:
+
+ * using an artifact manifest with fixed versions, or
+ * use the latest artifact version available in backing Maven repository.
+
+NOTE: The two mechanisms can be mixed within one channel, using static list to resolve some streams and Maven metadata for others.
+
+Channels using exclusively fixed version manifest can be more stable and make it possible to use 3rd party repositories. The artifact combination in the manifest can be tested before distributing, making sure there are no regressions.
+
+On the other hand channels using Maven metadata can make development and testing of new component updates easier. Any component updates deployed into the Maven repository are immediately made available to the subscribed servers.
+
+### Channels with fixed version manifests
+
+This kind of channels rely on manifests to list all available artifacts and their versions. This can be useful when the content of the repository cannot be fully controlled and might contain incompatible artifact versions.
+
+The manifest is a YAML file, containing a list of available streams:
+
+[source, yaml, title="manifest.yaml"]
+```
+schemaVersion: "1.0.0"
+name: "test-manifest"
+streams:
+  - groupId: "org.test"
+    artifactId: "artifact-one"
+    version: "1.2.0.Final"
+```
+
+The manifest has to be deployed in the channel repository and registered in the channel definition:
+
+[source, yaml, title="channel.yaml"]
+```
+schemaVersion: "2.0.0"
+name: "test-channel"
+resolve-if-no-stream: none #(1)
+repositories:
+  - id: "trusted"
+    url: "https://trusted.repository.org/maven/"
+manifest:
+  maven:
+    groupId: org.test.channel
+    artifactId: test-manifest
+```
+<1> this channel provides only artifacts explicitly listed in the manifest
+
+#### Updating components in fixed version channel
+
+Updating a component in a manifest channel requires publishing a new version of manifest.
+
+For example, if the `org.test:artifact-one` is updated to `1.2.1.Final` version, the new manifest would look like as follows:
+
+[source, yaml, title="manifest.yaml"]
+```
+schemaVersion: "1.0.0"
+name: "test-manifest"
+streams:
+  - groupId: "org.test"
+    artifactId: "artifact-one"
+    version: "1.2.1.Final"
+```
+
+This manifest has to be published in the channel repository with a new version:
+
+```
+mvn deploy:deploy-file -Dfile=manifest.yaml \
+-DgroupId=org.test.channels -DartifactId=test-manifest \
+-Dversion=1.0.1 \ #(1)
+-Dclassifier=manifest -Dpackaging=yaml \
+-Durl=https://trusted.repository.org/maven/
+```
+<1> note the updated version
+
+There are no changes required to the channel definition. Next time components are resolved from this channel, a new version of manifest will be used providing version *1.2.1.Final* of `org.test:artifact-one`.
+
+### Channels using backing Maven repository versions
+
+This type of channels expose the latest available versions of artifacts found in their repositories as the component version. The versions can be curated using either version patterns or a blocklist excluding specific versions.
+
+Channels based on Maven metadata can be useful when the content of repositories can be controlled and trusted. Any new artifact deployed into the channel repository, will automatically be made available for subscribed servers.
+
+An example of channel exposing all underlying artifacts can be seen below:
+
+[source, yaml, title="channel.yaml"]
+```
+schemaVersion: "2.0.0"
+name: "test-channel"
+resolve-if-no-stream: latest #(1)
+repositories: #(2)
+  - id: "trusted"
+    url: "https://trusted.repository.org/maven/"
+```
+<1> `latest` strategy means that if channel cannot find a stream matching the requested artifact, it will attempt to find the latest version in its repositories
+<2> the underlying repository for the channel
+
+#### Using version patterns to limit artifacts
+
+Sometimes the underlying repository might contain versions of artifacts that are not compatible with the installed server. For example the server might require a *1.2.x* version of a certain artifact, but the repository contains a newer version *2.0.x*.
+
+To filter incompatible versions, the channel can use a manifest file with `versionPattern` streams.
+
+[source, yaml, title="manifest.yaml"]
+```
+schemaVersion: "1.0.0"
+name: "test-manifest"
+streams:
+  - groupId: "org.test"
+    artifactId: "artifact-one"
+    versionPattern: "1\\.2\\..*" #(1)
+```
+<1> a https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html[regular expression] limiting versions to ones starting with `1.2.`
+
+NOTE: The `versionPattern` field uses regular expressions
+
+The above manifest file should be deployed in the channel repository with a classifier `manifest` and extension `yaml`. For example following command will deploy the file as `org.test.channels:test-manifest:1.0.0`
+
+```
+mvn deploy:deploy-file -Dfile=manifest.yaml \
+  -DgroupId=org.test.channels -DartifactId=test-manifest \
+  -Dversion=1.0.0 -Dclassifier=manifest -Dpackaging=yaml \
+  -Durl=https://trusted.repository.org/maven/
+```
+
+The channel definition needs to be updated to reference the new manifest file:
+
+[source, yaml, title="channel.yaml"]
+```
+schemaVersion: "2.0.0"
+name: "test-channel"
+resolve-if-no-stream: latest
+repositories:
+  - id: "trusted"
+    url: "https://trusted.repository.org/maven/"
+manifest:
+  maven:
+    groupId: org.test.channel
+    artifactId: test-manifest
+```
+
+Using this channel definition, all artifacts apart from `org.test:artifact-one` are still resolved to the latest versions available in the repository. The `org.test:artifact-one` will be resolved to the latest available "1.2.x" micro version. For example, if the repository contains versions [*1.2.0.Final*, *1.2.1.Final*, *2.0.0.Final*], the channel will pick version *1.2.1.Final*.
+
+#### Creating blocklist to exclude updates
+
+Another option to exclude certain artifact versions is to use a blocklist. A blocklist is a YAML file deployed in the channel repository listing blocked artifact versions.
+
+[source, yaml, title="blocklist.yaml"]
+```
+schemaVersion: "1.0.0"
+name: "test-blocklist"
+blocks:
+  - groupId: "org.test"
+    artifactId: "artifact-one"
+    versions:
+    - "1.2.2.Final"
+```
+
+Again, the blocklist has to be deployed in the channel repository. The blocklist artifact has to use `blocklist` classifier and `yaml` extension. For example:
+
+```
+mvn deploy:deploy-file -Dfile=blocklist.yaml \
+  -DgroupId=org.test.channels -DartifactId=test-blocklist \
+  -Dversion=1.0.0 -Dclassifier=blocklist -Dpackaging=yaml \
+  -Durl=https://trusted.repository.org/maven/
+```
+
+Finally, the channel definition has to be updated with the reference to the blocklist:
+
+[source, yaml, title="channel.yaml"]
+```
+schemaVersion: "2.0.0"
+name: "test-channel"
+resolve-if-no-stream: latest
+blocklist:
+  maven:
+    groupId: org.test.channel
+    artifactId: test-blocklist
+repositories:
+  - id: "trusted"
+    url: "https://trusted.repository.org/maven/"
+manifest:
+  maven:
+    groupId: org.test.channel
+    artifactId: test-manifest
+```
+
+Resolving `org.test:artifact-one` from this channel will exclude any versions not matching "1.2.*" pattern and version 1.2.2.Final. For example, if the repository contains versions [*1.2.0.Final*, *1.2.1.Final*, *1.2.2.Final*, *2.0.0.Final*], the channel will pick version *1.2.1.Final*.
+
+#### Updating components using "open" channel
+
+Updating component in an open channel requires only deploying the artifact into the channel repository. Neither channel definition not channel manifest has to be changed. Next time components are resolved from this channel, a new version of updated component will be used.

dist/docs/guide/concepts/channels.adoc

@@ -0,0 +1,71 @@
+### Channels
+
+Wildfly Channels are a way to provide a curated list of compatible, up-to-date components. A channel consists of a YAML definition file and one or more underlying Maven repositories.
+
+There are two types of Wildfly Channels supported by Prospero - open channels and manifest channels. Open channels allow access to the latest artifacts in the underlying repository, while manifest channels define a list of allowed list of components.
+
+For more information on Wildfly Channels see the https://github.com/wildfly-extras/wildfly-channel/blob/main/doc/spec.adoc[spec documentation].
+
+#### Manifest channels
+
+One way to specify channel content is by using manifests. Manifests are YAML file listing streams available in the channel. For example a part of wildfly-27.0.0 manifest looks like:
+
+[source, yaml]
+----
+schemaVersion: "1.0.0"
+name: Manifest for WildFly 27
+description: |-
+  This manifest provides updates for WildFly 27 Feature Pack.
+streams:
+  - groupId: "org.wildfly"
+    artifactId: "wildfly-galleon-pack"
+    version: "27.0.0.Final"
+
+  - groupId: "org.wildfly"
+    artifactId: "wildfly-ee-galleon-pack"
+    version: "27.0.0.Final"
+
+  - groupId: "org.wildfly.core"
+    artifactId: "wildfly-core-galleon-pack"
+    version: "19.0.0.Beta13"
+
+  - groupId: "org.wildfly.core"
+    artifactId: "wildfly-version"
+    version: "19.0.0.Final"
+
+  - groupId: "org.wildfly.security"
+    artifactId: "wildfly-elytron-jaspi"
+    version: "1.20.2.Final"
+----
+
+The channel using this manifest would be defined as:
+[source, yaml]
+----
+schemaVersion: "2.0.0"
+name: Channel for WildFly 27
+manifest:
+  maven:
+    groupId: org.wildfly.channels
+    artifactId: wildfly-27.0
+repositories:
+  - id: central
+    url: https://repo1.maven.org/maven2/
+----
+
+When resolving artifacts using such channel, the versions of artifacts will be dictated by the manifest, even if newer version is available in the channel repositories.
+
+NOTE: The manifest can be distributed as either a static file (referenced in a channel by a URL), or as an artifact in a Maven repository.
+
+#### Open channels
+
+Alternatively the channel can be defined to use the latest versions of artifacts available in its repositories.
+
+[source, yaml]
+----
+schemaVersion: "2.0.0"
+name: Channel for WildFly 27
+resolve-if-no-stream: latest
+repositories:
+  - id: central
+    url: https://repo1.maven.org/maven2/
+----

dist/docs/guide/concepts/galleon.adoc

@@ -0,0 +1,9 @@
+### Feature Packs
+
+Galleon is a provisioning tool used to assemble Wildfly distributions. In order for a piece of software to be provisioned by Galleon it needs to be packaged as a feature-pack.
+
+Feature packs are ZIP archives usually available in Maven repositories. They contain information about installed software's filesystem content, configuration and additional tasks needed to assemble and configure it.
+
+In case of Wildfly server, the feature pack also contains a list of Maven coordinates for all the components included in the server distribution. During provisioning, those coordinates are used to download the components and place them in resulting server.
+
+For more information on Galleon and the Feature Packs see the https://docs.wildfly.org/galleon/#_feature_packs[Galleon documentation]

dist/docs/guide/concepts/index.adoc

@@ -0,0 +1,11 @@
+## Concepts
+
+The Wildfly servers are assembled from a large number of dependencies from various projects. If any of those components is changed, in the traditional update process, the server has to be rebuilt with updated components and distributed as a new version.
+
+Prospero aims to simplify this process by allowing users to apply updated components to already provisioned servers.
+
+To achieve that, Prospero utilizes two projects - Galleon and Wildfly Channels.
+
+include::galleon.adoc[]
+
+include::channels.adoc[]

dist/docs/guide/customization/apply-customization.adoc

@@ -1,6 +1,6 @@
 ### Apply changes to the server
 
-After promoting artifacts to the channel, the changes can be applied to targeted servers.
+After promoting artifacts to the channel, the changes can be applied to subscribed servers.
 
 Promoted changes are treated as any updates, so `prospero update` command can be used to apply them.
 

dist/docs/guide/customization/customization-bundle.adoc

@@ -1,10 +1,10 @@
 ### Distributing custom artifacts
 
-The customized artifacts can be distributed as bundles. Each bundle is a ZIP archive containing an `artifact-list.yaml` file and a `repository` directory.
+The customized artifacts can be distributed as bundles. Each bundle is a ZIP archive containing a `maven-repository` directory and an optional `artifact-list.yaml` file.
 
 ```
 |-artifact-list.yaml
-|-repository
+|-maven-repository
   |- org
     |-custom
       |-artifact-one
@@ -13,8 +13,30 @@ The customized artifacts can be distributed as bundles. Each bundle is a ZIP arc
       |-artifact-two
         |-2.3.4
           |-artifact-two-2.3.4.jar
+    |-wildfly
+      |-prospero
+        |-customizations
+          |-1.0.0
+            |-customizations-manifest-1.0.0.yaml
+          |-maven-metadata.xml
 ```
 
+In addition to the custom artifacts, repository has to contain a channel manifest. The manifest should list all streams available in the customization bundle.
+
+```
+schemaVersion: 1.0.0
+name: Customizations manifest
+streams:
+  - groupId: org.custom
+    artifactId: artifact-one
+    version: 1.2.3
+  - groupId: org.custom
+    artifactId: artifact-two
+    version: 2.3.4
+```
+
+NOTE: the Maven metadata has to be present for the manifest artifact.
+
 The `artifact-list.yaml` lists all artifacts included in the bundle using following syntax:
 ```
 artifacts:
@@ -23,4 +45,48 @@ artifacts:
     version: "1.2.3"
     packaging: "jar"
     extension: ""
-```
+```
+
+#### Updating custom artifacts
+
+If a new version of customization bundle needs to be distributed, it should contain a customization manifest deployed under the same `groupId:artifactId` coordinates, but with an incremented version.
+
+The repository should contain all customized artifacts required by the updated manifest. For instance, the updated manifest included a new version of `artifact-one` and added `artifact-three`, resulting in following file:
+
+```
+schemaVersion: 1.0.0
+name: Customizations manifest
+streams:
+  - groupId: org.custom
+    artifactId: artifact-one
+    version: 1.2.4
+  - groupId: org.custom
+    artifactId: artifact-two
+    version: 2.3.4
+  - groupId: org.custom
+    artifactId: artifact-three
+    version: 3.4.5
+```
+
+The updated repository should contain three artifacts:
+```
+|-artifact-list.yaml
+|-maven-repository
+  |- org
+    |-custom
+      |-artifact-one
+        |-1.2.4
+          |-artifact-one-1.2.4.jar
+      |-artifact-two
+        |-2.3.4
+          |-artifact-two-2.3.4.jar
+      |-artifact-three
+        |-3.4.5
+          |-artifact-two-3.4.5.jar
+    |-wildfly
+      |-prospero
+        |-customizations
+          |-1.0.1
+            |-customizations-manifest-1.0.1.yaml
+          |-maven-metadata.xml
+```