Preliminary HTCondor-CE 23 docs

docs/architecture.md

@@ -54,7 +54,7 @@ owners that want to start contributing to a computing grid with minimal effort.
 ![HTCondor-CE-Bosco](img/bosco.png)
 
 If your site intends to run over 10,000 concurrent pilot jobs, you will need to host your own
-[HTCondor-CE](v6/installation/htcondor-ce.md) because the Hosted CE has not yet been optimized for such loads.
+[HTCondor-CE](v23/installation/htcondor-ce.md) because the Hosted CE has not yet been optimized for such loads.
 
 How the CE is Customized
 ------------------------
@@ -63,11 +63,11 @@ Aside from the [basic configuration] required in the CE installation, there are
 you decide any customization is required at all):
 
 -   **Deciding which Virtual Organizations (VOs) are allowed to run at your site:** HTCondor-CE leverages HTCondor's
-    built-in ability to [authenticate incoming jobs](v6/configuration/authentication.md) based on their OAuth
+    built-in ability to [authenticate incoming jobs](v23/configuration/authentication.md) based on their OAuth
     token credentials.
 -   **How to filter and transform the pilot jobs to be run on your batch system:** Filtering and transforming pilot jobs
     (i.e., setting site-specific attributes or resource limits), requires configuration of your site’s job routes.
-    For examples of common job routes, consult the [job router configuration](v6/configuration/job-router-overview.md)
+    For examples of common job routes, consult the [job router configuration](v23/configuration/job-router-overview.md)
     pages.
 
 How Security Works

docs/index.md

@@ -40,15 +40,15 @@ Benefits of running the HTCondor-CE:
 
 -   **Scalability:** HTCondor-CE is capable of supporting ~16k concurrent RARs
 -   **Debugging tools:** HTCondor-CE offers
-    [many tools to help troubleshoot](v6/troubleshooting/debugging-tools.md) issues with RARs
+    [many tools to help troubleshoot](v23/troubleshooting/debugging-tools.md) issues with RARs
 -   **Routing as configuration:** HTCondor-CE’s mechanism to transform and submit RARs is customized via configuration
     variables, which means that customizations will persist across upgrades and will not involve modification of
     software internals to route jobs
 
 Getting HTCondor-CE
 -------------------
 
-Learn how to get and install HTCondor-CE through our [documentation](v6/installation/htcondor-ce.md).
+Learn how to get and install HTCondor-CE through our [documentation](v23/installation/htcondor-ce.md).
 
 Contact Us
 ----------

docs/v5/configuration/authentication.md → docs/v23/configuration/authentication.md

@@ -1,13 +1,8 @@
 Configuring Authentication
 ==========================
 
-To authenticate job submission from external users and VOs, HTCondor-CE can be configured to use
-[built-in mapfiles](#built-in-mapfiles) or to make [Globus callouts](#globus-callout) to an external service like Argus
-or LCMAPS.
-The former option is simpler but the latter option may be preferred if your grid supports it or your site already runs
-such a service.
-
-Additionally, the HTCondor-CE service uses [X.509 certificates](#configuring-certificates) for SciTokens, SSL, and GSI
+To authenticate job submission from external users and VOs,
+the HTCondor-CE service uses [X.509 certificates](#configuring-certificates) for SciTokens and SSL
 authentication.
 
 Built-in Mapfiles
@@ -44,56 +39,10 @@ in `/etc/condor-ce/mapfiles.d/`:
 SCITOKENS /^https:\/\/scitokens.org\/osg-connect,.*/ osg
 ```
 
-### GSI ###
-
-To allow clients with GSI proxies with to submit jobs to your HTCondor-CE, add lines of the following format:
-
-```
-GSI /^<DISTINGUISHED NAME>$/ <USERNAME>
-```
-
-Replacing `<DISTINGUISHED NAME>` (escaping any `/` with `\/`) and `<USERNAME>` with the distinguished name of the
-incoming certificate and the unix account under which the job should run, respectively.
-VOMS attributes of incoming X.509 proxy certificates can also be used for mapping:
-
-```
-GSI /<DISTINGUISHED NAME>,<VOMS FQAN 1>,<VOMS FQAN 2>,...,<VOMSFQAN N>/ <USERNAME>
-```
-
-Replacing `<DISTINGUISHED NAME>` (escaping any `/` with `\/`), `<VOMSFQAN>` fields, and `<USERNAME>` with the
-distinguished name of the incoming certificate, the VOMS roles and groups, and the unix account under which the job
-should run, respectively.
-For example, to map any certificate from the `GLOW` VO with the `htpc` role to the `glow` user, add the following line
-to a `*.conf` file in `/etc/condor-ce/mapfiles.d/`:
-
-```
-GSI /.*,\/GLOW\/Role=htpc.*/ glow
-```
-
-Globus Callout
---------------
-
-To use a Globus callout to a service like LCMAPS or Argus, you will need to have the relevant library installed as well
-as the following HTCondor-CE configuration:
-
-1. Add the following line to the top of `/etc/condor-ce/condor_mapfile`:
-
-        GSI /(.*)/ GSS_ASSIST_GRIDMAP
-
-1. Create `/etc/grid-security/gsi-authz.conf` with the following content:
-
-    - For LCMAPS:
-
-            globus_mapping liblcas_lcmaps_gt4_mapping.so lcmaps_callout
-
-    - For Argus:
-
-            globus_mapping /usr/lib64/libgsi_pep_callout.so argus_pep_callout
-
 Configuring Certificates
 ------------------------
 
-HTCondor-CE uses X.509 host certificates and certificate authorities (CAs) when authenticating SciToken, SSL, and GSI
+HTCondor-CE uses X.509 host certificates and certificate authorities (CAs) when authenticating SciToken and SSL
 connections.
 By default, HTCondor-CE uses the default system locations to locate CAs and host certificate when authenticating
 SciToken and SSL connections.

docs/v5/configuration/job-router-overview.md → docs/v23/configuration/job-router-overview.md

@@ -40,6 +40,14 @@ in the following order:
 
 ### Deprecated syntax ###
 
+!!! warning "Planned Removal of Deprecated Syntax"
+    -   `JOB_ROUTER_DEFAULTS`, `JOB_ROUTER_ENTRIES`, `JOB_ROUTER_ENTRIES_CMD`, and `JOB_ROUTER_ENTRIES_FILE` are
+    deprecated and will be removed for *V24* of the HTCondor Software Suite. New configuration syntax for the job router
+    is defined using `JOB_ROUTER_ROUTE_NAMES` and `JOB_ROUTER_ROUTE_[name]`.
+    -   For new syntax example vist:
+    [HTCondor Documentation - Job Router](https://htcondor.readthedocs.io/en/latest/grid-computing/job-router.html#an-example-configuration)
+    -   **Note:** The removal will occur during the lifetime of the HTCondor *V23* feature series.
+
 Since the inception of HTCondor-CE, job routes have been written as a
 [list of ClassAds](https://htcondor.readthedocs.io/en/lts/grid-computing/job-router.html#deprecated-router-configuration).
 Each job route’s [ClassAd](http://research.cs.wisc.edu/htcondor/manual/v8.6/4_1HTCondor_s_ClassAd.html) is constructed

docs/v5/configuration/optional-configuration.md → docs/v23/configuration/optional-configuration.md

@@ -54,6 +54,81 @@ configuring them in unison.
         START_LOCAL_UNIVERSE = False
         START_SCHEDULER_UNIVERSE = $(START_LOCAL_UNIVERSE)
 
+Inserting IDTOKENs into the routed job's sandbox
+------------------------------------------
+
+If you want to insert IDTOKENS into the routed job's sandbox you can use the `SendIDTokens` route command, or
+the `JOB_ROUTER_SEND_ROUTE_IDTOKENS` global configuration variable. Tokens
+sent using this mechanism must be named and declared using the `JOB_ROUTER_CREATE_IDTOKEN_NAMES`
+and [`JOB_ROUTER_CREATE_IDTOKEN_<name>`](https://htcondor.readthedocs.io/en/latest/admin-manual/configuration-macros.html#JOB_ROUTER_CREATE_IDTOKEN_%3CNAME%3E) configuration variables.  Tokens whose names are declared in
+the `JOB_ROUTER_SEND_ROUTE_IDTOKENS` configuration variable are sent by default for each route that does
+not have a `SendIDTokens` command.
+
+- **To declare IDTOKENS for inclusion in glide-in jobs** for the purpose of advertising to a collector
+  add something like the following to `/etc/condor-ce/config.d/99-local-ce-token.conf`:
+
+        JOB_ROUTER_CREATE_IDTOKEN_NAMES = name1 name2
+        JOB_ROUTER_CREATE_IDTOKEN_name1 @=end
+            sub = "[email protected]"
+            kid = "POOL"
+            lifetime = 3900
+            scope = "ADVERTISE_STARTD, ADVERTISE_MASTER, READ"
+            dir = "/etc/condor-ce/gltokens/name1"
+            filename = "ce_name1.idtoken"
+            owner = "owner1"
+        @end
+        JOB_ROUTER_CREATE_IDTOKEN_Name2 @=end
+            sub = "[email protected]"
+            kid = "POOL"
+            lifetime = 3900
+            scope = "ADVERTISE_STARTD, ADVERTISE_MASTER, READ"
+            dir = "/etc/condor-ce/gltokens/name2"
+            filename = "ce_name2.idtoken"
+            owner = "owner2"
+        @end
+
+- **To insert one of the above IDTOKENS in the sandbox of a routed job**, include the token name in the `SendIDTokens` route
+   command like this.
+
+        SendIDTokens = "Name2"
+    !!! note "Route commands"
+        `SendIDTokens` is a route command, not a job attribute.
+        This means that you will not be able to manipulate it through
+        [transform verbs](writing-job-routes.md#editing-attributes) such as `EVALSET`.
+  **To add an IDTOKEN to a routed job in addition to the default tokens**, build a string containing the token name
+   along with the value of the global configuration variable like this.
+
+        SendIDTokens = "Name2 $(JOB_ROUTER_SEND_ROUTE_IDTOKENS)"
+
+  **You can use an attribute of the source job** to choose the IDTOKEN by writing an expression like this.
+
+        SendIDTokens = strcat( My.Owner, " $(JOB_ROUTER_SEND_ROUTE_IDTOKENS)")
+
+  It is presumed that the value of `My.Owner` above is the same as the `<name>` of an IDTOKEN and as the `owner` field
+  of that token.  For instance, the Fermilab CE config uses the above `SendIDTokens` expression and
+  the following token declarations at the time of this guide.
+
+        JOB_ROUTER_CREATE_IDTOKEN_NAMES = fermilab3 osg
+        JOB_ROUTER_CREATE_IDTOKEN_fermilab3 @=end
+            sub = "[email protected]"
+            kid = "POOL"
+            lifetime = 3900
+            scope = "ADVERTISE_STARTD, ADVERTISE_MASTER, READ"
+            dir = "/etc/condor-ce/gltokens/fermilab"
+            filename = "ce_fermilab3.idtoken"
+            owner = "fermilab"
+        @end
+        JOB_ROUTER_CREATE_IDTOKEN_osg @=end
+            sub = "[email protected]"
+            kid = "POOL"
+            lifetime = 600
+            scope = "ADVERTISE_STARTD, ADVERTISE_MASTER, READ"
+            dir = "/etc/condor-ce/gltokens/fermilab"
+            filename = "ce_osg.idtoken"
+            owner = "osg"
+        @end
+
+
 Enabling the Monitoring Web Interface
 -------------------------------------
 

docs/v5/configuration/writing-job-routes.md → docs/v23/configuration/writing-job-routes.md

@@ -24,6 +24,15 @@ Each example is displayed in code blocks with tabs to switch between the two syn
 Syntax Differences
 ------------------
 
+!!! warning "Planned Removal of Deprecated Syntax"
+    -   `JOB_ROUTER_DEFAULTS`, `JOB_ROUTER_ENTRIES`, `JOB_ROUTER_ENTRIES_CMD`, and `JOB_ROUTER_ENTRIES_FILE` are
+    deprecated and will be removed for *V24* of the HTCondor Software Suite. New configuration syntax for the job router
+    is defined using `JOB_ROUTER_ROUTE_NAMES` and `JOB_ROUTER_ROUTE_[name]`.
+    -   For new syntax example vist:
+    [HTCondor Documentation - Job Router](https://htcondor.readthedocs.io/en/latest/grid-computing/job-router.html#an-example-configuration)
+    -   **Note:** The removal will occur during the lifetime of the HTCondor *V23* feature series.
+
+
 In HTCondor-CE 5, the [deprecated syntax](job-router-overview.md#deprecated-syntax) continues to be the default and
 administrator's can move to the [ClassAd transform syntax](job-router-overview.md#classad-transforms) by setting the
 following in a file in `/etc/condor-ce/config.d/`:

docs/v5/installation/central-collector.md → docs/v23/installation/central-collector.md

@@ -25,7 +25,7 @@ Before starting the installation process, consider the following points
 (consulting [the reference page](../reference.md) as necessary):
 
 -   **User IDs:** If they do not exist already, the installation will create the `condor` Linux user (UID 4716)
--   **SSL certificate:** The HTCondor-CE Central Collector service uses a host certificate and key for SSL and GSI
+-   **SSL certificate:** The HTCondor-CE Central Collector service uses a host certificate and key for SSL
     authentication
 -   **DNS entries:** Forward and reverse DNS must resolve for the HTCondor-CE Central Collector host
 -   **Network ports:** Site HTCondor-CEs must be able to contact the Central Collector on port 9619 (TCP).
@@ -55,11 +55,6 @@ Installing a Central Collector
 
     This command will update **all** packages
 
-1.  Install the `fetch-crl` package, available from the EPEL repositories.
-
-        :::console
-        root@host # yum install fetch-crl
-
 1.  Install the Central Collector software:
 
         :::console
@@ -69,7 +64,7 @@ Configuring a Central Collector
 -------------------------------
 
 Like a site HTCondor-CE, the Central Collector uses X.509 host certificates and certificate authorities (CAs) when
-authenticating SSL and GSI connections.
+authenticating SSL connections.
 By default, the Central Collector uses the default system locations to locate CAs and host certificate when
 authenticating SSL connections, i.e. for SSL authentication methods.
 But traditionally, the Central Collector and HTCondor-CEs have authenticated with each other using specialized grid
@@ -204,7 +199,6 @@ The specific services are:
 
 | Software    | Service name                          |
 |:------------|:--------------------------------------|
-| Fetch CRL   | `fetch-crl-boot` and `fetch-crl-cron` |
 | HTCondor-CE | `condor-ce-collector`                 |
 
 Start and enable the services in the order listed and stop them in reverse order.

docs/v5/installation/htcondor-ce.md → docs/v23/installation/htcondor-ce.md

@@ -1,5 +1,5 @@
-Installing HTCondor-CE 5
-========================
+Installing HTCondor-CE 23
+=========================
 
 !!! tip "Joining the Open Science Grid (OSG)?"
     If you are installing an HTCondor-CE for the OSG, consult the
@@ -21,7 +21,7 @@ Before starting the installation process, consider the following points
 (consulting [the reference page](../reference.md) as necessary):
 
 -   **User IDs:** If they do not exist already, the installation will create the `condor` Linux user (UID 4716)
--   **SSL certificate:** The HTCondor-CE service uses a host certificate and key for SSL and GSI authentication
+-   **SSL certificate:** The HTCondor-CE service uses a host certificate and key for SSL authentication
 -   **DNS entries:** Forward and reverse DNS must resolve for the HTCondor-CE host
 -   **Network ports:** The pilot factories must be able to contact your HTCondor-CE service on port 9619 (TCP)
 -   **Submit host:** HTCondor-CE should be installed on a host that already has the ability to submit jobs into your
@@ -58,11 +58,6 @@ Installing HTCondor-CE
 
     This command will update **all** packages
 
-1. Install the `fetch-crl` package, available from the EPEL repositories.
-
-        :::console
-        root@host # yum install fetch-crl
-
 1. Select the appropriate convenience RPM:
 
     | If your batch system is... | Then use the following package... |

docs/v5/operation.md → docs/v23/operation.md

@@ -12,7 +12,6 @@ The specific services are:
 
 | Software                     | Service name                                |
 |:-----------------------------|:--------------------------------------------|
-| Fetch CRL                    | `fetch-crl-boot` and `fetch-crl-cron`       |
 | Your batch system            | `condor` or `pbs_server` or …               |
 | HTCondor-CE                  | `condor-ce`                                 |
 | **(Optional)** APEL uploader | `condor-ce-apel` and `condor-ce-apel.timer` |
@@ -62,20 +61,12 @@ before trying to operate the HTCondor-CE again.
 Checking User Authentication
 ----------------------------
 
-There are two primary authentication methods for submitting jobs to
-an HTCondor-CE: GSI (currently being phased out) and SciTokens.
+The authentication method for submitting jobs to
+an HTCondor-CE is SciTokens.
 To see which authentication method and identity were used to submit
 a particular job (or modify existing jobs), you can look in
 `/var/log/condor-ce/AuditLog`.
 
-If GSI authentication was used, you'll see a set of lines like this:
-
-```
-10/15/21 17:52:32 (cid:14) (D_AUDIT) Command=QMGMT_WRITE_CMD, peer=<172.17.0.2:41045>
-10/15/21 17:52:32 (cid:14) (D_AUDIT) AuthMethod=GSI, AuthId=/DC=org/DC=opensciencegrid/C=US/O=OSG Software/OU=People/CN=testuser, [email protected]
-10/15/21 17:52:32 (cid:14) (D_AUDIT) Submitting new job 1.0
-```
-
 If SciTokens authentication was used, you'll see a set of lines like this:
 
 ```

docs/v23/releases.md

@@ -0,0 +1,54 @@
+Releases
+========
+
+HTCondor-CE 23 is distributed via RPM and are available from the following Yum repositories:
+
+- [HTCondor stable and current channels](https://research.cs.wisc.edu/htcondor/downloads/)
+- [Open Science Grid](https://opensciencegrid.org/docs/common/yum/)
+
+
+Known Issues
+------------
+
+Known bugs affecting HTCondor-CEs can be found in
+[Jira](https://opensciencegrid.atlassian.net/issues/?jql=project%20%3D%20HTCONDOR%20AND%20status%20not%20in%20(done%2C%20abandoned)%20and%20component%20%3D%20htcondor-ce%20and%20issuetype%20%3D%20bug)
+
+Updating to HTCondor-CE 23
+--------------------------
+
+!!! note "Updating from HTCondor-CE < 6"
+    If updating to HTCondor-CE 23 from HTCondor-CE < 5, be sure to also consult the HTCondor-CE 6
+    [upgrade instructions](../v6/releases.md#500).
+
+!!! tip "Finding relevant configuration changes"
+    When updating HTCondor-CE RPMs, `.rpmnew` and `.rpmsave` files may be created containing new defaults that you
+    should merge or new defaults that have replaced your customzations, respectively.
+    To find these files for HTCondor-CE, run the following command:
+
+        :::console
+        root@host # find /etc/condor-ce/ -name '*.rpmnew' -name '*.rpmsave'
+
+HTCondor-CE 23 is very close in functionality yo HTCondor-CE 6.
+As such, upgrading should be very easy.
+
+HTCondor-CE 23 Version History
+------------------------------
+
+This section contains release notes for each version of HTCondor-CE 23.
+Full HTCondor-CE version history can be found on [GitHub](https://github.com/htcondor/htcondor-ce/releases).
+
+### 23.0.0 ###
+
+[This release](https://github.com/htcondor/htcondor-ce/releases/tag/v23.0.0) includes the following new features:
+
+-   Add grid CA and host certificate/key locations to default SSL search paths
+-   Verifies that HTCondor-CE can access the local HTCondor's SPOOL directory
+-   Can use condor\_ce\_trace without SciToken to test batch system integration
+-   condor\_ce\_upgrade\_check checks compatibility with HTCondor 23.0
+-   Adds deprecation warnings for old job router configuration syntax
+
+Getting Help
+------------
+
+If you have any questions about the release process or run into issues with an upgrade, please
+[contact us](../index.md#contact-us) for assistance.