Specific instructions to run harvester with ssh rpc middleware

Instructions

Basic description of SSH RPC middleware: see here

The queue configuration depends on the architecture of the cluster (HPC) and the use case.

Generic use case

Consider an HPC:

• No outbound connectivity on login nodes and worker nodes
• One can access login nodes via SSH
• Login nodes have the same environment and mount the same shared filesystem as worker nodes do
• Allowing service process to run on login nodes (no cputime limit per process or other limitations)
• With DTNs (data transfer nodes) which has outbound connectivity and grid data transfer tools (globus, gfal, xroot, etc.)
• DTNs are accessible from login nodes

Then it suffices to run harvester rpc_bot process on the login node of HPC, and let all harvester plugins run on the login node (run remotely).

That is, harvester runs the following plugins remotely:

• submitter
• monitor
• sweeper
• messenger
• preparator
• stager