Production Server Info

Intro

Information of production harvester servers at CERN are recorded here.

---

Specifications of node

• CERN Openstack VM managed by CSOps
• OS: CC7
• 4 cores, 8 GB RAM

---

Special setups

FIFO

FIFO is used on production node on motinor agent cycle.

---

Side services

HTCondor

NGINX

NGINX (openresty) is running on production node to serve as http gateway with token authentication of Harvester apache messenger.

Installation

• Yum install openresty-1.13.6.2-1 or above. Yum repo can be found here

• Get the latest release (v1.0.1) of nginx-jwt from GitHub and untar it a in proper directory (more info)

  wget -P /opt https://github.com/auth0/nginx-jwt/releases/download/v1.0.1/nginx-jwt.tar.gz
  cd /opt/
  mkdir nginx-jwt
  tar -xf nginx-jwt.tar.gz -C nginx-jwt
  

• Make a secret file for JWT token signature (must be the same file configured as secretFile in frontend section in harvester.cfg)

  ls -l /data/atlpan/harvester_jwt.secret
  

• Get nginx configuration file in place and make necessary modification. The nginx configuration template can be found here

  mv /usr/local/openresty/nginx/conf/nginx.conf{,.rpmsave}
  vim /usr/local/openresty/nginx/conf/nginx.conf
  

• Make the script nginx.service in place. The script example can be found here; make necessary modification of variables and paths in the script to fit your environment.

  ls -l /opt/nginx.service
  chmod a+x /opt/nginx.service
  /opt/nginx.service start
  

Service control

One can stop, stop, or reload the nginx service via the following commands respectively:

  /opt/nginx.service start
  /opt/nginx.service stop
  /opt/nginx.service reload

For CERN Central Harvester Instances Only:

CERN CSOps already has a puppet module to build up an instance as central production harvester server.

The harvester instance from CSOps has already done almost all the installation steps. After getting the instance, one only need to run this script to initialize:

# /cephfs/atlpan/harvester/scripts/nginx-init.sh

If successful, the instance will run the nginx service binding with port 25443.

After that, one can ask CSOps to open the port to outside CERN.