Pilot Configuration

The pilot configuration is stored in the default.cfg file (pilot3/pilot/util directory). The default values can be overridden locally but placing a config file in the pilot's launch directory. The various fields are described in the table below (in the order of appearance in the config file).

Section: Experiment

Field	Default value	Description
name	ATLAS	Pilot user name (currently unused).

Section: Pilot

Field	Default value	Description
pilotlog	pilotlog.txt	Main pilot log stored in the job log.
stageinlog	stageinlog.txt	Name of log file for stage-in.
stageoutlog	stageoutlog.txt.	Name of log file for stage-out.
pandajobdata	pandaJobData.out	Name of job definition file.
pandajob	real	Alternative: 'fake'. Used for pilot testing (see the following three fields).
testjobtype	production	Alternative: 'user'. Used for pilot testing.
testjobcommand	normal	Alternative: 'sleep' (sleep 1 will be executed as payload). Used for pilot testing.
testtransfertype	NULL	Alternative: 'direct'. Used for pilot testing.
pandaserver	(see code)	The URL for the PanDA server.
iddsserver	(see code)	The URL for the iDDS server.
heartbeat	1800	The heartbeat period in normal mode (30*60 = 1800 s).
debug_heartbeat	300	The heartbeat period in debug mode (5*60 = 300 s).
heartbeat_message	heartbeat.json	Heartbeat message file (only used when Pilot is not sending heartbeats to server).
jobid_file	pandaIDs.out	Job IDs can be stored to a file that is picked up by the wrapper.
free_space_limit	2 GB	The minimum required disk space for the pilot to run a job.
free_space_limit_running	1 GB	The minimum required disk space during running.
maximum_output_file_size	500 GB	The maximum output file size. The default is currently 'infinity'.
maximum_input_file_sizes	14336 MB	The maximum allowed sum of all input files (files accessed by direct access are not counted by the Pilot). Note: this is a fall-back value - the primary value is set in AGIS.
local_size_limit_stdout	2097152 kB	Size limit of payload stdout size during running (default: 2 * 1024^2).
looping_verification_time	900 min	Looping job time limits; if job does not write anything in N minutes, it is considered to be a looping. This field determines the frequency of the check.
looping_limit_default	7200 min	Time limit for looping jobs. Any output must have been updated within this time
looping_limit_min_default	7200 min	The minimum allowed looping limit, 2*3600 (reserved for future use).
kill_instruction_time	60 s	Kill instruction monitoring time. Pilot looks for kill instruction file every N seconds. The file may be placed in the run directory by an external actor (e.g. Raythena actor). When intercepted, Pilot will abort the payload.
kill_instruction_filename	pilot_kill_payload	Kill instruction file name.
proxy_verification_time	600 s	Proxy verification time (used by monitoring) in seconds.
payload_proxy_from_server	True	In case payload proxy should be downloaded from the server and be verified.
disk_space_verification_time	240 s	Disk space monitoring time (check remaining disk space every N seconds).
memory_usage_verification_time	60 s	Memory usage verification time (how often the memory monitor output will be checked)
process_verification_time	300 s	Process verification time. This verification sets the environmental variable PILOT_MAXNPROC to the maximum number of found (child) processes corresponding to the main payload process id, PILOT_MAXNPROC may be used by the payload.
output_verification_time	300 s	Output file size verification time. Currently not used.
thread_check	10 s	The default thread check time in seconds, used by thread monitoring.
cpu_check	60 s	The default CPU check time in seconds, used by CPU monitoring.
timing_file	pilot_timing.json	The timing file used to store various timing measurements.
error_log	piloterrorlog.txt	Optional error log. Currently not used.
redundant	(cvmfs path)	List of redundant files and directories to be removed prior to log file creation. Currently not used.

(more to come)