Monitoring of Drones

[giffels]

At the moment there is a lack of direct monitoring of drones. The only monitoring available is reported by the HTCondor central manager to an InfluxDB and does not contain any detailed information necessary for debugging. Currrently I think that the following two possibilities can be considered.

• Monitoring or status information (CPU, Memory, Disk Space, CVMFS, etc. ) is reported via HTCondor ClassAds, which are then stored in Elasticsearch or a similar DB. Update of ClassAds happens roughly any 12 minutes AFAIK.
• Collect HTCondor StartD logs in Elasticsearch for later analysis

The first still does not provide detailed logging, but is still enough to monitor the health of the drones, while the later allows more detailed debugging in case of problems arising.