Skip to content
Carlo Cabanilla edited this page Dec 17, 2013 · 10 revisions

How do I see what data dogstatsd is receiving?

We don't have debug logging in dogstatsd as it can potentially receive a huge amount of data and logging the incoming udp packets will kill cpu/disk. Sometimes we need to see that data though when trying to debug dogstatsd submission, so here is a hacky way to do it.

To apply it:

cd /
curl https://gist.github.com/clofresh/7883109/raw/75718cf42179d2199643daeea47fcec387e42281/dogstatsd_debug.py.patch | sudo patch -p 0
sudo service datadog-agent restart

To undo it:

cd /
curl https://gist.github.com/clofresh/7883109/raw/75718cf42179d2199643daeea47fcec387e42281/dogstatsd_debug.py.patch | sudo patch -p 0 -R
sudo service datadog-agent restart

Why is my agent process dying?

Best place to check are the logs in /var/log/datadog/. If you see something like:

2013-12-11 20:52:11,887 | ERROR | dd.dogstatsd | util(util.py:281) | Self-destructing...

It means an iteration of the agent took too long and its internal watchdog thread killed itself in order to prevent it from hanging.

How often is data collected?

  • Data from existing checks like mysql or nginx are collected every 15s via the collector process.
  • Data submitted via statsd gets aggregated locally and flushed every 10s to Datadog via the dogstatsd process.
Clone this wiki locally