Skip to content

MRTG Traffic Graphs

Jump to bottom Edit New page
Rowan Thorpe edited this page Aug 20, 2013 · 21 revisions

IXP Manager uses MRTG to poll switches and create traffic graphs for:

  • aggregate (total) traffic over the entire exchange (all infrastructures);
  • aggregate graphs for each infrastructure;
  • aggregate graphs for each switch;
  • aggregate graphs for inter-switch links; and
  • aggregate and per port graphs for each member / customer.

MRTG is particualr efficient in the SNMP sense as, irrespective of how many times an interface is used for different graphs, it is only polled once per run.

Please ensure you have configured your port types correct as discussed in the Port Types section of this page.

Graphs are generated for bits per/second, packets p/s, errors p/s and discards p/s.

Setting Up Your Server for Mrtg Collection

In all cases below, $APPLICATION_PATH is the base directory of your IXP Manager installation.

You need to install some basic packages for MRTG to work - on Ubuntu for example, install:

apt-get install libconfig-general-perl libnetaddr-ip-perl mrtg

You also need a folder to store all MRTG files. For example:

mkdir -p /srv/mrtg
mkdir -p /srv/mrtg/members

In your application.ini, you need to set two parameters:

;; destination for all MRTG output (log files, pngs, etc)
mrtg.conf.workdir = '/srv/mrtg'
;; destintation file for the MRTG configuration. If not set, the generator sends it to stdout
mrtg.conf.dstfile = '/etc/mrtg/mrtg.cfg'

Note that workdir is the same directory you created above (e.g. /srv/mrtg).

You can now generate a MRTG configuration by executing a command such as:

APPLICATION_PATH/bin/ixptool.php -a statistics-cli.gen-mrtg-conf

However, you need to complete additional actions from the Integrating with IXP Manager section below.

You also need to set up a cron job to regenerate the configuration periodically (see below). Note that our header template starts MRTG as a daemon (see below also).

Daemonising MRTG

On FreeBSD, MRTG comes with an initd script by default and you can kick it off on boot with something like the following in rc.conf:

mrtg_daemon_enable="YES"
mrtg_daemon_config="/etc/mrtg/mrtg.cfg"

However, on Ubuntu it does not but it comes with a /etc/cron.d/mrtg file which kicks it off every five minutes (it will demonise the first time and further cron jobs will have no effect). If you use this method, you will need to have your periodic update script restart / stop the daemon when the configuration changes.

To start and stop it via standard initd scripts on Ubuntu, use an initd script such as this (source):

cp $APPLICATION_PATH/tools/runtime/mrtg/ubuntu-mrtg-initd /etc/init.d/mrtg
chmod +x /etc/init.d/mrtg
update-rc.d mrtg defaults
/etc/init.d/mrtg start

Remember to disable the default cron job for MRTG on Ubuntu!

Periodic Updating of MRTG Config

You will need to update your MRTG configuration regularly so any additions / changes made to member ports will be reflected. To do this, create or update your periodic update script (e.g. /usr/local/bin/ixp-periodic-update.sh):

#! /bin/sh

PATH=/usr/local/bin:${PATH}
dir=/usr/local/bin
APPLICATION_PATH=/srv/ixp

# Synchronise configuration files
${APPLICATION_PATH}/bin/ixptool.php -a statistics-cli.gen-mrtg-conf

# Kick daemons
/etc/init.d/mrtg restart > /dev/null

Add the above script to cron to execute a few times a day (or as appropriate for your IXP).

Customising the Configuration

If you wish to change aspects of the configuration, copy the file(s) you wish to change from the APPLICATION_PATH/views/statistics-cli/mrtg directory to the same location under your own skin.

We've made these files very modular so, if you need to change them, you should only really need to change one or too. There are also empty files called:

  • custom-header.cfg
  • custom-footer.cfg

which are included just after the header and at the very end respectivily. Skinning these will allow you to easily drop additional content into your MRTG configuration.

Integrating with IXP Manager

Member / customer graphing will work immediately with one configuration option in the IXP table of the database. In the example above, our workdir was /srv/mrtg and so the configuration option you need is to set the mrtg_path of your IXP in the ixp table as follows:

UPDATE `ixp` SET `mrtg_path` = '/srv/mrtg';

If you are using the multi-IXP environment, you can set the above in the frontend under the IXPs menu option - and, in a muilti-IXP environment, you will need to set it per IXP. If you are in single-IXP mode, you can find a link to edit the IXP under the Infrastructres page.

That's it! If you want to use a separate server for collecting and generating the graphs, you can make this workdir available over HTTP/HTTPS (secured access by only the web server's IP of course!) and use a configuration option such as the following in place of `/srv/mrtg':

http://mrtg.example.com/mrtg

This is possible as graphs are read using readfile().

Aggregate IXP and Infrastructure Graphs

The MRTG script creates aggregate graphs for the entire IXP as well as per-infrastructure graphs. These are named as per the Aggregate Graph Name defined in IXP Manager for each IXP and infrastructure.

These graphs are available from the Statistics menu under Overall Peering Graphs. Also, the graphs on the admin dashboard are the monthly versions of these and will appear on the dashboard when configured as above.

Switch Aggregate Graphs

These are defined and built automatically from the switches you have defined. These graphs are the aggregate of all switch ports (not just peering ports) - which, if you think about it, makes sense as core ports do need to be included here. These graphs are available from the Statistics menu under Switch Aggregate Graphs.

Inter-Switch / Trunk Graphs

The above will generate graphs for all your customer ports and aggregate graphs for all items mentioned at the start of this page except inter-switch graphs. There is currently really no way to be able to create a sane definition of these in the IXP Manager database, so we chicken out and let each IXP do it manually. Simplicity r00lz.

Skin the trunks.cfg file as we have done as a working / production example here.

Once you have defined your own trunks as described above, then add a configuration line such as the following for each to application/configs/application.ini:

mrtg.trunk_graphs[] = "ixpid::core-sw1-sw2-lan1::PoP 1 to PoP 2 (LAN1)"
mrtg.trunk_graphs[] = "ixpid::core-sw1-sw3-lan1::PoP 1 to PoP 3 (LAN1)"

where:

  • ixpid is the ID (from the database table ixp) of the IXP where these graphs reside. In a single-IXP environment (i.e. you have not enabled multi-IXP), this will always be 1;
  • core-sw1-sw2-lan1 is the name as defined in mrtg.cfg.template; and
  • PoP 1 to PoP 2 (LAN1) is the title used on the frontend.

We've included matching sample mrtg.trunk_graphs in our skinned file above.

These graphs will be available in the Statistics menu under Inter-Switch / PoP Graphs.

Inserting Traffic Data Into the Database

There is a database table called traffic_daily which stores the daily / weekly / monthly / yearly average / max / total traffic in / out for bits / packets / errors and discards.

This table is very useful and is used, for example, for generating alert emails based on port utilisation, unusual traffic deltas, etc.

To have this information stored (which is gathered from the MRTG log files), put something like the following in your cron:

0 2     * * *   root    $APPLICATION_PATH/bin/ixptool.php -a statistics-cli.upload-traffic-stats-to-db

Deprecated Perl Based MRTG Configuration

In 2013/08 we switched to using the IXPtool generator. If you wish to use the older Perl version, here are the instructions:

In all cases below, $APPLICATION_PATH is the base directory of your IXP Manager installation.

You need to install some basic packages for MRTG to work - on Ubuntu for example, install:

apt-get install libconfig-general-perl libnetaddr-ip-perl mrtg

If you haven't already, install the Perl library for IXP Manager:

cd $APPLICATION_PATH/tools/perl-lib/IXPManager
perl Makefile.PL
make install

Then copy and edit the configuration file to set the database connection settings:

cp $APPLICATION_PATH/tools/perl-lib/IXPManager/ixpmanager.conf /usr/local/etc/
joe /usr/local/etc/ixpmanager.conf #and set database settings

You also need a folder to store all MRTG files. For example:

mkdir -p /srv/mrtg
mkdir -p /srv/mrtg/members

We bundle two files with IXP Manager for MRTG collection in this directory and they are:

  • mrtg.cfg.template - the basic MRTG configuration template which needs to be copied and edited for your specific set-up;
  • update-mrtg-config-from-db.pl - the Perl script to update your MRTG configuration based on the customers and ports defined in your IXP Manager database.

So, first we need to set up an appropriate template for your IXP:

cp $APPLICATION_PATH/tools/runtime/mrtg/mrtg.cfg.template /usr/local/etc/

and edit it and set:

  • set the WorkDir to the directory you created above (e.g. /srv/mrtg);
  • change any references to INEX to your own IXP name;
  • set up any inter-switch trunks as required (see below).

Now, copy the MRTG configuration update script:

cp $APPLICATION_PATH/tools/runtime/mrtg/update-mrtg-config-from-db.pl /usr/local/bin

You can now generate your full MRTG configuration by executing a command such as:

/usr/local/bin/update-mrtg-config-from-db.pl                                              
    --configdir /etc/mrtg                            # location of MRTG's mrtg.conf file  
    --templatename /usr/local/etc/mrtg.cfg.template  # our template from above            
    --datadir /srv/mrtg                              # our data dir from above            
    --owner root                                     # file owners                        
    --ixpname INEX                                   # Your IXP name

You need to set up a cron job to do this periodically (see below). Note that our template starts MRTG as a daemon (see below also).

Add a custom footer
Add a custom sidebar
Clone this wiki locally