User Tools

Site Tools


hannibal:rsyslog

If you have more than a few nodes on your network, their logs could easily drive you crazy! In order to preserve your mental state, we'll build a centralized logging server that collects all those logs and puts them into a database. In addition we'll install a small web application that supplies a sysadmin friendly userinterface to the database and that allows to deploy some basic filtering. Nothing fancy, but enough to keep you from going nuts!

By default Debian Etch supplies genuine good old syslog, however in 2008 more modern syslog software such as syslog-ng and rsyslog is available. We chose rsyslog as that might become the default in the Debian releases to come (eg. see this thread) and it already is the default in the Fedora-project. Rsyslog is a multi-threaded syslogd that supports remote logging over TCP, the MySQL database, permitted sender lists, ipv6 and nifty filtering. Detailed information is available at the rsyslog website.

Centralized syslog server

Pre-installation

As we want our centralized syslog server to put all logs in a MySQL-database, a MySQL-database should be available. While installing the rsyslog software, you will be asked to provide a database-account that has database-access from the centralized logging server and that has permissions to create a database and database-users. Read the Hannibal database server section for documentation on howto setup a MySQL server.

Make sure the time and timezone settings are correct. You can find documentation on howto keep time on your server at the NTP server section of this wiki.

Double check the configuration of your nameservers (and /etc/hosts files) in order not to get stuck with failing or slow connections.

Installation

The rsyslog software is packaged available in the following distribution repositories:

  • Debian-4.1 (etch) as version 2.0.0-2~bpo40 in backports
  • Debian-4.2 (lenny) as version 2.0.0-2 in main
  • Ubuntu-8.04 (hardy) as version 1.19.12-1 in universe

All above are fine. In order to obtain the required packages for the current Debian stable release, we'll first add a line to /etc/apt/sources.list

deb http://ftp.nl.debian.org/backports.org etch-backports main contrib

Now run:

apt-get update
apt-get install rsyslog rsyslog-doc rsyslog-mysql

During the installation of the mysql package, db-conf will ask you for the password of a mysql superuser (it assumes root), create a database and schema owner, creates two tables, and constructs /etc/rsyslog.d/mysql.conf with these information for the rsyslog db-connection-module.

Configuration

Instruct rsyslog to accept remote connection over UDP as well as TCP (port 514). Edit /etc/default/rsyslog and make sure that it contains a line like this:

...
RSYSLOGD_OPTIONS="-m 0 -r -t514"
...

The main configuration file of rsyslog is /etc/rsyslog.conf. You might want to adjust it according to your needs, however the Debian Backports rsyslog package we just installed provides the same defaults as the ones provided by genuine sysklogd.

Now you can restart the daemon:

/etc/init.d/rsyslog restart

Clean up the older sysklogd and klogd configuration.

dpkg --purge sysklogd && dpkg --purge klogd

Syslog clients

Your newly created centralized syslog server will accept client connections from regular syslog clients as well as from rsyslog clients.

Rsyslog client

Installation

The rsyslog software has been provided for Debian Etch by Debian backports. In order to obtain the required packages we'll first add a line to /etc/apt/sources.list

deb http://ftp.nl.debian.org/backports.org etch-backports main contrib

Now run:

apt-get update
apt-get install rsyslog rsyslog-doc

Configuration

Instruct rsyslog to send all logs to the logging server. Add a line like this to /etc/rsyslog.conf

If you want to send logs over TCP

...
*.*                          @@rsyslog.intra.example.com
...

If you want to send logs over UDP

...
*.*                          @rsyslog.intra.example.com
...

Now restart the daemon:

Genuine syslog client

Debian/Ubuntu

Remember, genuine syslog only can send logs over UDP. Instruct syslogd to send all logs to the logging server. Add a line like this to /etc/syslog.conf

...
*.*                          @rsyslog.intra.example.com
...

Now restart the daemon:

/etc/init.d/sysklogd restart

Solaris

On a standard Solaris 8 system, add a line like the one below below to /etc/syslog.conf. This also filters out the logging of the local MTA. Mention that the space between the two columns has to be one or more tabs!

...
*.debug;mail.none;local6.none;local7.none  <TABs-here>  @rsyslog.intra.example.com
...

Syslog webinterface

phplogcon is a small web application that supplies a sysadmin friendly userinterface to the logs in the MySQL-database. It does allow to deploy some basic filtering on the logs. Nothing fancy, but enough to keep you from going nuts!

Pre-installation

Download the software

At the moment of writing we used the latest stable version of phplogcon (1.2.x). Phplogcon is available for download at http://www.phplogcon.org/downloads.

Database access

Phplogcon needs access to the MySQL-database that contains the logs. It will add some tables to this database in order to store some user information as credentials and profiles. Create a MySQL-user that can do this.

mysql -u root -p -e "grant all privileges on Syslog.* to phplogcon@your_host identified by 'your_password'"
mysql -u root -p -e "flush privileges"

Read the Hannibal database server section for documentation on howto setup a MySQL server.

Install Apache webserver and PHP

Phplogcon is written in PHP so it needs a webserver that supports PHP and that is able to access the MySQL-database. The documentation on howto install Apache and PHP is available at the web server section of this wiki.

Installation

Unpack the software in your webserver:

cd /var/www
tar -zxvf /tmp/phplogcon-1.2.x.tar.gz
ln -s phplogcon-v1.2.x phplogcon

Change ownership and permissions on config.php. The permission are only necessary during the installation of phplogcon. Afterwards you must return the permissions to 640!

cd phplogcon
chown root:www-data config.php
chmod 660 config.php

Change ownership on phplogcon/install/install.php. This is only necessary during the installation of phplogcon. Afterwards you must (re)move the 'install'-directory!

cd phplogcon/install
chown root:www-data install.php

Configuration

Point your browser to http://your_webserver/phplogcon/install/install.php The Installation-Assistant will guide you through the installation. Don't forget to remove the phplogcon/install-directory and to return the permission to 640 on phplogcon/config.php!

Literature

hannibal/rsyslog.txt · Last modified: 2008/06/25 12:31 by Olivier Brugman