Sending Log Messages to Programs

Finally, to have another program handle the logs, use a pipe symbol (|) to redirect the messages to that program:

...............................................................................................

mail.* |/usr/local/bin/mailstats.pl

...............................................................................................

Note Traditionally, UNIX demands tabs between the columns in /etc/syslog.conf, but FreeBSD permits you to use spaces. Be sure to use tabs only if you share one syslog.conf between different UNIXes.

Rotating Logs with Newsyslog.conf

Log files grow and you must control their growth. The standard way to do so is with log rotation. When using log rotation, the oldest logs are deleted, each old log is renamed to the next oldest name, the current log is moved, and a new log file is created.

FreeBSD includes a basic log−file handler, newsyslog(8), which will also compress files, restart daemons, and in general handle all the routine tasks of shuffling files. Cron runs newsyslog once an hour.

Newsyslog reads /etc/newsyslog.conf and checks each log file listed there. If the conditions listed for rotating the log file are met, the log is rotated and other actions are taken as appropriate.

The /etc/newsyslog.conf file uses one line per log file, and each line has seven fields. For example:

...............................................................................................

/var/log/slip.log root:network 640 3 100 * Z

...............................................................................................

We'll examine each field in turn.

Log File Path

The first entry on each line is the full path to the log file to be processed (/var/log/slip.log in our example).

Owner and Group

The second entry (root:network in our example) lists the rotated file's owner and group, separated by a colon (such as root:wheel). This field is optional, and is not present in many of the standard entries.

Newsyslog can change the owner and group of old log files. By default, log files are owned by root and are in the wheel group. While it's not common to change the owner, you might have to use this ability on multi−user machines.

429

You can choose to only change the owner, or only change the group. In these cases you must use a colon, even though nothing appears on the other side of it. For example, :www will change the group to www, while user827: will change the owner to user827.

Permissions

The third field (640 in our example) is the permissions mode, in standard UNIX three−digit notation. (See Chapter 7 for details.)

Count

Next is the count field (having a value of 3 in our example), which represents the number of old log files that newsyslog will keep—kind of. Newsyslog starts counting archived log files at 0. Many computer systems start numbering at 0, but newsyslog includes 0 as well as the count number. With the default count setting of 5 for /var/log/messages, /var/log includes the following files:

...............................................................................................

messages messages.0.gz messages.1.gz messages.2.gz messages.3.gz messages.4.gz messages.5.gz

...............................................................................................

Those of you who can count will recognize that this is six backups, not five, plus the current log file! While, as a rule, it's better to have too many logs than not enough, if you're tight on disk space, deleting an extra log file or two might buy you some time. Some Web servers can have hundreds of sites on a single server; removing one or two files times 100 sites can create a lot of disk space.

Size

The fifth field (100 in our example) is the file size. When newsyslog runs, it compares the size listed here with the size of the file. If the file is larger than the given size in kilobytes, it is rotated. If the file size doesn't affect when you want it rotated, put an asterisk (*) here.

Time

So far, this seems easy, right? Well, the sixth field, time (* in our example), can make new administrators cry.

The time field has four possible values: an asterisk (*), a number, and two different date formats. If you don't want to rotate a log at a particular time, put an asterisk (*) here. If you use a plain naked number, newsyslog rotates the log after that many hours have passed. For example, if you want a log to rotate every 24 hours, but don't care exactly when this rotation happens, put 24 here.

The date formats are a little more complicated.

430

ISO8601 Time Format

Any entry beginning with an at symbol (@) is in ISO 8601 restricted time format. This is a standard used by newsyslog on most UNIX systems, and was the time format originally used in MIT's primordial newsyslog program. Unfortunately, this standard is not at all clear, but since it's a standard, FreeBSD supports it.

A full date in ISO 8601 format is 16 digits with a T in the middle. The first four digits are the year; the next two are the month; the next two are the date. The T is inserted after the date as a sort of decimal point, separating whole days from fractions of one. An ISO 8601 date must include the T.

The next two digits are hours; the next two are minutes; the next two are seconds. For example, the date and time February 2, 2002, 9:15 and 8 seconds PM is expressed in ISO 8601 as

...............................................................................................

20020202T211508

...............................................................................................

While complete dates in ISO 8601 are mostly straightforward, confusion arises when you don't list the entire date. For example, you can choose to specify only fields near the T, leaving fields farther away blank, which will be read as wildcards. For example, T23 matches the twenty−third hour of every day of the year. With a newsyslog time of @T23, the log rotates every day at 11 PM, and 4T00 matches midnight of the fourth day of every month.

As with crontab, you must specify hours. A date like @7T will run once an hour, every hour, on the seventh of the month. After all, it matches all day long! This can be useful for debugging, but isn't generally desirable.

FreeBSD−Specific Time

One problem with the ISO 8601 time system is that it doesn't allow you to easily designate weekly jobs (it's not uncommon to want to rotate a log on Mondays, for example), and it's impossible to specify the last day of the month. That's where the final time format comes in. Any time with a leading dollar sign ($) is written in the FreeBSD−specific month−week−day format.

This format works much like cron, allowing you to set particular days of the week to run a job on, and uses three identifiers: M (day of month), W (day of week), and H (hour of day). Each identifier is followed by a number indicating the particular time it should be run. Hours range from 0 to 23, while weekdays range from 0 (Sunday) to 6 (Saturday). M starts with 1, and goes up to the number of days in that particular month. For example, to rotate a log every Sunday at 8 AM you could use a time of $W0H8. To rotate the log on the fifth of each month at noon, you could use $M5H12.

One interesting feature of this system is that it lets you automatically schedule a job for the last day of the month by using L to represent the last day of the month. Without this, it's very difficult to do an end−of−month job without writing a script that knows how many days are in each month, compares the current date to the scheduled date, and decides if it will start the program. (That gets ugly quickly.) For example, to start your month−end log−file accounting two hours before the end of the month, use a time of $MLH22.

Note You can rotate logs at a given time, or when they reach a certain size, or both. If you use both, the log will rotate whenever either condition is met. If you're only rotating on one

431

condition (meaning you want to rotate every day, no matter how large the file gets), use an asterisk (*) in the other field.

Flags

Now that you know how to express the exact time that you want your log to run, we encounter the flags field (Z in our example), which offers two options for handling your log files. Some programs log their data in plain text, while others use a binary format; each sort of log needs to be treated differently.

Binary files can only be written to in a very specific manner. Newsyslog starts each new log file with a "log−file turned over" message, but adding this to a binary file will damage it. The B flag tells newsyslog that this is a binary file, and that the message should not be written. On the other hand, many log files are plain ASCII text, and compressing them can save a huge amount of space. The Z flag tells newsyslog to compress the rotated log files with gzip.

You can use only one of these flags; after all, compressing binaries doesn't save much room, and only text logs can use a "turned over" message.

Pidfile Path

The next field is the pidfile path (not shown in our example). A pidfile is a simple way to record a program's process ID (PID) so that other programs can easily view it. Not all programs have pidfiles; the ones that do store their pidfiles under /var//un (take a look and see what's on your system).

If you list the full path to a pidfile in /var/run, newsyslog will send a kill−style signal to that program when it rotates the log. For example, the Apache Web server needs to be notified when you rotate its logs. By listing its pidfile here, you can have newsyslog send a kill ‘ to Apache so it will handle its part of log−file rotation.

Most programs will handle log−file rotation on a kill ‘, or SIGHUP, but some programs need a specific signal when a log file is rotated. If you have one of these programs, you can list its exact signal number in the last field.

Example newsyslog.conf Entry

Let's slap this all together in a worst−case, you−have−got−to−be−kidding example. Assume you have a database log file that you want to rotate at 11 PM on the last day of every month. The database documentation says that you need to send the program an interrupt signal (SIGINT, or signal number 2) upon rotation. You want the archived logs to be owned by the user dbadmin, and only viewable by that user, and you need six months of logs. What's more, the logs are binary files. Your newsyslog.conf line would look like this:

...............................................................................................

/var/log/database dbadmin: 600 6 * $MLH23 B /var/run/db.pid 2

...............................................................................................

This is an extreme example; in most cases, you just slap in the filename and rotation condition and you're done. But I thought I'd make you twitch.

432