Chapter 15 Backing Up Your Linux System 459

One of the Linux system administrator’s most critical responsibilities is the backup and recovery of system and user data. Companies spend hundreds

of thousands of dollars on their network infrastructure, yet they never take into account the amount of money that may be wasted due to lost data and downtime because of a deficient backup plan. The importance of having a proper backup system that is routinely monitored and tested can’t be understated.

It’s not enough just to back up your user data; critical Linux system files must also be backed up on a regular basis. In the event of system file corruption or disk failure, you will be able to quickly restore your system to its original state if you have regularly backed up system files.

You should also regularly test your restore procedures. Many companies only test their backup procedures and never test their restores. This is a big mistake because the only way you can verify that a backup system is working properly is to test the restore procedure.

This chapter deals with planning, executing, and routinely testing a backup and restore system.

Disaster Recovery Planning

In planning a system for backup and recovery, the Linux system administrator must analyze several factors:

Types of backup data: What different types of data must be backed up, and what is their importance?

Frequency and scheduling: When and how to backup data?

Storage and media type: What types of storage units and media are available?

Recovery options: Is this regular user data, or special proprietary database information?

Media Storage: Should backup media be stored onsite or offsite? What extra safety precautions or facilities will be needed?

Types of data

Basically, two types of data must be backed up — user files and system files.

User files

Your Linux user files are typically stored in the /home partition or directory. Keep all user data in its own partition, so it won’t affect critical system partitions. For example, your user directories will quickly use up disk space because they continuously add files, programs, data, and personal software. If the /home partition runs

460 Part V Maintaining the Linux System

out of space, it won’t affect any of the system drives. If your system runs out of disk space, it may create a condition in which the server can crash.

Keeping all user files in the /home partition also aids your backup strategy by having all your user files backed up in one location. This means you won’t have to search all over your file systems for user files that need to be backed up, thus saving the administrator the task of creating lengthy backup scripts to accommodate all locations.

Some user data, however, may not be located in the /home partition; for example, special databases that all users can access. You must ensure that these are also backed up properly. Many databases store their information in a proprietary format that may involve special software or procedures to properly back up the data. For transactional databases, for example, you must back up both the database data and the transaction logs in order to recover from a disaster.

System files

Linux system files are critical programs and the data that they contain are necessary in order for the system to run properly. These critical programs include the core operating system, configuration files, and user-installed programs.

In the event of a severe system crash, you can install the base operating system by using the installation disks or CD-ROM. You can also reinstall patches and programs. However, some files can’t be reinstalled in this manner, such as special configuration files, user mail, and log files. Back up these files because they can’t be reinstalled by default.

Some of the major Linux system partitions and directories include:

/etc: Configurations for your services, programs, and daemons reside here. This partition is critical when restoring a system to its previous state.

/usr/local or /opt: Most user-installed files are kept in these partitions and directories.

/var/spool/mail: User mail that has not been downloaded resides here.

/var/log: This directory contains all your log files. It is a good idea to backup up your system log files on a frequent basis, as they contain information that can help you troubleshoot your system.

Frequency and Scheduling

After you know exactly what to back up, you must create a schedule to determine the frequency for performing backups. Certain data, such as Linux system files, change rarely and can be safely backed up once a week, or whenever an important configuration change takes place. User files that change very frequently should be backed up daily — at the very least. Some heavily used transactional database environments back up their data several times a day because of the large number of changes that happen within a short period of time.

Chapter 15 Backing Up Your Linux System 461

Backup types

You can perform four main types of backups:

Full

In this type of backup, all files are backed up every night. This is the most simple and effective type of backup, but can be limited by the amount of data that you have. A full backup may be impractical if your file systems are very large, containing hundreds of gigabytes. Depending on the size and speed of your backup device, a full backup may involve many sets of backup media, and the process may take a very long time to complete. The advantage of a full backup is that you always have the most recent files backed up properly, which makes restores very easy and convenient to perform.

Full backups should be performed prior to adding new applications or hardware, and prior to a change in server configuration.

Incremental

In this type of backup, all files that have changed since the previous backup are backed up. For example, a certain file is only updated on Wednesday mornings. Therefore, the next backup, which happens at 11:00 p.m., will backup that file. The following day, the file will not be modified, so it won’t be saved during that night’s backups. The advantages of using an incremental backup system is that you don’t use as much tape because you aren’t performing a full backup each night. Incremental backups also cut down the amount of time required to perform a nightly backup. One major disadvantage of an incremental backup system is lengthy restore times. If you need to restore a full set of files, you will need the last full backup and every incremental backup to restore them in order. Another disadvantage is that your backups will be spread across multiple media, making the restore process more complicated.

Differential

In the differential backup scheme, all files that have changed since the last full backup are backed up. For example, a certain file is only updated on Wednesdays. The backup that takes place at 11:00 p.m. will back up that file. On the Thursday night backup, the file is still backed up because it changed since the last full backup — even though the file did not change that day. The advantage of a differential backup is that to restore a full system, you only need the last full backup and the last differential backup. The downside of the differential scheme is that it uses much more disk media than an incremental backup.

Archive

In an archive backup, data from your file systems are backed up to a storage device, but the original files are then removed from the server. Use archive backups to permanently remove data that has not been used in a long time to a storage media, such as CD-ROM or tape. This frees up space on your file systems, and keeps your data directories clear of outdated files.

462 Part V Maintaining the Linux System

Backup rotation methods

After you have chosen backup types, you also need to organize your media to deal with these backups and to keep them for a certain period of time. Depending on the type of data, you may only want to keep it for one month, one year, or indefinitely.

You can choose from several existing backup media rotation methods, and you can always customize your own schedule, depending on the type and sensitivity of your data. Choose from the following:

Son

With this method, you perform a full backup, and then use the same tape for each subsequent backup. In other words, whatever has been backed up to tape is written over by the next backup operation. This format is not suitable for most environments, and does not offer much retention time for your files. It also causes the media to wear out much faster, which can cause corruption.

Father/Son

This method uses a full set of tapes, which are rotated every week. For example, suppose that you have five tapes representing Monday through Friday, and a sixth tape for Saturday. For the daily tapes, you perform an incremental or differential backup. For the Saturday tape, you perform a full backup. With this format, you can go back at least once a week to restore a file. In the event that you have to restore a full system, you only need the full backup, and up to five of the daily tapes. Although this scheme is better than the Son method, it still doesn’t offer enough retention time for most environments.

Grandfather/Father/Son

This is the most popular method of backup operations. Similar to the Father/Son method, this method’s cycle is an entire month, and allows for daily, incremental, or differential backups, with four weekly backups and one monthly backup. This method gives you a very large retention window for data because your weekly tapes are kept for up to a month, and your monthly tapes are kept for up to a year. You can also extend this time with a yearly tape at the end of the year. Although this method uses the greatest amount of backup media, it’s a worthwhile expenditure because you receive excellent data protection and retention.

Storage and media types

When you think of backups, the most typical type of storage media that probably comes to mind is the tape drive. Although the tape drive is still the backbone of most backup strategies, you can use other devices for backup purposes. Your choice for storage device and media will ultimately depend on the number of servers and size of your data backup requirements: