Human Factors Limit Detects

Another factor that limits the EOI we can detect and report is that people are part of the system. A typical day as an operator of an intrusion-detection system includes the recording and possible reporting of some number of detects. If you were to examine a year's worth of detects from a site, you might find that the detects cluster as 12 IMAPs, 5 portmaps, 25 ICMP ping sweeps, 30 Smurfs, 8 mscans, 4 portscans, 5 DNS zone Xfer attempts, 4 WinNukes, and so forth. If you check the site's Computer Incident Response Team (CIRT), you find that yup, these are the kinds of things being reported by those sites that do bother to report. So what's wrong with this picture? Not only does the IDS fail to report many events of interest because it does not have a signature for them, many times the analyst chooses not to report many of the events that are detected.

If you were to spend a day or two on the Internet doing web searches, you could easily collect a hundred different software implementations of exploits. Some won't compile easily, and others have limited documentation. Still others are variations on a theme. The simple fact remains, however, that you can easily collect more attacks than are commonly being detected and reported. So what's the problem? One part of the problem is the signature issue previously discussed. If the design of the system relies on signatures and a filter doesn't exist, the box cannot make the detect. Other factors that limit the detect capability of the system as a whole relate to the intrusion-detection analysts and the CIRTs to which they report.

Limitations Caused by the Analyst

Part of the reason for missed detects has to be laid at the feet of the intrusion-detection analyst. There are several issues here. Sometimes, an analyst might mentally evaluate an intrusion attempt and decide it isn't worth investigating. I have been guilty of this multiple times. Here is a classic example: Code Red is still active, because some people don't have the gumption to patch their IIS boxes. On a given day, I see a number of detects on port 80, but I do not tend to evaluate them in depth. I just figure it is Code Red. However, in February 2002, when the Apache PHP vulnerability was reported, I had to suddenly change my ways. After all, I run Apache.

Does an analyst report something he doesn't understand? Unknown patterns are challenging and require a significant understanding of TCP/IP and computer system processes to run to ground. What if the analyst doesn't trust her intrusion-detection system? It takes a lot of faith to sign a report based on a little picture on a console telling you such and such just happened. It takes even more faith to do this when the same IDS reports two email Wiz attacks (Wiz is a very, very old email attack) per day and six SYN floods per hour (and these are obviously false positives). Therefore, analysts are most certainly a weak link in the system. The reasons for this include the following:

∙Failing to report what the IDS detects

∙Lack of training needed to investigate new attack patterns