value of 4), it is greater than 3, meaning that the low-order reserved bit has been set. The scanner's hope is that the response to this bogus flag setting indicates something unique about the operating system.

Now, take a look at the response of target.com to scanner.com. Our interest and nmap's interest is the response to the bogus TCP flag bit set. Again, the normal TCPdump output display does not show the reserved bits of the TCP flag byte. The hexadecimal dump that does

show the TCP flag byte follows this:

target.com.domain > scanner.com.44388: S 4154976859:4154976859(0) ack 403915839 win 8855 <nop,nop,timestamp 16912287 1061109567,nop,wscale 0,mss 265> (DF)

[4500 003c e04e 4000 ff06 e6af 83da d684 83da d683] 0035 ad64 f7a7 ea5b 1813 443f a012 2297 fd3f 0000 0101 080a 0102 0f9f 3f3f 3f3f 0103 0300 0204 0109

Look at the response to the bogus TCP flag bits in the preceding TCPdump output. target.com responds with a SYN/ACK—nothing rancid here. It appears that target.com did not react to the abnormal TCP flag bit set. How do you know? The hexadecimal output of the transaction shows that the response has the SYN and ACK bit set in the TCP flag byte with a hexadecimal value of 12 (in bold). The ACK bit is in the low-order bit of the high-order nibble, so it represents the value 1. The SYN bit is in the low-order nibble, second bit from the left, and represents the 2 value. Therefore, the response discarded the bogus TCP flag bit. Another operating system

might have preserved that bit, and it would have been reflected in the TCP flag byte.

Anomalous TCP Flag Combinations

RFC 793 elaborates normal TCP flag state settings and transitions in extensive detail. It seems likely that most operating system TCP/IP stacks would conform to the specifications. For the most part, they do, but there are the rare exceptions that do not conform and are therefore identifiable by their lack of conformity. Look at the following TCPdump output from an excerpt of traffic produced by running nmap in operating system fingerprinting mode (-O command

line option) for a host named win98: nmap –O win98

20:33:16.409759 verbo.47322 > win98.netbios-ssn: SFP 861966446:861966446(0) win 3072 urg 0 <wscale 10,nop,mss 265,timestamp 1061109567[|tcp]>

20:33:16.410387 win98.netbios-ssn > verbo.47322: S 49904150:49904150(0) ack 861966447 win 8215 <mss 1460> (DF)

The scanning host sends a packet with the TCP flags of SYN, FIN, and PUSH simultaneously set. Logically, it appears that this is an anomalous flag trio because a SYN flag starts a connection, a FIN flag closes a connection, and a PUSH flag sends data after a connection is opened or before a connection is closed. It would seem a natural reaction that a host receiving this connection would ignore it or perhaps RESET it because it makes no sense.Yet, the Windows 98 target host appears to interpret this as session establishment and responds with a SYN and an ACK. This unique reaction helps identify the responding host as having a Windows TCP/IP stack.

No TCP Flags

As another example of nmap fingerprinting, look at the following TCPdump output. It shows a TCP segment with no TCP flag bits set. This is another instance of sending a mutant TCP flag byte setting. In this case, no flag bits have been turned on; this is also known as a null

session:

scanner.com.44389 > target.com.domain: . win 4096 <wscale 10,nop,mss 265, timestamp 1061109567 0,eol> (DF)

[4500 003c 7543 4000 3b06 15bc 0102 0304 0102 0305] ad65 0035 1813 443e 0000 0000 a000 1000 fa8d 0000 0303 0a01 0204 0109 080a 3f3f 3f3f 0000 0000 0000

Look at the previous hexadecimal output. The TCP flag byte field, which is in bold, has a value of 00. This means that no TCP flags have been set. Most hosts will not respond to a null session, yet some must, otherwise nmap would have no reason to send this kind of traffic.

A normal TCP flag byte has at least one flag bit set. The host target.com did not respond at all to this null session TCP segment. The lack of response provides some clue about the operating system. Another operating system might distinguish itself by responding differently, perhaps by replying with a RESET.

Using TCP Options for OS Identification

Look at the following TCPdump output from an nmap scan with the focus on the

bolded TCP options:

scanner.com.44388 > target.com.domain: S 403915838:403915838(0) win

4096

<wscale 10,nop,mss 265,timestamp 1061109567 0,eol> (DF) target.com.domain > scanner.com.44388: S 4154976859:4154976859(0)

ack

403915839 win 8855 <nop,nop,timestamp 16912287

1061109567,nop,wscale 0,mss 265> (DF)

One of the other methods that nmap uses to identify a particular operating system is to send many different TCP options. Some operating systems do not support all these options, and the response discards some. Also, some operating systems set different values for some of the TCP options, further differentiating the fingerprint. Unlike the other examples discussed so far, these are not unconventional stimuli, but are mentioned because they help identify the remote operating system.

Finally, different operating systems will store these options in a different order in the TCP header, which is indicated by the order in which TCPdump lists them. All this information can contain a bounty of identifying clues. As you see in the response to the preceding options, the order has been changed and some of the values have been altered (such as the wscale changing from 10 to 0 in the response). Also notice that the nop and eol options are rearranged or disappear in the response. These fields are used to pad TCP options to 4-byte boundaries and might not be needed in the response.

For an in-depth discussion of TCP options, take a look at RFC 1323. Some of the TCP options seen in the TCPdump output are as follows:

● -wscale. This option allows the TCP window size to increase to a value

greater than 65535 bytes. This is typically used to increase throughput of TCP

over high-bandwidth, long-delay networks.

● -timestamp. This option records round-trip time measurements. These

measurements are often necessary to optimize throughput based on changes in network conditions.

● -nop. This option is used to add a 1-byte pad to TCP options. TCP options

must fall on 4-byte boundaries; and if they are less than 4 bytes, the nop is used to pad.

● -eol. This is the end-of-list option used to pad a final byte to a 4-byte boundary.