542	Part VI Troubleshooting and Maintaining System Hardware

You can always test the service by using the following command:

ftp localhost

Or, try using FTP from a different machine to verify that the service is running.

Mail services

The failure of mail services — whether it’s incoming mail services, such as pop3, or imap, or sending services, such as smtp — is among the most frustrating problems. You can check most of these services by using the ps -ef command to find the running processes.

You can also test to see if these protocols are accepting connections for their particular port. For example, pop3 uses port 110 to listen for requests. By using the telnet command, you can telnet to that port to see if you get a reply:

telnet hostname 110

You can do the same for imap (port 143) and smtp (port 25).

If neither of these tests succeeds, then you must check your /etc/inetd.conf file to ensure that these services are enabled. If they are commented out, then that service is disabled. You must uncomment the line and restart the inetd process.

Basic Networking Troubleshooting

6.16 Use network utilities to identify network and connectivity problems (e.g., ping, route, traceroute, netstat, Isof)

Networks are very complex systems. Several devices, hosts, and software comprise a network, and problems with any one of these can have consequences ranging from smaller problems, such as the inability of a host to connect to the network, to an enterprise-wide network of problems.

You should always check network problems starting from the client side and moving up. Check the client’s networking settings and check the cables, and then move up the chain to the local wiring closet, and to the servers, central hubs, switches, and routers of your network.

By starting with the most obvious areas first and moving up, you can methodically trace a network problem to its root.

Networking connectivity

You can benefit from several Linux commands and utilities when troubleshooting network connectivity.

Chapter 18 Troubleshooting Software and Networking

543

Ping

You can use the ping command to test connectivity from one machine or device to another. Ping stands for Packet Internet Groper, and basically sends out a network packet to a specified machine. That machine, in turn, will send a packet back to the signal, stating that it is alive and on the network.

Ping uses the ICMP (Internet Control Message Protocol) to perform its functions. ICMP is an extension of the IP protocol, and allows error, control, and informational packets to be sent and received.

Suppose that you have just installed a Linux server on your network, and you want to test the connection from a client machine to see if the server is on the network. You can ping the server from your machine. If you don’t receive a reply, your server network is not set up properly. Similarly, you can test connectivity from the new server by trying to ping another host on the network — or, more commonly — the default gateway for the network.

The following is an example of the ping command:

ping www.hungryminds.com

PING hungryminds.com (168.215.86.100) from 10.1.1.5: 56(84) bytes of data. 64 bytes from websrv.hungryminds.com (168.215.86.100): icmp_seq=0 ttl=49 time=26.549 msec

64 bytes from websrv.hungryminds.com (168.215.86.100): icmp_seq=1 ttl=113 time=19.972 msec

64 bytes from websrv.hungryminds.com (168.215.86.100): icmp_seq=2 ttl=113 time=29.972 msec

In the preceding example, you try to ping a Web site to check for connectivity. Ping will show you how long it takes for the specified host to ping back a reply. Very long ping response times result in a “Request Timed Out” error. This error can occur during times of high network activity, if the host is not available.

Attempting to ping an address on the Internet is quite common, but if you are trying to ping a local host on your own private network, this may indicate network and routing problems. If you receive repeated request errors with no ping replies, then that host is experiencing networking problems, or it isn’t connected at all.

Some Internet and corporate sites use a firewall to block ping requests from outside hosts. Allowing ping requests can be a security risk, because a hacker can have a machine send repeated ping requests to a server to prevent it from processing any other network requests. This action is often called the “ping of death” type of attack.

Traceroute

Use the traceroute utility to trace a network packet from one host to another so you can see how many “hops” or Internet routers it passes through before reaching its destination. Traceroute is used most often to troubleshoot routing and latency

544	Part VI Troubleshooting and Maintaining System Hardware

issues between hosts in a WAN or Internet environment. As with the ping utility, traceroute also uses the ICMP protocol for its functions, specifically the TTL (Time to Live) data for each packet. When you initiate the traceroute command, network packets are sent to the destination host. The TTL setting for each packet is a measurement of how long that packet can remain alive in between hops until it is returned. If the TTL is too short, it is eventually returned without reaching its target. After each hop or router in a traceroute operation, the packets are sent with longer TTL settings until the final destination is reached.

The following example shows how to trace the amount of hops between a local machine and a Web site:

traceroute www.hungryminds.com

traceroute to hungryminds.com (168.215.86.100),30 hops max, 38 byte packets 1 gateway (10.1.1.1) 4.863 ms 1.264 ms 2.130 ms

2 Router (10.1.2.254) 3.907 ms 5.014 ms 2.315 ms

3 216.191.195.169 (216.191.195.169) 4.221 ms 4.088 ms 4.421 ms

4 atm7-0-71.core1-tor.bb.attcanada.ca (216.191.67.65) 8.745 ms 8.053 ms 12.336 ms

5 srp2-0.gwy1-tor.bb.attcanada.ca (216.191.65.243) 8.676 ms 8.692 ms 14.571 ms

6 12.125.142.5 (12.125.142.5) 23.113 ms 23.146 ms 22.749 ms

7 gbr6-p80.cgcil.ip.att.net (12.123.5.222) 19.464 ms 19.108 ms 19.437 ms

8 12.122.9.133 (12.122.9.133) 25.734 ms 73.168 ms 42.280 ms

9 12.122.11.57 (12.122.11.57) 43.276 ms 20.764 ms 21.492 ms

10 12.122.11.50 (12.122.11.50) 19.267 ms 19.324 ms 19.239 ms

11 gr1-p340.cgcil.ip.att.net (12.123.4.249) 26.212 ms 19.618 ms 19.271 ms

12 pa2-atm0-1-aads-igr01.chi.twtelecom.net (206.220.243.116) 29.284 ms 24.657 ms 45.476 ms

13 jr-01-at-0-1-0-1.chcg.twtelecom.net (207.67.50.85) 23.328 ms 23.336 ms 31.708 ms

14 jr-04-so-2-0-0-155m.chcg.twtelecom.net (168.215.53.37) 23.143 ms 27.103 ms 22.529 ms

15 jr-01-so-2-0-0-622m.iplt.twtelecom.net (168.215.53.18) 31.682 ms 31.464 ms 31.775 ms

16 cr-01-pos-5-0-0-155m.iplt.twtelecom.net (207.67.94.194) 32.500 ms 31.575 ms 32.648 ms

17 168-215-52-186.twtelecom.net (207.67.94.186) 40.461 ms 30.792 ms 29.252 ms

18 websrv.hungryminds.com (168.215.86.100) 30.077 ms 30.092 ms 30.376 ms

You can see the number of routers or “hops” that your network packets have to hit before reaching their final destination. If the request times out at some point, you will see a group of asterisks (*) in the destination, and it won’t go any further. These asterisks will let you know the exact point at which your network has lost communication. The traceroute command is particularly useful in internal local or wide area networks — especially if you are experiencing communication problems with a particular router or site in your network.

Chapter 18 Troubleshooting Software and Networking

545

Route

Use the route command to show and manipulate your local machine’s IP routing table. The routing table configures your system to route packets to certain addresses through another gateway or router. The routing table also allows you to choose which network interface you need to use for routing, because you have more than one network card in a system.

The entire topic of routing is beyond the scope of this chapter, but when troubleshooting network problems, you may have to make adjustments to your routing table in order for networking to function properly.

By simply issuing the following command, you can display your current routing table, which shows the destination address, gateway, and interface for each routing entry:

route

The following is a sample output from the route command:

Kernel IP routing table
Destination Gateway		Genmask		Flags Metric Ref Use Iface
10.1.0.0	*	255.255.0.0	U	0	0	0 eth0
127.0.0.0	*	255.0.0.0	U	0	0	0 lo
default	10.1.0.254	0.0.0.0	UG	0	0	0 eth0

This example shows the default gateway, 10.1.0.254, which is a central router for that particular network. The interface shows which network card is using that route; in this case, the network card is an ethernet card, “eth0”, and the local loopback interface “lo” is a virtual networking adapter used for internal loopback tests.

If you can’t reach a host because it is located on another network or subnet, you can add a static route to the proper destination gateway that will forward your request. The general syntax for adding routes is similar to the following:

route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.1.254

This syntax tells the system to add the network 192.168.1.0 to the routing table, and to go through the gateway address of 192.168.1.254, which can be a router, or routing interface on a device or host.

Similarly, you can delete static routes by using the route del command.

Netstat

Netstat is a very powerful command that can provide a wide variety of information about your network interface cards and network connections. Use the netstat command to show which network connections are currently being used on your system. This command also provides you with information about which daemons

546	Part VI Troubleshooting and Maintaining System Hardware

are listening on certain ports. This information is helpful in determining which ports are currently active, and waiting for network requests. If you are worried about security and want to see which connections you are using or listening to, use the following command:

netstat -l

This command shows information on all listening ports, while the netstat command on its own can be used to show active network connections. The following is a sample output from the netstat command showing connections, such as telnet, ssh, and ftp:

Active	Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address				State
tcp	0	124 host:telnet 10.1.0.188:4339 ESTABLISHED
tcp	0	0	host:ssh 10.1.0.189:4014 ESTABLISHED
tcp	0	0	host:ssh 10.1.0.189:3560 ESTABLISHED
tcp	0	0	host:ftp 10.1.0.190:21 ESTABLISHED

Active UNIX domain sockets (w/o servers)

Proto RefCnt Flags			Type	State	I-Node Path
unix 7	[ ]	DGRAM		338	/dev/log
unix 0	[ ]	DGRAM		1037545
unix 0	[ ]	DGRAM		181851
unix 0	[ ]	DGRAM		723
unix 0	[ ]	DGRAM		520
unix 0	[ ]	DGRAM		425
unix 0	[ ]	DGRAM		361
unix 0	[ ]	DGRAM		348

The netstat command has many other powerful options:

netstat -r: Shows the Routing table

netstat –I: Shows statistics for network interfaces

Kernel Interface table

Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flags

lo	0	0	4256	0	0	0	3185	0	0	0 BLRU
eth0	1500		0 873009		19	23	156 647383			123	0	0 BRU

These interface statistics show you the number of packets transmitted and received, including any errors that were encountered. A high number of errors indicate very high network traffic or a faulty interface card.

Lsof

Lsof (List open files) lists information about any files that are open by processes, and whether those files are currently running on any system. An open file may be a regular file, a directory, a library, a stream, or a network file, such as a network socket. Lsof is a great security tool for a Linux system administrator because it can show you open network sockets and files on your system.