- •Table of Contents
- •Preface
- •What This Book Covers
- •What You Need for This Book
- •Conventions
- •Reader Feedback
- •Customer Support
- •Errata
- •Questions
- •The Need for Cryptography
- •Privacy
- •Security
- •A History of the Internet
- •Holding the Internet Together
- •The Creation of ICANN
- •ICANN Bypassed
- •The Root Name Servers
- •Running the Top-Level Domains
- •History of Internet Engineering
- •The Internet Engineering Task Force (IETF)
- •RFCs—Requests For Comments
- •IETF and Crypto
- •The War on Crypto
- •Dual Use
- •Public Cryptography
- •The Escrowed Encryption Standard
- •Export Laws
- •The Summer of '97
- •The EFF DES Cracker
- •Echelon
- •The End of the Export Restrictions
- •Free Software
- •Free as in Verifiable
- •The Open Source Movement
- •The History of Openswan
- •IETF Troubles over DNS
- •Super FreeS/WAN
- •The Arrival of Openswan
- •NETKEY
- •Further Reading
- •Using Openswan
- •Copyright and License Conditions
- •Writing and Contributing Code
- •Legality of Using Openswan
- •International Agreements
- •International Law and Hosting Openswan
- •Unrecognized International Claims
- •Patent Law
- •Expired and Bogus Patents
- •Useful Legal Links
- •Summary
- •A Very Brief Overview of Cryptography
- •Valid Packet Rewriting
- •Ciphers
- •Algorithms
- •Uniqueness
- •Public-Key Algorithms
- •Exchanging Public Keys
- •Digital Signatures
- •Diffie-Hellman Key Exchange
- •Avoiding the Man in the Middle
- •Session Keys
- •Crypto Requirements for IPsec
- •IPsec: A Suite of Protocols
- •Kernel Mode: Packet Handling
- •Authentication Header (AH)
- •Encapsulated Security Payload (ESP)
- •Transport and Tunnel Mode
- •Choosing the IPsec Mode and Type
- •The Kernel State
- •Encryption Details
- •Manual Keying
- •Final Note on Protocols and Ports
- •Usermode: Handling the Trust Relationships
- •The IKE Protocol
- •Phase 1: Creating the ISAKMP SA
- •Phase 2: Quick Mode
- •The NAT Problem
- •Summary
- •Linux Distributions
- •Debian
- •SuSE
- •Slackware
- •Gentoo
- •Linux 'Router' Distributions
- •Deciding on the Userland
- •Pluto
- •Racoon
- •Isakmpd
- •More Reasons to Pick Pluto
- •Choosing the Kernel IPsec Stack
- •KLIPS, the Openswan Stack
- •ipsecX Interfaces
- •First Packet Caching
- •Path MTU Discovery
- •KLIPS' Downside
- •NETKEY, the 2.6 IPsec Stack
- •The USAGI / SuSE IPsec Stack
- •Making the Choice
- •GPL Compliance and KLIPS
- •Binary Installation of the Openswan Userland
- •Checking for Old Versions
- •Installing the Binary Package for Openswan
- •Building from Source
- •Using RPM-based Distributions
- •Rebuilding the Openswan Userland
- •Building src.rpm from Scratch
- •Openswan Options
- •Building the Openswan Userland from Source
- •Downloading the Source Code
- •Configuring the Userland Tools
- •Optional Features
- •Compile Flags
- •File Path Options
- •Obscure Pluto Options
- •Compiling and Installing
- •Binary Installation of KLIPS
- •Building KLIPS from Source
- •Kernel Prerequisites
- •Identifying your Kernel's Abilities
- •Using Both KLIPS and NETKEY
- •The Kernel Build Options
- •Required Kernel Options
- •Desired Options
- •NETKEY Stack Options
- •KLIPS Stack Options
- •L2TP Options
- •Patching the Kernel
- •NAT-Traversal Patch
- •KLIPS Compile Shortcut
- •Activating KLIPS
- •Determining the Stack in Use
- •Building KLIPS into the Linux Kernel Source Tree
- •Building a Standard Kernel
- •NAT Traversal
- •Patching KLIPS into the Linux Kernel
- •Verifying the Installation
- •Summary
- •Manual versus Automatic
- •PSK versus RSA
- •Pitfalls of Debugging IPsec
- •Pre-Flight Check
- •The ipsec verify Command
- •NAT and Masquerading
- •Checking External Commands
- •Opportunistic Encryption
- •The ipsec livetest Command
- •Configuration of Openswan
- •The ipsec.conf File
- •Host-to-Host Tunnel
- •Left and Right
- •The type Options
- •The auto Option
- •The rsasigkey Options
- •Bringing Up the IPsec Tunnels
- •Listing IPsec Connections
- •Testing the IPsec Tunnel
- •Connecting Subnets Through an IPsec Connection
- •Testing Subnet Connections
- •Testing Properly
- •Encrypting the Host and the Network Behind It
- •Employing Advanced Routing
- •Creating More Tunnels
- •Avoiding Duplication
- •The Also Keyword
- •KLIPS and the ipsecX Interfaces
- •Pre-Shared Keys (PSKs)
- •Proper Secrets
- •Dynamic IP Addresses
- •Hostnames
- •Roadwarriors
- •Multiple Roadwarrior Connections
- •Dynamic IP and PSKs
- •Mixing PSK and RSA
- •Connection Management
- •Subnet Extrusion
- •NAT Traversal
- •Deprecated Syntax
- •Confirming a Functional NAT-T
- •Dead Peer Detection
- •DPD Works Both Ways
- •Configuring DPD
- •Buggy Cisco Routers
- •Ciphers and Algorithms
- •Using ike= to Specify Phase 1 Parameters
- •Using esp= to Specify Phase 2 Parameters
- •Defaults and Strictness
- •Unsupported Ciphers and Algorithms
- •Aggressive Mode
- •XAUTH
- •XAUTH Gateway (Server Side)
- •XAUTH Client (Supplicant Side)
- •Fine Tuning
- •Perfect Forward Secrecy
- •Rekeying
- •Key Rollover
- •Summary
- •X.509 Certificates Explained
- •X.509 Objects
- •X.509 Packing
- •Types of Certificates
- •Passphrases, PIN Codes, and Interactivity
- •IKE and Certificates
- •Using the Certificate DN as ID for Openswan
- •Generating Certificates with OpenSSL
- •Setting the Time
- •Configuring OpenSSL
- •Be Consistent with All Certificates
- •OpenSSL Commands for Common Certificate Actions
- •Configuring Apache for IPsec X.509 Files
- •Creating X.509-based Connections
- •Using a Certificate Authority
- •Using Multiple CAs
- •Sending and Receiving Certificate Information
- •Creating your own CA using OpenSSL
- •Creating Host Certificates with Your Own CA
- •Host Certificates for Microsoft Windows (PKCS#12)
- •Certificate Revocation
- •Dynamic CRL Fetching
- •Configuring CRL
- •Online Certificate Status Protocol (OCSP)
- •Summary
- •History of Opportunistic Encryption
- •Trusting Third Parties
- •Trusting the DNS?
- •OE in a Nutshell
- •An OE Security Gateway
- •DNS Key Records
- •Forward and Reverse Zones
- •The OE DNS Records
- •Different Types of OE
- •Policy Groups
- •Internal States
- •Configuring OE
- •Configuring Policies
- •Full OE or Initiate-Only
- •Generating Correct DNS Records
- •Name Server Updates
- •Verifying Your OE Setup
- •Testing Your OE Setup
- •The trap eroute
- •The pass eroute
- •The hold eroute
- •Manipulating OE Connections Manually
- •Advanced OE Setups
- •Caveats
- •Summary
- •Where to Firewall?
- •Allowing IPsec Traffic
- •NAT and IPsec Passthrough
- •Configuring the Firewall on the Openswan Host
- •Firewalling and KLIPS
- •Firewalling and NETKEY
- •Packet Size
- •Summary
- •Microsoft Windows
- •Layer 2 Tunneling Protocol (L2TP)
- •Assigning an IP for VPN Access
- •L2TP Properties
- •Pure IPsec versus L2TP/IPsec
- •Client and Server Configurations for L2TP/IPsec
- •The L2TP Openswan Server
- •Configuring Openswan for L2TP/IPsec
- •Linux Kernel Runtime Parameters for L2TP/IPsec
- •Protecting the L2TP Daemon with IPsec using iptables
- •Choosing an L2TP Daemon
- •Configuring L2TPD
- •Configuring User Authentication for pppd
- •Microsoft Windows XP L2TP Configuration
- •Microsoft Windows 2000 L2TP Configuration
- •Apple Mac OS X L2TP Configuration
- •Server Configuration for X.509 IPsec without L2TP
- •Openswan Configuration for X.509 without L2TP
- •Client Configuration for X.509 IPsec without L2TP
- •Microsoft's IKE Daemon
- •Microsoft's Certificate Store
- •Clients using Microsoft Native IPsec Implementation
- •The ipsec.exe Wrapper
- •The Linsys IPsec Tool (lsipsectool)
- •Securepoint IPsec Client
- •TauVPN (iVPN)
- •The WaveSEC Client
- •Third-Party Replacement Clients for Windows
- •The GreenBow VPN Client
- •Astaro Secure Client
- •Mac OS X IPSecuritas
- •VPNtracker
- •Manual Racoon Configuration
- •Importing X.509 Certificates into Windows
- •Importing X.509 Certificates on Mac OS X (Tiger)
- •Summary
- •Openswan as a Client to an Appliance
- •Preparing the Interop
- •The Human Factor
- •Terminology
- •Preparation
- •IPsec Passthrough
- •Tunnel Limitations
- •Anticipate Known Problems
- •Update the Firmware
- •GUI Issues
- •Keepalives
- •ISP Filtering
- •Frequently used VPN Gateways
- •Webmin with Openswan
- •Cisco VPN 3000
- •Cisco PIX Concentrator
- •Nortel Contivity
- •Checkpoint
- •WatchGuard Firebox
- •Symantec
- •Frequently used VPN Client Appliances
- •ZyXEL
- •DrayTek Vigor
- •The Vigor Web Interface
- •Windows Logon Issues
- •Other Vigorisms
- •Unresolved Issues
- •NetScreen
- •Known Issues
- •SonicWALL
- •BinTec
- •LANCOM
- •Linksys
- •Lucent Brick
- •NETGEAR
- •KAME/Racoon
- •Aftercare
- •Summary
- •Methods of Encryption
- •Host-to-Host Mesh
- •Host-to-Gateway Setup
- •Single IP Extrusiautomation or L2TP
- •Opportunistic Encryption in the LAN
- •Non-OE-Capable Machines
- •Designing a Solution for Encrypting the LAN
- •Design Goals
- •Separation of WiFi and Crypto
- •Link Layer Protection
- •The Logical Choice: IPsec
- •Hotspot
- •WaveSEC
- •Full WaveSEC
- •Catch 22 Traffic
- •Building a WaveSEC Server
- •DHCP Server Setup
- •DNS Server Setup
- •Openswan Server Setup
- •Catch 22 Traffic Setup
- •Building a WaveSEC Client
- •DH Client Setup
- •Openswan Setup
- •Testing the WaveSEC
- •Starting the WaveSEC Connection
- •Known Issues with WaveSEC
- •WaveSEC for Windows
- •Design Limitations
- •Building a WaveSEC for Windows Server
- •Obtaining the Certificate and Client Software
- •Our Prototype Experiences
- •Openswan Issues
- •Windows Kernel Issues
- •Summary
- •Cipher Performance
- •Handling Thousands of Tunnels
- •Managing Large Configuration Files
- •Standard Naming Convention
- •The also= Parameter
- •The include Parameter
- •Openswan Startup Time
- •Limitations of the Random Device
- •Other Performance-Enhancing Factors
- •Logging to Disk
- •Disable Dead Peer Detection
- •Reducing the Number of Tunnels
- •OSPF Setup
- •BGPv4 Setup
- •High Availability
- •Heartbeat
- •Xen Migration
- •Using Anycast
- •Summary
- •Do Not Lock Yourself Out!
- •Narrowing Down the Problem
- •Host Issues
- •Configuration Problems
- •Connection Names
- •Interoperability
- •Hunting Ghosts
- •Rekey Problems (After an Hour)
- •Openswan Error Messages
- •IKE: Unknown VendorIDs
- •Network Issues
- •Firewalls
- •MTU and Fragmentation Issues
- •Debugging IPsec on Apple Mac OS X
- •Debugging IPsec on Microsoft Windows
- •Oakley Debugging
- •Debugging ipsec.exe
- •Microsoft L2TP Errors
- •You Suddenly Cannot Log in Anymore over the VPN
- •Software Bugs
- •Userland Issues: Assertion Failed or Segmentation Faults
- •Kernel Issues: Crashes and Oopses
- •Memory Issues
- •Common IKE Error Messages
- •Common Kernel-Related Error Messages
- •Common Errors when Upgrading
- •Using tcpdump to Debug IPsec
- •Situation A: No Communication on Port 500
- •Situation B: Failure at Third Exchange
- •Situation C: QUICK Mode Initiates, but Never Completes
- •Situation D: All IKE Messages Occur, but no Traffic Flows
- •A Final tcpdump Example
- •User Mode Linux Testing
- •Preparing the Openswan for the UML Build Process
- •Running the UMLs
- •Writing a UML Test Case
- •Debugging the Kernel with GDB
- •Asking the Openswan Community for Help
- •Internet Relay Chat (IRC)
- •The Openswan Mailing Lists
- •Posting to the Lists
- •Research First, Ask Later
- •Free, as in Beer
- •Do not Anonymize
- •Summary
- •Linux Kernel Developments
- •Kernel API Changes between 2.6.12 and 2.6.14
- •Red Hat Kernel Developments
- •Fedora Kernel Source/Headers Packaging Change
- •MD5 Insecurities
- •Discontinuation of Openswan 1 by the End of 2005
- •Update on UML Testing Suite Installation
- •Openswan GIT Repositories
- •Openswan on Windows and Mac OS X Updates
- •Known Outstanding Bugs
- •Vulnerability Fixes in Openswan 2.4.4
- •The OSI Model and the IP Model
- •No Layers, Just Packets
- •The Protocol
- •IP Network Overview
- •IP Address Management
- •The Old IP Classes
- •Classless IP Networks
- •The Definition of a Subnet
- •Calculating with Subnets: The Subnet Mask
- •The Rest of the Network
- •Linux Networking Commands
- •Routing
- •Routing Decisions
- •Peering
- •Network Address Translation
- •Port Forwarding
- •Openswan Links
- •Community Documentation
- •Generic Linux Distributions Containing Openswan
- •Specialized Linux Distributions Containing Openswan
- •Overview RFCs
- •Basic Protocols
- •Key Management
- •Procedural and Operational RFCs
- •Detailed RFCs on Specific Cryptographic Algorithms and Ciphers
- •Dead Peer Detection RFCs
- •NAT-Traversal and UDP Encapsulation RFCs
- •RFCs for Secure DNS Service, which IPSEC May Use
- •RFCs Related to L2TP, Often Used in Combination with IPsec
- •RFCs on IPsec in Relation to Other Protocols
- •RFCs Not in Use or Implemented across Multiple Vendors
- •Index
2
Practical Overview of the IPsec Protocol
The focus of this chapter is on the cryptographic theories and IPsec protocols you need to know about as a system administrator. We will not look at detailed mathematical formulas but instead will explain the basics of cryptography so that you can understand the key management and packet processing performed by the IPsec protocols. References to all the appropriate RFCs and drafts are in the appendix, so those who want to dive deep into the mathematical core of cryptography can do so.
A Very Brief Overview of Cryptography
Normal IP packets consist of the IP header and the IP data, or payload. The IP header contains information about where the packet came from, where it should be going to, what kind of (sub-) protocol the packet has, the size of the packet, the time-to-live (TTL, sometimes called hopcount), some option bits that tweak little things, and finally an extra verification number, called the checksum. The checksum is a simple addition of all numbers in the IP packet. If one number gets accidentally changed during transmission, the checksum will be different, allowing the packet to be recognized as 'broken'.
Practical Overview of the IPsec Protocol
For instance, this packet could be a UDP packet, in which case the protocol field in the IP header would have the value 17. It has a source and destination IP address, and a source port and destination port, since the UDP protocol uses ports. Within the header, there is a checksum that can be used by routers to see if the packet has been mangled. If so, the packet is dropped. Even though the checksum detects mangling that happens by accident, it is not sufficient to protect against the packet being altered, since if someone wants to change a packet, they can simply calculate what the new checksum needs to be and alter that as well. To protect against malicious packet tampering, one needs to have a stronger method for verifying the packet than a simple checksum. This is where cryptographic functions come into play. Instead of a simple checksum that everyone can generate, we need to add some kind of cryptographic checksum that only the sender and receiver know how to make and verify.
Valid Packet Rewriting
There is, however, a catch with this idea of a checksum that can only be made and checked by sender and receiver. As transmitted packets travel across the network, they are not immutable and some items in the IP header can change quite legitimately.
Probably the most important is the time-to-live (TTL). When a packet is passed along by a router, it must decrease its TTL value by 1. If that value would become 0, the packet has traveled over too many hops and is dropped. A special control packet, using the ICMP protocol, is sent back to the sender notifying it of the lost packet. The whole idea of TTL is to ensure packets don't travel in loops on the Internet for ever.
Other parts that may be changed are those that handle quality-of-service issues. Some networks may treat certain types of packets as less important, by giving such packets a lower priority. They can set QoS bits in the header to indicate these policies to their devices.
Ciphers
Ciphers and algorithms are the two main types of cryptographic functions used with IPsec. A cipher is nothing more than a deterministic scrambling scheme. If you have an unencrypted value X (the plaintext) and you push it through a cypher box, you get a scrambled text Y (the ciphertext). The sizes of the plaintext and the ciphertext are the same.
An often-used toy cipher is called ROT13. The ROT13 cipher works like this: Replace all letters with the letter 13 spaces further in the alphabet. If you get to Z, continue counting at A. If we put 'ABC' through this cipher, we could get 'NOP. If we put 'NOP' through our ROT13 cipher, we would get 'ABC'.
The problem is that the security of our cipher lies in the secrecy of it. Anyone who knows, or works out, how our secret cipher works, is able to decipher all our messages. It is very dangerous to rely on the secrecy of a cipher. You cannot ask a lot of people (mathematicians or cryptanalysts) whether your cipher is good without betraying your secret. And cryptography is deemed too difficult for a single person to securely invent in their basement.
A more common type of cipher is one where the cipher method is not a secret, and is known to everyone, but the cipher takes two inputs instead of one. One input is the plaintext, and the other
28
Chapter 2
input is a secret only known by the two parties involved. This secret is called a key, in this case a secret key. The cipher mangles the plaintext with the key and the result is the ciphertext. The receiver uses the same key and the ciphertext as input to the cipher to recreate the plaintext. This is called a symmetric cipher. Because of the use of a secret key, we do not need to keep the cipher secret. We only need to keep our key secret. And if someone steals our key, we can just decide on another key to use; we do not have to throw away the cipher.
DES, 3DES, and AES
The first cipher to enter widespread use throughout the world was DES, the Digital Encryption Standard. It was designed by IBM in the seventies, and slightly (but crucially!) modified by the NSA. Since then, computers have become much more powerful and DES is no longer secure against a brute force attack. This is an attack where all possible keys are tried on the ciphertext until the attacker stumbles upon the plaintext when they happen to use the right key.
Most installations using DES switched to triple-DES (3DES) years ago. Recently, the Rijndael cipher was chosen as the successor for DES, also called the Advanced Encryption Standard (AES) by the National Institute of Standards and Technology (NIST) in the US.
Algorithms
The cryptographic algorithms used with IPsec are mathematical one-way functions. A one-way function, like a cipher, takes an input and produces an output, but unlike a cipher, you cannot use the output and any other function to obtain the input. Hence the name 'one-way'. In IPsec terminology, these are often called the algorithms.
These types of algorithms are frequently used to make secure checksums for data. Again, the security of the algorithm does not lie with its secrecy, but in the fact that it is next to impossible to modify the input in such a way that it produces the same output. Algorithms that create a much shorter output than input are called hash functions. The two most commonly used hash algorithms are Message Digest 5 (MD5) and Secure Hashing Algorithm 1 (SHA1).
Recently a lot of media attention has focused on how MD5 and SHA1 have been hacked or broken. The truth is slightly less worrying. Researchers have found 'collisions' in these functions, which means that someone could find another plaintext that has the same MD5 or SHA1 hash as the original plaintext. However this does not mean it's possible to swap the plaintext with any other arbitrary plaintext at will. The implementation in IPsec uses a sequence number scheme called HMAC, which makes it even harder to find the proper alternative plaintext that would pass the MD5 or SHA1 algorithm.
MD5 and SHA1 are still safe algorithms to use for IPsec, but if a newer algorithm becomes available and you can use it, it is recommended to switch. A likely candidate for inclusion in the near future is SHA-256 or SHA-512.
29
Practical Overview of the IPsec Protocol
Uniqueness
One way eavesdroppers could still manipulate a secure communication would be to capture valid communication, and even though they cannot read this captured communication, to resend it. Since these resent packets are valid, digitally encrypted packets, they will pass the checks for proper authentication. This might disrupt the secure communication, or trigger other responses. This is called a replay attack. One way to avoid it is to add a counter to all the packets before running them through the cipher. Every time the counter is used, it is increased. This way, a replayed packet can be correctly identified as an old packet, and can be discarded.
Public-Key Algorithms
Of course, using a secret key is a Catch 22 situation. If you could communicate the secret key securely, you would not need a new form of encrypted communication to begin with. The solution to this problem was invented by Diffie and Hellman around the time DES was introduced. Without going into too much mathematical detail, the idea is to generate two very large numbers and use a one-way function in such a way that using one of the numbers as input to the function gives a ciphertext, but if you have the second large number, you can actually reverse the one-way function. This second number is a so-called 'trap door'.
The first large number you can give away to everyone, and is called the public key. The second number is your private key, and no one but you should know it. People can use the public key to encrypt a message that only you can decrypt with the private key.
Exchanging Public Keys
If two parties (by convention known as Alice and Bob) have never securely exchanged each other's public key, then they have a problem to solve before they can communicate. How do they know that they are talking to the person that they think they are talking to? How can each be sure that they are not talking to a 'man in the middle'? A man in the middle (often called Mallory) is someone who pretends he is Bob to Alice, and pretends he is Alice to Bob.
Digital Signatures
This public key encryption scheme can even be extended. Using a more complex algorithm, such as RSA or DSA, the private key can also be used to create a digital signature. The principle of a digital signature is similar to that of a handwritten signature. The assumption is that only the person represented by the signature is able to produce a correct signature. If your signature appears on a piece of paper, and it matches the reference signature that is on file that you have made in the past in the presence of a notary or bank employee, then it is assumed that only you could have written that signature, so therefore you signed that piece of paper. For a digital signature, the same assumption is used. Anyone with the public key can verify a digital signature, but only the person with the private key can make it.
Diffie-Hellman Key Exchange
The Diffie-Hellman (DH) key exchange allows you to exchange a secret key over a public channel, and thereby gain privacy no matter who listens in on this DH key exchange.
30