Building Firewalls With OpenBSD And PF, 2nd Edition (2003)

Section 10.2: Bandwidth Shaping	207

outside world

ext_if parent queue 5% reserve

queue dmznet bandwidth 50% of parent

queue prvnet bandwidth 45% of parent

Figure 10.3: Dividing bandwidth between two internal network segments with CBQ.

#others (default, connections from the firewall itself) altq on $ext_if cbq bandwidth 45Mb \

queue{dmznet, prvnet, others}

#CHILD QUEUE DEFINITIONS

#give the outbound traffic from the DMZ 50% of parent

#bandwidth

queue dmznet bandwidth 50% priority 6 cbq(red)

#give the outbound traffic from the private network 49Mb of

#bandwidth

queue prvnet bandwidth 45% priority 6 cbq(red)

#give the outbound traffic from the firewall host itself queue others bandwidth 5% priority 5 cbq(default)

#PACKET TRANSLATION

#Add NAT rules with clearly defined port ranges, so we know

#where the outbound packets are coming from (NAT will change

#their source address, and we'll need to use port numbers to

#know who's who

#Turn NAT off for connections between the private network

#and the DMZ segment

208	Chapter 10: Bandwidth Shaping and Load Balancing

no nat on $ext_if inet proto {tcp, udp} \ from $dmz_if:network to $prv_if:network no nat on $ext_if inet proto {tcp, udp} \ from $prv_if:network to $dmz_if:network

#do NAT between the DMZ network segment and the outside

#world

nat on $ext_if inet proto {tcp, udp} \

from $dmz_if:network to any -> $ext_if port 1024:32255

#do NAT between the private network segment and the outside

#world

nat on $ext_if inet proto {tcp, udp} \

from $prv_if:network to any -> $ext_if port 32256:65535

# PACKET FILTERING RULES BOUND TO QUEUES

pass out quick on $ext_if inet proto {tcp, udp} \

from ($ext_if) port 1024 >< 32255 to any queue dmznet pass out quick on $ext_if inet proto {tcp, udp} \

from ($ext_if) port 32256 >< 65535 to any queue prvnet

The ruleset shown above shows a basic implementation of packet queuing that imposes bandwidth limits on outbound packets sent from the DMZ and the private network segments. No advanced packet queuing is done for these networks. Notice that the parent queue uses 50% of the interface bandwidth. This is to leave some bandwidth for the incoming connections, otherwise the users on the private network segment may use up all of the available bandwidth.

Let's see how that ruleset could be expanded into a more complex queue tree that implements the following packet queuing policy:

ƒthe parent queue uses 50% of the total available bandwidth.

ƒthe DMZ segment uses 50% of the parent queue bandwidth. Packets are queued using the CBQ scheduler without specifying bandwidth allocation, but with speciŒed priority (like PRIQ, but we cannot use more than one scheduler, and the range of priority levels is smaller: 0 to 7).

ƒthe private segment uses 49% of the parent queue bandwidth.

ƒeach of the n private hosts gets 1/nth of the parent queue's bandwidth (25% for each of the four hosts in this segment).

ƒpacket queueing for each private host is done in a way similar to the queueing policy for DMZ hosts.

Section 10.2: Bandwidth Shaping	209

Here's a sample implementation of such policy:

#MACROS

#external interface ext_if = "ne1"

#DMZ interface dmz_if = "ne2"

#private interface prv_if = "ne3"

#PARENT QUEUE DEFINITION

#define a CBQ parent queue: bandwidth 45Mb, and

#three child queues: dmznet (hosts in the DMZ),

#prvnet (hosts in the private segment),

#others (default)

altq on $ext_if cbq bandwidth 45Mb \ queue{dmznet, prvnet, others}

#CHILD QUEUE DEFINITIONS

#give the outbound traffic from the DMZ 50% of bandwidth queue dmznet bandwidth 50% priority 6 cbq(red) \

queue(dns, ssh, www, mail)

#give the outbound traffic from the private network 49% of

#bandwidth

queue prvnet bandwidth 49% priority 5 cbq(red) \ queue(host1, host2, host3, host4)

#give the outbound traffic from the firewall host itself queue others bandwidth 1% priority 4 cbq(default)

#CHILD QUEUE DEFINITIONS: (for dmznet)

queue dns priority 6 cbq(red, borrow) queue ssh priority 5 cbq(red, borrow) queue mail priority 4 cbq(red, borrow) queue www priority 3 priq(red, borrow)

# CHILD QUEUE DEFINITIONS: (for prvnet)

queue host1 bandwidth 25% priq(red) {dns1, ssh1, mail1, www1} queue host2 bandwidth 25% priq(red) {dns2, ssh2, mail2, www2} queue host3 bandwidth 25% priq(red) {dns3, ssh3, mail3, www3} queue host4 bandwidth 25% priq(red) {dns4, ssh4, mail4, www4}

210	Chapter 10: Bandwidth Shaping and Load Balancing

#CHILD QUEUE DEFINITIONS: (for host1) queue dns1 priority 6 cbq(red, borrow) queue ssh1 priority 5 cbq(red, borrow) queue mail1 priority 4 cbq(red, borrow) queue www1 priority 3 cbq(red, borrow)

#CHILD QUEUE DEFINITIONS: (for host2) queue dns2 priority 6 cbq(red, borrow) queue ssh2 priority 5 cbq(red, borrow) queue mail2 priority 4 cbq(red, borrow) queue www2 priority 3 cbq(red, borrow)

#CHILD QUEUE DEFINITIONS: (for host3) queue dns3 priority 6 cbq(red, borrow) queue ssh3 priority 5 cbq(red, borrow) queue mail3 priority 4 cbq(red, borrow) queue www3 priority 3 cbq(red, borrow)

#CHILD QUEUE DEFINITIONS: (for host4) queue dns4 priority 6 cbq(red, borrow) queue ssh4 priority 5 cbq(red, borrow) queue mail4 priority 4 cbq(red, borrow) queue www4 priority 3 cbq(red, borrow)

#NAT RULES

no nat on $ext_if from $dmz_if:network		to $prv_if:network
no nat on $ext_if from $prv_if:network		to $dmz_if:network
nat	on $ext_if from $dmz_if:network to	any \
->	($ext_if) port 1024:32255
nat	on $ext_if from $prv_if:network to	any \
->	($ext_if) port 32256:40574
nat	on $ext_if from $prv_if:network to	any \
->	($ext_if) port 40575:48893
nat	on $ext_if from $prv_if:network to	any \
->	($ext_if) port 48894:57212
nat	on $ext_if from $prv_if:network to	any \
->	($ext_if) port 57213:65535

# FILTERING RULES ASSIGNED TO QUEUES: (for dmznet) pass out quick on $ext_if inet proto {tcp, udp} \

from ($ext_if) port 1024 >< 32255 to any port 53 queue dns

Section 10.2: Bandwidth Shaping	211

queue dmznet bandwidth 50% of total

queue dns priority 14

queue ssh priority 13

queue mail priority 12

queue www priority 11

queue other priority 1

outside world

ext_if parent queue

queue prvnet bandwidth 49% of total

queue host1			queue host2			queue host3			queue host4
bandwidth			bandwidth			bandwidth			bandwidth
25% prvnet			25% prvnet			25% prvnet			25% prvnet
queue dns1			queue dns2			queue dns3			queue dns4
priority 14			priority 14			priority 14			priority 14
queue ssh1			queue ssh2			queue ssh3			queue ssh4
priority 13			priority 13			priority 13			priority 13
queue mail1			queue mail2			queue mail3			queue mail4
priority 12			priority 12			priority 12			priority 12
queue www1			queue www2			queue www3			queue www4
priority 11			priority 11			priority 11			priority 11
queue other1			queue other2			queue other3			queue other4
priority 1			priority 1			priority 1			priority 1

Figure 10.4: A more complex arrangement of queues.

pass out quick on $ext_if	inet proto tcp \
from ($ext_if) port 1024	>< 32255 to any port {22, 23} \
queue ssh
pass out quick on $ext_if inet proto tcp \
from ($ext_if) port 1024	>< 32255 to any \
port {25, 109, 110, 143,	220, 995} queue mail

212	Chapter 10: Bandwidth Shaping and Load Balancing

pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 1024 >< 32255 to any port {80, 443} \ queue www

#FILTERING RULES ASSIGNED TO QUEUES: (for host1) pass out quick on $ext_if inet proto {tcp, udp} \

from ($ext_if) port 32256 >< 40574 to any port 53 queue dns1 pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 32256 >< 40574 to any port {22, 23} \ queue ssh1

pass out quick on $ext_if inet proto tcp \ from ($ext_if) port 32256 >< 40574 to any \ port {25, 109, 110, 143, 220, 995} queue mail1

pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 32256 >< 40574 to any port {80, 443} \ queue www1

#FILTERING RULES ASSIGNED TO QUEUES: (for host2)

pass out quick on $ext_if inet proto {tcp, udp} \

from ($ext_if) port 40575 >< 48893 to any port 53 queue dns2 pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 40575 >< 48893 to any port {22, 23} \ queue ssh2

pass out quick on $ext_if inet proto tcp \ from ($ext_if) port 40575 >< 48893 to any \ port {25, 109, 110, 143, 220, 995} queue mail2

pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 40575 >< 48893 to any port {80, 443} \ queue www2

# FILTERING RULES ASSIGNED TO QUEUES: (for host3) pass out quick on $ext_if inet proto {tcp, udp} \

from ($ext_if) port 48894 >< 57212 to any port 53 queue dns3 pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 48894 >< 57212 to any port {22, 23} \ queue ssh3

pass out quick on $ext_if inet proto tcp \ from ($ext_if) port 48894 >< 57212 to any \ port {25, 109, 110, 143, 220, 995} queue mail3

pass out quick on $ext_if inet proto tcp \

Section 10.2: Bandwidth Shaping	213

from ($ext_if) port 48894 >< 57212 to any port {80, 443} \ queue www3

# FILTERING RULES ASSIGNED TO QUEUES: (for host4) pass out quick on $ext_if inet proto {tcp, udp} \

from ($ext_if) port 57213 >< 65535 to any port 53 queue dns4 pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 57213 >< 65535 to any port {22, 23} \ queue ssh4

pass out quick on $ext_if inet proto tcp \ from ($ext_if) port 57213 >< 65535 to any \ port {25, 109, 110, 143, 220, 995} queue mail4

pass out quick on $ext_if inet proto tcp \

from ($ext_if) port 57213 >< 65535 to any port {80, 443} \ queue www4

10.2.6 Hierarchical Fair Service Curve (HFSC)

The HFSC scheduler offers similar features to CBQ, but enhances the administrator's toolbox with the ability to deŒne two types of packet schedulers: real-time and link-share. When real-time scheduling isn't used or no packets can be scheduled using this algorithm, link-share takes over. When more bandwidth is available, queues will use it, unless you place a cap on the bandwidth. The HFSC behavior is controlled with these parameters:

ƒrealtime Š controls the minimum bandwidth required for the queue. When more bandwidth is available, it will be used unless you use the upperlimit parameter. Packets are scheduled using the real-time scheduling algorithm Œrst. When no packets are legible for real-time scheduling, or when you do not use real-time scheduling, HFSC will use link-share packet scheduling. This parameter is optional.

ƒlinkshare Š sets the minimum `share' of the parent queue bandwidth for the current queue. When more bandwidth is available, it will be used unless you use the upperlimit parameter. If you also use the realtime parameter, it will take precedence over linkshare. This parameter must be used in HFSC queue deŒnitions, or you must use the bandwidth keyword in the queue deŒnition, before the hfsc() Œeld, or pfctl(8) will complain. The simplest HFSC setup is shown later in this section.

ƒupperlimit Š set the maximum bandwidth allowed for the queue.

214	Chapter 10: Bandwidth Shaping and Load Balancing

HFSC will automatically apply this limit to whatever scheduler it is using. When you use this parameter in your queue deŒnition, remember that it must be greater or equal to the limits deŒned in realtime and linkshare. This parameter is optional.

Each of these parameters can be followed by a single numeric value that deŒnes the amount of bandwidth that the queue can use, or it can be followed by a triple of service curve parameters that deŒne: (a) the initial level of bandwidth (m1), (b) time delay (d, measured in milliseconds), and (c) the level to which bandwidth will be adjusted after that time (m2). These parameters deŒne the so-called service curve, which can be convex or concave, with the exception of the realtime parameter, which must be convex. A concave service curve is created, when the m1 parameter is lower than m2. A convex curve is created when m1 is higher than m2. It is also possible to deŒne a •at service curve when m1 is equal to m2. This can be replaced with a single value (m1 and d are omitted).

Although it is possible to deŒne priorities of queues with the priority keyword (see sections describing PRIQ and CBQ), just like CBQ, HFSC will not really make much use of it. The in•uence of that parameter over the Œnal result is not huge and you can omit it to simplify your ruleset.

Another keyword that is largely redundant in queues that use HFSC, is bandwidth. There is really one case when it is handy, a simple HFSC queue conŒguration with a single level of queues, when bandwidth values are used as single-parameter linkshare settings:

altq on $ext_if hfsc bandwidth 45Mb \ queue{dns, ssh, www, mail, other}

queue dns bandwidth 20% queue ssh bandwidth 20% queue mail bandwidth 20% queue www bandwidth 20% queue other hfsc(default)

is the equivalent of:

altq on $ext_if hfsc bandwidth 45Mb \ queue{dns, ssh, www, mail, other}

Section 10.2: Bandwidth Shaping	215

queue dns hfsc(linkshare 20%) queue ssh hfsc(linkshare 20%) queue mail hfsc(linkshare 20%) queue www hfsc(linkshare 20%) queue other hfsc(default)

That was a simple example, let's see if we can come up with something more complex:

# PARENT QUEUE DEFINITION

altq on $ext_if hfsc bandwidth 45Mb \ queue{dmznet, prvnet, others}

#CHILD QUEUE DEFINITIONS

#backlogs that last less than 10 seconds get 50% or more

#of the total available bandwidth, after 10 seconds, that lim-

it

#goes up to 65% of the total available bandwidth

queue dmznet hfsc(linkshare (50% 10000 65%))

#backlogs that last less than 5 seconds get 50% or more

#of the total available bandwidth, after 5 seconds, that limit

#goes down to 25% of the total available bandwidth

queue prvnet hfsc(linkshare (40% 5000 25%)) queue others hfsc(default)

The Œrst important thing to remember is that the sum of the values of parameters of the same kind (e.g. the initial levels of bandwidth of the linkshare limit) must not exceed the total bandwidth available for the current level of queues. The default queue does not take part in this count. You can express these limits in the same units as those used with the bandwidth keyword (%, b, Kb, Mb, Gb).

In the previous example, the 45Mb of bandwidth assigned to the parent queue represents to 100% percent of the available bandwidth. It is divided into three queues: dmznet (takes at least 50% of the total), prvnet (takes at least 40% of bandwidth), and others (takes whatever's left). After the dmznet queue has been backlogged for more than 10 seconds, the HFSC scheduler will start increasing the amount of available bandwidth up to 65% of the total. When more bandwidth is available, it will be used, but that 65% is guaranteed (as is the initial 50%). When backlog disappears, the

216	Chapter 10: Bandwidth Shaping and Load Balancing

scheduler will slowly release the additional bandwidth and go back to 50%. Why increase bandwidth for this queue? To avoid backlogs and handle increased demand for bandwidth. Suppose there's a mail server, or an FTP server sitting in the DMZ and their patters on usage shown that there is an increase in demand near 10:00am and 4:00pm. A rule with two bandwidth levels will take care of this automatically, without tying up the additional bandwidth for the rest of the day. If you closely at the previous ruleset you will notice that the bandwidth allocated to that queue will be decreased after the backlog persists for 5 seconds. Why? To make room for the increased demand for bandwidth for the dmznet queue and to prevent users on the prvnet from hogging too much bandwidth.

As mentioned before, it is OK to use the linkshare parameter with the realtime parameter in the same queue, HFSC will automatically choose the one that will do a better job. You can use the same parameters for both, but some administrators differ them slightly. It's your choice, use whatever works best in your particular case, but remember that the service curve for realtime must be convex. In other words, when you use that parameter, you can only deŒne a decrease the amount of available bandwidth over time. If you want to deŒne an increase, use the linkshare parameter:

# PARENT QUEUE DEFINITION

altq on $ext_if hfsc bandwidth 45Mb \ queue{dmznet, prvnet, others}

#CHILD QUEUE DEFINITIONS

#backlogs that last less than 10 seconds get 50% or more

#of the total available bandwidth, after 10 seconds, that

#limit goes up to 65% of the total available bandwidth queue dmznet hfsc(linkshare (50% 10000 65%))

#backlogs that last less than 5 seconds get 50% or more

#of the total available bandwidth, after 5 seconds, that

#limit goes down to 25% of the total available bandwidth queue prvnet \

hfsc(realtime (40% 5000 25%) linkshare (40% 5000 25%)) queue others hfsc(default)

As was already mentioned, you can deŒne a concave service curve for the linkshare parameter to decrease the amount of bandwidth available to some users and prevent thm hogging too much bandwidth. A much better