Hardening OpenBSD Internet Servers:
OpenBSD 2.9 to 3.0 Changes

Significant changes between OpenBSD 2.9 and 3.0, at least as they impact the hardening procedures discussed in these pages, are the change from Darren Reed's IP Filter to the new Packet Filter maintained by the OpenBSD team and upgrades to Sendmail. There are, as always, minor changes to the kernel configuration files. An overview and impact of the changes is discussed here. Details are covered in the relevant sections.

The changes to Sendmail appear to be related to changes to the Sendmail product itself and are likely to be encounterd on any platform. Apparently it is now necessary for Sendmail to run as a daemon. "ps ax" shows "sendmail: accepting connections (sendmail)" which is somewhat disturbing. The accepted connections are only on the local host, not from any remote system.

Sendmail is now very insistent on doing DNS lookups and if it cannot properly resolve the IP address of the current machine, doesn't deliver mail. Todd C. Miller of the OpenBSD development team provided the necessary changes to the Sendmail configuration files. Unless Sendmail's DNS behaviour is changed, e-mail on a bridged firewall, or any system without proper DNS setups, is not delivered. Also Sendmail does not run SUID root but SGID as smmsp, an unprivileged user who cannot login. This change significantly reduces the damage that can be caused as a result of a sendmail bug and does not appear to relate to either of the two just mentioned issues.

A third Sendmail change will affect only those who build a custom kernel without IPV6 support and use the default Sendmail configuration file. If you do this then two "DaemonPortOptions" lines must be removed from the appropriate configuration file in /etc/mail. Otherwise Sendmail generates errors that prevent it from starting. If you use the GENERIC kernel or keep IPV6 support in a custom kernel there is no issue.

IP Filter was removed from the OpenBSD source tree before the official 2.9 CDs shipped, but was still included on the official 2.9 CDs. 3.0 includes Packet Filter and all signs of IP Filter are gone. It was my understanding that the replacement was going to be fully compatible with IP Filter but I've been unable to find support for this on OpenBSD web pages or mail list archives. While Packet Filter is conceptually very similar to IP Filter and most IP Filter rules work without change, the switch is certainly not transparent. The names of commands that control the firewall and logging and many options have changed. The OpenBSD team has used the change as an opportunity to improve the firewall.

The functions previously performed by ipf, ipnat and ipfstat are consolidated in pfctl. A review of the man pages will allow anyone familiar with IP Filter to easily adapt to Packet Filter. The rules now include a "macro" facility which is the ability to define variables which are substituted when the rule is interpreted. Used carefully, this should significantly simplify the creation of transportable rule sets. There is no need to include macros in a existing rule files.

There are several rules syntax changes. "with opt . . ." which allowed specific IP option settings to be blocked is no longer valid. Packet Filter by default blocks all packets with any options set, even if the default rule setting is to pass everything. The new "allow-opts" allows packets with options set to be passed. There does not appear to be any way to allow some IP option settings without allowing all in Packet Filter. Also, at least the "all" source and destination specifier, must be included in Packet Filter rules, where IP Filter would accept a rule with an implicit "all". The "head" and "group" keywords were also removed from Packet Filter rules. The "level" option of the log keyword is gone and "keep frags" is replaced with "scrub". Rules cannot be inserted with @n at a specific point in the rules.

There is an even more important and to the best of my knowledge undocumented (except in misc@openbsd.org in response to my question) issue in that Packet Filter may evaluate state in an identical rule set in a fundamentally different manner than IP Filter. IP Filter state applies to the firewall where Packet Filter state applies to the interface. In IP Filter, state created on one interface applies to others so a block all rule on the other interface wont block a stateful connection established on a different interface. Packet Filter states apply only to the interface on which they are defined.

This is most likely to affect anyone using a bridging firewall since IP Filter did not allow "out" rules to be defined on a bridging firewall. This necessitated using all interfaces to control traffic. Adapting a set of IP Filter rules for a dual homed, bridging firewall, to Packet filter is the simple task of switching all "in" rules on the inside interface to "out" rules on the outside interface. The outside interface obviously makes the best sense as this best protects the firewall itself as well as the machines behind it.

I call this undocumented because nothing in my reading of the state description of the pf.conf man page addresses the interface specific nature of Packet Filter state. This is a very frustrating problem because valid old IP Filter rules clearly don't work and state is obviously being handled differently but nothing in any official documentation that I've seen suggests the solution. In the FAQs related to Packet Filter in the "Differences between PF and IPF" subsection it's stated that "Packet Filter (PF) was designed to be very compatible with IPF's ipf.rules file." and then four rules syntax changes are listed missing two additional changes I've found. There is no suggestion that the same rules can produce entirely different results.

The largest change related to Packet Filter is the log format. The daemon, pflogd controls logging and creates logs in a binary tcpdump format. Fortunately, tcpdump has options to turn the standard log format into an easily readable text format. The command "#tcpdump -netttvf logfile_name" will cause tcpdump to read the saved binary log and output a text format with most of the same information and in approximately the same order as the IP Filter logs. Any scripts that may have worked with the old logs will need some adjustment.

From the limited comments I've seen on misc@openbsd.org and comp.unix.bsd.openbsd.misc, some wanted to stay with IP Filter and some are concerned about the all new Packet Filter, even from a team with OpenBSD's reputation, but a majority seem to think Packet Filter, as it's released on the 3.0 disks is already better than IP Filter.

Changing from OpenBSD 2.9 to OpenBSD 3.0 has been much more difficult than any of the previous changes that I've experienced from 2.6 thorough 2.9. Some of this was to be expected with the change from IP Filter to Packet Filter. The change would have been significantly easier if I'd seen and tried one of the two examples provided in the pflogd man page or carefully read the entire revised FAQ page. I did spend several hours with the relevant man pages and reached an erroneous conclusion that the log format did not include whether the packet was passed or blocked, the rule that caused the action or the interface on which the action occured. Had I simply tried the pflogd example for using tcpdump with the "-e" option I would have clearly seen what I needed but I did not. Nothing in the tcpdump documentation clearly indicated what this did. What is a "link level header"? I'd actually highlighted the "-e" option on the tcpdump man page because it did not make sense to me. Because the format was described as a standard binary format, I assumed there was no room for this information, not realizing that the tcpdump format was extensible.

Finally I wrote OpenBSD team with a preliminary draft of this web page explaining the problems I saw with the new log format. They were very prompt and helpful in describing how to get the information I needed and offering to modify the format if there was still something not included that made sense to include. Following the feedback and subsequent experiments, this page was extensively revised. Still, given that I was already quite comfortable IP Filter, I didn't think that I should have to read nearly every line of the new documentation to get Packet Filter to do what I wanted. I ended up doing this and still missed the answer. Perhaps I was impatient or this was just bad luck.

Had the OpenBSD statements describing Packet Filter and it's relationship with the replaced IP Filter been accurate my job would have been much simpler. OpenBSD.org should have said something to the effect that Packet Filter is a firewall that shares conceptual similarities and much of the rule syntax with IP Filter. Otherwise it is an entirely new product and in some cases, the same rules may produce entirely different results. I repeatedly made mistakes because I believed I could use knowledge of IP Filter in using Packet Filter.

Because the large majority of OpenBSD users are familiar with IP Filter, the most useful thing the OpenBSD team could have done to help us would have been to provide a complete list of known, user visible differences, between IP Filter and Packet Filter. Only the Packet Filter Designers, who had studied IP Filter and know were it is similar and where it is different could possibly do this. All users like myself, will only encounter certain subsets of the differences because we don't use all product features in a wide variety of situations.

Even without time consuming misunderstandings, there are still a number of adjustments to be made in switching from IP Filter to Packet Filter. In the long run, the improved functionality may offset the work but there is still a down side to Packet Filter. At least currently Packet Filter is an OpenBSD only product while IP Filter is used in the rest of the BSD world as well as on Sun, HP, and SCO products. In a heterogeneous UNIX environment this could be a significant issue.

Then there are the Sendmail changes which were not trivial. These may have been driven by changes in Sendmail beyond OpenBSD's control. One thing slipped through and should have been fixed before the OpenBSD 3.0 installation process was released. This is the first OpenBSD install that I've seen where selecting a default install option can result in a system that boots with error messages in one of the key components. The sendmail errors that result when DNS fails on a bridged firewall or other system without access to proper DNS setups should have been caught.

To put this in perspective though, compared to what we are often faced with on installs of other operating systems, this is pretty minor. OpenBSD users expect pretty close to perfection from the OpenBSD team, so when there is a problem, even a modest one, it is quite visible and atypical. The OpenBSD team creates a consistently excellent product. They have technical goals (quality and security) and philosophical goals (free software that may be used by anyone for any purpose) that they pursue without compromise. If you find a bug or suggest an improvement that will clearly improve the software, the members of the OpenBSD team with whom I've had contact, have been some of the most helpful people I've worked with.

The OpenBSD goals do not include making it easy for less technical users to learn to use the system. They appear almost indifferent to a more widespread use of OpenBSD. Of course they would like to see it more widely used; it's just that doing the things that would assist non users to become users clearly has a much lower priority than the pursuit of the core goals. The best example I know of this is the FAQ page that includes Building your own kernel. The opening instructions on getting the kernel source from the distribution CDs have been wrong since 2.8 replaced 2.7 in Dec. 2000 and are still wrong several weeks into the released of 3.0. Knowledgeable UNIX users can read the old instructions, find the new location and new formats on the 2.8 - 3.0 CDs, and use an appropriately changed tar command to install them. Anyone who does not know tar well will be lost with the newer CDs. Even today, Dec. 19, 2001, there was a question on misc@openbsd.org asking where on the CDs the source was located and how to use it; some answers were simply wrong though a clear correct answer also appeared. There is the valid argument that with limited resources, the core goals could not be pursued if key players in the development process diverted resources to lesser goals. OpenBSD would not be what it is were this to happen.

David W. Krause has ported IP Filter back to OpenBSD 3.0 (http://www.davidkrause.com/openbsd/#ipfilter) but it sounds like not all issues have been worked out and that building an OpenBSD 3.0 system with IP Filter may not be a simple task. For those with IP Filter on other systems or who prefer it for their own reasons, this project should be worth following.

Top of Page - Site Map

Copyright © 2000 - 2014 by George Shaffer. This material may be distributed only subject to the terms and conditions set forth in http://GeodSoft.com/terms.htm (or http://GeodSoft.com/cgi-bin/terms.pl). These terms are subject to change. Distribution is subject to the current terms, or at the choice of the distributor, those in an earlier, digitally signed electronic copy of http://GeodSoft.com/terms.htm (or cgi-bin/terms.pl) from the time of the distribution. Distribution of substantively modified versions of GeodSoft content is prohibited without the explicit written permission of George Shaffer. Distribution of the work or derivatives of the work, in whole or in part, for commercial purposes is prohibited unless prior written permission is obtained from George Shaffer. Distribution in accordance with these terms, for unrestricted and uncompensated public access, non profit, or internal company use is allowed.