The ISO OSI and TCP/IP network stacks are based on sound software engineering principles. Each stack is divided into independent layers based on functionality and requirements. Each protocol within each layer is interfaced through an abstract interface; protocols within a layer use the communication interface provided by the layer. Independent layers tradeoff security for flexibility, extendibility, and interoperability.
SOFTWARE METHODOLOGIES
There are three common methodologies for developing software projects: procedural, functional, and object-oriented. Each of these methodologies has its own strengths and limitations; no one method is better than another, although one may be better suited for a particular task. Each network stack, layer, and individual protocol implements one or more of these methodologies, and the selection is independent of other protocols. For example, a stack may be functional, a specific layer may be object-oriented, and a protocol within the layer may be procedural. The chosen methodology impacts the fundamental security aspects of the network stack.
Procedural Methodology
The procedural methodology divides functionality into sets of subroutines. Each subroutine may perform one or more complicated tasks and may modify global state elements. The inputs to a subroutine may come from passed parameters, global elements, or external systems. This makes parameter validation difficult. A subroutine may also return a value, set global elements, or activate external systems.
The inputs and outputs of a subroutine may be nondeterministic, which makes completeness (accounting for all situations) and testing difficult. Applications developed with a procedural methodology usually have many tightly integrated subroutines and can be rapidly developed. In addition, customized software interfaces and global elements can lead to faster performance.
Usually small programs are created using a procedural methodology. Large, procedural applications end up being monolithic, unstructured, and difficult to extend. Because network stacks are designed for flexibility, they are usually not procedural in nature. But when generic protocol interfaces are mostly unused, functionality can be simplified and combined into tight subroutines. Minimal network stack implementations, such as those found in some handheld devices, cell phones, and single-purpose networked devices (e.g., a networked power supply), may use procedural stacks to reduce overhead.
Functional Methodology
The functional methodology segments all actions into distinct and isolated units called functions. Each function performs a single task with little or no modifications to global elements. Functions are accessed through well-defined interfaces; one function can be replaced with an equivalent function as long as the interface definition does not change. The functional methodology also lends itself to recursion, where a function may call itself directly or indirectly without interfering with the
The functional methodology is very desirable for network stack development. Each layer is accessed through a well-defined interface and provides specific functionality. Both of these factors simplify parameter checking and software testing. Functional recursion is used when protocols are tunneled though other protocols, such as VPN technologies. Similarly, each protocol within a layer may be enabled as a function to allow protocols to be used interchangeably.
Object-Oriented Methodology
The third major methodology is object-oriented (OO). This methodology is a combination of functional and procedural. Each object defines a specific functionality and uses well-defined interfaces, similar to the functional methodology. But objects may perform multiple tasks—each object defines a set of related functions. In addition, each object may contain internal state variables that appear relatively global to the object, and functions within the object may be tightly integrated.
The result is a combination of functional and procedural methodologies: sets of objects use a functional methodology to communicate, but within an object, the elements may be procedural. Whereas network stacks are designed using a functional methodology, most individual protocols use an OO methodology. For example, TCP is a transport layer protocol. It uses a standard interface and can be readily replaced with UDP or SCTP. This gives TCP functional properties; however, TCP uses buffers for maintaining connections and reassembling data. This buffer space is shared among all applications that use TCP; if one application fails to manage its TCP connections properly, it can negatively impact all TCP connections, which is a procedural-based failure. Because TCP has both functional and procedural properties, it is an OO design.
If TCP were implemented using a functional methodology, then every TCP instance would have its own independent buffer space. The implementation of TCP is operating system specific. Whereas Linux and Windows use an OO methodology for the protocol, other systems may use functional or procedural implementations.
Converging Methodologies
Not all stacks are clearly part of one methodology. For example, the DOCSIS stack’s control messages are more procedural than functional, but they are not clearly one or the other.
Although general stack layers are functional, many protocols permit higher layer applications to directly influence internal protocol states. For example, the C functions ioctl() and fcntl() permit direct access to low-level drivers, including specific protocol state spaces. Although Linux and Windows provide IP and TCP as OO protocols, these implementations permit external manipulation of internal state elements. Similarly, the Unix command sysctl() (and the Windows Registry) can modify active and future connection states and protocol options.
Although the three methodologies define interactions, they are independent of the implementation method. For example, a stack may be developed using a functional methodology but implemented using a procedural programming language. An OO programming language, such as C++ or Java, supports many OO features but can still be used to develop procedural or functional applications.
The type of programming language is independent of the software methodology. From a network security viewpoint, knowing the type of methodology and programming language can assist in identifying potential attack vectors. For example, if a protocol is implemented using a procedural methodology, then it may not be easily tunneled or used as a VPN. Protocols that are strictly functional limit the scope of a security breach, but they may not support higher-level debugging, diagnostics, or real-time configuration changes.
SOFTWARE ENGINEERING PRINCIPLES
The overall OSI network stack is designed using sound software engineering principles. For example, there are well-defined APIs between layers that allow integration through standardization. Any system can use TCP through the sockets API—the interface does not need to vary between TCP implementations. The OSI stack’s architecture applies many design principles:
Modularity and Compartmentalization: Each layer and protocol performs one set of tasks. Network issues are isolated and distinct functions (e.g., session management versus network addressing) are kept independent.
Abstraction: The internals of each protocol and layer are hidden from higher and lower layers. A Web browser may connect to a server without ever knowing how the data is routed or even whether it is using IPv4 or IPv6.
Simplicity: Each protocol performs one set of functions. Different layers and different protocols perform distinct functions. As a result, new protocols can be used to replace other protocols, usually with no impact to other layers.
Flexibility and Extensibility: New protocols can be added to provide new functionality. Because each protocol performs a limited set of tasks, the type of data being received or transmitted is not a consideration to the protocol. For example, data passed to IP may be from a Web browser, SSL client, or VPN— IP treats all data the same way, independent of the type of data.
Reusability: Similar to flexibility, each protocol may be used and reused without interfering with existing connections.
Scalability: Each network layer and protocol is designed to be scalable. A single TCP implementation in a network stack may manage thousands of concurrent connections. The primary limitation for scalability comes from the hosting system’s resources. A slow CPU may become overwhelmed by many concurrent requests, but this limitation is independent of the protocol’s implementation.
Each of these software principles is independent of the methodologies. For example, a procedural method may use modular subroutines and support reusability. Conversely, a functional method may use very complex and dependent functions.
An ideal system will use a methodology that best fits the problem and apply the software engineering principles. By combining these principles, software can be quickly developed, maintained, debugged, or expanded with less effort than modifying a monolithic application. The OSI stack applies these principles. New protocols can be added to the OSI stack, existing protocols can be upgraded, and weak protocols can be enhanced.
IMPACT ON SECURITY
Each network protocol is designed using a standard interface and provides functionality independent of other protocols. For example, IEEE 802.3 routes data between systems on the same network but does not address intersystem routing. IP manages intersystem routing but not hostname resolution. The principles that lead to independent protocol development allow the network stack to be flexible, reusable, and extendible. However, this independence comes at a tradeoff: security at one layer is independent of security at other layers.
Transitive Authentication
Many protocols provide authentication, validation, privacy, and nonrepudiation (Table lohit1); however, each of these protocols only offers security for themselves. For example, a client that authenticates using WEP does not authenticate with the application. Each layer of authentication is independent of other layers. Passing the security precautions at any single layer does not imply security at any other—lower or higher—layer.
TABLE lohit1 Security-Oriented Protocols and Options
The result of protocol independence is a weakness for defense-in-depth. An attacker may need to overcome many security precautions, but each precaution is independent. For example, an attacker can crack a WEP password, impersonate a MAC and IP address, and hijack an established TCP connection to bypass security precautions in the first four OSI layers. Authentication, validation, and privacy at any one layer are not transitive—one secure layer does not imply security at other layers.
Defining Network Trust
Most network protocols only operate within one layer and are independent of other layers. As a result, the security found in one layer is not passed to other layers. Most network stacks assume that some layer will perform authentication, validation, and other security checks. This assumption establishes a false level of trust between protocols. Applications may assume that the information has not been intercepted or altered, but no steps are taken to validate this assumption. Most network connections operate with little or no explicit security precautions.
A few network protocols do explicitly enable security options and trust the results found in other layers. For example, HTTPS uses HTTP over SSL. The authenticated credentials from SSL can be passed to HTTP to create an authenticated
HTTP connection that spans two OSI layers. Similarly, most presentation and application protocols that use Kerberos can inherit the authenticated credentials and validated network information.
THE HACKER MENTALITY
Although the term “hacker” has undergone a variety of definitions over the past 30 years, the definition of the hacker mentality, or hacker factor, has remained consistent. The hacker mentality is the ability to see alternatives to existing situations. Sometimes called “thinking outside the box,” the hacker factor applies software, systems, and architectures to methods other than the original design. This important aspect of computer security identifies risks, limitations, and options beyond the original specifications.
One example of the hacker mentality is the use of NAT as a firewall. NAT was not designed for firewalls. It was designed as a solution to the IP address-assignment problem. NAT allows multiple hosts to share a common external address. The side effect—that inbound traffic cannot be routed when there is no outbound path—is as effective as an inbound filter on a firewall but is not a firewall by design. “NAT as a firewall” is a benevolent example of the hacker mentality.
As an example of deception, Sergey Lyubka (also known as “devnull”) developed a VPN solution that tunnels TCP over SMTP. The mproxy system (http://www.silversoft.net/projects.html) uses email to extend a private network. Although this VPN is slow, SMTP is ubiquitous, and this system can bypass most corporate firewalls. Attackers and administrators may create a backdoor path into a private network. An attacker would use it as an alternate (and likely undetectable) path into the network, and an administrator may use it as an obscure method around an otherwise-hardened perimeter.
Although some examples of the hacker mentality are benevolent, attackers can also use this approach to identify and attack network weaknesses. To identify potential attack vectors, detect exploits, and deter attacks, the security professional must understand that all network protocols can be used differently from the original design or be misused altogether.
Alternate Protocols Uses
Many protocols can be used outside of their defined specifications. This is usually done to bypass existing systems, such as firewalls, or to prevent detection by IDS and IPS applications.
Telnet Over ICMP: One of the early backdoor systems modified the telnet application to use ICMP rather than TCP. At the time, ICMP was usually permitted through firewalls, so this approach could bypass existing perimeter security precautions. Today, NAT and corporate proxies are widely used, so inbound ICMP cannot bypass most firewall configurations.
Knock-Knock Protocols: Port knocking uses connection attempts and packet options to signal clandestine services. Although a TCP SYN packet is usually used to initiate a connection, it can be used as a messaging system instead.
Covert Messages: Nearly all protocols offer some degree of flexibility in timing, data ordering, and information management (splitting, padding, unused bytes, etc.). These options and unused elements can be leveraged for passing alternative information. For example, DNS can be used as a generic distributed data system. DNS is not limited to storing hostname information.
Reconnaissance: Most protocols include system-specific information that can be used to profile hosts and identify possible attack vectors. SMTP leaks version and patch information, TCP SYN Scan and OS profiling can identify vulnerable services and systems, and HTTP timestamps can expose server location and nationality.
Alternate protocol uses are difficult to detect. A few high-end IDS and IPS applications learn expected network behavior. Unexpected network traffic is flagged as suspicious behavior, but poorly trained IDS/IPS applications generate many false reports. An attacker can generate suspicious activity as chaffing for an actual attack.
Misused Protocols
Some protocols can be misused to prevent communication rather than perform alternate communication functions. The DoS attack is a common example. Whether it is ICMP or SYN flooding, TCP disconnect attacks, or intentional SSH packet corruptions, a DoS disables communications. Nearly all protocols are vulnerable to DoS attacks either directly or through a dependency on lower layer protocols.
Not every protocol misuse is a form of DoS. Some manufacturers intentionally provide incompatible protocols to restrict usage. Most IM and VoIP implementations use nonstandard protocols to communicate. An IM client for Yahoo! Cannot access the Microsoft MSN system unless it supports two different IM protocols. Similarly, VoIP from Skype uses a different protocol than Vonage, even though they use the same audio communication standard. Adding to the complexity, some competing companies use the same server port number for different protocols or custom protocol extensions that conflict with competitors—both force incompatibilities.
NEW PROTOCOLS
New network protocols are constantly being added. The IANA provides official TCP and UDP assigned ports at http://www.iana.org/assignments/port-numbers. IANA also provides official lists for MAC address values, IP version numbers, and IP service types (all found at http://www.iana.org/numbers.html). The official protocol- value lists are only to resolve discrepancies between conflicting applications; technically, anyone can assign any protocol to any number. The only limitation is compatibility. For example, TCP is normally IP protocol type 6; however, TCP will work equally well as IP protocol 122. As long as all systems needing to use the same protocol use the same configurations, they will be compatible.
Flexibility Limitations
Although in theory, any protocol can use any configuration (and not just the assigned configurations), there are some limitations due to applications processing (Table lohit2). Using standards in nonstandard ways can prevent network routing or trigger alerts on IDS and IPS systems.
Fortunately, the impact from nonstandard usage is less severe for the upper layers (session, presentation, and application) because these protocols do not manage
TABLE lohit2 Lower Layer Impact from Nonstandard Configurations
routing across the network. Although a nonstandard DNS server may be incompatible with standard servers and trigger IDS alerts, the data will properly route across the Internet. In contrast, changes to lower-layer protocols may prevent packets from passing between networks. For example, many firewalls are configured to pass TCP or UDP transport packets but will drop all other packets. A custom protocol may fail to pass the firewall. If the custom protocol reuses the IP service type 17 (UDP), then it may be dropped as a corrupt UDP packet.
Protocol Conflicts
Reusing standard protocol assignments or using nonstandard configurations can lead to protocol conflicts. For example, if two different protocols both use IPv6 packet type 17 (standard UDP), then they can result in conflicts at the transport layer. The unknown protocol could be dropped as invalid, generate a resend request (to resolve corruption), or be misinterpreted by the recipient as a valid packet. Similarly, running a Web server on 22/tcp (the standard port for SSH) may result in many SSH connections to the Web server. The Web server must be able to handle invalid and malformed requests, and the SSH client must handle HTTP replies without crashing.
Multiple Applications: Many applications may use the same nonstandard port number. The most common examples come from computer viruses, worms, and trojans. The port 31337/tcp is used by many trojan systems, including BackOrifice, ADMworm, and LinuxRootkit IV. These applications are not compatible, and only one can use port 31337/tcp at the same time.
Server Discovery: A client must know how to contact a server. If the protocol operates on a nonstandard port, then general network clients may be unable to find the server. Requiring users to download custom applications for accessing nonstandard servers is usually undesirable; malware cannot be distinguished from desirable applications without some type of analysis.
Desirable Incompatibility
Sometimes protocol incompatibility is desirable. For example, if a private network reassigns IPv6 from Ethernet protocol 0x0800 to 0x0900, then most precompiled viruses will fail to spread across the network. Similarly, precompiled network scanners will fail to work if the OSI data link protocol reassigns the interface configuration for enabling promiscuous mode (or disables promiscuous mode altogether).
Although each of these options provide security-by-obscurity, they are likely to stop many common attacker vectors.
Many proprietary protocols use the same nonstandard configuration values as competing protocols. This allows a vendor to specify that not only will its software be used, but the competition’s software will not be used. Incompatible, proprietary protocols can lead to a business advantage in the marketplace.
Finally, protocol incompatibility can be desirable for bypassing existing restrictions. For example, if an ISP does not permit running a Web server on port 80/tcp, then the server can be moved to a different port number.