Osi Security

OSI Security

During the late 1970's, the International Organization for Standardization (ISO) began to formulate a theoretical model for networks of all kinds. Eventually they drafted what has become known as the Open Systems Interconnection (OSI) reference model which eventually became ISO Standard 7498. The foundation of the OSI reference model rests on the idea that networking can be broken into a series of related tasks, or layers, designed to conceptualize a single aspect of the communication process. This approach allows the complex world of network communications to be broken into a simplified series of interconnecting layers ranging from applications to hardware. A chain is only as strong as its weakest link and hackers will deploy whatever means necessary to determine the easiest way to compromise a network. Because of this divide-and-conquer approach, each layer must be able to send and receive data securely. This paper will describe what type of security is associated with each layer of the OSI model.

OSI Reference Model

Understanding the importance of security at each level of the OSI model requires understanding how the underlying model works. The OSI reference model breaks networked communications into seven layers (See Diagram 1). Each individual layer has its own set of well-defined functions, and the functions of each layer communicate and interact with the layers directly above and below it. The physical and data link layers, layers 1 and 2 respectively, define a network's physical media and the signaling characteristics necessary to send and receive data across the network medium. The network and transport layers, layers 3 and 4 respectively, move information from sender to receiver and handle the data to be sent or received. The session, presentation and application layers, layers 5, 6 and 7 respectively, manage communications across a network and deal with how data is to be interpreted and represented in applications or by clients across the network.

Every successful instance of communications on a network involves two machines and each utilizes the OSI model with a different layer sequence. One serves as the transmitting machine and the other the receiving machine. The transmitting machine is responsible for sending data and starts at the application layer (layer 7) and moves down the stack to the physical layer (layer 1). The receiving machine accepts the data from the transmitting machine and starts at the physical layer and moves up the stack to the application layer.

Diagram 1: OSI Reference Model

This graphic is taken from The Abdus Salam International Centre for Theoretical Physics.

Application Layer

The application layer provides a set of interfaces for applications to obtain access to networked services. This layer also handles general network access, the movement of data from sender to receiver, and error recovery for applications, where applicable. The layer would include all file-transfer protocols such as HTTP and FTP, user-oriented protocols such as DNS and WINS, messaging protocols such as SMTP, and access protocols such as Telnet and RDP (Reed, 2003).

The open-ended nature of the application layer makes it susceptible to poor design that can affect security. Examples include programs with well known back-doors that otherwise bypass secure controls and provide unauthorized access, programs that insecurely handle sensitive information by placing it in publicly accessible files, or programs that require weak or no authentication making them susceptible to unauthorized use (Reed, 2003).

By practicing strong design practices in the development of software many of the issues that affect poor design can be alleviated. Applications should be designed to utilize secure facilities available to them in the lower network layers, check incoming and outgoing data and require the use of strong authentication and encryption to validate and protect data as it travels across the network. Developers should assume that all data communications can and will be the subject of attack.

Other systems, such as firewalls and Intrusion Detection Systems (IDS), can prevent or monitor vulnerabilities. Firewalls can be used as a means of controlling access that applications have to the network and IDS systems can detect suspicious behavior that would indicate the presence of a Trojan horse or other virus attack.

Presentation Layer

The presentation layer handles data format information for networked communications. For outgoing messages, it converts data into a generic format that can survive the rigors of network transmission; for incoming messages, it converts data from its generic networked representation into a format that makes sense to the receiving application.

Security issues at this layer often stem from hackers taking advantage of buffered characters that feed unexpected or illegal-input into presentation layer facilities, producing undesired results. Examples include buffer overflows where program execution is redirected into unintended areas producing unexpected events. A specific example is the Unicode attack that takes advantage of a known issue with the Microsoft IIS that feeds the Unicode characters '%c0%af', that are translated to the plain text characters '../', into the URL that can potentially execute programs in a parent directory. By design, systems do not grant permission to parent directories, but by using Unicode the hackers bypass denial lists that only scan plain text (Surman, 2002).

Providing secure functionality at this level can be achieved by developing applications that carefully check input into applications, and by separating user input and program control functions. Applying patches supplied by vendors to known vulnerabilities will protect against attacks, such as the Unicode attack mentioned above.

Session Layer

The session layer permits two parties to hold ongoing communications, called sessions, across a network. This means applications on either end of the session can exchange data for as long as the session lasts.

Security concerns include weak or non-existent authentication mechanisms which session layer protocols often lack. Another authentication issue includes passing clear login credentials which are viewable to lower layers of the OSI model in an easily read format. A specific example is TCP Session hijacking which is when a hacker takes over a TCP session between two machines. This usually occurs after authentication so the destination will