Abstract Operating systems provide the fundamental mechanisms for securing computer processing.

Since the 1960s, operating systems designers have explored how to build "secure" operating systems operating systems whose mechanisms protect the system against a motivated adversary. Recently, the importance of ensuring such security has become a mainstream issue for all operating systems. For system designs that aimed to satisfy these requirements, the complexity of software systems often results in implementation challenges that are still exploring to this day. However, if a system design does not aim for achieving the secure operating system requirements, then its security features fail to protect the system in a myriad of ways. We also study systems that have been retrofit with secure operating system features after an initial deployment. In all cases, the conflict between function on one hand and security on the other leads to difficult choices and the potential for unwise compromises. From this book, we hope that systems designers and implementers will learn the requirements for operating systems that effectively enforce security and will better understand how to manage the balance between function and security. Operating System Security From the moment you boot your computer (after the BIOS loads), you are interacting with the operating system. This integral piece of software defines what you can do with the computer system and how you do it. Whether you're interacting with the file system or chatting with someone on an instant messenger program, the operating system is working behind the scenes to provide you with a (hopefully) flawless experience as it interprets your actions and converts them into something your computer can process. While operating systems vary on many levels, the most common operating systems provide much more than a simple interface between user and machine. Included are programs that provide the user with numerous extras, from simple screen savers to complex file-encryption schemes. However, it's important to understand that these programs are extras that are added to the OS and are not necessary for the computer to operate. Many users become intimately familiar with the operating system's accessories (such as Solitaire), but forget about the security features that are included to help the user maintain a safe and reliable operating environment. As a result, many information systems exist in an insecure state that leaves the system at risk to a virus infection or a complete compromise by an attacker. Operating System Overview

Before delving into the security side of an operating system (OS), it's important that you know where the OS begins and ends. This brief overview describes the functionality and purpose of the operating system and how it's used to create the computing experience. In Understanding the Linux Kernel (O'Reilly, 2002), Daniel P. Bovet and Marco Cesati provide the following definition: "Any computer system includes a basic set of programs called the operating system. The most important program in the set is called the kernel. It is loaded into [memory] when the system boots and contains many critical procedures that are needed for the system to operate. The other programs are less crucial utilities; they can provide a wide variety of interactive experiences for the useras well as doing all the jobs the user bought the computer forbut the essential shape and capabilities of the system are determined by the kernel. OS Functions In short, the OS must provide two main functions:

It must manage the resources available to the computer system. It must provide a reliable, stable, secure, and consistent interface for applications to access the computer's resources.

The first function is critical to the OS because it defines how applications access the system's resources. By controlling the various aspects of how hardware and software are used, the OS ensures that every application gets a chance to use the processor. The second, related function defines the methods by which an application can access these resources. Because the OS often acts as a buffer between an executing program and the hardware, it needs to provide some means of allowing applications to access resources without needing to know the details of each and every unique computer system. OS Types There are four main types of operating systems, classified according to the types of programs they support and way these programs interact with users:

Real-time operating system. This OS is most often found in robotic machinery and scientific devices. It doesn't provide much room for user operation, with the exception of some configuration changes. Typically, this OS contains highly polished timing mechanisms due to the impact even the slightest error could have in automated production or measurements.

Single-user, single task system. This type of OS is used by devices such as a PDA or other miniature computers. It basically allows one user to operate one program at a time. If another program is needed, the user must close the currently executing application. Single-user, multitasking system. This type of OS is most familiar because it includes most Microsoft Windows systems. In this OS, a user can open multiple programs and jump back and forth between applications as required. In fact, there is much debate that although Windows Server OSes appear to be multiuser systems, they're actually single-user, multitasking Oses (with the exception of Terminal Services). Multiuser system. A true multiuser operating system allows many users to access the computer's resources simultaneously. A common example of this type of OS is Linux. In this type of system, the OS manages requests from numerous users, and maintains rigorous control over the resources to ensure that one user doesn't affect any other user.

OS Tasks The OS is responsible for various tasks within the computing environment. These tasks are often what make one OS more reliable or easier to use than another, and also determine the power of the OS:

Processor management. The OS needs to ensure that each application gets a share of the processor's time, and that the processor is used efficiently to accomplish real work.

Memory management. This defines the methods by which the OS allocates memory to applications and OS functions. Device management. Because a computer system is composed of various hardware components (hard drive, monitor, mouse, keyboard, and so on), the OS must be able to manage how these components interact with each other. Storage management. The OS not only controls active resources, but defines how files and data are stored in a reliable fashion.

Application interface. An OS is really a bridge between applications and the computer's resources, which means that it must provide application programming interfaces (APIs) for applications to connect. User interface. Whether this is a command line or a graphical user interface (GUI), the OS is responsible for interacting with the end user.

This is a very brief summary of the major tasks that an OS should handle. The following sections describe security-related issues that the OS must also deal with to maintain confidentiality, integrity, and availability of system resources.
OS Security Weaknesses

Now that you have had a brief overview of what an operating system should provide for a user with regard to functionality, let's take a look at the security aspects of some favorite operating systems. In this section, we discuss the two most common operating system families and the security features they include. We also examine methods by which these security features can be attacked and/or bypassed, and how to protect against these types of attacks.

Windows Weaknesses
Microsoft Windows has long maintained a reputation for having inadequate security, but many security experts believe that Windows is not inherently weak. Instead, they place the blame squarely on the shoulders of the administrators who are responsible for the system. In other words, with proper maintenance and configuration, a Windows OS can be made relatively secure. There are several areas in which Windows is known to be vulnerable:

Uneducated users. Windows is an operating system for the masses. Many users don't understand or care about the security risks associated with improperly configuring the system. In addition to this rather extensive group, many businesses employ part-time administrators, elected from existing employees based on the fact that they know the most about computers. Unfortunately, this strategy often results in disaster the first time a hacker probes the gates looking for an easy target.

Commercial system. Windows has always been about providing the user with a simple and easy operating system. The early versions didn't do much in the way of security. Like other software companies, Microsoft is always looking to add features to their product to encourage existing customers to upgrade. In addition, their commercial nature requires backward compatibility with older, less secure versions. And with each new feature and service, a whole new set of security issues arises.

Poor auditing. When people think of Windows server-logging capabilities, the first thought is usually the Event Viewer. While this integral part of Windows does provide some useful information, the Event Viewer has long been considered a less-than-adequate logging tool with cryptic messages and missing information.

Size/complexity. Microsoft has its foot in every software door (and even some hardware doors). This is nice for integration; however, it makes it very difficult for the average administrator to keep up with the software in terms of understanding how to properly use and configure it. Insecure installation. One of the most common reasons that Windows servers fall prey to attackers is because they're installed and forgotten. Unfortunately, Windows is infamous for having little to no default security. This includes hidden shares, blank passwords, and no protection from known vulnerabilities.

Linux Weaknesses Linux is considered by many to be an operating system for the computer geek. While this was true at one time, for all practical purposes Linux OSes have evolved to the point where they're starting to attract the average user. From the Wal-Mart Windows machines to the implementation of Red Hat File Server, Linux is making some major headway into the mainstream market. Unfortunately, this means that the number of inexperienced Linux users is also growing. One of the most common blanket statements made with regard to Linux is that it's more secure than Windows. Unfortunately, this isn't exactly correct, and has misled more than one person into believing they're safe from hackers if they only use Linux. While it may be true that Linux can be made more secure than other OSes, Linux users face many of the same problems as owners of other OSes. These are the major security issues for Linux:

Complexity. By far, the most threatening issue Linux users face is a complex set of commands, concepts, and programs that need to be understood to properly implement security. In fact, this can easily be seen when a new user installs Linux for the first time. While some Linux distributions (versions) have started making decisions for the user, many Linux OSes require the installer to choose between cryptically named programsor just to install the entire OS. Unfortunately, the list of several hundred programs is often overwhelming. As a result, the user installs the entire OS, including the HTTP daemon, FTP daemon, mail daemons, and morenone of which are secure by default.

Networking OS. As Maximum Linux Security states, "Although Linux is well suited to personal use (even in non-networked environments), it is still inherently a network operating system. Default Linux installations run many Internet services, and unless you take proper precautions, attackers can target these services remotely throughout the duration of your online session."

Root account. If there's one rule in Linux that many people ignore, it's "Don't use the root account unless you absolutely need it." The reason for this rule is found in the power that root access offers to the user. Like the Administrator account in Windows NT, root is the highest-level interactive login account that exists in Linux. The danger lies in the fact that most exploited programs run with the permissions of the user who activated that program. In other words, if a user is surfing the Internet as root, he or she may inadvertently execute a script on a web page. This script would then have root access, and could conceivably access any file or even delete the entire file system. However, the root account is often used as the primary account in Linux. In fact, some distributions (such as Lindows) require the use of the root account during installation and operation. Open source updates. Much of the software created for Linux is written by students, research groups, or software companies who are trying to find a way to make Linux software profitable. Combine this with the fact that Linux is open source, which means that all the software is open to examination by the world, and you have a potential security nightmare. The problem is not that open source software is any more insecure than proprietary software. In fact, Linux vendors are known for having updates or patches within hours of a reported security vulnerability. Instead, the problem is that administrators never receive word about these updates. For example, Red Hat releases as many as five security bulletins a day that need to be reviewed by an administrator to see if they're applicable. While many of these alerts may be irrelevant, it only takes one missed warning to leave a system open to attack.

The paragraphs should have enlightened you about some of the security issues surrounding Windows and Linux operating systems. While these are not the only issues, most security experts would agree that these represent the more serious concerns. Now that you have a general idea of where to look for security risks, let's take a closer look at each of these OSes and examine some specific security threats. OS Security Pointers This section is dedicated to providing an overview of how to secure a Windows or Linux OS. This will only be a rough sketch of what procedures and steps you should perform. For more indepth and comprehensive information, consult the articles and books listed in the reference section. When a user installs or sets up an OS, several security issues must be acknowledged:

Properly setting up secure user accounts Ensuring the use of strong passwords Inventorying the services and programs running on the server Ensuring that the server is properly updated with the most current patches

And all this before the server is ever connected to a network! While many users are aware of these issues, there are still several OS-specific security issues that need to be dealt with before a server is released into the wild. The following sections look at both Windows and Linux servers and briefly outline the steps you should take to ensure that your server is set up securely. Windows While Windows has a reputation for being insecure, as mentioned earlier, it can be configured to be a secure OS. This takes an in-depth understanding of several key components of the OS, and some general guidelines that should be followed during installation. Selecting the File System The first issue any user will have to deal with when installing a current Windows OS is what files system to use. The options are FAT32 or NTFS, both of which have advantages and disadvantages. As Sean Deu by states in his article "Windows 2000 Server: Building, Maintaining, and Tuning the Box": The FAT file system is best for drives or partitions under approximately 200MB and is faster for sequential I/O. FAT32 can address much larger disks, but it still has none of the security features of NTFS. NTFS works best for file systems above 400MB. NTFS has its own log file for internal consistency; FAT has none. Additionally, an NTFS Version 5 volume is required for the Windows 2000 SYSVOL. Finally, NTFS has detailed security built into it. Since our primary concern is security, NTFS is the best and only choice. It includes the following security features:

File security. Files and directories can be assigned access rights that are used to control a user's access to a system's data.

File encryption. NTFS v5 supports encrypting and decrypting data as it's written to and read from the hard drive (not enabled by default).

Multiple file streams. Though not a security feature as such, NTFS supports the ability to write multiple data streams to one file. This feature can be used (or abused) to hide data behind another file.

Operating System Comparison An operating system or OS is a fundamental component of a computer system that manages activities and resources on the machine. As a host application, it handles the operations of hardware devices which make it easy for the associated programs to function. Nearly every computer including desktops, laptops, supercomputers, hand-held and even video game consoles use some type of operating system. While there are many different options, there are essentially four operating systems commonly used today: DOS, Linux, Windows and Mac OS X. DOS DOS (Disk Operating System) gained a reputation by supporting operating systems developed by IBM and Windows. DOS was prominent in the industry up until support was terminated for Windows ME. Though it's not as sophisticated as modern systems, DOS offers many advantages to the user. This system is very stable and rarely endures crashes. It can display content on an arcade monitor without the aid of special hardware and the use of batch files offer a high degree of customization. While the system is still used, DOS has become more difficult to manage as support is limited. Linux The Linux operating system is developed on a kernel based on UNIX. It is known as one of the most secure platforms, yet Linux is often viewed as more complex. Linux is the result of an open-source project, allowing users and developers to access the source code for free. Because it requires less hardware, Linux is capable of providing exceptional performance even on a smaller hard disk. Large communities of users exist for this system, constantly contributing to the code and making improvements. The server-based concept of Linux has resulted in fewer home users and a lack of multimedia support for the system. Mac OS X Mac OS X is Apple's trademark operating system software for their line of Macintosh computers. The system was based in part on UNIX and mimics its format with the administrative controls. Mac OS X software requires a low level of maintenance with fewer occurrences of computers worms, viruses and spyware. The Apple operating system does have some disadvantages, primarily in regard to software and hardware compatibility.

Windows The Microsoft Windows operating system is the most popular choice and currently has a stronghold over the market. This platform has made significant advancements from version 1.0 all the way to the new Vista system. The Windows system is highly compatible, feature-rich and has a much larger selection of software applications. Unlike the Linux kernel, Windows is proprietary software and tends to be more expensive than others. Despite widespread usage, Windows has been heavily associated with the term "insecure" as a number of security vulnerabilities have made it the most targeted system. Frequently exploited by hackers and malicious code writers, it is recommended that any Windows operating system with internet access be protected by some form of security software. Although the XP version is still popular among users, support for this system will conclude in 2009 as more emphasis will be placed on developing the Vista series. Conclusions The security engineer of the twenty-first century will be responsible for systems that Evolve constantly and face a changing spectrum of threats. She will have a large and constantly growing toolbox. A significant part of her job will be keeping up to date technically: understanding the latest attacks, learning how to use new tools, and keeping up on the legal and policy fronts. Like any engineer, shell need a solid intellectual foundation; she will have to understand the core disciplines such as cryptology, access control, information flow, networking and signal detection. Shell also need to understand the basics of management: how accounts work, the principles of finance and the business processes of her client. But most important of all will be the ability to manage technology and play an effective part in the process of evolving a system to meet changing business needs. The ability to communicate with business people, rather than just with other engineers, will be vital; and experience will matter huge.