Computer Network Intrusion Detection Software Development

Shaiful Jahari Hashim, Kasmiran Jumari, Mahamod Ismail Jabatan Kejuruteraan Elektrik, Elektronik dan Sistem, Fakulti Kejuruteraan, Universiti Kebangsaan Malaysia
s.hashim@,usa.net, kbi@pkrisc.cc.ukm.my, mahamod@,eng.ukm.my

Abstract Recent proliferation of computer network usage for business and communication has witnessed an alarming rate of intrusion. The phenomenon is a grave concern in academic environment since universities seldom have tight security measures and the high bandwidth available make them interesting targets as attack relay hosts. The software development process adopted open source approaches where a simple open source implementation of network intrusion detection is improved to tailor the needs in Universiti Kebangsaan Malaysia campus. The working prototype is then tested on real life implementation in the university environment for detecting, logging and alerting computer network intrusion.

Keywords: Intrusion Detection, Network Monitoring, Network Security

1.0 Introduction
The original ARPANET, which then gave birth to the Internet, was developed during the cold war to ensure the availability of the critical US information infrastructure avoiding a central point of failure. Until now, the issue of availability is still a number one security issue plus the secondary issues of integrity and confidentiality. Since the advent of the Internet, there were measures taken to ensure the availability, integrity and confidentiality of the computer network clusters connected by the Internet. The most popular measure is firewall. Firewall is like a gate that restricts access to a private intranet computer network. Since gate can be broken and sometimes has backdoor, so does firewall to private network. Given sufficient resources and time, any computer attacker can penetrate into the most secured computer network in the world [l]. Network intrusion detection system takes the measuring steps further by admitting that accident does happen. The so called accident will let someone breaks into a private computer network and denying or exploiting its resources. Network intrusion detection system tries to detect, log and alert any intrusion from the very beginning, since the very first probe sent to scan the network. Conventional network intrusion detection system relies on passive monitoring of the network choke point by sniffer like tcpdump and performs most of the analysis manually [ ] This approach has an advantage that 2. allows people to scrutinise every piece of information that went on the wire based on the tcpdump filter. Unfortunately, the analysis only can be done offline. A large-scale solution requires enormous effort in inspecting the voluminous traffic. Scripts written in perf, for example, can help to reduce the effort but then the detection fell at the mercy of the script rules and the process is far from neat since we need every script for detecting every malicious activity known!

0-7803-6355-8/00/$10.0002000 IEEE

III- 1 17

2.0 A Simple Anatomy of Intrusion

Usually there are four primary steps in any computer network intrusion namely scanning, exploit, sniffing and island hopping [3].

2.1 Scanning
Scanning is done to gather information about a particular target prior to launching the attack. It also can be used to map the network of any particular site. They are two primary type of scanners, port and vulnerability scanner. Nmap is an excellent example of a widely used port scanner and Nessus is powerful holes/vulnerability scanner that can be integrated with nmap. 2.2 Exploit The most basic attack is to guess password of any user on the target system. The main objective, however, is to get root/administrator password in order to own the system. If the password can not be guessed, steal the password file (via cgi for example) and do dictionary/bruteforce comparison against the encrypted password file. The majority of attacks exploit any hole or vulnerability that exists on the target system. For example, buffer overflow will bypass authentication mechanism to gain the administratorhoot account. Once the root is compromised, rootkit is used to hide the intruders activities from the system owner and backdoors are often installed to ease their subsequent return to the system. Another example is by exploiting holes in program running with elevated root privileges such as sendmail. Advanced attacker will find unknown holes but

normal attacker will use publicised holes as appear at www.rootshell.com. Failure to update the operating system by closing the holes is the main reason of compromised. The misconfiguration of the web/ft/smtp server or Jirewall also serves as reason for compromised. A classic way to get into an up-to-date system without using password is to fool system user to install a trojan such as Back Orzjke using the so called social engineering technique. 2.3 Sniffing Apart from backdoor, a sniffer will also be installed once a host has been compromised. A sniffer is a useful utility to get as much as passwords that can be captured on the network in order to break into other systems. It can also be used to passively mapping communicating hosts through the monitored Ethernet and gaining substantial amount of information such as trust relationship between hosts. Since Ethernet is a broadcast medium it is possible to promiscuously monitor the Ethernet segment. It is advisable to use some form of encryption as opposed to clear text communication. Switched Ethernet is the current networking technology that limits the sniffing only to a single machine where it resides, preventing the sniffer from capturing other system passwords and passively mapping the network. 2.4 Island Hopping Island hopping is a phenomenon where an attacker jumps from one system to another before launching an attack. Internet attackers often mask their identity by launching attacks not directly from their own computer, but from an intermediary host that they previously
2

111-1 18

compromised, i.e., a stepping stone [ ] 4. In order to figure out who is the attacker, you would need to be able to follow the chain all the way back. Thus, using relay hosts (stepping stones) when launching attacks, a potential attacker stands a much better chance of not getting caught as multiple log files are required to track them. Poorly guarded systems make great relay hosts. They are two types of Island Hopping, regional or external hopping. Regional means in the same Ethernet segment while external means from/to outside the segment. Regional hopping is relatively easy to perform. Once a host has been compromised, the rest of the hosts in the same Ethernet segment are in danger. This is due to the sniffing activity and relatively easy to spoof as other hosts using ARP manipulation. Similar to sniffing, the effect of ARP spoofing can be reduced using switched Ethernet technology.

management of the distributed monitoring sensors across the campus.

3.2 Software Development Approach

Reinventing the wheel is not necessary. Thus, we would like to use existing free utilities available on the Internet and tailor them to our needs. One of the most useful security sites on the Internet is packetstorm.securifv.com and it has abundance of free security utilities. A good example of this approach is presented by the infamous paper 'The Cathedral and the Bazaar' [ 5 ] . The main program used as the backbone for the development is pakemon, a simple open source network intrusion detection system implementation. Other useful utilities are incorporated into our improved version to make the routine of intrusion monitoring much easier for the system administrator.

3.3 Software Design

The software implementation uses conventional modular design although a modular plug-in design is more superior. The apparent advantage of plug-in programming is more easily extensible since change in internal architecture of the program is not needed for any functional addendum. Any extra feature is added in the form of plug-in which is connected via a set'of API (Application Programming Interface) to the main program. The conventional modular design was chosen because it is easy to implement compare to plug-in modular design. Any significant change made upon the program will require a modification in its internal architecture. Since we have consider most of the detection issues, the current model of network intrusion detection system is

3.0 Software Development

3.1 Software Requirement

In the last section, various techniques for compromising other system hosts were presented to show its diversity. Since there is no silver bullet solution for security, we would like to develop our own intrusion detection system that caters our needs in detecting computer network intrusion in real time and afterthe-fact sign of intrusion such as the existence of a malicious sniffer. The system should not impose significant overhead to the campus network, scalable and can be installed without interfering with existing system. A central management console should be available for reporting and basic

111-119

Central Alerting

+ Local Log l
Sniffer Detector ARP Registry Scan Detector String Matching -

HTTP Normaliser

LIBNET

LIBPCAP

ETHERNET SEGMENT

Figure 1: The improved pakemon design

Figure 2: Architecture of network intrusion detection system implementation

111- 120

adequate for the purpose of detecting intrusion in our campus area. The detection system is distributed in nature since each sensor is installed for every Ethernet segment we wish to monitor. Each sensor entity design is an improved version of pakemon that is shown in figure 1. Every sensor entity reports to the Central Management Console (CMC) that shows connected sensors and the alert details.

3.4.2 Detection and Analysis Engine

The detection and analysis engine incorporate four modules namely sniffer detector, ARP registry, scan detector and string matching. 3.4.2.1 Sniffer detector As the name implies, its function is to detect malicious sniffer that has been installed on the same Ethernet segment with the sensor. Basically, it uses simple ARP test. This method involves sending out ARP requests to the segment with all valid information except a bogus destination hardware address. A machine that is not in promiscuous mode will never see the packets and will not reply since they were not destined to them. If a machine is in promiscuous mode the ARP requests will be seen and the kernel will process them and reply. By the replies, we know which machine is in the promiscuous mode. 3.4.2.2 ARP registry This module registers the ARP and IP addresses pairing and notify any change. Change of pair usually indicates spoofing of hosts but this is not necessarily true in DHCP environment where pairs are changing dynamically. Fortunately, UKM does not use DHCP protocol. 3.4.2.3 Scan detector Detecting scans can be difficult since successive scans can be interleaved for a long time. Scan detection can also lead to denial of service since we are recording every scan corresponding to the same attacker source address in the memory. Thus more casual scans can be

3.4 Software Modules

3.4.I Low-level network packet handling/normalising library and routine

Since different operating system handles network packet differently, network interface library is needed for the sake of easy development, porting and deployment. Libpcap and Libnet are two excellent network interface libraries for handling network packet. Sometimes low level network packet data are normalised for the application before it is possible to use them, making the low level data incomprehensible if we want to read them directly. Thus Libnids and HTTP normaliser are required for normalising network packet data to avoid evasion by the attacker [ 6 ] . The functions of the libraries and routine used are given below: a. Libpcap - network packet capture library b. Libnet - network packet shaping, injection and handling library c. Libnids - IP stack emulator for packet de-fragmentation and reassembly d. HTTP normaliser - a routine to normalise HTTP packet

111-12 1

detected depending on the fiequency of the scan, with better detection for the higher frequency of scan. Logging of attack originating from the same source address is terminated once a threshold has been reached in order to avoid denial of service against the system.
3.4.2.4 String matching

3.4 Real Life Implementation Issues

Amongst the main issues in implementing the software package and the proposed solution are: a. The sensor only recognise IP version 4 thus IP version 6 data is unknown. Solution: The compatibility issue with next generation IP can be done by minor change in the source code. b. Encrypted data cannot be scrutinised for suspicious content. Solution: Encryption only blind the string matching module since it depends on the data content but the other modules should work. The encrypted data is impossible to tackle unless we use a totally different analysis technique or shift the sensor to every host that we want to monitor [7]. C Package is only good in . broadcast Ethernet segments. Solution: Smart hub and switched Ethemet are not broadcast in nature thus the sensor territory will be limited to the particular host that is served by the switch. Most of UKM Ethemet segments are broadcast , therefore, the problem is minimal. Figure 2 gives an architectural overview of the system. d. Clear text communication between manager(CMC) and the sensors. Solution: The obvious solution is to encrypt the communication. The solution can be extended to include authentication as well.
_/

The detection sensors look for malicious traffic according to the signature database in real time. Apart from the signature matching there is another popular approach called anomaly detection that looks for strange traffic behaviour. The approach has the ability to detect unknown attack but has the reputation of high rate of false alarm. One more drawback of this approach is that it is hard to tell the exact reason behind any detection apart from a threshold has been reached. String matching capability is inherited from pakemon using strstro function from standard C library string.h. This function is simple yet powerful that can match any string including binary string. At the beginning of program execution, the signature file is loaded into the memory then the strstro function scrutinises network packet data looking for the specified strings in the signature file. For the time being only transport protocol UDP and TCP are recognised by pakemon.

3.4,3 Logging and Reporting

Once an intrusion is identified, the sensor log intruders entire corresponding session. Alert will be sent to the Central Management Console (CMC) containing the detection summary.

111-122

Feature Details Sniffer Detector Detect malicious sniffer I Scan Detector I Detect network scan Register valid IP-ARP pairs, notifies invalid pair ARP Registry Scrutinises network packet content looking for suspicious string String Matcher Central management console for managing distributed sensors CMC Data Normaliser a)IP re-assembly b)TCP de-fragmentation c)HTTP normaliser The use of packet handling libraries hides operating system details Portability

The more advanced solution is to maintain a separate network for the intrusion detection system and this has an advantage of immunisation from the denial of service attack to the CMC. Unfortunately, the addition of private network will impose significant maintenance overhead.

enable us with development choices that are cheap and easy to be customised for our needs.

6.0 Reference
[1]S. Garfinkel and G. Spafford, Practical Unix Security, Oreilly, 1991 [2]S. Northcutt, Network Intrusion Detection, New Riders, 1999 [3]S. McClure and J. Scambray, Hacking Exposed, McGrawHill, 1999 [4]V. Paxson and Z. Yang, Detecting Stepping Stones, 2000 [5]E. Raymond, Cathedral and The Bazaar, Oreilly, 1999 [6]T. Ptacek and T. Newsham, Insertion, Evasion and Denial of Service, Eluding Network Intrusion Detection, Secure Network Inc., 1998 [7]J. Allen and J. Pickel, State of the Practice of Intrusion Detection Technology, Technical Report, Networked Systems Survivability Program , CMU/SEI-99-TR-028,2000

4.0 Summary of features

Features of the prototype system are tailored specifically to the university needs. Please refer to table 1 for the summary of the primary features. 5.0 Conclusion Network Intrusion Detection System can be considered as a risk mitigation technique to complement the risk avoidance techniques such as firewall. The knowledge gathered from the intrusion detection can be used to improve overall campus security and eliminating any weak link that exists in the university computer network infrastructure since a computer network is as strong as its weakest link. Open source software development approaches

111- 123