Parallel Password Cracker: A Feasibility Study of Using Linux

Clustering Technique in Computer Forensics

Johnny Bengtsson
Swedish National Laboratory of Forensic Science SKL
johnny.bengtsson[at]skl.police.se
Abstract
Today, nearly all computer forensic analyses are done on stand-alone computers. The
fundamental reason is that analysis software are not developed for parallel processing.
This paper describes a feasibility study in building a Linux based high-performance
computing cluster (HPC-cluster) cluster and the development of a parallelised password
cracker application using the Message Passing Interface (MPI) in order to gain
competence in designing parallel programs, especially in or near the field of computer
forensics.

1.

Background

As far as is known, computer forensic analysis tools only run on single computer systems
or at least not particularly designed to benefit from a clustered environment. If so, the
time to complete an analysis could in many cases be reduced.
Informationsteknikgruppen at the Swedish National Laboratory of Forensic Science
(Statens kriminaltekniska laboratorium SKL) has finished an initial feasibility clustering
project aimed to achieve competence in building a low-cost cluster and gain experience in
developing a parallel computing application in or close to computer forensics. The goal of
this project was to investigate if parallelism can be useful within this field.

2.

Related work

Cracking user passwords have always been fascinating people, as told by the numerous
password cracking software. However, only a few of these are specifically designed to take
advantage of parallel processing in a clustered environment. One example is MPICracker
by [16], a brute force cracker for Linux passwords. In some of the cases, the source code
has been modified or re-written to run in clustered or distributed environment. For example,
work has been done by [10] and [14] to parallelise the open sourced John the Ripper (JtR)
by using the Message Passing Interface (MPI).
Parallel password cracking can also be undertaken by other techniques, e.g. using load
balanced clusters or distributed systems. [07] has combined the load balancing openMosix
together with both John the Ripper-MPI (JtR-MPI) and the password cracker Cisilia. [11]
did minor modification to JtR for parallel processing by using parallel threads and
openMosix. It is also worth mentioning that successful attempts have been made to
distribute work to other nodes of the Forensic Tookit (FTK), described by [18]. Distributed
systems in a loosely coupled network are not directly comparable to this study, since the
cluster nodes connect to the same switch.
Techniques using pre-calculated dictionary based lookup hash tables is not realistic for the
MD5-based shadow files. The variable salt range up to O(248) would generate far too much
lookup data. Teracrack [13] uses this technique, but for the weaker DES-based passwd file
which only requires 4 096 (=212) hashes for a single password. It is worth mentioning one

of the original ideas on pre-calculation of lookup tables made by [09].

3.

Overview

3.1.

HPC-clusters and the MPI

A high-performance computing cluster (HPC-cluster) is a cluster model to perform

computations by sharing the workload between its nodes. There is also a HPC-cluster
concept called Beowulf (or Beowulf Project), where the basic idea was to get a descent
Linux hosted cluster for a low cost, often built by commercial-off-the-shelf or commodityoff-the-shelf (COTS) components.
To get the most out of a HPC-cluster the applications should use some kind of message
passing (MP) techniques to let cluster nodes communicate between each other. A
commonly used MP is the Message Passing Interface (MPI), a standardised library which
has several different implementations like the MPICH.
3.2.

Password complexity

The longer a password is, the more difficult it is to crack. That is said to be the general rule
but why? A correct four digit PIN code, where the allowed digits are [0-9], will exist
within the interval {0000, 0001, ..., 9999} as follows:
allowed_digits[0-9]^PIN-code[****] <=> 104 = 10 000 combinations
The same procedure is used to calculate all possible password combinations of a certain
length, along with a determined set of characters. An exponentiation operator describes the
password's exponential complexity: The base corresponds to set of characters used, the
exponent to the password length.
base^exponent <=> [character set]^password_length
Assume that we want to determine all possible password combinations with a length of six,
seven and eight characters within the interval [a-z].
[a-z]^password_length (6) <=> 266 = 308 915 776
[a-z]^password_length (7) <=> 267 = 8 031 810 176
[a-z]^password_length (8) <=> 268 = 208 827 064 576
If the character interval extends from [a-z] to [a-z, A-Z, 0-9], it would generate larger
numbers:
[a-z, A-Z, 0-9]^password_length (6) <=> 626 = 56 800 235 584
[a-z, A-Z, 0-9]^password_length (7) <=> 627 = 3 521 614 606 208
[a-z, A-Z, 0-9]^password_length (8) <=> 628 = 218 340 105 584 896
Obviously, from these calculated examples we can see that the most crucial variable is the
set of characters, since password length seldom are larger than eight to twelve characters.
3.3.

Crash course in password cracking methods

There are two major ways to crack passwords: dictionary based attack and brute force
attack.
The principle for the dictionary based attack can be described as a sort of lookup-table, a
suitable method for cracking unsophisticated passwords. uses one or more predefined
dictionaries containing common passwords of which are easy to remember, such as names
(nick names, relatives, pets, places, brands), numbers (phone numbers, social security

numbers, dates of birth) or different keyboard combinations (qwerty, 123456, 1qaz2wsx,

abc123).
Mandatory for today's password policies are a combination of majuscules (capital letters),
minuscules (lower case letters) and numbers, as well as the password interval, meaning
password alternations within a certain time.
A number of improvements to the dictionary based attack are made in order to comply with
some of these criteria: Letter alternations to match all possible combinations (mypass,
Mypass, mYpass, myPass, ..., MYPASS), reversing passwords (ssapym) or letter
replacement in accordance to leet speak style [20] (us3r, k!NGP!n, Se7eN, n0rM4L), also
known as leet, 1337 or hakspeak.
Rainbow tables are much alike dictionary lists, where passwords are replaced with precalculated hash tables.
The hybrid attack combines the dictionary based attack and the ideas of the brute force
attack. It adds a prefix or a postfix to a word found in the dictionary. The fictive password
mypass would, for instance, be altered as mypass1, mypass2, mypass3 and so on.
The brute force attack is an exhaustive method that tests all possible combinations within
certain delimitation boundaries, such as chosen character set and password length. The
method will eventually find the correct password, but it might take years or even longer.
3.4.

Passwords on Linux

Shadow passwords are normally used when handling user accounts on newer Linux
systems. These passwords are successor to the older passwd, a heritage from Unix. Both
password functions utilises the crypt()-algorithm, which from the beginning only supported
the Data Encryption System (DES) [06]. When shadow passwords was introduced, crypt()
was rewritten to support he MD5 Message-Digest Algorithm (MD5), a one-way hash
algorithm by [17].
The salt is blended with the password before hashing and contains characters within [a-z,
A-Z, 0-9 and ./], by default random and embedded with the hashed password. The salt size
was previously fixed to 12 bits or 4096 different hash values (212), but has been extended to
allow a variable Base64 encoded salt between 12 to 48 bits when MD5 was introduced.
Same password but different salts will therefore generate different hash values which
makes it more difficult to use dictionary based attacks and pre-calculated lookup hash
tables.

4.

This feasibility study

The initial project concentrates on specifying, buying and building a low cost high
performance computing cluster based on commercial off-the-shelf (COTS), installing an
appropriate Linux distribution and develop a demo application that benefits from cluster
computation, preferably directed to computer forensics. This, because of the opportunity for
further development within the same or similar fields.
4.1.

The cluster

When specifying the cluster, at 2004-2005, a trade-off between new technology (like true
64-bit processors), performance and cost was one of the issues. The final configuration
turned out to a be 42U 19-inch rack mounted Beowulf HPC-cluster. The cluster contains
six Iwill DK8ES motherboards with dual AMD Opteron 244 CPU:s, where each CPU was
dedicated 2 GB PC3200 REG ECC DIMM:s. Dual Maxtor MaXLine III SATA-150 disks
were striped in a RAID 0 array in order to cut down access time. The network latency is
usually considered a bottleneck. Due to the limited budget, 170 000 SEK (18 000 EUR,
May 2007), no high-speed interconnect such as InfiniBand or SCI could be afforded.

Instead, the motherboards were geared up with 3Com's 3C996-SX 1000BASE-SX card,
and connected to a cheap fiber gigabit switch GSM712F from Netgear.
The Slackware Linux distribution was installed and configured together with the MPI-1.2
implementation MPICH 1.2.6. The latter was, however, not considered better than other
implementations, like LAM/MPI or the Open MPI. The remote shell (rsh) service was used
for remote execution.
4.2.

The application a general description

The demo application (named brutest) utilises the MPI library for the parallelisation and
was developed in ANSI-C together with the MPICH package included compiler mpicc. A
few bash scripts were also written for testing and administration.
Brutest undertakes brute force attacks, as well as dictionary based attacks of passwd or
shadow files. Either of the attacking methods are selected by command-line options. If the
dictionary based attack is selected, a plain MS-DOS or Unix EOL coded ASCII-file is
required, while if the brute force method is selected, the user can chose between predefined
character sets or create a new. The chosen character set is then distributed to each slave
node.
The assignment vector is common for both methods. This is created by the root node and is
distributed one by one as work load to the slave nodes. An assignment vector contains three
components: a string base, UID and the passwd. The UID is the current username and the
passwd contains the salt and hashed password. The information is grabbed from the first
and the second field from the current row of the shadow or passwd file. The string base for
the dictionary based attack method contains the next word or phrase in the dictionary file
opened by the root node, while it for the brute force attack consists of one or more starting
counting up letter selected from the character set.
When a slave node has received an assignment vector, it disassembles its information and
then start the work. Here, the well-known crypt(password, salt) function (<crypt.h>) is
used for the matching purpose. The dictionary based attack function simply hashes the
string base information and compares it with the hashed password. The brute force attack
function locally concatenates three more characters (trigram) from the given character set to
the extracted string base, where after it hashes the resulting string and does the comparison.
Here, it means that the starting length is four characters, one from the string base and three
more attached by the slave node. The reason for that is to keep down the traffic between the
root node and the slave nodes which in the long run increases the total performance. No
combination collision will take place, since the root controls the content of the string base.
The root node is constantly listening to returning packages from any slave node, here called
node_result. When a slave node has completed an assignment, it assembles and sends the
node_result, a vector that keeps information about the current user, the actual password and
the outcome of the hash comparison result (true/false). If there is a negative match the slave
node will ask for another assignment. A positive match will stop the actual node and the
root node broadcasts a global stop.

5.

Performance and results

The BIOS circuit was unfortunately crippled on one of the motherboards, so only ten of the
twelve nodes were used during the tests. A shadow file with a root account was used along
with the brute force attack method. The length of the correct password was set to be greater
than seven characters to avoid any stops.
set of characters = [a-z], 26 characters
password length
4
5
6
7

MPI wall clock time (s)

24
951
24 755
648 618

estimated hashes per second

18 781
12 499
12 478
12 383

set of characters = [a-z, 0-9], 36 characters

password length
4
5
6

MPI wall clock time (s)

97
4 803
172 506

estimated hashes per second

17 302
12 587
12 618

As a result of the performed tests, an estimated number was approximated to 12 500 hashes
per second.
Now, assume that a local password policy requires an eight characters long password,
where the allowed characters are [a-z, A-Z, 0-9]. Is it worth using the parallel demo
password cracking application Brutest and the built computer cluster?
Recall the earlier mentioned formula:
[character set]^password_length = number of possible combinations
This gives us: [a-z, A-Z, 0-9]^8 = (26+26+10)8 = 628
628 [hashes] / 12 500 [hashes per second] ~ 17.5e9 s. ~ 202 000 days
The worst case scenario would be the password 99999999, and take over 500 years to
complete. That is why the dictionary based attack in some cases would be more
appropriate.

6.

Discussion

What to consider when building a new cluster is its actual purpose. If it is meant to be used
as a resource for hardcore computation, the HPC model would probably be appropriate, as
for this case. A Beowulf COTS-cluster can be relatively affordable and using a Linux
environment is mostly for free.
This project resulted in a Slackware Linux operated Beowulf HPC-cluster, all based on
commercial off-the-shelf (COTS). With a budget at 170 000 SEK (around 18 000 EUR,
May 2007) a 12 node 19 rack mounted high-performance cluster (HPC cluster) were built
from scratch during 2004 and 2005. At the time being, the socket 940 AMD Opteron was
one of the few realistic alternatives that offered a true 64-bit multi-processor architecture,
i.e. several processors on the same motherboard. The selected motherboard was the Iwill
DK8ES, built upon the nForce Professional 2200 MCP chipset. This motherboard supports
dual Opteron and, at that time, the new PCI-Express buses and SATA. A GPU, the ATI
Radeon XL (8MB), is also integrated which saves some money. (Remember that this
cluster was not intended to play the latest versions of World of Warcraft.) Components
were carefully chosen in order to reduce the bottlenecks. However, buying components and
new technologies might cause some problems, such as ending up in problems with the
supply chain, lack of compatible hardware drivers, risk of breaking parts when assembling
the computer nodes, etc.
Speed and performance was not a priority during the development of the demo application,
therefore no real scalability measurements were done (e.g. 2, 4, 6, 8 and 10 nodes
performance). Making linearity tests of these number of nodes would probably not give
very useful information. Optimisation for performance can most certainly be done: the
source code, kernel and Linux specific configuration files and drivers. Commonly known
scalability problem such as higher network load, increasing latency, caching problems,
memory leakage, etc., often appears at larger number of nodes.
A drawback with the one-way hash MD5 is the risk of a hash collision. Here, it means that
more than one hashed password can result in the same hash value. If a false password with
a hash value equals to the correct password, the latter would not be found, since the
password attack function stops after the first hash value match. Generally, this is not a big
problem for most applications, such as password protected files, if the correct password
string needs to be found.
The relevance of cracking shadow passwords when speaking of computer forensics can be
questioned if the computer is in physical control. The suspected disk is easily mounted to
another operating system, but that is not the point. Other algorithms can be used to crack
encrypted files, which is perhaps considered more relevant. The major purpose of this study
was to learn how parallelising code can be done in a clustered environment.
During the development process, the demo application was taking action in a sharp case. A
Linux based packet sniffer operated on a 8 GB Compact Flash disk was used to steal
transaction data from a grocery store. The sniffed data was bundled to packages,
presumably encrypted, and then broadcasted wirelessly using a WLAN card. For
investigating purpose, an unsuccessful attempt to crack the root password was done. A
revealed password might be used elsewhere.

7.

Conclusions

This feasibility study is not intended to compare different distribution or parallel techniques
nor hardware configurations, but to show that building a cheap cluster and taking advantage
of parallel computing can be a competitive alternative to traditional thinking. Certain parts
of a computer forensic analysis can obviously be speeded up, where problems can be
divided into independent sub-problems. Password cracking is such of an example, e.g. if
encrypted files or containers wants to be opened.
An even more important conclusion drawn from this study is that laboratories, institutes or
companies with small funds and a few personnel also are capable of setting up a high
performance computing resource and start developing applications that take advantage of
parallel processing using the Message Passing Interface.

8.

Future work

Computer forensics can and already does utilise parallelism. No specific proposals or
details are suggested, but interesting parallel applications might be developed within
password cracking of encrypted files or containers, data mining, database queries, statistics,
pre-calculation of lookup tables, parallel rendering, parallel signal processing...

9.

References

[01]

AccessData Corp., Forensic Toolkit(FTK), http://www.accessdata.com

[02]

Bar, M., openMosix (2002-), http://openmosix.sourceforge.net

[03]

Beowulf Project (1994-), http://www.beowulf.org

[04]

Benitez, C. E. et al., Proyecto CISILIA (2002-),

http://www.citefa.gov.ar/SitioSI6_EN/si6.htm

[05]

D4 B0rg (2001), Distributed, Keyboard, Brute-Force (dkbf),

http://dkbf.sourceforge.net/

[06]

Feistel, H. et al., Federal Information Processing Standards (FIPS),

FIPS PUB 46/FIPS-46-1/FIPS-46-2/(FIPS-46-3, withdrawn 2005),
Data Encryption Standard (DES) (1976-2005),
http://www.itl.nist.gov/fipspubs/fip46-2.htm

[07]

Frichot, C., An Analysis and Comparison of Clustered Password

Crackers (2004), http://scissec.scis.ecu.edu.au/publications/

[08]

Gopp W. et al., MPICH (1996-), http://www-unix.mcs.anl.gov/mpi/mpich1/

[09]

Hellman, M.E., A Cryptanalytic Time - Memory Trade-Off (1980),

IEEE Transactions on Information Theory, vol. IT-26, No. 4, July 1980,
http://www-ee.stanford.edu/~hellman/publications/36.pdf

[10]

Lim, R., Parallelization of John the Ripper (JtR) using MPI (2004),
http://cse.unl.edu/~rlim/jtr-mpi/report.pdf

[11]

Lunglhofer, J., Building a Parallel Password Cracking Environment,

IAnewsletter, volume 6 number 2 (2003), http://iac.dtic.mil/iatac

[12]

Message Passing Interface Forum (1994-), http://www.mpi-forum.org

[13]

Perrine T. et al., Teracrack: Password cracking using TeraFLOP and

PetaByte Resources (2003), http://security.sdsc.edu/publications/teracrack.pdf

[14]

Pippin et al., CS240A Applied Parallel Computing:

Final Report - Parallelization of John the Ripper Using MPI (2004),
http://www.cs.ucsb.edu/~gilbert/cs240aSpr2004/teamfinder.html

[15]

Solar Designer, John the Ripper password cracker (1996-),

http://www.openwall.com/john/

[16]

Stubbs, D., MPICracker 1.0 (1999),

http://www.ibiblio.org/pub/Linux/system/security/

[17]

Rivest, R., The MD5 Message-Digest Algorithm, RFC1321 (1992),

http://www.ietf.org/rfc/rfc1321.txt

[18]
[19]
[20]

Roussev, V. et al., Breaking the Performance Wall: The Case for

Distributed Digital Forensics (2004),
http://www.dfrws.org/2004/bios/day2/Golden-Perfromance.pdf
Volkerding, P., Slackware Linux Project (1993-),
http://www.slackware.org
Wikipedia, Microsoft, Google, et al., Leet, Leet speak, 1337, Hakspeak