1

Kolmogorov Complexity
Davide Basilio Bartolini University of Illinois at Chicago Politecnico di Milano dbarto3@uic.edu davide.bartolini@mail.polimi.it

Abstract What follows is a survey of the eld of Kolmogorov complexity. Kolmogorov complexity, named after the mathematician Andrej Nikolaevi Kolmogorov, is a measure of the algorithmic complexity of a certain object c (represented as a string of symbols) in terms of how hard it is to describe it. This measure dispenses with the need to know the probability distribution that rules a certain object (in general, the distribution may not be dened at all) and is thus more general than Shannons entropy, while describing basically the same fact from a slightly different point of view. This paper gives an overview of this eld of study from the origins to some of the latest developments and applications.

I. INTRODUCTION The idea of measuring the complexity inherent in a certain object in an algorithmic way was independently developed in the mid 1960s by Ray Solomonoff [12,13] , Gregory John Chaitin [4] and Andrej Nikolaevi Kolc mogorov [7] . This new measure (know today as Kolmogorov complexity, or algorithmic complexity) is dened according to a novel algorithmic way of giving a quantitative denition of information, in contrast with the combinatorial and probabilistic approaches that were already known at the time [7] . Roughly speaking, the new approach is to dene the complexity of an object depending on how hard it is to describe it. The universal measure of the inherent difculty of giving a description, needed for such an approach to be applicable, is provided by the theory of computation in terms of the length of the shortest program that, run on a universal Turing machine, will give the studied object (represented as a string of symbols) as its output. The solid basis provided by the theory of computation allows to apply algorithmic complexity to virtually any computable object, without the need for it to be ruled by a known probability distribution. A. Structure of the survey First, some useful concepts from the theory of computation are exposed (Section I-B), then a brief introduction to the main topic is given (Section II) and the basic denitions needed to understand the theory are presented (Section II-A); also, some of the most interesting results are reported (Section II-B) and parallels are made with classical information theory (Section III). Then, some works extending Kolmogorov complexity theory in different areas - namely games theory (Section IV-A), quantum information theory (Section IV-B) and computational complexity theory (Section IV-C) - are described, to give an idea of some developments of the original theory and links to other research areas. Last (Section V), a few applications of Kolmogorov complexity to real-life problems are shown, to highlight how this theory proved useful in understanding real phenomena and in solving practical problems. B. Notions from computability theory To understand the denition of algorithmic complexity, some notions from computability theory are needed, which are briey pointed out in this section. In particular, the existence of a universal model to evaluate the difculty of describing a certain string is crucial to permit the denition of algorithmic complexity. This model is provided by the Turing machine, described by Alan Turing in 1937 and its universality comes from Churchs thesis, which states that all (sufciently complex) computational models are equivalent (i.e. they can compute the same family of functions) and each of them can be simulated by a universal Turing machine [5] . Now that the use of the Turing machine within this context is clear, let us dene this computational model.

1) Turing Machine: Many equivalent denitions (more or less formal) are possible for a Turing machine; a formal denition is not necessary here and an intuitive idea should sufce. Intuitively, a 3-tape-symbol bounded transfer Turing machine [4] is a logical device formed by a control module (which is a nite-state automaton) and three tapes that can contain symbols or blanks; the tapes can be read/written, one symbol at a time, through dedicated heads. The machine is able to execute instructions written on its input tape (which is read-only) by changing its internal state, reading and writing symbols on its work tape and providing output by writing on its output tape. A graphical representation of such a device is shown in Figure 1, where the arrows indicate the symbol over which the heads are placed. From a mathematical point of view, a Turing machine can be viewed

Fig. 1.

Graphical representation of a Turing machine (adapted from [5] ). The arrows indicate the position of read/write heads

as a map from a set of nite-length strings (which are the programs to be written on the input tape) to the set of nite- or innite-length strings (retrieved from the output tape). In this representation, the work tape is not considered, as it is used only internally to the machine. In this context, when considering a binary Turing machine (binary computing machine [4] ), the set of functions f : {0, 1} {0, 1} {0, 1} computable by a Turing macine is called the set of partial recursive functions. 2) Church thesis: Church thesis (also known as Church-Turing thesis), is a statement about the nature of computable (meaning computable by a machine) functions. Just like for the denition of the Turing machine, many equivalent formulations of the thesis exist; the most known version of it is due to Kleene and it goes: Every computable numerical partial function is partial recursive. One of the main consequences of the thesis is that there exists a universal Turing machine able to simulate (with proper emulation instructions provided at the beginning of its input tape) any other Turing machine. The next section shows that this result is crucial, since the difculty of describing an object is dened in terms of the length of the shortest program (i.e. sequence of input symbols) needed for the universal Turing machine to halt having returned the wanted string on its output tape. II. MEASURING COMPLEXITY As already stated, the concept of algorithmic complexity emerged from three different authors around 1965. Each of these authors (Chaitin, Solomonoff and Kolmogorov) comes to the denition of complexity from a different background and using different notations. In particular, Chaitin [4] tries to apply information-theoretic and probabilistic ideas to recursive function theory; Solomonoff [12,13] comes to dening complexity when trying to obtain the prior probability of strings under an inductive inference point of view; Kolmogorov [7] directly proposes a novel algorithmic way of quantitatively dening information. It is very interesting to see how the three of them converged to the same results when starting from very different areas of research. The following tractation will try to unify the notation from the three sources, in order to provide a coherent and understandable overview of the main ideas in the theory. A. Denitions 1) Kolmogorov complexity: Given a nite-length binary string x of length l(x), denoting a univeral Turing machine as M and writing M(P ) to indicate the output of M when executing the program P (which, in turn, is a nite binary sequence of length l(P )), we can dene1 Kolmogorov (or algorithmic) complexity as the minimum length for a program that, evaluated by M, makes it return x as the output and then halt.
1 This

same denition can be found, even if with different notations, in all of [4] , [12,13] , [7] .

Denition 1 (Kolmogorov complexity): KM (x) min

P :M(P )=x

l(P )

(1)

An interesting interpretation of KM (x) is given by Solomonoff [12] , where the a priori probability of a binary string x is dened as 2KM (x) . This shows that the algorithmic complexity can be used as a measure of randomness where the higher the complexity, the more random the string happens to be. To give the feeling of this concept, Solomonoff [13] states that this method of inductive inference (i.e. of nding the prior probability of a string) can be, in a sense, seen as an inversion of Huffman coding where the minimal code of the string is given and, from this code, it is possible to derive the probability of the string. Note that these operations can be performed without the need for the string to be ruled by a known probability distribution of any kind. 2) Conditional Kolmogorov complexity: It should be quite clear that for any program (i.e. algorithmic description) to correctly yield the wanted string x when processed by a universal Turing machine M, it needs to include information about the length of the string l(x), if this information is not already present in the machine. When this is the case, the denition of complexity provided in Denition 1 can be used. On the other hand, it is possible to consider the case when the length of the string is hardcoded in or separately given to the Turing machine used for the computation; in this case, the needed program will be shorter than the one including information about l(x) and the conditional Kolmogorov complexity [5,7] is dened as: Denition 2 (Conditional Kolmogorov complexity): KM (x|l(x)) min
P :M(P,l(x))=x

l(P )

(2)

3) Examples of complexity: The denition of algorithmic complexity provides a really smart way of dening how difcult it is to dene one object and it should appear clear that this is related to the randomness of the object and to how much information it can convey. To get an intuitive idea of how this works, a couple of examples of more or less complex objects follow. Consider the following objects chosen from the domain of alphanumeric strings: 1) KolmogorovKolmogorovKolmogorovKolmogorovKolmogorovKolmogorov 2) acj73fd3HW24F3dj4S6e2dsG457hEW46fsda34701STHS45fa554naRY5782 It should be quite intuitive to see that the complexity of the second string is way greater than the complexity of the rst one. Keeping in mind that any computable function, with proper transformations, may be calculated with a Turing machine, the following considerations will stay at a high level of abstraction, to give an intuitive feeling of the matter. The rst string is simply the repetition of the word Kolmogorov for six times. So, its algorithmic complexity can be no greater than the length of the substring Kolmogorov plus a little more information about the length of the whole string. Its conditional complexity is even lower, since the information about the length of the string does not need to be included in the description. Considering that the overhead of describing an extension of this string (i.e. the repetition of the word Kolmogorov for n times, with n > 6) adds little to the length of the description that we need to provide, it is easy to see that the complexity of such a string, for n , is less than the length of the string. The second string looks quite random and, probably, there is no better description, in terms of an algorithm to derive it, than reporting the exact characters of the string, plus the overhead of having to indicate its length. In this case, the complexity of the string is greater than its length and the conditional complexity is about the same as the length of the string. Other interesting examples may be found when considering objects that can be efciently described using mathematics. For instance, the binary encoding of up to the n-th decimal digit can be algorithmically described with a program that has almost constant size for any n. Hence, this string of bits that could look pretty random at rst glance has in fact a fairly low algorithmic complexity. The following denitions and theorems should give a formal shape to the intuitions provided by the above examples. 4) Algorithmic randomness and incompressibility: The notions of algorithmic complexity, as dened above, permit to dene the conditions under which a string x = (x1 x2 . . . xn ) of length n can be considered random. A denition of this concept of algorithmic randomness [3,5] follows:

Denition 3 (Algorithmic randomness): x is algorithmically random iff. K(x|n) n (3)

Denition 3 states that a string is to be considered random if its conditional Kolmogorov complexity (i.e. the length of its minimal description, given the length of the string) is greater than the length of the string itself. This makes sense when considering the examples of Section II-A3, where the string that looks less random happens to have an algorithmic complexity that is less than its length, while the opposite holds for the random string. The concept of conditional Kolmogorov complexity allows to dene another property of a string x = (x1 x2 . . . xn ), correlated with its algorithmic randomness. This property, dened below, is incompressibility [5] : Denition 4 (Incompressibility): K(x|n) =1 (4) n Roughly speaking, a string is incompressible if the length of its minimal description - given the length of the string - tends to be equal to the length of the string as n goes to innity. This denition gives an interesting interpretation of Kolmogorov complexity in terms of how much a string can be compressed. In fact, a string with low Kolmogorov complexity can be computed by using a short program, which can be seen as a compressed version of the string. The original string can be reconstructed by running the program on an appropriate Turing machine, which plays the role of a decompressor. In the next section, some of the most interesting and meaningful results based on the denitions given above are presented. x is incompressible iff. lim
n

B. Main results The original papers that rst introduced the concept of algorithmic complexity and successive work by the same authors and others provide a wide range of results based on the denitions provided in Section II-A. Probably, the most fundamental one is that Kolmogorov complexity is not computable, but it can be approximated from above by a computable process (hence, an upper bound exists, as shown below). The following tractation shows this and some other interesting results. 1) Universality: The rst important result should look almost obvious after the brief discussion of computability theory provided in Section I-B and is presented in the following theorem [5] : Theorem 1 (Computer independence of Kolmogorov complexity): If U is a universal Turing machine, then for any other Turing machine (and, more generally, for any computing machine) A there exists a constant cA such that: KU (x) KA + cA The constant cA in the theorem is due to the overhead needed in the program for the universal Turing machine U to provide the instructions on how to emulate the other computer A and can be safely neglected when the length of x is big enough. Hence, due to the fact that there exists a universal Turing machine able to simulate any other computational machinery, the algorithmic complexity of an object is independent of any specic computer but for a constant term that becomes neglectable for l(x) = n . This result is really crucial to make Kolmogorov complexity a useful measure without the need to refer to a particular computing device. For the following two results, the eld is restricted to binary strings and binary computers; see that this restriction does really not result in a loss of generality, since any string may be represented in a binary alphabet just by translating it with a proper encoding. Also, the notations K(x) and K(x|l(x)) are used without specifying the specic Turing machine, thanks to the result shown in Theorem 1. 2) Upper bound on Kolmogorov complexity: An upper bound to the algorithmic complexity can be easily posed rst for conditional complexity: Theorem 2 (Upper bound for conditional complexity): K(x|l(x)) l(x) + c where c is a non negative constant. The proof is quite simple, as an effective program for obtaining a string x, when l(x) is known, is simply formed by the string itself with at most a constant overhead dependent on the computer the program is written for.

The above result can be extended to Kolmogorov complexity, when the length of the string is not known (i.e. a program to compute the string must be self-delimiting [3] ) as by adding a term that represents the inclusion of the information about l(x): Theorem 3 (Upper bound for Kolmogorov complexity): K(x) K(x|l(x)) + 2 log l(x) + c The upper bound comes from the fact that a trivial method of making the program self-delimiting requires at most 2 log l(x) + c bits [5] , but it can be rened by nding an optimized way to represent l(x) (i.e. by using iterated logarithms [4,5] ). 3) Lower bound on Kolmogorov complexity: A probably more interesting result is xing a lower bound on algorithmic complexity or, more properly, a lower bound on the number of strings within a certain complexity. This is interesting because, by doing so, one can get a avor of how likely it is for a string to be whether complex or simple to describe. Theorem 4 (Lower bound on Kolmogorov complexity [3,5] ): |{x {0, 1} : K(x) k}| < 2k The theorem shows that there are not many string with low complexity and is simply proven by the fact that the number of binary programs of length less than k is 2k 1 < 2k . 4) Information-theoretic version of G del theorem: A more advanced result shown by Chaitin [3] is briey o reported here (without any claim of being axhaustive), as an aside to show how algorithmic information theory can provide an alternative of G dels famous theorem. o Chaitin shows that, in an axiomatic theory, a lower bound n on the algorithmic complexity of a certain string of symbols dened in the theory can be established only if n is less than the algorithmic complexity of the axioms of the formal theory (i.e. the axioms used for the demonstration of the bound). This shows an inherent limitation of axiomatic theories that is assimilable to G dels incompleteness theorem by using an information o theoretic argument based on Kolmogorov complexity. III. COMPLEXITY AND ENTROPY Algorithmic complexity and entropy are dened from very different backgrounds and, apparently, measure different aspects of an object. Despite of these apparent differences, it is possible to show that complexity and entropy are both a measure for the randomness of a string and, under some hipotheses, a relationship between the two can be proven, as shown in Theorem 5. Theorem 5 (Relation between Kolmogorov complexity and entropy [5,8] ): Let {Xi } be a stochastic process n drawn i.i.d. f (x), x X , |X | < and let f (xn ) = i=1 f (xi ). Under these conditions, it can be proven that 1 E K(X n |n) H(X), as n . n The above theorem shows that algorithmic complexity and entropy turn out to be very similar measures. Of course, this result holds only under the specied hypotheses, which ensure that both complexity and entropy are well dened. This result makes Kolmogorov complexity an even more meaningful tool since it is assimilable to entropy when both are dened, but can be used in a more general context where the probability distribution (or a good estimate) is not available for the studied object. IV. FURTHER DEVELOPMENTS Section II showed the main denitions and the most interesting results of the original theory of algorithmic complexity. In about ten years after the rst papers (published in the mid 1960s), the theory grew to a complete theoretical system and, in 1977, Chaitin published a review of what he called algorithmic information theory [3] . Beyond these direct developments of the theory, Kolmogorov complexity has been extended to very different areas. The following sections show some of these branches, giving an idea of how wide is the scope of the concept of algorithmic complexity.

A. Game interpretation One of the most fascinating of the areas where Kolmogorov complexity has been applied is Game theory. The link between the two theories is provided by Muchnicks theorem, which relates the theory of recursive sequences (which algorithmic complexity theory is based on) and game theory. In particular, this theorem associates every statement of recursion theory with a two-players game G with innitely many moves represented by 0s and 1s. Considering this game, it is shown that if one of the players (called the Mathematician) has a computable winning strategy, then the statement is true, while if the other player (called Nature) has a computable winning strategy, then is false. Since Kolmogorov complexity is dened upon recursion theory, it is possible to exploit Muchnicks theorem to analyse the truth value of statements about the theory by the means of building a proper game, as dened in the theorem. This provides an interesting and unconventional point of view on the matter of Kolmogorov complexity and is extensively treated (with several examples) by Vereshchagin [15] . Other results in the eld of algorithmic complexity are obtained with a game-theoretic approach by Muchnick and others in a very recent paper [10] . Here, again, it is shown that it is possible to prove statements about Kolmogorov complexity by constructing a special game and a winning strategy in the game. B. Quantum complexity The increasing interest in quantum computing encouraged some studies aimed at extending Kolmogorov complexity to the domain of quantum computation. The extension of this theory to quantum computing can be performed only by a prior redenition of the relevant concepts of computation theory in this new area. Quite a lot of work has been done in this direction and Vitanyi gives an overview of the results in this eld in a paper [16] which title recalls the original 1965 paper by Kolmogorov. In particular, it is shown that Kolmogorov complexity can be quite naturally extended based on quantum Turing machines and it can be used to describe the amount of information contained in a pure quantum state (i.e. a set of variables that fully describe a quantum system in probabilistic terms). One of the analogies of quantum algorithmic complexity and the classical version is that the former, as the latter, is upper bounded and it can be approximated from above by a computable process with arbitrarily small probability of error. C. Complexity and problem spaces One of the elds where an extension of the ideas of Kolmogorov complexity seems more natural is the one of complexity theory. There is an interesting work by Allender, Buhrman and Kouck [1] that investigates y whether it is possible to characterize the complexity class PSPACE of problems solvable in polynomial space with a Turing machine by efciently reducing it to the set RK of algorithmically random strings (as dened in Denition 3). The paper is pretty technical and leaves many open problems; the interesting fact here is that the ideas behind Kolmogorov complexity can be employed in trying a new approach to the study of problems in a very wide variety of research areas. V. APPLICATIONS Kolmogorov complexity has so far been extended and exploited in a number of different research areas, as is shown in Section IV. Beyond this, the concepts of this theory have been used in a variety of practical applications, where it helped to provide a unique approach to the analysis of real world problems. The following section illustrates a few recent applications of Kolmogorov complexity. A. Information assurance Kolmogorov complexity can be applied to retrieve important information about the state of a system; in particular, it has been used by Evans, Bush and Hershey [6] to design an approach to monitoring an information system against security aws in both data and processes. In this work, the apparent complexity K(S) is dened as the best estimate of Kolmogorov complexity available to a party which is analysing the system and two metrics are proposed to evaluate the vulnerability of a process with input X and output Y : K(X.Y ) is dened as the complexity of the concatenated input and output of a process K(X|Y ) represents the relative complexity of a process Based on this measures, the vulnerability of both processes and data is analysed, showing that the higher both the quantities are, the less vulnerable the system is. This result states, intuitively, that the more complex the

operations carried out by a certain process (from the point of view of a potential attacker), the harder it is for an attacker to understand what the process is doing in order to harm the system. Thanks to these results, a method to monitor the security level of a system, based on the ability of estimating the apparent complexity of the system to an attacker, is proposed and the conclusion is that Kolmogorov complexity can be a good candidate for further developments in this area. B. Spam ltering Another smart application of Kolmogorov complexity has been published by Spracklin and Saxton [14] , who apply it in a email spam lter. The underlying idea is that, when highlighting in a text the words linked with spam, their distribution will be quite random in normal mail, while it will obey some ordered criterion in spam. This idea is applied by encoding the text of an email into a binary string where the words linked with spam are represented with a 1, while the other words are represented with a 0. Then, Kolmogorov complexity can be used to evaluate how random the resulting binary string is. The issue here is that Kolmogorov complexity is not computable; so, the idea is to apply a compression algorithm (run-lenght compression in the specic case) to get an estimate of Kolmogorov complexity. Based on this estimate, messages with high complexity are classied as non spam, while messages yielding a low complexity are identied as spam. The authors show that this approach ensures high accuracy (80% to 96%) and is much faster than other approaches used for the scope (for instance, Bayesian lters). C. Mental fatigue The capability of Kolmogorov complexity to be applied to any object encodable as a binary string permits to exploit it on virtually any data. For instance, Kolmogorov complexity is used in a medical context by Lian-yi and Chong-xun [9] to evaluate the level of mental fatigue of a person based on the signals coming from the EEG (Electroencephalogram). As in Section V-B [14] , an estimate of Kolmogorov must be computed to be able to analyse the data; in this work, this is done by using a modied version of the Lempel-Ziv alogrithm applied to the discrete sequences representing the EEG signals. The results are encouraging, as it is shown that the estimate of Kolmogorov complexity of the EEG decreases as the mental fatigue increases. This indicates that the EEG signal is somehow less random when a person is in a state of mental fatigue. D. Complexity in machine learning Machine learning is an applied research eld where randomness is often exploited in algorithms (for example, in the training algorithm of a neural network). One of the notions used in this eld is known as Occams razor (referring to the 14th-century English logician, theologian and Franciscan friar Father William of Ockham) and it can be - a little improperly - summarized as: the simplest explaination is the best. In machine learning terms, this means that the simpler the rules found to explain the training data, the better the generalization on the testing data. One problem in classical machine learning applications is the lack of a general and well founded way of determining which rules are the simplest ones for a given data set. Schmidhuber [11] discusses this issue and proposes a method based on Levins complexity (which is a time-bounded extension of Kolmogorov complexity) to pose an ordering on the complexity of sets of rules and allow to efciently choose the best ones. In this work, some experiments on simple problems (chosen to be computationally feasible) are described which show that the proposed method is able to yield solutions with generalization performance unmatchable by different training algorithms. VI. CONCLUSION Kolmogorov complexity is a fascinating concept which really captures how the complexity of an object can be described by scientic and rigorous means. This paper should have given an idea of how powerful and meaningful the ideas rst introduced by Solomonoff, Chaitin and Kolmogorov are and how these ideas have been useful in exploring new possibilities in a great number of research areas and applications. Beyond the achieved results, many open problems remain and today the idea of Kolmogorov complexity can help the understanding of still unsolved complex problems. For instance, it is believed that there exist not fully understood strong links between the concept of Kolmogorov complexity and different areas of physics, from thermodinamics to black holes. As the bottomline, the eld of Kolmogorov complexity is a mature but still very active research area, able to provide many useful results and answers, but leaving some appealing and challenging problems still to be solved.

R EFERENCES [1] E. Allender, H. Buhrman, and M. Kouck, What can be efciently reduced to the kolmogorov-random strings? Annals of Pure and Applied Logic, vol. 138, no. 1-3, pp. 2 19, 2006. (Referred to at page 6). [2] G. J. Chaitin, A theory of program size formally identical to information theory, 1975. (Not directly cited in text) [3] , Algorithmic information theory, IBM JOURNAL OF RESEARCH AND DEVELOPMENT, vol. 21, pp. 350359, 1977. (Referred to at pages 3 and 5). [4] , On the length of programs for computing nite binary sequences: Statistical considerations, Journal of the ACM, vol. 13, pp. 547569, 1969. (Referred to at pages 1, 2 and 5). [5] T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006, ch. 14. (Referred to at pages 1, 2, 3, 4 and 5). [6] S. Evans, S. Bush, and J. Hershey, Information assurance through kolmogorov complexity, DARPA Information Survivability Conference Exposition II, 2001. DISCEX 01. Proceedings, vol. 2, pp. 322 331 vol.2, 2001. (Referred to at page 6). [7] A. N. Kolmogorov, Three approaches to the quantitative denition of information, Problems of Information Transmission, vol. 1, no. 1, pp. 17, 1965. (Referred to at pages 1, 2 and 3). [8] S. Leung-Yan-Cheong and T. Cover, Some equivalences between shannon entropy and kolmogorov complexity, Information Theory, IEEE Transactions on, vol. 24, no. 3, pp. 331 338, may. 1978. (Referred to at page 5). [9] Z. Lian-yi and Z. Chong-xun, Analysis of kolmogorov complexity in spontaneous eeg signal and its application to assessment of mental fatigue, Bioinformatics and Biomedical Engineering, 2008. ICBBE 2008. The 2nd International Conference on, pp. 2192 2194, may. 2008. (Referred to at page 7). [10] A. A. Muchnik, I. Mezhirov, A. Shen, and N. Vereshchagin, Game interpretation of kolmogorov complexity, ArXiv e-prints, mar 2010. (Referred to at page 6). [11] J. Schmidhuber, Discovering solutions with low kolmogorov complexity and high generalization capability, in Machine learning: proceedings of the twelfth international conference. Morgan Kaufmann Publishers, 1995, pp. 488496. (Referred to at page 7). [12] R. Solomonoff, A preliminary report on a general theory of inductive inference, 1960. (Referred to at pages 1, 2 and 3). [13] , A formal theory of inductive inference, part l, Information and Control, vol. 7, pp. 122, 1964. (Referred to at pages 1, 2 and 3). [14] L. Spracklin and L. Saxton, Filtering spam using kolmogorov complexity estimates, Advanced Information Networking and Applications Workshops, 2007, AINAW 07. 21st International Conference on, vol. 1, pp. 321 328, may 2007. (Referred to at page 7). [15] N. Vereshchagin, Kolmogorov complexity and games, 2008. (Referred to at page 6). [16] P. Vitanyi, Three approaches to the quantitative denition of information in an individual pure quantum state, Computational Complexity, 2000. Proceedings. 15th Annual IEEE Conference on, pp. 263 270, 2000. (Referred to at page 6).