The Wysiwyg Universe

The Wysiwyg Universe
http://wysiwyg.stanford.edu/
by Simon Free
16,192 visitors have experienced this page.
Document Version 1.2, April 2004 Copyright 2002-2004 Simon Free Permission is granted to copy, distribute and/or modify this document under the terms of the Open Content License.
View Table of Contents
Introduction
WYS.I.WYG /'wi-zE-"wig, adjective: What YouSeeIsWhatYouGet
Much has been written in the fields of philosophy, psychology and physics regarding the role of the
observer in understanding the nature of reality. Physicists are zeroing in on what they hope to be a "Theory of Everything", a set of mathematical equations that describe all that there is to know about the basic building blocks of the Universe. At the same time, psychologists and neurobiologists are closer than ever to a complete map of how the brain allows us to see, think, feel, and even dream. Yet despite these advances in our scientific understanding, there remain some unsettling gaps between what we say about the world and what we actually experience. For example, no description of water, no matter how poetically vivid or mathematically correct, will quench your thirst. Similarly, the abstract beauty of Maxwell's electromagnetic field equations seems pale by comparison to the myriad hues of a sunset. Even the most elegant theories of the origins of the Universe seem somehow devoid of substance and leave us uncomfortably unsatisfied. In the words of physicist Stephen Hawking: "What is it that breathes fire into the equations and makes a universe for them to describe?" It will be argued here that these explanatory gaps arise from a simple ontological error commonly committed when discussing what we take to be reality. At the root of this confusion is the assumption of a world beyond what we experience. This assumption generally begins as a model of a set of experiences which is formulated
1 of 25
11/28/2008 9:50 AM
either verbally or in the language of mathematics. If a model is successful, it allows us to make predictions about observations within a certain domain of experience. However, the more successful a model becomes, the greater is the tendency to speak of the model as though it describes the way the world really is. But this confuses the map for the territory. Few would argue that a map of the Rocky Mountains actually is the mountain range, no matter how detailed or elaborate the map. Similarly, no neurological description of the taste buds in your tongue and their connections to your brain will enable you to experience a flavor you've never tasted before. It will be shown below that by adopting two simple postulates regarding the relationship between experience and reality, we can eliminate the confusion which tends to arise between our models of reality and reality itself. But before we take the plunge, let us examine two particular cases where the apparent gap between model and experience seems just too great to cross.
The Quandary about Qualia
There is a long standing debate in philosophy and psychology on the nature of qualia which are defined as
those elementary forms of experience such as color, taste, sound etc., with which each of us is intimately familiar. Despite great strides in the understanding of brain function over the past century, little progress has been made toward the elucidation of how the firing of neurons in the brain can give rise to these subjective qualitative states. For example, a pin prick to the finger causes nerve endings beneath the skin to send impulses to pain centers in the spinal cord and brain. These centers are connected to reflex motor actions which cause the withdrawal of the finger away from the pin. However, we not only react to the pin prick in this neuromechanical fashion; we also feel pain. How this subjective feeling of pain arises from the firing of neurons in the brain is a mystery. To put it another way, we have no idea why neurons firing in one part of the brain give rise to pain, while neurons firing in a different part of the brain give rise to sounds, and still others produce the experience of colors. It appears that qualia are made of a different "stuff" than the cells in the brain or the molecules which make up the cells. In fact, many have argued that there can never be a physical explanation of how we come to have experience. Others deny that we really have experiences at all! This apparent ontological "gap" between the physical world and the world of experience is what philosophers call the hard problem of consciousness or the mind-body problem.
Quantum Quirkiness
The second domain in which experience does not seem to follow directly from the physical world is the
realm of the very small, such as the level of atomic and subatomic processes. At the more macroscopic levels of experience, the process of measuring the property of an object, such as the position of a rock, generally yields a single value for any given state of the object. However, in the world of electrons and other subatomic entities, measuring the property of an object can yield more than one result, even though the initial state of the object is the same from one measurement to the next. For example, an electron has a property called "spin" which can take on only one of two values, "up" or "down". (Think of spin as a little arrow attached to the electron, where the arrow can point either up or down but not in between.) If a suitable measurement of spin is made on a large number of identically prepared electrons, half the time we get the value "up" and the
2 of 25
11/28/2008 9:50 AM
other half the time we get the value "down". For any given electron, there seems to be no way to predict which value will be measured. The branch of physics called quantum mechanics deals with this uncertainty by describing objects such as electrons using a formula called the wave function. The wave function of a quantum mechanical system prescribes both the set of possible values one can obtain during a measurement of a given property, as well as the probabilities of obtaining each result. For example, when the wave function is computed for an electron passing through a magnetic field, the formula admits just two possibilities: spin up and spin down. It also prescribes probability values of 0.5 for obtaining the value "up" and 0.5 for obtaining the value "down". So when measuring the spin of an electron, quantum mechanics can only tell us that the probability of measuring spin "up" is 0.5 and the probability of measuring spin "down" is 0.5. We will definitely obtain one or the other value, but which value we obtain is apparently determined by the toss of a coin. Needless to say, many physicists are not happy with this apparent irreducible randomness in the subatomic realm. Consequently, a number of theories and conjectures have arisen over the years in an attempt to understand what actually determines the value of a quantum measurement. In the case of electron spin for example, some physicists speculate that, prior to a measurement, the wave function of the electron actually describes a state of being both spin up and and spin down--a 50-50 superposition of opposite spin states. Since we know that we only ever observe either spin up or spin down, this superpositional state is not a physical one. To turn this non-physical state of superposition into a real value of the spin, the process of measuring the spin is said to "collapse" the wave function into one of its particular values. This seems to imply that the process of observation actually helps to create the reality which we ultimately observe. Imagine how strange this would seem at a macroscopic level. Suppose that a rose can appear either red or yellow. On half of the occasions we look at the rose it appears red, and the other half of the time it appears yellow. As much as we study the other properties of the rose, there seems to be no way of telling before hand which color we will see--only that there is a 50-50 chance it will be red or yellow. So what color does the rose have when we are not looking at it? Is it both red and yellow? Is it orange? Does it even have a color at all? Or is the question even meaningful? And what actually happens when we do look at the rose? We only ever see one color at a time so how does the unobserved and ambiguous color state of the rose (whatever this means) actually become the color we observe? Does the very process of our looking at the rose actually cause one color to be selected over the other?
The Wysiwyg Hypothesis
To clarify the respective roles of experience (qualia), models (science) and reality, we will begin with two
simple postulates. First Postulate: Experience is the only reality for an observer. Second Postulate: There is no reality other than experience. In general, postulates are assumed to be irreducible assumptions of a theory. However, one normally chooses these assumptions because they seem reasonable or necessary, at least on the surface. In this case, the two postulates may or may not seem self-evident. We therefore begin with arguments for their reasonableness before proceeding to the issues of qualia, quantum mechanics and a few other puzzling issues in science and mathematics. Arguments for the First Postulate
3 of 25
11/28/2008 9:50 AM
Since the issue is experience, and you are the only one who can examine your own experience, ask yourself the question: "What do I know of reality besides what I experience?" Suppose you answer: "I know that I have a brain in my head which allows me to have experience." Of course, you don't directly experience your brain--you can't even see it. So the "knowledge" that you have a brain in your head is an assumption based on what you have learned from the cultural database of knowledge that our society has collected to this point in time. Do you think this knowledge in any way affects the way you experience, say, color? Most non-scientists are probably completely unaware of the detailed neurobiology behind color vision, yet they see colors just fine. So the model we have of how brain activity enables us to perceive color is not the same as the experience of color itself. Hopefully this much is clear. Next you might argue: "While our model of brain function does not determine how I see colors, surely the function of the physical brain does." This is where the "map for the territory" error creeps in. To demonstrate how this comes about, imagine that you could peer inside your own head and actually see your brain in action. (In fact, we can do something just like this using the technology of functional MRI.) By viewing the workings of your own brain, what additional knowledge do you now have that relates to your experience of color? Let us focus on the visual subsystems which neuroscientists know to be involved in color vision. At first, you may not identify the various neural components in front of you. They appear to you as various shapes and sizes--in other words, as visual experiences. If you have read about the brain mechanisms behind color vision or you are a researcher in the field, you will have associations between these visual experiences and the names and functions of each component. In the end you could put together a verbal explanation (a model) of how you see color: "Light passes through the lens of my eye stimulating the rods and cones in the retina. The rods and cones activate my optic nerve which sends impulses to my visual cortex. Here I can see specialized neurons that have evolved to detect the ratios of wavelengths in the original light. For example, whenever I experience the color red, I see that cortical cells in region A are active. Whenever I experience the color blue, I see that cells in region B are active." In the end you have correlated one experience with another: your subjective experience of color on the one hand, and your subjective experience of your brain's activity (as observed in our imaginary scenario) on the other hand. This is actually very useful knowledge. For example, it allows you to predict that if you were to remove or otherwise damage the cortical cells in region A, you would probably have difficulty experiencing the color red. However, what you have not done is observe how a physical process (brain activity) gives rise to your experience. What we call the "physical brain" is actually a collection of experiences just like the experiences we are trying to explain. It makes no difference how detailed a description or model we make of the brain--right down to the quantum mechanical fluctuations in the spacetime continuum if you like: each component of the description will arise from its own collection of experiences and, as with the map of the mountain range, even the most articulated model of the brain is not the brain itself. The distinction between the physical world and the experiential world is therefore seen to be artificial; hence our first postulate: Experience is the only reality for an observer. Arguments for the Second Postulate It is one thing to say that experience is the only reality for a given observer (yourself for example). The second postulate takes this a step further: There is no reality other than experience. For thousands of years, thinkers in all cultures have wondered what is the "true nature" of reality behind our experience. Tribal societies imagine animated spirits behind objects and events. Religious leaders believe in the existence of an overarching theological plan. Philosophers have argued for a world of ideal platonic forms behind the objects we perceive with our senses. And scientists postulate a physical or mathematical foundation to reality. But for thousands of years, no one has come up with an answer that satisfies everyone. Perhaps there is a very simple reason for this: Experience is all there is. It should be clear from the arguments for the first postulate that ideas about a "reality behind experience" are really just models in disguise. Such ideas rely on constructions using verbal language or mathematics which
4 of 25
11/28/2008 9:50 AM
postulate unobservable entities that give rise to our experience. But this is just what a model is, and the hypothesized unobservable entities are just place holders for correlations among experiences. Furthermore, models are changing all the time as observers discover new phenomena to be explained. For example, the experience of gravity has been conceptualized and modelled very differently from the time of Aristotle's physics, through Newton's dynamics, and on to Einstein's theory of General Relativity. But rocks fall to the ground all the same. In any case, a model cannot be identified with something outside our experience. There is no way out of the experiential loop. Hence our second postulate: There is no reality other than experience.
Applications of the Wysiwyg Hypothesis
The arguments given above are meant to show that the postulates are not inherently unreasonable. Their
usefulness will become apparent when they are employed to solve a number of long standing puzzles in philosophy and science. We now proceed to the application of the postulates to the problem of qualia, the collapse of the wave function in quantum mechanics, and to a number of other troublesome issues, including the uncanny fit between mathematics and experience. Qualia, Qualia, All the Way Down Let us return to the example given above wherein you were able to view the activity of your own brain. Let E be an experience, such as the experience of the color "red". Suppose that whenever you experience E, you observe activity in a collection of cells, C, in your brain. Note that C is itself an experience: the experience you have when observing your brain cells and their activity. We denote by the notation, E ~ C, the correlation you have established between E and C. By the first postulate, E ~ C is the only reality you know: any additional facts you might acquire about the collection of cells, C, would simply add to the collection of experiences which we are calling C. By the second postulate, there is no other reality besides E ~ C. In other words, there is no hidden "physical reality" behind C nor is there some "ethereal reality" behind E. Since E ~ C is simply a correlation between like entities (experiences) there is no ontological gap which needs to be bridged between the "physical" world of neurons and molecules and the "experiential" world of mind or consciousness. In the end, there is only experience, all the way down.
Exorcising Quantum Ghosts
Recall from the introduction that measurements made on identically prepared quantum mechanical systems
can lead to different results. Which result arises during the measurement appears to be random, but each result can be assigned a probability of occurrence by computing the wave function of the system before the time of the measurement. The philosophical issue which concerns physicists is that it seems to be impossible to assign a definite value of the measured property to the object under study before someone actually observes it. Let us apply our postulates to the specific case of electron spin. What do we experience during the experiment? We experience a series of spin values, one for each electron which passes through our apparatus. If we count up the number of spin up electrons versus the number of spin down electrons, we find that they appear in roughly equal numbers. That's it. That is all we experience. According to the first postulate, reality for the experimenter is simply this: when you measure the spin of a batch of identically
5 of 25
11/28/2008 9:50 AM
prepared electrons, about half of them have spin up and the other half have spin down. At the present time, there does not seem to be a way to determine which value (up or down) one will obtain for a given electron at the time of measurement. So let E be the experience of measuring the electron spins. And let C be the collection of recordings of those spins--for example, writing a tally of "ups" versus "downs" on a piece of paper. Note that C is also an experience--the experience of recording the spins. After all, if we did not record our results (even in memory), there would be no way to do the analysis. We can then establish a correlation, E ~ C between the two experiences which says that the measurement of electron spins tends to generate a record of equal numbers of spin ups and spin downs. Physicists can take this correlation one step further. Quantum mechanics prescribes a precise method for actually computing the wave function of a system from its energy. The energy of the system is in turn computable from other well known properties of the system. We can therefore compute the wave function which represents the spin experiment ahead of time and thereby predict what the results of the measurements will be. The wave function therefore provides a model for how the system will behave. Since the wave function predicts the probability of obtaining spin up or spin down in the experiment, we can measure the spin of a large number of electrons and see if the relative numbers of spin ups and spin downs reflect the predicted probabilities (they do). In essence, the wave function takes the place of the correlation symbol, ~, between our experience of measuring spins (E) and our experience of recording the results (C). The only difference is that we now have a formula which enables us to predict the results ahead of time. What happens next is where all the confusion arises. Rather than recognizing the wave function as a succinct representation of the correlation between measurements and results, some physicists would like to endow the wave function with a reality beyond its role as a mathematical formula. The argument goes something like this: since the wave function appears to represent the state of the system before a measurement is made, then the system must actually be in that state before the measurement is made. Besides confusing the map for the territory, this is an especially puzzling view to take of the wave function, since the very role of the wave function is to represent the system before it has been observed. The ontological status of the unobservable wave function is therefore, in principle, untestable. If we apply our second postulate, there is no reality other than experience, to the problem of quantum mechanical measurement, we run into no trouble at all. In the case of the electron spin experiment, our experience includes the measurement of the spin, the recording of the results, and the wave function (formula) which correlates one with the other. We recognize the wave function for what it is, a recipe for generating probability predictions. There is no ghostly wave function in the experiment itself and therefore no need to collapse it when making a measurement. The observer therefore plays the same role as observers always do--they have experiences.
Mathematics and Experience (A more rigorous treatment of this section for the mathematically minded will appear in a forthcoming Appendix. The version presented here is intended for a more general audience.) One of the more interesting puzzles in the history of physics and mathematics is why mathematics applies to the world at all. Einstein once said, "The most incomprehensible thing about the world is that it is comprehensible." Other physicists and mathematicians have asked: Why should a symbolic system as abstract as mathematics be so good at describing physical reality? Our two postulates about experience and reality allow us to solve this mystery. First of all, it helps to know that all branches of mathematics are actually variations on a single theme. This common starting point is the
6 of 25
11/28/2008 9:50 AM
definition of a set of objects, together with a structure defined among the elements of the set. A structure is defined simply as a collection of pairings between elements of the set. In fact, a structure on a set is another set, a set where the elements are certain pairings of elements from the original set. For example, consider the set whose elements are the words "apple", "banana", "red", "yellow". We write this set using the notation {apple, banana, red, yellow}. An example of a structure on this set would be the pairings {{apple,red},{banana,yellow}}. Notice how the pairings of elements from the original set become new elements in the structure set. You can immediately begin to see how such structures can define a model. For example, the structure given above defines pairings between "apple" and "red", and "banana" and "yellow". If we were interested in the colors of various fruits, this model of fruit colors would predict the right colors for apples and bananas. A different model, say {{apple,yellow},{banana,red}} would make the wrong predictions. If this sounds familiar, it should. As we have seen, a model of a domain of experience is a correlation, E ~ C, between the set of experiences to be explained (E) and the set of recordings (C) made during these experiences. By our first postulate, recordings of results are just other experiences (like writing down numbers on a piece of paper). We can therefore combine the two sets of experiences, E and C, into a single set of experiences which we could call U. The combined set U would look something like this: {e1, e2, e3 ..., c1, c2, c3 ...} where e1, e2, etc., are experiences in the set E, and c1, c2, ... etc., are recordings (also experiences) in the set C. By our second postulate, these experiences are all we have. So the best we can do to model our experiences is to define a collection of associations between elements of this set. Such a model would look something like this: {{e1,c1}, {e2,c2}, {e3,c3}, ...} which implies that experience e1 predicts or correlates with recording c1 (and vice versa), e2 predicts or correlates with c2, and so on. But such a collection of parings is none other than a mathematical structure as defined above. We have therefore shown that any model of experience is by definition a mathematical structure. Conversely, any mathematical structure can be viewed as a model, although the vast majority of such structures will be very poor models when it comes to predicting results. Physics 101 It might seem at first glance that this definition of a model bears little resemblance to the formulas commonly found in physics, such as Newton's law of motion, F=ma, or the wave function in quantum mechanics. However, we can easily show that these formulaic models are just special cases of the more general mathematical structures defined above. Take the example of objects falling to the earth. The great Italian physicist Galileo Galilei recorded the time it took a ball to roll down a long inclined plane by using his pulse as watch (and later, a water clock). We see immediately that Galileo's experiment involved correlating one experience with another; namely, the experience of viewing the rolling ball at different points along the inclined plane, and the experience of feeling his pulse. Let E be the set of experienced pulse beats and let C be the set of positional recordings of the rolling ball. The set E can be written as {p1, p2, p3, ..., pN} where pk corresponds to kth pulse beat since the release of the ball at the top of the inclined plane. The set C can be written as {d1, d2, d3, ..., dN} where dk is the distance the ball has rolled down the plane by the kth pulse beat. The simplest model of the results would be the structure {{p1,d1}, {p2,d2}, ..., {pN,dN}} which says that the object will be found at distance d1 at pulse p1, at d2 at pulse p2, etc. Of course, this model would only work for one angle of the inclined plane. Fortunately, both pulse beats and distances can be given numerical representations. If we assume that the pulse beats are equally spaced apart, then we can represent the pulse beats with integers and write this new set as T = {1, 2, 3, ..., N} and call it "time". Next, regardless of the units used to measure distance, we can write their values relative to the distance traveled during the first pulse beat. What Galileo discovered was that, independent of the angle of
7 of 25
11/28/2008 9:50 AM
the plane, the distance measurements took on the relative values D = {1, 4, 9, ..., N2}. Our new model therefore relates time and distance expressed numerically and looks like this: {{1,1}, {2,4}, {3,9}, ... {N,N2}}. Even with Galileo's crude timing measurements, it was clear that these numbers indicated that the distance traveled by the ball (the second number in each pair) was related to the square of the time since the ball was released (the first number in each pair). What's more, this squaring relationship did not depend on the angle of the inclined plane or the mass of the ball used to do the experiment. These results implied two important properties of falling objects: that objects do not fall to the ground with a constant speed but are accelerating; and that this acceleration does not depend on the mass of the object. This latter result actually invalidated the model of falling objects proposed by Aristotle who thought that heavier objects fall faster than lighter ones. Only by slowing down the rate of motion by using the inclined plane was Galileo able to collect the experiences required to disprove Aristotle's claim. Given the numerical form of Galileo's model, we can write the results in more succinct form as D = kT2 where k is a constant of proportionality related to the angle of the inclined plane. It is this form of the model which we generally recognize as a mathematical or physical "law". But note that Galileo himself actually started with a formulation of his results exactly like the set notation used above. It was the numerical representation of the experiment that made the results amenable to a simple formulaic analysis. And this numerical representation was possible only because the experiences under study were of such a nature that numbers make a good representation of the results. How else would you choose to represent distance? Or what could be more natural than counting your pulse to measure time? The fact that certain aspects of our experience such as distance, time, mass, force, temperature, brightness, etc., can be easily represented numerically is the reason we can turn our observations of correlations among such properties into numerical formulas. Genetic Gymnastics Observations in other domains of experience, such as how the genes in biological cells direct the construction of proteins, do not lend themselves so naturally to numerical representation. Nonetheless, we can still generate a model of protein synthesis in terms of the genetic code which allows us to fully understand how genetic variation can produce changes in the structure and function of biological organisms. Furthermore, this model qualifies as an example of a mathematical structure using our more general definition. Biologists have known since the 1950's that all proteins are made up of combinations of 20 different amino acids. They also know that genes are composed of combinations of just 4 nucleotides. Somehow genes direct the construction of proteins but how? The set of all possible amino acid sequences and the set of all possible nucleotide sequences are both potentially very large. For example, even smaller proteins consisting of only 100 amino acids could come in potentially 20100 variations. This number is so huge (larger than the total number atoms in the universe) that it defies imagination. However, it can still be treated as a set. Similarly, there are approximately 100,000 genes in human cells which code for proteins and each of these genes contains anywhere from 1,000-2,000 nucleotides. So a gene can come in roughly 41000 variations which is again an unthinkably large number. The challenge before the biologists was to figure out which sequences of nucleotides in genes coded for which sequences of amino acids in proteins. And you thought Galileo had it tough! The story of how this coding problem was finally solved is one of the most intriguing pieces of detective work in the history of science. Theorists from physics, information processing, mathematics and biology all had a crack at it. The key was the realization that a genetic "alphabet" of four letters (nucleotides) could easily code for 20 amino acids if these letters formed at least three letter words, one word for each amino acid. For example, with four letters to choose from, one letter words give us only 4 possibilities. Two letter words yield 4x4 = 16 possibilities--still not enough to code for 20 amino acids. Three letter words come in 4x4x4 = 64 variations which would be more than enough to account for the 20 amino acids. Much theoretical speculation was aimed at understanding how these 64 genetic codes could be reduced to just 20 unique amino acid codes.
8 of 25
11/28/2008 9:50 AM
In the end, the solution had to be determined empirically, that is, by actually looking inside the cells to see how segments of a gene were transcribed into pieces of a protein--just as Galileo had to record the positions of the rolling ball at each beat of his pulse. The result was a mapping between three letter genetic sequences and amino acids which bore no resemblance to the theoretical models. As we might expect using 20-20 hindsight, Nature just threw things together one code at a time over the course of evolution as necessity demanded invention. The net result is a built-in redundancy whereby multiple genetic triplets code for the same amino acid. In this way, genes have a tolerance for genetic errors which the theorists hadn't considered. The empirical data laid out a mapping between three letter genetic codes and amino acids. Our mathematical problem can therefore be recast in terms of the set of amino acids, A, and the set of three letter genetic codes, C. If A = {a1, a2, a3, ..., a20} and C = {c1, c2, c3, ..., c64} where there are 20 elements in the set A and 64 elements in the set C, then the model of the empirical results takes the form: {{a1,c1}, {a1, c27}, {a2, c9}, {a2, c58}, ..., {a20, c16}, {a20, c32}} where we have indicated that a given amino acid may be coded by more than one genetic triplet. This is our mathematical model of the genetic code. We have defined a structure of pairings between genetic triplets and amino acids. One might wonder if we could take this a step further, as Galileo did for time and distance, and come up with a formula that would allow us to compute the value of an amino acid from the genetic code(s) that stand for it. This would require that we had some meaningful way of assigning numerical values to both the set of amino acids and the set of genetic codes. Of course, we could arbitrarily label the amino acids {1, 2, 3, ..., 20} and the codes {1, 2, 3, ..., 64}, but these values do not bear any relationship to the properties of amino acids or nucleotides. In other words, what would stop us from labeling the same amino acids in reverse order; i.e., {20, 19, 18, ..., 1}? So while it does not make sense to look for a formulaic version of our genetic model, it does make sense to figure out how genetic triplets are actually translated into amino acids using the processes available to a cell. The series of steps involved in this process is now well understood and is called protein synthesis. Segments of genes containing many coding triplets are lined up with another type of genetic molecule called messenger RNA. Messenger RNA then takes its mirror copy of the genetic triplets into the cytoplasm of the cell where some very mechanical processes in structures called ribosomes line up free floating amino acids with the appropriate genetic triplet along the length of the RNA molecule. When the assembly process is complete, the string of amino acids peels away as a freshly made protein molecule. Summary Hopefully it is now clearer why the world appears to obey mathematical laws. For when we say that the world "obeys" mathematical laws, we simply mean that there are correlations (pairings) among elements of our experience which enable us to make predictions from one experience to another. For some domains such as physics, the correlations between experiences can actually be given functional form, like the wave function in quantum mechanics or F=ma in Newtonian dynamics. These formulaic models are possible because the observations of interest can be given a numerical representation. Since formulas and equations are what most of us think of as mathematics, we marvel at how such abstract looking constructs seem to fit so nicely with our experiences. But from what we have shown above, even the most complex functional equations can be defined as examples of sets with structure. Conversely, models found in other sciences, such as the genetic code in biology, are no less mathematical in their essence than those found in physics. In these cases, the relevant objects under study do not lend themselves to numerical representation, but the structural mapping of one set of observations to another still forms a mathematical model. Unfortunately, the undeniable success of numerically precise models in physics has generated an implication that these models are somehow more "fundamental" with regard to an ultimate understanding of reality than
9 of 25
11/28/2008 9:50 AM
the models used in, say, biology or psychology. However, it is important to realize that even the most successful grand unified theory in physics will be essentially mute when it comes to many of the more macroscopic aspects of our experience. For example, the structure of the Empire State building is entirely compatible with the laws of physics, but those same laws cannot predict the existence of the Empire State building. In the same way, the model of the genetic code described above cannot be reduced to the laws of physics: the genetic code is a result of billions of years of haphazard biological evolution, and while these evolutionary processes are consistent with the laws of physics, they are not necessitated by them. The Wysiwyg hypothesis allows for "fundamental laws" at all levels of experience. There is no a priori reason to expect that one domain will be in some sense more fundamental than all the rest--after all, all such domains are ontologically equivalent insofar as they are all elements of our experience. For example, attempts to force an explanation of qualia in terms of Planck-scale quantum fluctuations in the spacetime continuum seem somewhat misguided. From our experience of brain function, the relevant level of description is that of neurons, their interconnections and their chemical messengers. And it is at this level that we are most likely to find the relevant correlations between qualia and brain mechanisms.
Experimental Support for Wysiwyg
The Wysiwyg postulates seem to clarify many otherwise confusing issues in philosophy and science.
However, is there some empirical set of observations which give the postulates a firmer ground on which to stand? Surprisingly, the answer is a qualified "yes". The Wysiwyg postulates claim that there is no "objective" reality other than experience--nothing behind the scenes pulling the strings or holding everything together. A similar implication can be drawn from the fallout of a paradox first described in 1935 by Einstein and two co-authors, Boris Podolsky and Nathan Rosen. It is therefore referred to as the Einstein-Podolsky-Rosen (EPR) paradox. The EPR paradox was meant to show that the quantum mechanical description of the physical world was not complete. The apparently irreducible randomness at the quantum level of matter greatly annoyed Einstein who felt that "God would not play dice with the Universe." He therefore speculated that there must be "hidden variables" lurking behind quantum measurements which, although they had not yet been observed, would allow us to explain the random behavior of the results once the variables had been identified. The basic setup of the EPR paradox is fairly easy to describe in general terms. Suppose we have two electrons A and B which are initially prepared such that if one of them has spin "up" then the other must have spin "down" and vice versa. Such spins are said to be "correlated" and can actually be produced in the lab. Now imagine that, before we actually measure the spin of either electron, we allow them to fly apart so that they are now separated by a very large distance. If we then measure the spin of electron A and find that it is "up", quantum mechanics must predict that a measurement of the spin of electron B will yield a value of "down". The question becomes: how does electron B "know" that our measurement on electron A resulted in a value of spin up? We can move the two electrons as far apart as we like so that even a light signal could not communicate the result of the measurement on A fast enough for B to be informed of the result. Einstein called this hypothetical result "spooky action at a distance" and concluded that the only way quantum mechanics could handle such a situation was to admit that additional hidden variables were at work which determined the spins of both electrons before they were separated. In other words, what you see is only part of what you get. Due to the technical sophistication required to actually carry out the correlated spin experiment, a resolution of the EPR paradox had to wait another 30 years. In the meantime, an Irish mathematical physicist named
10 of 25
11/28/2008 9:50 AM
John S. Bell took a keen interest in the EPR argument. Bell wondered, what if we accept the EPR hypothesis and assume that there are hidden variables behind the spin experiment? Bell was then able to prove a theorem which showed that for any such hidden variable theory, he could derive an equation (actually, an inequality) which put severe constraints on the types of experimental outcomes such a theory could predict. The inequality derived from Bell's Theorem for the case of correlated electron spins clearly showed that the results predicted by quantum mechanics could not be accounted for by any hidden variable theory. What's more, Bell's work led to the design of actual experiments which could test Bell's inequality in the lab and hence determine whether or not quantum mechanics was correct, despite the EPR paradox. A series of experiments followed, the most famous being that of Alan Aspect and co-workers. These experiments are very similar in principle to the original EPR thought experiment. The results are fairly unequivocal: quantum mechanics is right and Einstein was wrong. Bell's Theorem and the subsequent experimental vindication of quantum mechanics by Aspect and others has been widely regarded as "the most profound discovery in science." To appreciate the full impact of Bell's work, the EPR paradox can be recast so that it pits quantum mechanics against three fundamental assumptions about reality: 1. Deductive logic is valid. 2. No signal can travel faster than light. 3. There is an objective world independent of our experience. Since the quantum mechanical predictions for the results of the Aspect experiment were confirmed, one or more of these assumptions must be wrong. There is little reason to date to reject either deductive logic or the wealth of experimental evidence behind the maximum speed limit imposed by light. This leaves assumption 3 which, if rejected, becomes the equivalent of our second Wysiwyg postulate. In this way, one can see that the Wysiwyg hypothesis is not so far fetched as one might seem--in fact, it appears to find direct experimental confirmation in the physicists' lab.
Wysiwyg versus Mysticism
he Wysiwyg hypothesis puts all the emphasis on experience. Another pattern of thinking which tends to
emphasize experience is "mysticism". No doubt there will be a temptation to equate the Wysiwyg view with the picture of the world described by various mystical schools. However, this would be a mistake. First of all, unlike science, there is no one set of procedures or generally accepted truths about the world which we can lump into a single category called "mysticism". Scientists, by and large, speak the same language the world over. The results found in one lab need to be reproducible in anyone else's lab before the results enter the general database of scientific knowledge. New theories in science must be testable, at least in principle, and any data collected which contradicts the predictions of a theory necessarily invalidates it. It is very difficult to make a living in this day and age pedaling false science--someone will call your bluff in very short order. Mysticism generally begins with a simple process of reflection called meditation. The basic idea is to allow experience simply to "happen", without preconceptions and without intellectual analysis. In of itself, this may be the only way to truly appreciate the essence of experience as a foundation for all that is real. But from this simple beginning, whole schools of mystical thought have sprung forth over the millennia. Countless volumes are written on the insights gained through meditation regarding the true nature of reality and our place in the
11 of 25
11/28/2008 9:50 AM
cosmic order of things. Alas, as testable models of the world, most of these descriptions fall far short of even basic commonsense checks which anyone can perform in their daily lives. Needless to say, without testability, many such ideas are ripe for suggestibility, preconception and autosuggestion. Furthermore, many such descriptions date back thousands of years and seem starkly out of place in today's world. It would be as if we continued to follow Aristotle's model of physics simply because he was considered to be an authority two thousand years ago! As we have seen in previous sections, the Wysiwyg Universe is a Scientific Universe. There seems to be no end to the correlations among the rich and varied experiences each and every one of us enjoys on a daily basis. And these correlations tend to be agreed upon by anyone who cares to look. Should we happen to find new experiences during the course of meditation, these will simply add to our repertoire of experiences. If there are correlations between these experiences and others, time will reveal their structure and they will become generally known amongst a greater number of people. Perhaps the closest parallel one can find between the Wysiwyg hypothesis and mysticism can be summed up by the famous anonymous Zen saying: Before satori (enlightenment), the mountain is a mountain During satori, the mountain is no longer a mountain After satori, the mountain is a mountain again
Objects and Space

(This topic is treated in greater mathematical detail in Appendix A.)
Take a garden variety rock--no really--pick one up, heft it, turn it around in your hand. Feel the coolness of
the stone, the roughness of the surface, its weight against your palm. Perhaps it is colored in some interesting way or just a dull grey. If it has some crystalline structure, it might sparkle in the sunlight. Rotate the rock in your hand and notice the changing visual image. Toss the rock a few feet away and watch it fall through the air to the ground with a thump. What is this thing, this rock? By and large, our experiences seem to come packaged together as objects which are arranged in space. Traditionally, we say that these objects have certain properities such as shape, hardness, color, etc., which we come to know by way of our senses. And the arrangment of these objects in space is defined by their placement and orientation relative to some frame of reference (usually one's body). It is difficult not to think of objects as something "out there" that somehow give rise to our sensations. But from the Wysiwyg point of view, what we call objects are just those sets of experiences which are correlated in a certain way. Of course, it's easy to be swayed by the third person point of view, where objects really are outside the brain. But from this same perspective, brains are (ontologically) no different than the objects they are trying to comprehend. For example, if we are trying to build a computer vision system that can recognize different objects, the computer is obviously just another object. So objects are certainly outside the brain, but they are not outside experience. We can clarify this point by taking the first person point of view. Here the recognition of objects has the flavor of discovery. Imagine for a moment that you are having your first visual experience. All you see is a mosaic of color, shading, maybe some lines or edges. Now suppose you reach toward a smudge in your visual field and grasp something that the rest of us call a "rock". If you now rotate your hand, the smudge that is the
12 of 25
11/28/2008 9:50 AM
rock forms a succession of visual experiences that are related by the structure of rotations in three dimensional space. It is the rotational cohesion of these visual perspectives that defines the rock as bounded and rigid and therefore allows it to "pop out" from the visual background as an object. Note also that it is your action that generates these transformations. If, instead of rotating the rock, you were to move it left and right, you would generate a series of visual experiences that trace out a translation in space rather than a rotation. We are tempted to say that the rotated or translated rock forms a cohesive succession of visual images because it is a rigid object. But from the Wysiwyg perspective, it is precisely the correlated nature of these experiences which enable us to call the rock an object in the first place. If we replace the rock with a walnut, the visual experiences will be different, but the correlations between successive experiences during the rotation or translation will be exactly the same as with the rock. (Mathematically, we say that the succession of visual experiences forms a group; in this case the group of rotations and translations in three dimensional space.) What defines a visual experience as a rigid object is therefore not some property of the object "behind the experience" but the correlations among our changing experience. If you were to grasp a different smudge in your visual field that the rest of us call "water" and you then rotated your hand, the resulting visual experiences would not be of a rigid rotation but rather a "flowing" from your hand to the ground. In this case, you would be unlikely to consider the experience an "object" at all. So powerful is this rotational cohesion that we can see objects in a field of random dots, as long as those dots move in a way consistent with a rotation. Click on the icon below for a demonstration: Click to see an AVI movie (5Mb filesize). Notice how the square disappears as soon as the rotation stops. Physicists and mathematicians have a special word for this kind of transformational structure: it is called covariance. In fact, all meaningful "objects" in physics are defined by their covariant properties with respect to some set of transformations (such as rotations and translations). In other words, the features we observe during our interaction with some phenomenon must co-vary with our actions in a lawful manner before we will consider it an object. Have you ever noticed a spot on your windshield but weren't sure if it was really a spot or some distant object in the sky? And did you notice that the way you could tell them apart was to move your head a little? A spot on the windshield will appear to move one way with a tilt of your head, and a distant object will move another way. The resulting experience immediately disambiguates the two possibilities. But there is more. Not only do covariant transformations define objects, they also give away the structure of the space in which those objects interact. Consider again the rotation or translation of the rock in your hand. The correlated visual experiences define the rock as a rigid and bounded object; but these same transformations define space itself as three dimensional and (locally) Euclidean. Mathematically we say that the succession of visual experiences defines a representation of the rotation and translation group in three dimensions, symbolized as SO(3). In other words, objects and space are two sides of the same correlational relation. Einstein once said that physical objects are not in space; rather, objects are spatially extended. By this he meant that empty space has no meaning. We can now see that the notion of an object as a "thing in itself" is also meaningless: both objects and space are simply special correlations among our experiences. A key point about the Wysiwyg view of objects and space is that these correlational patterns generally arise from our own actions. For example, most objects do not oblige us by spinning on an axis against a fixed background. Instead, we must literally reach out and initiate a change in our experience before the necessary transformations can take place. No doubt most of us take care of these important exploratorations during infancy. It also suggests that what constitutes an object may be different for other species. For example, birds do a lot of flying and not much reaching and grasping. The kinds of transformations induced by flight
13 of 25
11/28/2008 9:50 AM
(optic flow patterns) characterize objects as flat or round, broad or narrow, moving or stationary, coming toward or moving away. A balcony railing to a bird is a perch, not something to keep it from falling off the balcony. The psychologist J.J. Gibson was the first to develop this idea which he called perceptual affordances. [Gibson, J.J. (1966). The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin.] In the field of modern physics, John A. Wheeler has proposed the notion of participators instead of "observers" to account for the fact that the properties of the world seem to depend on the kind of measurements we make; i.e. the way we choose to interact with the system under study. Objects are defined by additional properties which also tend to form correlated bundles. Take again the garden variety rock and this time close your eyes. When held in your hand, the rock has a certain felt shape. If you now rotate the rock within your palm using your fingers, the result would be a collection of tactile experiences, one for each orientation of the rock. These tactile experiences would be correlated in exactly the same manner as those of your earlier visual experiences; both sets of experiences result from a set of motor actions which generate the group of rotations. In this way, the rock is both a visual object and a tactile object with the same correlational structure. This common covariant structure is what "binds" together the separate visual and tactile properties of the rock. For imagine now that you open your eyes while you manipulate the rock in your palm with your fingers. Each orientation of the rock generates both a visual and tactile experience. Individually, these pairs of experiences need not bear any relation to one another; afterall, how can one compare a static visual impression with a single pattern of tactile sensations? However, the two sets of correlated experiences both form a representation of the three dimensional rotation group. What's more, these representations are isomorphic; i.e. given any pair of visual experiences, we can always find a pair of tactile experiences and a rotation such that the same rotation will map one visual experience into the other at the same time that it will map the corresponding tactile experiences into each other. In this way, the two collections of experiences are considered the same "object". But wait, you might object, why are these two groups of separate experiences isomorphic in the first place? Is it not because there is a "thing unto itself", a ding an sich, behind the experiences? According to the Wysiwyg view, this would put the cart before the horse. Any notion of platonic-like objects behind our experience is simply a model of the correlations among our experiences. We will delve into this issue in greater detail in a forthcoming section on Mind and Matter. The Case of Inverted Retinal Images Ever since Johannes Kepler first discovered it in 1604, people have puzzled over a curious fact of our visual system. As it turns out, when the lens of the eye focuses an image of an object onto the retina at the back of the eye, that image is inverted in relation to the orientation of the object and left-right reversed. Why then don't we perceive the world upsidedown and left-right backward?
A rose is a rose in any orientation.
This puzzle stems from a fundamental misunderstanding about perception that continues to cause confusion to this day. A widely held view about object perception is that brains "represent" the world around them. Taken literally, the word "represent" implies a re-presentation, rather like projecting a slide show onto a screen. Needless to say, such an idea suggests some form of agent or "homunculus" inside the head which then views the show. Many researchers realize that any theory that leads to this kind of "head within the
14 of 25
11/28/2008 9:50 AM
head" only pushes the problem back one step and therefore results in an infinite regress without explaining anything. The Wysiwyg viewpoint allows us to avoid this trap by understanding the experience of objects as bundles of covariantly transforming sensations. From the first person point of view, there is no such thing as a retinal image to begin with. No one has ever seen their own retinal image. (Go ahead, try it!) It is therefore irrelevant what its orientation might be relative to the object that produces it. In fact, the retinal image is just the first step in a long series of neural activations that run through the optic nerve, the thalamus and the visual cortex. At the same time, motor inputs from the muscles that control the eyeball, head and body are constantly producing activity in other parts of the brain such as the motor and parietal cortex. During an encounter with an object, all of these systems sweep through a series of activity patterns depending on the relative motion between the object and the observer. As detailed earlier, the result is a covariantly changing series of experiences that define what we mean by an object. It is the entire bundle of such sensations, together with the transformations among them, that corresponds to an object--not some intermediate blip of neural activity at some point along the optic pathway. It would be rather like looking inside the CPU of a computer, viewing the current set of 1's and 0's being processed, and trying to understand the computer program that was running at that time. In short, objects are not "pictures in the head". Instead, they are correlated bundles of experiences. In Appendix A we provide a simple mathematical example of how such covariant bundles can exist in the brain's visual system.
Questions and Answers

Wysiwyg and the Brain
When describing the relationship between experience and the brain, we seem to confront a classic chicken-and-egg problem: What comes first, experience or the brain?
When studying brain activity and its relation to experience, there is a tendency to draw a rather arbitrary
balloon around the head and say: everything inside the skull we will call "internal" and everything outside the skull we will call "external". Most of us go along with this since, after all, brains are very much hidden away inside their owner's heads, unlike, for example, hands or faces. Like most internal organs, we can't see our brain, or hear it, touch it, smell it or--ahem--taste it. So when we are confronted with the problem of, for example, how a brain hears the singing of a bird, we are tempted to say: the source of the sound is in the "external world" and there we find patterns of vibrating air molecules. But the brain operates in the "internal world" and here we find only the activity of neurons. So where exactly is the sound? Is it in the external world or the internal world? And if it is in the internal world, how can the firing of neurons actually be sound? Of course, this is where the whole mind-body problem sneaks in. While this division of the world into what's inside the head versus what's outside the head makes sense from a purely engineering point of view--for example, how could we make a computer recognize the sound of a bird--it has no ontological basis: the brain is just as much an object in the world as are birds and people and air molecules. When we stop to think about it, no one would argue this point, but our long entrenched speaking habits often lead us down an unnecessary dualistic path. The root of the confusion arises from our switching back and forth between a first-person perspective and a third-person perspective. For example, if you are listening to a bird singing on a branch before you, a
15 of 25
11/28/2008 9:50 AM
neuropsychologist might say that the singing does not exist in the "external" world, only in your brain (the "internal world"). What exists in the external world are just patterns of vibrating air molecules which emanate outward from the bird. These vibrations then impact upon the membranes of your inner ear, which in turn stimulate your auditory nerves, and ultimately the parts of your brain responsible for sound. For this reason, your brain is said to construct the experience of sound from the raw sense data of vibrating air. Of course, the psychologist doesn't actually have access to your experience, so the best he can do is say that your brain constructs a neural representation of a bird singing. This is the third-person perspective--the neurospsychologist is the third individual viewing the interaction between two other individuals, you, or rather your brain, and the bird. The first-person perspective is simply what you experience. From this perspective, the experience of the bird song really is over there coming from the bird sitting on the branch--not somewhere in your brain. In fact, you have no sensation of your brain activity at all. Indeed, if you had been raised by wolves rather than in a human society, you probably wouldn't even know that you had a brain inside your head. But you would presumably still have the experience of the bird singing over there on the branch. No matter how much physics or neurobiology someone throws at you, from your point of view, the bird is experienced as over there on the branch, not inside your head. Let's now bring these two perspectives together using a simple thought experiment. There you are, sitting in the garden, experiencing a bird singing on a branch. To better understand the relation between the sound of the bird and the activity of your brain, you attach a functional MRI device to your head. The device has a large color display which shows in fine detail the activity of the various parts of your brain. In this way you have taken your brain outside of your head without having to cut open your skull. As a result, your brain can now be seen for what it is: another object of your experience just like the bird. As the bird continues to sing, the MRI display reveals that various parts of your auditory cortex light up, along with regions of your association and visual cortex, and parts of the language areas in your left hemisphere (if you are right handed). There is even significant activity in your medial forebrain bundle which reflects the fact that you are enjoying the sound! You now have both first-person and third-person perspectives on your situation. You have the experience of the bird singing, and you have all the data you need to understand how the brain constructs the experience from the pattern of vibrating air striking your ear. Or do you? The MRI display appears to you as a visual experience which is very nicely correlated with the auditory experience of the singing bird. In fact, the correlation is of the very same nature as you might notice if you viewed a closeup of the motion of the bird's beak while it made the sound. In both cases you see a correlation between the sound experience and a visual experience. And that's it. Just correlations among experiences--mathematical structures which allow us to predict one experience from another. No ontological distinction between "internal" and "external" worlds. No need to wonder how the "physical" brain and environment give rise to our "mental" experience. In the Wysiwyg view, there is only experience. As an object in the world, the brain is a collection of correlated experiences--the experiences we have when studying the organ inside peoples' head. This structured set of experiences which we call the brain can in turn be correlated with other experiences we have such as the singing of birds, the color of the sky, sensations of hot and cold, hunger and thirst, etc. In this way we can see that experience is not a construction of the brain; rather, the brain is a structure in our experience. Wysiwyg and Occam's Razor
Doesn't the Wysiwyg hypothesis violate Occam's Razor?
16 of 25
11/28/2008 9:50 AM
When first presented with the Wysiwyg hypothesis, one might object to the second postulate as follows: "I
can accept that experience is the only reality for an observer. But how do we account for all the myriad correlations among our experiences without reference to an 'external world?' In other words, how can experience be all that there is?" First of all, we must be clear about what is meant by the phrase "external world". As we saw in the previous question on Wysiwyg and the Brain, we sometimes speak as if "external world" simply means everything "outside the brain". But the brain is clearly in the "external world" as much as any other object and certainly cannot be given any special ontological status a priori. So by "external world" we must mean "outside experience". Not just outside current experience, but outside all experience, even in principle. (Don't forget that observations we make using telescopes or microscopes or particle accelerators, etc., are all still part of our experience.) The Wysiwyg hypothesis postulates that there is no reality other than experience. The objection raised above can now be rephrased as: "How do we account for all the myriad correlations among our experiences without reference to anything outside experience?" When evaluating competing models of some domain of our experience, scientists often use a principle called Occam's Razor. William of Occam was a medieval philosopher who first put forth the idea that, given a choice between two explanations of some phenomenon, we should choose the simplest--the explanation which requires the fewest assumptions. For example, if you assume that the Sun and the rest of the solar system revolve about the Earth, as did the ancient Greek astronomer Ptolemy, then you need to also assume that the other planets follow very peculiar orbits in order to account for the observed motions through the night sky. On the other hand, if you assume that the Earth and the other planets revolve about the Sun, as Copernicus argued, then no further assumptions are required to account for the observed motions. Both the Ptolemic model and the Copernican model can explain the observed motions of the planets; however, the Copernican model makes far fewer assumptions and is therefore deemed preferrable. Readers familiar with Occam's Razor might argue that the second Wysiwyg postulate requires an infinite number of assumptions; namely, one assumption for every observed correlation among experiences. Wouldn't it be simpler to postulate the existence of variables or entities beyond experience to account for these correlations? If the number of these variables or entities is fewer than the number of correlated experiences, then this non-Wysiwyg view might be considered simpler and therefore preferrable to the assumption that there is no reality other than experience. There are a number of problems with this argument, some of which have already been spelled out in the earlier parts of this document. First of all, Wysiwyg makes no assumptions regarding correlations among experiences; they are simply observed. If we wish to model these correlations for a particular domain of experience, then we have to choose among the large number of all possible models (mathematical structures) which include the observed correlations as a subset. And it is at this stage that Occam's Razor can play a role in choosing among different models. Occam's Razor was not intended to be applied to the set of all experiences--or to any experience for that matter--only to models of a subset of our experiences. And even if we could apply the principle to the Wysiwyg postulates, which sounds simpler: that there is no reality other than experience; or, that in addition to everything we experience (including what we observe through microscopes and telescopes and infrared glasses etc.), there is a whole "world beyond experience" which we cannot observe but which somehow holds it all together? In this respect, Occam's Razor seems to actually favor Wysiwyg. But there is an even more compelling reason to let go of the idea of a "world outside experience": it is a mistake to assume that by simply referring to a world outside experience, we have then accounted for the observed correlations among experiences. For one could just as well ask: how then do we account for the structure in this other world beyond experience? We can best illustrate this point with a specific example.
17 of 25
11/28/2008 9:50 AM
If we regularly experience the sight of lightning followed by the sound of thunder, we could sum up our experience with the phrase (simple verbal model) "thunder follows lightning". Suppose we then ask the question: "But why does thunder always follow lightning?" If we are simply making things up as we go along we might answer: "Thunder is the sound of Thor grumbling whenever he is blinded by a flash of lightning." Of course, then we have to explain who Thor is, why he doesn't simply cover his eyes during a thunder storm, and why he never seems to appear in public. But if we are doing science, the first thing we do is broaden our set of experiences by looking for intermediate phenomena (experiences) between the sight of lightning and the sound of thunder. So we set up some lightning rods and some recording devices and wait for the next storm. When the data are collected, we notice the following chain of experiences: electrical surge, bright light, high temperatures, low pressure near the electrical surge, high pressure radiating out from the region of low pressure, loud sound. We put these together verbally and say, "An electrical surge through the air leads to a flash of light and a hot core of air around the surge. This hot air expands (low pressure), compressing the surrounding air (high pressure). When the high pressure air hits our ears we hear a loud sound." Notice that at no point in this explanation have we appealed to entities or processes beyond our experience. But by expanding our collection of observations, we are able to produce a more detailed model of why thunder follows lightning. Of course, one can play the "why" game to any level of detail you like: Why is there an electrical surge? Why does hot air expand? Why is there an atmosphere at all? Why does the Earth exist? And so on. In the end, you will always end up with a set of correlated experiences. What's more, the resulting explanation in terms of these correlations is just what we call a scientific model (physics in this example). If you are studying a reductive phenomenon like lightning and thunder, then you could conceivably collect a (nearly) "complete set" of predictive pairings which accounts for (almost) every "why" question in the explanatory chain. Some of the final questions might go something like: Why are there electrical charges? Why are there electrons? Why is there matter at all? Why was there a Big Bang? Why did Nothing suddenly become Everything? And then you are stumped, because the correlation {Nothing, Everything} is a pairing that makes no sense from a scientific point of view and cannot be further reduced. This is all just a long winded way of saying that the set of experiences fall into predictive pairings because that is the way the world is. If this were not the case, we wouldn't be having this discussion. As far as we can tell, science never answers the ultimate "why" questions. Why is the experience of "red" red rather than blue? Why does space have three dimensions instead of twenty? Why did the Universe spring from Nothing? Why does anything exist at all? The best science can do is to model the predictive pairings found among our experiences so that we can in fact use one to predict another. If you want to know the nature of your experiences "directly", then you should study your experiences themselves rather than a model of their correlations. The process of studying your experiences directly is called meditation. Wysiwyg and Idealism
Isn't the Wysiwyg hypothesis the same as Idealism?
The philosophical school called Idealism dates back to Bishop George Berkeley, an Irish philosopher and Anglican clergyman living in the 1700's. While the Wysiwyg hypothesis may seem similar to Idealism on the surface, the two approaches are actually quite distinct. In particular, Berkeley's Idealism is really a kind of theological epistemology, as many thinkers at the time were trying to reconcile the new empiricism of John Locke with their own religious convictions. First of all, the Wysiwyg hypothesis is contained entirely within the two postulates which are repeated here as a reminder:
18 of 25
11/28/2008 9:50 AM
Experience is the only reality for an observer. There is no reality other than experience. Berkeley used the word "idea" where we use "experience" or "sensation"; hence, the term "idealism". The modern connotations of the word "idea" suggests something beyond basic experience, such as "concept" or "thought." However, at the time of Berkeley's writing, "idea" denoted something much simpler and we can safely substitute "experience" in its place. Berkeley assumed that each of us has a "mind" and argued that "matter" cannot be conceived to exist independent of the mind. In essence, Berkeley sets himself up for a kind of implicit dualism: mind versus matter. However, according to Berkeley, matter can only exist if it is perceived by the mind. Furthermore, to account for the apparent ability of things to exist even when one is not observing them (which apparently contradicts his original premise), Berkeley held that all things must be constantly perceived by God. (Remember, Berkeley was a member of the clergy as well as philosopher.) By contrast, Wysiwyg simply says that each of us has experiences and that there is no reality other than experience. "Matter" and "mind" are labels we have for certain aspects of our experience. The notion of objects outside our experience is part of our model of the world. It permits us to summarize our experience and make predictions about other experiences. However, there is no reality outside experience. To the non-philosopher (which includes the author), Idealism and Wysiwyg will probably seem like birds of a different feather. But just in case the distinction is still unclear, we can summarize the differences as follows: Idealism Reality consists of mind and matter but... Wysiwyg Reality is experience. "Matter" and "mind" are labels for certain aspects of our experience. There is no reality other than experience. "Inside" and "outside" are also labels for certain aspects of our experience. The notion of objects outside our experience is part of our model of the world. Models of reality should not be mistaken for reality itself.
...Matter does not exist outside of the mind. Objects exists even in the absence of observers because they are continually perceived by God.
Ever since Berkeley put forth is ideas, philosophers have been somewhat annoyed to find that Idealism is very hard if not impossible to refute. What bothers them most is that an "external world" clearly seems to exist "outside of the mind". Dr. Samuel Johnson was a prominent physician and contemporary of Berkeley. In James Boswell's biography of Johnson, Boswell recounts: After we came out of the church, we stood talking for some time together of Bishop Berkeley's ingenious sophistry to prove the nonexistence of matter, and that every thing in the universe is merely ideal. I observed, that though we are satisfied his doctrine is not true, it is impossible to refute it. I never shall forget the alacrity with which Johnson answered, striking his foot with mighty force against a large stone, till he rebounded from it -- "I refute it thus." A refutation of Idealism, maybe. But what could be a more poignant confirmation of Wysiwyg than a stubbed toe! Avoiding Solipsism
The Wysiwyg postulates seem to imply that the Universe is only what I experience.
19 of 25
11/28/2008 9:50 AM
Doesn't this lead to solipsism?
Philosophers are generally skeptical of anything that smacks of solipsism, which is the notion that the only thing that is real is oneself and one's own experience. It is easy to caricature this position in such a way that the solipsist seems a least mildly disconnected from reality. Believing that the universe is the creation of one's own thoughts is such an example and suggests a rather delusional disposition. On the surface, the Wysiwyg hypothesis sounds like solipsism. But then solipsism, when stripped of its misplaced psychological connotations, is nearly a truism which few would dispute. For example, from a third person point of view, it is clear that everything a person knows derives from the activity in their brain. In a strict sense, no one ever has direct contact with the world outside their brain. All we know is the product of our own neurons. Nonetheless, no one would deny that the ultimate cause of this neural activity includes events outside the brain. So while it is true that a person's reality is circumscribed by the function of their brain, it is not logical to conclude that brain function creates the world it perceives. Similarly, the Wysiwyg hypothesis posits that there is no reality other than experience. But this does not imply that you or I create the universe of experience. At any given moment, our experience is infinitesimal compared to the vastness of all possible experience. Furthermore, we have very little control over what experience comes next. While we are immersed in the world of experience, we do not simply make it up. It would be as if the tail on an elephant, being able to see only the elephant to which it is attached, were to conclude that it created and controlled the beast lumbering about before it. Finally, one must carefully distinguish between experience and knowledge. While each of us seems privy only to our own experience, knowledge can be shared with others. What we acquire in this manner is a shared model of the world. And it is this shared model of the world which can be used to dispel any solipsistic delusions in the psychological sense. For example, an individual who has never experienced snow might claim: "Snow does not exist." To convince them otherwise, we might point out all the references to snow in books and on the Internet. Then we might enlist some of our friends to share their own personal experience with snow. If this is still not enough, we might take our friend to Canada in February and pelt him with snowballs. The knowledge that "snow is cold" is not the same as experiencing snow itself. And as far as we can tell, there is no way that one individual can have someone else's experience. So in the end, if you really want to know something about the world for yourself, you need to experience it yourself. All the knowledge in the world about chocolate cake won't taste as good as the real thing. But it will come in very handy if you want to bake a cake for yourself. He Said, She Said
How do we explain the ability of two or more individuals to agree on the identification of an object without reference to a world beyond experience?
Suppose you and I meet for the first time. We might not even speak the same language. Assume that I have an experience, B, to which I assign the verbal label, Blue. At the same time, suppose you are having the experience, b, and that my verbal utterance generates in you the experience, blah. Then for me, B ~ Blue, and for you b ~ blah. If at some later time I utter Blue, you will experience blah. If at the time you are also experiencing b, then by your correlation, b ~ blah, you will agree with my utterance. If you are not experiencing b, then you will disagree. So by using a verbal label when I have the experience B, I can induce in you a corresponding label for your experience b. Note that as far as we know, there is no way either of us can directly compare our experiences, B and b. It is only through the intermediate experience of language that we can agree on the correlations of our other experiences. Of course "language" here simply means any method by which we can signal each other. One need only try to understand two people talking in an unfamiliar foreign language to appreciate the arbitrariness of assigning labels to experience. Yet once the labels are habitually assigned, two individuals
20 of 25
11/28/2008 9:50 AM
can (generally) convey the sense of their separate experiences to each other.
Isn't Wysiwyg just another model?
Actually, Wysiwyg is just WYSIWYG! :-) Just remember that the finger pointing at the Moon is not the Moon.
Appendix A
In the section on mathematics and experience we discussed how all branches of mathematics begin with a set of objects and a structure defined on that set. We then defined a structure on a set as a collection of pairings of elements from that set. This definition allowed us to see how mathematics provides a natural way to model the correlations among experiences. At a very basic level, experiences seem to come packaged together as objects in the world. We now show how this comes about. Rigid Objects as Group Representations Take the example of a rock. Suppose you have never experienced a rock before and don't even have a word for it in your vocabulary. We'll assume all you know right now of the rock is that there is a smudge of something or other in your visual field. Let us call this smudge X. Now suppose you reach out in the direction of X, and having picked up something that might be X, you rotate it with your hand. This interaction with X generates two sets of experiences: a set of visual experiences, V = {v1, v2, v3, ...}, and a set of motor experiences (moving your hand), M = {m1, m2, m3, ...} . Since your motor actions are in step with your visual experiences, your actions generate the structure S = {{m1,v1}, {m2,v2}, {m3,v3},...}. This structure is not particularly "object like" since all you have done is paired particular positions of your hand during the rotation with the particular visual experience occurring at each orientation. If you had begun with your hand or X in a different orientation, the structure S would be completely different. Fortunately, the collection of rotations itself forms a structure of a very precise nature called a group. The elements of a group satisfy the following pairing relationships. If {a,b} and {b,c} are pairings in the group, then so is the pairing {a,c}. This property is called "composition". For example, if we rotate X around a fixed axis first through 5 degrees ({a,b}) and then through 10 degrees ({b,c}), then the resulting visual experience is the same as if we had rotated X through 15 degrees from the beginning; i.e. {a,c}. A second property of a group stipulates that there is one element which acts as an identity. If we symbolize the identity element by i, then any pairing {a,i} is the equivalent of just {a} alone. For rotations, the identity element is a rotation through zero degrees. From a visual point of view, the identity operation is the equivalent to no rotation at all. The final property of a group guarantees that for every element a, there is an inverse, a-1, such that the pairing {a,a-1} is the equivalent of the identity element. If we rotate X through degrees in one direction and then reverse the rotation through - degrees, then we end up with the same visual experience that we started with. The fact that rotations form a group allows us to define our rock as a collection of visual experiences which are related by the group of rotations in three dimensional space. (We could include translations too for completeness but we'll restrict ourselves to rotations for simplicity.) The generator of this group is the action of our hand during the manipulation of the rock. The set of visual experiences resulting from our manipulation of the rock are therefore not arbitrarily paired with each proprioceptive motor experience but are in fact bound together by the structure of the rotation group. Given any visual experience of the rock, additional visual experiences can be predicted by applying small rotations to the original experience. The
21 of 25
11/28/2008 9:50 AM
group structure of the set of rotations is therefore the "glue" which binds together the various visual experiences and allows the rock to "pop out" from the rest of the visual field. It is important to note that all we have done while exploring the rock is to move our hand. The result was a set of pairings between motor experiences and visual experiences. These pairings naturally satisfy the properties of a group--we did not impose this structure arbitrarily. We can therefore define a visual object as a collection of visual experiences which are related by the group of rotations (and translations). In mathematical symbols, our rock is defined by the set of visual experiences, V = {v1, v2, v3, ...} and the structure R = {{v1, v2}, {v3, v4}, {v5, v6}, ...} where, for any two visual experiences, vj and vk, there is an element, r, of the group of rotations (and translations) such that vj = r(vk). Note how we can now drop the motor experiences from our definition of the rock: the group structure is embedded in the relationship amongst visual experiences themselves. Recognizing the group structure behind the objects of our experiences enables us to understand how experiences in one modality (e.g. vision) can correspond to experiences in another modality (e.g. touch). For imagine that while manipulating our rock, we kept our eyes closed and, instead of rotating our whole hand, we simply turned the rock about in our palm using our fingers. The result would be a collection of tactile experiences for each orientation of the rock. These tactile experiences would have exactly the same group structure (rotations and translations) as those of our earlier visual experiences, since both resulted from a set of motor actions which themselves generate this group. In this way, the rock is both a visual object and a tactile object with the same structure. In fact, the two group representations afforded by the visual rock and the tactile rock are isomorphic. Two representations of a group are isomorphic if (a) there is a one-to-one mapping (correspondence) between the elements of each group and (b) the same mapping preserves group transformations. For example, if we label two visual perspectives of an object as vj and vk, while the tactile sensations for these same two orientations are tj and tk, then if vj = r(vk) for some rotation r, then tj = r(tk) for the same r. Similarly, given any one pair of corresponding visual and tactile experiences, {v1, t1}, we can (for small rotations or translations), transform each member of the pair by the relevant group operation and predict the new pairing {v2, t2} even though we may never have explicitly experienced this combination in the past. Objects and Brain Activity Neurophysiologists know that visual experiences correlate with the activity of neurons in the brain. In the previous section, we saw that the basic objects of our world are sets of experiences bound together by the group of rigid transformations (rotations and translations) in space. What aspects of brain structure or activity might correspond to such objects? Consider a collection of neurons in the brain whose activities are correlated with some aspect of visual experience such as the pattern of lightness and darkness in the visual field. We can represent a particular pattern of activity across the set of these neurons with an array of values v = [v1 v2 v3 ... vN] where the value vj is the activity of the jth neuron in the collection. Each pattern of lightness and darkness in the visual field is correlated with one of these activity arrays. Note that the dimensionality of the activity space simply reflects the number of neurons under consideration. It has nothing to do with the 3-dimensional space in which the objects are viewed. For example, the N visual neurons might be sensitive to the first N components of a Fourier decomposition of the original visual stimulus. A rotation of an object in the visual field will generate a succession of activity patterns over these N neurons. In symbols, if vi is the initial pattern of activity and vr is the pattern of activity after a rotation R, then vr = R(vi). In the meantime, the neurons in our network are interconnected amongst themselves via synapses that allow activity in one neuron to stimulate or inhibit activity in other neurons connected to it. Given any initial
22 of 25
11/28/2008 9:50 AM
pattern of activity, vt, at time t, the activity will evolve in the next instant to a pattern, vt+1, that depends on the strengths of the interconnections. If we have N neurons in our collection, then there are NxN possible connections between them. The values of these connection strengths therefore form an NxN matrix. It is easy to see that the transformation of the activity pattern in the N neurons from one moment to the next is given by multiplying the current array of activity values by this connection matrix. For example, if neuron v1 is connected strongly to neurons v2 and v3, then if v1 is quiet initially but v2 and v3 are moderately active, then in the next instant, v1 will become active due to the activity in v2 and v3 multiplied by the synaptic strengths between them and v1. If we let S stand for this matrix of synaptic connection strengths, then vt+1 = S vt. It can be shown that this kind of matrix difference equation has a solution of the form: v(t) = exp(k t S) v(0) where exp is the matrix exponential operator, v(0) is the pattern of activity at some arbitrary starting time t = 0, and k is a constant of proportionality. This equation has a simple interpretation if S is a matrix representing a rotation about at fixed axis in space. In this case, the activity in the visual cells will trace out a series of values that mimics a rotation of the original object about the fixed axis coded by S. The speed of rotation is given by the proportionality constant, k. Such a result is consistent with experiments done with human subjects who are asked to compare visual objects that differ only in orientation. In these experiments, subjects seem to mentally rotate objects at a fixed speed so that greater angular rotations take a longer period of time to complete. (See for example Shepard and Metzler, 1971). If the neural assembly is to process rotations of rigid objects, then the connection strengths must be modified to mirror the actions of the rotations, R. Mathematicians would say that the synaptic matrices must form a representation of the rotation group relative to the visual activation space. The careful reader might be wondering at this point how the NxN matrix of interconnections among our visual neurons could possibly encode representations of all possible rotations. After all, there are an infinite number of fixed axes in space about which objects can rotate. The answer lies in a subtle but powerful property of the rotation group in three dimensions. First of all, a finite rotation through some number of degrees about a fixed axis in space can be realized by applying an infinitesimally small rotation about the same axis over and over again. We call such an infinitesimal action a generator of the rotation. When we narrow our attention down to this microscopic level, it turns out that one only needs three such generators to produce a finite rotation about any axis. These generators are known as a basis for the Lie algebra associated with the three dimension rotation group SO(3). As an algebra, we can add and subtract these generators to produce new generators. In symbols, if L is the generator for any given rotation about a fixed axis, then we can always find three coefficients, a, b an c such that: L = a L1 + b L2 + c L3 where L1, L2 and L3 are three basis generators. Furthermore, it can be shown that to produce a finite rotation from a generator, we apply the exponential operator to the generator. In symbols, if R() is a rotation about some axis through angle , and L is the generator of that rotation, then: R() = exp( L) We immediately see the similarity between rotational generators and the time course of activity in our neural network. This enables us to conclude that our synaptic connections need only represent three different rotational generators since any other rotation can be generated from a combination of these three. Let us summarize: the visual experience of a rotating object corresponds to a continuously changing pattern of activity in a network of neurons sensitive to visual input. If the neural network is to recognize this changing visual pattern as a rotation of a single object rather than a random series of images, it must somehow
23 of 25
11/28/2008 9:50 AM
"abstract away" the rotation. As we have seen, this is much easier than one might think. For the entire rotation group in three dimensional space can be generated by just three infinitesimal rotations. Furthermore, these three generators can be easily encoded in the synaptic connection strengths between neurons. Finally, the operation that maps small rotations to finite rotations about an arbitrary axis in space is the same operation that governs the time course of neural activity from one moment to the next. For a more in depth treatment of the analysis presented here, see Goebel (1990), The Mathematics of Mental Rotations, Journal of Mathematical Psychology, 34:435-444. A Concrete Example in Flatland Let us use a simplified two-dimensional model where our visual objects are restricted to a single plane. In such a world, rotations can only take place around a single axis (the axis perpendicular to the plane), so our rotation group is only one-dimensional. (We are ignoring translations for the sake of simplicity although they could be handled in the same way.) Suppose we have just two neurons in our neural network which code the x-y coordinates of an arbitrary fixed point on an object. For any given orientation of the object, the activity of our two neurons will code the position of this point. These values form a two-valued array v = [vx vy]. If we rotate the object ever so slightly about the origin through an angle d, the neurons will take on new activity values v' = [v'x v'y]. Since v' will differ only a little from v, we can write the transformation as follows: v' = (1 + d L) v where 1 is the identity matrix and L is the generator of the rotation. In words we say that v' is basically the same as v (hence the identity matrix) plus a small "twitch" of magnitude d caused by the generator matrix L. Our task is to find the form of the generator matrix L. We can pin it down by studying its behavior at two special points: [1 0] along the x-axis and [1 0] along the y-axis. The point [1 0] will rotate to a new location given approximately by [1 d] since the rotation will leave the x coordinate essentially unchanged and will move the y coordinate about the same distance as the arc length of the rotation, which is d. Similarly, the point [0 1] will be moved to approximately [-d 1] using the same logic. Using these two results, we can solve for L in the equation above and we find that:
L = | 0 -1 | | 1 0 |
This simple matrix generates the two-dimensional rotation group. If the 2x2 matrix of synaptic connection strengths between our two neurons take on these values, then any initial pattern of activity in the two cells will transform over time in a way that mirrors a rotation of the original pattern. Indeed, one can show that exponentiation of this matrix leads to a finite rotation; i.e.,
exp ( | 0 -1 |) | 1 0 | = | cos() -sin() | | sin() cos() | = R()
which is the matrix for finite rotations through any angle . Learning Lie Generators in Neural Nets It has been shown by Rao and Ruderman (1999) that the Lie generators for more complicated rotations and in larger neural networks can be readily found by a simple learning algorithm (gradient descent). The result is that the synapses among neurons come to encode the generators for rotations. If an object is then presented to the network in a new orientation, the pattern of activity that unfolds naturally over time as a result of the network's interconnections is equivalent to "mentally rotating" the object. It is important to point out that the
24 of 25
11/28/2008 9:50 AM
encoding of the rotation generators does not depend on the particular object(s) used during training. Indeed, the generators encoded in the synapses do not correspond to any property intrinsic to the objects themselves, but to the abstract group structure of the way these objects transform when we manipulate them. It is the covariant behavior of changing patterns of visual experience that signifies the presence of fixed objects.
Acknowledgments
Many thanks to Jeffrey A. Gray whose penetrating discussions on the nature of consciousness got the whole ball rolling for this document.
References
Harrison, David M. (1999), Bell's Theorem Hayes, Brian (1998), The Invention of the Genetic Code Whitaker, Andrew (1998), John Bell and the Most Profound Discovery of Science Document Version 1.2, April 2004 Copyright 2002-2004 Simon Free Permission is granted to copy, distribute and/or modify this document under the terms of the Open Content License.
25 of 25
11/28/2008 9:50 AM

The Wysiwyg Universe - by Simon Free 112808

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

The Wysiwyg Universe - by Simon Free 112808

Загружено:

Авторское право:

Доступные форматы