!!!!FeelSpace Final Report

feelSpace
Report of a Study Project
Alper Aik c Isabel Dombrowe Tobias Kringe Robert Mrtin a
Boris Bernhard
Christine Carl
Christian Honey Christiane Kabisch Lina Jansen Christopher Lrken o Hannes Saal Moritz Stefaner Christian Stel o
Saskia Nagel
May 2005
Institute of Cognitive Science Department of Neurobiopsychology Supervisor: Prof. Dr. Peter Knig o
Contents
1 The 1.1 1.2 1.3 1.4 1.5 1.6
feelSpace Study Project Introduction . . . . . . . Theoretical Background Hardware . . . . . . . . Experimental Design . . Results . . . . . . . . . . Conclusions . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
1 1 1 2 3 5 6 10 10 10 12 12 12 12 14 14 14 16 19 19 20 21 22 24 26 30 30 33 34 34
A Organization, Documentation, and Public Relations A.1 Overview . . . . . . . . . . . . . . . . . . . . . . A.2 Organization . . . . . . . . . . . . . . . . . . . . A.3 Documentation . . . . . . . . . . . . . . . . . . . A.4 PR Activities . . . . . . . . . . . . . . . . . . . . A.4.1 Corporate Design . . . . . . . . . . . . . . A.4.2 Ethics Commission . . . . . . . . . . . . . A.4.3 Patent . . . . . . . . . . . . . . . . . . . . A.4.4 Fundraising . . . . . . . . . . . . . . . . . A.4.5 Public Presentation . . . . . . . . . . . . A.5 Conclusions . . . . . . . . . . . . . . . . . . . . . B Sensory Substitution B.1 Introduction . . . . . . . . . . . . . . . . . . . . B.2 Sensory Substitution devices and experiments B.2.1 TVSS . . . . . . . . . . . . . . . . . . . B.2.2 AVSS . . . . . . . . . . . . . . . . . . . B.3 How does sensory substitution work? . . . . . . B.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .
C Neural Plasticity C.1 Introduction . . . . . . . . . . . . . . . . . . . . . C.1.1 Perception and action are intermingled . . C.1.2 A new model of neural plasticity . . . . . C.1.3 Test of new model - the feelSpace project
feelSpace
D Magnetic Sense in Animals D.1 Introduction . . . . . . . . . . . . . . . D.2 The Earth Magnetic Field . . . . . . . D.3 The Map and Compass Hypothesis . . D.4 Magnetoreceptor Mechanisms . . . . . D.5 Testing for a Magnetic Sense . . . . . D.6 Case Study: Navigation in Sea Turtles D.7 A Human Sixth Sense? . . . . . . . . . E Augmented Reality E.1 Introduction . . . . . . . . . . . . . E.2 A Denition of Augmented Reality E.3 Technologies . . . . . . . . . . . . . E.4 Problems . . . . . . . . . . . . . . E.5 Compasses and Augmented Reality E.6 Vibrators and Augmented Reality E.7 Implications of these studies . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40 40 40 41 42 43 45 46 51 51 51 52 54 54 56 58 63 64 64 64 64 64 65 67 69 69 69 70 72 73 73 73 74 77 79 80 80 82 84
F Hardware F.1 The Compass . . . . . . . . . . . . . . . . . . . . . F.1.1 Introduction . . . . . . . . . . . . . . . . . F.1.2 Literature Search . . . . . . . . . . . . . . . F.1.3 GPS . . . . . . . . . . . . . . . . . . . . . . F.1.4 Digital Magnetic Compasses . . . . . . . . F.1.5 The Candidates . . . . . . . . . . . . . . . . F.1.6 Testing the PNI Compass . . . . . . . . . . F.1.7 The Decision . . . . . . . . . . . . . . . . . F.2 Vibrators . . . . . . . . . . . . . . . . . . . . . . . F.2.1 Criteria for Evaluating Stimulation Devices F.2.2 Overview of Tactile Feedback Devices . . . F.2.3 Sanko Electric 1e120 . . . . . . . . . . . . . F.3 Belt-Logic . . . . . . . . . . . . . . . . . . . . . . . F.3.1 Requirements . . . . . . . . . . . . . . . . . F.3.2 Basic Design Issues . . . . . . . . . . . . . . F.3.3 Hardware Design . . . . . . . . . . . . . . . F.3.4 Software Design . . . . . . . . . . . . . . . F.3.5 Power Supply . . . . . . . . . . . . . . . . . F.3.6 Enclosure . . . . . . . . . . . . . . . . . . . F.3.7 PC interface . . . . . . . . . . . . . . . . . F.4 Belt . . . . . . . . . . . . . . . . . . . . . . . . . . F.5 Specications and Final Device . . . . . . . . . . .
University of Osnabruck
ii
feelSpace
G Experimental Design G.1 Hypotheses . . . . . . . . . G.2 Experimental setup . . . . . G.2.1 Training . . . . . . . G.2.2 Experimental groups G.2.3 Test conditions . . . G.3 Experiments . . . . . . . . .
. . . . . .
. . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . .
89 89 90 91 91 92 93 95 95 95 97 99
H Orientation in Natural Environment H.1 Introduction . . . . . . . . . . . H.2 Methods . . . . . . . . . . . . . H.3 Results . . . . . . . . . . . . . . H.4 Discussion . . . . . . . . . . . . I
Orientation in Enclosed Environment I.1 Introduction . . . . . . . . . . . . I.2 Methods . . . . . . . . . . . . . . I.2.1 Subjects . . . . . . . . . . I.2.2 Apparatus . . . . . . . . . I.2.3 Procedure . . . . . . . . . I.3 Results and Discussion . . . . . .
- Blue . . . . . . . . . . . . . . . . . . . . . . . .
Velvet . . . . . . . . . . . . . . . . . . . . . . . .
Arena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
101 . 101 . 102 . 102 . 102 . 103 . 105 . . . . . . . . . . . . . . . . . . 108 108 109 111 113 114 115 115 119 120 120 125 125 125 126 129 131 131 132 137
J Navigation in Virtual Environment J.1 Motivation . . . . . . . . . . . . . . . . . . . . J.2 System Design . . . . . . . . . . . . . . . . . . J.3 Methods . . . . . . . . . . . . . . . . . . . . . . J.3.1 Implementation Details . . . . . . . . . J.3.2 Analysis of data . . . . . . . . . . . . . J.4 Results . . . . . . . . . . . . . . . . . . . . . . . J.4.1 Analysis of performance in time . . . . . J.4.2 Analysis of performance in path-length . J.4.3 Summary of Results . . . . . . . . . . . J.5 Discussion . . . . . . . . . . . . . . . . . . . . . K Physiological Tests K.1 Posturography . K.1.1 Methods . K.1.2 Results . K.1.3 Discussion K.2 Nystagmography K.2.1 Methods . K.2.2 Results . K.2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iii
feelSpace
L Subjective Report L.1 Introduction . . . . . . . . . . . . . L.2 Motivation . . . . . . . . . . . . . L.3 Methods . . . . . . . . . . . . . . . L.3.1 Diary . . . . . . . . . . . . L.3.2 Outdoor Training Questions L.3.3 Interview . . . . . . . . . . L.3.4 Evaluation . . . . . . . . . L.4 Results . . . . . . . . . . . . . . . . M Acknowledgements
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
140 . 140 . 140 . 141 . 141 . 141 . 141 . 143 . 143 146
iv
1 The feelSpace Study Project

1.1 Introduction
The aim of the feelSpace project was to investigate the eects of long-term stimulation with orientation information on humans. In order to do this, we constructed a belt which enables its user to feel his orientation in space via vibrotactile stimulation. This belt is equipped with a set of vibrators controlled by an electronic compass: it is always the element pointing north which is slightly vibrating. That way, the person wearing the belt is provided with permanent input about his heading relative to the earths magnetic eld. In humans, there is no natural sensory organ providing this kind of information. The central questions were concerned about how environmental information, not yet provided by an existing sensory organ, would inuence the human nervous system. Would this information be utilizable? If so, would it take conscious eort to use it, or would it be -somehow- integrated, to allow for its sub-cognitive use? If the latter be the case, what would be the nature of integration? What would be the subjective quality of it? Would it be that of a tactile stimulation or something else, yet unperceived? Would we have even have created created a new modality after providing access to a yet unknown domain of sensory information and its hypothetical qualitative novelties?
1.2 Theoretical Background

These big questions were not posed out of the blue. They were posed on the basis of a theoretical framework of challenging implications, formulated to explain astonishing ndings in experimental sensory prosthetics. These ndings were made in the early 1970s, when medical doctor Paul Bach-y-Rita pioneered the eld of sensory substitution by building and testing the rst sensory substitution device. These devices substitute a sense by providing a person with the necessary information through an other modality (e.g. in case of blindness dierent than the eye) [3, 1, 2]. One sensory substitution device transformed images captured by a video-camera into vibrotactile information transmitted to the back of the user, thus providing visual information via the somatosensory system. The blind subjects using this sensory substitution device were able to use the visual information in a purposeful and goal-directed manner. Furthermore, the subjective quality of this articial visual perception was dierent from anything the blind
feelSpace
persons experienced before. In this sense, it appeared to be a modality in its own right. Striving to explain the phenomenon of sensory substitution, experimental psychologist J. Kevin ORegan and philosopher Alva No developed the theory of e sensorimotor contingencies [5]. According to this theory, the qualitative dierence between two modalities neither is a result of the site of sensory processing in the cortex, nor of dierences inherent in some neural code. Instead, it is a result of the dierence in structural invariants of the change of sensory input during action. Verbalizing these invariants will result in the formulation of rules connecting ones action to the change in stimulation: When moving your gaze along a straight line of innite length, there will be no change in stimulation of the retina. Modalities are dierent because these sensorimotor dependencies are dierent for each sensory organ and stimulus source. The immense plasticity of the brain enables it to extract sensorimotor dependencies. For further particulars concerning the issues of sensory substitution and neural plasticity we refer to appendix B and appendix C, respectively. Picking up the trail where ORegan and No [5, 4] left it, the following train e of thoughts leads us to pose the questions already addressed in the beginning: Let us presume that, as claimed by ORegan and No, the dierence in sensoe rimotor dependencies constitutes the qualitative dierence between modalities. Consequently, providing access to a novel set of sensorimotor contingencies, not provided by any other sensory apparatus, will give rise to a qualitatively new perceptual experience, thus constituting a new modality. In order to verify/falsify this hypothesis experimentally, we needed to: nd a sensory domain diering suciently from existing human modalities to be describable by a new set of sensorimotor contingencies. provide subjects with an articial means to access and use information from this new domain. The sensory information we provided access to was about the local magnetic eld - indicating the direction of the north pole. We considered this to be a good decision, as humans cannot perceive magnetic elds, even though the existence of magnetic senses in certain animals shows that this information can be useful. (See appendix D)
1.3 Hardware
In order to provide continuous orientation information based on magnetic elds we built a belt equipped with a compass, tactile stimulators and a control unit to map compass information onto the stimulators. We provided tactile stimulation via 13 vibrating tactors attached to the inside of the belt. The compass feeds information about the direction of north to a microprocessor based control unit,
feelSpace
which in turn activates the vibratory element that is directed towards north. Thus, if the person wearing the belt turns clockwise, the site of stimulation jumps to a vibrator located in counter-clockwise direction. To support the integration of the belts information, we had to make sure that its reactions are very fast and reliable. Most compasses are designed for usage in vehicles and can not handle the fast and shock inducing movements of humans. MicroStrains DM-G compass, which has gyroscopes in addition to magnetic sensors, could provide information as reliably as we needed them. Still, the whole device had some noticeable latency which was mainly introduced by the vibrators, small rotation motors that need some time to get active. The belt had to be suited to be used for whole days for several weeks. We used an orthopedic rip fracture belt as the basis for the electronics because we could be sure that it would not cause any skin irritation and provides a comfortable, exible and strong support. The battery packs allowed continuous operation without the need for replacement for a whole day. In order to nd these components for our belt it was very useful to study literature from the augmented reality domain. Augmented reality devices provide additional information through existing sensory channels. It focuses on how semantic information can be added to the normal sensory input but not, as our project, on the integration of the information in the brain. See appendix E for more details. In addition to normal operation, the control unit can be interfaced from a PC. This feature was used in some of our experiments to provide wrong belt information or simulate a virtual north. Appendix F describes the belt hard- and software and the dierent components in more detail.
1.4 Experimental Design

After having built the belt, we were able to train subjects for several weeks. However, the sensorimotor contingency framework does not make any explicit predictions on what would be a measure of successful sensory integration. Therefore, we derived a number of hypotheses which were supposed to account for all possible eects of training with the belt, ranging from quantitative improvement in navigation tasks to qualitative changes in perception: Weak Integration: In principle, the sensory information provided by the belt can be processed. Thereby, training can improve performance. Strong Integration: Information on orientation of the belt can be rmly integrated in human perception. Sensory input of the belt inconsistent with other sensory inputs produces measurable responses of the test persons.
feelSpace
Subcognitive Processing: After training, attention is not needed to process belt information. New Modality: Mastery of the belt-imposed sensorimotor contingencies results in qualitatively new experiences. To test the Weak Integration hypothesis, we used a task with orientation in natural environment, a homing task in which the participants were blindfolded. They were guided along a predened path to a position from where they had to nd their way back to the starting point. The deviation of the walked path from to the ideal homing vector was used as a measure of performance. Given that the Weak Integration hypothesis holds, trained subjects should show improved performance in the orientation tasks when using the belt. We used a navigation task in a 3D virtual environment, where participants had to search the environment for target items, to test the eect of the belt during the absence of relevant proprioceptive cues. As there was only a limited amount of useful visual cues as well, we expected the belt information to increase performance in this task. Participants had to complete the task as fast as possible, thus leaving less time to consciously interpret the belt information. Hence, performance in the virtual environment task can be regarded as a measure of the Strong Integration hypothesis. In some trials, we provided the participants with false information about their heading direction in the virtual environment. Would trained subjects be able to ignore this misleading stimulation? If the Subcognitive Processing hypothesis was fullled, trained subjects should use the orientation information involuntarily and therefore not be able to simply ignore it. We applied two physiological measures, posturography and nystagmography, to quantify the eects in the domain of the Subcognitive Processing hypothesis and partly the the eects of the Strong Integration. Nystagmography measures an eye-reex elicited by angular acceleration sensed by the equilibrium organs in the inner ear. We expected rotatory information from the belt to eect the decay time of this reex when conicting with vestibular information. Posturography is a simple method to measure body sway. We hypothesized that articially induced false belt information would conict with proprioceptive and vestibular information and thereby lead to increased body sway in trained subjects. Of course, the success of our New Modality hypothesis, ultimately relying on subjective experience, was very hard to quantify. We monitored putative changes in subjective perception by the use of questionnaires, diaries and through face to face feedback. The whole experimental phase was divided into three parts: 1. A pre-training phase where baseline tests were performed with all participants.
feelSpace
2. A training period with regular controlled training in an outdoor orientation task. Subjects had to wear the belt during their wake time and were encouraged to make use of it whenever possible. This period lasted 6 weeks. 3. A post-training phase where all tests from the pre-training phase were repeated. The total number of participants was twelve, consisting of four subjects, four controls with training and four controls without training. Because subjects and the rst control group participated in training sessions, we were able to dissociate eects caused by training from those caused by constantly being stimulated with orientation information. (See appendix G)
1.5 Results
The results turned out to be very diverse, with large intra- and inter-subject variability. The orientation in natural environment experiments had the most clear-cut results. In these experiments we expected the experimental subjects performance to improve with correct belt information. And indeed, this was the case for all subjects but not for the control group: While there was no signicant dierence between the performance in dierent conditions for any subject before training, the post-training results showed a signicant improvement for the correct belt information condition only. Thus, the Weal Integration hypothesis is supported by this experiment, as the experimental subjects managed to use the belts information after training. (See appendix H and I) The orientation in virtual environment experiments, on the other hand, could not show any signicant tendency in the predicted direction. We expected the experimental subjects to improve their performance with correct belt information while the Strong Integration of the information would deteriorate their performance with wrong belt information - even though they would know that they should ignore the belt in this condition. Two experimental subjects, but also two of the controls, showed the predicted tendencies. Therefore we cannot support the Strong Integration hypothesis with this experiment. (See appendix J) The results of the physiological experiments did not allow any clear conclusion about our hypotheses either. The tendencies that we would have expected if the subconscious integration hypotheses holds was an increased body sway with conicting belt information and a decreased post-rotatory nystagmus with correct belt information. But the data suered from a high degree of variability for both the nystagmography and the body sway measurements and therefore did not show any clear results. Therefore, no statement regarding the subconscious integration of the belt information can be deduced from these experiments, even though some subjects showed the right tendencies. (See appendix K)
feelSpace
The subjective reports dier very much between subjects. Some subjects reported quite strong eects: [On a trip to Berlin and Hamburg] The orientation in the cities was interesting. After coming back, I could retrieve the relative orientation of all places, rooms and buildings, even if I did not pay attention while I was actually there. While wearing the belt, I didnt realize how much my perception had changed. After removing the belt, I experienced surprising eects. My subjective perception changed quickly. My living space diminished; the relative position to places beyond the visual horizon was rather unordered and chaotic. Other subjects were less enthusiastic about the belts eects: I really felt very little, but I had the feeling I was better with the belt at least at the natural-environment orientation tests. In summary, we can say that two subjects reported strong changes in their perception while wearing the belt. The other two subjects did not experience anything similar. One subject even ceased to realize the belts vibration at all if he did not concentrate on it. (See appendix L)
1.6 Conclusions
Let us review what we have done so far and how it all may t together. We have outlined the theoretical background of our project and described the technical details as well as experimental setups, procedures and results. We have shown that the information provided by the belt can be used in general. Active usage of the belt over a time span of six weeks signicantly improved performance in a natural orientation task. Thus, we were able to verify the hypothesized Weal Integration. However, the Strong Integration and Subcognitive Processing hypotheses were not supported by the data. Even though a number of subjective reports indicate that mentionable changes in spatial perception have occurred, we are unable to quantify these eects in all experiments. Still, we believe that our experimental design and setup was well conceived and the tests well conducted. In the remaining part of this report, we will discuss possible explanations for the results we obtained. One has to bear in mind that this discussion will be of speculative nature. One possible source of problems is the experimental setup itself. The validity of the results depended on a great number of factors, each introducing potentially confounding variables. For example, the device itself could be faulty, not providing sucient wearing comfort or an inappropriate kind of stimulation.
feelSpace
Another issue is the choice of participants. The total headcount of four experimental subjects is indeed very low and possible eects could easily be swamped considering the degree of variance in many results. It could be the case that our subjects simply were too old to harness the amount of plasticity required to rmly integrate the belt information. Yet, this explanation is not very intuitive, given the successful application of sensory substitution devices in adults. Perhaps we failed to render the orientation information behaviorally relevant enough. An extended and more rigorous training period could be a solution to this problem. Unfortunately, we already claimed a considerable amount of our subjects time, leaving almost no room for additional training. The diculty of training, on the other hand, is a point for possible improvements. Confronting subjects with more challenging tasks might improve the magnitude of integration and may have led to better results. Of course, there is always the option that the experiments themselves were not well suited to measure putative eects of the belt. But as mentioned above, we are content with what we did and do not believe that the tests were inherently awed. Our hypotheses are not directly included in in the theoretical framework. We rather adapted them to cover the whole spectrum of possible predictions that we think the theory allows. Of course it was not necessary for a successful outcome that all of our hypotheses could be supported. But assuming that none of the above factors is responsible for the weak experimental support for our hypotheses, an alternative explanation could be the existence of shortcomings in the theoretical background itself. When formulating the theory of sensorimotor contingencies, ORegan and No e only considered primary modalities, that is, modalities providing information about something that cannot be inferred from input received via other sensory channels. Information about direction or spatial location can readily be inferred by visual or proprioceptive cues. Therefore, sensing direction does not yield any substantially new information and may thus be categorized as a secondary modality. It is well conceivable that the predictions made by the sensorimotor contingency theory do not hold for secondary modalities. It may also be the case that knowledge of the sensorimotor contingencies acquired by explorative activity is an important part of, but not constitutive for the perceptual quality of a modality. Finally, some of ORegan and Nos theoretical assumptions might still be e erroneous, thus accounting for the failure of our experiment. But it goes without saying that the inability of our results to support the sensorimotor contingency theory in its full scale does not in any way disprove it. Further eort is needed both in the neurosciences and the theoretical conceptualization.
Bibliography
[1] P. Bach-y Rita. Brain mechanisms in sensory substitution. Academic Press, New York, 1972. [2] P. Bach-y Rita, C. C. Collins, F. Sauders, B. White, and L. Scadden. Vision substitution by tactile image projection. Nature, 221:963964, 1969. [3] C.C. Collins and P. Bach y Rita. Transmission of pictorial information through the skin. Advan. Biologic. Med. Phys, 14:285315, 1973. [4] E. Myin and J.K. ORegan. Perceptual consciousness, access to modality and skill theories: A way to naturalize phenomenology? Journal of Consciousness Studies, 9(1):2745, 2002. e [5] J. Kevin ORegan and Alva No. A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), 2001.
Organization, Documentation and Public Relations
Christine Carl & Christian Honey & Saskia Nagel & Moritz Stefaner University of Osnabrck u
A Organization, Documentation, and Public Relations

A.1 Overview
In the following, we will present an overview of our work on the internal organization and external presentation of our project. Due to the large size and heterogeneity of our group, the necessity of a clear organizational framework was evident right from the beginning. Our aim was to nd an eective way of structuring and planning the project in work packages and milestones. Closely connected to this, it was our ambition to maintain an ongoing documentation of work results. Finally, the presentation and communication of project results was assessed important enough to assign the task a work package of its own.
A.2 Organization
The key idea in our project organization was to have groups of people work on a particular subtask autonomously and use the group meetings for communication of results, further planning, revision and coordination. Work Packages One of the primary initial activities was the identication of dierent separable tasks and areas of activity. As a result, we decided on 17 work packages ranging from literature research, hardware and prototype construction to empirical research and project management. Each work package had a size of 2-7 people; typically, each member of the project was active in 2-3 work packages. An anonymous psychological test about the project members typical team roles was conducted and discussed in order to get a feeling for the complexity of group work and to reect ones own behavior. We changed the structure of the work packages during the project to take the actual necessities into account: Seven people left the project after the rst half to spend their semester abroad and the structure of the experimental work packages could not be established before the initial conception. Literature work packages Sensory Substitution Neural plasticity Enhanced Reality
10
feelSpace
Magnetic sense in animals Hardware work packages Vibrators Compass Belt and computational logic Experimental paradigm work packages Orientation and homing in natural environment Navigation in virtual environment Orientation in enclosed environment Nystagmography and Posturography Subjective reports Organization and PR work packages Chairmen Public Relations Organization & Documentation Finances Internal Communication We decided to use the Stud.IP platform as a common workspace to make all project resources available to all project members. We used the integrated WikiWikiWeb as a tool for communication and documentation of organizational issues. In weekly meetings, we discussed the progress of the project and future activities. These meetings focussed on topics relevant for the whole project and organizational issues; details were discussed within separate meetings of the dierent work packages. Milestones The project was planned for one year. For each work package and the whole project, we agreed on several milestones with concrete dates as well as satisfaction conditions. For instance, the completion of the belt prototype and the elaboration of experimental designs were among the major milestones of the project. The completion of each milestone required written documentation and a presentation of results in the plenary meetings. Dependencies and temporal order of these milestones were specied in a Gantt chart, which was continuously revised and adjusted.
11
feelSpace
Fig. A.1: The project logo
A.3 Documentation
Documentation is indispensable for any research project. Therefore, we tried to make high-quality documentation an integral part of the project work. This includes meeting protocols both of plenary meetings and work package meetings and progress reports in form of plenary presentations. Additionally, most major milestones were accompanied by a written report provided by the responsible work package. This allows reection and especially the reuse of already gathered and structured information for dierent purposes like posters, this nal report, the website etc.
A.4 PR Activities
A.4.1 Corporate Design
The starting point for our PR activities was to nd an appropriate name for the project. Several suggestions were discussed; the nal choice was determined in a voting. Subsequently, we decided to develop a Corporate Design to communicate the achievements of our study project adequately and consistently. This included logo design (see gure A.1) and document templates (gure A.3). Further PR tools like the poster for the Gttingen Neuroscience Conference and the website (gure o A.2) were developed on this basis. Being a study project with an innovative idea, we wanted to create a fresh impression whilst not risking to lose credibility among academic circles. The choice of strong colors in low amount, clear grid layout and the use of traditional, well-readable fonts reects this tightrope walk.
A.4.2 Ethics Commission

In order to make sure that our experiments conform with ethical standards, we presented our project and the planned tests to the ethics commission of the University. This included both written documentation as well as a personal presentation. The members of the commission did not have any objections against our plans and were interested in hearing about future results.
12
feelSpace
Fig. A.2: The project website
Fig. A.3: Document templates
13
feelSpace
A.4.3 Patent
In our initial investigation of related projects, we realized the innovative character of our device idea. To leave open the opportunity of future marketing, we decided to apply for a patent. In cooperation with n-transfer (the common platform for innovation transfer in Lower Saxony) and the consultants of the Technology Contact Point, we ran through the multi-stage process of the application. This included the creation of various application texts and schemata. Due to the unclear prospect of success and future pay-o compared to invested eorts, we desisted from completing the process.
A.4.4 Fundraising
Inspired by the potential of devices for subcognitive information access, we tried to raise funds for future activities in this eld. After identifying suitable foundations and contests, we decided to apply for the EXIST-SEED program initiated by the Federal Ministry of Education and Research (Bundesministerium fr Bildung u und Forschung). We haven chosen this programme among several alternatives; it is focussed on supporting the formation of a company from results of academic research without presupposition of a elaborate business plan, or extensive experience in a business context. The merits for chosen start-ups include coaching and monetary support over one year; the aim is to use professional support to produce a well-founded business plan in this period. Over the time of six weeks, we worked out a business plan proposal. Besides a description of the existing prototype, special emphasis was placed on the advantages and potentials of wearable devices, which provide a seamless information access without requiring attention. A rst projection of future market chances was given as well. The proposal was written in coordination and with advice from Annette Busch, the local advisor for start-up projects at the University. Unfortunately, our project did not qualify for funding. The jury did not share our opinion about the potentials and market chances and doubted the innovative character of our device.
A.4.5 Public Presentation

Mid-Term Presentation Each study project in the master program is obliged to present its work in two presentations. The mid-term presentation took place on December 12th, 2004. Besides an introduction to the theoretical background, we focussed on hardware and experimental design and provided an outlook for the tests in the future (see appendices). After the presentation, a lively discussion took place and members of the group were interviewed by a local radio station about the project.
14
feelSpace
Final Presentation The nal presentation took place on April 20th, 2005. Invited Talks We were invited to present our project at the Academy of Sciences of the Czech Republic in Prague in the context of the cooperation for the projects orientation tests in enclosed environment (see appendix I). Moreover, we were invited to discuss the theoretical background of our project at the Colloquium for Philosophy of Mind at the University of Osnabrck. u Website The study projects progress is documented on a website1 . The available information includes research background, experimental design, technological issues as well as contact information, acknowledgements and material download. The website is updated frequently. Poster at Gttingen Neuroscience Conference o We presented our work at the Gttingen Neuroscience Conference 2005. o
http://feelspace.de
15
feelSpace
A.5 Conclusions
Overall, the organizational structure worked out reasonably well, especially given the large size of the group, the fact that it was the rst major research project for most of the participants and the innovative character of the project. This is reected in the achieved results in a whole number of elds like literature research, hardware design, experimental design and conduction and data analysis. However, there were times where the lack of inter-group communication tended to complicate the common work. Additionally, the exact responsibilities of each work package turned out to be less clear than commonly believed in the beginning. In future projects, we would therefore put even more emphasis on the initial organization period and explicit descriptions of responsibilities. Another conclusion is the importance of a xed protocol to provide a framework for inter-group cooperation also in busy times, when the rst impulse is to drop apparently superuous bureaucratic activities, like meeting protocols and written documentation of results. For the rst half of the project, the milestone concept worked out well and
16
feelSpace
was a help for getting started into the project and planning the next steps. In the second phase, the explicit milestone structure was gradually replaced by a step-by-step planning. Concerning documentation, it turned out to be hard to foresee the importance of particular aspects and results of the current work in the context of future use and to agree on a common style of writing. In later project phases, more of the implicit knowledge, team decisions and experiences could have been documented. This would not only facilitate later reference for the group members themselves, but also make our experiences and conclusions more accessible and transparent for non-participants of the project as well. Feedback from externals was generally very positive. Especially the unique character of the project fascinated both researchers working on related topics as well as other students and laymen.
17
Sensory Substitution
Alper Aik & Christine Carl & Robert Mrtin c a University of Osnabrck u
B Sensory Substitution
B.1 Introduction
In a nutshell, a sensory substitution device provides sensory information from an articial receptor to the user, utilizing an existing sensory system as an interface [8, 9]. The person that can rightly be called father of sensory substitution is Paul Bach-y-Rita, a medical doctor and brain scientist dedicated to rehabilitation medicine. His initial motivation for the sensory substitution experiments was to fathom the plastic capacities of the brain and to explore their potential application in neural rehabilitation. For Bach-y-Rita, the central question was to nd out whether a blind person could learn to see, despite never having seen before. In these classic experiments that were conducted 40 years ago, a Tactile Vision Substitution System (TVSS) was built and tested [4, 11]. In a TVSS, the optical image captured by a video camera is transformed into tactile stimulation of the skin. The TVSS consisted of a tiny video camera, sending information through an electronic circuit board to an array of 20*20 vibrators on the skin of the back of the wearer. Recently, this vibrotactile array has been replaced by a miniature array of electrodes (Tongue Display Unit) for electrocutaneous stimulation of the tongue [7]. As the stimulators are located on the skin, the camera output is transduced to mechanical or electrical energy, which is sensed by the somatosensory system. However, after the training period the subjective percept is not located on the skin. Instead, subjects perceive the stimuli contained in the images to be out there, in the observed visual space. This amazing display of neural plasticity does only fully emerge when the wearer of a sensory substitution device is allowed to explore the environment and to actively manipulate the camera. Thus, simply adding a new (e.g. tactile) way of accessing visual information is not enough to give rise to a visual experience. Instead, the user needs to be fully able to act, thus embodying the sensory substitution device by exploring the relation of his action to the new sensory input. After training, the TVSS can be used independent of the location of the stimulation. TVSS devices allow access to perceptual phenomena that are unique to the domain of visual perception. Perceiving eects like motion parallax, occlusion, looming, zooming, depth and shadows along with many of the Gestalt laws and various optical illusions goes a long way towards being able to say: Thats what seeing is like. Even to the most skeptic scientist, sensory substitution is a
19
feelSpace
Fig. B.1: A blind person wearing a TVSS device. The camera is attached to the frame of the glasses, while the stimulator array can be seen in the left hand of the subject. From [4].
very good model to study the mechanisms of neural plasticity employed in using such a prosthetic device. For those with a less down-to-earth attitude, always striving to answer the more romantic questions, sensory substitution is the unique opportunity to observe the constitution of a new sensory modality. In the remaining part of this appendix we will provide the reader with an overview of the technical, experimental and nally theoretical approaches to visual sensory substitution.
B.2 Sensory Substitution devices and experiments

In this section, we introduce some of the sensory substitution devices and provide a brief summary of the observed perceptual changes and improvements as a result of training with them. This summary includes both laboratory experiments and case studies. The examples covered do not aim to be complete, but representative for the literature. Bach-y-Rita provides more exhaustive recent reviews [8, 9]. Some of the examples include brief experimental approaches, others make use of extended training to monitor the changes in qualitative aspects of perception [20, 13]. Even though we describe changes in the brain activity in experiments with imaging data, we wont speculate about the results. Laboratory experiments The aim of the laboratory experiments with sensory substitution devices is either to monitor the eects of training with them on the brain, or to see the perfor-
20
feelSpace
mance of these devices on clearly dened tasks like distance detection with a TVSS. These studies cannot provide a full account of the perceptual changes the subjects undergo, because the most interesting eects of sensory substitution are qualitative in nature.
B.2.1 TVSS
Tactile to visual sensory substitution devices are those devices, with which most of the classic experiments have been conducted. Early studies by Bach-y-Rita, making use of tactile input to transmit visual information, showed that blind subjects could start to make judgments about perspective, parallax and even depth, with a fair amount of training [6, 4]. One major nding was that if, after some practice with the device, the location of the tactile stimulation was changed (e.g from abdomen to forehead), the subject was able to cope with this change immediately. However, since the nature of these early studies is anecdotal, we review them in the Case Studies paragraph below in more detail. In a recent work, Ptito and colleagues used the Tongue Display Unit (TDU, as mentioned in the introduction), which is a TVSS device originally developed by Bach-y-Rita [7] to study improvement in orientation detection, together with accompanying regional cerebral blood ow (rCBF) changes [20]. Blind and visually intact, but blindfolded, subjects attended daily sessions over a week, where they used the TDU to detect the orientation of a T displayed on a Laptop. Using the mouse cursor they could scan the screen. The subjects learned the task at the end of the week, and the performance of blind subjects was signicantly higher than the performance of blindfolded controls. The rCBF data showed a signicant occipital activity increase in the blind subjects after training. No such change was observed for the control group. Hanneton, Lenay and colleagues developed a minimalist TVSS devices, coupling a graphics-tablet with a computer and a vibrotactile stimulation device [12]. Using the tablet, users were able to freely explore black and white bitmap images without intermediate gray values. The amount of vibrators changed between 1 and 5. Hanneton and colleagues rst studied the performance of their minimalist device with ve skin stimulators in shape recognition tasks. In the rst experiment, the orientation of the letter S was to be detected [12]. They found ceiling eects (100% performance) in twenty trials, where the subjects had as much time as they wanted to move the cursor. In the follow-up, they presented dierent upper case letters on the screen and the subjects had to identify the letter. After each response feedback was given. In three sessions separated with one week in between, subjects showed signicant improvement at correctly identifying the letters. In a second series of experiments, Lenay and colleagues demonstrated that even a single photoreceptor connected to a single tactile stimulator could be used to detect the direction and the distance of a luminous target [13]. However, this was only possible as long the users were able to move their arm to actively
21
feelSpace
manipulate the input of the sensor. In addition to the performance in the tasks, Lenay et al. also asked their subjects about the changes in the qualitative aspects of their perception using the TVSS [13]. Subjects reported that in the early phases of the task, they only felt the successive stimulations on the surface of the skin. But with training, these tactile sensations faded away from awareness, and were replaced by perception of objects located in the external world. Case Studies and Informal Examples In the early studies of Bach-y-Rita [4, 6], which he reviewed recently [8, 9, 5], the major nding was the qualitative character of the visual perception the blind subjects gained using the TVSS device: After sucient training with the TVSS, our subjects reported experiencing the image in space, instead of on the skin. [9] One common example, clearly demonstrating the sub-cognitive nature of the resulting changes, was the eects of zooming the camera without telling the subject [4]. As the subject was looking at an object, the experimenter zoomed the camera, unknown to the subject. The subjects response to the illusion of this very fast approaching object was throwing himself backwards in a reexive manner, even though the tactile stimulation regarding the object was located on his back.
B.2.2 AVSS
In the TVSS, the role of the interface modality is occupied by the skin, whereas in the AVSS it is played by the auditory system. Most AVSS devices share a common set-up: a video camera is linked to a mobile computer, which in turn is connected to pair of headphones. The advantages of AVSS devices are mainly of aesthetic nature, since earplugs are more easily integrated in the social environment than tongue-displays or vibrotactile arrays. The disadvantage, on the other hand, is the (at least partial) occupation of hearing capacities. The available devices mainly dier in the method of representing image information as sound waves. Arno & colleagues [2] and Meijer [17], for instance, used dierent techniques of mapping grayscale image information onto compositions of sound frequencies. The algorithm applied by Arno et al. processes a whole frame at a time, assigning each pixel in the frame a dierent frequency. The luminance information of each pixel is represented by the amplitude of this frequency. The sum of frequencies multiplied by their amplitudes produces a complex waveform carrying the image information. Interestingly, the camera resolution in the central region of the images is increased with respect to the periphery, thus creating a functional resemblance to foveal vision.
22
feelSpace
Fig. B.2: Meijers algorithm distributes image information in time columnwise. From [17].
While Arnos approach codes video information into soundwaves frame by frame, Meijer considers only one column of pixels at time. In doing so, the possible resolution that can be represented in auditory information is increased, by distributing the image in time columnwise. Unfortunately, there is a tradeo between resolution and time, because each frame has to be scanned from left to right[4, 11]. Meijer implemented this algorithm in an AVSS software called The vOICe. Using this software, one can easily build an AVSS device using o the shelf hardware. Because of its accessibility, The vOICe is already being used by blind persons and thus can already be observed in the eld. While we consider only the work of Meijer and Arno et al., many other researchers approached the eld of auditory to visual substitution with a rich diversity of techniques and experiments [10, 16]. AVSS experiments Arno and colleagues [3, 2] studied in two sets of experiments the pattern recognition performance of blind and sighted subjects using their AVSS device. The rst experiment was conducted to test whether their AVSS device [1] could aid blind subjects to use computers [3]. The increasing importance of using graphical user interfaces for computers in the past decades was the motivation of the study. The camera was not mounted on the head of the subjects, but it was at the tip of a pen, with which the subjects were scanning a computer screen. On the screen black and white shapes were displayed. The complexity of these shapes changed from very simple, i.e. dots or single lines, to quite complex, like oriented E s. Both groups proted from training with the device. Additionally, the performance of the blind subjects exceeded those of blindfolded controls, both in terms of accuracy to identify the shapes and the scanning time they needed
23
feelSpace
for the identication. In the second study, Arno et al. used the same set of stimuli, but the camera was mounted on the head [2]. Like Ptito and colleagues [20], they took PET scans before and after training with the AVSS device. Behavioral results again showed an improvement with training for both groups, and the performance of the blind subjects was signicantly better than that of the blindfolded controls. Additionally, occipital activation increased with training with AVSS device only for blind subjects. Case Studies and Informal Examples An exciting part of Meijers vOICe project concerns the reports of blind people on their subjective experiences with this device [15]. Both early- and late-blind persons describe the (re-)emergence of visual percepts. Of course, one has to keep in mind that these reports are of anecdotal nature and that the plural of anecdote is not data. The following short excerpt is taken from the report submitted by a late-blind female user. Well, the other day I was again washing the dishes. I had let the water out of the sink and turned to get a towel to dry my hands. Then, when I turned back to rinse the sink I was stunned to see the sink in a depth like image. I stepped away from the sink and walked slowly up to it again, to see if my mind was playing tricks on me. No, the feeling of seeing depth in the sink bowl was still there. Depth cues and their sensorimotor rules belong to the features exclusive to the visual domain. As already mentioned in the introduction, gaining access to these features comprises a big share of seeing.
B.3 How does sensory substitution work?

In the following paragraphs we will discuss how sensory substitution is working. It is obvious that we associate vision with eyes, or hearing with ears. The possibility of gaining visual perception in the absence of eyes is counterintuitive, yet exciting. We will focus on the theoretical approaches of the most prominent gures in the eld [5, 19, 14]. We will try to shed light on how, with the use of sensory substitution devices, perceptual experience arises, and how perception and action are related for the success of sensory substitution. Bach-y-Rita explains the success of sensory substitution devices in giving rise to visual experience by stating that we see with the brain, and not with the eyes [9, 8, 5]. As long as the brain is able to extract the visual invariants through some sensory channel (in the absence of the normal channel, which in this case is the eye) it is possible to obtain subjective qualities of seeing. The tactile or
24
feelSpace
auditory stimulation provided by the device carries spatially coded information. Our brains immense plastic capacity enables it to analyze auditory and tactile stimulation according to visual invariants. This instrumental sensory capacity, as one might call it, appears on the scene if the following criteria are fullled. 1. Functional demand exists. In case of the TVSS, this means that the visual apparatus is defective. Thus, information provided by the TVSS is not redundant. 2. Sensor technology to satisfy the demand is available (a sensory substitution device). 3. Training and psychosocial factors support the functional demand [9]. Even though it is not cited in the list quoted above, Bach-y-Rita also acknowledges that if the subject is not allowed to manipulate the device on its own (e.g. if the camera is not under the control of the subject), there is little if any success [4, 5]. This is especially true in terms of leading to the perception of the objects out there in visual space, and not on the skin. Without the sensorimotor links (re-)established, subjective perception of shape, form, size etc. does not seem to be possible. It seems like Bach-y-Rita regards the active exploration with the device as a process that helps the subject to achieve a subjectively visual experience [5]. Concluding, as perceptual experience is dependent on the information content rather than on the sensory channels, providing vision-like information to the subject should lead to visual experience. Charles Lenay, on the other hand, states that this sensorimotor aspect of sensory substitution is the most important one [14]. This is comprehensible, when we reconsider his and colleagues experimental methodology [13]. Their minimalist device has a single sensor, and a single vibrator; but still the subjects start to perceive the objects in the external world, as long as they are free to explore the environment with the device. Lenay even prefers to use the term Sensorimotor Substitution rather than Sensory Substitution. Accordingly, the sensory substitution studies become the empirical proof that there is no perception without action. This leads him to the conclusion that ... what is perceived, or recognized, does not derive from invariants in the sensory information but from invariants in the sensorimotor cycles which are inseparable from the activity of the subject. [14] For Lenay, a proper denition and/or modeling of what these sensorimotor invariants are, still remains to be formulated [14]. We think that ORegan and Nos e sensorimotor contingency theory, described briey below, provides the denition, borrowing examples from sensory substitution literature. Before starting the discussion of sensory substitution from the sensorimotor contingency theory point of view, we have to dene sensorimotor contingencies
25
feelSpace
[19]. Briey, these are the laws governing the sensory changes produced by various motor actions. These lawful changes depend on both the properties of the object and the properties of the sensor. Such a view is very close to Lenays, with a more comprehensive discussion of the sensorimotor invariants involved [14]. Accordingly, the dierences between modalities are explained as the dierences of object properties that are relevant for the sensory modality (e.g. reecting light versus reecting sound) and the sensory apparatus used in that domain. This leads to the denition of perception in a given modality as follows: ... the activity of exploring the environment in ways mediated by knowledge of the relevant [for that sensory domain] sensorimotor contingencies. [19] Let us look at how this theory is applied to sensory substitution studies. Similar to Bach-y-Rita, the neural channel through which the input is provided, is regarded as irrelevant. What matters are the sensorimotor contingencies involved. So what does a visual sensory substitution device do? It allows its user to master its sensorimotor contingencies that are similar to those of normal vision. Accordingly, this makes it possible to obtain sensations, which are qualitatively similar to the ones we see in case of intact vision [18].
B.4 Conclusion
Lots of observations have been made with sensory substitution devices. Both these observations and the theories developed to understand phenomena like the (re)-emergence of visual perception have contributed to our understanding of perception and the subjective feeling accompanying it. If we take up the trail where ORegan and No [19] left it, we can even think of new qualitative perceptual e experiences becoming possible 1.
26
Bibliography
[1] P. Arno, C. Capelle, M.-C. Wanet-Defalque, M. Catalan-Ahumada, and C. Veraart. Auditory coding of visual patterns for the blind. Perception, 28:10131029, 1999. [2] P. Arno, A. G. De Volder, A. Vanlierde, M.-C. Wanet-Defalque, E. Streel, A. Robert, S. Sanabria-Bohrquez, and C. Veraart. Occipital activation by pattern recognition in the early blind using auditory substitution for vision. NeuroImage, 13(4):632645, 2001. [3] P. Arno, A. Vanlierde, E. Streel, M.-C. Wanet-Defalque, S. SanabriaBohorquez, and C. Veraart. Auditory substitution of vision: pattern recognition by the blind. Applied Cognitive Psychology, 15(5):509519, 2001. [4] P. Bach-y Rita. Brain mechanisms in sensory substitution. Academic Press, New York, 1972. [5] P. Bach-y Rita. Tactile Sensory Substitution Studies. Ann NY Acad Sci, 1013(1):8391, 2004. [6] P. Bach-y Rita, C. C. Collins, F. Sauders, B. White, and L. Scadden. Vision substitution by tactile image projection. Nature, 221:963964, 1969. [7] P. Bach-y Rita, K. Kaczmarek, M. Tyler, and J. Garcia-Lara. Form perception with a 49-point electrotactile stimulus array on the tongue. Journal of Rehabilitation Research Development, 35:427430, 1998. [8] P. Bach-y Rita and W Kercel S. Sensory substitution and the humanmachine interface. Trends in Cognitive Sciences, 7(12):285295, 2003. [9] P. Bach-y Rita, M. E. Tyler, and K. A. Kaczmarek. Seeing with the brain. International journal of human-computer interaction, 15:285295, 2003. [10] C. Capelle, C. Trullemans, P. Arno, and C. Veraart. A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Trans Biomed Eng., 45(10):12791293, 1998. [11] C.C. Collins and P. Bach y Rita. Transmission of pictorial information through the skin. Advan. Biologic. Med. Phys, 14:285315, 1973.
27
feelSpace
[12] S. Hanneton, O. Gapenne, C. Genouel, C. Lenay, and C. Marque. Dynamics of shape recognition through a minimal visuo-tactile sensory substitution interface. Third International Conference On Cognitive and Neural Systems, Boston, USA, 1999. [13] C. Lenay, O. Gapenne, and Stewart J. The constitution of spatiality in relation to the lived body : a study based on prosthetic perception, emergence and development of embodied cognition. EDEC-2001, Symposium at the 3rd International Conference on Cognitive Science , Beijing, China, pages 2529, 2001. [14] C. Lenay, O. Gapenne, Hanneton S., Genoulle C., and Marque C. Sensory substitution, limits and perspectives. In Touch for Knowing. John Benjamins Publishers, Amsterdam, 2003. [15] P. Meijer. The voice - seeing with sound: User reports. http://www.seeingwithsound.com/users.htm. [16] P. Meijer. The voice - seeing with sound, website. http://www.seeingwithsound.com/. [17] P. Meijer. An experimental system for auditory image representation. IEEE Transactions on Biomedical Engineering, 39(2):112121, 1992. [18] E. Myin and J.K. ORegan. Perceptual consciousness, access to modality and skill theories: A way to naturalize phenomenology? Journal of Consciousness Studies, 9(1):2745, 2002. [19] J. Kevin ORegan and Alva No. A sensorimotor account of vision and visual e consciousness. Behavioral and Brain Sciences, 24(5), 2001. [20] M. Ptito, S. M. Moesgaard, A. Gjedde, and R. Kupers. Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. Brain, 128(3):606614, 2005.
28
Neural Plasticity
Boris Bernhard & Saskia Nagel University of Osnabrck u
C Neural Plasticity
C.1 Introduction
To cope with a constantly changing and dynamic world, the human nervous system is not static in architecture and connectivity, but rather undergoes constant adaptation. This phenomenon is labelled neural plasticity and it seems to be an ubiquitous mechanism in the human brain. [37] Mechanisms of plasticity allow our nervous system to meaningfully integrate peripheral information, to learn new motor skills and to develop these cognitive capacities which make us inherently human. Further, they allow adjustments in signal processing, following changes in both subject and environment. Accompanying these adjustments, changes on the gross macroscopical, as well as on the microscopical, synaptic level occur. Molecular mechanisms of neural plasticity Underlying mechanisms of neural plasticity may therefore be dierentiated across two dierent stages [37, 34]. First, molecular studies point to plasticity that happens on the synaptic level. [22, 10, 28] Since mature nerve cells have lost the capacity to divide, the synaptic connections between neurons are modied in strength and extent, enabling changes in signalling either on the short term, or on the long term [22]. Studies in vertebrates have shown that long-term memory diered from short-term memory on a molecular level. Long-term mechanisms aect the synthesis of new proteins, thereby globally changing the metabolism of a given cell, whereas short-term mechanisms often lead to the formation of new receptor molecules or the block of present potassium channels. [22] Processes involved in both long-term and short-term molecular mechanisms are second- messenger cascades. Prominent agents in the cytoplasm are proteinkinase A (PKA) and cyclic-AMP (cAMP). Long-term mechanisms take place in the nucleus. They seem to work on top of short term processes, and seem to require the synthesis of new proteins [23, 28]. Studies point to a prominent role of CREB, cAMP-response element binding protein, in inuencing transcription. Genetic studies have shown that the cAMP - PKA - CREB pathway seems to be highly conserved across species. This suggests that memory mechanisms have been modied and extended across evolutionary development [22]. Critical physiological mechanisms in long-term synaptic change may be processes such as Long-Term-Potentiation (LTP) [2], Long-Term Depression (LTD) [1], or the presence of specic transmitter molecules [34, 15]. There is evidence for timedependent as well as for activity-dependent mechanisms, which adjust changes in signaling of adjacent neurons. Concluding, investigations on the synaptic level
30
feelSpace
point to a variety of dierent molecular mechanisms involved. All of them seem to be highly inuenced by the extent and time-course of activity within their neuronal population. Cortical Mechanisms of neural plasticity The second evidence of neural plasticity can be found when one observes time- and activity dependent changes on the cortical level. In the case of primary sensory areas such as V1 [14, 4, 8], S1 [21], or the primary auditory cortex (A1) [33], cortical representations of the environment, so called cortical maps, are dynamically and continuously modied by sensory experience. These changes eect that larger amounts of cortex get devoted to those subparts of the peripheral space which seem to provide behaviorally more relevant input. Thus, the brains topography is not stabile, but rather underlies massive reorganization by means of highly malleable bers interconnecting specialized functional modules. But what are the mechanisms underlying cortical reorganization? Clinical studies, as well as observations on animal models, point to a variety of dierent mechanisms. Corresponding to the scheme drawn by Ebert [5], we outline three dierent basic principles that drive cortical reorganization: 1. First, there is evidence of an invasion of nearby branches following disuse or deaerentiation. Here, nearby areas literally take over the area of a functional module, resulting in shifts of functional specialization of that area. Such cases are illustrated mostly by clinical cases, such as the emergence of phantom limbs following limb transection [19], by studies on Braille reading in congenitally blind persons, or by studies in which visual aerents are rewired to auditory cortex in rodents [36]. Braille reading patients benet from the processes of cortical reorganization: Transcranial Magnetic Stimulation (TMS) applied to the occipital cortex, particular to V1, leads to a distortion of performance in Braille reading. Thus, it is assumed, that the region which is not used for visual processing any more has been invaded by non-standard branches from areas devoted to tactile discrimination [3]. Finney and colleagues have shown that a kind of cross-modal activity also occurs in human auditory cortex. In a fMRI study, they were able to demonstrate that visual stimulation leads to activity in auditory areas in early-deafened subjects.The demonstration of cross-modal plasticity in the blind and the deaf provides evidence for the brains ability to reorganize after removal of one sensory modality, but point also to a strong dependence of rerouting success on the onset of removal. In several studies [36] experimenters have rerouted the visual pathway from the retina of rodents onto those areas in the thalamus devoted to auditory information processing. The results were, that cell populations in auditory cortex and thalamus form retinotopic maps, usually seen in visual cortex. Crossmodal remappings can, if performed in early stages of development, guide visual behavior, although with a lesser acuity that that of the normal visual pathway.
31
feelSpace
2. Second, cortical representations increase due to increased stimulation and use. Training, for instance, alters somatosensory cortex maps: It has been shown that extensive training, activating a certain region repetitively, leads to an enlargement of that particular area. Musicians show broader regions devoted to their hands: In particular, violinists have extended region of the areas devoted to the left ngers, in comparison to the ngers of the right hand [5]. Additionally, a variety of studies have shown, both in animals [38] and in humans [24] that in areas of motor cortex, repeated stimulation of representations leads to an increase of both cortical representations as well as an increase in motor output from that repeated stimulations. These ndings provide a hypothesis for the cortical mechanisms accompanying the training and improvement of motor activity. 3. Third, maladaptations occur due to false interactions between environmental requirements and cortical functioning. Erroneous adaption of cortical functioning is illustrated by phantom limb pain, tinnitus, and synaesthesia. A phantom limb is an extremity, that was lost, but can still be felt. For instance, some patients feel they could still move their amputated arm. Others suer from phantom limb pain, where the non-existing limb itches, burns and cramps. 70-90 percent of adults, who have lost a limb, experience a phantom limb. The probability of experiencing phantom limb pain increases with increased age at time of amputation. Ramachandran and co-workers [32, 29] have argued, that phantom limb sensations stem from an imbalanced integration of aerent connections from and eerent activity to the missing stump. Besides, it is likely that existing neuronal connections from adjacent cortex are being remapped to cortical regions concerned with the missing limb. This hypothesis is supported by the nding that patients with phantom limb pain tend to have an imbalance in the size of the cortical representations.The underlying working of neural plasticity is further reected by the fact that the older a person is, the more likely a stump becomes a phantom limb. In this case the brain could not successfully unmask remapped connections, since probably long term mechanisms of protein synthesis and receptor formation decrease during aging. Tinnitus is a pathological condition in which auditory sensation without adequate stimulation occurs. Tinnitus is assumed to be based on a maladaptation of the auditory cortex [24]. If a peripheral lesion, which might be by itself manageable, is combined with an extraordinary intense, stressful experience, it can cause a negative cortical reorganization, such as that underlying phantom-limb pain and tinnitus. Inner hair cells in the organ of corti react to harmful inputs as if they were behaviorally relevant - and thus, there is more place devoted to the wrong frequencies in the auditory cortex.
32
feelSpace
In synaesthesia, a person either experiences sensations in one modality, although a dierent modality is stimulated (cross-modal synaesthesia), or experiences sensations in one feature domain of a modality, although another feature channel is stimulated (intra-modal synaesthesia). Popular cases of cross-modal synaesthesia involve visual experience during auditory stimulation. Well known cases of intra-modal synaesthesia are, for example, qualitative experiences of colors during the presentations of uncolored graphemes. Synaesthesia seems to be an inherently sensory, but nevertheless heterogenous phenomenon [30]. Pop-out tasks on synaesthetics revealed that synaesthetics are able to detect synaesthesia-inducing digits among competing stimuli more rapidly than non-synaesthetic controls, pointing to the fact that synaesthesia not a cognitive, memory-driven, but rather a perceptual phenomenon [30]. In order to explain why synaesthesia occurs, Ramachandran and colleagues [31] claim that synaesthetic associations are to a great degree genetically determined, since they remain fairly stable over time. Accordingly, grapheme - color synaesthesia may be induced by a genetic mutation. This mutation may lead to an increased crosstalk between areas in the fusiform gyrus responsible for color detection (V4, V8), and the adjacent visual grapheme area. However, they further suggest, that the mutation itself has just permitted synaesthesia, but that the nal association seems likely to be maladaptively learned. In summary, neuroscientic investigations on cortical plasticity suggest that the organization of the central nervous is malleable with respect to perceptual and behavioral demands. Genetic factors inuence regional specialization, as seen in diminished visual acuity after rerouting of cortical pathways, or the stability of grapheme-color synaesthesia. However, it additionally seems likely that a big deal of regional specialization as well as changes in function and organization are driven by ongoing behavior in a dynamically changing world.
C.1.1 Perception and action are intermingled

Behavior and natural world dynamics have a strong inuence on rearrangements in neural architecture. A huge variety of literature [27, 12, 26, 25] suggests that both domains are far more likely to be intertwined than previously [17, 6] suggested. Therefore, it might be that the interaction of motor activity and perceptual processing is one of the crucial mechanisms which drives plastic changes in the functional connectivity of the nervous system. A variety of studies [34, 15] have found indicators for the involvement of behavior-related neurotransmitter systems in the synaptic changes on the local level. The presence of specic neurotransmitters, especially acetylcholine from basal forebrain structures, seems to have a strong impact on how synaptic connections undergo changes both in the sensory and in the motor domain.
33
feelSpace
C.1.2 A new model of neural plasticity

For successful models of neural plasticity, it is crucial that they account for changes in sensory stimulation, changes in motor activity, and the changes in the relations between both in the light of behavioral and cognitive goals. As stated before, changes in stimulation and motor activity may occur due to conditions such as disuse, lesioning or deaerentiation, but also due to experience and training. Herein, behavioral relevance plays a very important role, and may be expressed by the presence of specic neurotransmitters, or by coincident activation of specic neural structures, especially the basal forebrain [34, 15] , ventral tegmental area [13, 18], the nucleus accumbens [11, 35], and the amygdaloid complex [9, 20]. The relations between both sensory stimulation and motor activity, and the meaningful and adaptive integration of both, may be driven by attentional mechanisms, balanced crosstalk as well cognitive and subcognitive capacities. These capacities allow to adapt cortical functioning and subjective experience to ongoing changes in stimulation. They are determined by genetic factors as well as activity and challenge. The interplay of both action and perception might lead to cortical reorganization in a goal - driven behavioral context, and additionally to the emergence of a specic qualitative experience associated with that very stimulation.
C.1.3 Test of new model - the feelSpace project

To account for such a dynamic model, which bears strong resemblance to the sensorimotor- contingency framework, or enactive approach, respectively of OReagan and No [27], Hurley & No [12] and others [7], we have started the study e e project feelSpace . In feelSpace, we [16] have developed a vibro-tactile belt device. When worn, the belt stimulates humans with information from an attached electronic compass device, always vibrating in the direction of the magnetic north pole. The bearers perceptual reality is thus enriched with additional real-time sensory information, which interactively changes with respect to her position and orientation in space. In our experimental sessions, participants wear the belt in a 6-week training period. Before the experimental trials, we recorded the participants performance in outdoor, indoor and virtual environments, as well as their vestibular-ocularreex with the help of Nystagmography. Ongoing behavioral tests and subjective reports are used to evaluate the condition of the subjects before, during and after the tests. After the rst run-up, participants performance in all conditions is matched and compared against controls. Further, we compare the pre-training data with the post-training results. Planned are further imaging studies that may shed light on the extent and signicance of cortical reorganization. We expect that movement of subjects under new relations between action and stimulation might result in new qualitative experiences, although solely so-
34
feelSpace
matosensory areas are concerned with the processing of stimuli. New qualitative experiences may occur if the subjects are indeed able to integrate the new form of action-dependent stimulation into their cognitive context and perceptual awareness. This approach might shed light on the development and nature of sensory modalities, the critical processes involved and the relationship between sensory information, a fairly static sensory apparatus, dynamic neuronal representations, active behavior, and qualitative, subjective feelings associated with this information. However, it is not very clear in how far compass information might inuence our actions and how behaviorally relevant it is for humans. Information about the geomagnetic orientation is unlikely to have a very dominant role in social contexts. In order to address the the topic of behavioral relevance, we need to nd out how signicant the dierence in changes of cortical reorganization in post-trained subjects compared to controls and pre-training subjects are, and in how far reorganization changes embedded behavior.
35
Bibliography
[1] M.F. Bear and W.C. Abraham. Long-term depression in hippocampus. Ann Rew Neurosci, 19:43762, 1996. [2] T.V.P. Bliss and T.J. Lomo. Long-lasting potentiation of synaptic transmission in the dentate gyrus of the rat following selective depletion of monoamines. Physiol., 232:331356, 1973. [3] L.G. Cohen and et al. Functional relevance of cross-modal plasticity in blind humans. Nature, 389:180183, 1997. [4] C. Darian-Smith and C.D. Gilbert. Axonal sprouting accompanies functional reorganization in adult cat striate cortex. Nature, 368:73740, 1994. [5] T. Elbert and C. Pantev. Increased cortical representation of the ngers of the left hand string players. Science, 270(5234):305307, 1995. [6] J.A. Fodor. The modularity of mind: An essay on faculty psychology. Cambridge, MA/Bradford MIT Books, 1983. [7] J.J. Gibson. The ecological approach to visual perception. Houghton-Miin, 1979. Boston:
[8] C. D. Gilbert and T. N. Wiesel. Receptive eld dynamics in adult primary visual cortex. Nature, 356:150152, 1992. [9] S. Hamann. Cognitive and neural mechanisms of emotional memory. Trends in Cognitive Science, 5(9):394400, 2001. [10] R.D. Hawkins and E.R. Kandel. Is there a cell biological alphabet for simple forms of learning? Psych. Rev., 91:375391, 1986. [11] Hooks and AUTORNAMEN VERVOLLSTANDIGEN. Amphetamine depresses excitatory synaptic traqnsmission via serotonin receptors in vta. J Neurosci, 19(22):97809787, 1999. [12] S. Hurley and A. No. Neural plasticity and consciousness. Biology and e Philosophy, 18:131168, 2003. [13] S. Jones and J. Kauer. Amphetamine depresses excitatory synaptic traqnsmission via serotonin receptors in vta. J Neurosci, 19(22):97809787, 1999.
36
feelSpace
[14] J.H. Kaas. Plasticity of sensory and motor maps in adult mammals. Ann Rev Neuroscience, 14:137167, 1991. [15] M.P. Kilgard and M.M. Merzenich. Cortical map reorganization enabled by nucleus basalis activity. Science, 279:17141718, 1998. [16] P. Knig, A. Aik, B.C. Bernhardt, C. Carl, T. Dierkes, I. Dombrowe, o c S. Gelesz, C. Honey, L. Jansen, C. Kabisch, T. Kringe, L.M. Kurzen, C. Lrken, R. Maertin, S.K. Nagel, K.H. Park, H. Saal, M. Stefaner, o C. Stel, and V. Willenbockel. feelSpace Poster. Report of a Study group. o Institute of Cognitive Science. University of Osnabrueck. (6th Meeting of the German Neuroscience Society - 30th Gttingen Neurobiology Conference, o Gttingen), 2004. o [17] E. Lepore and Z. Pylyshyn. What is cognitive science. Oxford: Blackwell, where it is entitled What is in your mind, 1999. [18] M. Leyton, I. Boileau, C. Benkelfat, M. Diksic, G. Baker, and A. Dagher. Amphetamine - induced increases in extracellular dopamine, drug wanting, and novelty seeking: A pet/[11c]raclopride study in healthy men. Neurobiopsychopharm, 27:6, 2002. [19] B.A. Linkenhoker and E.I Knudsen. Incremental training increases the plasticity of the auditory space map in adult barn owls. Nature, 419:293296, 2002. [20] J.L. McGaugh. Amygdala: role in modulation of memory storage, in: The amygdala: A functional analysis (aggleton, j.p. ed.) oxford university press. pages 391423, 2000. [21] M.M. Merzenich. Nature, 332:444445, 1998. [22] B. Milner, L. Squire, and E.R. Kandel. Cognitive neuroscience and the study of memory. Neuron, 20:445468, 1998. [23] P. Montarolo, S. Schacter, V.F. Castellucci, R.D. Hawkins, T.W. Abrams, P. Goelet, and E.R. Kandel. Interrelationships of cellular mechanisms for dierent forms of learning and memory. Levi Montalcini, R. , Calisano, P. and Kandel, E.R. and Maggi, A. (eds) Molecular Aspects of Neurobiology. Springer Verlag, Berlin, 1986. [24] W. Muehlnickel and T. Elbert. Reorganization of primary auditory cortex in tinnitus. PNAS, 95:1034010343, 1998. [25] A. No, J.K. ORegan, and E. Myin. Towards an analytic phenomenology e the concepts of bodiliness and grabbiness. Proceedings of the international Colloquium: Seeing and Thinking. Reections on Kanizsas Studies on Visual Cognition, UNiv. Tor. Vergata, Rome, 2001.
37
feelSpace
[26] A. No and E. Thomson (ed.). Vision and Mind: Selected Readings in the e Philosophy of Perception. Cambridge MIT Press, 2002. [27] K. ORegan and A. No. A sensorimotor account of vision and visual cone sciousness. Behavioral and Brain Sciences, 24(5):883917, 2001. [28] Goelet P., V.F. Castellucci, S. Schacter, and E.R. Kandel. The long and short - term memory: A molecular framework. Nature, 332:419422, 1986. [29] V.S. Ramachandran. Behavourial and magnetoencephalographic correlates of plasticity in the adult human brain. PNAS, 90:1041310420, 1993. [30] V.S. Ramachandran and E.M. Hubbart. Psychophysical investigations into the neural basis of synaesthesia. Proceedings of the Royal Society of London, 268:97983, 2001. [31] V.S. Ramachandran and E.M. Hubbart. Synaesthesia - a window into perception, thought, and language. J Consciousness Studies, 8(12):334, 2001. [32] V.S. Ramachandran and D. Rogers-Ramachandran. Synaesthesia in phantom limbis induced with mirrors. Proc. R. Sco. Lond. B., 263:377386, 1996. [33] G.H. Recanzone, C.E. Schreiner, and M.M. Merzenich. Plasticity in the frequency representation of primary auditory cortex following discrimination training in adult owl monkeys. JNeurosci, 13:87103, 1993. [34] M.A. Sanchez-Montanes, P. Verschure, and P. Knig. Local and global gating o of synaptic plasticity. Neural Computation, 12:519529, 2000. [35] N. Volkow and T.K. Li. Drug addiction: The neurobiology of behavior gone awry. Nature Reviews Neuroscience, 5:963971, 2004. [36] L. Von Melcher, S. Pallas, and M. Sur. Visual behaviour mediated by retinal projections directed to the auditory pathway. Nature, 404:871876, 2000. [37] L.M Ward. Human neural plasticity. Trends in Cognitive Science, 5(8):325 327, 2001. [38] U. Ziemann, G. Wittenberg, and L. Cohen. Stimulation - induced withinrepresentation and across-representation plasticity in human motor cortex. J Neurosci, 22(13):55635571, 2002.
38
Magnetic Sense in Animals
Lina Jansen & Hannes Saal University of Osnabrck u
D Magnetic Sense in Animals

D.1 Introduction
The main objective of our literature group was to gather information on what animal species use the geomagnetic eld for navigation, how such a sense could be realized biologically and what behavioral studies that tried to prove the use of geomagnetic cues looked like. Further, we searched for evidence against or for a magnetic sense in humans. Our motivation was to get background knowledge about the earth magnetic eld and to see how a magnetic sense could be helpful in navigational tasks. The knowledge of experimental conditions, which are performed to test if an animal has a magnetic sense, could possibly help our study project in the design of navigational tasks. Following this motivation this appendix will start with a short introduction to the earth magnetic eld. Then, the Map and Compass Hypothesis, which is suggested to explain the ability of homing animals to nd their home places, is described and three dierent possible magnetoreceptor mechanisms, which could enable the animals to sense the magnetic eld, are introduced. Further, dierent experimental paradigms when testing for a magnetic sense are explained. In the case study Navigation in Sea Turtles the afore introduced knowledge is combined and exemplied. The chapter appendix ends with a short look on a possible human magnetic sense.
D.2 The Earth Magnetic Field

The Earth Magnetic Field can be compared to that of a dipole. The main eld is generated in the core and contributes with over 90% to the overall eld. The anomalies are mainly due to currents in the crust. Two dierent parameters can be distinguished when describing the earth magnetic eld: direction and intensity. Direction (measured in degrees) can be subdivided into inclination (i.e. angle between earth surface and total eld vector) and declination (i.e. angle between magnetic north and true north). Intensity or total intensity (measured in tesla) can be represented as a three-dimensional vector, which consists of a two-dimensional horizontal and a vertical component. The total intensity of the earth magnetic eld ranges between 25,000 and 65,000 nanotesla. It is constantly changing and more stable at night [29].
40
feelSpace
Fig. D.1: Contours of equal magnetic intensity and inclination. These two parameters could in principle be used by Ascension island turtles for navigation. Adapted from [29].
D.3 The Map and Compass Hypothesis

Some homing animals are able to nd their home places with astounding accuracy even when they are displaced to a location where they have never been before. This ability is also called True Navigation. To explain this behavior the Map and Compass Hypothesis [26] was suggested. It states that these animals maintain a map to determine their position relative to their goal, and have some sense of direction to set position to the goal. It has already been shown that a lot of animals use stimuli from the geomagnetic eld to set the direction, but which stimuli are used to construct the map is still debated. What is needed are two dierent environmental stimuli, which at best should be orthogonal. Suggestions of environmental sources include solar arcs, natural odor sources or the gravitational eld of the earth [26]. But up to now perfectly orthogonal stimuli have not been found. Information from the magnetic eld could in principle be exploited in such a way, although they are not constantly right-angled. It was suggested that intensity of the magnetic eld vs inclination could provide animals with two reasonable stimuli to determine their position. Recent ndings show that in some regions of the earth both stimuli are varying nearly orthogonal, but these regions are clearly limited by other regions where this approach would not work (see Figure D.1). Nevertheless, some studies show that changing articially either intensity or inclination of the magnetic eldy aects the orientation of certain animals [19, 21]. There is a second hypothesis that states that intensity vs. intensity slope (i.e. the direction in which the intensity varies) could be used [29].
41
feelSpace
Fig. D.2: Chains of biogenic magnetite from magnetotactic bacteria. Adapted from [14].
Fig. D.3: Possible receptor based on a chain of single domain particles. Adapted from [29].
D.4 Magnetoreceptor Mechanisms

The biophysical mechanism underlying magnetoreception is still unknown - primary magnetoreceptors have not yet been identied with certainty in any animal. However, there are three kinds of proposed magnetoreception mechanisms that have been shown to be sensitive enough to detect magnetic elds of earth-strength or weaker [18, 28]. Mechanical reception Suggestions have been made for a ferromagnetic mechanism involving a magnetite-based magnetoreceptor [14]. It might be based on single-domain magnetite crystals twisting into alignment with the magnetic eld (see gures D.2 and D.3), but the hypothesis that they are free to move through all angles has shown to be wrong [31]. Other proposed types of magnetite-based magnetoreceptors include super-paramagnetic particles [13] or magnetite-containing liquid crystals [7]. Direct anatomical evidence is still scarce, although candidate magnetoreceptor cells have been identied in trouts and bobolinks [18, 30]. Chemical Reception The second proposed mechanism is a photoinduced radical pair mechanism where the geomagnetic eld inuences biochemical reactions that involve transitions between dierent electron spin states [18, 27]. Ritz et al. propose a model for a vision-based compass in birds, where the magnetosensory organ consists of a system of radical pairs orientationally ordered in the eye [27]. The assumption is that radical pair processes inuenced by the magnetic eld aect the sensitivity of the light receptors. There is no evidence for or against specic models, but evidence exists for a link between magnetoreception and the visual system [6]. Electric Induction The third mechanism is the principle of electromagnetic induction that has been evoked to explain how elasmobranch sh detect the earths magnetic eld. They are thought to have structures that function as a highly specialized electric sensor. However, there is no direct evidence yet that sh really use this mechanism [18].
42
feelSpace
Fig. D.4: Conditioned choice discrimination of magnetic stimuli: The birds are trained to hop on either the left or the right platform depending on the magnetic eld present during a trial. Adapted from [23].
D.5 Testing for a Magnetic Sense

Dierent experimental methods are applied to nd out if an animal can perceive the magnetic eld and if it uses this eld for navigation. If it is supposed that the animal has a magnetic sense, further tests will be performed to search for the locus of this sense. Conditioning Experiments Conditioning experiments can prove the existence of a magnetic sense. The idea is that if an animal can detect changes in the magnetic eld, it should be possible to use the eld as a conditioned stimulus. For example, conditioning of sharks showed that these animals can detect changes in the magnetic eld [22]. Activation of an articial magnetic eld was paired with the presentation of food at a target area. Following conditioning, sole activation of the magnetic eld resulted in an immediate behavioral response. The sharks started converging the target area. Conditioning plus Impairment Conditioning experiments can also be used to analyze where magnetic perception takes place. This method can be combined with induced impairments. If an impairment of a specic body part prevents the animal from developing a conditioned response, this part may be used by the animal for the perception of the magnetic eld. Homing pigeons are able to sense the magnetic eld. It is possible to train them to show a dierent behavior during the presence of magnetic anomalies than during the absence of these anomalies (see gure D.4). But this discrimination is impaired when magnets are attached to specic locations or when specic parts of the animal are anaesthetized. These experiments show that magnetoreception in pigeons may occur in the upper beak area. [23].
43
feelSpace
Changing the Magnetic Field Another method is to place the animal in a coil system and change the magnetic eld. If the animal uses the eld for navigation, a change in the eld will change the direction of movement. For example, sea turtles in a coil system change their swimming direction when the inclination angle is changed [19]. The same experiments have been done with spiny lobsters, which dont react to changing the inclination but rather the eld polarity of the magnetic eld [21]. Eects of an articial magnetic eld on movement is also reported for various sh species [9] and monarch butteries [25]. To show if animals use a magnetic sense, magnets can be attached to the animals. These magnets block the access to the magnetic eld and so the animal cannot use this cue for navigation anymore. Attaching the magnets to dierent body parts of the animal and thus creating specic disruptions may enable to nd the locus of magnetoreceptors. Attaching magnets to sea turtles resulted in a change in their direction of movement [17]. Changing the magnetic eld has also been shown to inuence the nesting behaviour for certain animals. They appear to prefer certain directions when building their nests. This behaviour has been shown in the Siberian hamster [5] and the blind mole rat [12]. Another method uses pulse-magnetization experiments. There, birds are magnetized with dierent polarities by applying a strong magnetic pulse of brief duration. showed signicant dierences from one another and from their controls in the direction they were oriented [2]. Tracking Satellite tracking experiments are carried out to explore the migration route of animals. The analysis of this route may make it possible to exclude other suggested cues that could help the animals during navigation. These kind of experiments revealed that the migration route of sea turtles is independent of the current direction of the ocean. This is an evidence against the hypothesis that sea turtles use chemical cues for navigation in the open sea, as these cues follow the current [24]. Testing for true Navigation For True Navigation not only a directional sense but also an internal map is needed. Experiments that test for this capability usually use displacement of subject animals and test whether they nd their way home. Such experiments have been successully performed with adult eastern redspotted newts [8] and spiny lobsters [3]. By changing parameters of the magnetic eld, the surrounding light or odours independently, it is possible to test more specically what stimuli are used to construct the map [26]. Using this method it was found that bumblebees use a variety of dierent navigational systems, like vision, smell and also a magnetic sense. Whenever necessary, they switch between them to allow for reliable navigation [4].
44
feelSpace
D.6 Case Study: Navigation in Sea Turtles

Sea turtles travel tens of thousands of kilometers over the span of their lifetime. One species has to swim 2000 kilometers to get from its feeding ground to the natal/nesting region. Another navigational task which sea turtle perform is that they always have to keep within an oceanic region favorable for growth and development. The rst arising question is, how sea turtles are able to navigate in the open sea. They cannot use the stars as navigational cues, because their eyes are too weak. But there are only very few other cues in the open sea. Three hypothesis try to explain this ability: Sea turtles could use chemical cues like an odor plume and it is shown that sea turtles can detect such a cue. They could use the magnetic eld, because they are able to detect dierent components of it. They could use a combination of both cues for navigation. Although it was shown that sea turtles can detect odor plumes [11], it is very unlikely that this is the only cue, because satellite tracking experiments revealed that the migration routes are independent of the current direction [24]. And by using odor plumes one has to follow the current. So, chemical cues could only be one type of navigational cues. To use magnetic eld as the main navigational cue, the sea turtle has to detect any two of the four parameters of the magnetic eld. There is experimental evidence showing that sea turtles can feel the inclination angle [19]. If a hatchling is put in a coil system and the inclination angle is changed, the swimming direction changes. As the inclination angle correlates with the latitude, it could help them to stay in the North Atlantic Gyre. Other experiments show that sea turtle can also sense the total eld intensity [20]. By changing this component, the swimming direction changes . They used intensities of a natural environment, where the sea turtles have to stay in a gyre. The changed swimming directions corresponded to the behavior in this gyre. By combining these cues one gets a bicoordinate magnetic map [16]. In some regions this map would be sucient for a navigation system, because the eld inclination and the eld intensity vary in a dierent manner. All these experiments were performed under very articial conditions. There was no possibility to test the magnetic sense in a natural environment. Now, a new experimental method enables researchers to make experiments in open sea [17]. A magnet can be attached to sea turtles to block the access to the earth magnetic eld. Sea turtles of control groups will carry at the same position a box of the same weight as the magnet. But this does not disturb the magnetic eld perception. With this technique it would perhaps be possible to localize magnetoreceptors, by disrupting the magnetic sense only around a specic part of the sea turtles body. But up to now researchers did only test this method in an articial environment. They showed
45
feelSpace
again that sea turtles seem to use the magnetic sense, because the magnet has an inuence on the direction of movement. But one still does not know anything about the physiological mechanism behind the magnetic sense. It is very likely that sea turtles use this sense and there are studies supporting this view. But it is still unknown how sea turtles could eectively combine their knowledge about inclination and intensity of the magnetic eld.
D.7 A Human Sixth Sense?

In the 1970s and 1980s a number of experiments at the University of Manchester seemed to provide evidence for a human magnetic sense [1]. Participants were blindfolded and displaced to an unknown location. Then, they had to point in the direction of the university, It was shown, that the participants were more successful, when they were blindfolded, then when they were not. The presence of strong magnets seemed to have a bad inuence on their ability. More experiments, involving orientation on city buses and spinning chairs, provided further evidence for a magnetic sixth sense. However, no other research group was able to reproduce the results (cf.[15]). Lately, the interest in magnetic elds inuencing neural behavior in the human brain has been reestablished (e.g.[10]).
46
Bibliography
[1] R.R. Baker. Human Navigation and the Sixth Sense. Hodder & Stoughton, London, 1981. [2] N. Beason, R.C.and Dussourd and M.E. Deutschlander. Behavioural evidence for the use of magnetic material in magnetoreception by a migratory bird. The Journal of Experimental Biology, 198:141146, 1995. [3] L. C. Boles and K.J. Lohmann. True navigation and magnetic maps in spiny lobsters. Nature, 421:6063, 2003. [4] L. Chittka, N.M. Williams, H. Rasmussen, and J.D. Thomson. Navigatin without vision: Bumblebee orientation in complete darkness. Proc. Royal Society London, 266:4550, 1999. [5] M.E. Deutschlander, M.J. Freake, S.C. Borland, J.B. Phillips, R.C. Madden, L.E. Anderson, and B.W. Wilson. Learned magnetic compass orientation by the siberian hamster, phodopus sungorus. Animal Behavior, 65:779786, 2003. [6] M.E. Deutschlander, J.B. Phillips, and S.C. Borland. The case of lightdependent magnetic orientation in animals. The Journal of Experimental Biology, 202:891908, 1999. [7] D.T. Edmonds. A sensitive optically-detected magnetic compass for animals. Proc. R. Soc. Lond., 263:295298, 1996. [8] J.H. Fischer, M.J. Freake, S.C. Borland, and Phillips J.B. Evidence for the use of magnetic map information by an amphibian. Animal Behavior, 62:110, 2001. [9] K. Formicki, A. Tanski, M. Sadowski, and A. Winnicki. Eects of magnetic elds on fyke net performance. Journal of Applied Ichthyology, 20(5):402, 2004. [10] M. Fuller, J. Dobson, G. Wieser, and S. Moser. On the sensitivity of the human brain to magnetic elds: Evocation of epileptiform activity. Brain Research Bulletin, 36(2):155159, 1995. [11] M. Grassman and D. Owens. Chemosensory imprinting in juvenile green sea turtles, chelonia mydas. Animal Behavior, 35:929931, 1987.
47
feelSpace
[12] T. Kimchi and J. Terkel. Magnetic compass orientation in the blind mole rat spalax ehrenbergi. The Journal of Experimental Biology, 204:751758, 2001. [13] J. Kirschvink, M. Walker, S. Chang, A. Dizon, and K. Peterson. Chains of single-domain magnetite particles in chinook salmon, oncorhynchus tschawytscha. J. Comp. Physiol. A., 157:375381, 1985. [14] J.L. Kirschvink, M.M. Walker, and C.E. Diebel. Magnetite-based magnetoreception. Current opinion in Neurobiology, 11:462467, 2001. [15] A. Kobayashi and J.L. Kirschvink. Magnetoreception and electromagnetic eld eects: Sensory perception of the geomagnetic eld in animals and humans. In M. Blank, editor, Electromagnetic Fields: Biological Interactions and Mechanisms, pages 367394. American Chemical Society Books, 1995. [16] K.J. Lohmann, J.T. Hester, and C.M.F. Lohmann. Long-distance navigation in sea turtles. Ethology, Ecology & Evolution, 11:123, 1999. [17] K.J. Lohmann and W.P. Irwin. Magnet-induced disorientation in hatchling loggerhead sea turtles. The Journal of Experimental Biology, 206:497501, 2002. [18] K.J Lohmann and S. Johnsen. The neurobiology of magnetoreception in vertebrate animals. Trends Neurosci., 23:153159, 2000. [19] K.J. Lohmann and C.M.F. Lohmann. Detection of magnetic inclination angle by sea turtles: a possible mechanism for determining latitude. Journal of Experimental Biology, 194:2332, 1994. [20] K.J. Lohmann and C.M.F. Lohmann. Detection of magnetic eld intensity by sea turtles. Nature, 380:5961, 1996. [21] K.J. Lohmann, N.D. Pentche, G.A. Nevitt, G.D. Stetten, R.K. ZimmerFaust, H.E. Jarrard, and L.C. Boles. Magnetic orientation of spiy lobsters in the ocean: Experiments with undersea coil systems. The Journal of Experimental Biology, 198:20412048, 1995. [22] C.G. Meyer, K.N. Holland, and Y.P. Papastamatiou. Sharks can detect changes in the geomagnetic eld. Journal of the Royal Society, 2004. [23] C.V. Mora, M. Davison, J.M. Wild, and M.M. Walker. Magnetoreception and its trigeminal mediation in the homing pigeon. Nature, 432:508511, 2004. [24] F. Papi, H.C. Liew, P. Luschi, and E.H. Chan. Long-range migratory travel of a green turtle tracked by satellite: evidence for navigational ability in the open sea. Marine Biology, 122:171175, 1995.
48
feelSpace
[25] S.M. Perez, O.R. Taylor, and R. Jander. The eect of a strong magnetic eld on monarch buttery (danaus plexippus) migratory behaviour. Naturwissenschaften, 86:140143, 1999. [26] J.B. Phillips. Magnetic navigation. J. theor. Biol., 180:309319, 1996. [27] T. Ritz, S. Adem, and K. Schulten. A model for photoreceptor-based magnetoreception in birds. Biophysical Journal, 78:707718, 2000. [28] T. Ritz, D.H. Dommer, and J.B. Phillips. Shedding light on vertebrate magnetoreception. Neuron, 34:503506, 2002. [29] M.M. Walker, T.E. Dennis, and J.L. Kirschvink. The magnetic sense and its use in long-distance navigation by animals. Current Opinion in Neurobiology, 12:735744, 2002. [30] M.M. Walker, C.E. Diebel, C.V. Haugh, P.M. Pankhurst, J.C. Montgomery, and C.R. Green. Structure and function of the vertebrate magnetic sense. Nature, 390:371376, 1997. [31] W. Wiltschko, U. Munro, R. Wiltschko, and J.L. Kirschvink. Magnetitebased magnetoreception in birds: the eect of a biasing eld and a pulse on migratory behavior. The Journal of Experimental Biology, 205:30313037, 2002.
49
Augmented Reality
Hannes Saal & Christian Stel o University of Osnabrck u
E Augmented Reality
E.1 Introduction
Enhanced or Augmented Reality (abbreviated AR) denes a state where the usual perception of the environment through our bodily senses gets enriched by the help of some external articial systems. The perceived reality of the user of those systems gets enhanced or augmented by additional features. The feelSpace belt would be such a system, as it enriches its users perception of the environment with an additional quality, namely the tactile information about the magnetic north pole. So to speak, the feelSpace belt can be stated to be an augmented reality device. In order to get a clear view on the eld of AR, we performed an intensive literature research. The goal was to stake the eld, to get a clear denition what Augmented (Enhanced) Reality really is, and to get an overview which devices have already been or are being developed within this domain. In particular, we wanted to concentrate on AR systems that augment reality via tactile stimulation in order to gain some insight which kind of vibrotactile devices are being used in the eld, and we looked for orientation-augmented reality, in order to get useful information on technical equipment and testing procedures. The terms Augmented Reality or Enhanced Reality are used synonymously.
E.2 A Denition of Augmented Reality

The term Augmented Reality was coined in the original publication [15] by Wellner et al., where he contrasted it to virtual reality. While virtual reality seeks to immerse the user in a purely synthesized environment, the goal of AR is to augment the real environment with information handling capabilities. In a virtual reality environment, the user is acting in a world that is completely generated by the system. An AR system augments the real world scene by for example tagging physical objects with additional information, that can be accessed by the user with special devices (cf. gure E.2). However, the user maintains a sense of presence in the real world. Thus, an AR system usually needs a mechanism to combine virtual objects and the real world. In this strong sense, AR can be dened as an environment that includes both virtual reality and real-world elements. For example, a user could see the real world as well as some computer generated images projected on top of it by wearing some special translucent goggles (cf. gure E.3). This idea of AR corresponds to the denition stated by Ronald Azuma, which is most often used in research
51
feelSpace
a)
b)
Fig. E.1: (a) A view through a see-through HMD shows a 3D model of a demolished building at its original location [2]. (b) An exemplar Augmented Reality system: A real desk with virtual lamp and two virtual chairs [1].
literature: He denes AR by the following three properties [1]: 1. It combines real and virtual objects in a real environment. 2. It runs interactively and in real time. 3. It registers (aligns) real and virtual objects with each other. However, this denition is limited to only certain parts of the whole eld that AR covers. Other works dene AR as a broadening of human perception via sensors for environmental properties that humans cannot sense themselves, such as radar, infrared, distance vision etc. This wider denition holds also for the augmentation with orientation information as provided by the feelSpace belt. Augmented Reality can be applied to all senses (vision, hearing, touch, ...), although most existing systems only augment vision. Certain applications also remove real objects. For example, an AR visualization of a building that stood at a certain location might overlap the building that exists there today. This is sometimes also called diminished reality, which can be regarded as a special case of mediated reality. Mediated Reality is the generic term for diminished, augmented, or otherwise altered perception of reality.
E.3 Technologies
One example of already existing AR systems is the real-time overlay of virtual markers in sport transmissions on TV, such as the distance to the goal for free kick situations in soccer, or the largest distance achieved in ski-jumping. It has to be noted, however, that these applications would not fall under the strong denition of AR by Azuma, as they are missing the interactive element. The major part of AR applications try to incorporate free movement and unrestricted
52
feelSpace
a)
b)
Fig. E.2: (a) An optical see-through HMD, made by Hughes Electronics [1]. (b) A video see-through HMD [1].
viewing in 3D, rather than having a xed-angle view on the real scene, such as the TV camera. In that way, AR has a clear connection to the eld of ubiquitous computing (abbreviated UC) and wearable computing. In order to move in the augmented environment and interact with it freely, the computations must be carried out mobile, which suggests wearable computing devices for AR systems. In those systems, the user has all computing units attached to his/her body, so that he can move around naturally without being connected to remote computers via cables or having to carry a laptop computer in his hands. UC aims for integrating the computation totally in the environment. UC devices should be able to sense changes in the environment and adapt to them based on the users needs and preferences. Although an environment which is enhanced in such a way could also be called Augmented Reality, the UC approach maintains the notion of explicit and intentional interaction with the computing system, whereas AR focusses on the disappearance of conscious and intentional interaction with an information system. Most AR systems augment the visual sense by using head-worn displays (HWDs), sometime also called head-mounted displays (HMDs). These displays are either transparent or opaque. Transparent displays allow a direct view on the world that is combined with additional information (e.g. virtual objects) projected onto the display. Systems using opaque displays need a camera to create a combination of the real and virtual world (cf. gure E.3). In order to do this, both types of systems need a method for accurately tracking the users viewing orientation and his/her position in the real world. Existing AR systems are usually hybrid systems, in that they combine several sensors, e.g. a compass, video and GPS for environment sensing.
53
feelSpace
E.4 Problems
Up to now, there are mainly prototype AR devices. There already exist some successfully running AR systems, for example surgery aids in the medical application eld. Furthermore, HWDs are already beginning to be employed in avionics, and augmented live television broadcasting is also already established. However, their use is quite often rather limited, and above all, stationary. In the eld of mobile AR, where applications are more numerous and auspicious, most systems are still only at a prototype stage, which is due to the fact that there are still a lot of limitations that need to be overcome. The rst problem, which is common to all mobile and wearable computing devices is the power supply. The currently availably accumulator batteries dont suce to supply power to mobile AR systems over longer periods of time, which is essential to create immersion into the Augmented Reality. Computing power is also often an issue, because real time rendering and tracking of natural image data needed by many systems poses huge demands on computational power. This is usually incompatible with light-weight wearable computing and low power consumption. The trade-o that has to be made in computing power often brings about the problem of latency. Real-time computing sometimes may not be fast enough to adopt to sudden changes in the environment, resulting in temporal incongruence of real and virtual information. Spatial incongruence is also a common problem of enhanced reality systems, which is also refereed to as the registration or the alignment of real and virtual objects. This problem is often caused by inaccurate measurements of the users position and viewing orientation. A lot of the sensors used in AR systems inherently produce errors, as they introduce noise during fast movement, or get otherwise disturbed by the environment, such as electronic compasses get disturbed by local distortions of the earths magnetic eld, issued by ferromagnetic materials. A problem specic to visual Augmented Reality systems using HWDs is data density. The image of the display easily becomes cluttered when lots of information is presented at the same time. Ergonomics is also an important factor, as most of the existing prototypes designed for mobile outdoor navigation are very heavy and cumbersome and could not be used as such in everyday life. As a last point, most systems have still to rely on extensive calibration procedures, which could no longer be accepted if a systems leaves the prototype state and is designed for common use.
E.5 Compasses and Augmented Reality

In this section we review some studies that describe navigation AR systems making use of a compass as orientation sensor. Most compass-related papers we found were about navigation systems, e.g for cars, ships, airplanes. We focused on papers dealing with personal wearable AR systems, in order to look for technologies we could use for our belt, and learning about similar problems that we
54
feelSpace
might encounter with our device. Motion-stabilized outdoor navigation Ronald Azuma and colleagues developed a head mounted tracking device which enables the user to see labels which are placed at the positions of dierent objects of interest. With the help of this system, the user is able to see spatially localized icons and labels superimposed on real world objects, identifying landmarks, path-to-travel and dangerous areas that need to be avoided [3]. The system described in the paper was a hybrid system, consisting of a see-trough HWD, a GPS sensor, a compass (TCM2, PNI Corporation) and gyroscopes for stabilization. Since small motions of the head make the labels unreadable, the compass had to be coupled to several gyroscopes in order to compensate the errors. The fusion of the sensor input allowed to estimate the correct head position and even to extrapolate head movements into the future. Still, some remaining problems are addressed: The system as presented is not easily portable, although it is designed as an outdoor system, because it consists of a lot of dierent components with high power consumption. Despite the stabilization resulting from the hybrid tracking system, the registration errors were still considerable, especially due to user movement and noisy compass output resulting from magnetic eld distortions. [4, 3]. A pedestrian navigation system In most AR applications that employ a compass, the orientation information is used to determine the users direction in the environment towards certain objects which are being tracked (or the user itself). Accuracy of the compass information is hereby a very important factor, as even slight abberations in the compass output can lead to serious misalignments of real and virtual objects or accumulate to severe errors in path integration for navigational tasks. There is only few literature about indoor navigation and navigation in an environment of magnetic disturbances. One researcher that extensively addressed these problems is Quentin Ladetto, who developed a pedestrian navigation system which was further developed by the NATO for dismounted soldier navigation. The solution he puts forward to improve the reliability of the compass azimuth determination is to couple the compass to a gyroscope (similar to the approach taken by Azuma [4, 3]), which can be used to bridge gaps in the heading information when the compass gets disturbed. When the results of using a compass alone are compared to the data of a gyroscope-enhanced device, it becomes clear that magnetic disturbances strongly inuence the compass reliability in indoor and outdoor navigation. [10, 11]. The Pedestrian Navigation System (PNS), which is presented in [10] oers in dierent forms: 3 azimuths (compass - gyroscope - GPS), 2 altitudes (barometer - GPS), 2 travelled distances (dead reckoning - GPS). This setup shows again that reliable information is very hard to achieve in dynamic environments and that it is best assessed with a multiple sensor hybrid approach.
55
feelSpace
E.6 Vibrators and Augmented Reality

In this section we review some of the studies in the eld of Augmented Reality that use vibration to convey information to the user. Technical details are omitted whenever they are not directly necessary to grasp the basic concepts. Feeling music Since music essentially is vibration, it is interesting to see how it would feel if it actually was transferred via the tactile sense. The MIT Media Lab developed a full-body suit with 9 vibrators placed all over the body, which can be activated and controlled independently of each other [9]. In a series of concerts they accompanied real music playback with matching vibrations as well as played pieces of music composed entirely for the tacile sense [8]. For stimulation they used all dimensions available to the tactile modality: frequency, intensity, duration, waveform and space (location on body). The frequency did not correspond to the pitch (of the music), since frequency discrimination is quite bad in human mechanoreceptors; instead it was mainly used as an additional qualitative component. Intensity was used to model the loudness, the location of the stimulation on the body was what mainly corresponded to the pitch. An interesting phenomenon they explored is the selective attention of the skin, where one is able to concentrate on a single point of vibration although concurrent vibration is happening in other places as well. Communicating via touch Vibrational stimuli are highly suited for mobile communication as they have a low social weight, that is they do not disturb ongoing conversations (contrary to cell phones for example). Experimental prototypes using this feature were developed in several studies. Toney et al. present a vibrotactile display in a standard shoulder pad for mobile presentation of information [14]. Their main objective was to explore how to design the shoulder pad to make it small and comfortable. Moreover, it should not be visible from the outside. This was achieved by using very small and lightweight vibrators (Sanko Electric 1E120), which we also use for the feelSpace belt (for details see section F.2). Four to six of these vibrators were incorporated into the pad and in a series of experiments the spatial resolution of the skin on the shoulder was determined (mainly via localization tests). Several design guidelines for mobile vibrational devices were proposed, in order to make it easy and comfortable for the user to decide between the dierent locations of vibration: Equal weight should be placed on each vibrator. The vibrators should be in good contact with the body. Boundaries of dierent regions between vibrators must not be blurred, i.e. the vibrators should not shift into dierent positions. One should take into account that the number of discrete stimulation points on the shoulder is limited.
56
feelSpace
Another mobile device that augments remote communication (via phone) with vibrational cues is the ComTouch system [6]. It consists of a handheld device that is able to convert hand pressure between dierent users in real time. Similarly, the VibroBod device [7] and the inTouch system [5] faciliate interpersonal communication via vibrationial cues. 3D awareness The POS.T-Wear System consists of whole body (torso) vibrotactile devices that are used to increase the presence of virtual objects, in order to support the interaction with moving objects [16]. Virtual objects of dierent shapes (line, disc, sphere) collide with the subject in a way that he feels the contact points of the objects (the plane of intersection respectively, as it moves through). Subjects were asked to estimate the direction and speed of movement of these objects. The vest contains ve rings of 12 electromotor vibrators, spaced at 30 degrees, which are tied together by a rubber band to provide minimal pressure. This is not a mobile system, so weight or power consumption was not a development criterion. Tactual illusions The MIT developed a device to explore tactual illusions on the skin [13]. It consists of a wearable tactile directional display, also called the rabbit display, because it takes advantage of a perceptual illusion, the so-called cutaneous rabbit. This sensory saltation phenomenon describes the feeling of a rather continuous, equally spaced stimulus sensation whereas the same region (e.g. the forearm) is stimulated at discrete places at certain frequencies (as if a tiny rabbit would hop up the arm). They developed a 3x3 vibrators tactile display with 8 cm inter-vibrator spacing, which gets attached to the back. The display is designed to be used in navigation tasks. As the sensory saltation eect was also observable on the back, higher resolution could be simulated, by activating certain vibrators in a row. They employed modied at magnetic speakers (FDK Corp., Japan) as stimulators and a LM383 (National Semiconductors Corp.) as amplier. The stimulation was set at 200-250 Hz, which is reported to be the optimal stimulation frequency for humans. Monitoring displays The Vibro-Monitor is a vibrotactile display for physiological data monitoring [12](cf. gure E.6). Its main eld of use is in operation room environments where for example the heart rate of some patient can be constantly monitored and vibrational alarms can be given if a threshold is passed. It was motivated by the fact that the auditory alarms that are used today are often overheard because of a lot of other noise sources. Vibration on the body would give a more direct way to attract the doctors attention. The device consists of 3 vibrators that are placed 6-10cm apart from each other on the forearm. This distance allowed for good spatial discrimination of the vibrational stimulus. The authors of this study were also able to show the so-called rabbit eect (see above).
57
feelSpace
Fig. E.3: The Vibro-Monitor device. It consists of three vibrating motors and is worn on the users inside forearm. [12]
E.7 Implications of these studies

Although the feelSpace system in not an AR system in the strong sense of Azumas denition, it can be regarded as such in a wider sense since it augments the natural environment in real time with additional information that the human is not able to grasp with his innate sensory organs. Several studies have already explored the possibilities to use compasses and vibrotactile devices for AR systems, and the reported results have certain implications for our system concerning which devices to use, how to employ them and which problems have to be taken care for. Most of the general problems discussed above, like power consumption, latency and ergonomics have also to be dealt with while constructing the feelSpace belt. Implications concerning the use of a compass The studies show that electronic compasses are indeed used in indoor as well as outdoor navigation AR systems. However, some problems arise, which have also been described extensively [10]. In all cases, the compass needed to be stabilized with the help of gyroscopes to compensate for sudden movements of the user. As in our case the compass is likely to be attached directly to the belt and therefore suers from all body movements occurring during natural movement, we should aim for a gyro-stabilized compass system. It also becomes clear that local magnetic disturbances have serious eects on sensitive electronic compasses, therefore we should shield the compass from strong environmental electromagnetic elds. Furthermore, it is recommended to place the compass not in the direct vicinity of other ferromagnetic or electromagnetic feelSpace belt-components. It is not absolutely clear, to which extend the errors of the compass-data may lead to inaccuracies in our device since we do not need absolute precise data if we only use 8 vibrators. Nevertheless we should consider using a gyro-enhanced magnetic sensor since it would be hard to show the subjective eects we want to show if the orientation which is indicated by the vibrators is not coherent with visual, vestibular and
58
feelSpace
proprioceptive information. The TCM2 compass (employed in [3]) could be an appropriate device for our purpose. Further considerations that led to the decision of the compass device we nally settled on for the feelSpace belt are exposed in section F.1. Implications concerning the use of vibrators From these studies, a couple of conclusions can be drawn that may guide of at least facilitate our choice of vibrotactile devices for the feelSpace belt. For small regions such as the ngertips, the use of piezoelectronic buzzers, for larger regions, the use motorvibrators or acoustic vibration devices is recommended [16, 13]. Thus, as we plan to attach the vibrotactile device on the torso, where spatial discrimination is not that sensitive, small motorvibrators that cover slightly larger areas then piezoelectronic buzzers should be optimal. The Sanko Electric 1e120 motorvibrator used in [14] could be a suitable device for our purpose. The minimal spatial distance on the torso reported to be 4cm [16]. Though no specication of dierent resolution on dierent areas on the torso is given, which may also vary from the front to the back region of the torso, 4cm can be regarded as a rough guideline of distance between the vibrators, and there is not much sense in bringing the vibrotactile devices closer together. The minimal temporal resolution (on the torso)is denoted with 200 ms [16]; the optimal stimulation frequency is 200-250 Hz [13]. These values give some hints at which frequency the vibrators should run to provide an optimal stimulation. Another interesting nding are the improved results in the discrimination of objects and their movement direction (see task in [16] for the orthogonal directions (3,9,6,12 on a clock) [16]. This result suggests that activation of the vibrotactile devices of the feelSpace belt for the orthogonal body axes might have a stronger impact than activation of the diagonal vibrator. This idea is consistent with the reported accuracy drop for diagonal directions in the abdomen region [16]. Furthermore, it is suggested that for this region 8 vibrators might be enough. The Cutaneous rabbit eect is also observable on the torso (back): the user has the illusion of continuous stimulus sensation when only discrete stimulation sites are activated; This eect could be used to simulate higher resolution than the actual number of vibrators would allow, but this is only possible with directional activation (meaning the vibrators get activated in a row), and not with spontaneous direction changes [13]. It is unclear, however, how this eect will inuence the continuous sensation of the north-indicating vibrator in the feelSpace belt.
59
Bibliography
[1] R. Azuma. A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6:355385, 1997. [2] R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre. Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6):3447, 2001. [3] R. Azuma, B. Ho, H. Neely III, and R. Sarfaty. A motion-stabilized outdoor augmented reality system. In Proceedings of IEEE Virtual Reality 99, pages 252259, 1999. [4] R. Azuma, J.W. Lee, B. Jiang, J. Park, S. You, and U. Neumann. Tracking in unprepared environments for augmented reality systems. Computers & Graphics, 23(6):787793, 1999. [5] S. Brave and A. Daley. intouch: A medium for haptic interpersonal communication. In Proceedings of CHI97, pages 363364, 1997. [6] A. Chang, S. OModhrain, R. Jacob, E. Gunther, and H. Ishii. Comtouch: design of a vibrotactile communication device. In DIS 02: Proceedings of the conference on Designing interactive systems, pages 312320, 2002. [7] K. Dobson, D: Boyd, W. Ju, J. Donath, and H. Ishii. Creating visceral interaction in mediated spaces. In Proceedings of CHI01, 2001. [8] E. Gunther. Skinscape: A tool for composition in the tactile modality. Masters thesis, Massachusetts Institute of Technology, 2001. [9] E. Gunther, G. Davenport, and S. OModhrain. Cutaneous grooves: Composing for the sense of touch. In Proceedings of the 2002 Conference on New Instruments for Musical Expression (NIME-02), pages 16, 2002. [10] Q. Ladetto and B. Merminod. Digital magnetic compass and gyroscope integration for pedestrian navigation. In 9th Saint Petersburg International Conference on Integrated Navigation Systems, Saint Petersburg, Russia, 2002. [11] Q. Ladetto, J. van Seeters, S. Sokolowski, Z. Sagan, and B. Merminod. Digital magnetic compass and gyroscope for dismounted soldier position & navigation. In Military Capabilities enabled by Advances in Navigation Sensors,
60
feelSpace
Sensors & Electronics Technology Panel, NATO-RTO meetings, Istanbul, Turkey, 2002. [12] J.Y.C. Ng and J.C.F Man. Vibro-monitor: A vibrotactile display for physiological data monitoring. Unpublished. [13] H.Z. Tan and A. Pentland. Tactual displays for wearable computing. In Proceedings of the 1st IEEE International Symposium on Wearable Computers, page 84, 1997. [14] A. Toney, B.H. Thomas, L. Dunne, and S.P. Ashdown. A shoulder pad insert vibrotactile display. In Seventh International Symposium on Wearable Computers, 2003. [15] Pierre Wellner, Wendy Mackay, and Rich Gold. Back to the real world. Commun. ACM, 36(7):2427, 1993. [16] U. Yang, Y. Jang, and G.J. Kim. Designing a vibro-tactile wear for close range interaction for vr-based motion training. In Proceedings of ICAT 2002, 2002.
61
Hardware
Christine Carl & Tobias Kringe & Christopher Lrken & Robert o Mrtin a University of Osnabrck u
F Hardware
To have a tool at hand that we could use for checking our hypotheses, we decided to build a belt-like device that delivers a global heading information, i.e. the direction of north, by vibro-tactile stimulation to the user. The belt needs to provide a continuous and reliable orientation information over a period of several weeks. This is highly demanding in terms of quality and design of the hardware that then is used by our subjects. First of all, the information has to be provided reliably and fast to account for quick movements of the subjects. Moreover, the belt should be easy to wear and use over quite long periods of time. This includes considerations like battery life, wearing comfort and skin friendliness. Regarding its functionality we settled for a design that uses an electronic compass which is read out by a control unit that maps the heading information to a set of vibrators covering the torsos perimeter. This yields that a user can sense the direction of north as a vibration on the waist in a way that the vibrator on the belt which is facing north is active. (see Fig. F.1) As the design and assembly of the belt hardware was a major part of the projects workload, we formed several workpackages to tackle dierent subproblems. Two groups were assigned to do research on the compass and vibrators - these components where the most crucial design decisions to make, putting restrictions on the further design of the belt and control electronics. A third group had to design the control unit that interfaces the compass and vibrator components and to choose and assemble the belts. In the following sections we will illuminate the
Fig. F.1: A schematic view of the feelSpace belt. The vibrator pointing north will vibrate.
63
feelSpace
work of the dierent groups, starting with the dierent components of the belt: The compass group, the vibrator group and the belt-logic group. Eventually the nal belt design and assembly will be discussed in section F.4.
F.1 The Compass

F.1.1 Introduction
One of the most fundamental decisions we had to make was the choice of the component measuring directional information and making it available to the other components. This component had to fulll a number of criteria. It should work both in- and outdoors. It had to be precise, portable, integrable and low in power consumption. Furthermore, the component should be shielded against environmental interference (movement, tilt, magnetic or electric elds) or be able to compensate for it.
F.1.2 Literature Search

In order to nd an orientation sensor fullling the above criteria, an extended literature search had to be carried out. Both the Technical Group and the Enhanced Reality Group E were involved in sifting through the material, most of which was found via the internet. The rst coarse skimming of the available literature revealed that we had two options to choose our direction sensors from, namely either a GPS-receiver or a digital compass.
F.1.3 GPS
The Global Positioning System (GPS) is a navigation system based on receiving information about the constellation of satellites via radio signals. Even though it can very accurately determine ones location, there are obvious shortcomings to a system depending on the availability of satellite signals. These signals are not always readily received inside buildings. Q. Ladetto and B. Merminod addressed this problem in their works and proposed the use of a digital magnetic compass for in- and outdoor pedestrian navigation [14][13]. (This system was developed further in cooperation with the NATO for military use [15]) Furthermore, GPS systems are good at nding your position, but most of them use a built-in compass to measure heading direction.
F.1.4 Digital Magnetic Compasses

We followed Ladettos advice and discarded the rst of our two options, leaving the use of a digital compass as the sole possibility.
64
feelSpace
Aside: Augmenting Sensors In the publications mentioned above, Ladetto and Merminod furthermore augment their digital compass with additional sensors to rid the raw compass data from the inuence of local magnetic disturbances and errors resulting from fast movements or tilt [14][13][15]. Augmenting sensors could either be gyroscopes which measure the rate of rotation or accelerometers measuring the earths gravitational force. These sensors are, of course, prone to error as well (e.g. drift in gyroscopes). Nevertheless, combining diverse sensors makes an orientation device more and more reliable. At this point in time we did not know whether we required any additional sensors or if a plain digital compass would be sucient for our needs. Sifting through the Market Our next step was to get an overview of the market for digital compasses. Unfortunately, most devices containing a compass were designed for outdoor application exclusively. Additionally, a lot of digital compasses serve as components in navigation systems for ships, automobiles or airplanes. Low power consumption has never been a major criterion in these environments. Thus, products from this domain could be excluded. Compasses using only two axes (realized with two orthogonal sensors) were of no use either, as these are easily disturbed by even the tiniest tilt. A three-axes compass seemed to be a better choice. These devices use three perpendicular axes and have an extended tilt range which is measured by an accelerometer or a liquid lled tilt sensor. The Plus-Compass We bought a cheap electronic compass from the local Plus supermarket to get an insight into the diculties that might occur using a digital compass in our usual environment. The performance of the Plus-Compass conrmed the conclusion of our literature search. This plain two-axes compass was extremely susceptible to tilt. Furthermore, using non-magnetic sensors to augment the compass appeared to be useful, as metal and electrical devices disturb the weak magnetic eld of the earth. These disturbances are abundant in our daily environment.
F.1.5 The Candidates

Bearing the above constraints in mind, we focused on three remaining products. Honeywell - Honeywell Magneto-Resistive (HMR), HMR3300 [8] PNI Corporation - TCM2 [6] MicroStrain Inc. - 3-DM-G-X1 [9] All of these compasses support to measure a third axis in space, thus becoming more resistant against tilt. We took a closer look at each of the candidates.
65
feelSpace
Honeywell - Magneto-Resistive (HMR), HMR3300 The data sheets of the HMR3300 looked very promising. Besides very small size and weight, it featured a two-axis accelerometer to detect tilt and compensate for it (tilt range of +/- 60 degrees) [8]. We refrained to buy a HMR3300 because communication with the Honeywell customer service was not satisfactory. Neither were we given the opportunity to test their product before buying it, nor was it guaranteed that the data sheets contained the correct specications. PNI Corporation - TCM2 In an augmented reality project, Ronald Azuma and colleagues developed a head mounted tracking display, which enabled the user to see virtual labels which seemed to be attached to dierent objects in the real world [1]. Azuma and colleagues used PNI Corporations TCM2, a three-axes compass with a liquid lled tilt sensor, to measure three dimensional head movements. Just like Ladetto and Merminod [14][13][15], Azuma augmented his magnetic compass with an additional sensor (a gyroscope) to compensate for magnetic disturbances. His results showed, that it was worth considering to use a gyroscope as well. The TCM2 tolerates tilt angles up to +/-50 degrees [6]. A potential disadvantage is the liquid lled sensor. During abrupt changes in movement velocity the liquid may swash around and disrupt the measurement [6]. Cooperation with the PNI corporation was very pleasant. The customer service answered all of our questions and PNI even provided us with a free TMC2-50 in exchange for some information and public relation work. MicroStrain Inc. - 3-DM-GX1 The 3-DM-GX1 is the compass we nally settled with. Our choice to buy the 3-DM-G compass was mainly inuenced by communication with Jrg Conradt, o a PhD student at the Institute of Neuroinformatics - ETH Zrich [5]. Being u experienced with this compass, he provided us with valuable advice. Jrg visited o us and held a talk on his Neuromorphic Robots, some of which used the 3-DMGX1. The 3-DM-GX1 is a gyroscope-enhanced orientation sensor providing a tilt range of 360 degrees over all three motion axes. Essentially, it works in every possible position. The various sensors (magnetic compass, gyroscope, accelerometer) can be combined to give a best estimate of orientation, compensating for the shortcomings of each individual sensor [9]. Furthermore, MicroStrain Inc. have tested the 3DM-G when worn on the human body. The compass was attached to a subjects knee while walking, jumping and rising from a chair. The results were very convincing. Provided that certain constraints are met, the 3-DM-G sensor is stable and accurate enough to measure the kinematics of human gait [4]. This property, we thought, could be very benecial to a device worn on the human torso.
66
feelSpace
Fig. F.2: The MicroStrain 3-DM-GX1 compass, a Gyro Enhanced Orientation Sensor [9].
F.1.6 Testing the PNI Compass

As soon as the sponsored TCM2 compass had arrived, we gave it a thorough testing. It was the rst high-end magnetic compass we laid our hands on. We had to know how reliable and quick it would be if worn on the body of a behaving person. General Testing Data acquisition At rst we explored the general functionality of the compass and had to nd a general procedure of measurement. As the continuous stream of data from the device could not be logged by the compass itself, a mobile computer had to be carried along as well. We chose to have this computer carried by a second person, thus ensuring maximal freedom of movement to the person wearing the compass. Compass output was measured during various activities such as walking, running, jumping or climbing stairs. These activities took place both indoors and outdoors. Furthermore, we tested the compass in environments containing a lot of metal such as elevators. Results These initial tests yielded the following results: Activating the built-in damping mechanism of the compass is a must during motion. When damping is activated, even the directional changes during abrupt movements are recorded in a quite smooth fashion.
67
feelSpace
Fig. F.3: The built-in damping function of the TCM2 compass is most benecial during rapid body movements. In this case, the wearer was jumping with damping either turned on (Sample 50-250) or o (Samples above 290). The sudden jump in the middle of the x-axis is a result of a 180 degree turn we performed to be able to discriminate between these two conditions.
Outdoor Testing Data acquisition To see how well the compass behaved in natural environment, we measured its activity during the traversal of an outdoor parcours. Its shape was quite simple, consisting of two straight line segments (length 23m each) interrupted by a circle (diameter 5m). This parcours was traversed in both directions at normal walking speed. The compass was worn alternately on the abdomen, hip, shoulder or head (attached to a helmet) to have a broader variety of movement kinetics act upon the device. Results By using a simple path - reconstruction algorithm, we were able to verify that the information acquired at the dierent parts of the body were congruent with one another and clearly contained reproducible path information. However, the sampling rate of the compass displayed an alarmingly high variance, as the recording application sometimes appeared to freeze for seconds. The origin of this problem is still an open question. Indoor Testing Data acquisition We were interested in the TCM2s susceptibility to magnetic perturbances present in our everyday environment. A further question
68
feelSpace
was whether these perturbances were consistent across repeated traversals of the same environment. We measured heading information during the mounting of a staircase connecting two storeys in a university building. After this, two doors had to be opened and passed. Both the railing of the staircase and the doors were fashioned of metal. Again, this path was walked multiple times and with dierent ways of wearing the compass. Results We were able to reproduce both positive (consistency of overall path data) and negative (inconsistencies on a smaller time-scale due to discontinuous sampling) aspects of the rst test. Furthermore, we noticed a considerable delay in reaction time. As mentioned above, the built-in damping algorithm of the TCM2 is a must for acquiring a stable signal during movement. We assume that this damping algorithm may cause the delay by averaging (or by performing a similar operation) over samples. This delay makes the TCM2 less suitable for a device that has to transmit directional information in real-time.
F.1.7 The Decision

The discontinuous sampling displayed by the TCM2 might have been a result of using the Windows XP operating system to run the sampling software, as this software had been developed for MS-DOS. Nevertheless, we were not willing to rely on this hypothesis and settle with the TCM2 anyway. As the temporal delay for the accurate sampling of directional information has to be minimized, temporal averaging (as it is done in the TCM2) may not be an optimal method to rid the signal of to noise and sudden uctuations. The 3-DM-GX1 oered by MicroStrain Inc. uses augmenting sensors to smooth the signal in real-time, thus providing reliable information with minimal delay. To eventually decide which compass to use, we tested both, the TCM2 and the 3-DM-GX1 with an early prototype of our orientation device. The TCM2 proved to be very slow in adaptation of directional changes. It was furthermore easily impaired by rapid movements or percussion like fast walking movements. The 3-DM-GX1, on the other side, convinced us immediately by its performance. It is relatively unaected by fast movements due to the built-in gyroscope and it reacts much faster. As the 3-DM-GX1 was moreover recommended to us by Jrg Conradt, we chose o it to be part of the feelSpace device.
F.2 Vibrators
F.2.1 Criteria for Evaluating Stimulation Devices
We evaluated the currently available tactile feedback devices according to the following criteria:
69
feelSpace
Safety and eects of long-term use Some devices currently used produce sudden pain or use high voltage which could be dangerous for people exposed to them. Noise level Since the belt has to be worn for the whole day in everyday life, the noise produced by the tactile device should be as low as possible. Size and weight The devices have to be light-weighted and small since we would like to use several of them that have to be carried around. Compatibility with compass Some tactile devices cause magnetic elds that may interfere with the compass. Power supply The devices should not have a too high power consumption since we power can only be supplied by battery packs. Preferably they should require the same voltage as the belt logic for that they can use the same power source. Price As we want to have four belts with at least eight vibrators, the price of the devices should not be too high.
F.2.2 Overview of Tactile Feedback Devices

Several types of tactile devices can be distinguished. Those relevant for our case include electromagnetic motors, piezo-electric devices, electrocutaneous systems and pneumatic devices. Electromagnetic motors and electrocutaneous systems have been used since the beginning of research in the area of tactile devices (Kaczmarek er al. [10] gives a very good introduction into both types as well as an extensive comparison). Only recently began the widespread use of piezo devices, mainly due to the technical progress that drastically improved their size and limitations of the high voltage needed. (cf. [19]). Electro-Magnetic Motors These devices use a counterweight attached to an electromagnetic rotating motor to generate the vibration. They are lightweight, have a low consumption and are generally very cheap. Moreover they are very easy to use and are therefore used in most tactile feedback systems (cf. e.g. [17]). Problems for our purpose arise from the fact that their induced magnetic eld might disturb the compass. Piezo-Electric Devices In some types of crystals, the application of an electric eld can induce distortion1 . So-called piezo-electric vibrators exploit this principle to generate mechanical vibration. Due to major technical improvements in recent years, those devices are nowadays as small and lightweight as electro-magnetic motors. Nevertheless, the voltage required is pretty high. One
1
Distortion is, in this context, some kind of mechanical stress.
70
feelSpace
major disadvantage is that they are very expensive in comparison to e.g. electromagnetic motors. Electro-Tactile Devices The mechanoreceptors in the skin can be directly activated by passing a local electric current through the skin. Kajimoto et al. [11, 12] claim that the dierent types of receptors can also be activated independently by varying the size of the electrode and the polarity of the current. Electrocutaneous devices also have a low power consumption, and they do not possess any mechanical parts. However, some problems arise with this technique as presented by Kaczmarek et al. [10]: Electro-tactile devices may cause skin irritations and even sudden pain, if pain receptors are activated by the electric current in addition to the desired activation of mechanoreceptors. The exact cause of these phenomena is still not clearly known. Moreover, the amount of current that should be used and the sensation that it generates is still unclear and unstable. Finally, the available systems are not recommended for long-term use. Pneumatic Systems Another method to stimulate the skin mechanically is by means of small pockets attached to the skin, that are dynamically lled with air. This technique allows the application of pressure as well as vibration. Pneumatic systems do not induce a magnetic eld. This represents a major advantage for both, the compatibility with a compass as well as the possibility to do fMRI experiments2 . Briggs et al. [2] and Zappe et al. [21] recently demonstrated this property. Because of the high consumption of compressed air, pneumatic systems are not suitable for mobile use though. Maucher [16] reports that his subjects complained about the pneumatic device being to bulky and uncomfortable. Furthermore, pneumatic devices are most probably too expensive for our purpose. After having presented some commonly used devices in tactile feedback systems, the suitable device for our system should be discussed. As stated above, pneumatic systems as well as electro-tactile devices seem to be inappropriate for long-term use, mainly because they are either user unfriendly or even harmful for the user. We therefore shortlisted electro-tactile motors as well as piezo-electric devices. Compared to electro-tactile motors piezo-electric devices were more expensive and hard to get. Therefore, our choice for a rst prototype was a electro-magnetic device: Sanko Electric 1e120. Above all we tested the compatibility with the compass, as a possible inuence of the vibrator on the compass seemed to be its major shortcoming.
71
feelSpace
a)
b)
Fig. F.4: (a) Sanko Electric 1e120. At the side one can see the contacts where the cable has to be attached. (b) Final version of the Sanko vibrator. The vibrator is glued into a cut down Coca-Cola cap.
F.2.3 Sanko Electric 1e120

Characteristics of the vibrator The electro-magnetic motor Sanko 1e120 was the rst prototype we chose (Fig. F.4). This type of vibrator was also used in another vibrotactile device (a shoulder-pad with vibrators inserted, see [19]). The vibrator has a size of 14,3x3,5mm, it requires a voltage of 2.4-4.8V with a current uptake of 30mA. Advantages of this particular electro-magnetic device are: Is is relatively small and light-weighted. The vibrator is very cheap. Security risks for the user are very low as it uses only low voltage. The voltage required is similar to the voltage used by the belt-logic. Construction of Mountable Vibrator The vibrator is very small and it proved to be somewhat dicult to attach a cable to it as its contacts tend to break o (see Fig. F.4). Therefore, we came up with a container-like structure that surrounds the vibrator and provides the stability needed for a robust functionality. This container was made of bottle caps of Coca-Cola 1.5l or 0.5l plastic/glass bottles. The vibrator with the cable already soldered to it has to be glued in the cut-down cap (see Fig. F.4). This prevents the solder joint from breaking o. We tried several dierent types of glues before we found the one fullling our demands.
2
Concerning the feelSpace project this might become interesting in future experiments.
72
feelSpace
Acrylic glue seems to get not hard enough so that the solder joints can break. Furthermore, the glue loosens the contacts of the vibrator. Two-component glue did show similar eects to acrylic glue. Hot-melt adhesive turned out to be the most suitable glue although one problem we encountered was the high temperature. If the glue is lled into the cap too quickly, the high temperature seems to destroy the vibrator. Slowing down the process, however, works ne. Performance and Compatibility Research on the psychophysical properties of the skin around the abdomen indicates that vibration thresholds are nearly consistent, i.e. detection of vibration is equally good on every position [3]. A frequency around 250Hz showed the lowest thresholds, but all other frequencies tested from the range 25-320Hz easily elicited responses as well. Dierences of performance in localization tasks depending on vibrator site were also examined. It turned out that middle center (belly button region) and back center (on spinal cord) could best be localized, while performance decreased with distance from those two reference points. Moreover, performance of localization decreases as the total number of vibrators around the abdomen increases [20, 3]. This could be due to the navel and the spine being the only reference points around the abdomen [18]. We tested the devices for a period of several hours on the skin and found no eects worse than adaptation to the vibration. This adaptation, however, was most probable due to the fact that only that single vibrator was activated for the whole period of the test. Tests conducted with the rst prototype of the belt for several days did not lead to any problems concerning the vibrators
F.3 Belt-Logic
F.3.1 Requirements
The basic function, which the belt logic has to provide is the interface to the compass and driving of the vibrators with regard to the heading. For performing experiments with the belt, we also wanted to have an interface from outside, to control the belts behavior with an external computer. As the belt-logic is the central electronic unit of the belt, it also has to provide power to all other components, e.g. the compass and vibrators.
F.3.2 Basic Design Issues

As the main tasks of the control unit are the communication with the compass or a PC via a serial connection and to perform some simple calculations to map the read data to the vibrators, many dierent realizations where possible and taken into consideration in the beginning.
73
feelSpace
A pocket PC, or PDA, could have been used as the computational main unit. As there are models available which already provide serial ports, the implementation of the compass communication would have been very ecient. A PDA allows the usage of an high level programming language and provides means of visualization for simple debugging. The problem of such a solution is its large distance to the hardware layer. For driving the vibrators it would have been necessary to have an other device which can be interfaced via a serial or parallel bus, as a PC does not provide any ports powerful enough to drive the vibrators directly. Because a PDA uses an internal power supply, another independent circuit would have been necessary to provide power to the compass and vibrator control. In addition a PDA has a high power consumption itself. No model on the market can be used for about eight hours continuously, as it was required for our long term training. A kind of contrary approach would have been to solely use low level hardware. As the computational demand of the control unit is very low, a simple 8 bit microcontroller could easily perform the needed computations. Without any high-level components as a PDA involved, driving of the vibrators and a centralized power supply are easily achieved. With the very low power consumption of small microcontrollers, the compass and the vibrators would have been the only signicant consumers. While such a low level solution would have been the best scaled for our purposes, the developmental costs are a lot higher than for a solution using high level hardware. The many external components that have to be used in addition to the microcontroller introduce a very hard to control complexity and many unaccessible error sources. For the small series of only four devices that were needed for the experiments, the development of such a system from the scratch would not have been feasible. The Conrad C-Control Unit II is an intermediate solution: The unit itself already integrates a micro-controller and necessary hardware for power regulation and RS232 serial communication. It provides us with a high level, multi-threaded programming language. Its program is stored in an on-board ash memory which can be written from a PC via serial connection. At the same time, the unit can be integrated into an electronic circuit, allows a centralized power supply, supplies regulated 5V power to the vibrator drivers and has a relatively low power consumption itself.
F.3.3 Hardware Design

The heart of the hardware is the C-Control Unit II from Conrad Electronic. The 82x60mm large unit sits on an circuit board which provides additional components for attaching power supply, vibrators and RS232 serial connections. The RS232 standard species an asynchronous serial protocol that is used for
74
feelSpace
data exchange between two systems. Hereby, a serial port sends and receives data one bit at a time over one wire. As the connection is asynchronous, there is no additional clock signal. The sender and the receiver have to use the same transfer rate. Typically, ports use 9600 baud (bits per second). The 3DM-G compass utilizes higher transfer rates of 38400 baud. The C-Control II Unit has two serial ports: A hardware port is used or the basic communications and for programming the unit. As the hardware port is the only port that is able to communicate at the speeds that are demanded by the compass, it is also used for the compass connection. A second serial port is emulated by the C-Control operating system. As the software emulation only allows transfer rates of 9600 bauds we use this port only for additional PC-belt communications during experiments. The IO-pins of the C-Control unit, which are used for serial communication, are connected to RS232 connectors on the circuit board, where the connecting cables can be plugged in. The C-Control Unit has 2x8 directly addressable IO-ports. Two of these are used by the software RS232 port. We use the other 14 ports to control the 13 vibrators and a status LED. As the ports power load is limited to 5mA and the vibrators consume about 40mA at 5 volts, the ports can not be used to directly drive the vibrators. We use four L293NE motor drivers, each able to drive four consumers with up to 1A. These are connected to 26 connectors on the board, to which the vibrators can be attached. The 5V regulated voltage that is needed by the motor drivers, vibrators and the status LED is generated on board of the C-Control unit. The C-Control II unit, as the compass, needs at least 8V supply voltage. This is provided by ten 1.2 AA-batteries, resulting in about 12V. On board of the C-Control II Unit a regulated 5V voltage is generated. This regulation is necessary, because the voltage provided by the batteries depends very much on their charging. The 5V power source can also be used by external consumers with up to 100mA continuous load, which is sucient to drive the additional hardware and up to two vibrators simultaneously. The circuit board that we used for our rst prototype was handwired. For the three additional units that were needed for the experiments, we designed a printed circuit board, which minimized the dimensions of the electronics and increased its reliability. The boards where manufactured by the Elektrotechnische Feinwerkstatt of the University of Osnabrck. We designed the board using the Eagle u Layout Editor from CADsoft. The two-sided design integrates all electronic components that are used except the compass, the vibrators and the battery pack.
75
feelSpace
Fig. F.5: Top side of the BeltLogic 1.0 circuit board.
Fig. F.6: Bottom side of the BeltLogic 1.0 circuit board.
76
feelSpace
F.3.4 Software Design

The C2 Programming Language Conrads C-Control II Unit comes with a (simple) high-level programming language, C2, and an IDE for compiling and transferring of programs to the unit. The language allows an object-oriented programming style, as dierent sourceles, or modules, can be included and used as detached units. The language itself is not object oriented, though, as it basically only supports global variables and functions. Its power is the possibility to write multi-threaded programs in a simple, straight foreward fashion. System libraries provide functions for dealing with all necessary hardware and operating system communications. The C2 language is easy to learn and allows fast development. Unfortunately, its simplicity is also its major drawback, as a couple of bugs within the system itself can hardly be addressed on the abstracted level of this programming language. Especially the multi-threading and the serial communications have major problems in the original rmware provided by Conrad. This resulted in several days of debugging, without nding any solution, neither in the hardware nor in our software. The solution are inocial driver and operating system updates that were developed by Andr Helbig [7] and removed the bugs within the les e provided by Conrad. Another problem induced by the programming language C2 is its low performance. The 20MHz, 16bit processor of the control unit should be more than sucient to do the few calculations and bus readings which are necessary for normal operation. As also the original system libraries are written in the C2 language, serial communication with the compass was too slow before we used the optimized routines of Andr Helbig. All these issues make the C-Control II e Unit an unsatisfactory choice, as a simple assembler programm would have done the same job on a much cheaper and smaller hardware. Programm Requirements On the one hand, the programm had to provide the basic belt function: Reading the compass via the hardware serial port and addressing the vibrators IO-ports accordingly. For our experiments, two additional features were required: A remote mode, that could be initialized via the software serial port, used for the virtual maze task, and an erroneous mode, where the actual heading could be distorted by a certain number of degrees, which was used for the posturography experiments (see appendix K). In any case, the most important aordances to the programm were its response time and its reliability. The latter point is important, because the programm should basically work for hours continuously without any need of human interference. As the serial port generates a large data load with a relatively high error rate, the stability of operation was a major issue that had to be addressed. Ob-
77
feelSpace
viously the reaction times are very important for our application. To allow the subjects to experience the belts feedback as a direct reaction to their actions, there should, in the best case, be hardly any time lack. As the computational load of mapping the raw compass data to the vibrators is relatively small, no signicant time lack should occur during the processing (in addition to the reaction times already introduced by the compass and vibrators). Programm Description As the programming language supports multithreading, we split up the whole problem into three subprograms. One thread is used to continuously read data from the compass and to update a variable with the heading. A second thread checks for input at the software serial port, reads commands from PC and sets options and belt heading accordingly. A third thread controls the outputs: It sets the vibrators and writes the compass heading to a PC, if logging is activated during experiments. The source furthermore is split up into ve modules: constants.c2 This module denes constants that are used by all other modules. This le species the version of the circuit board that is used (as the rst prototype slightly diers from the printed board version), buer sizes for the serial ports and similar things. main.c2 This module contains three important parts of the program. The rst initialization is done here. After initializing basic functions, the compassand the PC-connection threads are launched while the main threads takes over the vibrator control and logging. The main thread runs through the main loop: First the it is checked, if a heading has been read from the compass or the PC. Depending on which mode is active, the compass or the PC heading is translated into a vibrator activation and is sent to the IO-ports (using vibrator2.c2). If logging is active, the heading used for the vibrators is transformed into a string and sent via the software serial port. The functions for the PC-Connection are also located in the main module, as they need access to the same variables. The PC thread continuously checks for input for the software serial port. If a command is detected (rst character of the received message), the parameters are read and internal ags and variables are set accordingly (heading, mode, oset, etc). The heading and oset strings are transformed into integer representations of the heading by using functions of the module transform2.c2. compasscon2.c2 Compasscon2 provides functions for initialization and reading from the compass. Version 2 is optimized for communicating with the 3DMG compass, while compasscon.c2 was adapted to the format of the TCM2 compass.
78
feelSpace
The initialization function initializes the hardware serial port and puts the compass into non-continuous mode. That means, the compass only sends its present heading once if it receives the corresponding command. We avoid the continuous mode, in which the compass transfers its heading all the time and at a high frequency, because the program is not able to handle such a high data load. The compass main loop sends the command which lets the compass transfer its present orientation and reads the data from the hardware serial port. The input string is transformed into an integer value, which holds the present orientation by using the transform2.c2 module. If communication fails a couple of times, the connection and compass is preinitialized. transform2.c2 Provides functions for transforming character buers, that are read from serial ports in other modules, into integer heading values. Three transformations are available: The 3DM-G protocol, PC-Protocol (which basically codes the heading as plain text) and the Quake-Protocol (also used by the PC connection but more ecient than the plain text protocol). The TCM2 compass format is supported by the obsolete transform.c2 module. vibrator2.c2 Provides a function to map an heading integer to an arbitrary number of vibrators. The active orientations of dierent vibrators can be set in an array. The default number of vibrators is thirteen with only one vibrator active at a time. This is due to our experience, that increasing the resolution by having two vibrators active simultaneously at transitions from one vibrator to the other is not suitable as two vibrators at he same time feel to strong. Two old functions for driving eight vibrators with one or two active vibrators at a time are also available.
F.3.5 Power Supply

The aim of the power supply is to provide enough power for a continuous operation of more than ten hours. Power Pack Conception We used an array of standard 1.2V NiMH rechargeable batteries as supply unit. This solution is relatively cheap and scalable. The overall charge needed to drive all electric components of the belt for ten hours can be approximated as follows: The C-Control II Unit has an uptake of about 80mA, the vibrators need about 50mA and the 3DM-G typically 90mA. Therefore, the batteries have to supply about 2200mAh each for ten hours of operation. To achieve an supply voltage of 12V (which minimizes the current uptake compared to possible lower voltages), we used a series connection of 10 AA-cells with about 2200 to 2400mAh each.
79
feelSpace
Operation Every subject was provided with three power packs. As one pack should provide enough power for an whole day of operation, this allows an ideal charging cycle with one charger. The AFX-Charger allows variable charging currents. Therefore the charge time can be adjusted exibly - constant currents between 200 and 500mA result in charging times between 10 and 4 hours. While one pack is being charged over night or during the day, another pack can be used to power the belt while the third pack is used as a backup for the currently used pack. A clock timer allowed the subjects to start the charging, which normally has to be timed manually with the AFX-Charger, without worrying about switching it o later.
F.3.6 Enclosure
To nd an enclosure for the control unit which had the right dimensions and tted our needs in terms of stability and wearing comfort was a major problem. The rst prototype used an old butter box and later a simple bread box, which we purchased in a household supply store. While this was a cheap solution, neither the appearance nor the size of the resulting unit was very satisfactory. As the printed circuit board that we used for the three following belts minimized the dimensions of the electronics, we also had to nd an enclosure that more tightly tted these. We nally settled on an aluminium enclosure with black powder coating, the Teko - Tekal-T series. Its dimensions of 265 times 105 times 45mm are a perfect match for the electronics. As the enclosure has plastic caps on the left and right sides, we could insert a power switch, LED ttings and openings for the RS232 connectors as well es for the vibrator cables.
F.3.7 PC interface
Bluetooth Adapter In order to allow us to remote control the belt during experiments even if the subjects had to be able to move around freely, we replaced the serial cable by a serial-serial bluetooth bridge. This wireless connection mimics the behavior of a standard RS232 port. It turned out, that the bluetooth hardware has some severe problems, which make it almost useless in experimental conditions. As the nal experimental design had no need for a wireless connection anymore, this option was left out of the nal belt design. BeltConnect Software The BeltConnect software interfaces the belt from a PC via serial connection and allows it to be remote controlled. We developed the BeltConnect software as a Windows based C++ console application, which reads script les, interprets them and sends commands to the belt accordingly. The programms script provides commands to set the belt heading to a certain angle, to shift the compass angle
80
feelSpace
Fig. F.7: The circuit board with attached C-Control II Unit.
Fig. F.8: The Teko enclosure with attached cables.
81
feelSpace
by a certain degree or to let it rotate at a certain speed. The software was used for the posturography and nystagmography experiments. The virtual environment experiments used an other implementation to control the belt, as the computer game which we used for the experiments does not support C++ code. See appendix J for more details.
F.4 Belt
Why a Belt? We wanted to measure the orientation of the bodys main axis. Hereby, we dene the heading of the bodys main axis by the walking direction. Therefore, the compass heading should be correlated to this orientation. While arm-, leg- or head movements are often independent of the bodys main axis, the torsos movements are always directly related to it. Furthermore, the torso constitutes an area with good spatial resolution for sensitivity of vibration. This led to the decision that the compass signal should be mapped by the vibrators to the torso. But how could we attach the 13 vibrators in a stable position to the torso? A rst idea that comes to mind is the use of a belt. A belt does not have necessarily to be attached to trousers like an everyday belt. Therefore, we considered dierent belt types. Type of Belt The belt has to satisfy several requirements: As the belt has to suce long-term usage (several weeks the whole day), it has to be skin-friendly.3 The position of the vibrators needs to stay constant over the whole period of training as loosely attached or slipping vibrators might inuence the integration of the signal. The belt has to provide strong enough support to be able to attach at least some of the hardware components (e.g. vibrators and compass) to it. These devices should be rmly attached to the belt. Because of dierent girths of our subjects the belt should be exible in size, which also requires the vibrators to be adjustable in their position for an equal distribution. An original idea was to make the belt waterproof. However, we rejected this not very crucial requirement fast because it was hard to realize waterproofness from a technical point of view. Since the belt is not worn during
3
At the time of designing the belt we expected the subjects to wear the belt directly on the skin for an optimal vibration sensation. However, during the tests of the prototype it became clear that for a comfortable feeling the subjects preferred to wear the belt not directly on the skin. The tests revealed that wearing the belt over a shirt does not signicantly inuence the sensation of vibration.
82
feelSpace
Fig. F.9: The Thuasne Cemen 2900 orthopaedic belt for women.
night anyway the additional break during showering should not have any strong eect on training results. Alternatives reaching from the usage of a traditional everyday belt to a belt xed by suspenders were considered concerning the belt architecture. We nally decided to use a Thuasne Cemen 2900 orthopedic belt for women 4 (see Fig. F.9). This belt is designed for providing support for rip fracture patients who have to wear it continuously for several weeks on skin. As hence the belt is designed for long-term usage, it is per se made of material that is skin-friendly and comfortable to wear. Orthopaedic belts are designed to give strong support for the body. Therefore, they are made quite stable and do not slip around. This property perfectly fulls our requirement of being able to position the vibrators in static places. Furthermore, this belt is rm and broad enough to allow the attachment of all needed components. It is even strong enough to attach the computational unit as well as the battery pack to it, although most subjects choose to detach the batteries and wear them in their pocket. While providing strong support in terms of spatial stability and attachment of components, the Thuasne Cemen is still very exible as it is made of an elastic material and as it closes with Velcro fasteners. In addition, we ordered three dierent sizes to have optimally tting belts for all our subjects. As we furthermore attached additional Velcro pads for being able of to choose the locations of dierent vibrators freely we achieved a good adjustability to dierent users. Additionally, the belt is very user-friendly: The majority of subjects gave a very positive feedback concerning the comfortability of wearing. As a nice additional feature, the belt is also washable.
4
We chose the model designed for women as it is compatible to both genders.
83
feelSpace
F.5 Specications and Final Device

feelSpace Belt
Belt Device Belt Model No. of Vibrators Radial Resolution Weight (w/o batteries) Weight (with batteries) Operating Time Control Unit Supply Voltage Power Uptake max Uptake Typical Max External Load (5V) Max External Load (12V) Enclosure Enclosure Dimensions Core Unit Compass Connection PC Connection Compass Model Orientation Range Sensor Range Orientation Resolution Accuracy Thuasne Cemen 2900 (for women) 13 < 28 degree approx. 800g approx. 1200g > 11h 8..11V 280mA power < 200mA 100mA 500mA Tekal Teko-T 265 x 105 x 45 mm Conrad C-Control II Unit RS232 serial port, 38000 baud, 9-pin male RS232 serial port, 9600 baud, 9-pin male MicroStrain 3DM-GX1 360 degrees full scale (FS), all axes (Matrix, Quaternion modes) Gyros: 300 degrees/sec FS; accelerometers: 5 Gs FS; magnetometers: 1.2 Gauss FS < 0.1 degrees minimum 0.5 degrees typical for static test conditions, 2 degrees typical for dynamic (cyclic) test conditions and for arbitrary orientation angles 65 x 90 x 25 mm Sanko Electric 1E120 28 x 4 mm 3..5V 50mA @ 5V Tamya 10 x 1.2V AA, 2200mAh 77 x 60 x 32 mm
Size Vibrators Model Size (in Enclosure) Supply Voltage Power Uptake Battery Packs Connector Type Cells Package Size
84
feelSpace
b)
a)
c)
Fig. F.10: The nal belt. (a) The green Coca-Cola caps contain the 13 vibrators. The blue pack on the left is the battery pack. (Note that in the depicted belt version, only eight batteries were used.) (a/b) The black box in front contains the main control unit. The silver plug on its left is the RS232 connection to the 3DM-GX1 compass. The socket above that is to connect the belt to a PC. (c) The MicroStrain compass attached to the belt, demonstrating the use of the Velcro fasteners to attach the wires.
85
Bibliography
[1] R. Azuma, B. Ho, H. Neely III, and R. Sarfaty. A motion-stabilized outdoor augmented reality system. In Proceedings of IEEE VR, pages 252259. IEEE, 1999. [2] RW Briggs, I y Liacco, MP Malcolm, H Lee, KK Peck, KS Gopinath, NC Himes, DA Soltysik, P Browne, and R Tran-Son-Tay. A pneumatic vibrotactile stimulation device for fmri. Magnetic Resonance in Medicine, 51:640643, 2004. [3] R.W. Cholewiak, J.C. Brill, and A. Schwab. Vibrotactile localization on the abdomen: Eects of place and space. Perception & Psychophysics, 66(6):970987, 2004. [4] D. Churchill. Quantication of human knee kinematics using the 3dm-gx1 sensor. Technical report, MicroStrain Inc., 2004. u [5] J. Conradt. Homepage at ETH Zrich, 2005. http://www.ini.unizh.ch/conradt/index.html. [6] PNI Corporation. PNI TCM-50 Specications, 2005. http://www.pnicorp.com/supportProduct?nodeId=c4b2&tab=specsheet. e [7] Andr Helbig. CC2Net.de - Alles rund um die C-Control II, 2005. http://www.cc2net.de. [8] Honeywell Inc. Honeywell HMR3200/HMR3300 Data Sheet, 2005. http://www.magneticsensors.com/datasheets/hmr32003300.pdf. [9] Microstrain Inc. 3-DM-GX1 Product Information, 2005. http://www.microstrain.com/3dm-gx1.aspx. [10] K. Kaczmarek, J. Webster, P. Bach y Rita, and W. Tompkins. Electrotacile and vibrotactile displays for sensory substitution systems. In IEEE Transaction on Biomedical Engineering, volume 38, pages 116, 1991. [11] Kajimoto, Kawakami, and Maeda. Electro-tactile display with force feedback. In World Multiconference on Systemics, Cybernetics and Informatics(SCI2001), volume X1, pages 9599, Orlando, 2001.
86
feelSpace
[12] Kajimoto, Kawakami, and Tachi. Psychophysical evaluation of receptor selectivity in electro-tactile display. In 3th International Symposium on Measurement and Control in Robotics (ISMCR), pages 8386, 2003. [13] Q. Ladetto, V. Gabaglio, and B. Merminod. Combining gyroscopes, magnetic compass and gps for pedestrian navigation. Facult ENAC - Institut du Dveloppement Territorial, Geodetic Laboratory (TOPO), Lausanne, Switzerland. [14] Q. Ladetto and B. Merminod. Digital magnetic compass and gyroscope integration for pedestrian navigation. Facult ENAC - Institut du Dveloppement Territorial, Geodetic Laboratory (TOPO), Lausanne, Switzerland. [15] Q. Ladetto, L. van Seeters, S. Sokolowski, Z. Sagan, and B. Merminod. Digital magnetic compass and gyroscope for dismounted soldier position & navigation. Facult ENAC - Institut du Dveloppement Territorial, Geodetic Laboratory (TOPO), Lausanne, Switzerland. [16] Thorsten Maucher. Aufbau und Test eines taktilen Seh-Ersatzsystems. Diplomarbeit, Ruprecht-Karls-Universitt Heidelberg - Fakultt fr Physik a a u und Astronomie, Max-Planck-Institut fr Kernphysik, 1999. u [17] JYC Ng and JCF Man. Vibro-monitor: A vibrotactile display for physiological data monitoring. In Proceedings of Human Interface Technologies 2004 Conference, 2004. [18] M. Stolle, A, R. Hlzl, D. Kleinbhl, A. Mrsic, and H.Z. Tan. Measuro o ing point localization errors in spatiotemporal tactile stimulus patterns. In Proceedings of EuroHaptics 2004, June 2004. [19] Aaron Toney, Lucy Dunne, Bruce Thomas, and Susan P. Ashdown. A shoulder pad insert vibrotactile display. In Deeber Azada, editor, Seventh IEEE International Symposium on Wearable Computers, pages 3544, White Plains, New York, 2003. IEEE Computer Society. [20] J.B.F. van Erp. Tactile information presentation: Navigating in virtual environments. Lecture Notes in Computer Science, 2058, 2001. [21] A.C. Zappe, Thorsten Maucher, Karlheinz Meier, and Christian Scheiber. Evaluation of a pneumatically driven tactile stimulator device for vision substitution during fmri-studies. Magnetic Resonance for Medicine, 51(4):828 834, April 2004.
87
Experimental Design
Alper Aik & Christine Carl & Christopher Lrken c o University of Osnabrck u
G Experimental Design
G.1 Hypotheses
The last appendices gave an introduction to the theoretical background as well as the actual device we build. The next step is to specify what possible eects of the belt we expect to be able to occur due to its usage. We therefore formulated four hypotheses depicting four dierent levels of possible outcomes: 1. Weak Integration - In principle, the sensory information provided by the belt can be processed. Thereby, training can improve performance. 2. Strong Integration - Orientation information provided by the belt can be rmly integrated in human perception. Sensory input of the belt inconsistent with other sensory inputs produces measurable responses. 3. Subcognitive Processing - After training, attention is not needed any longer to process belt information. 4. New Modality - Mastery of the belt-imposed sensorimotor contingencies results in qualitatively new experiences. The rst three hypotheses claim for increasing levels of integration of the belt information. This goes from a merely usable information that can be used better if the subjects are trained to a second level of integration postulating that the belt cannot be neglected any more and can even confuse subjects that are exposed to faulty belt information. The third hypothesis postulates a level of integration that would be comparable to something like intuition as the belt information would become subcognitively available to a trained subject. The fourth hypothesis is even more demanding as it claims that a trained subject develops a qualitatively new experience or to put it in slightly dierent words, a new sensory modality.1 Please note at this point that these hypotheses are not meant to be our predictions or expectations of the belt eects. They are rather formulated in a way that they cover the whole spectrum of possible outcomes we might observe. So
1
Please refer to chapter 1 and appendices B and C for an explanation of the mentioned sensorimotor contingencies and how a mastery of these might be decisive for the development and experience of sensory modalities.
89
feelSpace
Fig. G.1: Schema of experimental setup. Two large blocks of experiments are conducted of which one is before the training (pre-training) the other one after the training period (post-training). Three dierent groups of subjects will be tested (see section G.2.2), namely the experimental subjects who wore the belt, the group of trained controls (Control 1) who took part in the weekly training without carrying a belt and a na control group (Control 2) who did not take part in neither the belt ve training nor the weekly training.
we do not expect them to become all clearly evident. However, we do certainly hope that we can monitor at least some eects that suggest the validity of the claims of some or even all of the aforementioned hypotheses. To test our hypotheses, we developed a quite broad variety of experiments. Since the level of integration of the orientation information provided by the belt cannot be accessed directly, we had to measure behavioral performance. Thus, we decided to quantify the eects of training with the belt on navigation skills in orientation tasks. To test the higher levels of integration even further, we designed additional experiments of neurophysiological measurements monitoring possible eects of a belt training. But before theses experiments are introduced in detail the following sections will give an overview over the general experimental design.
G.2 Experimental setup

As stated in section G.1, we would like to see changes in the subjects behavioral performance that are an eect of a long-term training. Therefore, we tested all experimental subjects and controls (see section G.2.2) before and after a training phase of seven weeks (see gure G.1). Thus we are
90
feelSpace
able to monitor any changes in subjects performances and behavior patterns as we can compare their results before and after the training period.
G.2.1 Training
The training period of seven weeks started immediately after the pre-training tests and consists of two main parts: Passive everyday training - The subjects that wear a belt have to wear it all day long over the whole training period. This means to put it on in the morning, wear it with as few interruptions as possible and to only switch it o at night. The subjects were requested to carry the belt outdoors for at least 90 minutes a day. Active weekly outdoor training - We engaged our subjects to do an outdoor training that was conducted on a weekly base (depending on the weather conditions). The reason to do this is in order to motivate them to actively use the belt information and thus try to take advantage of the additional information that they have at least during those training sessions. The weekly training session consisted of several simple pointing tasks. Subjects had to stand at a starting location in an empty area and face a reference point in form of a clearly visible structure, in this case a camera tripod, somewhere within that area. After that, they were blindfolded and an instructor gave them walking directions in terms of the direction to turn (left or right), the angles of turns (in degrees which are multiples of 30 or 45 degrees) and the length to walk (in steps). In this way, the subjects were guided along simple geometric shapes whose complexity increased during a training session. After they have walked according to the commands of the instructor, they were asked to point into the direction of the reference point they were facing in the beginning. When the subjects have pointed, they were asked to remove their blindfold to see how well they got the actual direction of their target. The training sessions lasted for roughly half an hour per subject. It is obvious that in urban outdoor environments, factors like localized sounds, changes in the texture of the ground or the direction of wind can confound purely path-integration based tasks like describe above. In our training sessions, the possible contribution of these factors was also inevitable, but it was assumed to be uniform over the subjects.
G.2.2 Experimental groups

To account for any dierences in subjects performance to either the passive or the active form of the training, we introduced two control groups in addition to the group of those subjects that actually wore the belt. Therefore, we ended up with three dierent groups of participants:
91
feelSpace
Group Experimental Subjects Trained Controls Na Controls ve Total
mean age 30.5 23.0 21.5 25.0
no. male 3 3 3 9
no. female 1 1 1 3
no. total 4 4 4 12
Tab. G.1: Demographic properties of the experimental groups. Each of the three dierent experimental groups consists of four members. All groups show a male/female ratio of 3 : 1 and comparable mean ages.
Experimental subjects (ES) - are those subjects that were equipped with a belt, wore it over the whole training period and took part in the weekly training sessions (of course while wearing the belt). Trained controls (TC) - are the group of subjects that did not wear a belt on an everyday basis but that, nevertheless, took part in the weekly outdoor training. But in contrast to the experimental subjects they did neither wear the belt during the weekly training sessions. Na controls (NC) - did neither wear the belt on a daily basis nor did they ve take part in the weekly training sessions.2 Each of the dierent experimental groups consisted of four people. For a detailed description of their demographic properties see table G.1. Of course, all participants took part in both, the pre- and post-training experiments.
G.2.3 Test conditions

Following the experimental schedule and the group segmentation as presented above and in gure G.1 we were able to relate results from dierent tests to each other. To further check especially for belt-related eects, we applied several test conditions in each experiment allowing us to control for various factors in order to isolate belt induced eects. We distinguish the three conditions of: Correct belt information (CI): The belt is vibrating towards north (real or virtual) consistently. No belt information (NI): The belt gives no information. Wrong belt information (WI): The belt is providing inconsistent information. This condition is optional in the dierent experiments.
Please note that, as we found no signicant dierences between the groups of trained and na controls, we will mostly omit this dierence in the following. ve
92
feelSpace
G.3 Experiments
Following the experimental setup presented in this appendix, we designed a variety of dierent experiments and evaluation methods to test for signs reinforcing the claims of our hypotheses (see G.1). These can be classied into three groups: Orientation/Navigation experiments - Two large groups of orientation and navigation based experiments, namely those of natural (both outdoor and indoor) as well as virtual environment tests, were conducted. While the natural environment experiment mainly aimed to be evidence of the Weak Integration hypothesis, the virtual environment setting was tailored to focus besides that on the Strong Integration and Subcognitive Processing hypotheses. See appendices H and I for the complete description and the results of the natural environment tasks and appendix J for those of the virtual environment. Physiological measurements - We realized the two physiological measurements of a nystagmography and posturography in order to test for the Strong Integration and Subcognitive Processing hypotheses that are explained in detail in appendix K. Subjective reports - Experimental subjects had to take part in weekly interviews as well as to keep a diary during the period of training. We took those insights into the subjectively reported feelings and experiences of our experimental subjects to look for evidence for the New Modality hypothesis. The results are presented in appendix L.
93
Orientation in Natural Environment
Alper Aik & Christiane Kabisch c University of Osnabrck u
H Orientation in Natural Environment

H.1 Introduction
According to the Weak Integration Hypothesis (see G.1 p.89) the information coming from the belt can be used eectively after some training. To test this simple navigation tasks are needed. In the literature, homing tasks are generally used to quantify the navigation skills of human subjects. In a typical homing task a subject is taken to a certain point, and the subject is reported that this is the so called home. Afterwards the subject walks over a trajectory, which might be simple (like single line) or quite complex (a trajectory with many turns and crossings). At the end of the trajectory (homing start point) the subject is supposed nd the home with the shortest way possible. The distance between the actual and recalled home is one way to measure the performance. Another way is to draw two vectors from the homing start point to the actual and recalled homes, and to look at the angle between them. These tasks can be visual or nonvisual according to the mode of navigation under question. In allothetic versions the subject navigates according to the salient targets around them, and use the relation between home and these landmarks. In idiothetic mode, the subject can only rely on recent displacements in space, like the turnings the amount of steps he/she underwent. The belt provides information regarding the latter, so we designed our tasks accordingly.
H.2 Methods
In a large and empty university courtyard, polygonal shapes were drawn on the ground (see gure H.1). The shapes had three to ve corners, according to which the complexity of the given shape was dened. We used two triangles, two quadrangles, and two pentangles, each of which was used once in correct belt information (CI), and once in no belt information (NI) condition (see section G.2.3); giving rise to twelve trials for each subject. Each CI trail was followed by a NI trial, and vice versa, to equalize learning eects. The blindfolded subject devoid of any knowledge regarding how the shapes looked like, was taken to one of the corners of the shape, and was led by one experimenter over all the shape but the last segment (see gure H.2). At that point s/he was released to nd the starting point with the shortest way possible (homing). That is, if the shape was triangular, the subject was led over two sides, and the third side would be the ideal homing vector. The relevant measurements taken at the experimental site
95
feelSpace
SHAPES USED IN THE OUTDOOR HOMING TESTS

a b a c SHAPE 3A = 45 a=5 = 90 b=7 SHAPE 3B = 45 = 30 = 105 a = 9.5 b=7 c=5 a b
a b
a a
SHAPE 4A
= 1 0 5a = 5 = 6 0 b = 3.5 = 90
SHAPE 4B
= 120 a=5 = 60
a b a a b d a b
SHAPE 5A
= 90 = 30 = 300
a=4 b=6
SHAPE 5B
= 45 = 240 = 30 = 135 = 90
a = 2.5 b=5 c = 4.5 d = 3.5
Fig. H.1: Shapes used in the outdoor homing tests. The Home, or the starting point is denoted by the green rectangle. The shapes are drawn such that the trajectory starting from the home is always clockwise. The ideal Homing Vector is seen as the red side of the shapes. The Greek letters denote the angles in degrees, the roman letters the length of the side in meters.
96
feelSpace
Fig. H.2: Homing in outdoor environment. The subjects were led by the experimenter over a trajectory, and then were released to nd the start again.
were the distance between the actual and recalled starting points, and the angle between the real and ideal homing vectors. All subjects are tested once before training and once after training. In such outdoor environments, the factors like heterogeneous distribution of light, localized sounds or wind direction can play a landmark role and change the purely idiothetic navigation mode. But please note that each CI trial was followed by an NI trial. Obviously this was our most interesting comparison, and in both kinds of trials all the factors involved were identical.
H.3 Results
In this section we only report data regarding the angular errors, because it is a more relevant measure keeping in mind the kind of information provided by the belt. The tests with the distance measurement gave very similar results. We combined the data from the six shapes we used, because there was no meaningful dierence, and for statistical testing this increased the size of trials in each condition. The statistical test we used for all group and belt condition comparisons was a signed rank test. Figure H.3 summarizes the results. We looked at how the mean error for each subject changes over time, i.e. with training with the belt, and if there was an eect of the belt condition (CI and NI conditions). For statistical comparisons we collapsed the subjects in each group. Before training there was no performance dierence between experimental and control subjects, regardless of if the belt was on or not (p-values > 0.05). Neither for experimental nor for the control subjects, the condition of the belt (CI or NI) changed performance levels. Thus, the performance of the two groups was similar, and the belt information did not make a dierence for neither of them.
97
feelSpace
Fig. H.3: Mean errors for each subject. The mean angular is plotted over time, i.e. before and after training. The dashed dotted line is for NI, and the solid line is for CI condition. Note that for experimental subjects, errors in CI condition get decreased after training.
98
feelSpace
30 Difference in Performance (CI NI) Control Ss Experimental Ss 20
10
10
20
30
Pre Time Course
Post
Fig. H.4: Dierence between CI and NI conditions. The mean error for all shapes in NI condition is subtracted from CI condition. Each line is for one subject (red - experimental subjects; green - control subjects). The x-axis refers to the time, the y-axis to the error. Note that after training this dierence becomes negative for experimental subjects.
After training, the same comparisons revealed identical results for the controls. Apparently the weekly training did not change the performance in neither of the belt conditions for the group (p > 0.05). According to our Weak Integration hypothesis, the experimental group should display decreased errors after training, only if their belt was on. A comparison of NI and CI conditions for that group after the training highlighted that the errors in CI were less than errors in NI condition (p < 0.05). This was not due to a decrease in performance in NI condition, since there was no statistical dierence between pre-training and posttraining NI trials (p > 0.05). Additionally, experimental subjects were better than almost any control subject, as long as their belt was on. In gure H.4, we plot a single line for each subject. Each line shows the evolution of the dierence between mean CI-NI conditions. The decrease in errors for CI condition is clearly seen for the experimental subjects. After training the points for experimental subjects cluster around negative values, since the error is decreased in CI; but the points converge around zero for the controls, because there is no dierence between conditions.
H.4 Discussion
We have shown that the direction information provided by the belt can be used eectively after training, without decreasing the performance if the belt is turned o. At the end of training experimental subjects were making decreased errors only if their belt was on, and this led to their superior performance compared to controls. Thus, we conrmed our Weak Integration hypothesis.
99
Orientation in Enclosed Environment Blue Velvet Arena
Alper Aik c University of Osnabrck u
I Orientation in Enclosed Environment Blue Velvet Arena

I.1 Introduction
The study of spatial cognition and its underlying neural mechanisms showed that there are mainly two modes of navigation, which can be tested separately for many animals. The allothetic mode makes use of salient landmarks; the subject estimates its position and/or direction according to his/her relative location to these landmarks around (cf. [5]). Idiothesis, on the other hand, is based on the vestibular and somatosensory feedback about the spatial displacements of the subject, together with the eerent copies of the motor commands responsible for the displacements (cf. [3]). In daily navigation, these two modes are used simultaneously, providing the subject with two overlapping maps of the environment (cf. [7]). We wanted to know, how these dierent modes of navigation would be inuenced by long-term stimulation with direction information. To put it dierently, how the performance of our human subjects would change, after carrying a device (a belt), which provides them continuously with information regarding their direction in relation to a stable point (north). As obvious, this information is independent of the layout of the landmarks in the environment. But it is highly correlated with the vestibular and somatosensory signals about the history of angular displacements of the subject. It follows logically, that if there is improvement as a result of training with this device, it should be exclusive to the idiothetic domain and the eects on allothesis should be little, if any (for more information on the rationale behind the development of the belt, and the technical details about the belt, please refer to the Expose and Appendix: Belt in this document). The animal spatial cognition research has developed many methods to separate these two modes of navigation in laboratory setting. For rodents, radial arm maze ([6]), Morris water maze ([4]) and various dry versions of these tasks have been used successfully to explore the navigation abilities for a variety of purposes. But similar methods for quantifying the navigation abilities of humans are unfortunately scarce. One remarkable example is the virtual environment developed by Gillner and Mallot ([1]), where the subjects learn to navigate in a virtual town. But here we restrict ourselves to navigation in real environments; the tests in virtual environments will be discussed elsewhere (see appendix J). In 2003, Stepankova and colleagues introduced a test battery, which was de-
101
feelSpace
Fig. I.1: Blue Velvet Arena. A photo made from inside of the arena. The ceiling is 3 meters high, and the diameter is 2.90 meters.
signed to study the dierent navigation modes of humans in the laboratory. All experiments were conducted in a circular arena covered by blue velvet curtains; hence it was later going to be called Blue Velvet Arena (BVA) ([2]). An important nding from their experiments was that the subjects were making angular errors, rather than distance errors in their homing tasks ([7]). This is exactly what we can expect to be improved by the belt. Using some tests of their battery, and adding a couple more we tested our subjects in BVA, to quantify the eects of training with the belt on allothetic and idiothetic modes of navigation separately, and combined.
I.2 Methods
I.2.1 Subjects
All the three groups of experimental subjects and controls have been tested (see G.2.2, p. 91). The training period (see G.2.1) ran until the onset of the BVA testing.
I.2.2 Apparatus
Blue Velvet Arena (BVA) is an enclosed cylindrical environment (diameter: 2.90m, height: 3m) (see gure I.1). It can be rendered totally dark inside to remove all visual cues. The use of wireless earphones, through which soft music is played, prevents the use of auditory cues during testing. The same earphones are used to give experimental instructions during the tests as well. These make sure that during testing there are no auditory or visual cues, except the ones
102
feelSpace
Fig. I.2: Simple Homing. Exemplary reproduction of a simple homing trial. Red reproduced part is the initial walk, and green is homing.
dened by the experimenter. A computerized tracking system continuously tracks the position of the LED pole, which is in some experiments held by the subject, and in others attached to the earphone-helmet. The trajectory of the subject can be quantied precisely with the use of this system. BVA also allows the projection of visual cues on walls and the ground of the arena. These signal either the starting point for the subject, or the visual cues used for allothetic orientation. Another innovation is the oor-related signals. In some tests when the subject enters a certain part of the arena, a sound is displayed to signal the discovery of the target.
I.2.3 Procedure
General Procedure Blindfolded subjects are taken into BVA by the experimenter. Here the subject is confused about direction, by being turned by the experimenter several times, which makes sure that the room frame is detached from the BVA frame. The start and targets are positioned such that the chance to establish contact with the walls of the arena is minimized. During the experiment the subject listens to soft music and hears the instructions from the experimenter through earphones. Each experiment is repeated ten times. Each trial starts at one of the ve random points in the arena. Each Correct belt information (belt on, CI) trial is followed by a No belt information (belt o, NI) trial. Tasks Simple Homing (SH) The disoriented subject is led to the start point and left alone in BVA. The task is to go to the opposite wall of the arena by walking straight, and when the wall is reached, the subject has to say turning, then turn around and nd the starting point with the shortest possible way. Here the subject says home which terminates the trial. A typical trajectory is shown in gure I.2.
103
feelSpace
a)
b)
Fig. I.3: (a) Complex Homing. The subject walks a complex path according to the instruction (red), and then starts homing (green). (b) Complex Homing 2. The subject is led over a trajectory (red), and then the experimenter leaves and the subject starts homing (green).
Complex Homing (CH) When the blindfolded subject is in BVA, following disorientation he/she receives the following instruction: After you will be released at the start point, go slowly straight ahead to the wall, turn right and walk ve steps along the wall, turn right again and walk to the opposite wall. Return from there to the start point using the shortest possible way. Stop for 1-2 s at each turning point (from [7]). At the last turning point the subject says turning, and when on recalled home home. A typical trajectory is shown in gure I.3a. Complex Homing 2 (CH2) After the subject is moved to the starting, point the experimenter leads the subject over a trajectory by holding him/her on the shoulder. The trajectory consists of 3 line segments with 2 turns. At the end of the walk the experimenter leaves the arena and with the instruction the subject has to nd the home. A typical trajectory is shown in gure I.3b. Reverse Route Reproduction (RRR) The subject is again blindfolded and disoriented in BVA. The experimenter who stays behind the subject leads him/her over a complex trajectory, which ends at the start point again, then the experimenter leaves the arena. At the turning points the experimenter says: This is point x; during reproduction the subject has to do this as well. The trajectory has straight-line components from one side of the arena to the other, and curved components, which are adjacent to the wall. The aim of the subject is to walk the same trajectory in the reverse fashion. A typical trajectory is shown in gure I.4.
104
feelSpace
Fig. I.4: Reverse Route Reproduction. Starting from the empty circle, the subject is led by the experimenter over cyan, pink and yellow reproduced parts. The experimenter leaves and the subject reproduces the walk in the reverse order, here this is denoted with blue, green and red. Endpoint is the lled circle.
Angle Reproduction (AR) The subject is again blindfolded and disoriented in BVA. The experimenter behind the subject leads him/her over a trajectory, which consists of three line segments, and two turnings. At the turning points and at the end of the trajectory, the subject hears point 1, point 2, end respectively Here the experimenter takes the subject to the starting point and leaves the arena. With the instruction, the subject has to reproduce this walk. Allothetic Task (AT) This is the only task that is aimed at measuring exclusively the navigation performance of the subjects using allothesis. In this task the eyes of the subject are left open, so he/she can have visual access to the landmarks. The landmarks are two dierent light patterns projected on the walls at the eye-level. In some circular portion of the area there is a virtual target, and nding the target (entering this small circular area) is signaled with a special noise to the subject. Additionally in each trial a laser light is seen somewhere on the ground which is the start point for that particular trial. In each trail the locations of the start point, visual landmarks, and the target is changed; but the spatial relationship between target and the landmarks stays the same. The location of the start point on the other hand, is changed independently of the other cues. Note that successful recall of the target depends only on understanding the relation between the landmarks and the target, where start point, hence idiothesis cannot play a role.
I.3 Results and Discussion

Hence this is collaboration with the Institute of Physiology at Academy of Sciences of Czech Republic the results will appear in a future publication
105
Bibliography
[1] S. Gillner and H. Mallot. Navigation and acquisition of spatial knowledgein a virtual maze. Journal of Cognitive Neuroscience, 10:445463, 1998. [2] E. Kalova, K. Vlcek, E. Jarolimova, and J. Bures. Allothetic orientation and sequential ordering of places is impaired in early stages of alzheimers disease: corresponding results in real space tests and computer tests. Behavioural Brain Research, in press, 2004. [3] H. Mittelstaedt and M.L. Mittelstaedt. Avian Navigation, chapter Homing by path integration., pages 290297. Springer-Verlag, Berlin, 1982. [4] R. Morris. Developments of a water-maze procedure for studying spatial learning in the rat. Journal of Neuroscience Methods, 11:4760, 1984. [5] J. OKeefe. An allocentric spatial model for the hippocampal cognitive map. Hippocampus, 1:230235, 1991. [6] D.S. Olton and R.J. Samuelson. Remembrance of places past: spatial memory in rats. Journal of Experimental Psychology: Animal Behavior Processes, 2:97116, 1976. [7] K. Stepankova, E. Pastalkova, E. Kalova, M. Kalina, and J. Bures. A battery of tests for quantitative examination of idiothetic and allothetic place navigation modes in humans. Behavioural Brain Research, 147:95115, 2003.
106
Navigation in Virtual Environment
Christine Carl & Christopher Lrken o University of Osnabrck u
J Navigation in Virtual Environment

J.1 Motivation
To test the Strong Integration and Subcognitive Processing hypotheses, we aimed at designing a task in which the belt information will be used without conscious deliberation. The best way to hold the belt information processing at a subcognitive level is to deprive the subjects of the time needed to think about the belt information. Therefore, we designed a time-competitive navigation task in a virtual environment. With this task we wanted to answer two questions: Will experimental subjects improve their skills of fast navigation with correct belt information after the training? Will wrong belt information deteriorate the experimental subjects performance after the training? By addressing these questions, we focused on the Strong Integration and Subcognitive Processing hypotheses but the Weak Integration hypothesis may obviously be reected by the results as well. Why is especially a virtual navigation environment advantageous? A virtual environment is a highly controlled environment. A natural environment always provides additional cues that cannot be accounted for like visual cues that help to orientate oneself. The light of the sun, for example, creates shadows that may serve as global landmarks 1 . Apart from visual cues proprioception in natural environments provides additional help for orientation. We wanted to separate proprioception as one source of information about orientation from the information provided by the belt. This can easily be achieved by a virtual setup. The belt design allows us to connect the belt directly to a computer (see appendix F). While the subject is navigating, we can provide information of a virtual north to the belt. This means, that if the subject is turning within the virtual environment, the belt provides the same vibro-tactile stimulation as if he would actually turn in the real world. Of course, it is easy to simulate any kind of heading information on the belt, which makes it also possible to provide wrong heading information. The main idea of the virtual environment experiment is to collect items in a predened order within a complex labyrinth-like structured world. Hereby, the
1
A landmark is a xed reference point.
108
feelSpace
a) Pre-Training.
b) Post-Training.
Fig. J.1: Expectation of training inuence on experimental subjects. (a) Before the training there should be no signicant dierence between the conditions. (b) After the training, the experimental subjects performance in the NI condition should increase only due to minor learning eects, while there should be a signicant performance increase in condition CI and a strong decrease in condition WI.
items have to be collected as fast as possible. To judge the subjects performances and to verify our hypotheses, we use time and path measurements (see section J.3). In gure J.1 you see the expected change in performance for our experimental subjects according to our hypotheses. Before the training there should not be any signicant dierence between the various conditions as well as the groups of experimental subjects and controls. After the training period, however, the experimental subjects performance should change in a way that correct belt information leads to better performance while wrong belt information will deteriorate their performance. Learning eects might result in an improved performance for the no belt condition as well but this eect should be more subtle than that of the correct belt information condition. Except from subtle overall learning eects we do not expect any notable changes in performance in the group of controls.
J.2 System Design

Our rst intend was to use an existing framework that provides a structure that is easily extendable and adjustable to our needs. In order to be close to natural vision the framework should provide a rst person perspective and a human like motion model. Earlier Approaches An early consideration for our virtual navigation tests was the use a virtual environment realizing Morris Water maze (cf. [6, 7]). The Morris Water Maze task is a well known experimental paradigm to study spatial learning and memory in rodents. The water maze consists basically of a round pool lled
109
feelSpace
with cloudy water in which a platform is submerged beneath the surface. In the original water maze learning task animals that are placed in the pool have to nd the hidden platform and swim towards that platform. The time they need for task completion over a number of trials is the performance measurement to test indirectly for spatial memory and learning. A virtual environment simulating a water maze is a convenient way to assess humans performance in a water maze learning task (cf. [3, 4, 1]). We have considered to use a water maze implementation to access possible eects of orientation performance of experimental subjects vs. controls. But as the water maze depends mainly on visual cues and provides a rather unnatural environment for human orientation in every day life. Since we wanted the environment to realize orientational demands close to real every day life orientation tasks we have abandoned this idea pretty soon. Another framework that we short-listed was Hexatown (cf. [8]). The Hexatown environment was developed by the group of H.P. Mallot, Tbingen. It u consists of a regular hexagonal grid of streets and junctions. The Hexatown environment provided several advantages for our concerns: Firstly, the environment was already tested within controlled experimental setups. Secondly, the hexagonal architecture seemed to be useful in terms of yielding a strongly structured environment where it should be easy to dene optimal paths for our collecting task. On the other hand, this structured environment seemed to be very restrictive prohibiting realistic map layouts. Because of the hexagonal conguration, the possible angles of the crossings were very limited resulting in a limited and thus unnatural amount of orientations that can be experienced by navigating subjects. This would be especially disadvantageous in our case because we wanted to test the integration of the belt for all possible body orientations covering the continuous space of 360 . In an environment where all body orientations are used by the subjects, the belt information gains in quality. Another shortcoming for our purpose was the absence of any model of physical laws. Since we were aiming at a fast navigation test with a labyrinth-like structure, we would at least need to model collision detection. A virtual environment based on Quake III To overcome the shortcomings we would have to face with the approaches presented above, we nally decided to use the complete system of the rst person ego-shooter Quake III Arena by ID Software ([2]). The Quake III engine provides a comfortable way of extending the program by customized add-ons (so-called mods). The program already has a built-in model of worlds physics and an embedded level editor that allows to construct new environments in a graphical user interface. The editor supports a convenient way of constructing new object models and inserting realistic textures. In comparison to the other approaches, this seemed to be the best way to build a natural environment without any restrictions given by the program design.
110
feelSpace
Furthermore, the C source code of Quake allowed us to access those system components needed to communicate with the belt and implement our experimental setup and logging. Besides these considerations about implementation, the choice of using Quake III Arena for our purpose was supported by the fact that some individually created Quake mods have already been tested in controlled experimental setups (e.g. [5]). Design decisions We wanted to have a labyrinth-like map layout in which the user should search for numbered items and collect them in ascending order. The maps we built look like labyrinths made of walls in outdoor environments. No features helping in orientation were provided apart from the shape and direction of the corridors, the numbered items2 , and three dierent textures of wood and stone which indicate three dierent districts in a map (see Fig. J.2a). The design of the maps ensured that no cues serving as global landmarks existed in the environments. The light, for example, was designed as a diuse lighting source which cannot be located. The items, the user has to look for, were white boxes with black numbers on them that had an approximate size of a third of the player gures height. To achieve environments close to realistic settings the map layout was inspired by city maps of Munich and Paris, of course in a down-scaled and featureless version (see Fig. J.2b). On the whole, we designed three dierent maps.
J.3 Methods
Our aim, as described above, was to measure the time or path length needed by the subjects to collect eight numbers distributed in the environment. The subjects were neither given a map of the environment nor did they have the chance to explore it beforehand. At the top of the screen the number of the box that has to be collected next was displayed. When a number was collected by running through the number box it did not disappear but the display at the top of the monitor was updated. Movements were executed by keyboard and mouse. Hereby forward/backward movements were initiated by the w/s keys and the direction of movement or the looking direction respectively was triggered by horizontal mouse movements. Other actions that are well known to experienced game players like free-look, jumping or strang were disabled. This was done rstly to prevent people having long experiences with ego-shooter games from performing much better than unexperienced persons. Secondly, especially people who are not at all familiar with this kind of games often suer from a phenomenon called cybersickness (see [9]), i.e. they show signs of nausea and sudden indisposition. A means to reduce cybersickness as mentioned in [5], is
2
The numbered items that had to be collected were not removed after collection. Hence, they served as additional local landmarks.
111
feelSpace
a)
b)
Fig. J.2: (a) Screenshot of the virtual environment inspired by a city map of Munich. The display at the top indicates the number that has to be collected. (b) Bird view screenshot of the map layout of the same map as (a).
the stabilization of the horizon so that pictures seem to be more static and only depend on the forward and backward moving of the game player. This can be achieved by prohibiting the players from jumping and looking around freely in all axes. Within one session subjects had to perform nine trials, each lasting roughly 10 minutes, and had to take short breaks after every three trials. Both mouse buttons could be used for zooming to recognize distant numbers more easily. The (in-game) heads-up-display did not show any information apart from the number of the next box to look for (see gure J.2a). We measured the time and path length to collect each single number as well as the overall time and path to collect all numbers. After each session subjects were asked about their strategies for orientation in the navigation task. Experimental Setup We decided to test each condition (CI, WI, NI) three times resulting in nine dierent trials within one session. Thereby, we had to consider two problems; If the same map is used all the time, learning eects cannot be prevented, whereas dierent maps or dierent arrangements of target items within one map necessarily lead to dierent levels of complexity that cause diculties in comparing the performance of the subjects. Therefore, we decided to test all three conditions in three maps to account for possible strong learning eects. Additionally, the order of tests for one subject was identical for pre- and posttraining tests so that the analysis in a within-subject-design is independent of the dierent complexities of the maps. To further reduce the eects of learning on our results we permuted the order of maps as well as the order of testing the conditions between dierent subjects. Although we have tried to design maps that were comparable in terms of the level of diculty for navigation, it is not crucial that all maps are equally dicult because we can compare condition dierences within one map.
112
feelSpace
Fig. J.3: Quake III map editor. The left part shows a top view of the map while at the upper right an inside view of the labyrinth is provided. The lower right presents a choice of textures that can be applied to the structures.
The location of the numbered boxes did not change for one map between the trials.
J.3.1 Implementation Details

The Quake III software structure is designed according to a client/server pattern. Hereby, the client side concerns everything that is related to the player, e.g. the player movements and perspective. Therefore, this part of the program provides several routines which are called either when the game is started (for initialization) or every newly rendered frame. We could expand these routines to include belt connection and logging functionality to the game. On the start of the game,a beltoptions le is opened and read. It contains information about which serial port is to be used, to which le the position of the player should be written for later analysis and if the signal to the belt should be correct or erroneous. The functions of the code which are called for every new frame have access to a data structure which hold information about the current player state - his position and view angle. This information is used to write the position of the player to a log le every 100 ms. The belt is updated every 50 ms with the present view angle. The belts software allows the vibrators to be remote controlled via a simple serial protocol (see section F.3 for details). The heading of the player is translated into
113
feelSpace
a 16 bit integer value and transmitted to the belt logic. The C programming language provides direct access to the serial port, even though port settings have to be done externally with a system command. The client side is also the place where we had to apply changes to the graphical appearance of the game. As the game normally displays a large number of items that were sidetracking for our experiments (e.g. the players weapon or the players score and status), we had to make a number of changes to the painting routines of the game. While all game-related information was removed, we added information about the next box to nd during the experiment. The server side on the other hand is related to maintaining e.g. the map and the communication to dierent players. As the game engine itself is unfortunately not open source, the communication process between the client and server part of the game is inaccessible. This pre-wired process exchanges only particular information which resulted in some problems of handling our additional belt related values. The number of the box that was looked for, for instance, was created in the client part as that part maintained the player coordinates within the world. To give the information of which box has to be searched in the GUI, this information, however, has to be transferred to the server side of the application. As it is not possible to exchange any additional information, we used a work-around that sends this information by simulating the grabbing of some ammunition. Thus, the server is told that the player got some extra ammunition and increases the number of available shots. We used this number to code for the next box to look for and to display it in the GUI. The maps were built with the Q3radiant map editor that supports a drag and drop way of map building. A natural looking sky model as well as the numbered boxes were custom made. After compilation, the boxes are rmly integrated into the map and cannot be moved any more. In order to recognize the nding of a box, i.e. the player passing its coordinates, the box location had to be stored in an additional le as they could not be accessed from the map itself (see gure J.3).
J.3.2 Analysis of data

In our analysis of data we compare performance between pre- and post-training in a within-subjects-design using a paired t-test. In the following, in section J.4.1 the methods used for a time dependent analysis will be described and the according results will be presented. Furthermore, in section J.4.2, the same will be done for the path related data.
114
feelSpace
a)
b) Fig. J.4: Example of the time needed by two dierent subjects to nd the according next box. The curves plotted here show the mean time over the three maps in condition NI, CI and WI for pre- and post-training conditions. (a) The subject does not show any notable learning eect. As an overall tendency, the time needed to nd one of the later boxes within a trial does not decrease. (b) The subject seems to gather a representation of the map while he walks through it within a trial. The time needed to nd the later boxes in a trial decrease remarkably.
J.4 Results
J.4.1 Analysis of performance in time
Before analyzing the data according to the expected changes in performance, we tested the data for learning eects that might have resulted from our experimental setup. Learning eects might occur either across trials with the last trials of a session being much shorter than the rst ones or within one trial, so that the last boxes in one trial can be found in less time. We found, however, that no signicant learning eect could be observed. Neither did a subject get notably faster in a map across dierent trials in one session nor was the time to nd the last boxes signicantly reduced within one single trial (cf. J.4(a)). Obviously, the maps seem to be highly complex, which makes it dicult to acquire a representation thereof. Although most of our subjects seemed not to show remarkable learning eects it might be interesting to investigate, if it was at all possible to acquire a representation of the maps. The data of one experimental subject show a drastically reduced time or path length needed for nding a box at the end of a trial (cf. Figure J.4 (b)). This reveals that it was in principle possible to get a more precise representation of the map by exploring it during a trial. However, such a learning eect was only observable in one subject. Therefore, we did not account for learning eects in the following analysis. The complete time for all trials did not dier remarkably depending on the
115
feelSpace
x 10
mean time in ms
Subjects Pretraining
mean time in ms
x 10 6 4 2 0 x 10
mean time in ms
Subjects Posttraining
6 4 2 0 x 10
5
NI
CI
WI
NI
5
CI
WI
Controls Pretraining
Controls Posttraining
mean time in ms
6 4 2 0 NI CI WI
6 4 2 0 NI CI WI
Fig. J.5: Mean time needed for a trial in the dierent conditions. Plotted is the mean time needed for one trial in the three dierent conditions (NI, CI, WI) averaged over the groups of experimental subjects (top row) and controls (bottom row) before (left column) and after the training (right column). There is clearly no overall group eect visible as predicted in gure J.1.
maps. Since this shows a comparable level of complexity of the dierent maps we do not distinguish between them in our further analysis. A rst analysis evaluating the time performance by calculating the mean over all subjects and maps for every condition did not show any eects. Figure J.5 shows the mean performance in time to complete a trial for the conditions NI, CI and WI for both experimental groups in pre- and post-training. Clearly, we cannot state any change in performance that corresponds to our predictions as described above (cf. gure J.1). The high variability suggests to evaluate each subject independently. Therefore, we continued our analysis in a within-subjectdesign. Since we were above all interested in the change in performance between preand post-training depending on the belt information provided, we compared performance in the dierent conditions (NI, CI, WI) in pre- and post-training. A very convenient way to compare the conditions is to look at the dierences between these conditions. To account for all possibilities that could lead to a change in performance as predicted, we evaluated the time dierences between the following
116
feelSpace
condition pairs: CI-WI, CI-NI, NI-WI. Looking at the changes in performance within conditions, we expected experimental subjects to improve their performance with the correct belt information, but to deteriorate it with wrong belt information. This means, for example, that a substraction of the time measured in the wrong belt information condition from the time measured for correct belt information should result in a smaller value after the training. And since furthermore performance with correct belt information should always be better than wrong belt information, the value should also be negative. The same holds for all other condition comparisons. In contrast to this change in performance for experimental subjects we expect no such eect for the controls. We therefore will use the following two terms for performance measurements: Downwards tendency - In the following we will test if the dierence in condition pairs (as mentioned above) changes signicantly from pre- to post-training. As described in the example above, we subtracted the time needed in the conditions in a way that we will always expect a decrease in the dierence value in the post-training for our experimental subjects. Since an increase of the dierence value after training does not display any possible eects of the belt-training, signicant changes between pre- and post-values of the dierence will only be considered if the change represents a decrease. We call this performance measurement the downward tendency. Negative dierence - We subtracted the time values of the conditions in a way that we expect the post-training result to be negative in our experimental subjects. Therefore, a signicantly dierent value from zero as shown by a t-test will only be considered if it is below zero. We call this performance measurement the negative dierence. Controls will be tested for downwards tendency as well as negative dierence although we do not expect any eects for them. For the performance measurements, we calculated the median of the time a subject needed to nd a single box in a specic trial.3 Since the complexities of the three maps were comparable, we calculated the mean over the paired dierences of the according conditions. Signicant results for downwards tendency and negative dierence were calculated with a paired t-test. The results are stated in table J.1 each time with signicant results in bold print. Please note here that the signicance for negative dierence is only meaningful if the subject did additionally show a downwards tendency. Figure J.6 shows exemplarily the results for the performance comparison in the condition CI-WI. Here, obviously two experimental subjects (CS and PK) show a downwards tendency while the other two experimental subjects show a tendency in the
3
Our data reveal high outliers for the time needed to nd a single box. Therefore, the application of a median for analysis seems to be advantageous over a mean since the eect of outliers is reduced in this way.
117
feelSpace
CS
1 0 1 2 3 4 pre x 10 1
4
x 10 1
x 10
ID
1 0 1 2 3 4
x 10
PK
1 0 1 2 3 4
x 10
UW
Time diff.: CIWI
0 1 2 3 4 post
pre x 10 1 0 1 2 3 4
4
post
pre x 10 1 0 1 2 3 4
4
post
pre x 10 1 0 1 2 3 4
4
post
BJ
BH
NF
SES
Time diff.: CIWI
0 1 2 3 4 pre post
pre
post
pre
post
pre
post
Fig. J.6: Time results of CI-WI dierences for all experimental subjects and trained controls. The y-axis shows the dierence of the time needed for nding a single box for condition CI minus the time needed in condition WI in ms. The values shown are averaged over all paired dierences of the median of the time needed for nding single boxes (see section J.4.1) in the pre- and post-training condition. Therefore, a value < 0 (negative dierence) codes for less time needed in condition CI compared to WI and thus for better performance with correct belt information than with wrong belt information. Moreover, if the curve has a negative slope the results have the downwards tendency we expected for experimental subjects, i.e. either the performance in CI condition was better after the training or the performance for WI condition was worse. The top row depicts the results of the experimental subjects, the bottom row those of the controls. Standard deviations are plotted for pre-and post-training.
opposite direction. One of the controls (BJ) shows also a downwards tendency which we expected only for the experimental subjects. Additionally the negative dierence in the post-training condition that shows a higher performance in using the correct belt information compared to the wrong belt information can be seen in all experimental subjects. But the negative difference was only signicant in the experimental subject CS while simultaneously showing the downwards tendency. Again, one of the controls (BJ) showed the very same eect for downwards tendency and negative dierence and even control SES showed a signicant negative dierence, while showing downwards tendency that was, however, not signicant. Resuming the results of the paired t-test for all condition dierences, as shown in table J.1, only one experimental subject showed two times signicant results for
118
feelSpace CI-WI p-value p-value (tendency) (neg. di.) 0.0090 0.0016 0.0574 0.0010 0.1982 0.2566 CI-WI p-value p-value (tendency) (neg. di.) 0.0104 0.0026 0.794 0.1580 0.5964 0.0006 CI-NI p-value p-value (tendency) (neg. di.) 0.0116 0.6321 0.0011 0.0954 0.5247 CI-NI p-value p-value (tendency) (neg. di.) 0.0099 0.0469 0.7465 0.0227 0.4779 NI-WI p-value p-value (tendency) (neg. di.) 0.0078 0.0339 0.5949 0.2369 0.9537 0.9193 NI-WI p-value p-value (tendency) (neg. di.) 0.6181 0.0373 0.4799 0.0691 0.3718 0.2611
Exp. Subjects CS ID PK UW
Controls BJ BH JS SES
Tab. J.1: Results of paired t-test for time measurements. p-values of the paired t-test for downward tendency and negative dierence are listed for all experimental subjects and the four trained controls. Signicant values are presented in bold print.
the downwards tendency as well as for the negative dierence of the post-training result. The same holds for one control. Furthermore, one other experimental subject as well as a further control showed signicant changes but only in either downwards tendency or negative dierence. As a result we can observe some signicant changes in performance after training for some experimental subjects but as well for some controls. This high variability prohibits a clear distinction between the two experimental groups.
J.4.2 Analysis of performance in path-length

Figure J.7 shows an example for one of the paths run during a trial by a control. Since it is not appropriate to expect from our subjects to traverse via the optimal path from one box to another, as this would require prior knowledge of the map, we restrained our analysis to measuring the path-length of the subjects trajectories. The analysis of performance in path-length was done very similar to that of the time. We therefore just refer to the signicance table for path-length (Table J.2), and summarize the results. We found results comparable to the time measurements for the experimental subjects CS and PK as well as for the controls BJ and SES. The only dierence compared to time performance regards the two experimental subjects ID and UW. This slightly dierent outcome, however, should not be taken into account since it refers to a signicant negative dierence without an expected signicant downwards tendency that is in one case even in in the opposite direction.
119
feelSpace
Fig. J.7: Example of a subjects trajectory. The gure shows one of the maps of Paris. The numbers indicate the numbered boxes, the blue dots depict the path a subject has run during one trial. As can be seen clearly, the boxes are distributed in a way that the subject has to traverse boxes that he has to collect later on. Hereby, we ensured that position of boxes which had to be collected later on could be learned during a trial. The gure shows that the subject prefers some of the routes to others, obviously often not knowing where a box is located.
As a result, the path-length analysis does not reveal additional eects which are not already observed in the analysis of time performance.
J.4.3 Summary of Results

Our analyses show some signicant changes in performance in some experimental subjects as well as some controls after the training period. However, the variability between individuals is too high to assume a general eect which distinguishes the group of experimental subjects from the controls. Hence, the results from the virtual environment experiment do not show evidence for the Strong Integration hypothesis and neither for the Subcognitive Processing hypothesis.
J.5 Discussion
Dierent possible explanations might account for the high variability in the data we acquired. We would like to address some of these possible problems here. Dierent navigational strategies - may have led to dierences in individual performance. Interviews after the training revealed a dierence in strategies
120
feelSpace CI-WI p-value p-value (tendency) (neg. di.) 0.0030 0.0022 0.8705 0.0012 0.0092 CI-WI p-value p-value (tendency) (neg. di.) 0.0124 0.0028 0.7482 0.0761 0.6271 0.0003 CI-NI p-value p-value (tendency) (neg. di.) 0.0625 0.1334 0.0427 0.8687 0.7912 CI-NI p-value p-value (tendency) (neg. di.) 0.0080 0.0461 0.6049 0.0040 0.2107 NI-WI p-value p-value (tendency) (neg. di.) 0.0024 0.0186 0.4017 0.3214 NI-WI p-value p-value (tendency) (neg. di.) 0.0806 0.4680 0.0733 0.2847 0.6217
Exp. Subjects CS ID PK UW
Controls BJ BH JS SES
Tab. J.2: Results of paired t-test for path measurements. The table is organized likewise to table J.1 (p. 119) but this time representing p-values for downward tendency and negative dierence of path-length measurements.
that the subjects employed to orientate in the virtual environment. While some subjects relied heavily on visual cues, others tried to navigate with the help of their intuition. Theses dierences in strategies might result in dierences in the eect of the belt: the inuence of the belt might be stronger if navigation is based on intuition rather than on visual cues. These dierences in strategies might cause high variability in performance of individuals. High complexity of the maps - Since we wanted to avoid conscious deliberation about the information provided by the belt, the environment was designed highly complex. However, this high complexity of the task might have induced highly variable results. To account for such a high variability, we could have increased the number of test trials. But since the workload for single subjects was already very high, the virtual environment test required already about 90 minutes, introducing more trials was not an option for our test settings. Crossmodal eects - During the navigation task subjects are provided visual cues that can be used for orientation as well as the orientation information provided by the belt. Since visual perception plays a crucial role in normal human perception it might dominate other perceptual cues. In our test, visual orientation might have covered any eect coming from the belt-originated perception of orientation. In this case, our results would reect eects of visual information on navigation performance rather than the eects of the belt. Absence of proprioception - In the virtual navigation task proprioceptive infor-
121
feelSpace
mation about orientation is completely absent. However, improving eects of the belt on orientation performance may rely on proprioception. I.e. if the belt just serves as an extension of orientation information already provided by proprioception, an elimination of proprioception might prevent any eects of the belt.
122
Bibliography
[1] N. Burgess, E.A. Maguire, and J. OKeefe. The human hippocampus and spatial and episodic memory. Neuron, 35:625641, August 2002. [2] ID-Software. Quake III Arena. http://www.idsoftware.com/games/quake/quake3-arena/, 1999. [3] W.J. Jacobs, H.E. Laurance, and K.G.F. Thomas. Place learning in virtual space I: Acquisition, overshadowing, and transfer. Learning and Motivation, 28:521541, 1997. [4] W.J. Jacobs, K.G.F. Thomas, H.E. Laurance, and L. Nadel. Place learning in virtual space II: Stimulus control. Learning and Motivation, 29:288308, 1998. [5] A. Ketzel. Untersuchung zur Verwendbarkeit von 3D-Spielumgebungen zur Simulation virtueller Umgebungen in psychologischen Experimenten. Diplomarbeit, Johann Wolfgang Goethe-Universitt Frankfurt am Main, 2003. [6] R.G.M. Morris. Spatial localization does not depend on the presence of local cues. Learning and Motivation, 12:239260, 1981. [7] R.G.M. Morris. Development of water-maze procedures for studying spatial learning in the rat. J. Neurosci. Methods, 11:4760, 1984. [8] S.D. Steck, H.F. Mochnatzki, and H.A. Mallot. The role of geographical slant in virtual environment navigation. Technical Report 87, Max Planck Institute for Biological Cybernetics, Tbingen, Germany, Jan. 2001. [9] J.J. La Viola. A discussion of cybersickness in virtual environments. SIGCHI Bulletin, 32(1):4755, 2000.
123
Physiological Tests
Isabel Dombrowe University of Osnabrck u
K Physiological Tests
According to the strong integration hypothesis the heading information provided by the belt can be rmly integrated in human perception. Sensory input of the belt that is inconsistent with other sensory inputs should therefore produce measurable responses. We used two dierent methods to test this hypothesis: posturography and nystagmography.
K.1 Posturography
We used posturography to investigate the inuence of conicting heading information on body sway using a standard procedure for balance testing (the Romberg test, see K.1.1). To test the strong integration hypothesis, we decided to provide additional as well as conicting heading information during posturography: Subjects received correct information by wearing the feelSpace-belt that continuously informed them about their orientation relative to the earth magnetic eld. In order to provide conicting information, the belt was driven externally by a computer program (see appendix F.3.4) that simulated a continuous rotation around the vertical axis. We expected an increase in body sway for the experimental subjects when they were provided with conicting information in the post-training tests.
K.1.1 Methods
There were two trials: eyes open and eyes closed. Each trial consisted of three conditions: no belt information, correct belt information and wrong belt information. We compared the wrong belt information condition where the belt provided information that was inconsistent with all other sensory inputs to the no belt information condition. Additionally, we compared the correct belt information condition to the no belt information condition. In each trial subjects had to stand still on a posturography - platform (Luzerner Messplatte by Otopront) for 30 seconds per condition while changes in their center of balance were continuously recorded. This procedure, the Romberg test, is commonly used for balance testing. The total length of the sway path was computed. Data were analyzed using the statistics toolbox of the matlab software package [2]. We computed the logarithmic ratios and the dierences between the no belt information condition that served as a baseline and the conicting information as well as the correct belt information condition.
125
feelSpace
We tested all participants (experimental subjects, trained controls and na ve controls) in the pre-training experiments. The na controls did not take part ve in the post-training experiments (see K.1.3).
K.1.2 Results
Pre-training posturography revealed large dierences between participants. Table K.1 exemplarily shows the means and standard deviations for the no belt information condition in the eyes closed trial. In order to get an overview of the mean sway path std. dev. sway path 39,1 cm 15.6153 cm
Tab. K.1: Means and standard deviations of the sway path the no belt information condition in the eyes closed trial.
results of the individual participants, we calculated the logarithmic ratio of the sway path in the wrong belt information and the no belt information condition (see gure K.1). At a rst glance pre-training results did not seem to reveal any signicant dierences between conicting heading information and no heading information. This impression was conrmed by a one-way analysis of variance. We additionally compared the no belt information condition to the correct belt information condition. There were no signicant dierences (i.e p > 0.05 for all tests) regarding the sway path among any of the conditions. Figure K.2 displays the complete data of the pre-training posturography experiments. Neither wrong belt information nor correct belt information inuenced the body sway of the subjects before the training phase. Post-training posturography yielded results that were inconsistent with our expectations. The body sway of two of the experimental subjects was not increased when they received conicting belt information after six weeks of training with the belt. Figure K.3 shows a comparison of the pre-training and post-training results for all four experimental subjects. The gure depicts the logarithmic ratio of the total length of the sway path. The ratio of the wrong belt information condition and the no belt information condition increased as expected for only two of the experimental subjects. For the other two experimental subjects this ratio was decreased. Figure K.4 gives an overview of the posturography results. Since there were no signicant dierences between trials and all tested conditions we only exemplarily show data of the wrong belt information in the eyes closed trial. In contrast to our expectations the body sway of the majority of the experimental subjects did not increase when the belt provided wrong information. Only one of the experimental subjects, but also one of the trained controls showed a clear increase in body sway.
126
feelSpace
0.12 0.1 log(belt rotates / belt is switched off) 0.08 0.06 0.04 0.02 0 0.02 0.04 0.06 0.08 subjects controls
Fig. K.1: Pre-training posturography: Each data-point in the plot displays the logarithmic ratio of the wrong belt information condition and the no belt information condition in one of the two trials (eyes open, indicated by stars, and eyes closed, indicated by crosses). The arrow indicates the expected direction of change for the post-training tests.
127
feelSpace
10
CINI
10
15
20
eyes open eyes closed
25 subject
10
WINI
10
15
20
eyes open eyes closed
25 subject
Fig. K.2: Pre-training posturography: The upper plot displays the absolute dierence in cm between the correct belt information condition and the no belt information condition for each trial (eyes open, indicated by crosses and eyes closed, indicated by stars). The lower plot shows the absolute dierence in cm between the wrong belt information condition and the no belt information.
128
feelSpace
sway path 0.16 0.14 0.12 log (belt rotates / without belt) 0.1 0.08 0.06 0.04 0.02 0 0.02 0.04 CS ID PK experimental subject UW pretraining posttraining
Fig. K.3: Post-training posturography: Data-points in the plots display the logarithmic ratio of the wrong belt information condition and the no belt information condition in the eyes closed trial for each of the experimental subjects. Crosses indicate pre-training results. Post-training results are indicated by stars.
K.1.3 Discussion
According to the strong integration hypothesis the heading information provided by the feelSpace belt can be rmly integrated in human perception such that beltinformation that is inconsistent with other sensory inputs produces measurable responses. We provided conicting belt-information by simulating a rotation of the subjects body around the vertical axis during posturography. In contrast to our expectations the body sway of the majority of the experimental subjects did not increase after six weeks of training with the feelSpace belt. Only one of the experimental subjects, but also one of the trained controls had a clearly increased body sway after the training phase. The results of the experimental subjects are consistent with the results of the other experiments we conducted (see appendices H, I and J) and with the subjective reports (see appendix L). The experimental subject that had a marked increase in body sway after the training phase when provided with conicting information also performed well in virtual and outdoor navigation tasks. He was the experimental subject that reported the most striking change in subjective experience. The interpretation of the results of the trained controls is somewhat unclear. The increased body sway of one of the trained controls could be due to the weekly orientation training during the training phase (see appendix G). In order to exclude or verify a possible training eect we would have needed
129
feelSpace
14 12 10 difference (WINI) 8 6 4 2 0 2 4 CS ID PK UW
pretraining time
posttraining
14 12 10 difference (WINI) 8 6 4 2 0 2 4 BJ BH SES
pretraining time
posttraining
Fig. K.4: Post-training posturography: A comparison of the body sway before and after the training phase. The total length of the sway path in the wrong belt information condition was subtracted from the total length of the sway path in the no belt information condition. After six weeks of training a marked increase in body sway could be observed for only one of the experimental subjects. For one of the trained controls the body sway was unexpectedly increased.
130
feelSpace
post-training data of the na controls. This was originally planned and the ve pre-training tests also included the group of na controls, but the time schedule ve of the whole project did not allow to include them in the post-training tests. Although the results of the posturography are consistent with the other experiments we conducted, they do not provide a clear support for the strong integration hypothesis.
K.2 Nystagmography
There are three vestibulo-ocular reexes that compensate for head movements in order to keep the image on the retina stable. The translational reex compensates for linear head movement, the ocular counter-rolling response compensates for head tilt in the vertical direction and the rotational vestibulo-ocular reex (nystagmus) compensates for head rotation: When the head rotates in one direction, the eyes slowly move in the opposite direction. This movement - the slow phase of the vestibular nystagmus - is followed by a rapid resetting movement the quick phase of the vestibular nystagmus. We used electro-nystagmography to investigate the eects of the additional heading information that was provided by the feelSpace-belt on the vestibular nystagmus. It is known that vestibular signals habituate during sustained rotation in the dark. Since the brain uses the visual system to supplement the vestibular system, this is not the case in a lit environment [1]. We expected an post-training increase in the number of nystagmi occurring during body rotation (per-rotatory nystagmus) for the experimental subjects, since the feelSpace-belt continuously informed them about the rotation. For the nystagmus occurring after three minutes of body rotation (post-rotatory nystagmus) we expected a decrease for the experimental subjects, because they received additional heading information by the feelSpace-belt that told them that the rotation had stopped.
K.2.1 Methods
Participants sat on a rotatory chair (Hortmann CNG) in a dimly lit room. After a short calibration procedure they closed their eyes and the chair started to rotate. After an acceleration phase of 30 seconds, the chair rotated at a constant speed for three minutes and then stopped abruptly. We recorded their eye movements with the help of six electrodes placed around the eyes of the participants (see gure K.5). Nystagmi occurring during the rst 30 seconds of constant rotation were summed up over the whole time period. The post-rotatory nystagmus was measured in time bins of ve seconds during the rst 30 seconds after the chair was stopped. There were two trials: correct belt information and no belt information. Each trial consisted of two conditions: chair rotates clockwise and chair rotates counter-clockwise that for technical reasons were always carried out in the same order. So each trial started with the chair rotates clockwise condition that was
131
feelSpace
Fig. K.5: Electrode arrangement for electro-nystagmography
followed by a break of four minutes that allowed the vestibular system to settle. This break was immediately followed by the chair rotates counter-clockwise condition. Data were analyzed using the statistics toolbox of the matlab software package [2].
K.2.2 Results
Pre-training nystagmography revealed large dierences among participants for the per-rotatory nystagmus as well as for the post-rotatory nystagmus (see table K.2 and table K.3). condition no belt counterclockwise correct belt counterclockwise no belt clockwise correct belt clockwise mean 44,1429 45,4286 46,7857 46,2857 standard deviation 9,9449 12,2645 12,5341 11,7632
Tab. K.2: Means and standard deviations of the per-rotatory nystagmus data.
condition no belt counterclockwise correct belt counterclockwise no belt clockwise correct belt clockwise
mean 46,5 41,1429 48,4286 45,4286
standard deviation 16,5006 10,5163 14,7111 13,34
Tab. K.3: Means and standard deviations of the post-rotatory nystagmus occurring in the rst 30 seconds after the rotation of the chair was stopped.
We compared the correct belt information trial to the no belt information trial. Figure K.6 shows an overview of the per-rotatory nystagmus data. Each datapoint displays the logarithmic ratio of the correct belt information and no belt information trial for each participant. An eect of the belt could not be detected. A comparison of the correct belt information to the no belt information trial for the post-rotatory nystagmus showed only a slight dierence of both trials when
132
feelSpace
0.15
0.1
0.05 log(belt on/belt off)
0.05
0.1
0.15
0.2 subjects controls
Fig. K.6: Pre-training per-rotatory nystagmus: Each datapoint in the plot displays the logarithmic ratio of the correct belt information trial and the no belt information trial trial avaraged over both conditions. The arrow indicates the expected direction of change.
we averaged over conditions. Subjects as well as controls had on average less nystagmi with the belt than without belt. Figure K.7 shows the timecourse of the post-rotatory nystagmus for subjects and controls separately. Post-training nystagmography did not yield the expected results for the postrotatory nystagmus, but for the per-rotatory nystagmus. Unfortunately, it was unclear if these results could be interpreted as supporting the strong integration hypothesis (see K.2.3). Figure K.8 depicts a comparison of the per-rotatory nystagmus data for the experimental subjects before and after training. The logarithmic ratio of the with belt and without belt trial increased as expected for three of the four experimental subjects. We obtained results that were contrary to our expectations comparing the pre-training to the post-training post-rotatory data for the experimental subjects. The number of post-rotatory nystagmi was not signicantly lower with the belt than without the belt after the training phase. Instead, the time course of the post-rotatory nystagmus diered to a large degree from experimental subject to experimental subject. Figure K.9 shows the post-rotatory nystagmus data for each experimental subject before and after training for both trials (with belt and without belt). As a common trend could not be detected, we did not do any further data analysis.
133
feelSpace
experimental subjects 70 without belt with belt 60
50 nystagmus (number)
40
30
20
10
05
010
015 020 time (seconds) controls
025
030
70 without belt with belt 60
50 nystagmus (number)
40
30
20
10
05
010
015 020 time (seconds)
025
030
Fig. K.7: Pre-training post-rotatory nystagmus: The two plots display the timecourse of the post-rotatory nystagmus after three minutes of rotation. The duration of the post-rotatory phase is 30 seconds, divided into six segments. In each segment, the averaged cumulative sum of observed nystagmi is depicted. Additionally, the standard deviations are shown. The arrow indicates the expected direction of change.
134
feelSpace
0.25 0.2 0.15 0.1 0.05 0 0.05 0.1 0.15 pretraining posttraining
log (with belt / without belt)
CS
ID PK experimental subject
UW
Fig. K.8: Per-rotatory nystagmus: Pre- and Post-training per-rotatory nystagmus data for the experimental subjects. Each data-point displays the logarithmic ratio of the with belt and without belt trial for each experimental subject. For three of four experimental subjects the number of nystagmi increased as expected. Unfortunately it was unclear if this result was valid (see K.2.3).
135
feelSpace
Fig. K.9: weeks of pre- and trials. A
Post-rotatory nystagmus of the experimental subjects before and after six training with the feelSpace-belt. Each plot displays the time-course of the post-training post-rotatory nystagmus for one experimental subject in both common trend could not be observed.
136
feelSpace
K.2.3 Discussion
We investigated the inuence of the heading information provided by the feelSpace belt on the vestibular nystagmus. Due to the interaction of the visual system and the vestibular system this vestibulo-ocular reex vanishes during a sustained rotation in the dark, but not in a lit environment. To test the strong integration hypothesis the participants in the nystagmography experiment received additional heading information during a three minute rotation without visual input. As the vestibular system habituates after some time and the subjective sense of rotation stops after some seconds, the belt-information was inconsistent with the usual perception during rotatory testing. We expected an increase in the number of per-rotatory nystagmi yielding results similar to a rotation in a lit environment. Additionally we supposed that the post-rotatory nystagmus would decrease since the subjects were informed by the feelSpace-belt that the rotation had stopped. According to our data the post-rotatory nystagmus did not decrease when the experimental subjects received additional heading information. Although our results indicate that the per-rotatory nystagmus of all but one experimental subject however was indeed increased after the training phase with the belt we still cannot conclude that the nystagmography experiments support the strong integration hypothesis. This is due to technical problems one of the experimenters experienced during the post-training tests. He reported a strange behavior of the equipment that he could not relate to any apparent cause. As a consequence, a validation of our results regarding the per-rotatory nystagmus necessitates a repetition of this experiment.
137
Bibliography
[1] E. R. Kandel, J.H. Schwartz, and T. M. Jessell. Principles of Neural Science. Mc Graw Hill, 2000. [2] MathWorks. Matlab Software Package. Natick, MA, USA, 2005.
138
Subjective Report
Christiane Kabisch University of Osnabrck u
L Subjective Report
L.1 Introduction
Subjective reports are a controversial means to measure experimental outcomes. Introspection is prone to error, i.e. false memories and problems with expressing subjective experiences. Results acquired via introspection are ill-suited for inter-subject comparison. When trying to obtain measures on a numerical scale, subjective reports are prone to distortion due to individual reference baselines. Still, for some questions, especially those concerning qualitative experience, subjective reports are the only way to nd an answer. Only introspection can decide whether a certain change in sensations has been experienced. Therefore we designed interviews and questionnaires for our subjects to evaluate the sensory quality of the belt after learning the sensory-motor contingencies.
L.2 Motivation
New Modality Hypothesis Neither physiological measurements nor navigational performance tests can give us insight what the quality of spatial perception with the belt is. However, the New Modality Hypothesis, our most ambitious hypothesis, makes an assumption about that quality: it states that the input from the belt will give rise to a new sensory experience after wearing the belt for a sucient amount of time. In order to test for this hypothesis, we formulated some questions to our subjects and control subjects. This was our main target hypothesis for the subjective reports. Subcognitive Processing Hypothesis Another hypothesis that might be either supported or refuted by subjective reports is the Subcognitive Processing Hypothesis: Did the subjects have to attend consciously to the belts vibrations or did they have an intuitive knowledge about where north is is? Did they have the impression that there was a constant subconscious information ow? Weak Integration Hypothesis Additionally, we also wanted subjective reports to test for the Weak Integration Hypothesis. We were interested whether the subjects themselves judged their spatial abilities as superior to their abilities before wearing the belt. Although the spatial performance improvements have already been tested in the outdoor and virtual environment tests (appendices H, J), self-evaluation might reveal a dierent aspect of improved spatial knowledge,
140
feelSpace
e.g. more certainty and reliability in navigating and relating locations to each other.
L.3 Methods
L.3.1 Diary
Our subjects were required to ll out a daily questionnaire during the six weeks they were wearing the belt. One purpose for using the diary was to assess the daily use of the belt and eventual technical problems. The other purpose was making the subjects consciously think about their subjective changes in navigational abilities and qualitative experience of space, so they would also notice small aspects of this. Over time, subjects might forget about changes in their experience, and we wanted them to notifying us rst. The diary consisted of 20 questions most of which only concerned the use of the belt in order to ensure that it had been worn without problems. Others addressed the sense of orientation and asked whether the quality of spatial perception had changed.
L.3.2 Outdoor Training Questions

After each of the almost-weekly outdoor training sessions, we asked the subjects and the controls questions about changes of their navigational abilities and whether they think their performance in the outdoor training had improved. Thereby, we wanted to nd out if subjects and controls would perceive the development of their performance in the outdoor training and navigational skills dierently.
L.3.3 Interview
After the post-training test, we had a nal interview with the subjects and the controls consisting of 20 questions. The purposes of this additional interview were to go more into detail, to compare the subjective change in spatial abilities of control subjects with the subjects that had been wearing the belt for six weeks, and to get information about possible developments over the weeks of wearing the belt. Typical questions addressed the spatial orientation and knowledge about angles and distances between landmarks in the subjects hometown before, while and after wearing the belt. The questionnaire from this interview is as follows (in German because all our participants are native Germans): Final Interview Questions Wie bewusst war euch der Grtel? Wenn etwas strte, war es der Grtel, u o u die Vibration oder beides?
141
feelSpace
Warst du whrend des Tragens in der Lage, die Positionen (persnlich) a o wichtiger Landmarks, in Bezug zur eigenen Position, anzugeben (z.B.: Zuhause, Uni, Mensa, Bahnhof...). War das vorher schon so? Wie war es nachher? Wusstest du, wie je zwei Landmarks im Verhltnis zueinander lagen? a War das vorher schon so? Wie war es nachher? Hattest du das Gefhl, durch den Grtel von einer Graphen-Reprsentation u u a (nur das Wissen uber Orte und Verbindungen) zu einer 2D-Reprsen a tation (kartenhnlich) gewechselt zu haben? a Nach Ende der Tragezeit: Fehlt etwas? Falls der Grtel Orientierungsvorteile gebracht hat: Blieb der Orientieru ungsvorteil erhalten, nachdem der Grtel abgelegt wurde. u Hattest du das Gefhl, Magnetfelder oder hnlich orientierte Felder (Nordu a wahrzunehmen? Sd Achsen o.A.) u Hatte der Grtel uberhaupt einen Eekt? Welche Vernderung ist uber die u a Tragezeit aufgetreten? Musste man sich gezielt auf den Grtel konzentrieren um ihn zu nutzen, u oder war die Informaton intuitiv vorhanden und intuitiv nutzbar? Hattest du das Gefhl, dass die Richtungsinformation eine Eigenschaft der u Umwelt war, oder musste man sie von der Haut ablesen? Hattest du den Eindruck, dass die Richtungsinformationen, die der Grtel u lieferte, eine besondere Qualitt hatten? Waren sie VOM GEFUHL etwas a anderes als taktile Stimulation? Gab es noch andere Besonderheiten in der Richtungswahrnehmung, die sich schwer in Worte fassen lassen, aber denitiv QUALITATIV NEU und an den Grtel gebunden waren? u FALLS eine spezielle Qualitt vorhanden war : Wrdest du sagen, dass a u es ein neuer Sinn war? FALLS eine spezielle Qualitt vorhanden war : Hast du dieses Gefhl a u jemals ohne Grtel gehabt? u
142
feelSpace
L.3.4 Evaluation
The diary and the outdoor training questions were checked carefully for important aspects, but not evaluated quantitatively. The results of the interview were rated on a point scale. An answer that was consistent with our hypotheses would be rated with one point, an answer that was inconsistent with our hypotheses would be rated with zero points. For example, we would ask the questions: 1. While wearing the belt, did you know how dierent landmarks were located in relation to the others? 2. Was that already the same before you had the belt? 3. Was that still the same after you stopped wearing the belt? We would assign a point if the subject answered question 1 with yes, because according to our theory (in particular the Weak Integration Hypothesis and the Strong Integration Hypothesis), the subject should indeed be able to relate between the locations of dierent landmarks. If the subject had answered question 2 with yes, this would be no point, as it means that the subjects sense for orientation had not actually improved because of the belt. The subject must have had (or at least thought to have had) an adequate internal representation of its surroundings already before wearing the belt. If, however, the subject had answered question 2 with no, this would be a consistent with the assumption that the belt had played a major role in building up an internal spatial representation or improving it; therefore we would assign a point. The answer to question 3 would be rated with one point if it was no. If the subjects knowledge about the orientation of landmarks remained stable after removing the belt, then the sensory input of the belt had been required no longer. This suggests it hadnt been used after an initial phase of building up or correcting an internal spatial representation of the surroundings. So, if the answer to question 3 was yes, no point would be assigned. It is necessary to note that this evaluation design leads to a baseline that may be more than zero, i.e. somebody who had not been trained with the belt but nevertheless has a good sense of orientation might reply to question 1 with yes and thereby yield one point. However, our aim is not to compare the subjects results with zero but to compare the experimental subjects results with the control subjects results. To get a 10-point scale, the sum of points was nally multiplied by 0.2.
L.4 Results
The diaries did not provide us with important information about changes of the sensory experience or spatial representations with the belt. Likewise, the outdoor
143
feelSpace
training questions showed no obvious dierence in the self-perceived improvement in navigational skills between experimental subjects and controls. The interviews after the post-tests were more revealing. The reported experiences within the experimental subject group varied highly. Overall, two of the experimental subjects (UW and PK) experienced a qualitative change in their spatial representations whereas the other two (ID and CS) did not. As expected, for the control group there was no qualitative change. Experimental subjects UW and PK The two subjects that reported qualitative changes said the input from the belt reected properties of the environment rather than being a mere tactile stimulation: It was dierent from mere tactile stimulation because the belt mediated a spatial feeling. [quote by UW]. Their orientation improved while they were wearing the belt, and they were more aware of the spatial relations between dierent locations: I was intuitively aware of the direction of my home or of the AVZ. For example, I would wait in line in the Mensa and spontaneously think: Im living over there. The orientation on a trip to Berlin and Hamburg was interesting. After coming back, I could retrieve detailed memories of all places, rooms and buildings and describe them. [quotes by PK]. For one subject (PK), this eect disappeared when he stopped wearing the belt: While wearing the belt, I didnt realize how much my perception had changed. After removing the belt, my living space diminished quickly: the world appeared smaller and more chaotic because relative positions to places beyond the visual horizon were rather unordered. [quote by PK]. However, UW did not feel a change in his spatial perception after removing the belt. Both PK and UW reported that, after some time, they did not have to focus their attention on the belt to retrieve the information where north was located: During the rst two weeks, I had to concentrate on it; afterwards, it was intuitive. [quote by UW]. Experimental subjects ID and CS The other two experimental subjects ID and CS did not experience changes in their sensory experience.They said they had to concentrate to access the information provided by the belt. The stimulation was perceived as tactile sensation and even disturbing at times, especially for ID, whereas CS sometimes even did not notice when the belt was accidentally switched o:
144
feelSpace
a) Experimental Subjects.
b) Trained Controls.
Fig. L.1: These diagrams depict the results on a 10-point scale for the subject (a) and trained control group (b).
I really felt very little, but I had the feeling I was better with the belt at least at the natural-environment orientation tests. Subjectively, I couldnt feel the dierence between the virtual environment test trials with incoherent, with coherent and without belt information. [quotes by CS]. None of the subjects rated the sensory experience as comparable to a new sense. Control subjects As we expected, the control subjects did not experience a change in their spatial perception, although one of them reported that, while wearing the belt, he had a better understanding of the spatial properties of the environment. They had to concentrate on the vibration to be able to use the information. Point distribution for the interviews In the experimental subject group, the mean score of points was 4.5 points. In the control group, the mean score of points was 1.5 points. The resulting distribution is depicted in gure L.1.
145
M Acknowledgements
We would like to thank: PNI corporation, Santa Rosa (CA, USA), for the donation of a TCM2 compass, ESD Kringe, Netphen, for technology support and housing the belt workcamp, the Universittsklinikum Mnster for consultancy and providing the equipa u ment for the posturography and nystagmography, Kamil Vlcek and Jan Bures at the Institute of Physiology - Academy of Sciences Czech Republic, Prague, for their collaboration concerning the Blue Velvet Arena, the Elektro Werkstatt and Feinmechanische Werkstatt of the University of Osnabrck for manufacturing the circuit boards. u
http://www.feelSpace.de
146

!!!!FeelSpace Final Report

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

!!!!FeelSpace Final Report

Загружено:

Авторское право:

Доступные форматы

feelSpace

Report of a Study Project

Alper Aik c Isabel Dombrowe Tobias Kringe Robert Mrtin a

1 The 1.1 1.2 1.3 1.4 1.5 1.6

1 The feelSpace Study Project

1.2 Theoretical Background

1.4 Experimental Design

Organization, Documentation and Public Relations

A Organization, Documentation, and Public Relations

Fig. A.1: The project logo

A.4.2 Ethics Commission

Fig. A.2: The project website

Fig. A.3: Document templates

A.4.5 Public Presentation

B.2 Sensory Substitution devices and experiments

B.3 How does sensory substitution work?

Boris Bernhard & Saskia Nagel University of Osnabrck u

C.1.1 Perception and action are intermingled

C.1.2 A new model of neural plasticity

C.1.3 Test of new model - the feelSpace project

Magnetic Sense in Animals

Lina Jansen & Hannes Saal University of Osnabrck u

D Magnetic Sense in Animals

D.2 The Earth Magnetic Field

D.3 The Map and Compass Hypothesis

D.4 Magnetoreceptor Mechanisms

D.5 Testing for a Magnetic Sense

D.6 Case Study: Navigation in Sea Turtles

D.7 A Human Sixth Sense?

Hannes Saal & Christian Stel o University of Osnabrck u

E.2 A Denition of Augmented Reality

E.5 Compasses and Augmented Reality

E.6 Vibrators and Augmented Reality

E.7 Implications of these studies

F.1 The Compass

F.1.2 Literature Search

F.1.4 Digital Magnetic Compasses

F.1.5 The Candidates

F.1.6 Testing the PNI Compass

F.1.7 The Decision

F.2.2 Overview of Tactile Feedback Devices

Distortion is, in this context, some kind of mechanical stress.

F.2.3 Sanko Electric 1e120

F.3.2 Basic Design Issues

F.3.3 Hardware Design

Fig. F.5: Top side of the BeltLogic 1.0 circuit board.

Fig. F.6: Bottom side of the BeltLogic 1.0 circuit board.

F.3.4 Software Design

F.3.5 Power Supply

Fig. F.7: The circuit board with attached C-Control II Unit.

Fig. F.8: The Teko enclosure with attached cables.

We chose the model designed for women as it is compatible to both genders.

F.5 Specications and Final Device

G.2 Experimental setup

G.2.2 Experimental groups

Group Experimental Subjects Trained Controls Na Controls ve Total

mean age 30.5 23.0 21.5 25.0

G.2.3 Test conditions

Orientation in Natural Environment

Alper Aik & Christiane Kabisch c University of Osnabrck u

H Orientation in Natural Environment

SHAPES USED IN THE OUTDOOR HOMING TESTS

a = 2.5 b=5 c = 4.5 d = 3.5

30 Difference in Performance (CI NI) Control Ss Experimental Ss 20