D4.8.3 BLUE Experiment Results and Evaluation v1.0

D4.8.
3 BLUE Experiment Results and Evaluation

2013-10-07
George Lepouras, Angeliki Antoniou, Jenny Rompa, Costas Vassilakis (University of Peloponnese) Ioanna Lykourentzou, Yannick Naudet, Eric Tobias (Henri Tudor Public Research Centre)
This deliverable presents the results of the BLUE experiment, regarding the EXPERIMEDIA components employed by the "My Museum Story" and "My Museum Guide" applications and regarding the usability, efficiency and effectiveness of the applications. The document outlines the experiment process, presents and analyses the results and discusses the experiment outcomes.
www.experimedia.eu
EXPERIMEDIA Project acronym EXPERIMEDIA
Dissemination level: PU
Full title Experiments in live social and networked media experiences Grant agreement number 287966 Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Experimentation (FIRE) Project start date 2011-10-01 Project duration 36 months Activity 4 Experimentation Workpackage 4.8 EX8: BLUE Deliverable lead organisation University of Peloponnese Authors George Lepouras, Angeliki Antoniou, Jenny Rompa, Costas Vassilakis (University of Peloponnese) Ioanna Lykourentzou, Yannick Naudet, Eric Tobias (Henri Tudor Public Research Centre) Reviewers Sergiusz Zieliski (Pozna Supercomputing and Networking Center) Stephen C. Phillips (IT Innovation Centre) Version 1.0 Status Final Dissemination level PU: Public Due date PM24 (2013-09-30) Delivery date 2013-10-07 Future Internet Research and
Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013
EXPERIMEDIA
Table of Contents
1. 2. 3. Executive summary ............................................................................................................................ 5 Introduction ........................................................................................................................................ 6 Experiment overview ......................................................................................................................... 7 3.1. Experiment design .................................................................................................................... 7 3.1.1. Changes since D4.8.2 ........................................................................................................... 7 3.1.2. Target group .......................................................................................................................... 7 3.1.3. Experimental factors ............................................................................................................ 7 4. Experiment results............................................................................................................................ 11 4.1. Method...................................................................................................................................... 11 4.1.1. Qualitative analysis.............................................................................................................. 12 4.1.2. Quantitative evaluation metrics ........................................................................................ 12 4.2. Analysis of experimental results ............................................................................................ 13 4.2.1. Facebook game results analysis ......................................................................................... 13 4.2.2. On-site results analysis ....................................................................................................... 18 4.2.3. Usefulness of cognitive style in recommendations ........................................................ 22 5. EXPERIMEDIA component assessment .................................................................................... 24 5.1. 5.2. 6. 7. 8. EXPERIMEDIA Experiment Content Component (ECC) ............................................ 24 EXPERIMEDIA Social Content Component (SCC) ....................................................... 24
Discussion .......................................................................................................................................... 25 Conclusion ......................................................................................................................................... 29 References.......................................................................................................................................... 30 Semi-structured interview ............................................................................................. 31 Avatars, pets and tools Icons ....................................................................................... 37
Appendix A. Appendix B. B.1. B.2. B.3.
Avatar icons ............................................................................................................................. 37 Pet icons ................................................................................................................................... 37 Tool icons................................................................................................................................. 38 Museum templates ......................................................................................................... 39
Appendix C.
EXPERIMEDIA List of Tables
Table 1. Phase T0 - Factors ....................................................................................................................... 8 Table 2. Factors captured by the questionnaire ...................................................................................... 8 Table 3. Users' Estimated and Actual Cognitive Style ........................................................................ 14 Table 4: opics associated to exhibitions, corresponding to cognitive style dimensions. ............. 23
EXPERIMEDIA
1. Executive summary
The goal of the BLUE experiment of EXPERIMEDIA is to explore the use of visitors' cognitive styles and content interest in order to personalize their experiences inside a museum. The related experimental runs were conducted at the Foundation of the Hellenic World (FHW) in Athens. The goals and requirements for the experiment have been described in EXPERIMEDIA deliverable D4.8.1 and the initial experiment design in D4.8.2. Four experimental sessions were conducted on four different dates. The total number of participants was 30: 15 male and 15 female with an average age of 30. Of the 30 participants 6 people visited alone, 17 with one or more friends, and 7 with family members. The sampling processes followed replicated the processes the museum will apply in order to attract new visitors, advertise exhibitions and also collect relevant information for user profiling. Experiments revealed that a good proportion of visitors liked and followed recommendations, had a positive impression of using the recommender and reported the mobile guide enhanced their quality of experience. The Facebook game could predict user interests and cognitive profile. Furthermore, visitors were happy with the provided Point of Interest descriptions. Overall the results are promising, as the analysis of our experimental runs has shown the usefulness of the BLUE technologies and their suitability to improve user experience.
EXPERIMEDIA
2. Introduction
Following their design and implementation, the EXPERIMEDIA BLUE applications, "My Museum Story" (MMS) and "My Museum Guide" (MMG) were assessed in an experiment performed at the Hellenic Cosmos Cultural Centre of the Foundation of Hellenic World in Athens, Greece. The experiment's aim was twofold: first, to investigate the extent to which the EXPERIMEDIA BLUE experiment achieved its goal of enhancing the Quality of Experience for visitors within the museum, and second, to assess in a real-world setting the EXPERIMEDIA components used in building the BLUE applications. The rest of the document is structured as follows: section 3 gives a brief overview of the experiment, focusing on the changes that had to take place since the initial description of the experiment in D4.8.2, the target group, recorded variables and experiment structure. Section 4 presents the data collection process, and the methodology employed during data analysis as well as the experiments' results. Section 5 describes the assessment of the two EXPERIMEDIA components used in the development of EXPERIMEDIA BLUE, namely the EXPERIMEDIA Experiment Content Component (ECC) and the EXPERIMEDIA Social Content Component (SCC). Section 6analyzesthedata collected during the experiment in relation to the hypotheses made and discusses the findings. The document ends with final conclusions and future work in section 7. The template of the questionnaire employed during experiments can be found at Appendix A.
EXPERIMEDIA
3. Experiment overview
3.1. Experiment design
3.1.1. Changes since D4.8.2

Since the time Deliverable 4.8.2 (describing the initial experiment design) was submitted, changes in FHW exhibitions and ticketing policy caused alterations on the experiment's design. The Mathematical exhibition was replaced by a new one, titled "Your Planet Needs You!", which offers glimpses into the world of 2050 and explores how mankind will survive on a changing planet. Due to this exhibition change, new content had to be created for the recommendation engine to cater for different user profiles. That resulted in abandoning the geolocalised message posting functionality, in an effort to make up for the extra time and effort, as well as to keep up with the original trial schedule. Furthermore, this new exhibition had a more classical monolithic structure, where the visitor has to follow a specific path, making it impossible to build additional virtual exhibitions like we expected to do (see D4.8.2, section 2.2.2). The small number of items to recommend (8 in total plus cafeteria and gift shop), and the fact that 6 of them were shows, which have a fixed schedule and were played most often once per day, meant that we could not give visitors the choice between multiple paths or visit sequences. Actions have been proposed to visitors by the MyMuseumGuide (MMG)application on the sole basis of their interaction with the system, since it was found that the location system was not sufficiently reliable. To help attract subjects for the experiments, FHW altered its ticketing policy offering free entrance for the visitors invited to our trials. We could therefore have a better mix of subjects, with different backgrounds as opposed to what had been originally envisaged in D4.8.2 (section 2.2.3) for phase 2, i.e. the controlled experiment. To this end, Phase 2 and 3 described in D4.8.2 (section 2.2.3) were merged in one single phase of open experiment where for each run we had 7 or 8 participants. As anticipated in D4.8.2 for the open experiment, not all visitors had played the MyMuseumStory (MMS) Facebook game.
3.1.2. Target group

Changes in ticketing policy helped us attract more subjects for our experiments. The experiment was advertised through Facebook. The majority of the subjects were Facebook users who played the MMS game, while the rest had learnt of the experiment from friends. Nevertheless, the number of participants was small.
3.1.3. Experimental factors

As defined in Deliverable 4.8.2, there were 4 main experimental phases: T0, before the user goes to the museum; T1 and T2 which, for the purpose of variable generation can be treated as one step, the preparation of the visit and visit of the museum; and T3, after the visit has been concluded. At T0, different factors were captured, all related to the user. Of these factors, some provided demographic information and others were related to the users game choices and behaviour on the social network. Table 1 summarises the identified factors:
EXPERIMEDIA
Table 1. Phase T0 - Factors
Category Demographics
Factors Age Gender Mother Tongue Facebook ID as provided by Facebook, Number of Facebook friends Avatar Choices: Old-Wise, Artist, Engineer, Scientist, Alien, Rapper, TV-Persona, Mad Scientist, or Diplomat Pet Choices: Cat, Dog, Gold Fish, Monkey, or Owl Tool Choices: Book, Clock, Disco Ball, or Heart Choice of Museum Template Number and type of exhibits they have won playing the game Number and type of exhibits they have placed in-game Number of games they have played
Facebook behaviour Game choices
Also, during phase T0 and after the above factors had been collected, a number of other factors were computed. From the players game choices, their cognitive styles were estimated, since different choices were related to different dimensions of the cognitive style. Similarly, peoples interests were also computed both from their game choices and their cognitive profiles. Additional factors, collected during the experiment in T1|2, had been elaborated and committed in a Google document. These encompass the visitor's: identifier, position, visited exhibitions, pictures taken, comments on pictures and choice to upload their visit to Facebook. A visitor's identifier is either their Facebook identifier, if they chose to provide it by logging in, or a randomly generated UUID. It is used to keep track of a visitor's use of the system. A visitor's position is provided as a geographical position in latitude and longitude as separate but related variables. A visitor's position is used to establish their visiting style. The visitor's engagement with the system is measured by the number of comments and the number of pictures taken using the devices. Their readiness to use and interactivity with social media is measured by their willingness to upload their visit, sharing it with their social network. After the museum visit, in T3, a questionnaire was used to gather data from the visitors. This data had been organized into different categories. A template of the questionnaire is included as appendix. Table 2 summarises the questionnaire categories and questions.
Table 2. Factors captured by the questionnaire
Category General Participant Information
Factors A visitor's Facebook ID (if known) to relate the questionnaire to data collected in T0, T1 and T2. Participants Name
EXPERIMEDIA
Category Factors
Educational Background Whether they were part of a group and if so, whether they held the tablet. Whether they played the Facebook game The short version of the MBTI cognitive style questionnaire My Museum Story related questions Avatar choice, pet choice, tool choice Difficulty in making these choices Rationale behind these particular choices Museum layout template choice Rationale behind this particular choice Self-reported visiting style Number of exhibits they collected Problem regarding process of choosing exhibits Rationale for choosing specific exhibits Whether they invited friends to the game Whether they published their game results on Facebook Whether they enjoyed playing the game Likes and dislikes regarding the game's features Improvements suggestions Their stance on games for cultural institutions Exhibition rating (interesting or not) Whether or not the exhibition in question was recommended by the application Whether they mainly followed the recommendation Whether they would upload photos and a diary of their visit to Facebook Rating ease of use (of MMG app) Whether they enjoyed using the application Whether the presentation of the information was satisfying Additional information they would have liked to see Whether they would make changes in the presentation of the recommendations Whether they would have preferred seeing a map With whom they visited In case they visited with a group: 1. Whether they followed the group's decisions or the app's recommendations 2. Whether they checked other group member's screens In case they visited alone, whether they followed the app's recommendations Their opinion on advantages and disadvantages of the mobile application Preferences while visiting a museum Minimum and maximum time they would have liked to spend at the FHW
My Museum Guide related questions
EXPERIMEDIA
Category Factors
Whether MMS and/or MMG helped make their visit more enjoyable.
10
EXPERIMEDIA
4. Experiment results
According to the success scale proposed in D4.8.1 section 8, the BLUE experiment has achieved a moderate success and some steps towards success have been realized. The criterion for baseline success, i.e. being an EXPERIMEDIA test bed, has been fulfilled since the ECC has been fully integrated and used, and knowledge and expertise gathered accordingly. Then, the analysis of our experimental runs has shown the usefulness of the BLUE technologies and their suitability to improve user experience. QoE has been evaluated qualitatively, thus the impact of using BLUE technologies has been shown. Four research hypotheses have been evaluated based on both qualitative and statistical analysis.. QoS has been taken into account at design time.
4.1.
Method
Four experimental sessions were conducted. The first took place on 18th June 2013 with 8 participants, the second on 23rd June 2013 with 7 participants, the third on 30th June 2013 with 8 participants and the fourth on 16th July with 7 participants. The total number of participants was 30, 15 male and 15 female with an average age of 30. Of the 30 participants 6 people visited alone, 17 with one or more friends, and 7 with family members (there were also two children visiting with their parents but they were not included in the sample due to legal constraints). Consistent with the literature describing how people usually visit museums in a group (Antoniou, 2009), our sample also reflected this trend. In addition, data collected from participating families is of significant value as data regarding family museum Figure 1. Visitor playing MMS game visit behaviour is still lacking. Summarizing, the experimental procedure required invited visitors to first play the Facebook game, then visit the museum and use our mobile application and finally fill in a questionnaire. More specifically: Two applications were developed, MyMuseumStory (MMS) and MyMuseumGuide (MMG). MMS is a Facebook game giving its users the ability to populate their own virtual museum with various exhibits. At first, users are asked to choose an avatar, a pet and a tool to play with. Based on their choices, the MMS calculates and extracts each users cognitive style. Moreover, the chosen virtual museums template indicates the users visiting style. As the user wanders in her museum she can win objects to add as exhibits by playing mini-games. All Figure 2. Visitor using MMG application exhibits earned and placed inside the virtual museum throughout the game are recorded and stored as they reveal users personal interests.
Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013 11
EXPERIMEDIA
The MMG computes recommendations in the form of predictions. These are then provided in the form of a sequence of Points of Interest (POI) to the visitor. The procedure of computing recommendations has been published in (Naudet, 2013). After his visit, the visitor is asked to fill a questionnaire, giving his personal likings and opinions. The experiments provided data that can be used to evaluate the experiment both qualitatively and quantitatively.
4.1.1. Qualitative analysis

The qualitative analysis examines the effect of multiple factors in regards to the Facebook game, the effect that the on-site experiment had on visitors' QoE and, as an additional element, briefly discusses QoS (Quality of Service) handling during the on-site experiments. Facebook game (MMS): Gameplay choices Game usability
QoE during on-site experiment. We examine the effect of the following factors: Overall Visitor Satisfaction and Evaluation Photo Option and Online Visit Sharing (MMG) Tablet Usability (MMG) Effect of Personalization and Content Adaptation (MMG) Group Visitors Evaluations
Additional element: QoS evaluation
4.1.2. Quantitative evaluation metrics

In regards to the quantitative analysis, four elements were examined. For each of these elements, a hypothesis (in the form of null hypothesis) was made and statistically examined. These elements, and the null hypothesis made to test them, are the following: 1. Estimation of cognitive styles by the Facebook game : The Facebook game cannot reveal players cognitive styles. 2. Prediction of user interests by the Facebook game : The Facebook game cannot predict players museum interests. 3. Success of on-site recommendations : The recommender cannot provide the best-suited exhibitions for each user 4. Prediction of visiting style by the Facebook game : Facebook game players choices of museum templates cannot reveal their preferred visiting style for a physical museum.
12
EXPERIMEDIA
4.2.
Analysis of experimental results
4.2.1. Facebook game results analysis

Gameplay choices: All participants, apart from 3 that did not have a Facebook account, had played the MMS Facebook game. From the 9 available avatars in the game, three were never chosen (TV persona, Diplomat, Rapper). From the remaining 6, the mad scientist was the most popular since 7 people chose it, 5 chose the Engineer, 4 the Old-wise, 3 the Artist, 2 the Judge and 2 the Alien. This clear preference for the mad scientist might be due to the position of the item in the game, since it was the very first avatar in the top left corner of the table with the 9 avatars. From the available pets, the gold fish was never chosen. The dog was the most popular with 14 people choosing it. 4 people chose the monkey, 4 the owl and 1 person chose the cat. Again the dog was placed at the top left corner of the table with the pets. From the available tools, the Book was also the most popular (14 choices), being placed at the top left corner of the choice table. 4 people chose the clock, 3 the disco ball and 2 the heart. Avatars, pets and tools are shown in Appendix B. A clear preference for the first item in the tables with the avatars, tools and pets was observed. Items in the top left corner of these tables were chosen significantly more than the remaining items. In addition, 4 people also reported that they had problems selecting these items, since by default the first item was chosen and a single click was selecting the items without an undo option. This usability problem, together with the known preference of the western world users for the top left corner, could possibly explain the results observed here. For this reason, any new version of the game should handle the usability problems reported and also vary the position of items in the top left corner of the choice tables. Furthermore, participants were asked for the reasons they made the above choices. As it was found during pilot testing, the pictures of avatars, tools and pets alone were not sufficient to stress the main feature we would like people to look at. In order to increase the success rates of the pictures chosen, a short sentence was placed under each picture to show people the main characteristics of the item. For example, the picture of a diplomat was accompanied with the text I like to avoid conflicts, because the concept of a diplomat might mean different things to different people. From the participants answers it was clear that this practice was very useful and people did read the text in order to choose avatars, tools and pets. Hypothesizing that the different choices of avatars, tools and pets would reflect peoples personalities was also supported since 15/20 people explicitly stated that they chose these items since they reflected their personalities. Some of their answers were: Because I am strange and they reflect my personality (male, 19 years old) Descriptions closer to my personality (female, 23 years old) Read the description and was close to my personality (male, 20 years old) Representative of my personality (female, 23 years old) Reflect my way of thinking (female, 38 years old)
13
EXPERIMEDIA
Game Usability: Playing the game, most users did not face any problems. Only 4 out of 30 reported problems, mostly regarding selection issues with the avatars, navigation problems and avatar movement problems (at times it was getting stuck). In addition, 26 out of 27 people reported that the game was easy to play, 24 out of 27 said that they enjoyed the game and most people managed to collect exhibits by playing, with average exhibits per player being 7. Almost half of the users invited their Facebook friends to play the game (13/27) but only 8/27 reported their scores on their wall on Facebook. As for ameliorations, most participants suggested that the game graphics could be enhanced (some found them old-fashioned). Additionally, some participants mentioned that they had navigation problems, although a map was used in the game to show players their exact position within their museum. As for other points for improvement, a few people said that the games were easy and they would prefer more difficult ones, while others suggested adding a short description to each game. One participant suggested adding tablet version of the game. Estimation of Cognitive Style by Facebook Game: One of the main hypotheses of the present study was whether a social networks game can reveal peoples personality traits, such as cognitive style. In the following table the participants actual visiting style, as it was calculated from their interview questions is presented together with the one the game estimated from their choices. The different letters represent the 4 dimensions of the cognitive style (the Myers-Briggs Type Indicator), which can take 2 values each. In particular, there are 4 dimensions of two opposite points, like Extraversion-Introversion (letters E and I in the table), Sensing Intuition (letters S and N), Thinking-Feeling (letters T and F) and Judging Perceiving (letters J and P).
Table 3. Users' Estimated and Actual Cognitive Style
Facebook App Cognitive Style Estimation1 N P I N T N T I N F N T E T N T
Real Cognitive Style from interviews E N T E N T I N F I N F E N T P I S T J I S F
Successful estimations are indicated in bold blue font.
14
EXPERIMEDIA E N T E N J T I N T E J T J I N T N J E N F T T J I N T S T E N P J T E S T I S T S T J E N T P E S T P E N F J I N T J I S T I S T J I N T P I S T J E N T P I S T J I N F P I S F P E N T P E N T P I N F I S T J E N F J E N F P I N F
The missing information on the estimated cognitive style (when cognitive style could not be entirely computed from the game, due to player actions denoted in the table above by the + sign) was randomly filled by the application in order to assign users a cognitive profile. The recommender was using the completed information. For the purposes of the presented analysis the actual data, without the additions, are used to see the effectiveness of the game in cognitive style estimation.
15
EXPERIMEDIA
The last dimension of cognitive style (Judger-Perceiver) is the only one that can change during a persons lifetime. The other dimensions are relatively constant. This last dimension seems to be affected by age and most people seem to start as Perceivers and end as Judgers later in life. For this reason, together with the fact that most of our participants were young adults, some people scored between the two categories (denoted in the table above by the * sign). In these cases, any estimation of their cognitive style from the game is considered correct, since these participants could handle equally well both types of information (i.e. information designed for Perceivers and information designed for Judgers). For the first dimension, Extraversion-Introversion, the game correctly estimated this dimension with 69.2%. For the dimension Sensing Intuition the success rate was 58.8%, for the JudgerPerceiver it was 77.7%. The lower score was for the Thinking-Feeling dimension, which was only 55%. The collected data seem to partially reject null Hypothesis
The game could in many cases correctly estimate players cognitive style, with success rates of 55% to 77.7%. The experimenters are particularly satisfied with the above result, since the game could reach this estimation after only three choice screens. It seems that the game is certainly moving towards the right direction and more features need to be included to make it more accurate. From different pilot studies and preliminary results, certain correlations were found that could be used in the next version of the game to increase its success rates. For example, a high correlation between peoples cognitive style and music, fashion and decoration preferences was found. In a future version, the player will be able to dress her avatar, to decorate her museum space and choose what type of music she wants to listen while playing. All these choices provide further information for the better calculation of cognitive style. In addition, it was also found that different cognitive styles might have different game preferences and preferences for museum content. In a future game version, games from different categories should be included and players choices should be also recorded, together with their choices for different exhibits and content. Prediction of user interests by Facebook Game In addition, 21/23 participants reported that their exhibits choice in the game reflected their actual museum interests (91.3%). The recommender used information from the game and peoples exhibits choices in order to suggest different exhibitions to different users. People reported their preferences for the 8 exhibitions available at FHW, from very interesting to medium interesting to not interesting. The number of stars provided to the users describing the exhibitions relevance to them, was compared to the users self-reports on their interests levels for those exhibitions. The results rejected the null hypothesis and , implying that the developed applications adequately predicted user interests and suggested best suited exhibitions in many cases (please, see below for more information about recommender success on-site). From the interview answers it was clear that the game was easy to play, with straightforward rules, engaging and most users also found it enjoyable. In particular, regarding participants choice of exhibits, in most cases this was a very interesting process, since most reported that
16
EXPERIMEDIA
these exhibits reflected their personal interests. Only one person reported that she chose items from all the available categories and one more that did not really think about it. All other participants had very clear arguments about their choices: Because I love Darwin (male, 19 years old), I like ancient exhibits (male, 20 years old), They reflect my interests (female, 19 years old), Because, I usually visit museums about ancient Greece (female, 23 years old), etc. Despite the fact that for copyright issues the images were processed and were not very clear, this did not seem to affect users choices. In fact, only 6 /30 people reported that the quality of the images was affecting their choices. It also seems that a game that uses such features can accurately predict peoples interests inside the museum. Although, based on the overall observations, the null hypothesis concerning prediction of visitors interests could be rejected, a closer look revealed that this was due to certain categories. Other categories, like peoples interest for Biology could be correctly predicted in more than 65% of the times. It seems that possibly the quality of the images used in the game might have affected the outcome or the methodology followed in order to collect user data, might also affect results. More specifically, the recommender predicts interests in generic interest areas (e.g. Biology, Ancient Civilizations, etc.) but the users rated specific exhibitions (in this case, exhibition content was made for school children and a few users mentioned that they found them childish). In addition, the selfreports used might have produced biased results, since users might have told the experimenter that they liked an exhibition more than they actually did. In any case, the fact that the recommender was successful in certain cases is promising since it provides an indication that social networks games could predict peoples museum interests. Therefore, information from social networks could be particularly valuable to museums and curators, in order to 1) correctly advertise different exhibitions to different users and 2) appropriately guide users inside the museum, by increasing the quality of their experience. Prediction of visiting style by Facebook game: From the available museum templates, most people chose the free museum (11 people), 10 people chose the open museum and only 1 chose the linear museum (the museum templates are available in Appendix C). In addition, most participants reported a preference for an ant visiting style in museums, 6 preferred a butterfly style, 4 a fish visiting style and 3 a grasshopper style. These metaphors showed the nature of the movement. An ant visitor moves in a clear line, views almost all exhibits, spends a good amount of time for each exhibit, pays attention to details, moves close to the exhibits and the walls, avoids empty spaces, follows the curators suggestions and rationale. A fish visitor moves in the centre of rooms, does not avoid empty spaces, does not pay attention to details but rather shows interest in the larger picture, spends short time in front of the exhibits and does not stop very often. A butterfly visitor does not follow the curators paths or a clear line in her movement, changes the direction of the movement frequently, usually avoids empty spaces, moves close to the exhibits, sees almost everything, looks at details, seems to be attracted by the exhibits accessibility, is affected by other visitor traffic (environmental affordances (Gabrielli et al. 1999, Marti et al. 2001) and stops frequently. Finally a grasshopper visitor seems to have clear
EXPERIMEDIA
preferences and views only the exhibits that interest her. Such visitors do not stop very often, cross empty spaces and they spend a long time in front of the exhibit they choose to see. Ant visitors need the most time to view an exhibition from all other visitors, butterfly visitor follow in time demands, fish visitors need less time than the two above and grasshopper visitors have the shortest visits of all (Oppermann & Specht, 2000). The museum templates were used in the game, since it was hypothesized that they could reflect visiting style preferences. When the museum templates choices in the game were compared to the interview questions about visiting style preferences, it was found that visiting style was correctly predicted from the template choices in 6 cases and wrongly predicted in 16 cases. The results failed to reject null Hypothesis . Regarding the choices of the museum templates in the game with the reported visiting style, it was found that we could not accurately predict the preferred visiting style of visitors by simply using their game template preferences. However, the nature of the images used in the game might have affected users choices. As some participants reported during the interview, the shape of the museum template was very important for their choices, more than the practicality of that template. One participant mentioned, it looked minimal (female, 24 years old), another said, I liked the shape (male, 19 years old), and others commented, looked more symmetrical ( male, 23 years old), looked more contemporary (male, 32 years old), looked more spacious (female, 24 years old) Trying to avoid the issue of image aesthetics, in a next game version, the templates will need to be very carefully designed. In addition, it is also important to note here that this should not count as a point of failure for the experiment, since the use of visiting styles in the on-site experiment was decided not to be exploited in the end, due to the lack of path diversification ability by the recommender (because of the few items to suggest aside from shows), as explained above.
4.2.2. On-site results analysis

This section provides an overview of the analysis of the results achieved during the on-site deployment of the BLUE experiment, and the effect that these had on the QoE of visitors. An analysis of the QoS results experienced during the on-site experiments is also provided. QoE: Recommender Satisfaction and Evaluation: Consequently, due to the success rates of the recommender, 73% (11 out of 15) of participants reported that they followed the recommendations. When asked what they would do if they visited alone, 71% (17/24)of the participants mentioned that they would follow the recommendations, 17% (4/24) reported that they would not follow the recommendations and only the remaining 12% (3/24) said that they would try to combine the recommendations with their personal opinions.
18
EXPERIMEDIA
Finally, people were asked to provide their overall feedback about the recommender. In general 75% (15 /20) of the participants were very positive with the use of the recommender and only a 10% (2/20) mentioned that the recommender should have more information. Here are some of the most interesting participant comments: It is nice to see all the available exhibitions (male, 39 years old) If you combine it with more information it could be very useful. I like the photo feature that you can upload on FB. Together with your comments it is a very good advertisement (female, 38 years old) It is quite interesting to see what was pre-recommended according to my game results (female, 25 years old) The recommendations were mainly close to my interests. Made my visit easier and not time consuming (female, 22 years old) It was pleasant and different from usual. Made the visit easier and more fun (male, 25 years old) It is nice to see details of exhibitions and assist choice (male, 19 years old) Participants were also asked if they would prefer to visit with or without the recommender. This gives an impressive result: 82% (22/27) of the participants answered that they liked having the tablet and the quality of their experience was enhanced. Some of the user comments were: I prefer having the tablet to see what is available (male, 32 years old) I enjoyed it a lot (female, 23 years old) Both the game and the tablet made my visit more enjoyable (female, 22 years old) I was better prepared for the visit (male, 46 years old) Finally, three last questions attempted to discover how important different features of the recommender were, like avoiding visitor traffic, not missing important exhibits and to have most effective movement patterns within the museum. Participants had to rate these features from 1 (not very important) to 5 (very important). Participants provided the score 4.6 for the importance of seeing all relevant exhibits in a museum and not miss information important to them. Having most effective movement patterns and not wasting time by walking back and forth was of medium importance, since the average score was 3.9. Finally, avoiding visitor traffic also scored 3.9. Success of on-Site Recommendations When self-reports were compared to the recommendations, it was found that from the total of 250 recommendations, the recommender was successful in 57.9% of the times (average success rate). However, when individual categories were studied separately, the recommender was
EXPERIMEDIA
particularly successful to suggest certain types of exhibitions like Ancient Cities (61.3%) and Biology Darwin (65.2%). There were lower success rates for other types of exhibitions like Biographies-Kazantzakis (42,1%) and Environmental exhibitions (57.7%). It is interesting to note that recommender was also suggesting activities like a short break at the museum caf or museum gift shop. These recommendations were not based on the individual profiles but only on the exhibitions time schedules and gaps between shows. In this case, of the non-targeted suggestions the users reported that in only 51.3% of the times they were pleased with these recommendations. In other words, and although the sample used in these studies does not allow a smooth statistical analysis, it seems that targeted recommendations based on individual profiles score a lot higher than random, non-targeted ones. Certainly this is only an indication that the present approach of creating and using visitor profiles, moves towards the right direction. With the current results, we have a partial support from Hypothesis , since users sometimes provided contradicting statements (i.e. the satisfaction levels reported do not always match the stars given by the users to the different exhibitions). The reasons for these inconsistencies are not clear, especially when the Facebook game seems to correctly reflect user interests in more than 90% of the cases. However, the very nature of the exhibitions seems to have played a very important role. The exhibitions were highly targeted to school students and in many cases, our participants found the content below their expectations or childish as a few put it. In the Discussion section below, a possible explanation is provided in greater detail. Finally, the difference of results between interests predicted in the game and those given by the recommender on site can be technically explained by the following: regarding interests in game, people gave feedback on the suitability of the objects they choose to represent their actual preferences. Contrarily, recommendations were evaluated by comparing computed ratings (on a 1-5 scale) with the users ratings, which leaves much more place for errors than with a binary answer as was the case with the in game evaluation. Photo Option and Online Visit Sharing: Using the tablet application, the user could take photos from their visit and upload them to their Facebook account. Although only 39% (9/23) of the participants answered that they would upload these pictures or that they already had, at the end of the interview when asked about their general views of the experiment, most people said that they really liked the photo feature and most took pictures using the tablets during their visit. An interesting situation was observed in regards to the photo option in the MyMuseumGuide application. Although only 9 people explicitly mentioned that they would upload pictures of their visit on their Facebook account, most users used the photo option and many said that they liked it. The finding is not contracting, since people might want to document their visit, but not publish it on social networks, protecting their privacy. In addition, our average participant age was 30 and older adults might have a different approach to privacy issues than younger adults concerning information they share on social networks. Nevertheless, the photo option seems to be useful and liked. The only addition would be to provide the option of sharing the information on Facebook or keeping it for personal use. Facebook already provides this option and it could be explicitly incorporated in the MyMuseumGuide application. Tablet Usability: Almost all users stated that the tablet application was very easy to use. Only one said that it was medium easy. All (100%) of the participants explicitly stated that the
EXPERIMEDIA
application was enjoyable and 90% (19/21) of the users mentioned that the information was presented in a satisfying manner. When asked if a map would be useful, 74% (17/23) of the participants agreed. Although this was a suggestive question, it seems that a map option would be useful. While the My Museum Guide mobile application featured a map, it was not activated due to an unresolved problem as to how best switch floors when a user transits between floors. To not confuse users, it was chosen not to display a map. Participants were also asked if the recommendation notations were clear and if they would make any changes. Most participants were satisfied with the notation system used (i.e. stars for each exhibition). One participant mentioned that he did not realize that these stars were the recommendations and another one said that perhaps a bar with percentage of interest would be more useful than the stars. Finally, all participants were asked about other information they would like to have. Most frequent requests were a map and more pictures from the exhibitions, and three participants also asked for a feature that they could also rate the exhibitions. Effect of Personalisation and Content Adaptation: Although all content provided to the users was adaptive and prepared for the different cognitive styles, the users did not realize this content adaptation. Although they were happy with exhibition descriptions, they did not realize that their partners and friends had different descriptions and often did not open the content pages that provided more in depth information for each exhibition. Only 6 participants said that they checked their friends screens but did not pay any more attention. Thus, none of the users were aware of the adaptive content in the descriptions of the exhibitions and only 6/23 checked their friends screens and saw that there were different stars provided to different users. Consistent with literature describing how users do not read text (Chairman & Claremon, 1977, Falk et al., 1986), our observations confirmed that finding. It seems that adaptive content is not necessary at least in the form of text, since users do not really read this information. However, users did notice the pictures used for the different exhibitions and some asked to have more pictures from each exhibition available at their tablets. This observation might imply that adaptive content is relevant but in alternative forms to text, like audio or visual content. Further research is required however to determine which form of adaptive content is best for different visitors, possibly related to their individual cognitive styles. Group Visitors Evaluations: Since most museum visitors visit in groups, it was important to record group visitors behaviour with the tablet application. Participants were asked to describe their behaviour within the museum and whether they did what the recommender suggested or followed the groups wishes. Only one participant mentioned that he would not follow the group and continue with his personal preferences as reflected on the recommender suggestions. Similarly, only one person said that he only followed the groups decisions. It seems that most group visitors tried to combine different suggestions for the different group members and proceed accordingly. However, the situation was very different for participants visiting with children, since they all said that they only did and would do what the child wanted. QoS evaluation: Upon examining the list of metrics collected during the experimentation, we realised that measures regarding QoS were not necessary for our experiments. Indeed, as we are not running a
21
EXPERIMEDIA
real-time system and neither QoE measures nor the visitors experience are affected by factors relating to QoS, measures regarding the latter could be neglected. In hindsight, evaluating comments received by visitors, the initial assessment can be confirmed. No user expressed any concerns about the QoS, hence, there was no need to re-evaluate the need for those measures.
4.2.3. Usefulness of cognitive style in recommendations

The sequence of recommendations displayed on MMG was the final service proposed to guide visitors and supported fully the user experience during a museum visit. In addition to gathering user feedback and their satisfaction regarding the recommendations provided, we tried to observe the influence of using cognitive style on recommendations. The Facebook game was very important in the experiments since it provided both the cognitive styles and the exhibitionrelated interests (through items chosen for the virtual museum). For users who did not play, the personal profile was empty and the cognitive style taken randomly. With the specific characteristics of the museum, recommendations had to follow strict rules to ensure their usefulness. (1) The small number of exhibitions (resp. POI) implied that no filtering was applied: all exhibitions and POIs were shown, whatever their matching score for the user. (2) The fixed schedule of some exhibitions implied that recommendations were ordered according to the moment they should be visited, respecting the time schedule. (3) Because some exhibitions had very similar topics and thus had the same matching score, they were presented with a slightly different number of stars (around the normally computed value). We postulated here that diversity in the recommendation scores displayed to the user would help him making a choice (which might be difficult when a set of items having all the same score are presented).The sequence matching was computed using Cosine Similarity on the set of matching scores POI in the sequence (details on the recommendation algorithm can be found in Naudet, 2013). We have analysed user satisfaction regarding the recommendations to assess the accuracy of computed sequence matching scores. A first statement was that user profiles with only a cognitive profile (CP) (considering only the cognitive style estimation derived from the game without other factors like choices of exhibits, etc.) did not contain enough information to provide relevant recommendations (a complete profile includes information about cognitive style together with players exhibit choices in the game, which provided a more complete picture on possible museum preferences). Indeed the analysis conducted before the trials to map cognitive styles to museum exhibitions topics revealed only two rules that have been exploited: (1) people with cognitive styles containing the feeling dimension (F) usually like literature and biographies; and (2) people with cognitive styles that contain the thinking dimension (T) usually like natural sciences. According to the topics associated to the museum exhibitions, 4 at most could match the (T style, with only one matching a CP containing both the (F) and the (T styles, as shown in the table below:
22
EXPERIMEDIA
Table 4: opics associated to exhibitions, corresponding to cognitive style dimensions.
Exhibition Cognitive Style dimension Darwin Earth is our home Ice Worlds Your planet needs you! Other exhibitions
Literature and bibliographies Natural science Feeling (F) Thinking (T) -
On the whole set of experiences, the two visitors for which we had only the CP rated Darwin high, which corresponds to their supposed behaviour according to their cognitive profile that contains feeling (F). However they also liked other things, thus nothing could be concluded. Knowing this, having in the user profile also personal interests complementing CP-based interests was mandatory for the conducted experiments, in order to get recommendations that are sufficiently diverse. The synthesis showed that almost all visitors were satisfied by the recommendations and have followed them, whatever the matching score of the sequence they were proposed. The visitors that did not follow the recommendations did it for reasons independent from the recommendation quality (e.g. following a childs decisions). This tends to show that the computed scores were relevant. Finally, it was interesting to observe that while sequences matching scores are low when only the cognitive profile prevailed (i.e. no or just a few personal interests are known), they rise to medium or high as soon as the number of times personal interests prevailed in the computation of exhibition matching scores is higher than for the CP. Since we have shown that only few exhibitions match with cognitive styles mapping rules, we have the obvious result that the richer (or the more diverse) the profile, the better the chances to make a relevant recommendation. In conclusion, the experiments did not allow showing the usefulness of using visitors cognitive style in a recommendation process.
23
EXPERIMEDIA
5. EXPERIMEDIA component assessment

5.1. EXPERIMEDIA Experiment Content Component (ECC)
The ECC was used to automate data collection in a secure and centralized location in Greece. The component was integrated with our systems as described in Sections 2.3 and 2.4 of deliverable D4.8.2. The deployment of the RabbitMQ messaging bus proved a bit tricky but doable with a hand from IT Innovation staff. The same held true for switching the system to operate exclusively using SSL. After the installation of the server-side ECC, the infrastructure was briefly tested with the provided sample implementations. Due to the client interacting with the server-side ECC being based on Android, the first version of the libraries had some compatibility issues. However, IT Innovation was able to provide an Android implementation of their client libraries in a short time period and we could proceed with the implementation. The implementation of the ECC running for our experiments was pulling data every four to five seconds from all connected devices. Due to improvements in the ECC design and experiment state handling, we were able to allow for dynamic client connections, drops, and reconnections; all transparent to the user. We were happy with the performance and like to commend the quick response time and uptake on proposals for improvements by the team at IT Innovation. As of now, the only point where we were not completely satisfied is the support for data analysis. As the ECC compiles a humongous amount of data, it is rather cumbersome to prepare the data for subsequent analysis. Also, due to the complexity of the data model, extracting the data from the database in a usable format also proves rather cumbersome. However, these points have been noted by IT Innovation and will be subject to further analysis and improvements.
5.2.
EXPERIMEDIA Social Content Component (SCC)
The SCC sub-component used in the My Museum Guide Android application is the Social Integrator (SI). It allows us to propose users to log into their social medium of choice and, thereby, connect him through our application to his social environment. The SI was thrown into disarray by an adjustment of the Facebook API. However, the NTUA team was able to quickly address the issue. As of now, the SI performs admirably and, without further change to the Facebook API, should continue to do so. A proposal for the development of a client-side monitor of social interaction was taken into account by the development team and proposed as a web-service this summer. We were able to deploy the server-side of the social web-service but, unfortunately, development of our own application had proceeded to a point where it would have been a non-negligible risk to attempt integration. However, we are confident that future EXPERIMEDIA partners will find the development useful as it complements the ECCs EM nicely when it comes to collecting social data.
24
EXPERIMEDIA
6. Discussion
The two applications developed within the framework of EXPERIMEDIABLUE were tested with real users at the FHW in four separate sessions. The sampling processes replicated the processes the museum will apply in order to attract new visitors, advertise exhibitions and also collect relevant information for user profiling, since the museum will also its Facebook page to host the game and invite new players/potential visitors. Having used Facebook for sampling purposes, the networks efficiency was also observed since people responded. Users found both applications to be highly usable and enjoyable. As users also mentioned, it was very interesting to link peoples personalities and actions prior to the visit with the physical museum visit. The novel approach of using a social networks game for museum visitor profiling was successful in numerous ways but most importantly because it opens a road towards the exploitation of the vast quantities of information available in social networks and its use in adaptive technologies. Overall, visitors QoE has been enhanced and their final impression within the museum was very good. Roughly (see previous section for exact numbers), 82% liked using the tablet with MMG, 75% had a positive impression using the recommender and 73% actually followed the recommendations. Those results, despite an average precision in recommendations of 58% (on exhibition ratings, on a 1-5 scale) are very encouraging knowing the constraints induced by the museum venue and the little amount of data exploited for profiling users. With the MMS, we were able to show with a high success rate (resp. 91.3% and 67.17%) that user interests and cognitive style could be predicted from a simple game targeting museum topics. In the following sections we discuss the results of data analysis regarding our four hypotheses. H1: The Facebook game can reveal players cognitive styles. The data collected in the present study provided partial support for the above hypothesis. As discussed above, this first version of the game only explored the possibilities of a game that could reveal cognitive styles. There were clear tendencies towards this direction that definitely require further development. The game only used three screens to calculate players cognitive styles. From the available results from our previous studies (preliminary results from pilot studies), we know that different features that can be added to the game can provide further information and possibly more accurate results. The main challenge for designing the game was to identify the key pictures to best reflect the different dimensions of the cognitive style. Keeping in mind that for the proper estimation of cognitive style, a highly trained psychologist should interview a person over a significant amount of time, the game had to significantly accelerate this process by reducing it to item selection in 3 screens. Considering the above, the results were more than satisfying and promising for further development. In addition, this is a highly novel approach, not simply because it attempts to estimate users cognitive styles from a simple game, but also because information from social networks can be used in a user profiling process in adaptive systems, whether these will be used in cultural heritage, education, etc. H2: The Facebook game can predict players museum interests. The Facebook game was able to correctly predict players interests and support the 2nd Hypothesis. Most users reported that they chose items that reflect their personal interests and not simply because of item aesthetics or image quality. Information deriving from social networks can be particularly valuable to different
EXPERIMEDIA
professionals like teachers and curators. Knowing users interests information can be tailored and directed accordingly. In particular, museums can use this information to target specific groups in order to advertise different exhibitions but also in order to support the physical museum visit, significantly enhancing the quality of visitors experience. H3: The recommender can provide the best-suited exhibitions for each user. Based on the game data, the recommender was indeed successful in providing the most appropriate exhibitions to the different users but for some and not all exhibitions. So far, my museum guide has been designed for single visitors, focusing on individuals and single cognitive styles. However, as known from the literature but also observed during our experiments, visitors usually visit in groups. The recommender could significantly increase its success rates when it will combine the information of the individual cognitive styles and the information about the group member in a single visit. For example, exhibition 1,2 and 4 will be the best for user X. Knowing that X is visiting together with Y and her best options would be exhibitions 1,2 and 5, the recommender can suggest exhibitions 1 and 2, informing the users that these would be the best choices for their group. Especially, for families visiting, the recommender could calculate the cognitive styles of the adults and the age and gender of the accompanying children, in order to suggest best options. These future additions of the recommender would hopefully make it suitable for group visits, targeting a known museum technology problem that of group visits and content provided. H4: Facebook game players choices of museum templates can reveal their preferred visiting style for a physical museum. The above hypothesis was not supported with the available data. The main reason seems to be the aesthetics of the images used, since certain images seemed to attract the majority of preferences. Attempting to capture the preferred visiting style before ones visit seems to be a very demanding task and alternative ways should be further explored. One possibility would be to use the known link between cognitive and visiting style (Antoniou, 2010). Once the cognitive style is known from the Facebook application, an estimation of possible visiting style preferences can be derived and later used at the museum. This is a field for future exploration that remains to be studied. The EXPERIMEDIABLUE team faced numerous challenges during the design, implementation and testing of the applications, mainly due to the nature of the venue. FHW was not a traditional museum hosting object-based exhibitions. In addition, the content of the exhibitions was highly targeted to school students. The information was simplified and generic, mainly in the form of 3D films. Only one exhibition was using interactive technology to present environmental issues. The layout of the museum and the nature of the exhibitions (i.e. highly targeted, film form, specific time schedule) have affected the outcome of the experiments, since none of the users was of school age. Most users were adults invited through Facebook and they found the exhibition content of a school level (some called it childish). This had enormous implications in the present work, since although someone might like Biology for example, they could give the Darwin exhibition a low mark because it was not of the desired depth. Combined with the fact that most users did not access, read or even notice the adaptive content, possibly adaptivity at this level might not be meaningful. Post-adapting a strongly focused exhibition (as these "exhibitions" were focused on children) does not have an overall effect (positive or negative) on the visitors' experience. Adaptation is something to be considered beforehand, to cater for different target groups and different views/interpretations of the exhibition's message. Altering
EXPERIMEDIA
text messages does not have a strong effect on the experience. However, recommendations are certainly useful. Given that the museum is a small one with a lot of limitations on the visit (hours, presentations, ticketing policy) the overall visitors' feedback is positive. For a large museum, with lots of exhibits (more than what an average visitor would be able to view in a day's visit) and more freedom of movement the impact would have been even greater. In larger museums, such a system would be particularly useful, since it would allow curators to know visitor interests even before their visit, could suggest routes inside the museum to avoid visitor traffic and provide opportunities for both visitors and museums to connect the visit with social networks. In the present work, efficient recommendation depended on two main things: the quantity of information contained in both the user profiles and the items to be recommended; and the matchmaking algorithm that actually computes the items to recommend. In the case of our experimental context, the following issues emerged: Insufficient number of topics attached to each exhibition; Insufficient user profiles: items in the FB game mapped to one topic only and the number of different items was not enough; Number of topics used to characterize both POI (in particular exhibitions) and user interests was clearly not rich enough: a multi-topic classification of exhibition should have been used; Only two rules mapping cognitive styles to exhibitions is not enough, making the recommendations based on cognitive styles not necessarily relevant; There were not enough items to recommend; There were only few participants, but this is not a factor impacting the precision of recommendations.
For future research, the following important points will have to be taken into account. To see the effect of knowing the cognitive styles on recommendations, we have to (1) know the users cognitive profile and (2) have a significant set of rules mapping each cognitive style to interests related to exhibitions topics. Then, to be able to increase recommendation accuracy, we need to have: (1) more interests deduced from MMS; (2) exhibitions (resp. POI) described by a set of topics instead of only one. In summary, the profiling process through the MMS game only was too weak to get accurate results. The average results that we have tend to show nevertheless that we should carry on investigating our approach because in the end people were satisfied by the recommendations. In general, EXPERIMEDIABLUE attempted to combine cultural heritage, social networks and peoples personalities in a unique way. The academic community recognized the novelty and the significance of the approach, since parts of the present work have already been presented in international conferences and published in conference proceedings. A new door opens in the exploitation of the available information in social networks for adaptivity purposes. The uses of such practices can be numerous and remain to be studied. Moreover, the users found the approach engaging, entertaining and promising. The popularity of social networks and social network games make it an excellent field for use in cultural heritage. Finally, the present study
27
EXPERIMEDIA
showed that certain improvements of the two applications (My Museum Story, My Museum Guide) could significantly improve the quality of the visitors experiences by: 1) predicting cognitive styles more accurately 2) providing even better recommendations 3) combining existing information and individual profiles with group visits and family visits
28
EXPERIMEDIA
7. Conclusion
EXPERIMEDIABLUE was an attempt to connect physical space attributes, human personalities, and social networks. The experiment investigated novel ways to extract museum visitors profiles and use them in order to provide personalized information. Viewing the museum visit as a process that starts before the actual visit and finishes long after that, social media were employed to provide a continuation of the visit, in digital space. The visitor could use material from their visit to create personal diaries of the visit, publish them on social media and share the experience with friends. During the trials at the Foundation of Hellenic World in Athens, Greece, we observed the interactions of visitors with the MyMuseumStory (MMS) game and with the MyMuseumGuide (MMG) application. Furthermore, visitors were asked to fill in a questionnaire in order to gather feedback on their Quality of Experience and the usefulness of the different personalization and recommendations they were provided. Experiments revealed that a good proportion of visitors (over 70%) liked and followed recommendations, had a positive impression of using the recommender and reported the mobile guide enhanced their quality of experience. While we were able to show that the Facebook game could predict user interests and cognitive profile for a small sample of 30 persons, the overall precision of resulting recommendations was around 60% and although they have been happy with provided Point of Interest descriptions, we do not know if the cognitive profile-specific adaptation is responsible for this. However, these results are encouraging regarding the small amount of data available to make the recommendations (e.g., we could only use a few rules to map cognitive styles to interests). Overall, visitors feedback shows that BLUE tools improve the quality of visitors museum experience, and the interest in individualization of visits and personal technology-enhanced guiding. Moreover, the introduction of social gaming as an extension of visits trough social network has received positive feedback. From a scientific perspective, the usefulness of gaming to derive visitors interests has been proven, while promising results have been obtained regarding cognitive profile detection through gaming and its use for visit personalization. Quoting an experiment participant: I really liked the experiment. It gave me the opportunity to see various exhibits that piqued my interest. I believe that they were based on my character. They suited me a lot. It was a nice experience. There was also the application, which was very important, as I could go through everything and not miss anything. And it was very interesting. [sic] BLUE opens a very promising road. The vast numbers of social network users implies that there might be immense data available for exploitation; data that could be directly used for the creation of personalized applications for spaces of different characteristics, like museums. To this end, BLUE can help in the introduction of a whole new set of social and networked media experiences.
29
EXPERIMEDIA
8. References
Antoniou, A. (2009) A Methodology for the Development of Museum Educational Applications: visitor inspired museum adaptive learning technologies. Doctoral Thesis, University of Peloponnese. Antoniou, A., &Lepouras, G. (2010) Modelling visitors profiles: adaptation for museum learning technologies. ACM Journal on Computing and Cultural Heritage, 3(2), 1-19. Gabrielli, F., Marti, P., Petroni, L., 1999. The environment as interface, i3 Annual Conference: Community of the Future. Siena, Italy, p. 44-47. Marti, P., Gabrielli, F., Pucci, F. Situated Interaction in Art. Personal and Ubiquitous Computing 2001; 5: 71-74. Naudet, Y., Lykourentzou, I.,Tobias, E., Antoniou, A., Rompa, J., Lepouras, G., (2013), Gaming and Cognitive Profiles for Recommendations in Museums, Semantic and Social Media Adaptation and Personalization (SMAP) workshop 2013, Bayonne, France, Dec. 2013. Oppermann, R., Specht, M., 2000. A Context- sensitive Nomadic Information System as an Exhibition Guide, Ubicomp00. Bristol, UK, p. 127-142.
30
EXPERIMEDIA
Appendix A. Semi-structured interview

Date: Participant #: Name: Age: FB user name: Occupation/ educational background: If in a group, who had the tablet (additional information about the number of group members and other details): The Facebook game - Did you play the FB game? If not, who played it? Is this person visiting with you? - Which avatars, pets and tools did you choose? -Did you have any problems choosing avatars, pets, tools?
-Why did you choose these particular avatar, pet and tool?
-Which museum layout did you choose?
-Why did you choose this particular template?
- During a typical museum visit do you: a) I usually see most exhibits, most of the times in a serial order (in a line). I also move close to the exhibits, trying to see details. ____ b) I usually see most exhibits, most of the times in a non-linear fashion (not following a clear line). I also move close to the exhibits, trying to see details.____ c) I usually see most exhibits, since I mainly moved in the center of rooms, trying to get the general picture. ____ d) I usually see only the exhibits which are particularly interesting to me.____
EXPERIMEDIA
- How many exhibits did you collect? -Did you have problems choosing exhibits because of the quality of the images?
-Why did you choose these particular exhibits?
- Did you invite any of your friends to play the game? - Did you publish you score on FB? - Was the game easy to play? - Was the game enjoyable? - What features of the game did you like and dislike?
- Can you suggest any improvements?
- What is your opinion about cultural institutions using games to engage future visitors?
- Are there any circumstances that the above practice would not be desired/ acceptable?
32
EXPERIMEDIA
Inside the Museum: The mobile app recommender
- How interesting did you find the recommendations? Were they recommended to you? Not at all interesting 1 Darwin Ancient Miletus Ancient Agora Ancient Priene Antartica Kazantzakis Your Planet Needs you! Ice Worlds 2 3 4 Totally Interesting 5 Was it recommended to you? YES NO
Cafeteria Gift shop Take a photo
- Did you mainly follow the recommender suggestions? What were the reasons?
- Would you upload photos and a diary of your visit to FB? Tablet Usability - How easy was it to use? - Was it enjoyable to use? - Was the information presented in a satisfying manner?
- What other information would you like to have?
33
EXPERIMEDIA
- Would you have made any changes in the recommendations notations (the number of stars) that were provided? Why?
- Would a map have been useful to help you find exhibitions?
General questions - Who did you visit with?
- If you visited in a group, what did you do, what the group wanted or what the recommender suggested?
-If in a group, did you check other members screen? What did you notice?
- If you were alone, how likely is it that you followed the recommender suggestions?
- Overall, what were the main advantages and disadvantages of the recommender?
- In visiting a museum, which of the following are important to you (rate from 1-5)? Totally Disagree 1 To see as many exhibits that I might find interesting as possible Not to waste time by walking back and forth To avoid visitor traffic - What is the minimum and maximum time that you could have spent today in the museum? Minimum time (minutes): Maximum time (minutes): 2 3 4 Totally Agree 5
34
EXPERIMEDIA
- In the end, do you think you would have enjoyed the museum visit the same way, better or worse without the tablet and/or without having played to the MyMuseumStory Facebook game?
Please, answer the following questions. BOTH lists are equally valuable. Try to answer as your really are, not how you may wish you were, or have to be at work. Tick the sentence that best describes you most of the times in your everyday life. Either tick the sentence on the right or on the left.

I have high energy I talk more than listen I think out loud I first act, then think I like to be around people a lot I prefer a public role I can be easily distracted I prefer to do lots of things at once I am outgoing and enthusiastic
S
I have quiet energy I listen more than talk I think quietly inside my head I think, then act I feel comfortable being alone I prefer to work behind the scenes I have good powers of concentration I prefer to do one thing at a time I am self-contained and reserved
N
I focus on details I admire practical solutions I remember facts I see how things are I live in the here and now I trust actual experience I work at a steady pace Learning new skills is tiring I want clear instructions
T
I focus on the big picture I admire creative ideas I only notice new things I see how things could be I plan the future I trust my gut instinct I work in bursts of energy I enjoy learning new skills I prefer to figure things out
F
I make decisions based on facts I appear cool and reserved I am direct I prefer honesty and fairness
I decide based on my values and feelings I appear warm and friendly I am diplomatic and tactful I prefer harmony and compassion
35
EXPERIMEDIA I take few things personally I am motivated by achievement I enjoy debates

J
Dissemination level: PU I take many things personally I am motivated by appreciation I avoid conflicts
P
I make decisions pretty easily I am serious and conventional I am seldom late I work first and play later I see the need for most rules I like to make and stick with plans
I have difficulties making decisions I am playful and unconventional I am usually late I play first and work later I question the need for many rules I like to keep plans flexible
Thank you for your time!
36
EXPERIMEDIA
Appendix B. Avatars, pets and tools Icons

B.1. Avatar icons
The different avatars represent different ends of the cognitive style dimensions. The mad scientist uses the stereotypical view of an introvert individual, lost in his own studies. The TV persona represents extraverts. The Judge is for people that view justice very important (Thinkers). The Diplomat is for people that wish to avoid conflicts and find a middle solution (Feelers). An Engineer is for people that prefer clear instructions and create practical solutions (Sensors). The Artist is for individuals that have vivid imagination and increased creativity (Intuition). The Old-wise is for individuals that like to plan things in advance and like routines (Judgers). The Rapper is for individuals that like to mix work and fun, question rules and like action (Perceiver). Finally, the Alien is for people that feel different from the rest of the world (Introverts).
B.2. Pet icons

The dog implies high Intuition. Dogs can plan for the future since they can hide their bones for later (this characteristic will be known to the player). The cat implies high Sensing, since she will eat her food immediately and lives in the now. The monkey implies high Extraversion since monkeys like interaction with others and live in groups. The gold fish implies high Introversion and the Owl implies Judgers, since it is associated with wisdom.
37
EXPERIMEDIA
B.3. Tool icons

The book represents Thinkers since it is associated with the search for facts and logic. The heart represents Feelers, since it is associated with sensitivity to peoples needs and feelings. The clock indicates a preference for punctuality (Judgers) and the disco-ball can represent both extraversion and need for fun (Perceivers).
38
EXPERIMEDIA
Appendix C.
Museum templates
Open Museum Template, with easy to see rooms and exhibits. Possibly best for fish visitors that want to have a quick overview of the museum space, exhibitions and exhibits
Linear Museum Template, which is possibly best for ant visitors that need to follow a clear-linear path during their visit
Free Museum Template, possibly best for butterfly visitors that prefer a non-linear movement.
39

D4.8.3 BLUE Experiment Results and Evaluation v1.0

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

D4.8.3 BLUE Experiment Results and Evaluation v1.0

Загружено:

Авторское право:

Доступные форматы

D4.8.

3 BLUE Experiment Results and Evaluation

EXPERIMEDIA Project acronym EXPERIMEDIA

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Appendix A. Appendix B. B.1. B.2. B.3.

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

EXPERIMEDIA List of Tables

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

3.1.1. Changes since D4.8.2

3.1.2. Target group

3.1.3. Experimental factors

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Facebook behaviour Game choices

Category General Participant Information

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

My Museum Guide related questions

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

4.1.1. Qualitative analysis

Additional element: QoS evaluation

4.1.2. Quantitative evaluation metrics

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Analysis of experimental results

4.2.1. Facebook game results analysis

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Facebook App Cognitive Style Estimation1 N P I N T N T I N F N T E T N T

Real Cognitive Style from interviews E N T E N T I N F I N F E N T P I S T J I S F

Successful estimations are indicated in bold blue font.

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

4.2.2. On-site results analysis

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

4.2.3. Usefulness of cognitive style in recommendations

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Table 4: opics associated to exhibitions, corresponding to cognitive style dimensions.

Literature and bibliographies Natural science Feeling (F) Thinking (T) -

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

5. EXPERIMEDIA component assessment

EXPERIMEDIA Social Content Component (SCC)

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Appendix A. Semi-structured interview

-Which museum layout did you choose?

-Why did you choose this particular template?

-Why did you choose these particular exhibits?

- Can you suggest any improvements?

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Inside the Museum: The mobile app recommender

Cafeteria Gift shop Take a photo

- What other information would you like to have?

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

- Would a map have been useful to help you find exhibitions?

General questions - Who did you visit with?

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

Copyright University of Peloponneseand other members of the EXPERIMEDIA consortium 2013

EXPERIMEDIA I take few things personally I am motivated by achievement I enjoy debates

Thank you for your time!