Proceedings of the

OpenLivingLab Days
Conference

Co-creating Innovation:
Scaling-up from Local to Global

2019

2
 ENoLL Office
Pleinlaan 9
B-1050 Brussels
Belgium

T: +32 2 614 85 47
www.enoll.org
3
E: info@enoll.org
This report is a compilation of the papers presented
between the 3rd and 5th of September 2019, in
Thessaloniki, Greece, as part of the OpenLivingLab
Days 2019 conference. The publications here
contained are a result of the double-blind review and
evaluation procedure launched on February of 2019
as part of the “Call for Papers” responding to the
theme of the OpenLivingLab Days 2019 conference:
“Co-creating Innovation: Scaling-up from Local to
Global”

The “Call for Papers” encouraged contributions from

three different paper categories to stimulate a diverse
participation of actors: Research Papers providing
consolidated scientific research; Innovation Papers
showing case studies from a practitioner perspective;
and Research in-Progress works, presenting relevant
preliminary results.

ISBN (e-book): 9789082102796

© 2019 ENoLL - European Network of Living Labs
All rights reserved
4
Review Panel

Chair
Prof. Panagiotis Bamidis. Aristotle University of Thessaloniki.

Steering Committee

Top Selected Papers Chair:

Dr. Dimitri Schuurman. imec.

Health and Wellbeing Chairs:

Ömer Onur. Başakşehir LL Istanbul.
Prof. Panagiotis Bamidis. Aristotle University of Thessaloniki.

Smart Cities and Regions Chair:

Dr. Ali Padyab. Luleå University of Technology.

Living Lab Sustainability Chair:

Dr. Joëlle Mastelic. Haut Ecole Spécialisée de Suisse occidental.

Theoretical and Methodological Challenges Chair:

Prof. Tuija Hirvikoski. Laurea University of Applied Sciences.

Doctoral Consortium Chairs:

Prof. Anna Stålbröst. Luleå University of Technology.
Dr. Brigitte Trousse. Inria, University Côte d'Azur.
Evaluation Committee:
Abdorasoul Habibipour – Botnia Living lab
Anja Overdiek –The Hague University of Applied Sciences
Antonis Billis – Knowle West Media Centre
Aya Rizk – Luleå University of Technology
Balatsas-Lekkas Angelos – VTT Technical Research Centre of Finland
Benjamin Nanchen – HES-SO Valais-Wallis
Bianca Ceccarelli – LifeTechValley/University Hasselt
David Jamieson – Northumbria University
Eric Seulliet – La Fabrique du Futur
Francesco Molinari – XR8 sas & Lunigiana Amica
Gareth Priday – Australian Network of Living Labs (ALLIN)
Hanna-Greta Puurtinen – TAMK Living Lab
Isis Gouedard – HES SO Valais
Joelle Mastellic – Energy Living Lab
Josep M. Pique Huerta – La Salle - Ramon Llull University
Judith Urlings – Happy Aging LifeTechValley
Masataka Mori – Miratuku
Ömer Onur – Başakşehir Municipality / Başakşehir Living Lab
Rob Wilson – Newcastle Business School
Stefano Tarantola – Joint Research Centre-European Commission
Suvi Konsti-Laakso – Lappeenranta-Lahti University of Technology
Athanasios Priftis – University of Applied Sciences in Geneva (HES-SO)

ENoLL Office Contributors

Clara Mafé

Thess-AHALL Contributors
Despoina Mantziari
Table of Contents
Top-6 Papers selected by the Evaluation Committee
A Creative Citizens Model for Smart Urban Planning by Helen Manchester
and Carolyn Hassan ............................................................................................ 14
Agile Piloting for Smarter Cities: 3 Cases of Engaging Ecosystems and
Communities in Co-creation by Kaisa Spilling, Janne Rinne and Matti
Hämälainen ........................................................................................................... 28
Co-Creating Technology for Societal Change: A Mobile App Addressing
Homelessness by Rachel Burrows, Antonette Mendoza, Sonja Pedell, Leon
Sterling, Tim Miller and Alexi Lopez-Lorca .......................................................... 41
Living Lab Activities for Social Problem-Solving R&D Projects in Korea by
Ji Eun Seong and Ji In Park ................................................................................. 61
Living Labs and Circular Economy. The case of Turin by Federico Cuomo,
Nadia Lambiase and Antonio Castagna .............................................................. 83
To Get Things Right for Children. Implementation of a Public Social Living
Lab Model for Coordinated Support for Children in Need by Angelika Thelin,
Torbjörn Forkby and Mats Anderberg ................................................................ ..99

Health and Wellbeing

Co-creating innovative tools with and for people with Intellectual
Disabilities: The case of DS Leisure e-Training Platform by Maria Metaxa,
Foteini Dolianiti, Ioanna Dratsiou, Evangelia Romanopoulou, Dimitris Spachos,
Theodore Savvidis, Vasiliki Zilidou, Maria Karagianni & Panagiotis Bamidis .. 115
Creating an anonymous, at-home screening for sexually transmitted
diseases sent by letter mail: the cross-border development of a
standardized urine collection device and associated testing service by
Judith H.J. Urlings, Bianca Ceccarelli, Claire A.G.J. Huijnen, Paulette J.J.
Wauben, Joke Donné, Ronald Van den Bossche, Alejandra Rios-Cortes, Koen
Beyers and Vanessa Vankerckhoven ................................................................ 127
IoT –based Smart living Environments for ageing well in Greece by Sofia
Segkouli, Stefanos Stavrotheodoros, Nikolaos Kaklanis, Konstantinos Votis,
George Dafoulas, Christina Karaberi and Dimitrios Tzovaras .......................... 139
Participatory design and validation of an innovative training program to
maintain Autonomy of older adults with Alzheimer’s Disease by Despoina
Mantziari, Antonis Billis, Giorgos Arfaras, Maria Karagianni, Vasiliki Zilidou and
Panagiotis Bamidis ............................................................................................. 152
Responsible Design for an Older Future by Gareth Priday and Sonja Pedell
............................................................................................................................. 170
Smart Cities and Regions
A Case Study of a Living Lab through a Bus Improvement Committee in the
Yeongjong area of Incheon City by Min-Ho Suh, Junghyun Park, Minki Kim and
Won-Kyun Joo .................................................................................................... 189
Adapting Living Lab approaches to marginal contexts and urban
regeneration: the case of Mapping San Siro Lab by Elena Maranghi and
Francesca Cognetti ............................................................................................. 205
Intelligent Living Lab: Supporting data-centric decision-making using ICT
tools by Minki Kim, Junghyun Park, Min-ho Suh, and Won-Kyun Joo ............. 200
Open Innovation Camp (OIC) – A Tool For Solving Complex Problems
Rapidly by Teemu Santonen, Julia Nevmerzhitskaya, Aletta Purola and Harri
Haapaniemi ......................................................................................................... 226

Sustainable Living Lab Processes, Business Models and Goals

Building a platform of social entrepreneurship and living together by
Athanasios Priftis, Leonor Afonso, Theo Bondolfi and Jean-Philippe Trabichet
............................................................................................................................. 244
Business model review for Living Labs: Exploring business challenges and
success factors of European Living Labs by Justus von Geibler, Julius
Piwowar and Linda Weber.................................................................................. 254
Facilitate innovation and collective intelligence through play by Yves Zieba
and Isis Gouédard............................................................................................... 273
Launch Process of a Living Lab and Required Leadership for Practitioners
by Masataka Mori and Kyosuke Sakakura......................................................... 280
Living Labs need sustainable business models: the Funding Mix
Framework to bridge the gap between theory and practice by Edoardo
Gualandi and Flavia Fini ..................................................................................... 295
Sustainable person-centered Living Lab for regional management as
extension of Japanese dementia care activities by Atsunobu Kimura, Mizue
Hayashi, Fumiya Akasaka and Masayuki Ihara ................................................. 312
The value of participatory approaches in developing energy services by
Joelle Mastelic and Stéphane Genoud .............................................................. 322

Theoretical and Methodological Living Lab Challenges

Blockchain, a promising way for scaling up co-creation of innovation from
local to global by Eric Seulliet .......................................................................... 338
Cross-cultural Differences in Living Lab Research by Nele A.J. De Witte,
Ingrid Adriaensen, Leen Broeckx, Vicky Van Der Auwera and Tom Van Daele
............................................................................................................................. 352
Developing a test and validation protocol based on quasi-experiment and
analogue observation to evaluate the performance of a Living Lab output by
Benjamin Nanchen, Emmanuel Fragnière, Patrick Kuonen, Joëlle Mastellic,
Randolf Ramseyer and Henk Verloo.................................................................. 368
Identifying Living Lab orchestrators’ individual-level skills by Anne Äyväri,
Tuija Hirvikoski and Heidi Uitto ........................................................................... 382
Improving Quality in Higher Education by using Living Lab Methods by Karin
Axelsson, Yvonne Eriksson and Anders Berglund ............................................ 394
LivingLab 65+ - Co-creation with retirement and nursing homes by Veronika
Hämmerle, Stephanie Lehmann, Cora Pauli and Sabina Misoch ..................... 408
Living Labs for small retailers – in search of methodology and tools by
Heleen Geerts, Gabriela Bustamante Castillo and Anja Overdiek .................... 416

Doctoral Consortium Papers

Barriers for Test and Adoption of Digital Innovations by End-users in a
Living Lab Context by Abdolrasoul Habibipour ............................................... 434
Beyond participation: exploring citizen stakeholder empowerment in the co-
creation of innovation by Shelly Tsui .............................................................. 450
Can Open Innovation offer a new perspectives for development of
ecosystemic business models? by Julia Nevmerzhitskaya .......................... 459
The Roles, Functioning and Culture of Urban Innomediaries by Jimmy
Paquet-Cormier................................................................................................... 467
Urban Living Labs as a smart city approach: how does socio-technical
innovation transform urban development? by Hui Lyu ................................ 475
 Acknowledgments
This publication is a collaborative effort of several
individuals representing the European Network of
Living Labs and its network members.

11
Top 6 papers
selected by
the Evaluation
Committee

12
13
 A creative citizens model for smart urban
planning
Helen Manchester1 and Carolyn Hassan2

1
Associate Professor Digital Inequalities and Urban
Futures, School of Education, University of Bristol, United Kingdom
2 Director, Knowle West Media Centre, United Kingdom

Category: Research in-progress

Topic: Smart Cities & Regions

Abstract
Recent moves, led by the Living Labs movement and others, have begun to
place the citizen at the centre of Smart City discussions. But questions around
what theories and forms of learning are required for citizens to play a role in the
development of digital, urban futures are rarely asked. This paper adopts
ethnographic methods to study the assumptions about learning in a Europe-wide
smart city project that included a component of work led by Bristol Living Lab
(KWMC). Our paper provides important messages for Living Labs and others
keen to include citizens in smart city development. It suggests that the current
‘banking’ models of learning adopted in relation to citizen participation are not fit
for purpose and that new models are needed. This needs to recognise citizen
learning as situated in social and material contexts and embedded in unequal
relations of power, knowledge and resources. We make the case for smart city
initiatives to offer city inhabitants critical, creative learning opportunities that
begin to address the inequalities that constitute the contemporary smart city.

Keywords: Smart City, citizen, learning, digital inequalities, critical, creative

citizen

14
1 Introduction
The educational challenge implicit in citizen involvement in smart cities is visible
in urban theory literature which argues that there is a distinct mismatch between
the rhetoric of the potential of Smart Cities worldwide to create more inclusive,
democratic or more innovative cities and the actual practice of Smart City
planning (Hambleton, 2014; McFarlane, 2011; Campbell, 2012; McFarlane and
Söderström, 2017). In addition, whilst citizens are increasingly placed at the
centre of Smart City visions problems with questions of governance, citizenship
models and relationality of power have been noted (Hollands, 2015; Joss, Cook
& Dayot, 2017; Gabrys 2014; Cardullo and Kitchin, 2018). Batty, for example,
asks whether,

“Smart cities are equitable cities…. that… the sort of infrastructure,

expertise and data that will characterise the smart city will enable
equity to be easily established and such cities will improve the quality
of urban life.” (Batty et al, 2012 p. 516)

This paper draws on learning theory, which has been little utilised in relation to
living labs, smart cities and collaboration in smart city work, to explore an
emerging model of learning that might inform more equitable design of future,
digital cities.

In the first section of the paper we utilise learning theory to explore a model of
digital learning that we believe can have distinctive effects on participation and
inclusion in conversations about digital, urban futures. In section 2 we go on to
introduce our research questions and to briefly discuss the collaborative
ethnographic methods we used to explore the approach of the Bristol Living Lab
within one European Smart City project in the city of Bristol, UK. In Section 3 our
findings explore how placing creative, collaborative models of learning at the
heart of Smart City planning might enable more inclusive approaches to learning
about and designing urban futures.

1.1 The Creative Citizens Model

In 2016 we were brought into a major city-wide, project in Bristol in which the
question of how to build the capacity of citizens to engage with Smart City
developments was central. Our role was to ‘document and explore the role of
citizens in co-designing digital futures’ in the city. Located in a School of
Education we came to the project with a particular set of assumptions about both
learning and how learning might play a role in citizens’ shaping of a city.

First: our assumption was that learners are active. Theories of critical digital
literacies have long suggested that citizens should be considered active learners
when engaging with new technological developments (Potter & McDougall,
2017; Eynon, 2015). Seeing the citizen as an active learner places value on
attending to how their diverse knowledges, creative approaches and critical
social actions might contribute to positive future urban development. Here,
learning can be understood as a tool to enable people to understand new and
emerging digital technologies in order to change the Smart City, not adapt to it
(McFarlane and Söderström, 2017).

15
Second: we recognised literacy practices as being ideological not neutral.
Current Smart City discourse presents the introduction of a particular set of
digital literacy skills as an unalloyed good (Tapscott, 1998; Jenkins, 2007). The
long history of literacy studies (Street, 2003), however, would suggest the
importance of situating literacy practices in social, cultural and historical
contexts, paying attention to relationships between literacy, power and
knowledge and attending to the inequalities and knowledge hierarchies that
these produce (Gee, 2000; Buckingham, 2006; Selwyn and Facer, 2013). These
literatures suggest that models of learning in Smart City projects must involve a
deeper understanding of the already existing cultural knowledges and
experiences of the city (Barton and Hamilton, 2012; Erstad & Sefton-Green,
2013).

Third: As the digital increasingly becomes the ‘stuff’ of everyday life (Miller, 2010)
the development of knowledge hierarchies becomes invisible, embedded in
hidden infrastructures and algorithms that permeate the instrumentation of the
city (Star, 1999). This perspective suggests that learning is distributed across
digital and material spaces (McFarlane, 2011). This requires us, as researchers,
to consider that learning practices in the Smart City also include the material
infrastructures being embedded to make smart cities work, the place-based
memories held by the communities involved, and the new material goods such
as electronic cars that suddenly appear on street corners.

Our position on entering this Smart City project, then, was one that understands
learning as situated, ideological and material, drawing attention to why and how
particular ideas about citizen learning become dominant over others and how
these ideas might be questioned and reframed (Gee, 2000; Street, 2013). It sees
the learner as creative, active and critical, as wanting (rather than needing) to
learn certain things in relation to making urban futures. It recognises that, given
the historical patterns of inequality that produce patterns of ownership, access
and control of technologies, there are obstacles to city inhabitants finding routes
to influence policy and technology shaping the development of the city.

2 Introducing the Mimeo Project and research aims

The Mimeo project took place in Bristol, a medium sized city in the UK. The city
partnered with two other European cities to successfully bid for a large European
Union Horizon 2020 project. The project lasts for 5 years (2016 – 2021) and is in
its implementation phase as we write in 2019. The project is managed by the
local municipality with around 16 partners including small and medium sized
enterprises, community organisations as well as academic partners working on
technology implementation and other evaluation.

The increasing focus on the need for citizens’ involvement in Smart Cities is
stated in the overall objective of the project:

“To increase the quality of life for citizens across Europe by

demonstrating the impact of innovative technologies used to co-

16
 create smart city services with citizens, and prove the optimal
process for replicating successes within cities and across cities”
(Mimeo Project documentation).

The ‘co-creation’ approach is intended to enable innovative, replicable Smart

City services to emerge in dialogue between citizens and project team that tackle
familiar urban problems such as traffic congestion, poor air quality and
unsustainable energy use. However, the project proposal documentation had no
specific aims relating to citizen engagement or learning. Further, it is worth noting
that while the development of technology accounts for half of the budget, citizen
engagement in the process has just over 10% of the budget.

The management of citizen participation in the project is led by a media arts

organisation and ‘Living Lab’ in the city (KWMC). The organisation is not based
in the Mimeo case study area but is known for its innovative work around digital
inclusion in the city. KWMC have been working closely with the municipality and
the universities in the city for a number of years to develop a framework that
creates a smart city where ‘the power of technology is harnessed to tackle the
issues that people care about.’ (KWMC, 2016). Their approach rejects an
emphasis on the role of data, hardware and software in Smart City projects by
seeking to include the knowledge and lived experience of the citizens within the
community to address actual needs (Soderstrom, Paasche & Klauser, 2014).
The stated goal of their approach is to collaborate with citizens to ‘tackle local
issues by enabling participatory processes, bottom-up sensor infrastructures and
collecting, making sense and sharing relevant open data’ (Balestrini, Creus,
Masfarre & Caniguearal, 2016, p. 3).

The Mimeo project is working in a ‘case study’ area in each city on the
implementation of specific smart infrastructure. In Bristol a vibrant and diverse
area of the city was selected. The population living in this area (N= 50,000
approx.) has the highest percentage of black or minority ethnic citizens (BME) in
residence (44% whereas the city average is 16%). 51% of all accommodation in
this area is flats, whereas the city average is 20%. The district also has the lowest
levels of car availability in the city with (46% of households with no car, city
average 29%). According to the Department of Energy and Climate Change two
areas within this district are in the top 10% of households in the country
experiencing fuel poverty. Almost one third of the neighbourhoods within the
district are classified as amongst the 10% most deprived neighbourhoods
nationally. The number of recipients of out of work benefits is significantly higher
than the city average (12.1%), with the rate in one neighbourhood amongst the
city’s highest at 26.2%. These multiple deprivations resulted in the area being
granted European Union Objective 2 status and 'New Deal for Communities'
status by the UK government1.

1
Statistics from
https://www.bristol.gov.uk/documents/20182/928407/Ashley%2C+Easton+and+Lawrence+Hill.pdf/c83
444ac- a3d8-4417-b967-b1c19ec3512f accessed 14th Feb, 2018

17
Mimeo is directed towards developing a suite of new technological designs
including innovations around smart homes and retro-fitting of homes to become
more energy efficient, the installation of electric bikes and vehicles and an
electric vehicle charging infrastructure, an on demand electric bus service, the
development of a Smart City platform and an energy demand management
service. The challenge of diverse partners with competing motivations and
desires meant that, although attempts were continually being made to embed
citizen engagement and co-design into the project, this was challenging. KWMC
recognised this and created a specific stream of work within the Mimeo project
called ‘Created by Us’ in order to achieve their aims related to increasing the role
of citizens. The focus was on ‘Citizen Sensing’: a process involving people
building and using small and generally low-cost sensor technology to help them
to collect data about issues of importance to them. It was hoped that the ‘Citizen
Sensing’ element of the work would also feed into the design of some of the
larger scale technological infrastructure projects.

Our own position on the project was as researchers tasked with researching the
citizen engagement activities on the project. Our research aimed to explore the
way that citizens, and their learning, were imagined and included in the Mimeo
project. Instead of only offering description or critique our research was designed
to uncover and explore the learning models and practices of citizen involvement
in the Mimeo project. Our purpose was not to provide a framework for citizen
engagement (as other projects have tried to do, see European Union, 2017) but
rather, through in- depth empirical work, to illuminate approaches to digital
learning that might support the participation of those often marginalised from
conversations about digital, urban design.

The key project partners and associated partners brought into the Mimeo project
in Bristol included a diverse array of organisations. In the process of seeking
informed consent confidentiality and anonymity were not promised to those
involved. However, in reporting our findings names are not used and labels given
when quoting individuals are deliberately ambiguous in order to avoid, as far as
possible, directly identifying actors.

2.1 Question, methodology and methods

In this paper, we explore the following question through detailed analysis of the
citizen sensing element of the project, led by KWMC:

• How might a Creative Citizens model offer opportunities for critical, digital
learning in Smart City planning?

Our research recognises the need to ground critical Smart City scholarship within
specific places, foregrounding the distinctive knowledges, concerns and
challenges of marginal, often excluded citizens (McFarlane & Söderström, 2017).

Our engagement in the field began as the project began and will finish in October
2019 when the major intervention in the city is due to be completed. By exploring
learning in a Smart City in a specific location and time we are attempting to
understand ‘situated bodies of practices, into which human actors are differently

18
enrolled’ (Cowley et al, 2018, p.55). In order to understand citizen learning on
the project we recognise a need to account for the conditional and changing local
situatedness of people, policies, technologies and places that create the dynamic
landscape in which the project operates. We therefore sought to explore how
citizen learning was constituted in relation between multiple and diverse actors
‘rather than as only imposed by state or corporate actors from ‘above’ (Cowley
et al, 2018, p. 55).

We conducted participant observation, working closely with project partners. The

data generated includes observations of nine whole project team days, twenty-
seven other meetings, including engagement and communication group
meetings and ten workshops held with local organisations and citizens,
producing in- depth fieldnotes from meetings and events. Details on how citizen
learning was discussed by these project partners were carefully noted and any
changes over time specifically recorded. We also conducted twenty face-to-face
interviews with project partners and organisations, which were transcribed,
including those who were not formal partners on the project but were brought in
as the project evolved, for instance, anchor community organisations in the
locality.

Discourse analysis of key texts, project documentation and online

communications was also undertaken. As we were particularly looking at the
citizen engagement package in the Bristol case study site, we spent time with,
and worked alongside, the KWMC staff, joining their team meetings and
recording their approach throughout the project. In all of our work we were
looking for where accounts of learning became visible and we followed the
actors, attending smaller meetings and having informal conversations as and
when interesting data emerged.

Data were analysed iteratively and analysis was carried out using Nvivo
software, following a thematic approach. As the themes emerged from the data
analysis, the approach to the research could be iteratively developed, in
collaboration with KWMC and others involved in citizen engagement activities.
Interview schedules were adapted to reflect findings as they emerged and to
build on our understandings of citizens and their learning as the project was
rolled out.

3 Findings
In this section we use our data to illustrate, unpick and discuss the work of the
Bristol Living Lab (KWMC) and their particular approach to learning that was
evident in their ‘Citizen Sensing’ work. For a full discussion of the learning models
that circulated within the wider project see Manchester and Cope (2019).

3.1 Creative Citizens Model

The Creative Citizens learning model operated in the citizen sensing element of
the Mimeo project run by KWMC. As discussed earlier this approach situates
learning in historically constructed, unequal relations of power, knowledge and
resource, it recognises the significance of learner diversity and is focused on how

19
inequality might be challenged through working alongside individuals and
communities to discover what animates them, but also in thinking about how they
might be connected into powerful processes at work in the city.

In initial conversations with anchor community organisations in the area it

became obvious to KWMC and ourselves that there was a disconnect between
the everyday lives of many of the inhabitants in the case study neighbourhood,
and the ‘smart’ technologies being introduced and discussed on the project. This
key tension was discussed with one of the project managers at KWMC who
suggested that,

“…the technology doesn’t mean anything to the community and so

[we are] trying to steal that space where technology is developed
based on its meaning and its purpose to real people. But these real
people having a part in that is challenging.” (Interview project
manager, media arts organisation).

As KWMC did not have a long history of working in the targeted area they
recognised that they needed to understand the histories and knowledges held in
the targeted community. A first step involved inviting local anchor organisations
to discuss their possible participation. Many of the community practitioners were
very dubious about the effectiveness of using digital media to tackle inequalities
and felt that there were a range of other priorities in citizens’ lives that should be
given priority (fieldnotes, March, 2017). Considering historical relations of power,
and negative feelings towards both the municipality (in a context of UK austerity)
and the EU, meant that drawing these local knowledges from local civil society
organisations and inhabitants into Mimeo needed to be carefully managed. As
one of the project managers observed:

“Quite rightly some of the organisations were apprehensive even

before it started but for me having seen how those negotiations have
been done and how careful you have to be this has been a great
learning process. And I think if you want to test out anything with
people then you have to build in the groundwork to introduce the
project properly, finding and respecting the key stakeholders who
are already in the community and working from there”. (Interview
project manager, media arts organisation)

Connecting with community anchor organisations in the area, which had been
overlooked in the bid writing, here helped KWMC to acknowledge the huge
amount of knowledge, understanding and research around the key assets,
concerns and challenges faced by those living in the area. This included
integrating the findings from a timely publication of a community research led
project that had identified a key set of priorities around quality of life for
inhabitants of the area. For instance, when asked which services were important
to people’s wellbeing, over half of respondents in the area pointed to parks and
green spaces. These were also identified as a key focus for municipality
investment (Up Our Street, 2017). This community-led research questioned the
assumption in the Mimeo project that ‘smartness’ will deliver a better quality of

20
life. Instead, these findings suggested more green spaces (we might argue the
opposite of smartness) was what was truly desired.

Starting with the everyday issues of concern to residents and providing diverse
opportunities for engagement and learning, meant that this work stream began
to switch away from a focus on the technology and the narrow, technocratic
deliverables of the Mimeo project. As a result, this work was silo-ed, largely
because the inhabitants’ concerns did not connect with the technological
developments listed in the project deliverables. Despite this, the experimental
work led by KWMC continued and the ‘Citizen Sensing’ strand of activity began
to take shape. Citizen Sensing involves collaborating with citizens on the
development of low-tech sensor infrastructures and supporting them to make
sense of and share the knowledge gained through the sensor technology
development.

Initial activity designed by KWMC in the project area involved in depth, informal
on the ground work involving artists and community development workers
‘hanging out’ and beginning conversations in local chicken shops, cafes and
nailbars. Several issues emerged including concerns around damp homes and
poor air quality. Both issues connected with wider concerns that had been
expressed in the community-led quality of life research (Up Our Street, 2017).
Initially the damp homes issue gathered momentum, partly because it connected
with multiple local concerns including health issues, social stigma and poor-
quality housing, demonstrating the complexity of connections between issues
faced by marginalised communities. In order to challenge knowledge hierarchies
in the city it was deemed essential to connect these emerging local concerns
with powerful processes and people in the city. This was possible as these
concerns around health, social and housing inequalities were also shared by
many local organisations and by policy makers in the municipality. In addition,
computer scientists and technologists, open data and other experts were
interested in exploring how sensor data might be utilized by citizens to
understand the issues of concern in more depth. Out of this dialogue, the ‘Damp
Busters’ project emerged.

In developing this work, the importance of connecting lay knowledges with expert
knowledges in order to generate dialogue between them became clear. The
mutual exchange and learning required was challenging, particularly when
innovative technologies, often seen as ‘irrelevant’ in the area, were at the heart
of the inquiry,

It’s great to have different partners from different worlds, it’s a

challenge as well, but the level of expertise is amazing - to speak to
someone who is creating a cutting-edge network the like of which
has not been seen before is a challenge but it’s also exciting to
connect people up who might not have had the chance to access
those sorts of technologies. (Interview, project manager, media arts
organisation)

21
The role of KWMC was vital here in ‘holding’ (in the psychoanalytical sense of
‘holding’ emotions and doubt so that they are manageable, see Bion, 1984;
Bibby, 2009) the collaboration at the early stages, which involved supporting the
translation of knowledge and ideas across the different groups involved. The
importance of offering multiple and varied opportunities for participation and
involving a diverse team in order to facilitate the inclusion of citizen knowledge
and to challenge unequal relations of power quickly became obvious. Artists,
technologists and engagement specialists designed and offered a wide range of
opportunities for learning and participation including workshops, hack days and
working on design briefs together. Participation across the activities varied, as a
KWMC project manager explained,

We didn’t get hung up on everyone attending everything, it wasn’t

that we were going to do lots of hackdays. We offered a range of
different activities to keep people on board.

Learners were seen here as active in understanding what they might want or
need to learn and how Citizen Sensing might help them to do so. Digital learning
opportunities involved diverse groups of people coming together in practice
based, material encounters bringing the technology design into relation with the
everyday lives of those living with damp. Learning here was understood as both
relational and material, involving engagement with artefacts and policy agendas.
For instance, social tenants worked alongside the tenancy officer in the
municipality to identify and target private landlords who ran damp properties in
order to challenge those who were not acting ethically.

In addition, a focus on critical engagement with the actual technologies was

visible in events throughout the damp project that offered people opportunities
to play with sensor technologies and other digital, material devices.
Technologists as well as residents in the area, academics, civil servants and
others took part. The events were designed to share knowledge and expertise
and to demystify terms like ‘data’ and ‘sensor technologies’. In this way an open
design process was created where different knowledges were put into
conversation in order to open up new questions around digital, urban futures
(Storni, Binder, Linde & Stuedahl, 2015). Autonomy and creativity were
encouraged in these workshops where the processes and the digital tools and
interfaces were designed as ‘tools for conviviality’ to encourage engagement
among and between people and the city environment (Illich, 2002).

The Citizen Sensing group built a prototype sensor together using open
technology. The tool was designed in response to a real concern raised by those
involved in the design process. This ‘making together’ approach (Ingold, 2013)
enabled those who were not technology experts to see how things are put
together and to build knowledge about sensors and their design. The co-design
process also supported inhabitants of the area to ask questions around data and
ethics that related to their everyday lives and concerns. As a KWMC project
manager suggested, it was important that the interface itself was user friendly,
however,

22
 “what was really important to the citizens was ‘I want to choose when
it’s switched on, what happens with the data, I want to know who the
data is being shared with”.

The interface that was co-designed in order to house the sensor technologies in
people’s homes was in the shape of frog. The frog, who loves to live in damp
places, became an accessible material symbol for the project that linked the
digital data being collected with the everyday lives of those experiencing damp.
It was designed to be as attractive and easy as possible to use. Notably, the
intention here was not to make the infrastructure invisible to citizens, rather,
tenants in the five households involved in the testing phase were trained so that
they all understood how the technology worked and what data it was collecting.
Through the co-construction of a data agreement, issues related to data
ownership were experimented with, allowing different voices to be heard in the
process, and raising awareness of the various concerns of all of those involved
from the sensor developers, through to the landlords, municipality officers and
residents themselves.

In terms of the sustainability of the work a national sustainable energy

organisation worked alongside 16 people to support them to become
‘Community Damp Busters’- a knowledgeable local team of people able to
support others with damp issues in their neighbourhood. Interestingly, the focus
of the action very firmly moved away from the frog sensors and their deployment
to the issue itself. As the KWMC project manager explained,

“Although the frog became a symbol for the project the real focus is
on making a difference. If we had more information, like if we knew
who owned the property, then we could ask are these buildings fit
for purpose? The key thing to keep momentum going is to empower
people to make changes.”

The Damp Busters project offers us insights into how practices of situated, critical
learning might be adopted with citizens on a Smart City project. This involved
accounting for the everyday lives and unequal relations of power, knowledge and
resources in the area. The approach stresses the need to provide multiple and
varied opportunities for participation, including those that are creative, encourage
autonomy and involve linking citizens into powerful processes and people in the
city. Direct engagement in building technologies together also has the effect of
making visible technology design processes and, in particular, ethical issues that
may be significant barriers to the implementation and sustainability of Smart City
infrastructure.

4 Conclusion
Despite a genuine desire to include citizens in the Mimeo project it remained
difficult for the team to do so. Writing a successful, ambitious and innovative bid
had created a constant tension running through the project in balancing the roll-
out of the technological infrastructure with what could be delivered on the ground
that might be of benefit to local residents. Our argument in this paper is that in

23
order to include citizens in Smart City planning much greater reflexivity is needed
around models of learning, and assumptions about citizens and their capacities
and interests. In particular, early discussion of different learning models and the
collaborative design of developmental practices with all partners is necessary.

Our argument, drawing on extensive literature in education studies, is that a

model of learning that foregrounds collaborative, creative, critical learning for all
involved is necessary in order to include citizens in powerful urban design
processes. This requires significant commitment from policy makers, technology
companies, funders and citizens themselves and the work needs to be brought
out of the silos of sub projects and ‘engagement activities’ and into mainstream
Smart City policy and practice. It is only in this way that we can begin to weaken
the hold of neoliberal, technocratic approaches in the critical imaginations of
those involved in smart urban planning (Cardullo and Kitchin, 2018; Joss et al,
2017).

Funding details
The research was funded by the European Union Horizon 2020 fund under grant
number 691735.

References
Balestrini, M., Creus, J., Masfarre, G. & Caniguearal, A. (2016) A Future in
Common: Understanding and Framing Commoning Strategies for Bristol
Accessed, June 2018
https://marabalesdotcom.files.wordpress.com/2016/01/bristol-approach_
phase2_d2_v6.pdf
Barton, D., & Hamilton, M. (2012). Local literacies: Reading and writing in one
community. Routledge.
Batty, M., Axhausen, K. W., Giannotti, F., Pozdnoukhov, A., Bazzani, A.,
Wachowicz, M., & Portugali, Y. (2012). Smart cities of the future. The
European Physical Journal Special Topics, 214(1), pp.481-518.
Bibby, T. (2009) 'How do children understand themselves as learners? Towards
a learner-centred understanding of pedagogy' Pedagogy, Culture &
Society 17 (1): 41- 55
Bion, W. (1984) Learning from Experience, London: Karnac Books
Buckingham, D. (2006) Is there a Digital Generation? In Buckingham, D & Willet,
R. (eds.) Digital Generations: Children, young people and digital media.
Campbell, T. (2012). Beyond Smart Cities: How Cities Network. Learn and
Innovate, Earthscan, New York (NY).
Cardullo, P. & Kitchin, R. (2018) Smart Urbanism and smart citizenship: The
neoliberal logic of ‘citizen-focussed smart cities in Europe. The
Programmable City Working paper 39,
http://progcity.maynoothuniversity.ie/

24
Cowley, R., Joss, S., & Dayot, Y. (2018). The smart city and its publics: insights
from across six UK cities. Urban Research & Practice, 11(1), pp. 53-77.
Erstad, O., & Sefton-Green, J. (Eds.). (2013). Identity, community, and learning
lives in the digital age. Cambridge: Cambridge University Press.
European Union: Market Place of the European Innovation Partnership on Smart
Cities and Communities (2017) Inclusive smart cities: A European
manifesto on citizen engagement. Retrieved October 19th, 2018,
https://eu-smartcities.eu/sites/default/files/2017-09/EIP-
SCC%20Manifesto%20on%20Citizen%20Engagement%20%26%20Inclu
sive%20S mart%20Cities_0.pdf
Eynon, R. (2015) The quantified self for learning: critical questions for education,
Learning, Media and Technology 40 (4) pp.407-411.
Gabrys, J. (2014). Programming environments: environmentality and citizen
sensing in the smart city. Environment and Planning D: Society and Space,
32(1), 30-48
Gee, J. (2000) Chapter 3 : Identity as an Analytic Lens for Research in
Education. Review of research in Education.
https://doi.org/10.3102/0091732X025001099
Hambleton, R. (2014). Leading the inclusive city: Place-based innovation for a
bounded planet. Bristol: Policy Press.
Hollands, R. G. (2015). Critical interventions into the corporate smart city.
Cambridge Journal of Regions, Economy and Society, 8(1), 61-77.
Ingold, T. (2013). Design and anthropology. Ashgate Publishing, Ltd..
Jenkins, H., with Purushotma, R., Robison, A.J. Weigel, M., and Clinton, K. (2007)
Confronting the challenges of participatory culture: Media education for the
21st century (Part One) Nordic Journal of Digital Culture, 1:2, pp. 23–33
Joss, S., Cook, M., & Dayot, Y. (2017). Smart cities: Towards a new citizenship
regime? A discourse analysis of the British Smart City Standard. Journal
of Urban Technology, 24(4), 29-49.
Knowle West Media Centre (2016) The Bristol Approach in action: working
together to create tech for the common good. Accessed June 2018
https://issuu.com/knowlewestmedia/docs/bristol_approach_booklet_issu
McFarlane, C. (2011) Learning the City: knowledge and translocal assemblage.
Oxford: Wiley Blackwell.
McFarlane,C. & Söderström, O. (2017) On alternative smart cities, City, 21:3-4,
312- 328, DOI: 10.1080/13604813.2017.1327166
Manchester, H. and Cope, G. (2019) Learning to be a smart citizen. Oxford
Journal of Education 45: 2, 224-241.
Miller, D., (2010). Stuff. Cambridge: Polity.
Potter, J. & McDougall, J. (2017) Digital Media, Culture and Education: Theorising
third space literacies. London: Palgrave MacMillan.

25
Selwyn, N. & Facer, K. (2013) The Politics of Education and Technology. London:
Palgrave Macmillan.
Söderström, O., Paasche, T., & Klauser, F. (2014). Smart cities as corporate
storytelling. City, 18(3), pp. 307-320.
Storni, C., Binder, T., Linde, P., & Stuedahl, D. (2015) Designing things together:
intersections of co-design and actor–network theory, CoDesign, 11:3-4,
149-151, DOI: 10.1080/15710882.2015.1081442)
Street, B. (2003). What’s “new” in New Literacy Studies? Critical approaches to
literacy in theory and practice. Current issues in comparative education,
5(2), 77-91.
Street, B. (2013) (2nd edition) Social Literacies: Critical Approaches to Literacy in
Development, Ethnography and Education. Routledge: Oxon
Tapscott, D. (1998). Growing up digital: The rise of the net generation. New York:
McGraw-Hill.
Up Our Street (2017) Understanding wellbeing through community research in
Easton and Lawrence Hill (Research Report). Accessed, June 2018:
https://upourstreet.org.uk/project/486/wellbeing-research-project.

26
27
 Agile piloting for smarter cities: 3 cases of
engaging ecosystems and communities
in co-creation
Kaisa Spilling*1, Janne Rinne1, Matti Hämäläinen1

*Corresponding author
1 Forum Virium Helsinki

Category: Innovation paper

Topic: Open Track

Abstract
Agile Piloting Programme is a proven method that supports and facilitates smart
city development and engage a wider stakeholder network to co-create new
services. During 2016-2019, more than 50 pilots have run on the different
platforms in Helsinki. The thematic piloting rounds have ranged from climate
positive and resource wise solutions to innovative local services and wellbeing.
The model has been adopted in different domains of smart city and used widely
in the network of six biggest cities of Finland. In this paper we briefly present the
model and give examples of three cases that highlight different aspects on how
co-creation and experimentation has been applied in different city platforms:
Smart Kalasatama (Health & Wellbeing), Jätkäsaari Mobility Lab (mobility and
transport) and Helsinki schools (education).

Keywords: Smart Cities, Living labs, Open innovation, Experimentation, Start-

ups, Ecosystems

28
1 Introduction
Smart city development has become the mainstream. It is not only about
technology. Cities are looking for new ways of working together with companies
and other stakeholders to solve the problems of rapidly growing and evolving
cities. The new solutions require more agile ways of operating and collaboration.
Agile piloting was developed as a method to activate the innovation ecosystem,
and it opens the city infrastructure, data and services as an urban lab for
experimentation. It also offers companies and start-ups an authentic real-life
environment to test and develop their services, together with residents who may
participate in the process as experts of everyday life. Lean development,
experimental culture and a permission to fail are what it takes to create the urban
futures.

From 2013, Forum Virium Helsinki — a city owned innovation company — has
been orchestrating the innovation platform activities in Kalasatama, the model
district for smart city development in Helsinki. One of the methods created is Agile
Piloting Programme, a proven method that supports and facilitates smart city
development and engages a wider stakeholder network to co-create new
services. During 2016-2018, Smart Kalasatama Agile Piloting Programme has
run and facilitated 21 agile pilots. By 2019, more than 50 pilots have run on the
different platforms in Helsinki. The model has been adopted in different domains
of smart city and used widely in the network of six biggest cities of Finland and
the city of Stavanger in Norway.

Conventional smart city development often focuses on long term infrastructure

projects and is driven by large players, such as construction companies, energy
industry, etc. The programme model for Agile piloting was developed in order to
accelerate smart city development and enable the participation of smaller players
— such as start-ups and SME’s — and to share any learning outcomes with a
much wider audience. Agile piloting also provides a means to engage citizens
and the users of the services as pilot initiators, co-developers, users and to
provide learnings about what smart city development is all about. The model has
been documented in CookBook – Recipes for Agile Piloting, a guidebook for city
innovators.

Experimentation and small, agile pilots provide useful means to approach an

uncertain future. Agile piloting is a good method for creating something new in
order to uncover the best solution and to gain insight on how users experience
the service. Experimentation produces new understanding. The aim is to learn as
much as possible, to speed up the development of solutions, and to create new
business models. All parties involved gain value from the different learnings
generated during experimentation; the company carrying out the pilot and
platform facilitating the process, as well as the city and the wider network.

In this paper, we briefly present the model and give examples of three cases that
highlight different aspects on how co-creation and experimentation has been
applied in different city platforms: Smart Kalasatama (Health & Wellbeing),
Jätkäsaari Mobility Lab (mobility and transport) and Helsinki schools (education).

29
1.1 Agile Piloting programme
The Agile Piloting programme invites Start-ups and SMEs by open call to test and
co-develop their services in real-life environment during a 6-month period. The
programme procures pilots for a small compensation (e.g. 1,000–10,000€). The
pilots are typically financed with external project funding, but the city, corporate
partners or any other actor can act as a funding partner. The programme offers
tools to innovate, co-create and experiment new services together with the start-
ups, the city and other stakeholders. Learning is in the heart of the process.

The programme engages the different actors of the quadruple helix, e.g. the
companies, residents, city officials and researchers, and creates value for all
participants. Small start-ups have innovative ideas, but they need support for
executing pilots in real-life environment and getting feedback from users. Co-
creative process is in the core, and the programme supports the pilots by
providing them with methods for co-creation, access to local communities and
city infrastructure, new networks, market reference and visibility.

The startups have been satisfied: agile piloting accelerates learning, and positive
visibility in media for the pilots has been appreciated. Engaging corporate
partners to the process allows them to offer technologies or platforms and
connect with startups, the city and residents. The residents gain understanding
of future solutions and have the opportunity to explore and affect new services.
For the city, the pilots are a way of getting a sneak peek into the future, and they
work as an anticipation method to forthcoming changes.

1.2 How it works: the process

Running a piloting round requires intensive facilitation work from the Living Lab.
Facilitation of the experimentation process is about leading networks,
communications and hands on work. The most effective use of resources is
achieved when pilots are run together as a programme. There are many
synergies when the various stages of instruction, facilitation and evaluation are
carried out simultaneously with several pilots. A crucial task is the creation of trust
among the various players. In addition to the managerial activities related to
innovation processes and resources, mediation among stakeholders and
activities is needed to create trust and shared meanings enabling shared
learning, a shared vision and shared value creation among the multiple actors
needed in living labs. (Äyväri, Jyrämä, Hirvikoski, 2018)

The facilitator organisation, the living lab or the city engages the various
stakeholders in the process in the different stages: defining themes, open call,
selection of pilots, experimentation and evaluation. The process is described
more in detail in the Cookbook for Agile Piloting Cookbook (Mustonen, Spilling,
Bergström. 2018)

1) Stakeholder mapping and defining themes

The key questions to start with when initiating a piloting programme are the
following:
• Who owns the programme? What are the learning goals?

30
 • Who are the central cooperation partners who will participate in
defining the theme, in funding the pilots and in co-development?
Cooperation partners can also provide an environment or other
resources for experiments.

The first step when planning an agile piloting programme is defining the themes
and the challenge together with the collaborators. This stage is an important one
as it is a way to set the common goals and a means to get the essential parties
committed.

2) Open Call
A piloting programme starts by an open call for pilots. Typically, in open calls for
agile pilots, the solutions can be based on existing data, such as open data and
use technology in an innovative way. The piloting programme can be used to
enhance an existing solution with a new user group, or to add new features to it.
Another aim is to test and validate new business models or new ways for involving
partner companies. The selected pilots should be linked to the local platforms or
the collaborator partners activities.

Information about the call is shared through a broad range of channels. The
collaborator network’s channels are used widely for communications. The
collaboration network is also engaged in the selection of the pilots.

3) Selection of pilots
The pilots are selected in collaboration with an expert jury. The living lab is
responsible for making agreements with start-ups about the pilots. A common
kick-off is arranged for the selected pilots. The key stakeholders are engaged
from selection and the kick-off to participate to the process.

4) Experimentation
The experimentation phase is an up to 6-month long process, facilitated by the
living lab that helps with user recruitment, stakeholder collaboration, integrating
to the infrastructure and co-design. The multiplayer network requires active
communications with stakeholders about the content and progress of the
experimentation. Best way to succeed is to collaborate with partners and engage
them to actively communicate through their own channels, and encourage all
parties to share their experiences.

The living lab facilitates the experimentation process and supports the pilots by
recruiting users / professionals, organising workshops, helping to integrate to city
infrastructure and service.

5) Evaluation
In order to maximise the learnings from the process it is essential to evaluate the
process itself, as well as the pilots. Evaluation is an ongoing process, the model
consists of several tools to support this such as reporting in the form of light
questionnaires in the different phases of the experimentation, and physical
facilitated events that gather the relevant stakeholders. The results of the pilots

31
are documented, and the learnings and best practices disseminated widely - in
the best cases, the services are scaled up wider. The overall learnings from the
piloting programme can be scaled up and taken forward in all participants own
activities.

2 Case 1: Kalasatama Wellbeing - co-creation to engage wider

ecosystem
2.1 Smart Kalasatama - smart & sustainable district as a Living Lab
Smart Kalasatama in Helsinki, is a former brownfield area, developed to a
residential district. By 2035, the district will offer a home for approximately 25,000
residents and jobs for 10,000 people. Currently, there are 3,500 people living in
the area. It is a vivid Smart City innovation platform to co-create smart sustainable
urban infrastructure and services. Smart Kalasatama is developed flexibly and
through piloting, in close collaboration with 200+ stakeholders including
residents, companies, city officials and researchers. The Smart Kalasatama
Innovator’s Club, gathers the area’s companies, organisations and residents,
who take part in defining needs and participate as users in pilots.

The vision of the smart district is that smart services save one hour of citizen’s
time every day. Building the common vision with the stakeholders served to
identify and focus the thematic areas for experimentation. In Kalasatama, the
agile pilots have been exploring different areas of smart and sustainable everyday
life, such as energy and resource efficient services, sharing economy, health and
wellbeing.

2.2 Co-creating wellbeing

The Kalasatama Health and Wellbeing Center opened in February 2018. The new
centre provides a range of public health and social services available under the
same roof. The spaces are flexible and multi-purpose. Already during the
construction of the centre, Smart Kalasatama team initiated collaboration with the
City of Helsinki in order to define a framework of development and the themes to
be taken up further (Hirvikoski, Lehtonen & Äyväri 2016). The new centre opened
possibilities for piloting together with professionals and using the Health and
Wellbeing centre as a platform for experimentation.

The recent Kalasatama Wellbeing piloting programme, in 2018, accentuated the

role of corporate partners on the side the City. The partners beyond the city
(Kalasatama Health and Wellbeing Centre) were the corporations all active in the
district2. Laurea University of Applied Science served as a research and
development partner.

2.3 Co-creation throughout the process

2 Construction company SRV Group is an active developer with a commercial center and residential
buildings in the district, Kesko retail group has wide occupational health services, with their new campus
located in the heart of the district and IT consultancy CGI Finland, has a range of solutions and services
for cities.

32
The process started by defining the challenge to be solved together, gathering
the corporate partners, the city and the living lab team. The thematic framework
for the Kalasatama Health and Wellbeing centre as a platform for experimentation
defined with the professionals of the city of Helsinki, served as a starting point
(Hirvikoski & al 2016). The challenge formulated together was seeking to answer
the following question: how can the residents to better take care of their personal
wellbeing and health on a daily basis?

The open call was launched in January - February 2018 and engaged over 30
start-ups offering their pilots. The expert jury selected the top 10 pilots evaluated
with a set of common criteria. These start-ups were invited in a Co-creation Jam,
organized to give a better understanding of the collaboration partners aims, the
start-ups’ goals and enable to finetune their pilot ideas. The event also gave
possibilities to find opportunities for collaboration with the other teams. The Co-
creation Jam concentrated on the pilot proposals business model and value
proposition, experimentation goals and user experience. The pilots were selected
in a pitching session followed by the event. An essential ground for the selection
was to identify the real-life platform for each pilot, to enable a quick start for the
experimentation phase. The five pilots selected, presented digital services
focusing on healthy nutrition and wellbeing, in particular stress management,
helping the residents to take better care of their health in daily life.

2.4 Engaging wider ecosystems

Agile piloting programme targets to create service ecosystems around the piloting
themes, and a wider audience can be invited to the process. These can include
the representatives of other sectors of the City, and even non-health and welfare
companies, all in all actors supporting the activities and interested to explore the
possibilities for cooperation. The pilots serve as a neutral ground for co-creation
with a wider ecosystem.

Within the 6-month long Kalasatama Wellbeing programme, the start-ups and
collaboration partners were gathered in two common meetups. In the first one,
the aim was first to strengthen collaboration across pilots and with other partners,
help to overcome boundaries in the starting phase. In the next one, the aim was
to share learnings from the pilots, as well as to discuss potential to scale the
learnings and focus on next steps. The thematic co-creation workshops arranged
served to engage additional external partners. The end event of the programme
gathered a wide audience to discuss the pilot experiences as well as more
generally innovation platform activities in the field of health and wellbeing.

A total of 450 end users and 20 professionals participated in the piloting

programme. The corporate collaboration partners brought dynamics to the
programme by providing optional test beds, potential partners and customers for
the start-ups, which was especially appreciated by the start-ups. Furthermore,
the programme tested in practice how the collaboration with a university partner
can support the living lab phase within the Health and Wellbeing centre, and thus
make more intense piloting possible. The students from Laurea University of
Applied Sciences were a valuable asset to support the living lab work in practice.

33
Evaluation played a central part of the programme in order to generate learnings
for all parties involved. The pilots were evaluated from the perspective of the end
users and the professionals process. On the other hand, the programme was
evaluated from the perspective of the start-ups and the collaboration partners.
Bringing in corporate partners demands more from the facilitating organisation,
but is valued both by the start-ups and the city. The best results occur when the
corporate partners have a clear role and can for example provide their
technologies or platforms or act as an alternative test bed for pilots.

3 Case 2: Jätkäsaari - Engaging residents to mobility

innovation
Recently, alongside infrastructure development and regulation, urban mobility
challenges have been approached through experimentation and citizen-centric
innovation. In Jätkäsaari Mobility Lab, inclusive co-creation and facilitated
experimentation are used to boost mobility innovation and to tackle local mobility
challenges. Here, we describe how the model for agile piloting, originally
developed in Smart Kalasatama, has been applied and adapted in the domain of
smart mobility.

3.1 Tackling a real-world mobility challenge

Helsinki is one of the busiest passenger ports in Europe, due to extensive ferry
traffic between two capital cities, Helsinki in Finland and Tallinn Estonia. The
former logistic port of Jätkäsaari has moved to the outskirts of Helsinki, but the
passenger port remains in Jätkäsaari, with about seven million passengers
passing through the neighbourhood annually. Jätkäsaari is also the largest district
development project in Finland, converting the brownfield areas of the former
logistics port into homes of almost 20,000 people. Jätkäsaari is connected to the
city centre with only two main bridges and suffers from traffic peaks from the port
and construction-related traffic.

As Jätkäsaari is undergoing serious transformation and as its mobility challenges

are very concrete and real, it has been selected as the testbed for novel mobility
services, technologies and solutions in Helsinki. There is a long history of
mobility-related R&D projects in the area. What differentiates Jätkäsaari from
many mobility labs in Finland and abroad, is the involvement of the residents in
the co-creation and living lab activities.

3.2 Citizen-centric mobility experimentations

The approach applied in Jätkäsaari follows the model of agile piloting developed
in Smart Kalasatama. The role of residents in the co-creation is highlighted and
they are deeply involved in the key phases of the process - in defining the mobility
challenges to be tackled, in co-designing the pilots, in using the piloted services
and in giving continuous feedback about them.

The local mobility challenges are very tangible and have an impact on the daily
lives of the citizens. For this reason, the local community and residents were
invited in the process from the very beginning, in defining the mobility issues that
the agile piloting would address later.

34
The content and scope of the piloting round was defined in collaboration with
residents, city professionals (urban planners and traffic planners), and private
companies. To this end, local mobility needs, development areas and pain points
were addressed in four open co-creation workshops and via an online
questionnaire targeted at residents during December 2019 - January 2019. The
results of the web survey (n=150) gave input to the iterative workshops, that
identified the mobility pain points and potential solutions to them. An open call
was formulated on the basis of the workshop results.

In January-March 2019, the open call was published, with the aim to find new and
innovative mobility solutions. Companies were invited to offer pilots that would
offer practical solutions to the challenges faced by the residents or to reduce
congestion, emissions or other negative impacts of traffic.

An expert jury scored the proposals according to the evaluation criteria: novelty
and innovativeness, feasibility, and impact. Traffic planners and urban planners
from the City were represented in the jury. A total of four pilots were ultimately
selected for implementation: 1. A local cargo bike sharing scheme, 2. Shared
hobby and sports transports, 3. Local cargo bike logistics, and 4. Smart
pedestrian crosswalk.

The open kick-off event was organised on 27 May 2019, with co-design sessions
open to residents, planners and other local stakeholders. The aim was to gather
further ideas for the implementation of the pilots. Once all pilots will be running
and open to end-users, the residents will have an active role as co-designers,
end-users and feedback providers. The active piloting period with the end-users
takes place June-October 2019. The results of this piloting round will be available
and shared with the wider ecosystem by the end of the same year.

3.3 Insights to engaging the residents

In Jätkäsaari, residents do have high interest in their living environment and
strong opinions about its development. The neighbourhood is under construction
and the residents want to influence how the whole neighbourhood will turn out to
be. However, having them involved in practical terms requires a significant effort
from the facilitator of the pilots — in this case Forum Virium Helsinki.

A further challenge is to ensure that the process would be as open as possible,

and not biased to specific individuals, organisations or interest groups. Utilising
existing communication channels, networks, communities, and “gatekeeper
organisations” is the key for engaging with the diverse local community. In
Jätkäsaari, Forum Virium Helsinki has teamed up with local key actors. Formal
neighbourhood association (Jätkäsaari-Seura), informal social media
communities (Jätkäsaari-liike), Jätkäsaari district development project of the City
of Helsinki, and Port of Helsinki have been valuable partners in the process.

Forum Virium Helsinki has presented the upcoming pilots and related co-creation
activities in various local events. Some of them have been addressed specifically
agile pilots and related living lab activities. In engaging with new residents, the

35
existing local events organised by others have been very valuable, for example
the participatory events of The City, meetings of Jätkäsaari association, and local
fairs organised by the Port of Helsinki.

The communication of the pilots has been diverse, ranging from websites to
posters in the library and local grocery stores, and from social media campaigns
to a stand in the events. A dedicated living lab website for piloting and
experimentation will be launched during summer 2019. This website will be an
arena to communicate topical issues about pilots and their results, a low-step
channel for residents to get involved and to exchange ideas about the piloting
and living lab activities. The goal is to make the involvement easier for the
residents and to make the facilitation of the participation more structured and
organised.

4 Case 3: EdTech co-creation with Schools

Education Technology is one of the fastest growing fields of business today,
already worth more than $6 trillion per year, and growing at the rate of ~7%
annually. Looking at these numbers, it’s no wonder that there is great interest in
developing new products and services that could take a slice of this pie. However,
the field of education is quite protective by nature, and the education community
does not react well to outsiders who might be suggesting changes to the
prevailing practices. As the world around is changing rapidly through continuous
advances in digital technologies, the schools cannot stay inert to the change, but
will need to adapt. At the same time, the schools cannot expose their students to
the numerous edtech fads that might jeopardise their learning outcomes for the
price of convenience or cost-effectiveness.
The city of Helsinki, among many other regions, has recognized the situation, and
made commitments to speed-up and facilitate the diffusion of innovations into the
public-school world. One revelation along this track has been that, there is a clear
need for a platform, on which the teachers, students, and edtech companies can
co-create, co-develop, and test novel educational tools and methods — and do
so while developing corresponding pedagogical practices. Whether the process
is driven by the schools or by the business, it is clear that there needs to be clear
benefits for both parties; otherwise nobody would be willing to commit to it.

4.1 Towards a new EdTech Co-Creation Platform

In 2017, an agile piloting programme begun in order to establish a new,
permanent edtech co-creation platform. The plan was to initiate a transparent
model for edtech start-ups to co-develop or pilot their products at public schools
in Helsinki. Before the pilot programme began, there were already numerous
pilots running at Helsinki schools. Due to uncertainty in regulations and policies,
these pilots were mostly conducted under-the-radar, without any formal approval
from, nor communication with the educational authorities.

In order to tackle the lack of transparency, the programme structure was designed
to be as flexible and inclusive as possible. The main objective of the piloting
programme was to make the piloting more transparent within the Helsinki
education community, primarily in order to spread the results and to build up a

36
culture of co-creation, but also to increase the impact of any pedagogical
discoveries or revelations. A plan was made for a piloting programme structure,
in which the facilitator organisation was to offer facilitation and formal support in
order to achieve higher quality and better outcomes. Following the format of Agile
Piloting (Mustonen et. al. 2018) and the Espoo KYKY Model (Sutinen et. al,
2016), the pilot programme structure was following a standard format, while in
the spirit of lean development, most of the process facilitation was planned to
take place at the beginning of the program. Rather than acting as gatekeepers,
the organisers took the role of matchmakers between the schools and
businesses, and tried to make sure that any proposed pilot projects had a clear
match with an identified need from the educator community. Once the screening
and matching had been completed, the facilitators assisted with the planning of
the pilot project; after which the co-creators were given almost complete
autonomy over their pilot projects.

During the piloting programme, the projects had to be initiated by the co-creation
platform rather than waiting for either of the parties to make the first move (which
would be the expectation for the more permanent platform in the long run). For
this reason, a one-day workshop was organised in order to identify specific needs
from the user community (i.e. schools), and to transform those needs into formal
challenges that would be used to invite edtech companies to participate in the
program. About a dozen teachers were invited to as experts to this workshop that
took place in two stages. First, a discussion was facilitated around the topic of
trends in education and digitalisation. Based on this discussion, a few dozen
explicit and latent needs were identified for the education sector. Later on, these
needs were categorised and screened in order to forge them into specific
challenges.

Co-creators were then selected through an open call, followed by a workshop in

which facilitators helped the participants to create solid plans for their Agile Pilots.
After this workshop, the facilitators stepped back, and allowed the co-creators to
focus on conducting their pilots. After the piloting period was over, a facilitated
wrap-up meeting was held between the co-creators. At this meeting, the
experiment was evaluated on two levels. First, the execution of the pilot project
was analysed, making note of how accurately the piloting plan was followed, what
type of facilitation took place during the process, how well the schedule held, etc.
Second, the outcomes of the pilot were evaluated from participants perspectives;
did the results satisfy the expectations, were any new discoveries made, etc.

During the first two rounds of the piloting programme, 16 edtech start-ups were
running a total of 27 co-development or pilot projects in collaboration with 17
schools, involving roughly 50 teachers and more than 1500 students. The piloting
programme evolved to a model labelled as Easy Access Co-development, which
is expected to become the core of the more permanent edtech co-creation
platform.

5 Scaling up learnings
Piloting is a way to start anything - a systemic change, new services and activities.
A city makes the most out of piloting by linking the themes to the strategic aims.

37
It also provides a means to scale up: after running several piloting programmes,
and tens of pilots we have explored that the learnings from the pilots keep on
living and are available for the stakeholders to take further in their own activities.
In some cases, the pilots as such or in a format incorporating learnings are scaled
up to other districts.

Agile piloting in smart city development is a fast way to gain more insight into
problems and make them visible as well as to engage a range of stakeholders. It
enables the whole urban community to learn as much as possible during the
intensive piloting period. Piloting reveals the challenges in scaling up smart
solutions. Most of the solutions share the same challenges, such as lack of
business ecosystems, legislative and regulatory issues, lack of interoperability
and the slowness of change in behaviour of the users in adapting new services.
(Mustonen & al, 2018).

5. 1 Added value by engaging diverse actors

Although the approach of agile piloting has been adapted in varying contexts, the
engagement of diverse actors remains as one of its key elements. Co-creation
with wider network - private companies, citizens, civil servants, research partners
- adds value but does not happen by itself. Implementing a high-quality piloting
programme requires resources, in terms of working time, expertise and other
resources from the facilitating organisation. This should not come as a surprise
to the facilitating organisation.

The experience showed that engaging corporate partners as collaborators in the

piloting programme brought new dynamics to the process, and also engaged
more mature start-ups to participate. Both start-ups and corporate partners
appreciate an efficiently facilitated process: it saves time and resources for other
actors, which is a major benefit to start-ups in particular.

The co-creation workshops and user feedback provide valuable lessons learned
and follow-up ideas concerning the development of services and business
models. Piloting in real-life environment also provides unanticipated learnings
that were proven particularly valuable. The start-ups gave positive feedback on
the large number of contacts that the programme allowed them to establish and
especially appreciated the visibility given to the pilots.

Running a piloting programme, is not only effective use of resources for the living
lab, but it also creates synergies among the start-ups participating in the
programme. The facilitator supports collaboration by the various activities
arranged during the different steps in the piloting programme.

City professionals are crucial stakeholders for many reasons. In a health and
wellbeing context the professionals are an essential and truly valuable user group
- that often acts as a gatekeeper organisation to the end users. In mobility context,
the planners participated in the definition of the mobility challenges and in the
selection of the pilots, bringing necessary expertise and realism in the most high-
flying ideas from the residents. Bringing civil servants on board in early stages
increases the commitment and ownership in the process.

38
Even when part of the costs of the pilots are being covered for the start-ups,
money is not the primary reason for taking part in agile piloting. According to our
experience, what is valued the most are the real urban testing environment and
real end-users, facilitated piloting process, support in communication, and access
to networks of key stakeholders and potential cooperation partners.

5.2 Trust and clear rules of the game

Developing a trustful communication with the residents takes time and effort. It is
crucial to identify the existing networks, organisations and gatekeepers and
cooperate with them. Identifying the key local actors is efficient timewise, and
cooperation with them helps to understand the local context, wishes and
concerns of the locals, and it also increases the commitment of the local
community to the process.

When collaborating with the residents, it is important to speak their language and
not to confuse them with “innovation jargon”. It is important that they understand
that pilots provide something practical and useful that they can understand and
relate to. Communicating the results of the pilots back to active residents is
necessary in order to foster the relationship and commitment to future pilots as
well.

Working with the school or health care centre provides a different living lab
environment, where most often the communications to the end users runs
through the teachers and healthcare professionals, if not agreed otherwise. There
the key is clear rules of the game with the professionals whose working
environment and processes are central.

6 Summary
Agile piloting provides a flexible framework for co-creating urban futures.
Experimentation also provides learnings for the living lab - the model is an
adaptive process that can be modified and further developed to better accelerate
smart city development and provide value for the different stakeholders. This has
been proven by the wide adaptation in the cities in Finland, with over 100 pilots
run and interest from other Nordic countries.

For us the Agile Piloting programme model has been a learning process on how
to best accelerate the creation of smart urban services and engage the urban
community. It has also provided means for creating wider ecosystems that take
the learnings of the pilots further in their own activities. Piloting in different real-
life environments of a smart city has different requirements in terms of users, city
professionals like teachers or healthcare professionals and city infrastructure.
What is common for all of them is that co-creation is a valuable means for
developing better future services and agile pilots to provide a neutral platform for
co-creation.

The three case examples of this paper demonstrate that the model for agile
piloting can be applied in very different domains. It is essential for the facilitator
organisation to know the thematic and local context and adjust the activities

39
accordingly. As highlighted in the three case examples, different actors may have
very different perspective towards piloting. The facilitator organisation should be
aware of the different perspectives, expectations and concerns and recognise
these in the process.

We conclude that Agile Piloting has a lot of potential to spark innovation and to
solve concrete challenges in different contexts. However, the programme for agile
piloting is not a rigid model that fits automatically everywhere, but rather a flexible
framework that should be - and has been - adapted in varying ways in different
settings.

References
IBIS Capital (2013) Global e-Learning Investment Review
Hirvikoski T., Lehto P., Äyväri A., (2016). Development and experimentation
platform for social, health and wellbeing services in the context of
Kalasatama health and wellbeing centre., Laurea-ammattikorkeakoulu.
ISBN 978-951-799-441-5
Mustonen, V., Spilling, K., Bergström, M., (2018). Cook Book – Recipes for agile
piloting. Helsinki: Forum Virium Helsinki
https://drive.google.com/file/d/1WeLZoDVVkf-
QkjL1LUSsOXY_XL9MDs6U/view
Rill, B. R., & Hämäläinen, M. M. (2018). The Art of Co-Creation. Palgrave
Mcmillan. ISBN 978-981-10-8500-0
Sutinen P., Erkkilä K., Wollstén P., Hagman K., Hirvikoski T., Äyväri A., (2016).
KYKY Living Lab handbook for co-creation by schools and companies.,
Espoo: City of Espoo. ISBN 978-951-857-732-7
Äyväri A., Jyrämä A., Hirvikoski T., (2018). The Circle of Mediators: Towards a
governance model for tackling sustainability challenges in a city. ,
European Network of Living Labs. URN:NBN:fi:amk-2018090414850
Äyväri A. (2018), Kalasataman terveys- ja hyvinvointikeskus kehittämis- ja
kokeilualustana: tavoitteet, palvelut ja prosessit yritysten ja alustan
näkökulmasta, Laurea-ammattikorkeakoulu

40
 Co-Creating Technology for Societal Change:
A Mobile App Addressing Homelessness
Rachel Burrows*1,2,3, Antonette Mendoza2, Sonja
Pedell3, Leon Sterling2,3, Tim Miller2 and Alexi Lopez-Lorca4

*Corresponding author
1PsyLab Ltd, Cambridge Science Park, United Kingdom
2 University of Melbourne, Australia
3Swinburne University of Technology, Australia
4
Semantic Web Company, Vienna, Austria

Category: Full Research

Topic: Open Track

Abstract
Living Lab projects often involve the collaboration of diverse stakeholders. This
is particularly true with new technology that aims to tackle the systemic and
societal problem of homelessness. In this paper, we present a mixed-method
approach to understand the perspectives of key stakeholders. We discuss our
findings and their implications for the development of a mobile app that aims to
help people who are homeless. We measure usage of the mobile app which
currently attracts over 10,000 users each month in Australia. We also conduct
semi-structured interviews with 30 participants who are either homeless, ex-
homeless or service providers. Our study provides insights and an approach that
may help others in developing similar systems. We discuss barriers and enablers
of success relating to (i) organisational concerns from service providers, (ii)
maintaining awareness of the system in the homeless community, and (iii)
supporting user needs in software design. We propose and demonstrate our
emotion-led approach to bring a novel perspective on the concerns from key
actors influencing the adoption of new technologies.

Keywords: Thematic Analysis, Homelessness, Emotion-led Design, Mixed-

Method, Technology innovation, Living Lab approach, Socio- technical systems

41
1 Introduction
Homelessness is a systemic societal problem that requires a holistic approach to
progress towards a solution. In Australia, many organisations and communities
aim to help people experiencing homelessness transition into stable living
situations. Existing service providers may help with a range of needs from
temporary accommodation, meals, or finding a job. Despite efforts, the number
of people in Australia experiencing homelessness has increased by 14% in the
five years leading up to 2016 according to the Australian Bureau of Statistics.
This urgent situation has led to new initiatives to improve the access that people
experiencing homelessness have to relevant information about services. One
way to achieve this is through the development of new technology that helps
people experiencing homelessness access the help that they need.

Unfortunately, introducing new technology to help those who are homeless brings
unique challenges. Existing research outlines challenges that are faced by
technology developers (Woelfer et al 2009, 2011a, 2011b, 2012; Hersberger
2003, 2012; Chatman et al 1996; Le Dantec et al 2008; Munõz et al 2004;
Humphry 2014; Deng et al 2016). For instance, Woelfer (2009) and Chatman
(1996) suggest this group of users may not have the necessary devices or
motivation to access information online. Hersberger (2003) suggests that this
group of users suffer from information overload. New technology may, therefore,
risk contributing to their confusion. Finally, Weise et al (2017) outline how new
technology may be resisted due to incompatibility with existing service provider
protocols.

Due to the sensitive nature of this application domain, and the complexity of the
socio- technical system, we use an emotion-led approach to guide our analysis
and understanding. New emotion-led approaches to software design and
evaluation are now being increasingly utilised. These approaches aim to
understand how we can resolve concerns of all stakeholders and address their
needs. People will reject technology if it does not address their emotional
concerns (Dix et al 2003; Krumbholz et al 2000; Norman 2005; Pedell et al 2014;
Miller et al 2015). Emotional experiences with technology are formed and then
change over time based not only on the actual technology engagement
experience but also are layered with associated experiences (Saffarizadeh et al
2017; Alatawi et al 2018). This need is heightened when creating technology for
vulnerable users. For instance, users may wish to feel in control, connected,
hopeful, cared for, or empowered, among others (Toscos et al 2013; Pedell et al.
2014; Saffarizadeh et al 2017).

This work reacts to questions and uncertainties around new technology for
homelessness. There is a lack of understanding over if and how new technology
can be introduced into this complex system of service providers. While there are
some studies that focus on designing for vulnerable user groups, and even those
who are homeless, these studies do not focus on emotional experiences and also
do not evaluate a deployed system. Our work has contributed to the development
of a web app that helps homeless Australians find information about services that
can help them. The app has been deployed since 2016, contains information
about services providers, and currently attracts over 10,000 users each month.

42
There are 16 service categories including food, housing, everyday needs, money
help and counselling. The objective of this work is to (i) provide an in-depth
understanding of the key barriers and enablers to success, and (ii) provide advice
for living lab projects who are facing similar challenges.

The remainder of this paper is structured as follows: We outline relevant literature

on homelessness, and approaches for designing technology that helps those
experiencing homelessness. We then outline our research method and present
findings. Quantitative user engagement data was collected to understand usage
trends. We also conduct semi-structured interviews with participants who were
either homeless, ex-homeless, or service providers. Seven themes emerged that
are associated with emotional viewpoints or experiences of key actors. These
themes represented both barriers and enablers of success depending on how
they were addressed. We finally discuss and provide advice in relation to three
challenges that may be faced by similar initiatives.

1.1 Homelessness and Service Providers

The Australian Bureau of Statistics defines homelessness as:

“...in a dwelling that is inadequate; or has no tenure, or if their initial tenure

is short and not extendable; or does not allow them to have control of, and
access to space for social relations.”

There are many government services across Australia whose purpose is to help
those experiencing homelessness to find the help that they need. Service
providers allocate an extensive range of support, and become largely responsible
for the diffusion of new information to those who are homeless via a mix of
government funded organisations and grassroots organisations (Woelfer et al
2009).

There are many opportunities to support existing service providers by developing

technology that facilitates online help-seeking activities. The new means of
accessing information can often be met with resistance by existing service
providers, though. One study by Woelfer et al (2011) suggests that we should
take a precautionary stance when it comes to providing access to complex
service information online and has even suggested that new technology may not
be an appropriate solution. This is because most information for homeless people
is currently exchanged in face-to-face situations (Hersberger et al 2013; Le
Dantec et al 2008). The traditional way of interacting with these services allows
service providers to tailor advice and recommendations on a case by case basis.
It also allows service providers to retain control over what information is provided
to each person. This control is often preferred by service providers since the new
means of accessing information creates new expectations upon the service
providers. Such new expectations may be hard for the service providers to meet
(Le Dantec et al 2008; Weise et al 2017). Work by Hersberger et al (2013) and
Chatman et al (1996) suggests that those who are homeless are already
overwhelmed by information provided by services and are unlikely to own a
smartphone to access this information.

43
Those experiencing homelessness represent a unique user group. The major
causes of homelessness according to the Australian Bureau of Statistics include
family violence or financial difficulties. In fact, only 6% of people who are
considered homeless are characterised as rough sleeping; many live in severely
overcrowded homes or transitional accommodation. Additional problems
accumulate over time, such as drug and alcohol abuse, creating a viscous cycle
and a worsening situation (Woelfer et al 2009).

Despite the scepticism around communicating complex service information

online, there is a counter body of evidence that suggests it would be beneficial to
many. A mobile phone is often viewed as a necessity and a critical lifeline (Le
Dantec et al 2008). A more recent study estimated most people experiencing
homelessness prioritise retaining their mobile phone to keep in touch with family,
friends and necessary service providers (Humphry 2014). Even those without a
smart phone may have access to the internet via alternative means, such as the
library or with the help of case workers. Work by Woelfer et al (2012) indicates
that the use of social media by young homeless people is extensive and that
younger adults utilise numerous online information seeking strategies.

2. Method
2.1 An Emotion-led Approach
People will reject technology if it does not address their emotional concerns (Dix
et al 2003; Krumbholz et al 2000; Norman 2005; Wood et al 2006; Pedell et al
2014; Miller et al 2015). For this reason, there is a growing body of work that calls
for an increased focus on information about the emotions of stakeholders to
improve the design and evaluation of new technology. Emotional experiences
may be related to aspects of the software design, such as a particular feature that
is displayed. Work on socio-materiality and technology affordances (Orlikowski
et al 2008; Vaat et al 2013; Leonardi 2013) shows how aspects of design can
trigger positive and negative emotional perceptions. Emotional experiences
associated with a particular technology are also influenced by external factors,
including other individuals or organisations that are associated with the
engagement experience. Misplaced expectations may still be attributed to the
technology itself due to multiple experiences becoming aggregated and
associated with each other (Wood et al 2006). These experiences could include
those occurring during the progression towards a goal (Clore et al 2008; Luce et
al 2001). Consequently, in our case study, participants may interact with a service
provider and their reaction is attributed to the technology that facilitated the
interactions.

While there are a variety of popular psychological frameworks that characterise

emotions, their content and utility for system design and evaluation will vary. In
fact, the current range of emotional concerns that are considered has already
been criticised as being ‘narrow’ (Dunne et al 2001). Some psychological
frameworks are grounded in primary (also referred to as basic) emotions such as
fear, anger, or joy (Ekman et al (1992); Schwarz and Clore (1983)). Other
emotional frameworks contain different types of emotions, including those that
are more reflective. For instance, some emotions are characterised by having

44
relatively lower levels of arousal and involve relatively higher levels of reflective,
cognitive processes; examples include the characterisation of shame and
resentment (Martin and Tesser (1996); Desmet and Hekkert (2007); Plutchik
(2003)). Due to the sensitive nature of this domain, we utilise an emotion-led
approach. This means we characterise results according to the way stakeholders
wish to feel when using technology and place emphasis on emotional concerns
when analysing results. Similar to work by Deng et al (2016), we do not rely on a
single classification or framework to interpret our results. This is because it is
important to inquire about the wide ranging and often subtle feelings and
perspectives of stakeholders that may not be accurately represented by a
generalisable psychological framework.

Our research has contributed to the design and evaluation of a web app for
homelessness. In this paper, we present a mixed-method study that investigates
the challenges for developing technology for homelessness. The quantitative
phase provides an overview of technology use collected from all users, spanning
the two years since deployment. Quantitative data enabled us to discover usage
trends over time. The qualitative phase is centred around 30 in-depth semi-
structured interviews that were conducted with a range of stakeholders. Seven
themes emerged from the thematic analysis as a basis for further discussion.

2.2 The Web App

The web app aims to tackle the problem of homelessness by assisting those who
are homeless in finding useful information. It provides information from over
350,000 services in Australia. The listed services range from help with health
issues, food, shelter through to legal and financial advice. Figure 1 shows three
example screenshots. Our recent work has already focused on evaluating
specific design decisions (Burrows et al 2019a 2019b). Differently, this paper
takes a different perspective to understand the impact and role that Ask Izzy has
as part of a broader socio-technical system.

A typical use involves the browse/search page. A user is presented with 16 help
categories. The user can choose to give their location and is guided through a
series of category-specific questions. Based on these answers, the user is
presented with a service list compiled via a service filter process, detailing results
of services that match their criteria - and potentially ordered by relevance.

45
 Figure 1. Screenshots of second version of Ask Izzy.

A user can select a particular service to view its detailed service page. The
detailed service page displays information about how to connect with the
particular service, how to get there, who it is for, and what clients should expect.
2.3 Data Collection and Analysis

46
Usage data was collected over a period of two years. We also collected data on
how often users returned to the web app. An overview of the number of people
that used the app is shown in Table 2. These time windows were selected in order
to capture data that was representative of the normal use while spanning enough
time to account for seasonal variations. A key concern for this system was that
the people would cease to continue to use it in the long term. We therefore
capture usage data over a longer-term. For instance, demand for services
typically spikes in winter and around New Year (Australian summer). The
returning behaviour for a user captures the number of days since their last
session. For this finer-grained analysis, we present aggregated data based on
percentage change to preserve anonymity. Returning behaviour is compared by
taking four 12-week time windows and then analysing how often users returned
to the web app for seven days after the initial session. A session starts when a
user engages with the web app and ends after the default of 30 mins of inactivity.

Table 1. Interview Participants

Stakeholder
Number Sample Coverage
group
Adult, Youth, Family Violence, Veteran,
Homeless and
14 Mental or Emotional Difficulties, Drugs and
Ex-Homeless
Alcohol Problems, With Children

Service Official Service Providers including Government

15
Providers funded providers and Charities.
Software
1 A representative from the software company
Owner

Two researchers conducted a series of semi-structured, one-hour interviews with

30 participants, six months after the deployment of the web app. We took care to
ensure the recruitment procedures and interview locations were appropriate;
participants were recruited via existing service providers who were also able to
provide a familiar environment for the discussion to take place. Participants who
were homeless had some experience with the app. This ranged from a single use
to frequent use over the 6-month period. Service providers were aware of the
web app and therefore played a role in raising awareness with the homeless
people with whom they were in contact. Table 1 gives an overview of the
participants and different stakeholder groups that were interviewed. Participants
were selected to represent a range of people who have a stake in the success of
the application and who have had first- hand experience with the web app. This
included those who were homeless, ex-homeless, service providers, and the
software company.

A semi-structured interview was chosen to give flexibility to the conversation. It

allowed participants to diverge and discuss contextual factors that may be
unexpectedly related to their perceptions of the web app. With regard to the

47
software design, we asked what they liked, disliked, and what they would change
in the mobile app. We also asked how using it made them feel. We discussed
interactions and experiences outside of the application including how they heard
about it and if they had recommended it or supported others in using it. We then
asked what they thought were the barriers to uptake. If they chose not to use the
web app, we asked for the reason.

The qualitative data comprised of audio files recorded from the semi-structured
interviews. All transcripts and audio recordings were imported into the NVivo tool
(Bazeley 2013). The results were analysed by two authors following a thematic
data analysis process (Braun 2006) in order to identify, analyse, and report the
themes from the data. Codes (i.e. quotes) were extracted from the transcripts that
were related to emotional experiences or concerns. The codes of the transcripts
were grouped individually into themes and then later discussed and merged to
form a final agreed set of themes.

Following guidelines for thematic analysis, and similar to comparable approaches

(e.g. Deng 2016), no specific framework of emotions was used to categorise the
elicited codes. Any text phrases that were thought of as representing the way the
participant would or would not like to feel were marked and extracted. We
retained information about the role of each participant in order to contrast views
of service providers with those who are homeless or ex- homeless. Each theme
we analysed according to different stakeholder groups.

3 Results
We now describe key findings from the quantitative and qualitative phases of the
study. The subsequent section discusses implications of these findings.

3.1 Quantitative
Data about the numbers of users is shown in Table 2. This gives an overview of
the numbers of users that were visiting in a two-year period. Each row shows the
weekly mean, median, standard deviation, range, minimum and maximum
number of users.

Figure 2 shows information about the number of users from four different time
windows over the entire two-year data collection period. Figure 3 shows two
charts that capture returning behaviour.

Table 2. Descriptive Statistics

48
 Year 1 Year 2
Weekly
number of number
Measure
users of users
Mean 817 2287
Median 836 2328
Std. Dev. 219 622
Range 895 2557
Min. 473 1046
Max. 1368 3603
Total 42464 118949

Returning behaviour is a popular measure to understand how often users

repeatedly visit the app. There is no ‘ideal’ time period, however by looking at the
patterns and changes over time it is possible to see how using the app is used
sporadically, routinely, or how new features that are designed and implemented
change the patterns of engagement. To give an example, let’s consider the chart
on the right. We can see that, in time window 4, the number of users that returned
to the web app after 7 days has increased by 800% when compared to the 1st
time window. Time window 1 is used as a baseline for comparison and therefore
not shown on the charts (as it would appear as 0%). If a user has more than one
session in the same day, the extra sessions are counted as ‘0 Days’.

Figure 2. Overview of User Numbers

49
 Figure 3. Percentage Change in Number of Sessions.

The chart on the left is showing users that belong to an active user base (multi-
user sessions) whereas the chart on the right is showing all users.

New users are also counted as ‘0 Days’; and for this reason it is popular practice
to report multi-session users1 separately. A focus on multi-session users can
therefore be seen in the left-hand chart in Figure 3. Multi-session users are those
who have had more than one session within a particular time window and are
considered to represent an active user base. For more information about we refer
the reader to google analytics documentation3.

3.2 Qualitative
Table 3 describes seven themes that emerged from analysis. They represent
barriers or enablers to the uptake and use of the app. Each theme is represented
and articulated as a positive goal to be addressed. These goals therefore can be
interpreted as enablers of success if they are adequately addressed, or, barriers
if they are not. After reading the transcripts, 107 codes were extracted and
subsequently grouped into the 7 themes to represent how users would like to feel.
These themes were Empowerment and Control, Assurance, Cared For, Identity
and Belonging, Clarity, Unashamed / Without Stigma and Hopeful. We give a
brief description of these themes in this section and their implications are
discussed in the following section.

Figure 4 visualises the key groups of people who are either directly or indirectly
influential in the success of Ask Izzy. The figure depicts system entities who are
either individuals, organisations or communities in this socio-technical system. It
depicts the collaboration and the communication between these entities
(represented by the arrows). Finally, it depicts the technology engagement.
Technology engagement is also integral to the flow of information as we are able
to monitor and react to user engagement from in-app measures.

3
Google Analytics: https://analytics.google.com/analytics/web/

50
 Table 3. Emergent Themes

Number of
Codes Description

New means of accessing information empowers

information-seekers and increased choice provides a
Empowerment greater amount of control over how they interact with
and Control 23
service providers.

Assurance is a theme related to the levels of trust and

confidence in information received via the system or a
Assurance 16 person.

If participants sawevidence that individuals were

working to provide them with help then this provided
Cared For 5 them with a sense that they were cared for and cared
about.

Visual imagery within the application was important to

Identity and signal that this application was created inclusively for the
Belonging 12 cultural needs of multiple user groups.

This theme relates to the need to present relevant

Clarity 16 information without overwhelming users.

Unashamed The stigma around being homeless reduces the

and Without 13 motivation of people to seek help.
Stigma
Hopefulness encourages further help-seeking behaviour.
It is also important to be realistic about the extent and
Hopefulness 22 existing service provider can help.

4 Discussion
Overall, the project had a positive impact and was used by many homeless
people to seek help. The quantitative findings indicate that overall use and the
percentage of returning users are increasing. The average number of users in the
first year of deployment was 817, this increased to 2287 in the second year each
week. Over 10,000 users are now accessing the web app each month which is
evidence that a large proportion of people are choosing to seek information online
via the web app. In this section we discuss the barriers and enablers of success
in this project and reflect on the implications with respect to the living lab
methodology. We break down the discussion into three sections. Firstly, we
discuss how the themes were associated with practices and protocols regarding

51
how members of the public interacted with service providers. Secondly, we
discuss how these themes provided insights on how to maintain momentum and
awareness of the system. Finally, we illustrate how themes were associated with
aspects of the software design.

Figure 4. Collaboration Pathways between Organisations, Communities, and Individuals

4.1 Organisational Implications: New ways of discovering services

The system forged a new means for those who are homeless to discover and
interact with service providers. This was seen as empowering by those who are
homeless. Differently, service providers, were concerned about the lack of control
they had about how people were passed between coordinating organisations.

This challenge was further debated when designing the information that was
presented in the web app itself. On one hand, if the service list within the
application presented services which were usually only recommended in-person
(by a qualified service provider), then there was a risk that a client would attempt
to access a service that was inappropriate to their situation. On the other hand, if
these services were removed from the list, then the reduction in options would
limit the choice and control for the client with regard to the ways of interacting and
accessing services.

If we consider Figure 4, it is possible to understand how two key information flow

links could easily break. Service providers had great influence on adoption
success as they would cease to recommend the web app if they perceived the
presented information as risky. Similarly, clients recommending to other clients

52
were at risk of ceasing to recommend the app if they did not feel their needs were
addressed.

One service provider said:

“we like to think that information only exists in hard copy form and that’s
been a challenge we’ve had for a long period of time. But even in a meeting
we had recently the number of people that identified the [web app] and
how much they use it and how useful it is”

Results indicate that a large number of people who are homeless chose to find
service information online. One person who used to be homeless stated:

“I would go to internet cafes to research information on homelessness and

where I can get help from. I had no idea, I had no idea”

It is possible to see that the number of users who returned to the app also
increased over time. Further barriers were removed as the software company
provided free battery packs so that those who are homeless could charge their
phones and access the website for extended periods of time. Additionally, the
website was unmetered which meant that accessing the website did not cost any
money.

4.2 Maintaining Momentum

Projects like Ask Izzy rely on the continued sharing of information in order to
maintain user engagement momentum. This is because new people are
constantly becoming homeless, or finding a home. Similarly, service providers
often undergo organisational changes, meaning new people take on the
recommending roles. These are just two examples of how the people who could
recommend or benefit from Ask Izzy are constantly changing. Communication
pathways need to be maintained so that those who are homeless are aware and
made aware of Ask Izzy.

If service providers do not feel assured that the data is accurate, or if they feel
that the amount of control that is given to clients may have adverse
consequences, then they may choose not to recommend the application. Even if
service providers are willing to promote the web app, there is still uncertainty
around who will be responsible for ensuring that this awareness is maintained.

One service provider stated:

“I guess your question was who should be responsible for it, I mean I guess
it goes to who owns [the app] and I would think that the perception [is that

53
 the software company] owns it, so it’s their thing. And that’s problematic in
terms of ongoing funding”

The participants in our study, including service providers and those who had
experienced homeless outlined that a broader group of individuals and
organisations could contribute to maintaining awareness of Ask Izzy. Based on
results from interviews, these people could be: (1) service providers and case
workers who are client-facing, (2) prominent members of Aboriginal and Torres
Strait Islander communities (e.g. Elders), and (3) the general public. In many
instances, the interviewees suggested increasing awareness in hospitals, police,
and public transport workers as these groups frequently come into contact with
people needing help in their day to day work.

A participant who had been previously homeless said:

“it would be good to do some advertising on the Metro or trams or

something, some advertising, when people are sitting on the train or the
tram and they’re approached by people who are asking for money”

Peer-to-peer recommendation of the application was often described as trusted

source of information. People felt assured and cared for and trusted the advice
of others in a similar situation. Consequently, awareness of the web app was
effectively communicated socially at events.

“some of the services or they go and have a meal there or catch up with
people they know … So there was a lot of this talk bubbling around and I
think that’s for other people that were homeless it tends to be quite
effective”

In this quote the participant was describing how the tips and strategies from other
people who had a lived experience of homelessness were effective. In these
cases, certain services, such as food vans became locations where information
was routinely exchanged.

Interestingly, there is an increasing number of users who return to the web app
after 7 days. This may be explained by people accessing the web app after
hearing about it via peer-to-peer recommendation at a repeating gathering,
meeting or event. There is a possibility that an event or activity that occurs weekly
triggers motivation to use the web app. In fact, quite often an expert user had
initiated interaction to help another person access a particular service. Young
adults often asked questions to their case worker whose hands-on guidance
would take the stress away from searching for appropriate courses of action. One
participant who used to be homeless explained how he often helped others who
didn’t have access to a smart phone and WIFI. Interactions and raising
awareness about the web app often occurred during activities that were social.
By understanding how multiple stakeholders are able to collaborate we are able
to understand the ongoing success of Ask Izzy as dynamic and constantly
changing.

54
4.3 Reconciling Diverse Viewpoints and Trade-offs in Software Design
The themes elicited from the interviews were also related to aspects of the
software design. In this application domain, the interaction with the web app is
one small part of a much longer help-seeking journey. However, it is important to
consider how the initial interaction with the web app creates expectations that
may have consequences on future actions. If the expectation created in design
is not realised by the service provider then this may have negative consequences
on both service providers and the success of the web app itself.

As an example, consider the theme of hopefulness. In this circumstance, the

overriding expectations and hope of finding what is needed (i.e. a home) can vary
greatly. One service provider stated that they “spend a lot of time managing client
expectations”. Consequently, avoiding disappointment remains an ongoing
challenge. The engagement with the web app becomes a trigger where hope is
created. The expectation that is initially created is then acted upon and then
changes with each subsequent interaction with service providers. For many
individuals experiencing homelessness will be a number of years. Therefore, the
more this need is addressed in design (the greater the initial hope), the greater
the risk of negative consequences in the long-term (expectations not met). One
service provider stated that “you’re actually setting the expectation that there’s
hope there when actually that’s misplaced”.

Someone who used to be homeless recalled her experience after finding a

service advertised to only find that the service was not able to help:

“It’s so depressing when you’re trying to find accommodation and you read
something and say oh my god, they’ve got support to transitional
accommodation. I remember this happened with me when I was homeless
and I’d get really excited, oh great, I’ve found a place, maybe they can help
me, hang up after hang up, after hang up.”

In this sense, one of the greatest challenges for technology design is to provide
clarity and hope about what can be achieved by seeking help without reducing
motivation to seek help.

Again, taking a holistic approach allowed technology developers to consider the

consequences that specific design features would have on future actions and
interactions with service providers.

5 Conclusion
This paper investigates the key challenges of developing new technology to help
those who are homeless. We believe the lessons from our project may be useful
to others who are creating new technology for socially-complex projects. We
conducted a mixed-method study based on the development of a web app. We
contrast the different perspectives of people who were homeless, ex-homeless,
service providers and a representative from the software development company.
We found that an emotion-led approach was useful to understand stakeholder
concerns in the context of the broader socio-technical system. People who are

55
homeless wanted to feel empowered, in control, assured, cared for, unashamed
and hopeful while interacting with the web app. In particular, we discuss the
technology design and the impact it had on existing organisational practices and
protocols. We also discuss the challenge of maintaining momentum and
awareness of the web app. We explain the importance of being mindful so as not
to raise unrealistic expectations. Finally, we give an example of how our approach
was used to inform design trade-offs.

Acknowledgements
This research has been made possible by the support and collaboration with
Infoxchange a not-for-profit social enterprise developing technology for social
change. This project was funded by the Australian Research Council Discovery
Grant DP160104083 “Catering for individuals' emotions in technology
development”.

References
Alatawi, E., Mendoza, A., and Miller, T. 2018. “Psychologically-Driven
Requirements Engineering: A Case Study in Depression Care,” In 25th
Australasian Software Engineering Conference (ASWEC), IEEE.
Ask Izzy Website. “https://askizzy.org.au/”: accessed 15/01/2019.
Bazeley, P., and Jackson, K. 2013. Qualitative Data Analysis with NVivo, Sage
Publications. Burrows, R., Mendoza, A., Sterling, L., Miller, T., Pedell, S.
2019. Evaluating Ask Izzy: A Mobile
Web App for People Experiencing Homelessness. In: Proceedings of the 17th
European Conference on Computer-Supported Cooperative Work.
Burrows, R., Lopez-Lorca, A., Sterling, L.; Miller, T., Mendoza, A.; Pedell, S.
2019. Motivational Modelling in Software for Homelessness: Lessons from
an Industrial Study. In Proceedings of the 27th IEEE International
Conference on Requirements Engineering.
Chamberlain, C., Johnson, G., and Robinson, C. 2014. Homelessness in
Australia, UNSW Press. Chatman, E. A. 1996. “The impoverished life-
world of outsiders,” Journal of the American
Society for information science (47:3), pp. 193–206.
Clore, G. L., and Ortony, A. 2008. “Appraisal theories: How cognition shapes
affect into emotion.”
Deng X. N., Joshi K. D., and Galliers R. D., 2016. “The duality of empowerment
and marginalization in microtask crowdsourcing: giving voice to the less
powerful through value sensitive design” MIS Q. 40, 2 (June 2016), 279-
302.
Desmet, P., and Hekkert, P. 2007. “Framework of product experience,”
International Journal of Design (1:1).
Dix, A., Finlay, J. E., Abowd, G. D., and Beale, R. 2003. Human-Computer
Interaction (3rd Edition), Prentice-Hall, Inc.

56
Ekman, P. 1992. “An argument for basic emotions,” Cognition & Emotion (6:3-4),
pp.169–200.
Griffiths, P. E., and Scarantino, A. 2005. “Emotions in the wild: The situated
perspective on emotion,”.
Hersberger, J. 2003. “A qualitative approach to examining information transfer
via social networks among homeless populations,” The New Review of
Information Behaviour Research (4:1), pp. 95–108.
Hersberger, J. A. 2013. “Are the economically poor information poor? Does the
digital divide affect the homeless and access to information?” in
Proceedings of the Annual Conference of CAIS/Actes du congrès annuel
de l’ACSI.
Hou, Y., Lampe, C., Bulinski, M., and Prescott, J. J. 2017. “Factors in Fairness
and Emotion in Online Case Resolution Systems,” in Proceedings of the
CHI Conference on Human Factors in Computing Systems, ACM.
Humphry, J. 2014. Homeless and Connected: Mobile phones and the Internet in
the lives of homeless Australians.
Krumbholz, M. a., Galliers, J., Coulianos, N., and Maiden, N. 2000. “Implementing
enterprise resource planning packages in different corporate and national
cultures,” Journal of Information Technology (15:4), pp. 267–279.
Le Dantec, C. A., and Edwards, W. K. 2008. “Designs on dignity: perceptions of
technology among the homeless,” in Proceedings of the SIGCHI
conference on human factors in computing systems, ACM.
Leonardi, P. M. 2013. “Theoretical foundations for the study of sociomateriality,”
Information and Organization (23:2), pp. 59–76.
Lowry, P. B., Gaskin, J., Twyman, N., Hammer, B., and Roberts, T. 2012. “Taking
fun and Game seriously: Proposing the hedonic-motivation system
adoption model (HMSAM)”.
Luce, M. F., Bettman, J. R., and Payne, J. W. 2001. “Emotional decisions:
Tradeoff difficulty and coping in consumer choice,” Monographs of the
Journal of Consumer Research (1), pp. 1–209.
Majchrzak, A., Faraj, S., Kane, G. C., and Azad, B. 2013. “The Contradictory
Influence of Social Media Affordances on Online Communal Knowledge
Sharing,” Journal of Computer- Mediated Communication (19:1), pp. 38–
55.
Massimi, M., Dimond, J. P., and Le Dantec, C. A. 2012. “Finding a new normal:
the role of technology in life disruptions,” in Proc. of the ACM Conference
on Computer-Supported Cooperative Work, ACM.
Miller, T., Pedell, S., Lopez-Lorca, A. A., Mendoza, A., Sterling, L., and Keirnan,
A. 2015. “Emotion-led modelling for people-oriented requirements
engineering: the case study of emergency systems,” Journal of Systems
and Software (105), pp. 54–71.

57
Muñoz, M., Vázquez, C., and Vázquez, J. J. 2004. “A comparison between
homeless, domiciled and vulnerable populations in Madrid,” Population
(59:1), pp. 129–142.
Norman, D. A. 2005. Emotional design: Why we love (or hate) everyday things,
Basic books. “Census of Population and Housing: Estimating homeless-
ness, 2016,” http://www.abs.gov.au/ausstats/abs@.nsf/mf/2049.0.
Orlikowski, W. J., and Scott, S. V. 2008. “10 Sociomateriality: Challenging the
Separation of Technology, Work and Organization,” Academy of
Management Annals (2:1), pp. 433–474.
Pedell, S., Keirnan, A., Priday, G., Miller, T., Mendoza, A., Lopez-Lorca, A., and
Sterling, L. 2017. “Methods for Supporting Older Users in Communicating
Their Emotions at Different Phases of a Living Lab Project”.
Pedell, S., Miller, T., Vetere, F., Sterling, L., and Howard, S. 2014. “Socially-
Oriented Requirements Engineering: Software Engineering Meets
Ethnography,” in Perspectives on Culture and Agent-based Simulations,
Springer, pp. 191–210.
Plutchik, R. 2003. Emotions and life: Perspectives from psychology, biology, and
evolution. American Psychological Association.
Saffarizadeh, K., Boodraj, M., and Alashoor, T. M. 2017. “Conversational
Assistants: Investigating Privacy Concerns, Trust, and Self-Disclosure,”.
Schwarz, N., and Clore, G. L. 1983. “Mood, misattribution, and judgments of well-
being: Informative and directive functions of affective states.” Journal of
personality and social psychology (45:3), p. 513.
Toscos, T., Connelly, K., and Rogers, Y. 2013. “Designing for positive health
affect: Decoupling negative emotion and health monitoring technologies,”
in Pervasive Computing Technologies for Healthcare (Pervasive Health).
IEEE.
Vaast, E., and Kaganer, E. 2013. “Social media affordances and governance in
the workplace: An examination of organizational policies,” Journal of
Computer-Mediated Communication (19:1), pp. 78–101.
Vines, J., McNaney, R., Clarke, R., Lindsay, S.,McCarthy, J., Howard, S.,
Romero,M., and Wallace, J. 2013. “Designing for -and with- vulnerable
people,” in CHI’13 Extended Abstracts on Human Factors in Computing
Systems, ACM.
Website, Homelessness Australia 2018. “/www.homelessnessaustralia.org.au:
accessed 15/01/2019”.
Weise, S., Coulton, P., and Chiasson, M. 2017. “Designing in between Local
Government and the Public–Using Institutional Analysis in Interventions
on Civic Infrastructures,” Computer Supported Cooperative Work (CSCW)
(26:4-6), pp. 927–958.
Woelfer, J. P., and Hendry, D. G. 2009. “Stabilizing homeless young people with
information and place,” Journal of the American Society for Information
Science and Technology (60:11), pp. 2300–2312.

58
Woelfer, J. P., and Hendry, D. G. 2011. “Designing ubiquitous information
systems for a community of homeless young people: precaution and a way
forward,” Personal and Ubiquitous Computing (15:6), pp. 565–573.
Woelfer, J. P., and Hendry, D. G. 2012. “Homeless Young People on Social
Network Sites,” in Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems, CHI ’12, New York, NY, USA: ACM.
Woelfer, J. P., Iverson, A., Hendry, D. G., Friedman, B., and Gill, B. T. 2011.
“Improving the safety of homeless young people with mobile phones:
values, form and function,” in Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, ACM.
Wood, S. L., and Moreau, C. P. 2006. “From fear to loathing? How emotion
influences the evaluation and early use of innovations,” Journal of
Marketing (70:3), pp. 44–5.

59
60
 Living Lab Activities for Social Problem Solving
R&D Projects in Korea: Achievements and
Challenges from Case Studies
Jieun Seong*1, and Ji In Park2

*Corresponding author
1 Science and Technology Policy Institute (STEPI), South Korea
2 MSIT (Ministry of Science and ICT), South Korea

Category: Full Research

Topic: Theoretical &

Methodological Challenges

Abstract
The purpose of this study is to investigate the role of “living lab” activities in social
problem- solving R&D projects conducted in Korea and to derive its
achievements and tasks. This study analysed the representative cases in which
living lab activities are prominent among social problem-solving R&D projects in
Korea. The analysed cases are ‘develop portable fundus camera for eye disease
screening test to resolve health inequalities’ and ‘auto-sensing integrated system
development in rural pedestrian crosswalk’. Through this study, we identify the
contents of the living lab activities of these social problem- solving R&D projects
and present policy issues. The characteristics derived from the analysis are as
follows: 1) living labs are being introduced as a methodology for user and
demand-oriented research innovation in Korean R&D projects. 2) these projects
conduct new policy experiments that try to overcome the limitations of the Korean
innovation system, such as top-down approach in policy making led by central
government; R&D planning focused on technology providers’ convenience; and
industrial innovation emphasizing economic growth.

Keywords: Korean living lab, R&D innovation model, social problem-solving

R&D project, R&SD (Research & Solution Development), Case Studies,
Achievements and Challenges

61
1 Introduction

“Living labs” have been introduced and implemented in recent years in Korea
both as innovation models led by social actors (e.g., local communities and users)
and as venues for field-based innovation.

Not only the central government, such as the Ministry of Science and ICT (MSIT,
renamed from Ministry of Science, ICT and Future Planning (MSIP) in 2017), but
also local municipalities and other entities are adopting living lab projects for a
variety of efforts including the development of products and services,
establishment of public infrastructure, reform of local communities and tackling of
local problems. For instance, the “Social Problem-solving R&D Project” led by the
MSIT and the “Energy Technology Acceptability Enhancing and
Commercialization Promotion Project” led by the Ministry of Trade, Industry and
Energy (MOTIE) are efforts to improve field applicability and demand-oriented
ness through living lab approaches. Similarly, a variety of IoT living lab projects
led by municipalities and public institutes, such as the BukchonHanok Village IoT
living lab in Seoul and the Seongnam Senior Experience Center’s senior living
lab, are examples of other efforts to apply modern technology, such as IoT, to
enhance public services and solve local problems (Seong, Han & Park, 2016).

This study aims to analyze samples cases of living labs in social problem-solving
R&D projects that are implemented in such a way as to promote innovation in
national R&D projects. To this end, we looked at case studies involving the
development of “fundus camera technology” and “automated pedestrian
detection system” (in particular, the general overview, objectives, planning and
management systems, achievements and challenges of such projects) and were
able to draw policy suggestions. As for research methods, in-depth interviews
with major participants were conducted along with literature reviews of policy
reports, academic papers, newspaper articles, and other sources.

2 Theoretical background

2.1 Innovation Models in Korea & Living Labs

Innovation models in Korea thus far have been based on the linear model.
Embodied in this model is the perspective that innovation will occur when R&D
results from research labs become commercialized. Therefore, in this model,
R&D activities themselves constitute the critical source of innovation (Seong &
Song, 2007). Traditional ICT policies, largely built upon TDX, CDMA and other
cutting-edge technologies, are also based on this linear model, wherein users are
assigned the role of merely informing tech-providing companies about their needs
or serve as passive bystanders who adapt themselves to developed products
(Song and Seong, 2013).

However, such linear models focusing on R&D have gradually come up against
their limits. Structural limitations imposed by an ageing society, fierce competition
from emerging economies, coupled with stagnant economic growth and other

62
factors, have made it difficult to steadily expand human and financial input,
thereby reducing the efficacy of traditional input-driven innovation models (Song
and Seong, 2013).

A living lab is a user-driven innovation arena where users function as actors

rather than objects of research innovation and activity, and the significance of a
living lab lies in that it is an open innovation model led by end-users who actively
engage in problem solving. As such, living lab activities are most prevalent in
areas related to everyday living, such as energy, housing, transportation,
education, and health care, where user experience and intuition matter the most.

Recently, Korea has seen a wide variety of actors (e.g., the central government,
municipalities, and social economy organizations) considering living labs as an
alternative model for technological and social innovation. MSIT, the Ministry of
Health and Welfare (MOHW), and other government agencies are examining
living labs as a novel way to induce social problem-solving innovations that utilize
ICT and other technologies. On the other hand, the municipal governments of
Seoul, Daejeon and other cities are beginning to see living labs as new social
innovation models distinguished from conventional approaches. Finally, public
welfare institutions (e.g., nursing homes and hospitals) and social economy
organizations are paying attention to living labs as experiments for realizing
welfare goals and improving welfare service delivery systems.

2,2 Introduction of Living Labs as Implementation Mechanisms for National

R&D Projects
The MSIT’s social problem-solving R&D project, a national R&D initiative, is a
new approach to tackle societal challenges such as the diminishing quality of life
and social polarization (MSIP, 2016). Unlike traditional R&D projects that focus
on the provision of technology and industrial innovation, this project pursues
values such as socially responsible technology as responses to social demands,
while integrating technology, law, organizations, culture, and other fields to solve
problems facing end- users (Song & Jeong, 2016).

However, more than anything, the project fosters open-innovation activities,

wherein the introduction of a living lab approach allows end-users and
researchers to work together to develop, substantiate and evaluate products in
actual living spaces. As such, this project emphasizes civic participation, and
based on that, explores social issues closely related to everyday living.
Furthermore, by encouraging participation by social innovation organizations to
effectively deliver developed products and services, the project seeks to raise
their efficacy.

63
 Table 1. The Characteristics of Korea’s Social Problem-solving R&D Projects

AS-IS technology TO-BE social problem-solving

acquisition program
Growth-based, Human-centric, pursuing better
focusing on national quality of life as well as
economic development economic development
Objective
R&D, R&BD → R&SD
(Research & Solution Development)
Acquire scientific and
Primary objective technological Solve social problems
competitiveness
· Problem -solving convergence*
· Technological
*technology + humanities and
convergence
Characteristics social science + regulation
· Provider -centric R&D
· Recipient -centric R&D
Research division- Research division and policy
Planning
centric division cooperation-centric
R&D progression- Problem-solving and change
centric management management
Management
(program manager) (solution consultant)
Characte- Extent of social problem
ristics by · Research results such
resolution through the
stage as papers or patents.
production and delivery of
Evaluation · Verification of research products and services or by
results, dissemination systemic transition
Exploration of social problems
Technological
Main impetus and systemization of
development
service delivery

The ‘social problem-solving R&D project’ is being implemented across three

sectors: living environment, disaster safety, and social gap reduction. The living
environment sector focuses on everyday-living problems facing citizens—in
particular, health and environmental issues. The disaster safety sector, on the
other hand, deals mainly with disasters (e.g., toxic gas release and explosion
accident) that can occur at the regional and national levels and seeks to develop
products and services that can tackle these problems. Finally, the social gap
reduction sector centers on socioeconomic gaps that are apparent in everyday
living, seeking to develop low-priced quality products and services to help protect
the underprivileged.

Those projects introduced living-lab methodology as a new way of R&D and tried
to promote open innovation activities in which end users and researchers jointly
develop, demonstrate and evaluate products in real life spaces. The Living Lab
is an infrastructure that enables professionals and end users to continuously
improve their products, services, and demonstrations with customer interactions.

64
 Table 2. List of Korea’s Social Problem-solving R&D Projects

Source: Seong et al. (2016). Revised.

The purpose of this study is to investigate the living lab activities in the social
problem-solving R&D projects conducted in Korea and to derive its achievements
and tasks.

Figure 1. Living Lab as methodology of ‘Social problem-solving R&D Project / Source:

MSIP (2016)

65
3 Case Studies on Living Labs in the ‘Social Problem-solving
R&D Project’

3.1 Case Study on the R&D of a Portable Fundus Camera

3.1.1 R&D goals
This project aims to develop and distribute portable fundus cameras that can be
used for early diagnosis of ocular fundus lesions occurring in people without
proper medical care. According to The Korean Retina Society (KRC), the number
of patients with age-related major retinal diseases categorized into four types
rose from 382,247 in 2009 to 510,413 in 2013, a 35.1% increase over five
years(Health Chosun, 2012). The primary cause of blindness among these
patients is diabetic retinopathy, which can be prevented by early diagnosis
(Seong et al., 2016: 196). In particular, medically vulnerable demographic groups
(e.g., seniors, those with mobility problems, and residents of remote areas) are
in ever greater need of visiting/portable fundus examinations with retinography.

As such, the research team is focusing on developing a portable design and

intuitive GUI software to achieve convenience. To realize this goal, the team is
operating a user-driven living lab to apply the outcome of interactions among a
variety of actors in subsequent product development.

In operating the living lab for this fundus camera project, a wide variety of
stakeholders participated, including technology developers; experts in living lab
management, validation and authorization; and organizations related to
commercialization and distribution. The living lab management team and these
participants together operated the living lab throughout the exploration and
experimentation phases. The first living lab came up with a product that reflected
the preferred functionalities of users surveyed. The second living lab, on the other
hand, mainly aimed to evaluate the performance and usability of prototype
cameras. These prototype models were constantly modified based on the pros
and cons discovered and evaluated through use in real-life settings.

3.1.2 Building a system of Living Lab

Infrastructure design
As part of preparations before the launch of the living lab, the team engaged in
various activities including the handling of legal procedures related to clinical trials
and the development of guidelines for participants. First, the project had to pass
IRB review, and the authorization for the manufacturing of medical equipment
also had to be obtained. MOUs were signed with medical cooperatives, clinics,
and university hospitals for the operation of the living lab. Next, product manuals
and questionnaires were issued. The manual was offered in three different
versions: an equipment manual, a usage manual, and a quick manual for non-
experts containing explanations on how to make a simple diagnosis.

At the same time, the questionnaires were produced for two phases: specification
design and evaluation during real-life use. Questions related to the former
included: (a) What are the usage requirements? (b) What type of assistance is
needed? (c) How should the living lab be structured? As for the second phase, a

66
three-day survey focusing on user convenience was conducted to evaluate the
proficiency obtained by repeated use of the product. Survey results were also
uploaded to a website for real-time feedback.

End –user organization

The living lab proceeded in two phases. As each phase had different
characteristics, different user groups were formed for each. The first living lab
was related to the design and exploration of product concepts, so opinions from
core users were collected to determine specifications. These core users were a
group of some 200 ophthalmologists specializing in retinal and vitreous diseases
and therefore were familiar with fundus photography as well as constituting the
real force behind the market for fundus cameras. By asking them “What
functionalities need to be added or eliminated?” in specific, detailed questions,
the team was able to determine development direction and detailed
specifications. Not only was this survey useful in exploring the needs of real users
before product development, but it also provided the extra benefit of promoting
the product to be developed.

The participants of the second set of living labs consisted of an expert group and
a non-expert group. The reason behind such classification was to differentiate the
way these living labs were operated to reflect the participants’ knowledge of
fundus cameras and their skill levels. The expert group was subdivided into small
eye clinics and large general hospitals. The non-expert group, on the other hand,
consisted of medical cooperatives. Although the members of this latter group did
not have prior experience in fundus photography, they had a deep understanding
of the need for early diagnosis of eye diseases and telemedicine, thereby serving
as a well-organized civic group that is best aware of the purpose of product
development and that actively participates in the process. The non-expert group
played the dual role of end-users and a service delivery system for end-
beneficiaries.

All of these groups shared the fact that they were all potential buyers as end-
users. However, depending on their expertise and familiarity with related
products, each provided its own unique input on the product. The general hospital
group often had their own fundus photography equipment, so their experience
allowed them to compare the research team’s prototype with other existing
equipment on a practical level. The eye clinic group, on the other hand, consisted
of private and small-sized clinics and was regarded as comprising the majority of
potential buyers. As such, the living lab for this group acted as a market tester.
The group was also considered an appropriate test bed for evaluating the
usability of a portable fundus camera as the members of the group came into
contact with a large number of patients. The medical cooperative group, on the
other hand, were composed of the uninitiated who were able to provide fresh
perspectives and assist in adding user-friendliness to the product by participating
in the development process and the drafting of the manual.

Table 3. Organization of actors of fundus camera Living Lab

67
 Actor’s Expertise Actor’s Roles in the
Type characteristics (fundus camera- Living Lab
related skills)
Potential buyers Ample experience Evaluate photo quality
in fundus Share prior experience
photography and in fundus photography
General analysis
Hospitals Evaluate usability
Possess portable
fundus cameras /
ample experience

Expert
group
Potential buyers Ample experience Share prior experience
in fundus in fundus photography
photography
Most suitable group Evaluate marketability
for evaluating Highly skilled in and usability
Eye Clinics marketability fundus photography
as they deal with
numerous patients
Potential buyers No prior experience
Non- Provide perspectives of
expert the uninitiated
Medical Best aware of the
group necessity of early
Co-ops Offer advice on the
diagnosis and product manual
telemedicine

Source: Seong et al. (2016); Seong & Han & Jeong (2018)

3.1.2 Exploration: co-design

This phase involved an online and offline survey conducted during December
2015 involving 83 ophthalmologists with ample experience in handling fundus
cameras. 95.3% of the respondents said they were currently using fundus
cameras, while only 2.3% said they were using portable ones, indicating
significantly lower usage rates of the latter. As for reasons why they did not use
a portable version, the respondents cited high prices (30.1%), low picture quality
(29.2%) and difficulty of use. When asked a specific question, namely “What
functionalities need to be added or removed?” they chose “high resolution,”
“lightness” and “photography of the eye in a non-mydriatic state.” Based on these
responses, the team abandoned its original plan of developing a budget-priced
device using existing camera modules, and instead opted for developing its own
unique product (Table 1). Reflecting the feedback from these experts, the team
adjusted its target price range and developed a prototype that features a 1-inch
CCD and a dual-light source that combined IR LED and white LED modules.

3.1.4 Experimentation: co-production

This phase was conducted to evaluate the performance of the prototype, with
emphasis on image quality, usability, device learning curve, and differentiation
from existing cameras. The evaluation was carried out over one month during
September 2016 on the three end-user groups and 301 subjects. As for image
quality, a group of experts assessed videos taken by living lab participants. As

68
for usability, doctors, nurses, optometrists, and other professionals assessed
difficulty of use. Finally, as for the learning curve, in order to determine whether
users can easily learn how to manipulate the camera with basic training, medical
doctors not specializing in ophthalmology were repeatedly tested to evaluate their
camera use ability.

The experiments provided feedback on a variety of aspects including the level of

difficulty in using the camera, accessibility of the software, appropriateness of the
weight, comparison with existing devices, and the addition of an LED display. The
level of difficulty felt when using the camera varied greatly depending on the user
group; those with experience in using a portable fundus camera mostly felt that
the prototype was similar to existing cameras in usability, whereas those with no
prior experience found it difficult to use. On the other hand, those who are not
eye experts had to undergo much trial and error during early experiments as they
found it challenging to recognize a target point. After three days of usage, their
manipulation of the camera gradually improved, but these subjects still found the
camera more challenging to use than fixed-type cameras.

As a solution, the team is currently developing its own original display and
guidance software that provides a fixation point. Moreover, a variety of design
choices were made to enhance usability (e.g., a safety strap to prevent falls). The
feedback also indicated a need for a separate device manual for non- skilled
users, a training program, and the development of indicators to enhance image
quality. In particular, the participants raised the need for a guidebook containing
medical knowledge that would allow non-experts to make a simple diagnosis.

3.1.5 Achievements
In this case, the living lab, wherein a variety of actors participated in repeated
experimentation, acted as a catalyst in upgrading the product and making it more
user-friendly. The living lab was conducted in three phases. First, institutional and
systemic designs were devised to facilitate smooth operation of the living lab.
Next, the exploration phase driven mostly by expert groups was conducted to
determine product concepts and create prototypes. Finally, the team evaluated
usability through the living lab by expanding user groups to include small eye
clinics and medical cooperatives. In short, problems and basic concepts for
products and services were defined through interactions with core users before
determining specifics by expanding the scope of end-user groups and collecting
feedback.

Participation by different occupational groups allowed the team to collect different

feedback from groups varying in expertise and familiarity with related products.
Such varied participation aided in the development of a product that features a
user-friendly design and technology while increasing the attractiveness of the
product. Moreover, direct feedback from potential buyers (e.g., eye disease
experts, hospital professionals and medical cooperatives well aware of the need
for the product) helped to promote it and raised the commercialization possibilities
of the R&D results.
On the other hand, the learning effect from these various groups led to the
creation of a number of alternatives to cope with problems likely to arise during

69
telemedicine. For instance, a more detailed manual for uninitiated users, a
training program, and the need for developing indicators to improve image
quality— these were ideas ophthalmologists had not conceived of, and would
serve as complements that can encourage a broadening of the user base. In
addition, efforts such as developing standard fundus photograph samples and a
guidebook for non-experts served as opportunities to create services that would
allow regular users to make simple diagnoses of eye diseases.

The living lab for the development of portable fundus camera technology is an
effort to reduce disparity in healthcare access. By developing a high-quality
portable fundus camera at an affordable price, the lab aimed to expand the use
of such cameras. In addition, by lowering the technical barriers surrounding
fundus photography (which had been the exclusive domain of ophthalmologists)
and by making it accessible to ordinary citizens, the living lab transformed fundus
photography into an open service. Moreover, by running separate training
programs that can promote simultaneous distribution of product and services, the
team also strived to reduce gaps in medical care in a tangible way.

3. 2 Case Study on the Development of an Automated Pedestrian Detection

System
3.2.1 R&D goals
Unlike major cities, many rural areas in Korea have limited traffic safety
infrastructure, which puts vulnerable pedestrians (e.g., children, the elderly, and
the disabled) at a high risk of accidents at crosswalks. To prevent such risks, the
case in question aims to develop an automated pedestrian detection system that
can enhance convenience and safety for these vulnerable pedestrians.
Technically, the system aims to achieve a 99% video detection rate for
pedestrians; it will be connected to existing facilities and apply object recognition
and tracking technology in a detection zone.

To develop a commercially viable system, the R&D team worked with traffic
engineers and a company specializing in video detectors to operate a living lab
in areas with limited traffic safety infrastructure. The living lab was executed in
three phases. In the first phase, living lab sites and participants were selected
and analyzed with VISWALK, a pedestrian simulation software tool, in order to
set up a basis for comparison. In the next phase, the Living Lab Experience
Community (LEC) consisting of local residents, public servants, facility managers
and other experts was established to verify the living lab and collectively proceed
with the entire process of planning, examination, experience, application,
enhancement, and verification. In the final phase, an integrated auto-detection
system was installed and operated to conduct research on its continuous
enhancement. The feedback from the LEC and a log analysis were also
considered as basic tools for the distribution and upgrade of the system.

3.2.2 Building a system of Living Lab

Infrastructure design
Regarding the selection of living lab sites, Jeollabuk-do Province was chosen as
the optimal region as roads in the province show typical characteristics of rural
roads, posing a high risk of pedestrian traffic accidents. Moreover, the provincial

70
government expressed a strong willingness to support the living lab project and
relevant organizations in the province were well coordinated.

As for living lab participants, there were two general considerations used as
selection criteria. First, a variety of factors were considered including the
presence of facilities that can induce pedestrian traffic, opinions of local residents,
the way traffic signals are operated, and the possibility of camera-based auto-
detection. In addition, local residents informed the research team of sites with a
history of pedestrian accidents or sites that are accident-prone, allowing the team
to include bus stops, roadside residential areas, senior welfare centers,
commercial facilities, factories, churches, and schools as its living lab sites. Next,
based on this feedback, the team also considered other factors such as the
amount of pedestrian traffic, the diversity of vehicular traffic, the regularity of road
geometry, and the frequency of traffic accidents, before selecting four areas in
Jeonju City, Jeollabuk-do Province as living lab sites.

LEC Organization
In order to raise the efficacy of the system, an LEC platform was established so
that all participants could regularly share and exchange opinions. Researchers,
administrative agencies, police stations, research institutes affiliated with
universities and administrative agencies, the Korea Road Traffic Authority (which
oversees traffic systems), and local residents—all participated actively in the
planning, development, and installation process.

The LEC was classified into system developers, living lab operators, and local
resident representatives, depending on the role they played. The system
developers consisted of a group of experts involved in the development and
enhancement of the automated pedestrian detection system. The Korea Road
Traffic Authority took charge of the operation of traffic signals and system
certification. Police stations provided guidance on development direction and
precautions based on the current state of the living lab sites. Tech companies
involved examined, based on existing pedestrian detection technologies, the
overall conditions related to the operation of a network that is tailored to the local
context.

The living lab operators consisted of seven people: four field employees from
administrative agencies and three academics specializing in traffic-related
matters. The former provided advice on administrative matters related to the
installation and operation of the system and took practical measures, whereas
the latter offered opinions on the overall system operation, including system
installation, signal operation, and future planning.

Resident representatives consisted of a group of 14 volunteers capable of using

the Internet. Those interested in solving social matters and recommended by
local administrative agencies were included in this group, and other members
were selected in such a way as to ensure diversity in gender, age, and
occupation. By sharing opinions on the inconveniences and issues related to
system installation, these resident representatives helped the research team

71
handle a range of challenges. They were also responsible for promoting the
system to other local residents after installation.

Table 4. LEC Organization

Actor’s Roles in
Actor’s characteristics Participant
the Living Lab

System System developers

Secure institutional
compatibility through development Local police officers
Expert
group collaboration between system Product certification Korea Road Traffic
developers and working-level Authority
people related to laws and
institutions
Secure the effectiveness and Living lab operation Academic faculties
professionalism of the operation
Working through collaboration with System risk Administrative
group working- level traffic system management agencies
experts and professors
specializing in traffic- related
fields
Resident representatives from Living lab Local communities
four areas interested in solving participation and
End user social issues feedback
group
Possess IT literacy Promotion of the
system to local
residents

3.2.3 Exploration: co-design

As transportation-related living labs are closely related to daily safety, it is difficult
to test prototypes immediately in real situations. Thus, as a complementary
alternative, pre-simulations were conducted on four sites using the VISWALK
simulation program. The analysis showed a reduction of 70% in average traffic
delay times in three of the sites as well as the reduction of 40-50% in the average
pedestrian crossing time.

Figure 2. VISWALK simulation screen / Source: Seong et al. (2016)

72
This result confirmed that the introduction of the system would bring positive
benefits to both drivers and pedestrians. The details were shared with the LEC
by having them posted on the project website, wherein LEC members were able
to discuss operation-related issues in real-time.

Figure 3. LEC organization (top: Inhabitants briefing session, Bottom: Expert meeting) /
Source: Seong et al. (2016)

The next step involved receiving real-time feedback from experts and residents
before making a prototype. Minimum legal requirements exist for pedestrian
crossing systems. As such, the team sought cooperation for system installation
through meetings with experts while receiving demands and a variety of ideas
from the policy agency, which is the final authority, and working-level public
servants from municipal governments. In addition, to prepare for any risks that
may arise during system installation and operation, the project team collected
opinions from resident representatives and tried to apply those ideas
immediately. The team also held information sessions to explain the system
under development to them and encourage its use; conducted public hearings to
help prevent the invasion of privacy related to the installation of cameras; and
had opportunities to identify system requests.

3.2.4 Experimentation: co-production

System log analysis
The controller on the automated detection system allowed detailed records of
signal operations before and after the system installation to be compared and
analyzed, thereby enabling quantification of the effects of the installation.
However, at the present stage, as the stability of the system has not yet been
verified, a green (walk) signal had never been assigned; thus, the number of

73
pedestrians detected were used instead as an indirect alternative to the number
of walk signals assigned.

The log data for the first living lab was collected from the four sites during
September 2016. The analysis of the data indicated that in outer suburbs the
number of walk signals assigned were reduced to approximately 10-23% of the
normal figure. This means that a pedestrian crossing that assigns walk signals
720 times per day can function with signal levels roughly 10-20% of that number,
while providing both pedestrians and passing vehicles with improved safety and
time savings. It can thus be concluded that in outer suburbs, the system can
dramatically enhance safety and convenience for pedestrians. However, our
analysis suggests that in inner cities, where the amount of pedestrian traffic varies
greatly between weekdays and weekends, the efficacy of operating the system
is relatively lower. For instance, if there are an excessive number of walk signals,
that can disrupt the flow of vehicle traffic. Thus, we drew the conclusion that at
inner city sites, the options of adjusting signal intervals or flexibly operating the
system only during certain hours need to be considered.

Feedback of LEC
Above all, it was shown that the product should be improved for better pedestrian
safety. For example, some demanded audible guidance and electronic message
boards that could allow pedestrians to easily recognize whether the automated
detection system was in operation. It was also apparent that the wait time should
be flexibly controlled according to the type of pedestrian (e.g., the elderly) and
the number of pedestrians crossing.

There was also demand for enhancing the reliability of the product. To this end,
the product should be carefully designed so pedestrians can see clearly whether
it is in operation. It was also requested that the camera display the detection
range and a signal indicate whether the detection function is activated. Moreover,
video needs to be record when the system detects pedestrians in order to analyze
user behavior patterns during the living lab. However, this requires further
consideration since users may feel uncomfortable about being filmed and since
the act of filming pedestrians may also be against the Personal Information
Protection Act and privacy laws. Thus, it was necessary to stipulate that "Video
images will be deleted within one day unless there is an accident." pursuant to
the Personal Information Protection Act. In addition, the living lab management
needed to be revised in such a way as to consider the time and place in which
users actually use the system in order to accurately identify residents' behavior
patterns.

There was also a request for the enhancement of the product’s feedback system.
For example, a supplementary manual control system was needed for
unexpected accidents. Some also suggested that if the product is equipped with
real-time recording and partial storage through the application of the camera
technology as well as a vehicle control feature, it could even play a role in
reducing accidents caused by drivers running red lights. There was also a
suggestion for establishing national technical standards for the system. Although
there is currently no product based on the technology in question, it will be

74
necessary to establish technical standards for utilization and diffusion of the
system in the future.

3.2.5 Achievements
At end of the first year, the results of the living lab were as follows. The living lab
research community, LEC, was formed to build a system through which anyone
could participate in product development and installation in a variety of ways. In
particular, feedback from local resident representatives created an environment
in which users could take advantage of the product with enhanced convenience
and safety. The installation costs of the automated pedestrian detection systems
at the four sites were about 25% of the costs for traditional systems. The new
system therefore offered significant potential advantages in commercialization
and distribution. Log records confirm that the system has reduced travel time for
both pedestrians and vehicles.

When it came to site selection, this case tested the system at as many varied
crosswalks as possible. The development team adopted an approach that
gradually narrowed down crosswalk types to just one that provided the system
with the best efficiency, and this approach proved to be very appropriate.
Moreover, the living lab sites were recommended and selected by utilizing the
experience of the living lab participants. As such, the sites were situated in
context by local participants, which was not achievable through a field survey by
outside researchers.

The case of the automated pedestrian detection system development constitutes

an urban transport arena, and serves as an urban transition lab. The constant
contact and communication between new technology and local communities
develops new meaning and culture within the context of traffic technology by
inviting change in the daily behavior of civil society. In the end, it is an effort to
establish a technical system that promises to reduce the risk of accidents on the
roads in rural areas, and to find fundamental solutions through steady interaction
with relevant local actors. This case has been praised as a successful alternative
approach to traffic problems and is now spreading to other municipal
governments (Seong & Han & Jeong 2018).

3.3 Cases Summary and Learning from the Cases

The social problem-solving R&D project led by the MSIP has the goal of solving
social problems and has used living labs as an effective management method to
enhance the efficacy of technology development. From examining the
aforementioned two cases, the following characteristics have become apparent.

First, the purpose of a living lab was clearly defined, and focus was put on
organizing the end-user community to enable living lab participants to play a
central role. In the case of the fundus camera, the product concept and
specifications centered on core users were first determined, before completing
product development by analyzing their opinions from much richer and more
diverse perspectives. As such, this can be viewed as an exemplary case wherein
R&D activities helped solve a medical gap issue and secure new markets. On the
other hand, in the case of the development of crosswalk technology for rural

75
areas, there was active participation by residents in tackling problems, and it
served as an Urban Transition Lab. At the same time, the LEC formed a new
innovation ecosystem and created a common vision among participating
members, while striving to solve urban problems through experiments.

Second, ways to improve the delivery system and legal/regulatory systems were
explored concurrently in order to commercialize the product. In the case of the
fundus camera, the team tried to deliver its products and services to those who
actually needed them by incorporating social innovation groups as participants.
In the case of the crosswalk technology, the team is collaborating with related
organizations to build a safety evaluation system.

Third, operating living labs for each stage of technology development (from
product planning to field testing) created concrete participatory spaces for end-
users, maximizing the technology development. This confirms the effectiveness
of user-driven open-innovation, while at the same time providing idea- based
SMEs with possibilities for new (Seong & Han & Jeong 2018).

The following aspects with regard to the planning and operation of living lab can
be extrapolated from this case study. First, in planning living lab, the concrete
presentation of social issues and user demand to be addressed and organization
of end users are important. The main challenge of the living lab project is to
concretize abstract social problems to a level at which they can be tackled
through R&D projects. Both cases emphasize this process, and in particular,
various efforts were made to organize the end user group and encourage their
participation.

76
 Table 5. Comparison of cases. Source: Seong & Han & Jeong (2018)

Development of
Development of an automated
portable fundus camera
pedestrian detection system
technology
Reduce healthcare
Solve regional traffic problems
Purpose of development disparities

Set up
Request cooperation from traffic
institutional Contact potential buyers,
system-related institutions,
infrastructure etc. and pass an IRB
regarding sites with severe traffic
and select review
problems
sites

Organize expert groups

Organize working-level experts
for product planning
for product planning
Living lab Organization
design of end-user Expand user groups to
Expand user groups to include
groups include potential buyers
local residents for product testing
for product testing in
in actual settings
actual settings

Establish product
specifications Pre-simulation analysis
Exploration
Share analysis findings with LEC
Living lab
Utilize surveys, and receive feedback
interviews, etc.

Produce a minimum Proto type installation and log

viable product (MVP) analysis
Experimentation
Living lab
Feedback from potential Apply site-specific product
buyers specifications

Enhance service delivery Spread solutions through the

system with participation establishment of an LEC
from social innovation
organizations Spread policies for problem-
Living lab outcomes
solving by establishing an urban
and commercialization
Develop a transport area.
strategies
commercialization model
that includes user Develop a risk-management
education services and model and a certification system
explore new markets related to traffic laws

Second, the construction of a cooperative governance between the public,

industry, academia, research institutions, and government institutions is
necessary. To this end, the comprising parties must consist of civil groups with
sufficient representation and public nature, strong-willed governments with a
commitment to problem solving, industry members with capacity for innovation
and high commitment to social values, and interdisciplinary research and
academic institutions that can handle the problem in a holistic manner. The two
cases studied differ in the nature of participatory actors’ activities and degree of
cooperation, but the effort to create a kind of cooperative body among the actors
with the above characteristics is a common factor.

77
The third major element is the challenge to select the appropriate space or site
for living lab. As seen in the case, there are noteworthy efforts to choose an
effective site to implement living lab, with consideration to areas with a
concentration of transportation and safety issues, areas where the residents are
committed to addressing the problems, and areas where there is a history of civil
participation and action.

Fourth, appropriate methodology for developing and implementing, depending on

the problem type and situation being addressed, is essential. In the case of
fundus camera development, the project was implemented across two groups,
one comprised of optometric professionals and the other of non- professionals;
the professional group was further divided into doctors, nurses, and optometric
photography technicians. In the case of automatic pedestrian sensor system
development, the VISWALK simulation and various other methods were
employed to suit each situation.

Fifth, the improvement of legal frameworks with relation to planning and operating
living lab is important. Since the cases surveyed here dealt with medical
equipment and transportation systems that may have serious consequences on
health, important issues were raised with regard to the safety and ethics of
technology, products, and services. The important challenges of living lab also
include IRB certification, improvement of transportation law, etc., as well.

Sixth, various communication methodologies and educational programs including

dedicated mobile applications, quick display, manuals and guidebooks, etc. for
the continued interaction between the professional developer groups and the end
user groups in the process of developing, testing, and assessing technology were
developed.

Figure 4. Learning Elements in the Cases’ Living Lab Activities

78
4 Policy Challenges from the Cases
Korea has pursued science and technology innovation activities focusing on
economic growth and industrial development. For the rapid growth, Korea has
taken a strategy to develop capable subjects and areas that can grow fast first.
In recent years, inclusive innovation has been emphasized in Korea to reduce
social disparities and strengthen social integration. To this end, that the paradigm
shift from R&D-oriented policy to consumer-oriented problem-solving innovation
policy is emphasized. There is an attempt to integrate the subjects, fields, and
areas previously excluded in the process of science and technology innovation.
The emergence of new R&D categories in 2010, such as 'Social Problem- Solving
R&D Project', reflects the situation of Korea nowadays. In particular, these
projects introduced living-lab method as a propulsion system and tried to promote
open innovation activities in which end users and researchers jointly develop,
demonstrate and evaluate products in real life space. The living lab is an
infrastructure that enables professionals and end users to continuously improve
their products, services, and demonstrations with customer interactions.

Figure 5. Significance of Living Labs Activities for Social Problem-Solving R&D Projects

First, living labs are a new concept in Korea and are currently in their early stages.
Since various experiments are being conducted, it is necessary to examine and
analyze them in depth through a systematic framework. It is also important to
monitor and evaluate on-going living lab projects, share experiences and
achievements, and explore models suitable for Korean society.

Second, the introduction and spread of living labs requires strategic niche
management. If a new system undergoes testing through various living lab
prototypes and is successful, it should expand to become a bigger experiment.

79
To this end, it is important to form a common vision among interested parties
through living lab activities and to draw a consensus on the transition.

Third, living labs need to be used as an important concept in changing the

paradigm of innovation policy from a technology-based approach to a user- and
demand-based. Only by shifting the focus of the innovation activities and policies
from acquisition of technologies to consumer needs and problem- solving can
living labs become a platform that links the creativity of ordinary citizens with the
official innovation process. It is imperative to bring innovation to the entire range
of existing R&D systems, such as planning, budgeting, evaluation, as well as
infrastructure and ecosystem development, through the concept of living labs.

Finally, flexible and adjustable policy management is vital for the diffusion and
utilization of products and services developed through the production of ultra-
disciplinary knowledge in the process of operating living labs. The whole system
needs to be critically evaluated to create an environment in which policy errors
can be corrected based on long-term socio-technical systemic perspectives
(Seong & Han & Jeong 2018).

References
Health Chosun (2012), 1 out of 4 patients with retinal blindness had 'diabetic
retinopathy',
http://health.chosun.co.kr/site/data/html_dir/2012/10/17/2012101702299.
html
National Science and Technology Commission (2012), “New Science and
Technology Program Promotion Strategy”.
MSIP ((2016), Guidelines for Solving Social Problem-Solving R&D.
Schuurman, D., Marez, L.D., & Ballon, P. (2016), “The Impact of Living Lab
Methodology on Open Innovation Contributions and Outcomes”,
Technology Innovation management Review, 6(1): 7- 16.
Seong, J. & Song, W. (2007). Theory and Application of Total Innovation Policy,
Journal of Technology Innovation, 10(3): 555-579.
Seong, J. & Song, W & Park, I (2014), “Living Lab as User-Driven Innovation
Model: Case Analysis and Applicability”, Korea Technology Innovation
Society, 17(2): 309-333.
Seong, J. et al. (2016). Selected as a specialized agency for the construction and
operation of 'Citizen Research Mentor Team' for social problem-solving
technology development project. Ministry of Science and ICT.
Seong, J. & Han, G. & Park, I, (2016), Status and Tasks of Living Lab Activities
in Korea. STEPI Insight, 184, STEPI.
Seong, J. & Han, G. & Jeong, S, (2018), Analysis of Living Lab Cases in R&D
Initiatives for Solving Societal Problems and Challenges. Journal of
Science & Technology Studies, 18(1): 177-217.

80
Song, W., and Seong, J. (2013), Science and Technology Innovation Policy to
Solve Social Issues, Paju: Hanwool Publishing Co. (Text in Korean).
Song, W. & Jeong, S. (2016). Status and Tasks of the Research and
Development Project for Solving Social Problems. STEPI Insight, 185,
STEPI.

81
82
 Living Labs and Circular Economy:
the case of Turin
Federico Cuomo1, Nadia Lambiase2 and Antonio Castagna3

1 University of Turin-City of Turin, Italy

2 University of Turin, Italy
3 Managerial Trainer

Category: Research in-progress

Topic: Smart Cities & Regions

Abstract
This paper aims to present the case of the Torino Living Lab on Sharing and
Circular Economy in an attempt to highlight possible future scenarios for policies
to stimulate urban innovation in the environmental and social fields. The case
study is analysed in three phases. First of all, it is described the approach of the
local public administration to the tool of the Living Lab as a stimulus to innovation.
In the second part, the Turin Living Lab on Sharing and Circular Economy is
deepened and potentialities and weaknesses are highlighted. In the last section
we focus on understanding how the selected case can open possible fields of
comparison between administrations in order to improve globally by sharing their
local experiences.

Keywords: Living Lab, Sharing Economy, Circular Economy, Regeneration,

Turin

83
1 Introduction
Last Autumn the Municipality of Turin launched the Torino City Lab as an
initiative-platform aimed at creating simplified conditions for companies interested
in conducting tests of solutions for urban living.

With this action, the City officially committed itself to become a promoter of public
and private initiatives aimed at improving the urban ecosystem and proposing
ideas in different fields of innovation: from IoT (Internet of Things) to collaborative
and circular economy activities.

Adopting the perspective of a public actor not only as a regulator, but also as an
hub of boost to local development, the Torino City Lab as permanent platform in
the urban area was created for social, economic and administrative conditions.

As reported in the Giorgio Rota Report of 2018, the area of the City of Turin is
characterized by a high rate of small and medium-sized bodies operating in the
tertiary sector with less than 10 employees. Moreover, concerning capital
corporations of the tertiary sector, Turin has an average of 14 employees per
company. This aspect distinguishes Turin from the other large cities in the Centre-
North, where the size of the company is usually larger (Centro Einaudi 2018).
Even more significant is the low number of new companies registered in the Turin
metropolitan area, which in 2018 stood at around 13 thousand units, recording
the lowest result in the last decade (Camera di Commercio di Torino 2018). This
stagnant entrepreneurial context, increased due to the economic crisis, is more
frequently leading small and medium-sized enterprises to seek the support of
public players, especially in the taking off phase of their market. This dynamic of
local market is being progressively combined with the expressed commitment at
national level to develop practical policies to foster and accompany development
of start-ups considered innovative, as regulated in the Decree Law 179/2012,
known as "Decree Growth 2.0" (Ministero dello Sviluppo Economico 2016).

Therefore, the City is planning to become a laboratory in which companies can

establish direct contact with the final users of their products. The main purpose
of the action consists in covering the weaknesses of local companies in pre-
commercial phase. According to the local actor perspective, this initiative would
contribute to regenerating of the local entrepreneurial landscape. At the same
time, this mode of partnership with private sector could also attract investments
from Italian or international companies to the territory.

Starting from this framework, the following contribution presents the case of the
Torino Living Lab on Sharing and Circular Economy as a possible development
basis for innovative environmental policies on a local and global scales. In order
to provide the best possible structure for the research, the contribution is divided
into five sections: (I) the description of the new permanent laboratory proposed
by the City of Turin; (II) the past experiences of living labs in Turin; (III) the
approach and the birth of the Torino Living Lab on Sharing and Circular Economy;
(IV) the presentation of the experimentations admitted to the Living Lab; (V) the
interpretation of the case study as a basis for future projects on local and
international scales.

84
2 The Torino City Lab: a permanent platform for experimentation
To achieve the objectives described in the previous paragraph, the Torino City
Lab presents itself as a platform that aims to generate four main outputs in the
urban ecosystem.

First of all, the Lab ensures the access to public spaces through streamlining the
administrative process. The initiative is promoted by adopting a new strategy on
the part of the local authority, which is capable of acting by making all its sectors
work with an integrated perspective. More specifically, the Innovation Area of the
City is committed to working in agreement with the Environment and Green
Spaces Area to ensure simplified procedures for experimenters. This cooperative
management is born from the desire to quickly coordinate all the local offices and
to meet all needs of experimental bodies.

Secondly, the Laboratory is addressed to connect local actors operating in

economic sectors considered to be innovative. Through experimentation
activities, small and medium-sized companies have the opportunity to create
partnerships with large public and private multiutilities that manage the services
sector in the area. In addition, the experimenting subjects have the opportunity to
deal directly with the world of research. Adopting this approach, a start-up might
have the chance to collaborate with big local multiutilities as SMAT (management
company of hydric sector), IREN (management company of energy sector),
AMIAT (management company of waste) but also with the University and the
Politecnico of Turin.

As a third point, the Torino City Lab aims to make it possible to test products and
ideas that might be exported on a larger scale. From this point of view, every
project in the Lab are not planned to fill out only local needs, whereas they should
be designed to be reused and fitted on wider scales. This designed process is
addressed to match the transnational co-creation strategy (Santonen, Creazzo,
Griffon, Bòdi, Aversano 2017).

Finally, the Torino City Lab is based on the involvement of citizens as final users
and aims to adapt the experimentations to the needs expressed by peoples. For
this reason, in addition to the permanent chance to propose to the City innovation
ideas, the public administration works to open specific calls based on identified
challenges to fill out emerging needs of urban areas or European Union
directives.

This last aspect puts the Laboratory in its own right in the category of Living Lab,
providing the urban territory to create a public-private-people partnership,
achieving the innovation model of the quadruple helix. As described in recent
literature, this pattern shapes the collaboration of four main actors: public
authorities, industry, academia and citizens (Varmland County Administrative
Board 2018). Following this purpose, the City of Turin decides to plan in detail the
Torino City Lab, in order to avoid wasting energy and to set specific objectives.

85
As shown in the Table 1 below, the City identifies specific mission, vision and
values to be pursued in the development of the platform.

Table 1. The features of the Torino City Lab. Source: City of Turin4

Mission Vision Values

● Facilitate testing ● Positioning Torino at ● Agility in the execution

operations in real
conditions of
innovative solutions of
public interest.
● Offer constant support
to facilitate access
and then facilitate the
conduct of trials, in
relations with Internal
Services and Utilities.
European and of activities.
international level as a
place where ● Transparency of the
innovation is easier process.
and is a shared
challenge for the ● Openness of the
territory. partnership.
● Attracting companies
from Europe and the
world to engage new
trajectories of
economic
development in
sectors with high
added value and to
serve the citizens of
tomorrow.

This platform enables the City to promote new challenges in environmental fields,
which are difficult to address with classic regulatory tools, involving a huge variety
of public and private actors as well as citizens. Eventually, within the City
Laboratory the Municipality decides to promote one of the policy areas
considered most important both to offer new opportunities for local development
and to match European inputs in environmental policies: The Circular Economy.

3 Turin Municipality and Living Labs

As previously described, the Torino City Lab foresees the chance of hosting
actors from different innovative fields. However, the City has decided to adopt the
tool of living labs, paying particular attention to environmental impacts and the
promotion of sustainable development. For this reason, the several steps that
have marked the construction of the Torino City Lab have always combined
innovation with environmental sustainability to achieve a long-term goal:
becoming a Circular Economy Hub.

At the end of 2015, for the first time the European Commission designed a
Circular Economy Action Plan. On the one hand, it claimed the necessity to
change the economic model to face the lack of resources in a sustainable way.
On the other, it sets up almost 10 billion to boost the transition towards a new
plan of development, financing projects based on redesign, reuse and recycle
values (European Commission 2019). Despite this, European research institutes,

4
https://www.torinocitylab.it/it/

86
as the Ellen MacArthur Foundation, have stressed that the Circular Economy
cannot be supported neither exclusively through top-down investments of funds,
nor merely introducing regulative limits to the industrial processes.

From this point of view, the Circular Economy theory has been based on
promoting the model of the 3 R (Reduce, Reuse, Recycle) starting from the
capability to choose and act of the purchasers and end users of all services: the
citizens (Yang, Zhou, Xu 2014).

The reference idea started from the assumption that the citizen can represent the
engine of change. In this sense, the end user goes to affect not only reducing
their consumption or reusing as much as possible finished products, but also on
the systems of product design and durability of materials with their choices of
purchase.
Adopting this perspective, in recent years the City of Turin began to imagine the
Living Lab tool as a potential stimulus to the Circular Economy. In this sense, the
Public Administration has started to experiment laboratories in several fields
directly linked to the Circular Economy topic.

In 2016, the City launched the first Living Lab in its history in the Campidoglio
district, providing the urban area for an experimentation of technologies and
innovative ideas related to the Smart Cities sector. For a year the neighbourhood
became the home of 32 experiments that changed the area ecosystem through
data sharing technologies and air quality monitoring systems, urban farming and
projects against food waste.
In 2017, the local Public Administration decided to open another Living Lab
spread over several suburbs of the city, focusing on the more specific issue of
IoT. As in the previous laboratory, technological innovations were brought into
contact with citizens with the clear aim of boosting new companies committed to
environmental sustainability and improving the quality of life in urban contexts.
Therefore, IoT technologies were selected with reference to specific areas of
application related to the environment and the daily lives of citizens: the quality
of the urban ecosystem (air and noise monitoring systems); mobility; energy
efficiency; security and management of buildings; culture and social inclusion.

3.1 Turin Living Lab on Sharing and Circular Economy

Although both the described laboratories had a close link with environmental
sustainability, neither of them had been specifically focused on the Circular
Economy paradigm.

The real chance of implementing a Living Lab on the Circular Economy was
opened in the summer of 2017, when the City of Turin received a budget of 18
million euros to implement the AxTo (Actions for the Turin suburbs) programme
through a Presidential Decree from the Council of Ministers aimed at fostering
urban regeneration.

Through this broad programme, the City was committed to implementing 44

specific interventions in the selected suburbs of the City, focusing on five areas
of action: Public Space; House; Work and Innovation; School and Culture;

87
Community and Participation. Within the third pillar, which combined the
challenge of stimulating businesses and employment with innovation, the idea of
planning a Living Lab on Sharing and Circular Economy was born. Therefore, this
specific laboratory was inserted as action 3.02, focused on innovation in the
suburbs as mechanism capable of dealing with the crisis of local businesses
(Comune di Torino 2018).
The reference pattern was based on creating open-air laboratories that give a
chance to companies engaged in sectors such as sharing economy, internet of
things, digital manufacturing, circular economy, environmental sustainability and
food. In addition, innovation was also interpreted from the point of view of
recovering the craft heritage of the reference areas to keep alive sectors of the
craft industry that are strongly linked with circular economy (shoemakers,
carpenters, hardwars). For these reasons, four main goals were identified by the
City: boosting the local private sector; stimulating new ideas of business; creating
a network of sustainable development composed by entrepreneurs; planning
conditions to host in public spaces innovative experimentations.

To achieve these goals, the project was planned by the Development and
Innovation Area of the City over 18 months, from May 2018 to December 2019,
with a maximum time allowed of 9 months for each experimentation. In the spring
of 2018, an external “Managing Authority” of technical support to companies and
communication with citizens was identified through a public call for tenders. This
initiative of the City was addressed to non-profit companies, associations and
foundations specialised in development strategies and activities of territorial
promotion.

In the same period, the City publishes the call for the selection of private
experimenters, open for two months (May-July 2018) to companies in partnership
with community associations or Universities and research institutions. The
contribution made available by the City amounted to 100 thousand euros. Each
testing action could receive a grant up to a maximum of 15 thousand euros, equal
to 50% of the total eligible investment to cover the costs of experimentation.

The identified areas of experimentation of the Laboratory were concentrated in

the North and South suburb neighbourhoods. However, proposals that provided
actions spread throughout the city area were also allowed.

88
 Figure 1. The map of admissible areas of the Living Lab / Source: City of Turin5

To objectively examine proposals that were could be deeply diversified from each
other, the City set up an ad hoc evaluation committee to select the projects. This
evaluation body was composed bringing together experts in the different fields of
activity. Accordingly, to the committee effort, the evaluation process was based
on five criteria considered decisive for access to public funding: (I) Technical
Feasibility; (II) Uniformity; (III) Level of Innovation; (IV) Level of Engagement; (V)
Economic Sustainability.

The type of contract chosen by the City to start the testing phase with the selected
subjects was the Partnership Agreement. The latter was considered by the public
administration as the most suitable to start the Living Lab because it clarified the
conditions and facilitates the administrative procedures for the transfer of grants.

3.2 The methodology

From the point of view of small companies, planning projects in the context of the
circular and collaborative economy, activating partnerships with other private
subjects, cooperatives and universities are not simple actions. These difficulties
can represent obstacles to innovation. For this reason, the City of Torino wanted
to include a Management Authority that could apply an accompanying
methodology.

In the design phase of the Living Lab on Sharing and Circular Economy, the
experts of the Authority met the proponents with the aim of informing and
encouraging new networks. These preliminary meetings were an opportunity to
share ideas and encourage their development. This action enhanced the

5
http://www.comune.torino.it/sfogliato/axto/files/assets/basic-html/page-1.html#

89
strengths and highlighted the weaknesses of projects in terms of technical
feasibility and economic sustainability, urging subjects to improve specific
aspects.

In the evaluation of projects phase, the team constructed a summary table in

which the strengths and weaknesses of presented projects were indicated with
reference to each specific assessment criteria. They also guaranteed their
presence during the discussion, with the objective of facilitating the work of the
evaluation commission.

In the system and planning phase, the proponents followed a structured training
course in four workshops with the aim of validating the idea; identify monitoring,
evaluation and impact indicators; find consistent community engagement
methodologies; develop the action plan. Design thinking and systemic design
were the basis of the working method. System map and Social business model
canvas were the main tools used. The output of the entire activity was a roadmap
for each of the eight projects admitted to the trial.

In the testing phase the team specialists met the proposers, individually or with
the whole project team, on at least four different occasions with the coaching
methodology. On the one hand, this initiative was implemented to reaffirm the
value of the partnership between public and private subjects. The meetings could
be experienced as a form of bureaucratic control. However, the difference
between monitoring as a form of support instead of control has not been easy to
perceive and it has made necessary to build a relationship of trust with the
proponents. On the other, emphasizing weaknesses has represented a value for
the project, as well as insisting on the identification of precise indicators of
success or failure.

Form the point of view of the City, the accompanying methodology carried out by
the Authority has given rise to some points for reflection. First of all, coaching
activities have allowed to gather information and data useful for evaluating the
effectiveness. The collected data have concerned the individual projects, the
identification of the critical phases, the prevailing orientations of the proposers,
the blind spots of the projects, the ability to involve and engage the population,
with the aim of identifying suitable forms of support and accompaniment for future
living labs. In addition, information has been collected on the entire
experimentation process, the ability to facilitate the activation of territorial
networks, the visibility and the communicative impact, with the aim of developing
processes capable to promote innovation in the circular and collaborative
economy.

The accompaniment has also made it possible to provide adjustments during the
course of the Living Lab, in order to deal with difficulties that have arisen in the
process phase. Furthermore, the in-progress accompaniment has facilitated the
activation by the Public Administration, in terms of authorizations and definition
of agreements with the involved public departments.

90
Table 2. The methodology pathway of the Living Lab (Adapted by authors with permission
of SocialFare)

The latter aspect has represented a tough step. Social innovation and economic
innovation require the presence of an ecosystem capable of accepting and
enabling changes, both from the point of view of norms and local regulations and
authorization processes. For instance, one of the companies needed to conclude
an agreement with the city to experiment with hydroponic production in a city
park; another needed to collect plastic with the contribution of citizens, avoiding
that it ends up in the waste stream; a third decided to produce and distribute hot
meals in the homes of the first night shelter for homeless people also using food
from charitable collections from the city markets. In these cases, Authority has
activated various services to cooperate within the short time frame of
implementation.

Another aspect to emphasize has been the support of the Authority for the
dissemination of the 3.02 program aimed to insert and position the City of Turin
in a European debate on the circular and collaborative economy, identifying
opportunities for meetings, workshops, conferences, where to present the case
of the experiments conducted on the territory of the City.

4 The experimentations
The time frame for developing the eight selected projects is from January to
September 2019. To have a better understanding of the characteristics of the
Living Lab it is necessary to present the projects admitted to experimentation on
the urban territory.

Abbasso Impatto (Lower Impact)

91
Abbasso Impatto is a project conceived and developed by the Verdessenza
cooperative, based on the collaborative economy and built on the model of
Solidarity Purchase Groups (collective purchasing groups). The objective is to
reduce the environmental impacts in the consumption of catering and hospitality
establishments and to guarantee sustainable prices for supplies thanks to
collective purchasing. The experimentation area identified is the San Salvario
district of Turin.

Verdessenza is primarily concerned with assessing the needs of catering and

hospitality establishments, and simultaneously identifies and selects the groups
of products and services to be offered to them. In order to carefully choose the
suppliers of the necessary products and services, it draws up the Minimum
Environmental Criteria through which to select the suppliers that guarantee a
production process more attentive to socio-environmental sustainability.

Edilizia Circolare (Circular Building)

Edilizia Circolare, a project conceived and developed by the Emmegi company,
was born with the aim of applying the concept of reuse and recycling in the
construction sector.

The first step is the establishment of a team of professionals for reuse, made up
of architects, designers, companies and artisans. This will be followed by the
identification and collection of potentially reusable materials that will be donated
by citizens, businesses and local artisans, to be transformed and come back to
life in new furnishings and finishes. Throughout the project, workshops and focus
groups will be organized for both professionals and DIY (do it yourself)
enthusiasts.

The final product of the trial will be the restyling of a room located in Via
Montevideo 41, entrusted by the Municipality of Turin to the Paradigma Social
Cooperative, which will host a café for members and new laboratories.

Suolo sostitutivo (Replacement soil)

The project, conceived and developed by Horizon srl, aims at the re-use, in the
context of territorial planning, of inert material, following an appropriate treatment,
coming from excavations carried out in the city for infrastructural works. Normally,
in fact, this material is classified as waste and is stored in landfills.

The main objective of the project is the development of a technical protocol for
the constitution of a soil capable of replacing the natural one, suitable to sustain
a plant substrate over time.

Large volumes of inert materials that are difficult to dispose would be transformed
into secondary raw materials, in line with the principles of the circular economy
and with the current provisions of the European community regarding waste
reduction and re-use and recycling of waste materials in order to guarantee the
conservation of ecosystems.

92
The information taken from the experimentation can then be used to redesign or
convert parts of the industrial waste generation process and to develop a mixture
that can become a marketable product.

UrbanAquaFarm
UrbanAquaFarm, an experimental project proposed by Carlo Prelli Service,
wants to develop and test innovative systems for horticulture. Within the
framework of the "Orti Urbani Torino" system, a pilot project is proposed that
creates a collaborative system of production and consumption of plant products
based on "hydroponic" culture techniques. Specifically, the project will build
prototypes and experiment with circular horticulture practices, in the area of the
"urban gardens" inserted in the Parco dei Laghetti in the north of the city,
inaugurated during the 2018 spring in an area currently undergoing
redevelopment.

During the implementation of the project, specific dissemination activities will be

carried out, as well as additional parallel and collateral initiatives, directly
applicable to the entire system of "Urban Gardens" of the city, such as training
courses on the "Hydroponic" system, vocational training seminars, visits guided
tours for students and groups of citizens.

Humana, RicuciTo project

The project arises from the fact that 5,000 of the 20,000 tons of used clothes
stored each year in the Pregnana Milanese warehouse have any market value.
Humana has chosen to build a pilot project capable of dealing with one of the
main components of this quota, denim. To do this it has built a partnership with
the design course of the Polytechnic of Turin, which has allowed 240 students of
the second year of Design to work on the problem, and with the social cooperative
"Il Gelso" which has also activated the laboratory of the District House Lorusso
and Cotugno of Turin where three women work.

Izmade, Beautiful Precious Plastic project

Izmade creates design objects and furniture, working wood and metal. Beautiful
Preciuos Plastic aims to include the Precious Plastic open source machine in the
laboratory, so as to expand the range of materials and objects. The machine
consists of a shredder, an extruder, an injector and a press. To carry out the
process it is necessary to engage the local population that is called to participate
in a dedicated collection of plastic and that will then be able to access the
workshops dedicated to specialists and amateurs in the multifunctional centres
of Barriera di Milano area.

Magma, the Balon's Marketplace project

The project plans to create a marketplace dedicated to vintage goods and
antiquities marketed in the traditional Balôn market in Turin, which has been held
since 1857. The markeplace has two main objectives: 1) to increase the
commercial capacity in a traditional sector, opening it to a foreign clientele or
resident far from Turin; 2) supporting evolution and transparency in a traditional
sector closed to innovation and comparison with wider and more developed
markets.

93
Stranaidea, project CON il cibo 2
The project is the evolution of a previous experimentation with which the
Stranaidea Social Cooperative had already begun to distribute hot meals in one
of the three-night shelter facilities it manages. The objectives of the project are to
guarantee at least one hot evening meal to the guests of the facilities managed
directly by the cooperative and to encourage the empowerment of guests by
involving them in food collection preparation and distribution. The cooperative is
trying to activate a negotiation with the City, the client of the service, also in
agreement with other subjects that manage other reception centers that could
benefit from the service.

5 Towards a circular economy Hub for the Municipality of Turin

Cities are in a critical position with respect to the transition to the circular
economy. On the one hand they have a very high impact on the environment for
all the activities they carry out. On the other hand, their characteristic of having a
high concentration rate of resources, capital, data and talents in a relatively
narrow geographical area can be an opportunity.

In this perspective, it becomes interesting to think about the city in terms of a

peculiar ecosystem of social and economic innovation, able to face the transition
from a linear to a circular economic model (Ministry of the Environment for the
Protection of the Territory and the Sea, Ministry of Economic Development,
2017). In order to increase awareness of its existence, there are two fundamental
ingredients that those who administer a city have to consider: a) intention to invest
in and with the city community; b) establish clear and shared governance
mechanisms.

Within this perspective is moving forward the ongoing experimentation of the

Living Lab on Sharing and Circular Economy of the City of Turin, for which the
Municipal Administration has chosen to be flanked by a Managing Authority. The
objective of this choice is to experiment a methodology and a practice aimed at
building the structure of a territorial hub of Circular and Collaborative Economy of
Turin and at formulating the definition of a city policy concerning the circular
economy issues. Specifically, a territorial hub is meant to be an ecosystem of
public, private and civil society subjects that interact with the aim of bringing to
value, economic, social and institutional the environment in which they operate.

In order to function at its best, this ecosystem has to plan a governance structure
capable of effectively putting the various actors involved into relation. Therefore,
the action of the Municipal Administration in assuming the management of
direction and involvement plays a key-role. As regards the strategic political
address, it is desirable to create an inter-council Control Room (environment,
innovation, work, social inclusion, culture and education). Concerning the
involvement, it is necessary to consider that the boundaries of the Circular
Economy, as literature is defining it (Lacy P., Rubqvist J., Lamonica B., 2016), is
very broad and extends well beyond the only field of recycling. Rather it includes
all the phases of the realization of a good and service, and therefore: the

94
conception and design, the production, the distribution, the modality of fruition
and of transformation. The circular economy business models are different, and
they cover several phases of the production cycle: circular chain from the
beginning (biobased material and energy, and/or second raw material as
productive inputs); recovery and recycling; upcycling; extension of product life;
sharing platforms; product as a service.

Moreover, could be considered crucial the creation of a subject as a Circular

Economy Manager Group, which acts as the executive branch of the Control
Room. This body would operate with the mandate to coordinate and bring to
system the various initiatives, practices and projects that already are moving in
the city in the various sectors of the circular economy.

Several are the key actors to be involved in the Hub governance structure. First
of all, the different universities in the area: The University of Turin, through the
new Doctoral School Innovation for the circular Economy; the Polytechnic of
Turin, thanks to the multiple skills related to Systemic Design; the University of
Gastronomic Sciences of Pollenzo which carries out research related to food and
circular economy.
Secondary, strategic players that must be involved in the Hub's governance
structure are trade associations such as the Chamber of Commerce, Industrial
Union, Confindustria, Confartigianato, CNA, Confcooperative, Legambiente.

Thirdly, the various utility companies are key-actors that have to be involved. In
this regard, the experience carried out by Maribor is interesting. In June 2018
Maribor became the first city in Slovenia to define a strategy for transition in the
circular economy, in close synergy with the Charter of Sustainable Development
Goals. This strategy was conceived and developed by Wcycle Maribor - Institute
for the circular economy - founded by five utility companies in the city. The
objective is to implement a management system for all flows of materials and
resources available in the city, capable of generating cross-sectoral cooperation
between seven different fields: urban waste; construction and demolition waste;
mobility; water; power; territorial planning; collaborative economy. The five
companies aim to achieve the highest rate of reuse of material, energy and water,
sharing information and activities.

Finally, the concept of hub recalls the network paradigm (Buchanan M., 2003).
The constituting Hub is to be understood both as a new network of territorial
actors and at the same time connection of existing ones. Additionally, the Hub
has to be meant as a node of a wider network, connected to supra-local scales
of action. With regard to the connection with the supra-local networks, the work
of the Circular Economy Manager Group is strategic, which makes it possible to
relate what is moving, developing and is learned at the Hub level with the rest of
the Italian and international circular economy context.

To sum up, adopting the City perspective it will be important to identify one or
more significant places in the urban area, able to rise to physical places of
networking and exchange, in which companies, citizens, schools and universities
can directly interact. Some of these can be identified starting from the same

95
experiments in progress through the Living Lab on Sharing and Circular
Economy. At the same time, the City is thinking of exploiting disused spaces,
such as the Remida centre, to build physical clusters for the new Hub.

Therefore, the next step will be to find a suitable home for the Hub, making the
most of the huge architectural heritage left unused by the crisis of local industry.

6 Conclusions
This paper has addressed the practical case study of the Living Lab on Sharing
and Circular Economy of the City of Turin. Deepening the Turin policy of using
Living Labs as a stimulus for innovation, the case study suggests three main
points of discussion.

First of all, the Living Lab on Sharing and Circular Economy has shown how there
is an emerging network of local businesses, associations and committees of
citizens increasingly active in the field of sustainable entrepreneurship. Through
the facilities made available by the City and the guidance of the Managing
Authority, these bodies have managed to get in direct contact with citizens. This
direct approach provided through the Living Lab could be a tool to improve their
innovative ideas and modify them to better match the needs of citizens.

Secondly, the case study has suggested how the Living Lab methodology allows
to build the foundations to turn cities into innovation hubs. In the recent history of
the City of Turin, no classic regulatory or business incentive tool has ever
managed to bring together the City, research institutions, businesses and citizens
on such a key-issue as the Sharing and Circular Economy. This Living Lab will
therefore be the starting point for transforming the Torino City Lab into a real Hub
of Circular Economy.

Finally, the case presented has highlighted how local action is directly linked to
global policies. The need to find sustainable business solutions that preserve the
environment by working on reuse, material recovery and recycling is shared by
most of the world's institutions. However, shared policies on paper often find
barriers insurmountable in practice. Nevertheless, living labs could represent
practical tools to connect cities and to scale from local to global policies in support
of the sharing and circular economy.

To sum up, the case of study has clarified that the greatest challenge is to bring
together different actors on both a local and global scale to promote real changes
in environmental regeneration and citizen services policies. A challenge that can
only be faced by cities through dialogue and positive exchanges addressed to the
planning of future living labs.

References
Buchanan M., (2003), Nexus. La rivoluzionaria teoria delle reti. Perché la natura,
la società, l’economia, la comunicazione, funzionano allo stesso modo,
Mondatori.

96
Centro Einaudi, (2018), Servizi: uscire dal Labirinto. Diciannovesimo Rapporto
“Giorgio Rota” su Torino. Torino, Centro Einaudi
Comune di Torino, (2018), AxTo Azioni per le periferie torinesi. Schede descrittive
delle azioni, Torino, Comune di Torino
Lacy P., Rubqvist J., Lamonica B., (2016), Circular economy. Dallo spreco al
valore, Egea. Santonen, T., Creazzo, L., Griffon, A., Bòdi, Z., Aversano,
P., (2017), Cities as Living
Labs-Increasing the impacts of investment in the circular economy for sustainable
cities, Luxembourg, European Union
Yang, Q., A., Zhou, J., Xu, K., (2014), A 3R Implementation Framework to Enable
Circular Consumption in Community, International Journal of
Environmental Science and Development

Web-Site References
Abbasso Impatto www.verdessenza.to.it
European Commission, (2019, March 4), Report from the Commission to the
European Parliament, the European Economic and Social Committee
and the Committee of the Regions on the implementation of the Circular
Economy Action Plan. Retrived from http:
//ec.europa.eu/environment/circular-economy/index_en.htm
Camera di Commercio di Torino, (2018), Nati-mortalità delle imprese torinesi
nel 2018. Retrived from https://www.to.camcom.it/sites/default/files/studi-
statistica/Lungo_Natimortalita_2018.pdf
Circular Glasgow https://circularglasgow.com/
Comune di Torino, Centro Riciclaggio Creativo Remida
http://www.comune.torino.it/iter/servizi/centri_di_cultura/arte_e_creativita/
centro_remida/index.shtml
Edilizia circolare www.emmegiservizi.com
European Project Urban Wins https://www.urbanwins.eu/
Ministry of the Environment for the Protection of the Territory and the Sea,
Ministry of Economic Development (2017) Towards a circular economy
model for Italy. Framework document and strategic positioning. Retrived
from
https://circular-impacts.eu/library/1789
Ministero dello Sviluppo Economico, (2016, February 4), Scheda di sintesi della
policy a sostegno delle start-up innovative. Retrived from
https://www.mise.gov.it/images/stories/documenti/Scheda_di_sintesi_poli
cy_startup_innovative_0 4_02_2016.pdf
Politecnico di Torino
https://didattica.polito.it/laurea_magistrale/design_sistemico/it/presentazi
one
SocialFare https://socialfare.org/

97
Slovenia Times http://www.sloveniatimes.com/wcycle-waste-treatment-in-the-
circular-economy-for-the-city-of-m aribor
Suolo sostitutivo https://horizon.to.it
Tavolo del Riuso http://tavolodelriuso.it/
Università degli Studi di Torino, Scuola di Dottorato Innovation for the circular
economy https://inno-ce.campusnet.unito.it/do/home.pl
UrbanAquaFarm www.cpsenergia.it
Varmland County Administrative Board, (2018, June 12), A quadruple Helix
Guide for innovations. Retrived from
https://northsearegion.eu/media/5326/quadruple-helix-guide-version-
20180612.pdf

98
 To get things right for children.
Implementation of a public social living lab model
for coordinated support for children in need
Angelika Thelin*1, Torbjörn Forkby1 and Mats Anderberg2

*Corresponding author
1 Department
of Social Work, Linnaeus University, Sweden
2 Department of Pedagogy and Learning, Linnaeus University, Sweden

Category: Full Research

Topic: Health and Wellbeing

Abstract
There is a large need in Sweden and internationally for the development of
knowledge-based approaches to improve children’s well-being, promote
learning, school attachment and self-efficacy early in life. This includes both the
articulation of comprehensive policy frameworks and the implementation of
targeted interventions. One response to this is presented by the Scottish model
Getting It Right for Every Child (GIRFEC). Central pillars are to improve
children’s well- being and learning through early intervention, universal service
provision, and multi-agency coordination. The model has gained substantial
interest in Sweden, where the most challenging implementation is taking place
in the county of Kronoberg, including eight municipalities and several health
service organizations. This research paper is based on material from the ongoing
evaluation that aimed to establish an interactive research in support of the
implementation process. The paper describes the early process that followed the
implementation decision and discuss how it might be understood as a public
collaborative social living lab and what this demands from the researchers.
Emphasis is put on the researcher’s role to balance between partaking in the
innovative work and standing aside and giving critical reflections.

Keywords: coordinated interventions, children, well-being, implementation,

social living lab, on-going evaluation

99
1 Introduction
Social innovations share the commonality of being social to their both ends and
means (Young Foundation 2012). To work with social innovations through inter-
organisational and multi-actor collaboration has been presented as a way
forward to meet financial as well as democratic challenges in the public welfare
sector (Nicholls, Simon & Gabriel 2015, Sørensen & Torfing 2015). Social
innovation can be applied with the aim to create something entirely new, but also
in adaptive processes when policy or interventions are transferred between
contexts (Sørensen & Torfing 2015). The social living lab methodology offers a
broad framework for how to address societal challenges on local and global level
through innovative and co-productive processes between users, professionals,
researchers and other stakeholders (Garcia Robles et al. 2016). It suits therefore
well with policy intentions to establish trust-based processes for enhanced
quality in the public sector.

To integrate research in these innovative processes are furthermore in line with

the objectives for evidence-based practice in the welfare sector (Greener &
Greve 2014, Nicholls, Simon & Gabriel 2015, Sørensen & Torfing 2015).
However, research provides limited empirical examples of social innovation
processes in the public sector that integrated research (Garcia Robles et al.
2016, Nicholls, Simon & Gabriel 2015, Sørensen & Torfing 2015). This paper
discusses how an implementation of a knowledge-based praxis model (GIRFEC)
could be facilitated by the use of a public social living lab methodology, and what
this implies for the researcher role. Interactive research was connected to the
implementation process from the beginning. The intention with the
implementation is to bring about early and coordinated interventions for
children’s and youth’s wellbeing and learning. The implementation (here labelled
BBiK6) takes place in the county of Kronoberg involving eight municipalities and
health care services in the south of Sweden.

1.1 The Swedish context

Sweden has a universal child welfare model (Robertson 2011), realized as the
offering of comprehensive public assistance generally to children and families.
For example, public health and dental care are free of charge for all children from
birth until 20 years of age. Preventive health care is integrated in all schools and
child benefits or grants are allowed all families until the end of upper secondary
school. In spite of this, the trust for the public social policy is challenged due to
perceived inefficient in public sector, inflexible adaption to meet needs, and lack
of cooperation between welfare actors (Bringselius 2018). The relative income
poverty among single households with children has increased in the last
decades, mental health problems among children and youth has developed in a
problematic way, and it is hard for young adults to find housing and
employments. In this context, child welfare services are criticized for spending
too much time on documentation and assessment, and too little time to give voice
and support to children and families in need. It has also been argued for more
influence from evidence-based methods in the educational system and in the

6 Acronym from ”Barnets Bästa Gäller! I Kronobergs län” – Get It Right for Every Child! In Kronoberg.

100
social services (Grefve 2017, The National Agency for Education and the
National Board of Health and Welfare 2018, The Swedish Agency for Public
Management 2014). Turning the UN Convention of Children’s right into a law
also present a challenge for the practical work.

1.2 The praxis model in Scotland

Child welfare systems in the Western world can broadly be divided into two
categories: universal systems exemplified by Sweden and residual systems
found for example in the United Kingdom (Robertson 2011). The former aims to
provide support for families in a more comprehensive manner, and at prevention
rather than managing the consequences from risks. The latter emphasises the
importance of parents’ rights and the integrity of family life. Public assistance is
concentrated towards protection of children at more severe risks.

Scotland has been described as having a hybrid child welfare approach

(Robertson 2011, Stafford et al. 2008), highly concern with the families, parents
and individual’s discretion at the same time as stressing assets and resilience of
communities for achieving wellbeing among children (Coles et al. 2016, Peterson
2015). An expression of the latter intention is the policy framework Getting It
Right for Every Child (GIRFEC). This was worked out for over a decade, and
were to be fully implemented in 2016 (Coles et al. 2016). In the short run, the
aim of the policy framework was to provide better quality of life for children by
focussing on early, universal and, when needed, coordinated efforts for those at
risk. In the end, the aim was to reduce inequalities stemming from poverty and
narrow life expectancies. Children and youth were to be offered the opportunity
to develop, reach their full potential, and become successful, confident, and
responsible citizens.

More specifically GIRFEC consists of a praxis model with the following key
elements: (1) the Named Person or the Named Person Service and Lead
Professional, (2) three assessment tools – the SHANARRI well-being indicators,
the Resilience Matrix, and the self-assessment tool My World Triangle and (3)
the coordination document Single Child’s Plan (Coles et al. 2016). The “named
person” is the appointed actor for every child targeted through universal
provisions from health and education services. For those children and youth in
need of more extensive support, a Lead professional (usually a social worker)
could be appointed. The responsible professional is to make sure that needs,
risks and resilience are assessed and suitable measures are undertaken if
needed. The aim of the assessment tools and the single child’s plan is to facilitate
common understanding of notions of well-being in all concerned agencies, based
on a holistic, ecological child development theory (Coles et al. 2016, Peterson
2015).

The policy framework intends to stop agencies operating independently of one

another, only concerned with one or the other aspect of the child’s life. It is
stressed that success of the overall work must be searched in terms of children’s
and youth’s wellbeing, not in scrutinizing professional processes (Coles et al.
2016). GIRFEC considers children’s’ perspectives and rights, aims to ensure that
children and families are listened to and are included in as well as understand

101
decisions affecting them. Positive outcomes particularly in respect to children’s
protection and changes in professional practice and culture have been reported
from the implementation of the model. At the same time, different views on
professional roles and the balance between intrusions in family life, supporting
well-being and protecting children have been noted (Coles et al. 2016).

2 Previous research on implementation and interactive research

Implementation research has pointed at several challenges involved in the
process of reforming policy, or making a new model/method to work:
● Even in the less complicated processes, focus on the original intentions is
often lost (Alexanderson, 2006; Pressman & Wildavsky, 1983).
● It is difficult to overcome conflicting goals, governance forms and views
on what can be considered as quality between organisations (Darlington
& Feeney, 2008).
● There are often different views on what should be done for individuals
among users, different professionals and organisations (Danermark &
Kullberg, 1999).
● There are seldom any objective and knowledge-based standpoint
available to argue from in resource-demanding, complicated or complex
processes (Julian et al., 1995; Rogers 2008).
● Legal and financial obstacles can prevent good ideas and collaboration
between organisations (Huxham & Vangen 2005).

All these challenges need problem-solving processes aiming to establish long-

term trust, confidence in each other's work, and commitment to a common goal.
These would often be realised as multi-actor negotiations about overstepping
boundaries protected from different stakeholders (Gieryn, 1983; Lamont &
Molnár, 2002). It furthermore points to the need of provoding the parties with
required knowledge, inspiring their will to participate and giving them adequate
conditions for doing so (Lundquist 1987:47, Logan & Graham 1998).

Dynamic methods for collaboration between the academic world and community
actors, in everything from joint problem formulation, knowledge-generating
processes to actual change processes is required to meet many of today’s
challenges (Nowotny et al. 2001, Svensson et al. 2002). The so-called fifth
generation of ongoing evaluations or interactive research has underlined that
implementation must acknowledge also local contexts and individual
idiosyncratic interpretations (Sjöberg et al. 2009, Svensson et al. 2007, Weiss
1979). At the same time, the researcher’s role is to regularly “inject” more general
scientific feedback into the practice – leading to new questions, analysis and
organisational action supporting local development. The researchers are
supposed to act as a “critical friend” (Sjöberg et al. 2009), however thereby
putting him/herself in front of other questions:

● How to maintain an independent and critical role when becoming involved

in the development processes and acquainted to the participants?
● How to keep focus on long-term results, when the organisations work
under the pressure of efficiency?

102
 ● How to capture the views and wishes of different stakeholders if identified
as part of the processes?
● How secure sufficient resources for the rather resource demanding
interactive research?

Complicated and complex development processes will take (sometimes

extensive) resources. Nevertheless, an iterative message from research is to
establish partnership, integrating practice and research when striving for
increased quality and efficiency in public sector for the best of the citizens’
wellbeing. In addition, these co-productive processes should also include users,
citizens, different actors and organisations in the public, private or voluntary
sector, as well as hybrids inbetween. However, there is still a lack of knowledge
about how to establish these collaborations in different forms of practice and
contexts, as well as what different forms of collaborations might bring about – for
example in relation to coordinated efforts for children’s and families’ wellbeing
and needs.

3 Method and materials

The on-going evaluation of or interactive research in BBiK started in December
2018, and included three researchers in social work and pedagogy (the authors
of this paper). Additional financial support has been granted from the Kamprad
Family Foundation for entrepreneurship, research and charity for three more
years, something that makes an expansion of the scope and research group
possible.

The details of how the interactive research should be formed was held quite open
at the beginning. Generally, the role of the researchers was to introduce scientific
perspectives and knowledge as well as ”disturb” the implementation by raising
questions and problematize the process based on collected material, acting as
critical friends (Sjöberg et al. 2009). Since working close to an evolving and
changing implementation process, we have to continuously reflect on and
reconsider our “temporary” understanding, position and research methods to
adapt. This approach serves to make the research flexible enough to contribute
to ongoing knowledge and development processes, without creating
unnecessary uncertainty or vagueness through jumping between research
positions. Changes in the research direction should instead depend on the
development and needs arisen during the implementation process, identified
through analysis of connections between the collected material and scientific
perspectives and knowledge.

The findings in this paper is the results of such analysis in the early
implementation processes. Following empirical data have been collected and
considered for the purpose of writing this paper:
● One focus group interview with the three regional process leaders, lasting
1,5 hours.
● Eight hours reflexive dialogues between researchers and the regional
process leaders.
● One group interview, 1,5 hours, with three local facilitators from the
biggest municipality.

103
 ● Observation of two steering committee meeting. Written documentation
prior to and memoranda after these meetings.
● Participant observations of one operational group meeting and one
“creative workshop” at the regional level.
● Participant observation of one operational group meeting in a smaller
municipality.

The researchers and authors of this paper have met several times and
considered the collected material through a dialogue. The dialogue has been
about how the overall implementation process can be understood through a
connection between the collected material, scientific perspectives and
knowledge; what strengths and challenges can be identified; and what does this
imply for the interactive research design. Between the meetings, the researchers
have search and shared information about possibly relevant scientific
perspectives and knowledge. The section following below share the results of
this dialogue. It describes how the researchers understand the early
implementation process, but present above all scientific perspectives and
knowledge that has been found relevant to consider in relation to the
development. The paper then ends in conclusions about what lessons have been
learned through this intellectual work.

4 Findings
4.1 The early implementation processes
In the County Kronoberg, the first step of the implementation process in practice
consisted of setting up a supporting organisation. In 2018, at the time for the
decision to implement GIRFEC from the end of the year, a regional steering
committee with representatives from the involved stakeholders’ top officials was
set up. After this, an operative regional group consisted of first line managers
from all municipalities and the health services within the county council were
given mandate to work with the implementation process, together with the three
regional process leaders. From the beginning of 2019, local operational groups
with both first line managers and professionals in each municipality and the local
health care sector have been formed. These local groups are to include the first
line managers that are part of the regional operational groups.

All groups that have been formed are interdisciplinary with representatives from
the school, social services and health care. However, only the regional steering
committee and one local operational group has so far succeeded to include the
police, since they could not prioritize this before emergency matters. One parallel
disciplinary group has been formed within the health care sector, due to a felt
need to strengthen their internal collaboration first (among psychiatry for children
and youth, maternity welfare, childcare centre), and an un-familiarity of
prioritizing broader collaborative network outside their own sector.

Early in the implementation process, the steering group wanted an examination

of the County’s preconditions for implementing the GIRFEC model. However,
they soon changed this position and called for more of direct action. Argument
such as that action needed to be taken in order to keep practitioners focused,

104
motivated and encouraged were put forward. The risk that BBiK would enjoy
decreased attention over time, be seen as a diffuse and too complex process
had to be prevented by immediate action. Some also argued that action was
needed on behalf of the children and youth who suffered unnecessarily (partly)
because of bad organisation.

The demands for quick and targeted change processes came, in the regional
process leaders’ interpretation, to be translated into working with the ideological
and theoretical foundation of the model. Something that could evolve as a
common glue holding the actors together in future work. The chosen way of doing
this was to arrange “creative workshop” for the regional operational group of first
line managers, followed by discussions in the local operational groups,
occasional tests in local practice, followed by feedback back the regional level.

The first process started with a processing of well-being indicators of children,

expressed in existing assessments tools compared to those in the SHANARRI
model. The workshop then turned to creative work inspired by The Logical
Framework Approach (SIDA 2006), which ended in an outline of a new common
well-being assessment tool for the involved organisations in BBiK. While not far
from assessment tools already in use in their organisation, the new outline had
a clearer focus on the child as being the subject. This outline was then send to
the local level for discussion and tests.

The regional facilitators have also encouraged the local operational groups to
take into consideration the views of children and families on the assessment tool.
One municipally has returned feedback from views of some five years old
children, and some others are still discussing how to go on with including
children, parents, private and non-profit organisations in the process. A final
version of the well-being indicators is to be decided upon by the steering
committee and implemented after the summer of 2019. The implementation idea
is to go through similar processes also with the rest of the central elements of
the GIRFEC praxis model.

Parallel interviews and observations at the local level so far indicate that BBiK
locally primarily is associated with other efforts than those related to GIRFEC.
Still there is an obvious lack of knowledge about GIRFEC. Whereas GIRFEC is
a comprehensive coordination tool for investigation, planning and evaluation of
any possible individual efforts – leaving room for any kind of intervention
depending on needs. The efforts in focus at the local level is so far primarily
about predetermined, fixed organized and already existing coordinated actions
developed through the last five years.

4.2 Challenges for the interactive research design

As a response to the steering group early wishes of an examination of the
County’s preconditions for implementing the GIRFEC model, the interactive
research found inspiration from The Ottawa Model of Research Use (OMRU)
(Logan & Graham 1998). The model’s aim is to promote research use in health
care practices and provide six themes for the researcher–practitioner interface:
1) organisational terms/the practical context, 2) those who shall receive and

105
implement the method, and 3) the new method/model in itself. These themes are
used to clarify the objectives of the development process and identifying,
preventing and managing obstacles and difficulties expected in the
implementation process. Dialogues and knowledge-processes are facilitated in
which researchers and practitioners target the 4) implementation strategies when
the method/model is to be put into practice, 5) the actual implementation, and
follow-up data 6) the use of the evidence, and health-related and other results
from the process.

However, the changed position of the steering group and the decision to
implement the GIRFEC praxis model immediately forced the interactive research
to jump straight into an implementation process without prior assessment. This
meant that the implementation stages explained by the OMRU methodology had
to be sidestepped, even though they could have a profound importance for
whether or not to succeed. Even though the Ottowa model is dynamic, it still
follows a systematic logic starting with the investigation, followed by planning
and then action. Barriers and supporting factors are identified in a time where
they still can be addressed, providing direction for adjusting chosen strategies.
This way of working therefore presupposes a predetermined work process that
the actors involved follow.

The process leaders’ draw to move back in the implementation circle can be
interpreted as a kind of soft resistance that forced the process into a slower pace,
and in a way trying to bring about an imitation of the original development of the
GIFREC model. The workshops as a strategic implementation tools, were set
out to organise multidisciplinary and multisectoral creative work, aiming at a
common language, understanding and trust between all involved actors. At the
same time, this introduce an ambiguity of whether it is GIFREC that is
implemented in its original form or whether the process now rather is aiming at
change through imitation of innovative solutions from elsewhere through a
process of collaborative adoption and adaptation. The later – especially since
the process also have interactive research connected to it – could then be
understood in terms of a public collaborative social living lab.

4.3 Implementation through a public social living lab

Living labs integrate innovative processes and research, have a user-centred
perspective as their starting point, and use local experiences and real contexts
as their basis (Garcia Robles et al. 2016). They can be organised in many
different ways and involve various methods, while characterised by an overall
co-creative approach. Input from research is deemed to be useful, relevant and
applicable in practice. While clearly influenced philosophically from pragmatism,
the need of incorporating rigour research should not be downplayed (Mulgan
2015).

The application of social living labs usually emphasises power relations and
evolves as creative organic/dynamic processes (Hughes, Wolf & Foth 2017,
Scholl & Kemp 2016), as parts of a neverending history of development
(Sørensen & Torfing 2015). The products from social innovations consist of ideas
addressing social needs, which are translated into practice, implemented and

106
result in changes (Nicholls, Simon & Gabriel 2015). They can be portrayed as:
“a conflict-ridden attempt to find joint solutions to shared problems through
provisional and disputed agreements” (Sørensen & Torfing 2015:155). The goal
is to develop innovations with high degree of acceptance and credibility among
involved parties (Scholl & Kemp 2016), at the same time as previous research
has shown that innovations are likely “to create value for some and destroy it for
others” (Nicholls, Simon & Gabriel 2015:5).

The expansion of a public social innovation agenda has been understood as an

advancement of New Public Management (NPM). Previous trust in NPM,
individual entrepreneurial capacity or designated development units might have
enhanced efficacy to some point but has also increased fragmentation, distrust
and auditing and resulted in administrative overload. Innovations have
furthermore seldom been shown to be the result of the efforts of a single actor,
but of the meeting between different actors, organisations and collaboration
(Sørensen & Torfing 2015).

Attention has begun to turn to New Public Governance (NPG) and towards
innovation processes through multi-actor, and inter-organisational collaboration
in networks and through partnership. The underlying ideas are based on
collaboration around knowledge, ideas, resources and praxis and through this
enhance trust, stimulate mutual learning and innovation processes, not on the
present salience given to competition. The facilitation from an adaptive,
pragmatic, distributive, horizontal and integrative leadership with focus on meta-
governance has been pointed out as helpful in such processes (Sørensen &
Torfing 2015).

One strength with understanding the interactive research in focus of this paper
in terms of a public collaborative social living lab is that it puts action in the centre
of the process. Starting in action might be a more successful approach for
upholding motivation, focus and encouragement in necessary developments for
increasing citizen’s wellbeing under rather challenging circumstances. Through
the social living labs methodology, public, private and civil functionaries can
possibly attain executive power, take majority decision on strategies, test
proposed solutions, and sort out those working. The inclusion of different
experiences, forms of knowledge give a creative tension potentially useful for
successful innovative work, when giving structure and process facilitation. The
deliberative solution finding process can result in jointly held strategies that gain
high degree of acceptance and credibility broadly in the communities.

5 Discussion of the researchers’ position

The development described above has placed the interactive or on-going
research at a crossroad. Should it take a critical stance to the change in direction,
pointing out the risks of losing the initial praxis models knowledge base and
creating even more resource demanding, complicated as well as complex
processes then was initially planned for? Should the researchers continue to
contribute to the process by trying to make the practitioners slowing down the
process, and stay at implementing the original praxis model? This would make

107
the process more rational through adapting a predetermined work process that
all actors follow. A more manageable implementation could then be carried out
through pilots.

An alternative is however to take a more critical stance towards the initial planed
implementations process and interactive research, understanding it as not being
dynamic, integrated, collaborative as well as efficient enough to meet the
complex and changing challenges in society and practice. In order to enhance
success in implementation processes in current welfare systems in Europe, it
can be argued that there is a need to build on historical learning in the new
context. When a new way of working is to be established, there will necessarily
be processes both of a) understanding the model and the implementation soil,
b) initiate common knowledge-processes both aiming to learn the new and de-
learn malfunctioning routines, c) construct the new and de-construct various
hinders. These implementation processes will by necessity involve social
innovation processes and be traceable back to several knowledge sources and
historical development processes. In the welfare sector, such a process is
furthermore to be done under a situation where citizens as well as politicians,
managers and practitioners might be of the opinion that there is no time and not
enough resources to carry out a slower and firmly established development
process, which starts in examinations and thinking, before action and finely
hopefully positive outcome.

But if the implementation is understood as a public collaborative social living lab

– what position do we as researchers see as most fruitful within it? Should we
continue by following the process less co-creative, being prepared to both inspire
and disturb the development with external scientific perspectives and knowledge
as well as describing and analysing failures and successes? This approach
would make us researchers to maintain an independent and critical role, even
when we are involved in the development processes and closely acquainted to
the participants. It could help in keeping focus on long-term results, when
organisations work under the pressure of efficiency. The researcher would not
put all their energy in practitioners’ current interests or in increasing the credibility
giving for politicians. The sight would not be narrow down focussing on low
hanging apples in a strive to enhance efficiency and positive short-term outcome
to the social problems highest on the political agenda of today.

6 Conclusions
The integration of social living lab methodology and ongoing evaluation could
potentially embrace a pragmatic methodological in research, feeding the process
with knowledge and research activities that involved parties find to be useful,
relevant and applicable for enhancing quality and efficiency in the public sector.
The question then turns into how researchers can be part of public collaborative
social living labs in a way so that they become as efficient as possible, at the
same time as they secure multi-actor and inter-organisational collaboration on
an innovative as well as knowledge-based foundation.

108
Rather than focusing on describing and analysing problems in the process, to
examine the processes of finding and experimentation of solutions within
contextual limits and challenges becomes central. Such examination includes
considering the value of feeding and disturbing the development process with
scientific perspectives and knowledge. The researcher must always be aware of
his/her role of contributing from a relative alien position.

It furthermore becomes important to make sure that research capture different

stakeholders’ wishes and views. The critical scientific approach will still have to
include the articulation of contradictions, differences in interest, conflicts and
ethical dilemmas when needed. There is also a need for posing question about
what the “new” in the creative processes running; asking for verification for un-
justified claims; investigate the theoretical base for different innovations; and also
to strictly evaluate its short run outcome and search for long terms results.

However, there seems to be no good reason for letting some actors focus on
problems and others on solutions. That is one important reason why to look for
this integrated model between on-going evaluation and the social living lab.
Wellbeing of children, or anyone for that sake, would be helped by deep
collaboration holistic views and of joint and never-ending problem formulations,
articulation of objectives and knowledge-generating processes.

The early implementation process in Sweden has made the connected

interactive research change from a more rational implementation model to focus
on how to incorporate collaborative social living lab theory and methodology. The
next question to consider will be if, and in that case how, this changed
understanding of the overall development process also calls for changed
methods planned for or already put into action through the early interactive
research design.
Appendix 1: The research questions in the qualitative
interviews
1) Describe how BBiK has evolved? What key events have led to
here you are today?
2) What are the strengths and weaknesses with the current
content of BBiK?
3) How does BBiK related to Scotland's praxis model in GIRFEC?
4) What difficulties and enabling factors can you see in the history
of developing BBiK?
5) What strategies do you have to manage obstacles that expect
for the future?
6) How do the local and regional development process relate to
each other in BBiK?
7) What thoughts do you have about how you want to cooperate
with the academy in the continuing process?

109
References
Alexanderson, K. (2006). Om evidens i socialt arbete: möjligheter och
begränsningar för ett evidensbaserat arbetssätt inom socialtjänsten.
[About evidence in social work: opportunities and limitations for an
evidence-based approach in the social services]. Falun: Dalarnas
forskningsråd.
Bringselius, L. (ed.) (2018). Att styra och leda med tillit. Forskning och praktik. [To
manage and lead through trust. Research and practice]. Stockholm:
Tillitsdelegationen.
Coles, E., Cheyne, H., Rankin, J., & Daniel, B. (2016). Getting It Right for Every
Child: A National Policy Framework to Promote Children's Well‐being in
Scotland, United Kingdom. Milbank Quarterly, 94(2), 334-365.
Danermark, B. & Kullberg, C. (1999). Samverkan. Välfärdsstatens nya
arbetsform. [Cooperation. The new working method in Welfare states].
Lund: Studentlitteratur.
Darlington, Y. & Feeney, J. A. (2008). Collaboration between mental health and
child protection services: Professionals' perceptions of best practice. In
Children and Youth Services Review, 30(2), 187-198.
Garcia Robles, A., Hirvikosky, T., Schuurman, D. & Stokes, L. (2016). Introducing
ENoLL and its living lab community, European Network of Living Labs
Brussels
Gieryn, T. F. (1983). Boundary-work and the demarcation of science from non-
science: Strains and interests in professional ideologies of scientists. In
American Sociological Review, 48(6), 781-795.
Greener, I. & Greve, B. (2014) Evidence and evaluation in Social Policy.
Broadening Perspective in Social Policy. Wiley-Blackwell
Grefve, C. (2017). Barnets och ungdomens reform. Förslag för en hållbar framtid.
Slutrapport från den nationella samordnaren för den sociala barn- och
ungdomsvården. [The reform for children and youth. Suggestions for a
sustainable future. Final report from the National Coordinator for Social
Child and Youth Care]. Government Offices, Ministry of Social Affairs
Hughes, H., Wolf, R., & Foth, M. (2017). Informed digital learning through social
living labs as participatory methodology: The case of Food Rescue
Townsville. In Information and Learning Science, 118(9/10), 518-534.
Huxham, C. & Vangen, S. (2005). Managing to collaborate: the theory and
practice of collaborative advantage. London/New York, NY: Routledge.
Julian, D.A., Jones, A. & Deyo, D. (1995). Open Systems Evaluation and the Logic
Model: Program Planning and Evaluation tools. In Evaluation and Program
Planning, 18(4), 333-341.
Lamont, M. & Molnár, V. (2002). The study of boundaries in the social sciences.
In Annual review of sociology, 28(1), 167-195.

110
Logan, J. & Graham, I. D. (1998) Model of Health Care Research Use Model of
Health Care Research Use. In Science Communication, 20(2), 227-246
Lundquist, L. (1987). Implementation Steering. An Actor-Structure Approach.
Lund: Studentlitteratur
Mulgan, G. (2015) Foreword: The Study of Social Innovation – Theory, Practice
and Progress. In Nicholls, A., Simon, J. & Gabriel, M. (eds.) New Frontiers
in Social Innovation Research. Palgrave Macmillan
Nicholls, A., Simon, J. & Gabriel, M (2015) Introduction: Dimensions of Social
Innovation. In Nicholls, A., Simon, J. & Gabriel, M. (eds.) New Frontiers in
Social Innovation Research. Palgrave Macmillan
Nowotny, H., Scott, P. & Gibbons, M. (2001). Re-thinking science: knowledge and
the public in an age of uncertainty. Cambridge: Polity Press.
Peterson (2015). Personalizing Education at Scale: Learning from International
System Strategies. Cambridge, Massachusetts: Harvard Graduate School
of Education.
Pressman, J. & Wildavsky, A. (1983). Implementation. Berkley: University of
California.
Robertson A. (2011) The Fork in the Road: A Scottish Perspective on Relationship
Building as a Catalyst toward Prevention in Child Welfare [dissertation].
Urbana- Champaign: University of Illinois.
Rogers, P. J. (2008). Using programme theory to evaluate complicated and
complex aspects of interventions. In Evaluation, 14(1), 29-48.
Scholl, C. & Kemp, R. (2016). City Labs as Vehicles for Innovation in Urban
Planning Processe. In Urban Planning, 1(4), 89-102
Sjöberg, K., Brulin, G. & Svensson, L. (2009) Lärande utvärdering – följeforskning:
En syntes. [On-going evaluation – interactive research: A synthesis] In
Sjöberg, K. (ed.) Lärande utvärdering genom följeforskning. [On-going
evaluation through interactive research]. Lund: Studentlitteratur
Stafford, A., Vincent, S. & Morgan-Klein, N. (2008) Child protection: an overview.
In Stafford, A. & Vincent, S., (eds). Safeguarding and Protecting Children
and Young People. Edinburgh, United Kingdom: Dunedin Academic Press
SIDA (2006) Logical Framework Approach – with an appreciative approach. [Sida
is a government agency working on behalf of the Swedish parliament and
government, with the mission to reduce poverty in the world]
Svensson, L., Ellström, P.-E., & Brulin, G. (2007). Introduction – on interactive
research. In International Journal of Action Research, 3(3), 233-249
Sørensen, E. & Torfing, J. (2015) Enhancing Public Innovation through
Collaboration, Leadership and New Public Governance. In Nicholls, A.,
Simon, J. & Gabriel, M. (eds.) New Frontiers in Social Innovation
Research. Palgrave Macmillan
The National Agency for Education and the National Board of Health and Welfare
(2018). Uppdrag att genomföra ett utvecklingsarbete för tidiga och

111
 samordnade insatser för barn och unga. Delredovisning. [Assignment to
develop early and coordinated efforts for children and youth. Sub-report]
The Swedish Agency for Public Management (2014). Evidensbaserad praktik
inom socialtjänsten. Utvärdering av överenskommelsen mellan regeringen
och SKL. Slutrapport. [Evidence-based practice in the social services.
Evaluation of the agreement between the Government and the Swedish
Association of Local Authorities and Regions. Final report]
Weiss, C. (1979) The many meanings of research utilization. In Public
Administration Review 3, 426-31
Young Foundation (2012) Social innovation Overview – Part 1: Defining Social
Innovation. A deliverable of the project: “The theoretical, empirical and
policy foundations for building social innovation in Europe” (TEPSIE).
Brussels: European Commission, DG Research.

112
Health and
Wellbeing

113
114
 Co-creating innovative tools with and for people
with Intellectual Disabilities: The case of
DS Leisure e-Training Platform
Maria Metaxa1, Foteini Dolianiti1, Ioanna Dratsiou1,
Evangelia Romanopoulou1, Dimitris Spachos1, Theodore
Savvidis1, Vasiliki Zilidou1, Maria Karagianni1 and
Panagiotis Bamidis1

1 Lab of Medical Physics, School of Medicine, Faculty of Health Sciences,

Aristotle University of Thessaloniki, Greece

Category: Research in-progress

Abstract
Participation is a key factor and a central concept when considering interventions
for supporting people with Intellectual Disabilities (ID). Building upon the
potentialities of the Long Lasting Memories Care service -an integrated ICT
platform for cognitive and physical training- for enhancing the daily performance
of people with Down Syndrome (DS), and taking into consideration the
importance of participatory design for removing the barriers that undermine
inclusion, DS Leisure - an e-Training Program for improving Quality Of Life
Through Inclusive Leisure- was introduced by Thessaloniki Active and Healthy
Ageing Living Lab (Thess-AHALL). The present work presents the design,
development and validation steps of the DS Leisure, where all Thess-AHALL
actors -including people with DS and ID, their families, educators and Lab of
Medical Physics- were actively involved in the process as co-creators. The aim
of this paper is to evaluate the usability and adequacy of the Games and Virtual
Scenarios included in the e-Training platform, as tools for training people with
DS and other IDs on inclusive leisure. Eleven specialists with previous
experience in the field participated in the survey. Findings suggested that both
Games and Virtual Scenarios are easy to use, consistent and provide a sense
of control to the user, although support from a trainer may be required.
Additionally, the adequacy of the Games and Virtual Scenarios was positively
rated, as they were considered to be constructive and interactive educational
experiences for everyday skills enhancement. Overall, the positive feedback
received sets the ground for further research to reveal whether designing with-
and not just for- people with ID is an approach that results to successful
outcomes.

Keywords: assistive technology, human- centered design, participatory, co-

creation, intellectual disability, inclusion

115
1 Introduction
Intellectual Disability (ID) is characterized by significant limitations in both
intellectual functioning and adaptive behavior, with the latter covering a wide
range of everyday social and practical skills (American Association on Intellectual
and Developmental Disabilities, n.d.). Among the genetic causal factors related
to ID, Down Syndrome (DS) is the most common one, affecting more than 5
million people worldwide (ECNP, n.d.). DS is a chromosomal abnormality that is
associated with consequences in the areas of cognitive, linguistic, speech-motor
and social development (Chapman & Hesketh, 2000).

Many studies have empirically supported the role of social relations and active
leisure experiences in happiness and well-being (Holder & Coleman, 2009;
Holder, Coleman & Sehn, 2009). Although leisure has been associated with
emotional and psychological benefits for people with ID (Williams & Dattilo, 1997;
Caldwell & Gilbert, 1990), leisure participation for people with DS is restricted to
solitary and sedentary activities (Oates, Bebbington, Bourke, Girdler, & Leonard,
2011). This suggests the need for ongoing support of people with DS and other
IDs to develop and enhance the skills necessary for effective use of leisure time
and to facilitate full inclusion (Buttimer & Tierney, 2005).

1.1 Assistive technology for Inclusion

It is a fact that barriers in performing daily activities can be reduced or removed
by the use of Assistive Technology (AT) (Reed, 2007). Such technologies as
computer games or computer-based activities enable people with ID to make use
of their leisure time and to participate in social activities (Erdem, 2017). In regard
to people with DS, AT contribution is crucial in independent living and could
contribute significantly in building self-confidence, independence as well as in
achieving high quality of life (Reed, 2007; Feng, Lazar, Kumin & Ozok, 2010).
Commonly, the integration of serious games in learning can be an effective
approach for people with ID, as long as these games are specifically designed
considering their unique characteristics and cognitive abilities (Cano, Fernández-
Manjón & García- Tejedor, 2018).

1.2 Living Lab, Participatory design and Co-creation

Living Labs are user-centered experimentation environments where users are
part of the innovation’s co-creation and co-production process (Ballon, Pierson &
Delaere, 2005). This approach advocates that “the user is not simply a source of
information or evaluator of the final product, but an active contributor of design
ideas and a decision-maker in the process, often referred to as “co-creator” or
“co-designer” (Sanders & Stappers, 2008).

Regarding the Living Labs operation, it is important that the roles of each
participant- actor are distinct. In particular, the involved actors are: (a) providers,
who “enter into Living Labs networks to co-develop new products”, (b) users
including “both current and potential clientele of products and services” (c)
utilizers, namely, “non-producers that seek efficiency gains and new knowledge
from the Living Labs” and, (d) enablers, namely, “organizations that provide
supportive technology, and other necessary resources to the use of participants”
(Leminen & Westerlund, 2012).

116
In this vein, participatory design has been implemented as an approach that can
help remove the barriers that undermine inclusion of people with DS when
technology- enhanced learning tools are designed without accessibility in mind
(Buzzi, Buzzi, Perrone et al., 2018). For example, Engler and Schulze (2017)
involved people with DS, their families and trainers in the POSEIDON project,
which aimed to increase their independence and autonomy by providing support
in the areas of time management, mobility and money handling, and as it is
proved, there is great benefit to increase inclusion of people with DS into society.

2 Previous work of Thess-AHALL

Thessaloniki Active and Healthy Ageing Living Lab (Thess-AHALL) (aha-
livinglabs.com), operating since 2014, is the Living Lab governed by the
Laboratory of Medical Physics, Medical School of the Aristotle University of
Thessaloniki (medphys.med.auth.gr). The lab’s main strength lies with the health
and social conditions’ improvement and independent living facilitation, by actively
pursuing co-creation and co-design with end-users and relevant community
stakeholders.

In this manner, co- created work of Thess-AHALL includes Long Lasting

Memories Care (LLM Care) service – an integrated ICT tool- which combines
state-of-the-art mental exercises with physical activity. In this context, wFitForAll
platform has been developed by the Lab including the physical training exercises
(Bamidis et al., 2015). In contrast with mono- therapeutic interventions, LLM Care
suggests a holistic approach against cognitive deterioration. Initially, the LLM
Care service was exploited to improve or maintain the quality of elderly’s people
life. Bamidis and his colleagues (2015) performed a LLM Care- based
intervention to 322 older adults aged ≥55 years with or without neurocognitive
disordered (NCD). The results of the intervention showed significant
improvement to participant’s global cognition (p= 0.002).

Following this, LLM Care service appears to be a vital tool to people with DS and
other IDs providing important potential towards health, social and daily living
improvement (Romanopoulou, Zilidou, Savvidis, et al., 2018).

In this direction, Thess-AHALL co-creation network was exploited for the

“Training Program for Improving Quality of Life through Inclusive Leisure for
Persons with Down Syndrome”, called DS Leisure Project aims at increasing DS
peoples’ competences in terms of attitudes, skills and knowledge enhancing their
daily performance. This project is based on the wFitForAll Platform and includes
training Games and Virtual Scenarios (VS).

Table 1. DS Leisure design steps and how they were applied.

Design steps Involved Actors How this step was applied in the project

117
 Users:
People with DS

Utilizers: Participants: 46
Organizations for wFitForAll Platform of LLM Care –a
Step 1: people with Thess-AHALL’s service- appears that may
LLM Care disabilities, Educators enhance health and daily living skills of
implementation & Parents people with Down Syndrome
(Romanopoulou et al., 2018).
Enablers:
Lab of Medical
Physics (AUTH)

Users:
People with DS Participants: 22
In the context of LLM Care’s new
Utilizers: innovation services, Thess- AHALL
Organizations for organized a Co-Creation Session in
Step 2: people with Thessaloniki by actively involving Users
Co-Creation Working disabilities, Educators and Utilizers into the design and selection
Session & Parents process of the Training Materials,
Assistive Technology, Contents and
Enablers: Methods of the DS Leisure Project in the
Lab of Medical wFitForAll.
Physics (AUTH)
Step 3: Based upon the Users need emerged
Design and Enablers: from the Co-Creation Working Session,
Development of the Lab of Medical design and development of the Training
Training Materials Physics (AUTH) Materials and the DS Leisure e-Training
and the DS Leisure Platform were carried out on the
e- Training Platform wFitForAll Platform by the Enablers.
Users:
People with DS
Participants: 14
Utilizers: At the Co-Validation Working Session, the
Organizations for Training Materials were presented and,
Step 4:
people with through participatory and interviews, all
Co-Validation of the disabilities, Educators participants (Users, Utilizers and
Training Materials & Parents Enablers) discussed with regard to the
Activities extracted from the Co-creation
Enablers: Working session, such as the content and
Lab of Medical structure of Methodological Guide.
Physics (AUTH)
Participants: 11
Step 5:
Finally, specialists were asked to fill in a
Validation of the e- Utilizers: questionnaire regarding the usability
Training Platform Specialists and adequacy of the Games and
(Games & Virtual
Virtual Scenarios of the DS Leisure e-
Scenarios)
Training Platform.
Considering the importance of the participatory design, the involved actors and
the steps of the DS Leisure Project design process by Thess-AHALL are
described in Table 1 above.

118
3 Purpose of the study
Among the five steps presented in the previous subsection (see Table 1), this
work mainly focuses on the validation of the Games and Virtual Scenarios (VS)
of the DS Leisure e-Training Platform (step 5). In particular, this study focuses on
the usability and adequacy of the Games and VS as tools for training people with
DS and other IDs on inclusive leisure.

4 Materials & Methods

4.1 Participants
In order to validate the e-Training Platform (Games and VS), 11 participants took
part in the survey (10 female and 1 male). All of them were specialists with
previous experience in working with people with ID and came from a range of
professional fields, such as psychologists, special educators and LLM Care
trainers.

4.2 Games and Virtual Scenarios

The results of the Co-Creation session have highlighted the need to enhance
users’ memory performance and money management, as well as to get them
aware of the everyday life challenges. Based upon these specific needs, DS
Leisure e-Training Platform was enriched with two Games and ten VS, for which
specialists’ validation considered necessary.

In the first game, called the “Memory Game”, the user is asked to invert upside-
down cards displaying different leisure activities (e.g., cinema, party) and find
pairs of matching images. In the second game, called the “Money Game”,
banknotes and coins appear on the screen and the player has to make the right
selection in order to reach a specific amount of money and fill his or her wallet.
VS represent daily living leisure activities, such as going to the cinema or a
concert, making a restaurant reservation, organizing a party, watching a football
game, using the public transportation etc. Here, the goal is for the user to choose
among different options and follow the path of the right decisions in order to
correctly fulfil the activities.

119
4.3 Assessment & Procedure
The questionnaire comprised of both closed-ended and open-ended questions
was devised in order to assess the usability and adequacy of the Games and VS
included in the e-Training Platform. In the closed-ended questions, participants
were asked to determine in a 5-point Likert scale their degree of agreement to a
set of statements regarding different usability and adequacy dimensions. In the
open-ended questions, participants were asked to provide feedback regarding
the strong and weak features of the Games and VS as well as their suggestions
for improvement.

The questionnaire was administered through an online survey tool with an

introductory e-mail. The e-mail explained the general purpose of the study and
the particular goals of the Games and the VS. It also contained the online links to
the e- Training Platform and the online survey. Participants were asked: a) to visit
the e- Training Platform and play the Games and the VS, b) to fill in the
questionnaire.

5 Results & Discussion

5.1 Validation of the Games
This subsection presents the results regarding the usability and adequacy of the
games included in the e-Training platform. As Figure 1 shows, perceived ease of
use was rated as equal or above neutral (i.e., 3) with the majority expressing an
agreement (n = 5) or a strongly agreement (n = 3) that games were easy to use.
Most respondents (n = 6) strongly believed that games were consistent and
agreed that people with ID feel in control over the games. On the other hand,
respondents did not express a clear view as to whether people with ID need the
support of a trainer to use the games, and the majority (n = 7) gave a neutral
rating.

Figure 1. Responses regarding with the Games usability by people with DS and other IDs.

120
Regarding game adequacy, on the whole, respondents gave positive ratings, as
shown in Figure 2. The majority (n = 8) argued or strongly argued that the games
serve the special needs of people with ID, they enhance the performance of
people with ID and they offer direct interaction. Additionally, with the exception of
two respondents, feedback appropriateness was considered to be above neutral.

Figure 2. Responses regarding with the Games adequacy.

Among the open-ended responses, strong game features were identified by all
11 participants. Specifically, the entertaining character of the games was
mentioned by the majority (n = 8), followed by the visual stimuli (i.e., images,
colors) (n = 3), the learning goals (n = 3) (e.g., “people can learn how to organize
activities”) and the interactivity (n = 2). Self-paced training (n = 1), ease of use (n
= 1) and feedback provision (n = 1) were, also, identified as positive game
attributes. On the other hand, game weaknesses were mentioned to a lesser
extent. Participants (n = 3) emphasized on the lack of auditory stimuli -namely,
sounds, music, audio feedback and audio instructions- while there existed some
doubts regarding the ease of understanding (n = 2). Short game-play duration (n
= 1) and lack of different difficulty levels (n = 1) were also mentioned.

5.2 Validation of the Virtual Scenarios

Considering the VS usability (Fig. 3), a contradictory result is that they are not
considered to be difficult for people with ID (n = 9), though support of the trainer
appears to be still recommended by the evaluators (n = 6). Furthermore, most of

121
them argue that VS are not lacking in consistency (n = 8), as such give full control
to the end- user (n = 8).

Figure 3. Responses regarding with the Virtual Scenarios usability by people with DS and
other IDs.

As shown in Figure 4, most of the participants agree or strongly agree that VS

respond to everyday living (n= 8), as well they are constructive educational
experience (n= 9). In fact, everyday skills as money handling and use of public
transportation were both rated as equal or above neutral (Fig. 4).

Regarding the open-ended responses about the VS, the evaluators mentioned
content correspondence to the reality (n = 3), decision- making enhancement (n
= 2) and design simplicity (n = 2). On the other hand, the lack of enough auditory-
visual stimuli has been identified as weakness, as it is possible to make the VS
hard to understand (n = 3). Hence, the support of an educator is claimed to be a
necessity (n = 2).

Overall, the results from the specialists’ evaluation revealed some interesting
insights. Among the open-ended responses, both positive and negative
suggestions were reported. Moreover, most of the qualitative responses argue
that Games and VS were accessible, pleasant and easy to use by people with
disabilities (n = 6).

122
 Figure 4. Responses regarding with the Virtual Scenarios adequacy.

Additionally, no technical obstacles were reported by the facilitators during their

experience with the e- Training Platform (n = 11). Besides that, educational
contribution of the games and the VS to social inclusion considered to be a
significant advantage (n = 6), as they “enhance skills targeted to the needs of
people with disabilities for autonomy and social engagement” and “support
everyday living”. Finally, suggestions such as graded difficulty of levels (n = 3)
and auditory provision of instructions (n = 2) were stated.

6 Conclusion
It appears that the technology can significantly contribute to the process of
providing everyday skills to people with ID. However, in contrast with most of
digital games and web-based applications for people with DS, which are designed
with no consideration of their special needs (Feng et al., 2010), this study
innovation lies with the human-centered design (Maguire, 2001; Sanders &
Stappers, 2008; Feng et al., 2010) followed by Thess-AHALL. As it was
considered crucial to this survey considering not only the obstacles to be faced,
but also the special needs of the people with ID, Thess-AHALL actively involved
people with ID in the design process by choosing the Training Materials, AT tools,
Contents and Methods of DS Leisure e- Training Platform.

In addition, the special of this survey as opposed from previous approaches is

the objective of the e- Training Platform itself which focuses on leisure activities
that are yet considered to be restricted for people with DS (Oates et al., 2011),
as well as the original material that was developed in a Living Lab network for this
purpose.

123
Furthermore, this work presented the dynamic process that led to the DS Leisure
e- Training Platform, offering an example of how sustainability –a key principle of
Living Labs (Bergvall- Kåreborn, Eriksson, Ståhlbröst & Svensson, 2009) - can
be achieved. Taking into consideration that innovation and evolution are two inter-
dependent phenomena (Yurchyshyna, Khadraoui, Opprecht & Léonard, 2011),
Thess- AHALL continuously builds upon and enriches the accumulated
knowledge, by bringing new features to the LLM Care service and deploying its
components (i.e., wFitForAll platform) to develop new products (i.e., DS Leisure
e-training Games). Specifically, beginning with the Living Lab’s LLM Care service
implementation to elderly people that subsequently applied to persons with DS
and ID, DS Leisure Project co-creation followed.

Regarding the results of this study, the validation of the Games and the VS by
specialists revealed successful outcomes. Although, there is a remaining
question concerning the positive effect –whether it is affected by the way the skills
were trained or due to the fact that the Training Materials were co-created with
the people with DS- that is yet not clear. In our point of view, both factors are
considered to have been equally crucial to this matter, but more research is
needed to clarify this question. In view of that, further research is also in the
process of investigating whether or not autonomy and quality of life of people with
ID may be enhanced by the DS Leisure e- Training Platform.

Acknowledgements
This research was supported by DS LEISURE project funded by the European
Commission within the ERASMUS+ 2017 Programme, as well as the business
exploitation scheme of the ICT-PSP funded project LLM, namely, LLM Care
which is a self-funded initiative at the Aristotle University of Thessaloniki
(www.llmcare.gr).

References
American Psychiatric Association (APA) (n.d.). What is Intellectual Disability?
Retrieved at 30 May 2019 from: https://www.psychiatry.org/patients-
families/intellectual-disability/what-is- intellectual-disability
Ballon, P., Pierson, J. & Delaere, S. (2005). ‘Open innovation platforms for
broadband services: benchmarking European practices’, Proceedings of
the 16th European Regional Conference, Porto, Portugal.
Bamidis, P. D., Fissler, P., Papageorgiou, S. G., Zilidou, V., Konstantinidis, E. I.,
Billis, A. S., ... & Tsilikopoulou, G. (2015). Gains in cognition through
combined cognitive and physical training: the role of training dosage and
severity of neurocognitive disorder. Frontiers in Aging Neuroscience, 7,
152.
Bergvall -Kåreborn, B., Eriksson, C. I., Ståhlbröst, A., & Svensson, J. (2009). A
milieu for innovation: defining living labs. In ISPIM Innovation Symposium:
06/12/2009-09/12/2009.

124
Buttimer, J., & Tierney, E. (2005). Patterns of leisure participation among
adolescents with a mild intellectual disability. Journal of intellectual
disabilities, 9(1), 25-42.
Buzzi, M. C., Buzzi M., Perrone E., & Senette C. (2018). Personalized
technology- enhanced training for people with cognitive impairment.
Universal Access in the Information Society, 1–17.
https://doi.org/10.1007/s10209-018-0619-3
Caldwell, L. L., & Gilbert, A. A. (1990). Leisure, health, and disability: A review
and discussion. Canadian Journal of Community Mental Health, 9(2), 111-
122.
Cano, A.R., Fernández- Manjón, B., & García- Tejedor, Á. J. (2018). Using game
learning analytics for validating the design of a learning game for adults
with intellectual disabilities. British Journal of Educational Technology,
49(4), 659–672.
Chapman, R. S., & Hesketh, L. J. (2000). Behavioral phenotype of individuals
with DS. Mental Retardation and Developmental Disabilities Research
Reviews, 6(2), 84 – 95. https://doi.org/10.1002/1098-
2779(2000)6:2%3C84::AID-MRDD2%3E3.0.CO;2- P
ECNP (n.d.). Down Syndrome and Other Genetic Developmental Disorders
Network. Retrieved at 1 June 2019 from: https://www.ecnp.eu/research-
innovation/ECNP- networks/List-ECNP-Networks/Down-syndrome
Engler, A., & Schulze, E. (2017). POSEIDON - Bringing Assistive Technology to
People with Down Syndrome: Results of a Three Year European Project.
Studies in health technology and informatics, 236, 169-175.
Erdem, R. (2017). Students with Special Educational Needs and Assistive
Technologies: A Literature Review. The Turkish Online Journal of
Educational Technology, 16(1), 128–146.
Feng, J., Lazar, J., Kumin, L., & Ozok, A. A. (2010). Computer Usage by Children
with Down Syndrome: Challenges and Future Research. Journal ACM
Transactions on Accessible Computing (TACCESS), 2(3), 1-
44. https://doi.org/10.1145/1714458.1714460
Holder, M. D., & Coleman, B. (2009). The contribution of social relationships to
children’s happiness. Journal of happiness studies, 10(3), 329-349.
Holder, M. D., Coleman, B., & Sehn, Z. L. (2009). The contribution of active and
passive leisure to children's well-being. Journal of health psychology,
14(3), 378-386.
Leminen, S., & Westerlund, M. (2012). Towards innovation in Living Labs
networks. International Journal of Product Development, 17(1-2), 43-59.
Maguire, M. (2001). Methods to support human- centred design. International
Journal of Human-Computer Studies, 55, 587-634.
doi:10.1006/ijhc.2001.0503

125
Oates, A., Bebbington, A., Bourke, J., Girdler, S., & Leonard, H. (2011). Leisure
participation for school-aged children with Down syndrome. Disability and
rehabilitation, 33(19-20), 1880-1889.
Reed, P. (2007). A resource guide for teachers and administrators about assistive
technology. Oshkosh: Wisconsin Assistive Technology Initiative.
Romanopoulou, E., Zilidou, V., Savvidis, T., Chatzisevastou, L. C., & Bamidis, P.,
(2018). Unmet Needs of Persons with Down Syndrome: How Assistive
Technology and Game- based Training May Fill the Gap. In Hasman et al.
(Eds.) Data, Informatics and Technology: An Inspiration for Improved
Healthcare, IOS Press, doi:10.3233/978-1-61499-880-8-15
Sanders, E., & Stappers, P.J. (2008). Co-Creation and the New Landscapes of
Design, CoDesign, 4 (1), 5- 18.
Williams, R., & Dattilo, J. (1997). Effects of leisure education on self-
determination, social interaction, and positive affect of young adults with
mental retardation. Therapeutic Recreation Journal, 31(4), 244-258.
Yurchyshyna, A., Khadraoui, A., Opprecht, W., & Léonard, M. (2011). Innovation
and evolution of services: role of initiatives. In: Tambouris E., Macintosh
A., de Bruijn H. (eds) Electronic Participation. Lecture Notes in Computer
Science, vol 6847 (pp. 262-273). Springer, Berlin, Heidelberg.

126
 Creating an anonymous, at-home screening for
sexually transmitted diseases sent by letter mail:
the cross-border development of a standardized
urine collection device and associated testing
service
Judith H.J. Urlings1,2*, Bianca Ceccarelli1,2, Claire A.G.J.
Huijnen3, Paulette J.J. Wauben3, Joke Donné4, Ronald Van
den Bossche4, Alejandra Rios-Cortes4, Koen Beyers4 and
Vanessa Vankerckhoven4

*Corresponding author
1 Happy Aging | LifeTechValley, Diepenbeek, Belgium
2 Hasselt University, Hasselt, Belgium
3 Centre of Expertise for Innovative Care and Technology (EIZT) at Zuyd

University of Applied Sciences, Heerlen, The Netherlands

4 Novosanis, Wijnegem, Belgium

Category: Innovation Paper

Abstract
Novosanis' Colli-Pee is a unique device for standardized and volumetric self-
collection of the first-void fraction of urine. The aim of the present living lab
project was to optimize the design of the Colli-Pee for postal mailing. Novosanis
wishes to combine the Colli-Pee device with a sexually transmitted infection (STI)
testing service via letter mail. Possible fitting of the product and associated
service in the existing health care systems of Belgium and The Netherlands was
explored by interviewing professional stakeholders. Additionally, a co-creation
session and large scale live-test were performed with voluntary end-users in both
countries. Both the outcomes of the innovation project as well as learnings for
the living labs are discussed in this paper.

Keywords: Cross border, Healthcare innovation, Product design, Business

model design, Sexual health care

127
1 Introduction
The first-void or first-catch urine (first 20 ml of the urine flow) is being used more
and more for the detection of sexually transmitted diseases (STIs). A large
research project from The Netherlands has shown that the prevalence of
chlamydia can be significantly reduced by active screenings, as larger groups
can be included in this type of screening (van den Broek et al., 2010). Online STI
self-tests allow for home-based sampling and returning the sample kit to the lab
via letter mail. This anonymous and non-invasive procedure might help patients
to overcome feelings of shame or stigma, and might in turn lead to increased
participation in screenings (von Karsa et al., 2015).

Typically, a standard urine cup is used and the patient is asked to collect first
void. However, this cup does not allow collection of a standardized first-void urine
fraction. The first-void of urine can easily be missed, or the interruption of the
urine stream is not successful. Additionally, the traditional urine containers do
not allow efficient letter mail, except as a postal package. Novosanis' Colli-Pee
is an unique device for standardized and volumetric self-collection of the first-
void fraction of urine. It is a patented technology that automatically separates the
first-void from the mid-stream urine without active user involvement.

Novosanis wished to optimize their Colli-Pee urine detection device with respect
to user-experience as well as letter mail delivery. The aim of the present
project was to design a new generation of the Colli-Pee device that is
optimized in architecture and materials to be optimally suited for postal
mailing. Novosanis wishes to combine the Colli-Pee device with an STI-
testing service via letter mail. Possible fitting of the product and associated
service in the existing health care system is explored by interviewing professional
stakeholders. Both Belgian and Dutch end-users and stakeholders where
involved in various stages of the design process:
● Co-creation sessions
o to gain knowledge on end-users wishes, needs, barriers and habits to
STI screening.
o to have various alternative Colli-Pee 3D print prototypes evaluated by
end-users.
● Live-tests:
o to gain knowledge on end-user experience in the process of ordering,
using and returning the sample kit.
● Business-model interviews:
o to gain understanding of the current practices in the field of STI
detection and treatment.
o to collect opinions of professional stakeholders on the Colli-Pee
device and associated STI test service via letter mail.

Those stakeholder involvement activities were carried out by two existing Living
Labs: EIZT (Centre of Expertise for Innovative Care and Technology), associated
with Zuyd Hogeschool in the Netherlands and Happy Aging, associated with
Hasselt University in Belgium. Both Living Labs have extensive experience with
like-wise innovation trajectories. Therefore, the network and panels of the Living

128
Labs – carefully curated over the course of several years- were crucial for
carrying out the described project.

The main purpose of the present paper is to disseminate learnings captured from
this cross-border innovation project to the wider community of European living
labs.

2 Methods
2.1 Background
The current innovation project takes place within the framework of CrossCare.
CrossCare is an Interreg Vlaanderen-Nederland project that is ran by six Living
Labs: three located in Flanders (Belgium), three located in The Netherlands.
Small and medium sized companies (SME) can propose an innovation project to
CrossCare. If the project is accepted, the SME receives both financial as well as
living lab support to further develop its product or service and business model.
The typical duration of a CrossCare innovation project is between 12 and 24
months. Every innovation project is coached by two Living Labs: one in the
Netherlands, one in Belgium. In the current project, the existing Colli-Pee device
is further developed and the feasibility of an associated STI screening via letter
mail – known as ‘Pee-post’ is explored.

2.2. Co-creation session

Before the start of the session, all participants signed written informed consent
forms, guaranteeing mutual confidentiality. This means that information about
and input from participants is kept confidential and only shared in an anonymous
format, but also that participants keep information that they receive on product
design and the business model confidential. The co-creation session consisted
of three parts: The Round Robin exercise, getting to know the Colli-Pee and the
User experience/user acceptance quadrant exercise. All parts are described in
more detail below.

2.2.1 Round Robin

The Round Robin is a concept ideation technique in which ideas evolve as they
pass from person to person (Luma Institute, 2012). The method is used to create
fresh new ideas based on shared authorship. Passing a first idea to the next
author can make the idea grow in unexpected ways. In our case, the Round
Robin assignment was built up out of 3 questions:
● Where/how would you apply for an STI-test?
● What would the test look like?
● How would the results be communicated to you?

Every participant started with an empty sheet with the 3 questions. He or she
filled in question one, then passed on the paper to their left-hand neighbor. This
second participant read the answer to question 1 and formulated his answer to
question 2, given the framework outline in answer 1. The process continued until
the three questions were answered. The assignment was followed by a plenary
feedback session.

129
2.2.2. Getting to know the Colli-Pee
As a second step in the co-creation process, the company Novosanis presented
4 different prototypes of the Colli-Pee device. Prototypes differed in terms of
materials used, assembly needed by the user and recyclability. For the sake of
confidentiality, the different prototypes presented are not discussed in detail
here.

The Colli-Pee is typically delivered to the end-user in a standard size cardboard

box that meets the maximum size for letter mail delivery. This box contains the
Colli-Pee, and a plastic sealable bag to wrap the urine sample for delivery to the
laboratory. Participants of the co-creation sessions were also asked to view the
user manual of the Colli-Pee that is presented on the inside of the delivery box
in images. Secondly, the available manual-video is shown to the participants.
Participants are asked to provide feedback, both on the device-prototypes, as
well as on the instruction video.

2.2.3 User experience and user acceptance

The group of participants was presented with one A0 size poster divided into
quadrants. On the horizontal axis ‘user experience’ is represented. This scale
runs from ‘makes the product easy to use’ to ‘makes the product difficult to use’.
On the vertical axis ‘user acceptance’ is represented. This scale runs from ‘is
important for in my choice to use the product’ to ‘is not important for me in my
choice to use the product’.

Eight product characteristics were pre-identified by Novosanis and summed up

below. Every participant individually noted each characteristic on a post-it note
and placed it on the desired quadrant on the poster:
● Pricing of the device and associated service
● The time the total testing procedure takes (application to test result)
● Recyclability of the device and packaging
● Material of the Colli-Pee
● Hygiene while using the device
● Chance of leaking
● Ergonomics of the device
● Ease of use

2.3 Live-test
Eligible participants for the live-test (adults between 18-45 years old) were
recruited through the CrossCare and Living Labs newsletter, social media
accounts and websites of the Living Labs and through personal communication.
Participants were recruited in two gender (male, female) and three age (18-24,
25-34, 35-45) groups. As the Belgian living lab, Happy Aging recruited 60
participants, equally distributed over the six groups. At the point of writing this
paper the Dutch live-test was still on-going. Forty-one participants were already
sent a Colli-Pee parcel (19 male). A total of 36 urine samples were returned, and
34 participants completed the live-test, including the evaluation survey up till
now. No further recruitment is ongoing.

130
In the Belgian part of the study, participants applied for participation through an
online web-form (Google Forms) and gave digital informed consent there. In The
Netherlands, participants were sent an informed consent form via e-mail.
Participants were ensured that no analyses except for volume determination
would be performed on their urine sample. Age, gender and contact details
including postal address was asked from each participant. Every participant gave
consent to receive a urine-collection kit with the Colli-Pee device.

After entrance into the study, participants received a confirmation mail (in
Belgium with a digital link to the informed consent form for safekeeping by the
participant), and a link to the online video-manual on the Novosanis YouTube
channel. The Colli-Pee parcel was sent to the participant by the living lab.

In Belgium, both the living lab and the participant could track their individual
parcel through the online portal of the postal company. Instructions on how to
return the parcel to a postal drop-off point once the Colli-Pee was used were
included in the mail. In The Netherlands, the Colli-Pee parcels were sent in an
envelope especially designed for medical transport. This procedure was adopted
after consulting with the Dutch postal company and did not allow tracking of the
parcel. For returning of the urine sample, Dutch participants could drop the return
envelope in any of the orange mail boxes available in all neighborhoods.

Once the parcel was delivered at the participant, a follow-up mailing was sent
with again a link to the video-manual and a link to the evaluation survey.
Individual participation codes (owned by the living lab researcher) ensured
anonymous participation in the survey. If no response was recorded after 2
weeks, a reminder e-mail was sent. As a last reminder, a phone-call was made
to the participant 5 days after the original reminder.

2.4 Business model interviews

To gain insight on how the Colli-Pee and associated testing service can be
embedded in the current care system, the living labs administered interviews with
professional stakeholders, active in primary care in both The Netherlands and
Belgium. Business model interviews were prepared through the method
described by Albert & van der Auwermeulen (2017).

Interviewees were recruited from the Living Lab partner organizations, both in
Belgium as well as in The Netherlands. To increase the chance of participation,
interviews were conducted at a time and location preferred by the interviewee.
In Belgium, one general practitioner, one pharmacist and a manager from a
healthcare insurance company participated in the interviews. In the Netherlands,
one general practitioner, one pharmacist and one medical doctor working in a
center for sexual healthcare participated.

Due to the confidential nature of these interviews for both the interviewee and
the company, the results of the interviews are not extensively reported in this
paper.

131
3 Results
3.1 Co-creation session
3.1.1 Session results Belgium
The co-creation session was visited by 8 eight participants (3 male), between 18
and 49 years old in Belgium. In the Belgian session, from the Round Robin
exercise we learned that the family physician is considered as the trusted party
to turn to for questions regarding sexual health. Trust in the family physician is
higher than the level trust felt for commercial parties. Although in Belgium a
patient is free to visit another general practitioner than the one that holds a
person’s medical health record, most participants preferred to visit their personal
family doctor.

If participants would buy a test in a physical store (i.e. pharmacist, drugstore),

one would assume that the results are readily available, similar to a pregnancy
test. Participants would prefer a complete test-kit, rather than a separate test for
a single pathology.

The majority of the participants expect that a urine and blood sample are
necessary for a full STI screening. An important insight is that participants expect
to receive extensive additional information from their general practitioner. An
informative conversation should cover both the test-procedure as well as
information on STI´s and reliable preventive measures. Despite the fact that the
family doctor is regarded as the preferred partner in STI testing, the majority of
the participants would prefer to provide a self-sampled urine or a vaginal swab
rather than have the procedure done by the doctor if this was a possibility.

Most participants would prefer to receive the test-results in a face to face

meeting. In case of online communication, the possibility should be given to have
an (online) consultation with a doctor, especially in case of a positive screening.
In such a consultation, treatment, future prevention and the need for notification
of sexual partners should be discussed.

From the user experience and user acceptance exercise, various important
characteristics of the Colli-Pee and associated service were derived. First, the
cost of the total service influences the user acceptance. If the cost exceeds the
cost of a test by the general practitioner, then testing in the doctor’s office is
preferred. Secondly, the duration of completion of the entire procedure was
named as a factor in user acceptance. About half of the participant thinks a
quickly available test results will influence their choice for an online test. Thirdly,
half of the participants think choice of materials and recyclability influenced their
user experience.

3.1.2 Session results The Netherlands

Nine people took part in the Dutch co-creation session (2 male). In the Round
Robin exercise Dutch participants indicate that they would contact their general
practitioner or a center for sexual health for an STI screening. Others would
prefer to buy a test at a drugstore or online. Participants expect that either the
general practitioner, the assistant of the general practitioner or the patient
him/herself would carry out the test. One would expect testing of a blood and

132
urine sample, either in the doctor’s office or in an external laboratory. For a test
bought online or at a drugstore, one would expect immediate results. Participants
have very diverse expectations towards the way they receive the test results.
Various options that are named are: 1. To be contacted by telephone by the
general practitioner, 2. To call a ‘test-results line’ or log in to a ‘test-results
website’, 3. In a physical consultation with the doctor, 4. Directly read the results
of the test.

The user experience vs user acceptance quadrant is filled in per presented

prototype. No large differences on characteristic placement arise between the
four prototypes. Choice of materials and recyclability is important for the user
acceptance. In contrast to the session in Belgium, time needed to complete the
test and costs of the test are not regarded as top 3 characteristics in terms of
user experience and user acceptance.

Based on the learnings and insights gathered during the two co-creation
sessions and the business model interviews, Novosanis optimized the design of
the Colli-Pee and the entire Pee-post service for Belgium and The Netherlands
separately. Approximately one year after the co-creation session, the product
and service were ready for live-testing.

3.2 Live test

3.2.1 Test results
In starting up the live-test important knowledge was gathered with respect to
delivery options in both countries. Although the Colli-Pee parcels were optimized
for letter mail, this type of delivery has an important disadvantage in Belgium.
The postal service does not allow tracking of letter mail, therefore Colli-Pees sent
by letter mail can´t be tracked. Delivery as a parcel allows for tracking of the
shipment but has the disadvantage that it is not delivered in the letter mailbox at
home. Additionally, for returning the parcels to the lab, the participant needs to
hand in the parcel at the postal service centers for scanning, rather than drop it
in the regular (red) letterboxes available in many neighborhoods.

3.2.2 Live test results Belgium

Via Happy Aging, 60 adults between 18 and 45 years (20 in each category: 18-
24, 25-34, 35-45 years old) of age participated in the project. Sixty Colli-Pee
parcels were sent out, 58 arrived at the participants home. Two participants did
not pick up the parcel at the postal office (where the parcel is available after it
could not be delivered at the home address) within two weeks, after which the
parcel was returned to sender. Nine participants did not return a urine sample to
the lab. A total of 12 participants did not complete the evaluation questionnaire
despite one reminder per e-mail and one reminder per telephone. Twenty-six
participants completed university education, 18 completed a professional
bachelor level education.

The quality of the information that participants received prior to participation in

the project was rated as 7.36 on a scale from 1 to 8. The Colli-Pee parcel was
delivered in the letter mailbox of the participant in 24 cases. In 15 cases, the
participant was at home at the time of delivery and the parcel was handed over

133
by the postman. In all other cases, the parcel was available at the postal service
center or delivered at a ‘safe location’ (e.g. on the terrace of the house) that could
be communicated by the participant to the postal company online by using the
tracking code of the parcel. None of the participants reported difficulties in
receiving the Colli-Pee. Three packages were slightly damaged in transport: in
one case the seal of the package was broken, in one case a corner of the box
was damaged and in one case, the buffer solution was leaking from the sample
tube.

Information on how to use the Colli-Pee and how to return the urine sample was
presented in a cartoon format on the inside of the box. The quality of this
information was rated as 6.4 on a scale from 1 to 8. The anonymity of the parcel
was rated as 7.34 on a scale from 1 to 8. Surprisingly, one in three participants
did not view the instructional video before using the Colli-Pee. Two participants
reported that they would rather not use the Colli-Pee again to collect a urine
sample.

3.2.3 Live test results Netherlands

In the Dutch part of the live test, coordinated by the living lab EIZT, bags were
used for mailing instead of boxes, as strongly advised by the Dutch mail. Forty-
one participants were included and were sent a Colli-Pee parcel. Of those, 34
participants (13 male) completed the procedure, including the evaluation
questionnaire. Fourteen participants were over 35 years old, 15 were between
25 and 34 years old, 5 were between 18 and 24 years old. Twenty participants
completed university education, nine completed professional bachelor level
education.

The quality of the information that participants received prior to participation in

the project was rated as 7.29 on a scale from 1 to 8. All Colli-Pees were delivered
in the letter mail box at the home of the participant. One participant had to request
for another Colli-Pee as the first one was not delivered. As no tracking of parcels
was possible in the Dutch part of the study, the whereabouts of this parcel is
unknown. None of the participants reported any damage to the package after
transport.

Information on how to use the Colli-Pee and how to return the urine sample was
presented in a cartoon format on the inside of the bag. The quality of this
information was rated as 7.18 on a scale from 1 to 8. The anonymity of the parcel
was rated as 7.09 on a scale from 1 to 8. In this case, more than half (20) of the
participants did not view the instructional video before using the Colli-Pee. Two
participants reported that they would rather not use the Colli-Pee again to collect
a urine sample, four were doubtful on whether they would use the Colli-Pee
again.

3.3 Living Lab Learnings

It was striking that large percentages of participants of both the co-creation
session as the live test completed higher education. This might have influenced
the results of the session. For example, it was found that a large proportion of
the participants would turn to their general practitioner for an STI screening. This

134
preference for care delivery through the general practitioner might be different in
other socioeconomic status (SES) groups. Previous Belgian research for
example has shown that lower SES is associated with lower participation in
cervical cancer screenings by general practitioners (Lorant, Boland, Humblet, &
Deliège, 2002). Therefore, in future projects sufficient attention should be given
to the recruitment of a variety of end-users, not only in terms of age and gender,
but also in terms of educational level, employment or native background.

The user experience vs user acceptance quadrant method is not easy to use in
a group discussion. Evaluating product characteristics on both scales at once
makes the assignment unnecessary complex for participants, and leads to a
situation where one of both aspects gets more attention than the other. For future
co-creation sessions we would prefer to split the assignment into 2 steps. First,
have all pre-identified characteristics sorted on their importance in making the
device and service easy to use. In a second step, all characteristics can be sorted
in order of importance for the device and service to be accepted by the end-user.
Alternatively, prices could be allocated to product characteristics. Participants
could in turn be given a fixed amount of fake money and be asked to ‘buy’ specific
product characteristics.

A relatively large percentage of participants was lost to follow-up in the live-test

procedure in Belgium, despite strict monitoring of the participant (following the
parcel online, written and personal reminders for the participant). In the Belgian
part of the study, from a total of 60 participants, ten participants did not complete
the evaluation survey even though some of these participants did return a urine
sample. As the living lab project ran during the months July and August the low
response rates could possibly be due to the summer holidays. Additionally, the
obligatory effort of going to the postal office to return the sample package might
have influenced the lost-to-follow up rate. If samples could be returned via the
standard postal boxes, this might have reduced the effort needed to participate
in the study.

One important aspect of an anonymous STI screening procedure has not been
simulated in our live-test, i.e., receiving the test-outcome for example through an
online platform. This aspect may however have a large impact on the user-
acceptance of the service. Additionally, the follow-up on a positive screening by
the patient could not evaluated in the current live-test as no real analyses were
performed on the urine samples. For example, if a patient receives the outcome
that an STI is detected, he or she should still turn to his or her general practitioner
for medical treatment or further follow up.

Cultural and organizational differences between countries matter in innovation

projects. In the case currently presented, user-preferences for obtaining an STI
screening differed between Dutch and Belgian end-users. For example, Dutch
participants preferred the well-known centers for sexual health to turn to, in
addition to their general practitioner. Belgium however, does not have such an
extensive range of sexual health centers. Additionally, the cost of the test and
the time needed to complete the test was regarded as more important in Belgium
than in The Netherlands. Lastly, letter mail delivery options differ between

135
countries. These contextual factors influence the feasibility of an innovation like
the Colli-Pee to a large degree.

With respect to the business model interviews, it was learned that it is difficult to
motivate professional stakeholders (e.g. general practitioners, pharmacists) to
participate in a focus group if no financial compensation is foreseen. Therefore,
we changed our approach from an original focus group set-up to individual
interviews. Flexibility in terms of timing and location for the interview is necessary
for Living Labs. Additionally, we learned that almost none of the consulted
stakeholders held a negative attitude towards the product or associated service.
It is the fitting of the product within the strict regulations in the healthcare sector
(for example related to reimbursement by health care insurers) that most likely
has a large influence on the market adoption.

4 Conclusion
The present Living Lab project has involved both end-users as well as
professional stakeholders in the development of the Colli-Pee device and the
associated STI screening service via letter mail, marketed as ‘Pee-post’.

Two co-creation sessions in two neighboring countries have raised interesting

insight on citizens’ preferences with regard to sexual health care. Extensive live-
testing in both Belgium and The Netherlands has shown that participants valued
the anonymity of the service and found the product easy to use with the provided
instructions. Almost all participants would consider using the Colli-Pee again.

Additionally, the business model interviews with professional stakeholders led to

additional knowledge on how a new product and service could be integrated into
the existing primary health care systems. Belgium and The Netherlands –
although neighboring countries – differ to a large extent both on user
expectations as contextual factors (existing care systems, postal service etc.).

On a living lab level, we conclude that other factors, next to gender and age,
should be considered in recruiting participants to avoid bias because of
educational level, socioeconomic status etc. Secondly, living lab experiments, in
contrast to lab experiments, do not take place in controlled environments.
Therefore, a relatively large drop-out and lost to follow up rate of participants
should be anticipated. In addition to that, sufficient should be anticipated for the
living lab manager to follow up on participants to complete the full study, including
the evaluative survey. Thirdly, the neutral role of the Living lab is crucial in both
setting up anonymous live tests as well as in facilitating (non-commercial)
contacts with various stakeholder from the wider network of the Living Lab, such
as, in this case, health care professionals and postal service providers.

References
Albert, J., & Van der Auwermeulen, T. (2017, March). Why classic Business
Modelling doesn't work for complex business domains–A new Business

136
 Modelling approach for Digital Health. In ISPIM Innovation
Symposium (pp. 1-18).
Lorant, V., Boland, B., Humblet, P., & Deliège, D. (2002). Equity in prevention and
health care. Journal of Epidemiology & Community Health, 56(7), 510-516.
Luma Institute, L. (2012). Innovating for people: Handbook of human-centered
design methods: LUMA Institute, LLC.
van den Broek, I. V., Hoebe, C. J., van Bergen, J. E., Brouwers, E. E., de Feijter,
E. M., Fennema, J. S., de Coul, E. L. O. (2010). Evaluation design of a
systematic, selective, internet-based, Chlamydia screening
implementation in the Netherlands, 2008-2010: implications of first results
for the analysis. BMC infectious diseases, 10(1), 89.
von Karsa, L., Arbyn, M., De Vuyst, H., Dillner, J., Dillner, L., Franceschi, S.,
Suonio, E. (2015). European guidelines for quality assurance in cervical
cancer screening. Summary of the supplements on HPV screening and
vaccination. Papillomavirus research, 1, 22-31.

137
138
 IoT –based Smart Living Environments for
ageing well in Greece
Segkouli Sofia*1, Stefanos Stavrotheodoros1, Kaklanis
Nikolaos1, Votis Konstantinos1, Dafoulas E. George2,3,4,
Karaberi Christina2,4, Tzovaras Dimitrios1

*Corresponding author
1 Information Technologies Institute-ITI, Centre for Research and Technology
Hellas-CERTH, Thessaloniki, Greece
2 CitiesNet- Intermunicipal Development Company, Digital Cities of

Central Greece SA, Greece

3 Faculty of Medicine, University of Thessaly, Larisa, Greece
4
E-Trikala S.A Developmental Municipal Company of Trikala, Greece

Category: Research-in-progress

Abstract
This paper aims to provide an overview of the Greek Deployment Site, one of
the nine (9) deployment sites of the Large Scale ACTIVAGE project, a European
Multi Centric Large-Scale Pilot for Ageing well. Main goal of ACTIVAGE is to
provide an IoT Ecosystem Suite (AIOTES) resolving interoperability at different
layers between heterogeneous existing IoT Platforms for Active and Healthy
Ageing (AHA). Greece is one of the most “aged” countries in the EU. Specifically,
according to a research made in 2014 [1], people at 65 years and over account
for 20.2% of the total population. This large scale IoT pilot connects in one large
scale pilot site three of the most innovative Greek regions (i.e. Municipality of
Pilea-Hortiatis, 10 Municipalities in Central Greece, Municipality of
Metamorfosis) that are representative of different, complementary, geopolitical
and socioeconomic realities. This paper highlights the initial goals,
achievements, technical solutions, critical technological, organizational, privacy
and security challenges and also the best practices that have been initiated by
the Greek Large-Scale Pilot, in order to address successfully a) pilots’
performance and b) new business models’ acceptance and ecosystem
sustainability. To this end, a ‘reference evaluation framework’ has been initiated
as a key aspect of the ACTIVAGE project for the smart living ecosystems’
assessment. It also stresses the obstacles that have been faced so far by the
various stakeholders involved (end- users, healthcare professionals, relatives,
social environment, caregivers) and the lessons learned during the Greek pilot
recruitment, installations, training and running. Last but not least, the Greek
Deployment Site has been assigned to coordinate the ethical and legal activities
of the ACTIVAGE consortium. Thus, an outline is provided about the study of the
ethical and legal requirements in depth and the optimum coordination of data
management as it is experienced by the Greek deployment Site (GR DS) in
compliance to the new regulation (GDPR). The ultimate goal was to address

139
trustworthiness, privacy, data protection and security in project level and also in
each DS internally.

Keywords: IoT Ecosystem, Ageing well, Active and Healthy Ageing (AHA), data
management, data privacy and security

140
1 Introduction
ACTIVAGE is a large scale IoT pilot aiming at setting –up and deploying a number
of Use Cases (UCs) to provide added value services in elderly, across several
European countries (Spain, France, Italy, Germany, Finland and United
Kingdom). The project vision was to implement a strategic reference sustainable
ecosystem, the IoT Ecosystem Suite (AIOTES), of smart living solutions to deliver
interoperability at different layers between heterogeneous platforms and enable
the interconnectivity of heterogeneous IoT devices with the ultimate goal to
provide smart living services and ensure the autonomous living of seniors. Active
and independent ageing is among the priorities of EU initiatives related to older
adults [2].

AHA services based on the Internet-of-Things are promising and various IoT
solutions are deployed in order to sense, measure and control indoor and outdoor
activities to support the independent living of seniors [3]. Nevertheless, the
existing infrastructures in the view of global IoT landscape lack the critical
component of inter-connection which is expected to bring significant added value.

Moreover, various interoperability barriers exist in the current IoT ecosystems, as

the lack of IoT protocol interoperability (systems are often vendor locked by
design), interconnected smart objects of different owner’s demand data sharing
that raises serious privacy concerns, large-scale integration imposes rules that
sometimes are not equally accessed by all users [4].

Therefore, the heterogeneity of IoT devices and sensors along with the
communication technologies and interoperability in different layers is still a
challenge for expanding IoT solutions in a global level [5]. ACTIVAGE aims at
exploiting the plethora of IoT platforms into a dynamic ecosystem of connected
devices, and overcome the fragmentation of architectures, and applications
towards integrated environments and open systems in order to offer solutions to
different stakeholders (users and their families’ service providers, public
authorities).

In line with this main concern, the Greek pilot of ACTIVAGE project envisaged to
use unified IoT solutions at service level, involving different municipalities in terms
of various pilots’ use cases used as an IoT enabled single pilot to provide smart
living services and maximize the effects of IoT technologies in elderly
autonomous life. The smart IoT environments were tested and validated in
controlled and realistic environments before their large-scale deployment in the
selected sites. Specifically, the smart home of CERTH/ITI was used for living lab
testing, providing a fully controllable environment similar to the real cases.

2 Related Work
Relevant research work demonstrated that Internet of Things (IoT) innovation
could have tremendous opportunities for seniors willing to live autonomously at
homes. Moreover, IoT ecosystems can provide healthcare services indicating
great potential to support effective self-care and the independent living of elderly
[6].

141
For example, for the We-care project [7] an IoT solution for the elderly living
assistance has been developed that monitors patients’ vital information, and
additionally provides mechanisms to trigger alarms in emergency situations.
Mighali et al. [8] in the context of City4Age, presented a non-intrusive IoT-based
system that collects information related to movement and body mobility and
through analysis detects automatically behavioral changes in elderly people for
the prevention of Mild Cognitive Impairment.

Similarly, Wang et al. [9] designed a situation-aware abnormality detection

system for the elder people that utilize a u-tiles sensor network to detect their
position and trace. Zhang et al. [10] created a Smart Assistive Living platform that
aggregates sensor information about environmental, cognitive, physical and
physiological factors and integrate them to support the delivery of telehealth and
telecare services to older people with mild dementia.

Other solutions provide activity-monitoring functionalities with the utilization of

wearable devices. For example, Popleteev [11] presented a system that uses a
wearable magnet for activity tracking and indoor positioning, while Belmonte-
Fernández et al. [12] presented a similar tracking system that uses a Smart-watch
wearable device that acquires the Wi-Fi strength signals of surrounding installed
Wireless Access Points.

Although all the aforementioned solutions follow the IoT paradigm and provide
useful insights of to improve the autonomous living of elder people, they have not
been tested in a large-scale environment. Scale. Providing an open source,
interoperable system based on AAL services that will offer high quality AAL
services supporting home care and health of elderly through interoperability and
low cost, for numerous elder users, is still a challenge.

3 Proposed framework
The Smart Home Scenario is directly related to the following use cases: (UC1)
Daily activity monitoring, (UC2) Integrated care and (UC3) Monitoring outside. It
targets the older adult population that lives independently in 3 different Greek
regions. A number of qualified Key Performance Indicators (KPIs) have been
defined in terms of the project in order to assess the progress of pilots’ operations
in respect to the Deployment progress, preparation, the installations, the
experiment running, the open calls etc. Each DS of the ACTIVAGE project has to
report periodically a number of KPIs in order to allow the progressive assessment
of their status. Also a number of questionnaires and scales have been defined for
administration in elderly and their caregivers to provide qualitative information
about the impact of IoT technologies in daily life.

The key stakeholders of the proposed framework are the following:

● Elderly end –users
● Caregivers: relatives or health professionals
● Researchers and Developers
● Entities willing to provide AHA services (i.e. municipalities)

142
Despite the typically of the set –up, the innovative point is that for the first time
AHA services are envisaged to be provided through IoT technologies in such a
wide scale.

Concerning the GR DS, the target values that have to be achieved by the end of
the project in respect to recruitment and installations KPIs and also the current
values so far are quoted below:

Table 1: Key Performance Indicators, Current and Target Values in Greek Deployment Site

Elderly Caregivers
Recruitmen Recruitment Elderly Installations Installations
Greek
t Status Status Target (Current (Target
Deployment Site
(Current (Current Values Values) Values)
Values) Values)
DCCG 108 83 150 17 150
Municipality of
Pylaias – 70 56 150 16 150
Chortiati
Municipalityof
68 68 150 21 150
Metamorhosis

The inclusion criteria for participation acceptance in the research were the
following:
● elderly above the age of 65
● living alone
● have no serious medical or clinical condition that acquires 24-hour
monitoring by another person

In terms of the Smart Home environment, an IoT-based system that is consisted

of commercially available devices was installed in each user’s home. This system
uses the universAAL IoT platform and collects data from 6 wireless devices that
are either wearables (e.g. panic button) or placed on specific places/rooms of the
home. The architecture of the technical solution used in the Greek deployment
site is depicted below:

143
 Figure 1: Greek Deployment Site Monitoring Platform

All data collected from the devices and also the measurements (Table 2) are
transmitted to the cloud server via a gateway (Raspberry Pi) in real-time.

Table 2: Devices, sensors utilized in the Greek Deployment Site

No. of
Wireless sensor Sensor placement Measurements
sensors

1 Panic button Wearable Alarm (binary)

1 x bedroom
Presence (binary)
1 x living room
4 Motion sensor Temperature (°C)
1 x kitchen
Luminance (Lux)
1 x bathroom

Open/close (binary)
1 Door opening/closing detector Central door
Temperature (°C)

Blood Pressure
1 Personal Health System Tele BP/glucometers and/or Glucose
measurement

Then, this data is automatically analyzed in order to enable: a) daily behavioral

activity monitoring, b) health status monitoring, c) alarms/notifications provided to
caregiver in urgent cases.

All the information collected from the smart home installations is transmitted to
the corresponding caregivers through the ACTIVAGE Smart Home monitoring
platform. In particular, a dashboard for each installation indicates in real time all
the data as received from the sensors (Figure 1).

144
 Figure 2: Sensors' data displayed in the monitoring platform of the Smart Home

Moreover, rules are used within the monitoring platform to alarm for abnormal
events displayed accordingly in the dashboard (Figure 3).

Figure 3: Notifications displayed in the ACTIVAGE monitoring platform

Additionally, the ACTIVAGE monitoring platform provides access to all collected

data from each sensor. This data could be visualized through diagrams with
environmental measurements (e.g. temperature). A sample of data is depicted
concerning the temperature of a smart home as sent during a concrete period
(Figure 4).

145
 Figure 4: Diagram of temperature measurements

4 Methodology, challenges and barriers

Within ACTIVAGE, a concrete methodology has been followed for the
assessment of smart living solutions for ageing well. In particular, specific
indicators related to QoL, accessibility and usability and also services’ models
introduced during pilots’ performance.

Among the main objectives of ACTIVAGE was to monitor and track specific
metrics at a local and global level towards a coherent and ‘’Glogal’’ evaluation
framework. This framework is based on the main Triple Win indicators of the
European Innovation Partnership on Active and Healthy Ageing (EIPonAHA):
Impact on QoL, Sustainability, Innovation & Growth.

This evaluation framework is defined to be communicated to different

stakeholders through the ACTIVAGE Evidence Open Database, a global data
repository consisted by three user interfaces:

o The LSP dashboard, which tracks real time data by each deployment site
o The ‘ACTIVAGE Public Evidence website’, an open data base which will
be available also after the project’s end
o The ‘AHA –ADVISOR’, an interface provided as a web-based ICT platform
which offers valuable services enabled by IoT technologies for active and
healthy ageing.

In line with these objectives, the GR DS is dedicated to enable new IoT based
services to support seniors in different Greek regions and also exploit these
solutions to create IoT business cases for healthy ageing.

4.1 Uses Cases implementation in IoT context

146
The GR DS has to implement 3 use cases in respect to the domain of AHA, AAL
and e-Health, which were mentioned in the previous section. Concerning the UC1
& UC3, the health personnel from each Municipality has to recruit a sufficient
number of potential users, in order to accelerate the initiation of installations. The
UC2 services have been initiated since the procurement of the routers for the tele
BP/glucometers had been completed.

In terms of the ACTIVAGE project, the IoT Ecosystem has been validated and
tested in realistic environments having as a main target to collect technical values
the KPIs. Concerning the GR DS, relevant KPIs have been defined for users’
recruitment, installations, and also for the scales/questionnaires administered in
elderly users and their caregivers.

Three evaluation periods (baseline, intermediate and final) have been defined
during the pilot and the evaluation tools used are the following:
o CarerQoL-7D to measure the impact on quality of life of the caregiver [13]
o EQ5D-3L to measure the impact on quality of life of the elderly end user
[14]
o UT-AUT to evaluate the end user’s acceptance of the service [15]
o Global Questionnaire which is a common evaluation tool designed for all
9 Deployment sites that provides input on QoL and Acceptability of
ACTIVAGE IoT platform by users (16)
o ADL and IADL to measure the end users’ level of independency on
everyday chores [17] [18]
o UEQ to measure the end users’ level of empowerment [19]
o UCLA to measure the end users’ level of isolation and loneliness [20]
o FES-1 to evaluate the end users fear of fall [21]

4.2 Users’ Recruitment, consent procedure

In respect to the GR DS, the recruitment process of the users was and still is
among the most challenging issues for personnel assigned to conduct this task.
Timing and synchronization of user recruitment and pilots’ installations was
among the most critical factors for the pilot progress.

From the very beginning of the project within the GR DS concerns and afterwards
when the new regulation came into force (May 2018) [22], the consent procedure
has been addressed as an integral part of data management. Therefore, what
considered important was to produce consent forms in compliance to GDPR and
met all requirements in respect to privacy –related data management.

4.3 Personnel/Developers Training

Before and during the installation phase, the GR DS leader (CERTH) played a
supportive role concerning the main functionalities and technical requirements
that have to be followed in respect to devices’ configuration and installation.

Therefore, training sessions have been scheduled in order to provide the proper
guidelines and adequate assistance in installations’ procedure. Moreover, a

147
manual has been prepared and handed out to developers in order to proceed
smoothly in the procedure.

Additional on-line training sessions took place in order to train the health
personnel responsible for the recruitment the evaluation about the official forms
and evaluation tools distribution and data gathering that takes place throughout
the pilot.

4.4 Technological challenges

The main role of the Greek DS is to support the ACTIVAGE development and
infrastructure of IoT services/tools (existing and new). In particular, the Greek DS
was assigned to define and implement an infrastructure for data storing
processing; including cataloguing, indexing and searching of all ACTIVAGE IoT
data (services, applications, devices). The Greek pilot site and more specifically
CERTH, was the responsible for developing large scale data Analytics
mechanisms to extract maximum value from both historical and real-time data
stored in the Data Store will be analysed and integrated.

The focus is particularly on predictive data analysis mechanism – finding patterns

and associations in the data, which enable us to predict issues, spot anomalies,
and understand relationships between different important factors. CERTH
developed a multi-functional platform with multi-criteria analysis and detection of
spatiotemporal patterns in order to ensure the optimization of IoT management
operations.

4.5 Open calls

Smart living spaces for ageing well supported by the IoT ecosystems were a great
business and market opportunity to be extended in the course of ACTIVAGE and
also after the end of the project. From this perspective open callers were
anticipated to integrate and /or exploit existing the work, project functionalities
and services, retrieving new offerings adapted to specific user requirements.
However ethical and legal points of data management as well as technical issues
(technologies integration etc.) have to be solved.

4.6 Ethical and legal risks

IoT technologies and ecosystems could offer the potential of big data use and
data access with tremendous potential for personalized services. From the
beginning of the project it was clearly realized that connected smart things and
their services should be adapted to an ethical and legal IoT environment.

In respect to the core principle of data protection, Ethics by Design [22], a data
policy framework has to be defined according to devices, platforms’ design and
development from the design phase of the project. In addition, it is of high
importance that as the project was running and on May 2018 a new regulation
came into effect, General data Protection Regulation (GDPR) and ethical and
legal issues of a vast amount of data handling had to be managed properly.

In line with the new regulation (CDPR) and due to the need of ‘special categories’
(health data, known as sensitive data) of data processing in terms of the ‘UC2

148
Integrated care’ on a large scale and through a systematic monitoring and use of
IoT technology, the GR DS had to plan and conduct carefully a number of
organizational activities:
● DPO and controllers, processors’ assignment to monitor GDPR oriented
actions
● Data flow processing
● DPIA documentations of each municipality adapted to the needs and the
concerns of the use cases
● Initiate a Data Protection Methodology (e.g. personal data collected for
specific, legitimate and explicit purposes, processed in consistency with
the purposes defined from the DoA of the project, access is permitted in
authorised persons)

Moreover, a number of technical mitigation measures and Privacy Enhancing

Technologies (PETs) have been initiated to minimize the risk of potential data
breach and ensure data protection and humans’ rights. In particular a number of
encryption algorithms have been used (RSA RC4, DES, 3DES), RSA RC4 (40,
56, 128, 256-bit keys), DES (40, 56-bit keys) and 2-key 3DES ή 3-key 3DES (112
or 168-bit keys) as well as software components based on technologies such as
blockchain [23] etc., to enhance the security and privacy in AIOTES architecture.
Moreover, authentication mechanisms have been utilised for resource accesses
in universAAL based on user roles and associated access rights in order to
ensure that personal data will be submitted to the right data subject [24]. Data
anonymization mechanism is reversible only for authorised entities.

5 Conclusion and Future Steps

From the current status perspective of the GR DS, more than 90% of the
recruitment has taken place and all installations, thus baseline evaluation will be
completed before July 2019. Intermediate evaluation that will take place during
August will provide a more complete feedback in respect to both the local and the
global KPIs.

Despite the technical problems that all the stakeholders of the GR DS

(developers, health personnel, psychologists etc.) have to overcome, IoT
technologies have been proven so far a very challenging and effective solution
for people living alone and need for daily support and guidance.

In terms of ACTIVAGE, our study has been devoted to address elderly

requirements and different demands in relation to care requirements through IoT
–based remote monitoring. However, despite the importance of IoT provided
services for elderly care and healthcare, personalization issues have to be further
addressed in future oriented IoT based systems for elderly. Consequently, what
is of outmost significance is the dynamic nature of seniors’ needs, hence
personalized monitoring should be among the most challenging aspects of future
oriented IoT based systems for elderly remote monitoring and well-being.

Acknowledgements

149
This paper is part of the ACTIVAGE project that has received funding from the
European Union's Horizon 2020 research and innovation program under grant
agreement No 732679.

References
[1] https://greece.greekreporter.com/2014/03/12/greece-holds-one-of-the-
highest-aging-rates-in- europe/
[2] Christensen, K., Doblhammer, G., Rau, R., & Vaupel, J. W. (2009). Ageing
populations: the challenges ahead. The lancet, 374(9696), 1196-1208.
[3] Vermesan, O., & Bacquet, J. (Eds.). (2019). Next Generation Internet of
Things: Distributed Intelligence at the Edge and Human Machine-to-
Machine Cooperation. River Publishers.
[4] Arbel, I., Baker, K., Bailey, T., Bouraoui, M., Chartier, I., Contin, L., ... &
Vermesan, O. (2015). AIOTI WG5-Smart Living Environment for Ageing
Well. Alliance for Internet of Things Innovation (AIOTI), Report v1. 0.
[5] Rechel, B., Grundy, E., Robine, J. M., Cylus, J., Mackenbach, J. P., Knai, C.,
& McKee, M. (2013). Ageing in the European union. The Lancet,
381(9874), 1312-1322.
[6] World Health Organization. (2011). mHealth: new horizons for health through
mobile technologies. mHealth: new horizons for health through mobile
technologies.
[7] Pinto S., Cabral J., Gomes T. (2017). We-care: An IoT-based health care
system for elderly people. IEEE International Conference on Industrial
Technology, ICIT, IEEE, pp. 1378-1383
[8] Mighali V., Patrono L., Rodrigues J. J. P. C., Solie P., Stefanizzi M. L. (2017).
A smart remote elderly monitoring system based on IoT technologies.
Ninth International Conference on Ubiquitous and Future Networks ICUFN
2017.
[9] Wang J., Cheng Z., Zhang M., Zhou Y., Jing L. (2012). Design of a situation –
aware system for abnormal activity detection of elderly people. Proc.
Eighth Int'l Conf. Active Media Technology (AMT), pp. 561-571.
[10] Zhang, Q., Su, Y., Yu, P. (2014). Assisting an elderly with early dementia
using wireless sensors data in smarter safer home. Service Science and
Knowledge Innovation, pp. 398–404.
[11] Popleteev, A. (2015). Activity tracking and indoor positioning with a wearable
magnet. In: Proceedings of the 2015 ACM International Joint Conference
on Pervasive and Ubiquitous Computing and Proceedings of the 2015
ACM International Symposium on Wearable Computers, pp. 253–256
[12] Belmonte-Fernández, Ó., Puertas-Cabedo, A., Torres-Sospedra, J.,
Montoliu-Colás, R., Trilles- Oliver, S. (2016). An Indoor Positioning System
Based on Wearables for Ambient-Assisted Living. Sensors 17(1), 36.

150
[13] Brouwer, W. B. F., Van Exel, N. J. A., Van Gorp, B., & Redekop, W. K. (2006).
The CarerQol instrument: a new instrument to measure care-related
quality of life of informal caregivers for use in economic evaluations.
Quality of Life Research, 15(6), 1005-1021.
[14] Van Reenen M., Oppe M.(2005). EQ-5D-3L User Guide Basic information on
how to use the EQ- 5D-3L instrument, Version 5.1. EuroQoL R. F.
[15] Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User
acceptance of information technology: Toward a unified view. MIS
quarterly, 425-478.
[16] ACTIVAGE public deliverable (2017). D 6.1 Consolidated list of KPI and
coordinated methodology for evaluation
https://www.activageproject.eu/communication-room/public- documents/
[17] Katz S. (1983). Assessing self- maintenance: Activities of Daily living,
mobility and instrumental activities of daily living. J Am Geriatr Soc.
Dec;31(12):721-7. PMID: 6418786
[18] Graf C. (2008) .The Lawton Instrumental Activities of Daily Living (IADL)
Scale. Best Practices in nursing care to older adults. The Hartford Institute
for Geriatric Nursing. April, VoL.108(4):59
[19] Wanner, M., Hartmann, C., Pestoni, G., Martin, B. W., Siegrist, M., & Martin-
Diener, E. (2017). Validation of the Global Physical Activity Questionnaire
for self-administration in a European context. BMJ open sport & exercise
medicine, 3(1), e000206.
[20] Russell, D., Peplau, L. A., & Ferguson, M. L. (1978). Developing a measure
of loneliness. Journal of personality assessment, 42(3), 290-294.
[21] Yardley, L., Beyer, N., Hauer, K., Kempen, G., Piot-Ziegler, C., & Todd, C.
(2005). Development and initial validation of the Falls Efficacy Scale-
International (FES-I). Age and ageing, 34(6), 614- 619.
[22]http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/hi/et
hics/h2020_hi_ ethics-data-protection_en.pdf.
[23] Theodouli, A., Arakliotis, S., Moschou, K., Votis, K., & Tzovaras, D. (2018,
August). On the design of a Blockchain-based system to facilitate
Healthcare Data Sharing. In 2018 17th IEEE International Conference On
Trust, Security And Privacy In Computing And Communications/12th IEEE
International Conference On Big Data Science And Engineering
(TrustCom/BigDataSE) (pp. 1374-1379). IEEE.
[24] Raschke, P., Küpper, A., Drozd, O., & Kirrane, S. (2017, September).
Designing a GDPR- Compliant and Usable Privacy Dashboard. In IFIP
International Summer School on Privacy and Identity Management (pp.
221-236). Springer, Cham.

151
 Participatory design and validation of an
innovative training program to maintain
Autonomy of older adults with Alzheimer’s
Disease
Despoina Mantziari1*, Antonis Billis1, George Arfaras1, Maria
Karagianni1, Vasiliki Zilidou1 and Panagiotis D. Bamidis*1

*Corresponding Author
1 Thessaloniki Active & Health Ageing Living Lab, Medical School, Aristotle
University of Thessaloniki, Greece

Category: Full Research

Abstract
Dementia and its most common type, Alzheimer’s disease, constitute one of the
most challenging global health priorities. Nowadays, several dementias- friendly
initiatives insist on the support of Autonomy of older adults with initial/mild
Alzheimer’s and their caregivers. AD-Autonomy project aims to enhance (ICT)
skills/competences/attitudes of PwAD and their carers through an innovative
training program. The project adopted the participatory design approach in two
milestones, a) the co-creation of the training programme methodology and b) the
co-validation of the training material and the eLearning environment. This paper
focuses on Thess-AHALL’s co-design sessions with end-users, as one of the five
pilot sites of the program.

Keywords: Alzheimer’s Disease, autonomy, e-learning, assistive technologies,

co- creation

152
1 Introduction
According to the World Health Organization (WHO 2019), around 50 million
people have dementia in 2019, worldwide, while there are 10 million new cases
every year. As dementia is an umbrella term for a variety of brain disorders,
affecting cognitive function and memory, Alzheimer’s disease (AD) is the most
common type of dementia, representing around 60%-70% of the total number of
cases in the world. Accordingly, the joint OECD and EU report on the 2018 State
of Health among the EU countries estimates that 7% (9.1 million people) of the
population aged over 60 are living with dementia in EU member states, compared
to 5.9 million in 2000. The total number of persons with dementia (PwD) is
projected to double by 2030. Countries in the European south that have higher
rates of older adults, generally have a greater proportion of persons with
dementia (Italy, Greece, France, Spain). It is estimated that around 8% of
population aged over 60 is living with dementia in these countries (OECD/EU
2018). As dementia progresses to Alzheimer’s, person’s abilities of memory,
thinking, orientation, comprehension, calculation, learning capacity, language
and judgement are significantly affected.

Moreover, the cognitive function impairment, in most of the cases, is followed by

loss of emotional control, changes in social behaviour and limited self-esteem,
which, in combination with the social stigma and discrimination, experienced by
people with AD, often lead to depression and social isolation. Also, WHO includes
dementia among the highest costly chronic diseases, without treatment currently
available to cure or change its progress, estimating that its societal cost
amounted to 1.1% of the global GDP (USD 818 billion) in 2015 and most of the
care to be provided by family caregivers (WHO 2017).

Having a high physical, psychological, social, and financial impact, not only on
persons with dementia, but also on their families, caregivers and the society at
large, dementia has been established as the leading cause of dependency and
one of the ten leading causes of mortality, worldwide. The WHO and the EU have
placed dementia as a growing challenge and one of the main priorities for the
global public health in the next decades. Both have invested in dementia-friendly
initiatives, increase of the social awareness and limitation of stigmatization, early
diagnosis and the training of patients, families and carers, as well as healthcare
professionals in order for PwD to preserve their competences and Quality of Life
(QoL), while the disease progresses.

In recent years, innovative approaches (Bamidis, Antoniou, & Sidiropoulos 2014)

allow persons with dementia to preserve their life as full as possible outside the
healthcare environment, introducing alternative residential care models and
improving their Quality of Life (QoL), while the development of assistive
technologies and ICT Tools enhance their daily life competencies (Bamidis et. al,
2015) in a secure environment with the relevant supports (Bamidis,
Konstantinidis, Billis, & Siountas, 2017). AD-Autonomy is a European-funded
project that aims to “increase the (ICT) competences (attitudes, skills, knowledge)
of Persons with Initial/ Mild Alzheimer's Disease (PwD), Families and Caregivers,
about how to improve Autonomy and subsequently Quality of Life (QoL) of PwD
through an innovative training program”. Embracing the co-creation methodology

153
and the participatory design principles, the AD-Autonomy project partners set
end-users in the spotlight, by actively engaging end- users (PwD, families &
carers, healthcare professionals) from Spain, Greece, Slovenia, the UK and
Turkey in two milestone phases of the training program development:(a) the co-
creation of the methodological training guide, and (b) the co-validation of the
training program, by testing the e-platform (eLearning environment) and
evaluating the final training material.

2 Materials and Methods

AD-Autonomy is launched having as its main objective to enhance the
competences (attitudes, skills, knowledge) of Persons with Initial/Mild
Alzheimer/Dementia (PwA/PwD), Families and Caregivers, in order to improve
their Quality of Life of PwA and preserve/increase their Autonomy through an
innovative training program.

Through the development and implementation of this programme, AD-Autonomy

aims to address the following specific objectives:
● To aware and motivate the targeted stakeholders on the importance of
maintaining the autonomy of people, within a security and support
environment, as an element of QoL for PwD and their families.
● To increase the Autonomy of PwD for decision-making and independent
living, adopting a universal Well-being and QoL approach.
● To transfer good practices and recommendations to PwD, as well as to
their formal and informal caregivers, to promote Autonomy through
training in the execution of daily routines
● To transfer tools, including assistive technologies and ICTs, to support
the processes of empowering PwD and increasing their Autonomy.
● To transfer good practices for the emotional management of impairment
associated with dementia and related problems, based on the concept of
mindfulness.
● To involve professionals and experts, coming from different specialties,
regarding PwD health and care, based on the exchange of their
knowledge and experiences and with end-users, their relatives and the
research team.

To achieve its objectives, launching a training programme that really meets the
needs and requirements of end-users (PwD, families and caregivers), the AD-
Autonomy consortium followed a participatory approach, based on the co-design
methodology and the mutual exchange of experience and know-how among all
the involved stakeholders. Over the last decade, the co-design methodology has
emerged as a significant bottom- up process - social in the means that it uses
and in its ends - in which researchers, social innovators, creative communities,
citizens, vulnerable groups and civil servants co- create solutions to tackle
societal challenges from every aspect of the public life (social inclusion, health &
well-being, employment, migration, climate change, etc.) and address the unmet
needs of the society, based on the strong collaboration and involvement of all the
different types of stakeholders (Moulaert, et. al. 2013; Terstriep, et. al. 2015).

154
The participatory approach has been implemented through all the preparation
steps of the training programme, to collect requirements, to develop the training
materials and specifications, to build the ePlatform, as well as during the testing
phase, regarding the co-validation and evaluation of entire training programme
(face-to-face sessions, experiential activities with end-users, use of the
ePlatform). In order to maximize the impact of stakeholders’ involvement in the
development of a coherent and effective training programme, the AD-Autonomy
also asked them to co-define Autonomy and the aspects of daily life that are
crucial for a person to live independent (Figure 1). To achieve this, a staged co-
creation model was followed: i) expertise and know-how on related projects and
initiatives were collected through structured questionnaires and interviews with
experts from the dementia healthcare sector, in order for the research team to
establish the “State-of-the-Art” of Autonomy and its importance for PwD, ii) open
dialogue with older adults and their caregivers/relatives, regarding the co-creation
of a universal definition of Autonomy and its impact on the main aspects of the
everyday living.

Figure 1. The training co-creation methodology followed in the five pilot sites

3 Co-creation of the methodological training guide

Two rounds of co-creation sessions with end-users (older adults with
dementia/mild cognitive impairment, their relatives and health professionals)
were conducted in late spring 2018 in the five pilot sites (three patient
associations and two universities) of the AD-Autonomy involved countries: the
AFA-Castellon Alzheimer Association (AFA), Turkish Alzheimer Association
(TAD), Spomincica-Alzheimer Slovenia, the Thessaloniki Active and Health
Ageing Living Lab from the Aristotle University of Thessaloniki (Thess-AHALL)
and the Bournemouth University Dementia Institute (BUDI). The conclusions of
these sessions led to the development of a co-created methodological guide for
the training program, including, i) the definition of Autonomy and its main aspects
in one person’s everyday life (dimensions of Autonomy), ii) the design -with and

155
for end-users- of the experiential training activities (good practices, strategies),
supported by technological solutions (existing ICT tools and apps), simulating real
life contexts and situations, and iii) the setting of the user requirements for the e-
training platform for the program. Both sessions were organized as focus groups
of two-hour duration.

3.1 Co-defining Autonomy and its main dimensions

In Greece, the first “AD-Autonomy” co-creation session was organized and held
by the Thessaloniki Active & Healthy Ageing Living Lab (Thess-AHALL), located
and governed by the Medical Physics Laboratory of the Aristotle University of
Thessaloniki (AUTH) in early May 2018.Five older adults (4 females, mean age
77.2 y.o.), suffering from Mild Cognitive Impairment (MCI) or having subjective
memory complaints, participated. One of them had the dual role of the family
caregivers, since they provided daily care at their sister, suffering from AD (Billis,
Mantziari, Zilidou, & Bamidis 2018). In addition, three healthcare professionals
participated in the session, sharing their experience and perspectives on the
caring and autonomy of the PwD. Thess- AHALL trainers guided the discussion
to specific issues, related to i) the perception of Autonomy and its main
dimensions, ii) good practices, strategies and iii) ICT tools and apps that in
everyday routine contexts could help PwD to preserve theirautonomy and
improve their QoL and assist caregivers to promote patients’ independency.
Greek PwD summarized Autonomy as: “to be able to serve yourself”, and remain
“physically, mentally and emotionally healthy”. Regarding the Autonomy
dimensions, experts provided the initial “AD-Autonomy” list of nine proposed main
aspects of the everyday life, as presented in Table 1.

Table 1. Initial AD-Autonomy list of dimensions

Table 1: Security & Safety

Personal Hygiene

Health & Recreation

Food Preparation & Eating

Housekeeping

Outdoor Activities

Finance, Administration

Orientation

Entertainment & Social Interaction

Participants commented
on the proposed
dimensions and promoted their own views and suggestions on the design of a
second, more comprehensive and inclusive list. PwD highlighted the importance
of “Security & Safety” dimension, arguing that it is one of the primary aspects for
both patients and their caregivers. Moreover, they mentioned personal hygiene,
money management, medication management, housekeeping and meal
preparation, sleep routines, orientation and public transportation as core

156
dimensions for the everyday life and the preservation of the autonomy of people,
suffering from memory problems. Greek participants also commented on the
significance of socializing and the need for limiting the social cultural stigma
towards people, facing cognitive function impairment. On their side, professionals
and the caregiver emphasized on emotional control, referring to communication
and effective personal relationships, expression of emotions and understanding
of one’s feelings, the patience and the skills that relatives and caregivers should
train. Regarding emotional skills and personal relationships, PwD agreed with
patience and raised the issue of companionships and each other care for elderly
couples, as well as sexual relationships and inappropriate behaviour of people,
affected by Alzheimer’s.

Older adults with dementia, their relatives and health professionals worked in a
similar way in the rest four pilot sites, providing their personal beliefs on
Autonomy and its dimensions. Accordingly, based on the results of the first co-
creation sessions, and with respect to cultural differences, raised by participants
in the five pilot sites (e.g. praying for Turkish older adults, sexual relationships for
Greek PwD), the AD-Autonomy research team consolidated the final eight main
Autonomy dimensions and an elementary dimension to introduce and assess the
Autonomy, on which the training program activities have been designed (Table
2):

Table 2. Final AD-Autonomy list of dimensions

Table 2: Autonomy & Autonomy Assessment (Dimension Zero)

Security & Safety

Personal Hygiene

Health Management & Sleep

Food & Housekeeping

Meaningful Activities (Including Leisure, Social Activities,

Recreation)
Finance

Orientation & Navigation

Emotional Skills

In a second round of co-creation sessions, end-users in every participating

country were informed about the results and the suggestions of end-users in
every pilot site, which led to the transformation of the initial dimensions’ list. In
Greece, Thess-AHALL organised in early June 2018 its second session, as a
focus group with the active involvement of four older adults (3 females, mean age
74.8 y.o.), with MCI or subjective memory complaints -one of which had the dual
role of family carer- a family caregiver and three experienced health
professionals. Greek participants agreed on the final list, also expressing their
satisfaction for having common views on the Autonomy issue with participants in
the rest of the project involved countries. Moreover, Greek older adults agreed

157
on the co-created term of Autonomy, as it stemmed from the contribution of all
participants’ views, highlighting that “maintaining physical and psychological
health” is the most complete approach, which encompasses all aspects of
Autonomy in one person’s life. Providing additional comments on the Autonomy
dimensions, Greek MCIs and professionals emphasized on the “Security &
Safety”, “Health Management & Sleep”, “Orientation & Navigation” and
“Meaningful Activities”, also confirming participants from other countries that they
constitute the highest priorities for people with memory problems to preserve their
competences in everyday life.

Summarizing, the evaluation and determination of Autonomy, as part of the co-

created methodological guide sessions, motivated participants to express their
personal views of the impact of dementia in persons’ daily and inclusive life.
Within the same context, participants also provided valuable information about
the impact of dementia in both the emotional and physical well-being and the
possible barriers and supports needed for the implementation and preservation
of Autonomy by PwD and their carers.

3.2 The co-design of the training program material

A second objective of the AD-Autonomy co-creation sessions was the
participatory design and determination, with and for end-users, of the training
material -types of material, good practices, activities, ICT tools- as well as of the
skills and key competences related to the implementation of the training program
by PwD, their family caregivers and healthcare professionals.

In order for the AD-Autonomy research team to develop the training program
material and activities, based on the needs and experience of both PwD and
caregivers, participants in the five pilot sites were asked to provide practical
and empirical information about their everyday life activities, the possible
barriers and how they find ways to cope with them, preserving the levels of
their Autonomy and their QoL. In the framework of the first co-creation session,
Greek MCIs highlighted the importance of developing and keeping habits and
daily routines as key factors for maintaining their independence. On their point of
view, professionals insisted on the prioritization of daily duties focusing on
positive aspects, like what older adults can do without any support and which
activities increase their self-confidence ("do easy tasks instead of spending a lot
of time to something that you cannot figure how to do by oneself"). Both older
adults and professionals recommended the adjustment of the everyday activities
to the pace and extend that the PwD can effectively perform them, avoiding any
comparisons to what they used to do in the past. Reminders, keeping notes, and
to-do lists were promoted by all Greek participants as example strategies for
scheduling and prioritizing daily duties, while also dealing with daily problem-
solving. Professionals recommended that people with cognitive impairment
should exercise their memory, following practical strategies, like trying to
remember information as the shopping list, bringing into mind pictures of their
indoor spaces, e.g their fridge, while older participants added the brain exercise
by reading books and playing crosswords, puzzles. The feeling of anxiety, fear
and discouragement by relatives were identified by PwD as the main reason of
failure for developing the required Autonomy skills.

158
In addition to the proposed experiential strategies and good practices,
participants in each pilot site were asked to provide their views on how
existing apps and ICTs could act as assistive technological tools to support
the levels of Autonomy of PwD, enhance their competences and QoL, as
well as those of their caregivers. Greek participants appeared positive in such
a possibility, stating that every kind of support for Autonomy is welcomed and
admitting their basic knowledge and previous experience in using ICT tools and
technologies -mainly smartphones and tablets- through their active participation
in other Thess-AHALL’s research and co-creation. Moreover, participants
characterized the use of ICT Tools/Assistive Technologies as valuable for the
preservation of their Autonomy, the maintenance of their social interaction and
communication levels and their daily life and health management. However,
participants, older MCIs and professionals highlighted that training should be
provided on an individual learning pace and on a continuous base, so as to
sustain in the long term the benefits obtained by the use of technology. Older
adult participants provided real-life context scenarios, where apps and ICTs could
potentially act as complementary or substitute traditional methods and
experiential strategies, used by PwD and their caregivers. Concerning the
possible barriers and doubts on assistive technologies, except for the need for a
long-lasting learning process, the Greek MCIs mentioned the complex design of
existing apps and tools, as well as some ethical issues of using the Internet and
ICTs, like misleading information and personal data use and protection.

In the framework of the second round of co-creation sessions, Greek participants

received feedback on the responses given by end-users in the rest pilot sites,
concerning good practices and tools (e.g. alarms, smoke detectors, pill-boxes,
key/object finders), thinking positively of integrating some of these additional tools
to meet their own needs for maintaining the Autonomy levels. Besides the review
of the Autonomy dimensions and the lists of tools and techniques proposed by
the total number of end-users, the main objective of the second co-creation
sessions was the participants from all the stakeholder groups (PwD/MCIs,
relatives and healthcare professionals) to provide their views over the
development of a training programme for Autonomy, as well as to express their
requirements for both the training material and the e-Platform.

As far as it concerns the Thess-AHALL participants, they found the idea of

developing a training program innovative, very interesting and useful. Older
participants positively commented on the co-creation of an e-Training Platform
that would include and combine technological and non-technological tools and
activities, which could help both PwD and caregivers to maintain the Autonomy
levels and enhance the QoL. On their side, professionals and family caregivers
appeared willing to attend the “AD- Autonomy” online training programme, while
the older adult participants recognized their crucial role for the final design of the
platform and the training material.

All the Greek participants agreed that a complete training programme, based on
the conclusions of the co-creation sessions, would be very helpful and an
innovative approach for both patients and caregivers. Regarding the type and

159
content of the training material, both older participants and professionals
recommended the training program be “everyday life” orientated and respond to
patients needs and interests. MCIs insisted on the use of simple and practical
activities, based on step-by-step guidelines, while they also highlighted the need
for caregivers’ training on the platform, requesting for activities on their behaviour
and patient towards persons with dementia or other memory impairments. Health
professionals recommended the use of printed manuals; written instructions with
screenshots of the presented tools and technologies and family caregivers
suggested the use of multimedia, like informative videos, which combine visual
and audio instructions on good practices and tools. Referring to the type of
training tools and assistive technologies, older participants appeared willing to
use easy- to-use programs and applications, which could help them manage daily
routines, like online lists, alerts and medication reminders, electronic device
control apps, GPS and tracking tools (like Google Maps and the public
transportation timetable apps), social interaction apps (including Viber and
Skype).

3.3 Setting up the e-Training Platform requirements

The AD-Autonomy e-Training platform has the two following specific objectives:
i) to operate as a repository of all the practical training materials and strategies
required to train end-users on the importance of maintaining autonomy, within an
environment of security and support and ii)to give an insight of how assistive
technologies can help PwD and their carers to implement and increase
Autonomy, providing tools, including existing apps and ICTs and online practical
activities and self-evaluation tests. During the second co-creation sessions,
participants were asked to express their opinion about the functional
requirements and the kind of the information that the e- Platform should
contain.

In the Greek pilot site, Thess-AHALL all the Greek MCIs noted that it is an extra
motivation for them to participate in the design of an e-Training platform, which
could help not only their lives, but also the lives and the preservation of the
Autonomy levels of many other people with memory complaints. MCI participants
and their relatives sided for a flexible, in their own pace, online training program
that would allow them to repeat the activities and use the tools as many times as
they want. Family caregivers also appeared willing to use and show the platform
to their relatives, suffering from memory problems. Both asked for clear
instructions, as well as a user-friendly and simple online training environment,
which will include as less as possible information, so as not to be “chaotic” and
confusing for persons with dementia.

Greek participants followed the common line for the information that should be
included in the e-Training Platform, in relation to what the Training Program
should contain. Specifically, they mentioned the need for simple written
instructions, clear objectives of activities and tools, step-by-step videos and
guidelines, repeated actions and not technologically advanced terms and training
materials. Tools and activities should meet the need for lifelong learning and
continuous training and become a daily life routine for participants, who will use
them to enhance their or their patients’ Autonomy levels and competences. In

160
any case, Greek older adult participants and family caregivers asked for face-to-
face sessions, practical activities and repetition to be trained on how use the e-
Platform.

4 Co-validation working sessions with end-users

A new round of participatory co-validation sessions with end-users has started in
each participating country since May 2019, a year after the co-creation sessions
for the design of the training materials and the e-Training platform and is still
ongoing, expected to finish by end of June 2019.In the meantime, the AD-
Autonomy project team developed the final material, considering the
requirements, suggestions and limitations that all participants (n=135) provided
for both the practical strategies and assistive technologies, as well as e-Training
platform features and content.

The five pilot sites invited the participants from the first two co-creation sessions,
as well as new ones, persons with dementia or other memory and cognitive
impairment issues, their relatives and healthcare professionals to attend three co-
working sessions (Face-to-Face meetings, experiential workshops) and to have
some online practice in the training platform, to test and assess the material and
methodology of the training program. The initial goal was a minimum number of
40 participants (30 older adults with memory impairment and their family carers,
as well as 10 professionals) to validate the training programme in each country,
divided in four groups, with a balanced representation of all the different
stakeholders.

In Greece, a total number of 29 participants (3 male) -18 older adults with MCI or
other memory complaints, 3 family caregivers 8 healthcare professionals, some
of whom had the dual role of taking care PwD relatives-attended the co-validation
sessions organised by Thess-AHALL. It is noted that the three out of the four
groups with Greek end-users ran in the premises of Thess-AHALL, while one
group ran with participants from the 11th Municipality of Thessaloniki “Open Care
Centre for the Elderly” (KAPI). Participants of each group attended two Face-to-
Face sessions of three-hours duration each, breaks included. In the first Face-to-
Face session Thess-AHALL’s trainers presented the Autonomy dimension and
the main objectives of the co-validation procedure, as well as asked for
participants consent to be their first collaborators in testing the e-Training
Platform. In addition to the introductory dimension, trainers presented one of the
two main dimensions. The dimensions for each group were divided, as following
(Figure 2):

161
 Group A: Group B:
Personal Hygiene Security & Safety
Orientation & Navigation Food & Housekeeping

Group C: Group D:
Emotional Skills
Finance

Figure 2. Final AD-Autonomy list of dimensions

The second Face-to-Face session included the presentation of the second main
dimension and training to the use of the ePlatform, for end-users to be able to
navigate and explore the material assigned to the dimensions they attended, as
well as to access the rest at their own pace at home. These two sessions were
organized as focus groups and participants were motivated to interact and
discuss on their personal views, experiences or possible doubts and questions,
so as the sessions not have the structure of a passive classroom with distinct
roles (teacher- student). Therefore, all the meetings and workshops that held in
Thess-AHALL’s premises were hosted in the Living Lab’s eHome (Bamidis,
Konstantinidis, Billis, & Siountas, 2017), a specially designed room, similar to a
living room, like a meeting with friends with coffee and treats in a cosy and well-
lit place (Figure 3).

Figure 3. Co-creation in the real-life setting of Thess-AHALL

The third meeting of each group had the form of an experiential workshop, during
which end-users had the opportunity to try some ICTs and apps, related to their

162
assigned dimensions, as well as to have the experience of the practical
implementation of acquired knowledge and strategies in real-context or simulated
user-scenarios. In the closing of this third meeting all the participants completed
a Likert-scale questionnaire, which differed from MCIs to family caregivers and
professionals, assessing the materials, the online platform and the trainers of the
program in terms of functionality, usability, content, type of proposed tools and
strategies, concerning all the Autonomy dimensions.

According to early, pre-compiled results, Greek participants appeared satisfied

and appreciated the fact that their needs and opinions were not only heard, but
also included in the development of the final training materials. Especially, the
MCIs, who either attended the co-creation phase or have cooperated many times
with Thess-AHALL in the framework of other research projects, expressed their
appreciation of acting like partners and ambassadors of the AD-Autonomy
project, representing the Greek end- users community.

4.1 Validating the co-created training materials

Concerning the theoretical and experiential materials of the programme, the AD-
Autonomy research team implemented a similar approach for all the Autonomy
dimensions. For each one out of the eight main and the one introductory
dimension, the research team followed end-users’ request for written informative
materials, regarding dementia and its impact on PwD and their caregivers’
everyday life aspects. A handbook, including all the necessary information,
strategies, tools, and step-by-step instructions for their implementation, have
been designed for each dimension. The handbooks mainly address to trainers,
the people who would train the PwD, either professionals or family caregivers.

A number of practical strategies for the enhancement of competences and skills

of PwD and the support of caregivers and healthcare professionals have been
developed, according to the good practices and personal experience that end-
users shared during the co-creation sessions. The most common strategies,
addressed in most of the Autonomy dimensions were use of lists and diaries for
everyday duties (e.g. the shopping list, cooking recipes, health management,
financial management), brain exercise activities (e.g. crosswords, reading, brain
games),practical information for the adjustment of home as a secure place (e.g.
furnishing, colours, signs and indoor lighting), social/communication skills and
activities (e.g. scheduling of outdoor activities, reminders for calling family and
friends on the phone, physical exercise, expression of emotions).

The practical advice provided mainly by PwD and family caregivers has also
constituted the base for the respective experiential activities of the dimensions.
The user stories of participants from the five pilot sites, as well as their doubts
and possible barriers on how to cope with everyday duties and needs were used
as the raw material for building the real-context or simulation scenarios for the
third round of the validation co-working sessions. Attending the experiential
workshops, older adults with cognitive impairment acted similar to their everyday
life (e.g. scheduling the weekly households in an individual plan/ organising
medication in a pill-box), while family carers and professionals focused on the
same issues, assisting the older participants with their activities (e.g. motivation

163
and encouragement to build up an effective household plan/ practice with older
adults on how to use the pill-box). Non- technological tools and ICTs were
demonstrated and used by participants, meeting the request of end-users for
traditional methods (pill-box) and assistive technology alternatives (apps for
medicine management).

Explanatory videos, external links to additional information and sources, as well

as repetition and combination of methods and tools for more than one dimensions
were also adopted for the training materials. Regarding the multimedia sources
and the apps which have been included in the programme, the AD-Autonomy
research team focused on videos, for which the use of subtitles or narration were
not necessary, and on apps, supporting many languages (e.g. the Google maps
for navigation, viber/ whatsapp for communication) or apps, easy-to-use and
supported by pictures and video examples for their use (e.g. sudoku and other
brain games, money counting apps, emotion flash cards).

4.2 Validating the e-Training Platform

The e-Training Platform (http://www.adautonomy.eu/training/) (Figure 4) was
validated in a two-step process by Greek participants, who attended a live
demonstration and practice with the support of Thess-AHALL’s trainers, during
the second Face-to-Face co-working session, as well as on their own pace online
practice at home for those older adult participants, who had Internet connection
and access to smartphones or tablets. Although the AD-Autonomy research team
did not provide any written instructions for navigating the e-Platform, Thess-
AHALL’s approach on demonstrating the online training materials through a one-
shot visit in the framework of the group meetings and the guidance for exploring
the platform and the entire material at home has proved effective. Family
caregivers, but mostly MCIs got familiar with the structure of the e-Platform, its
main features and the type of the materials, expressing their satisfaction for
unlimited use and self-paced training at home, either alone or with the help of
their family.

The platform included all the training materials presented and used for the
Autonomy dimensions in the group meetings, like the informative material about
dementia, the strategies/good practices, trainers’ handbooks, experiential
activities and a full catalogue of the assistive technologies/ICT tools promoted for
the maintenance of Autonomy. The training material has been organized
according to the nine dimensions of the programme and split in small, but
comprehensive pieces of information, using clear and simple language, not much
text, pictures and videos, additional links and simple colours, as requested by the
majority of end-users.

The absence of login was helpful for older participants with memory problems,
who perceived the training platform as a useful repository of all the necessary
material and information for training themselves or other people on the Autonomy
issue. At the same time, responding to end-user’s request for being part of this
European initiative and be aware of how people from the rest participating
countries deal with similar to their needs and problems, the e-Training Platform
included a community forum.

164
Also, although this is for a European research project, there were no language
issues, meeting the end-user’s expectations, for having the material and the
platform translated in their native language. Moreover, in terms of navigation
though dimensions, the e- Platform adopted both a linear and a non-linear
approach, offering the opportunity to users for one-click access to the activities
and materials of their interest, also providing the possibility of exclusively
selection of either the non-technological strategies and experiential practices or
the assistive technologies, ICTs and online tools.

Figure 4. The e-Training Platform home page

5 Conclusions
The co-creation approach for the development of an innovative training
programme for increasing and maintaining the Autonomy levels and QoL of PwD
and their caregivers, led to a round of participatory sessions that provided useful
information from different participants’ perspectives for both the importance of
Autonomy and its main dimensions in everyday life, as well as for the design of
the training material itself.

The main contribution of this work is the involvement and active collaboration of
Dementia Care multidisciplinary experts (both clinical and engineers) and end-
users at the receiving end of care, such as older adults and their informal
caregivers. Building on top of existing knowledge and experience, older adults
and their caregivers attempted to juxtapose experts’ knowledge on their daily
lives and routines. Self- reflection and real-life situations, allowed to work on
examples and develop new strategies in a way that: are fit for the older adults’
real-life living environments, and maximize the adoption of assistive technologies
at home settings.

The AD-Autonomy research team took into consideration the majority of the
needs, suggestions and expectations of end-users in the five involved in the
project countries (Spain, Greece, Slovenia, Turkey, the UK) to design a complete

165
training programme and an online platform that include both experiential practical
activities and assistive technologies for the maintenance and support of
Autonomy of people, suffering from memory problems.

At the same time, realizing their integral role, as members of a European-wide

co- design initiative that could help other people with similar issues, Greek end-
users, and especially older adults with MCI or memory complaints, appeared
willing and felt comfortable to share their thoughts and personal experiences on
everyday living with cognitive impairment and the preservation of Autonomy
levels. Acting like a kind of ambassadors for the training programme, Greek
participants also contributed to Thess- AHALL’s co-validation working sessions
of both the training material and the e- Platform. They expressed their
appreciation of consulting the initiative, as well as of their needs and requests
having been counted for the final development of the programme.

Acknowledgements
The work has been funded by the Erasmus+ Programme of the European Union
under grant agreement no 2017-1-ES01-KA204-038608 as well as, it has been
supported by members of the 11th Municipality of Thessaloniki “Open Care
Centre for the Elderly” (KAPI) and members of the “Collaboration & Research
Community for the Independent Living” of the Medical Physics Laboratory,
Faculty of Health Sciences, School of Medicine, Aristotle University of
Thessaloniki. For more details, please see http://www.adautonomy.eu/ and
http://medphys.med.auth.gr/

166
References
Bamidis, P. D., Antoniou, P., and Sidiropoulos, E. A., “Using simulations and
experiential learning approaches to train careers of seniors,” Proc. - IEEE
Symp. Comput. Med. Syst., pp. 119–124, 2014.
Bamidis, P. D. , Fissler, P., Papageorgiou, S. G., Zilidou, V., Konstantinidis, E. I.,
Billis, A. S., Romanopoulou, E., Karagianni, M., Bearatis, I., Tsapanou, A.,
Tsilikopoulou, G., Grigoriadou, E., Ladas, A., Kyrillidou, A., Tsolaki, Α.,
Frantzidis, C., Sidiropoulos, E., Siountas, A., Matsi, S., Papatriantafyllou,
J., Margioti, E., Nika, A., Schlee, W., Elbert, T., Tsolaki, M., Vivas, A. B.
and Kolassa, I. T., “Gains in cognition through combined cognitive and
physical training: The role of training dosage and severity of
neurocognitive disorder,” Front. Aging Neurosci., vol. 7, no. JUL 2015.
Bamidis, P. D. , Konstantinidis, E. I., Billis, A. S., and Siountas A., “From e-homes
to living labs: founding and organising the Greek active and healthy ageing
living lab (Thess-AHALL) and its networked services,” J. Hell. Soc. Nucl.
Med., no. Supplement Proceedings of the 10th Panhellenic
Interdisciplinary Conference on Alzheimer’s Disease and Related
Disorders and 2nd Mediterranean Conference on Neurodegenerative
Diseases, pp. 112–125, 2017.
Billis A., Mantziari D., Zilidou V., Bamidis P.D., “Co-Creation of an Innovative
Vocational Training Platform to Improve Autonomy in the Context of
Alzheimer's Disease”, in Proceedings of the 16th International Conference
on Informatics, Management and Technology in Healthcare (ICIMTH
2018), Athens, Greece, July 2018. Permalink: http://www.icimth.com/
Moulaert, F., MacCallum, D., Mehmood, A. & Hamdouch, A. (2013). The
international handbook on social innovation: collective action, social
learning and transdisciplinary research. Cheltenham, UK: Edward Elgar
Publishing.
Terstriep, J., Kleverbeck, M., Deserti, A. & Rizzo, F. (2015). Comparative Report
on Social Innovation across Europe. Deliverable D3.2 of the project
«Boosting the Impact of SI in Europe through Economic Underpinnings»
(SIMPACT), European Commission – 7th Framework Programme,
Brussels: European Commission, DG Research & Innovation
OECD/European Union (2018), "Health at A Glance: Europe 2018. State of
Health In The EU Cycle" (Revised version: February 2019). Retrieved
June 4, 2019, from
https://ec.europa.eu/health/sites/health/files/state/docs/2018_healthatgla
nce_rep_en.pdf
World Health Organization (2017) "Global action plan on the public health
response to dementia 2017 - 2025"Retrieved June 5, 2019, from
https://www.who.int/mental_health/neurology/dementia/action_plan_2017
_2025/en/
World Health Organization (2019) "Adopting a healthy lifestyle helps reduce the
risk of dementia". (n.d.). Retrieved June 5, 2019, from

167
 https://www.who.int/news-room/detail/14-05-2019-adopting-a- healthy-
lifestyle-helps-reduce-the-risk-of-dementia
World Health Organization (2019) "Dementia". (n.d.). Retrieved June 4, 2019,
from https://www.who.int/news-room/fact-sheets/detail/dementia.

168
169
 Responsible Design for an Older Future
Gareth Priday1 and Sonja Pedell1

1 Future Self and Design Living Lab, Swinburne University of technology,

Australia

Category: Full Research

Abstract
In this research we focus on a Living Lab co-design process for the development
of an ‘active ageing’ portal with a local council and the community as part of their
WHO age-friendly city strategy plan. Narratives of ageing have been defined by
models of deficit and decline where technology is meant to provide a solution;
only recently notions of capabilities and individual interests as part of the
technology provision are emerging and are coming to the foreground and getting
more attention. We found that co- design acts as a protective process against
generalised deficit ideas of ageing and to avoid more subtle ageist assumptions.
We show that, on what may be considered a straightforward process and ICT
product, co-design can play a significant role in preventing unintended ageism.
The co-design process in our case provided a pathway to increased adoption
and uncovered future opportunities for a Living Lab to play role in systemic
change regarding the perception of active ageing that extends beyond the
technology solution.

Keywords: Active Ageing, Co-design, Ageism, Ageing World, Web design,

Communities, Age-friendly cities

170
1 Promoting Active Ageing
The world is getting older. The world health authority reports that the percentage
of people over 60 will nearly rise from 15% in 2015 and 22% percent by 2050
(WHO, 2018). By the end of 2020 it is expected there will be more adults over 60
than children under 5 years old. Australia follows this trend with the dependency
ratio (the number of dependants for every 100 workers) changes from 50% in
2013 to between 58-61% by 2033 (ABS, 2014). As a result, the pension age in
many countries is changing; in Australia the pension age is moving from 65 years
to 67 years by 2023.

At the same time, we are extending people’s lives without necessarily improving
the quality of life resulting in a long period of decline nor enable people being able
to actively enjoy or contribute to society (Gore, Kingston, Johnson, Kirkwood and
Jagger, 2018). Providing adequate health and social care for an aging population
is a major economic change for governments at all levels. The greater number of
older people economically dependent on fewer people of working age creates
fiscal challenges for state and local governments (Productivity Commission,
2005). Rural local councils are at particular risk with higher rates to older people
needing services that are more expensive in remote areas, while the rate paying
base is diminishing owing to youth migration to the major cities.

These changes drive different needs in community infrastructure and create the
need to tackle multiple complex transitions at the same time often with competing
agendas; for example, the transition to smart cities, resilient cities, age friendly
cities and dementia friendly cities. As Lancet notes, "how to provide adequate
health and social care to an ageing population is a major economic challenge for
all countries, threatening to cripple already overstretched health and social care
budgets in some" (Lancet, 2018 p.587).

As a result of these drivers there has been a significant interest in promoting

“successful ageing” and more recently, since WHO coined the term in the late
1990’s, “active ageing” (Bounty, 2012). Active Ageing has become a key policy
driver (EU, 2012) with an associated Active Ageing Index (Zaidi, 2012). Active
ageing and its predecessor “successful ageing” are part of a positive ageing
narrative that attempts to counter negative attitudes towards ageing.

Active ageing has become a significant area of interest for local councils
(municipalities) in Australia as a mechanism to improve health of older adults and
prevent unnecessary or premature decline. The objective is to improve quality of
life through increased participation in physical, mental and social activities,
consequently reducing the need for health care (especially the costs for chronic
diseases) and increase the ability of older adults to participate actively in society.

1.1 Ageing and Design

There is increasing research and criticism about the tendency in society and
design to treat older adults as a homogenous group with similar needs that can
be documented and designed with a one-fits-all approach (Lindley et al, 2008;
Vines et al, 2015). There can be pressure for older adults to accept these norms
and active hostility against older adults who openly disregard prescriptive norms

171
of behaviour. “Given the different types of age stereotypes and prejudices that
older adults face, it is not surprising that age-based stigma can negatively affect
them” (Chasteen & Cary, 2015, p.106). Self-adoption of negative stereotypes can
also have negative impacts (González-Domínguez et al, 2018) which may result
in non-use when concerned with inadequate technology. There is therefore a lost
opportunity for these users to engage with technologies that could be in
accordance with the way they like to see themselves and feel about ageing.

Research shows that there are not only diverse pathways in ageing (Browning et
al, 2018), but “Progresses in terms of longevity, healthy ageing and technological
innovation have shaped today’s older people as a generation that actively
contributes to both society and family” (Bordone et al, 2019 p.2). This should also
be reflected in technology design.

HCI designers are not immune to an old model of ageing either and were
historically “concerned with the downside of aging; focusing on assistive
technologies that can help compensate for people’s frailties and the assumed
needs that arise when getting older” (Rogers et al, 2014 p. 3913). This often came
with a view of older people as passive technology receivers, resulting in ICT that
is produced with design that looks and feels as if it has been tailored to a disabled
audience contributing to this suggestion and stereotyped elderly users as such
(Renda & Kuys, 2013). This notion is now being challenged with the HCI field’s
focus moving towards interests, cultural and leisure activities rather than decline;
and design with older adults to mitigate against social ideas of ageing (Sayago,
2019). Durick et al (2013) suggest that “when designing for ‘older users’ we need,
first and foremost, to regard them as ‘users’ who are defined by their specific and
contextual needs, and not their age” and “to remain mindful of how embedded
notions of what ageing means may limit our design thinking (p.473).” Mallenius
(2007) similarly suggests design should not be focused on age at all, but
functional capacity.

1.2 Living Labs and Wellbeing

Living Lab approaches are well established for creating solutions for wellbeing
particularly for older adults with about 30% of ENoLL members having a health
and wellbeing interest and Australia having four living labs that are concerned
with Ageing. The Future Self and Design Living Lab is an ENoLL and ALLIN
(Australian Living Lab Innovation Network) member. The Lab has a focus on
developing technology solutions using emotion led design to increase the quality
of life with a focus on older adults. The Living Lab has success with the emotion
led design as a key element of its co-creation approach to inform tailored
technology development and increase adoption rates (Pedell et al, 2017).

172
1.3 Active Ageing Project
Our project concerned the development and implementation of an ‘active ageing
portal’ on a local council website. The Council is a foundation partner of the Future
Self and Design Living Lab hosted by Swinburne University of Technology.

The Council's main objective is to improve ageing outcomes for their constituents
through greater involvement in physical, mental and social activities. The Council
is approaching this aim as part of their age friendly city strategy in several ways:
through the new website, peer to peer training, promotion with community activity
providers (e.g. U3A, community houses) and health care providers. The success
of the implementation for the Council will be measured by the number of people
using the website and, over the longer-term, increased participation in activities
by local providers. Most website design standards for ageing focus on adoptions
for declining physical capabilities, such as eyesight changes resulting
recommendations for increased contrast and font size (e.g. W3C, 2008). This is
a vital part of website design to make the website accessible. However, these
types of standards do not address the other elements of the design that make the
website attractive to a wide range of people, with very different capabilities. The
Council’s interest in local people adopting the website as a central portal for
Active Ageing created an opportunity to investigate these critical elements for
adoption in a more holistic manner. The investigation into the website design
included language, visuals (aesthetics as well as photographs as representation
of ageing), content, search interactions and hardware use as key elements of the
investigation.

Taking a Living Lab approach, we included other actors in the eco-system as part
of the research this included council staff, doctors and aged care providers. This
paper focuses on the results of our co- design research and the issues it
uncovered.

2 Approach
A prototype of the website portal had already been developed at the time that the
Living Lab got engaged in the project. The core component of the site was a
search function that returned activities available from local providers at no or low
cost. Activity types included physical exercise (e.g. aqua aerobics), mental
stimulation (e.g. language courses, creative classes) or more social activities
(such as trips, social gatherings) and events (e.g. art exhibitions at the council
hall). The Council has 17 centenarians within its borders The Council has a higher
proportion of people post retirement age than other councils in the Melbourne
Metropolitan Area.

Three sessions including interview and prototype scenario-based walk through

were conducted with each end-user participant and a focus group was run with
other eco-system stakeholders (doctors and council staff). As is typical of this
type of project the website development was ongoing. The Council was keen to
have the early findings incorporated into the updated Website. An interim report
was developed after the first 5 interviews of phase 1 and 2; consistent with
literature where 5-6 users are sufficient uncover major needs and usability

173
problems. The prototype site supported wording and picture changes without
developer intervention; updates were made to the site and included in the
remaining phase 1 interviews. Updates of major findings during phase 2 were
provided continuously to enable addressing these issues for final development
(Figure 1.).

Figure 1. Overall participant consultation process

2.1 End -User Approach

The research was based on the concept of co-designing a revised solution with
volunteers sourced by The Council. The Council recruited volunteers for the end-
user research from its networks. Participants input was acknowledged after the
final session with a gift voucher for a local store. Interviews were scheduled for
an hour each in their homes. The first two sessions focused on the initial prototype
website and participants’ attitudes and understanding of active ageing walking
through a set of use scenarios on the prototype as part of the session. These
could have been performed as a single interview, but it was considered that an
interview of two or more hours would be too long. An additional benefit was that
participants often reported additional feedback on a prior section having had time
to reflect on the first session between interviews. The first interview included
questions about general health, technology use including hardware (iPad, mobile
phone, computer), Internet usage and search engine type. It also covered
questions about their understanding and attitudes towards active ageing. The
remainder of the interview in the second interview protocol covered the major
elements of the website including functionality, wording and visual elements
mostly guided through typical use scenarios.

Based on the initial findings a revised high-level prototype was developed. This
included most of the critical and highly prioritised recommendations from the first
two interviews. The third session focused on an updated Website version.

Seven females and five males were taking part in the end-user study. Participants
had an age range from 55 to 80 years. Two of the volunteers were acting on their
own behalf and in the role of carer for their 90+ year old father who had vision
impairment which was considered a common scenario for the Website use.
Several of the participants were in semi-retirement. All the participants were still

174
driving, were living in their own home and were mobile, although one needed
some support and could not walk far.

An interview script (Figure 2) was developed and agreed upon with the Council.
This included a standard introduction to the project, a review and signing the
necessary consent forms. Consent procedures concerned the audio recording of
the interview and an additional request for video recording for two of the
volunteers. The short videos were made with two users of very different physical
and technological capabilities for the development team to have a clear sense of
the range of capabilities and the impact the design had on participants. The audio
transcripts and notes were analysed using content analysis according to Patton
(2002).

Figure 2. Session script extract

In all the sessions participants were encouraged to interact with the website in
ways that were natural to them and explore the site as they saw fit. The guide
with the scenarios was used as a checklist to ensure that all the functions were
covered and for prompting questions about content, visuals and other aspects
when they were not covered by the participants. As with many website
developments there are more user requirements than budget and time to deliver
them straight away. The final design was based on the high impact revisions; high
being defined as based on number of users requesting the same or similar
changes and considering them as important.

2.2 Focus Group Approach

Older users’ data were complemented by a data collection with additional key
stakeholders. A focus group discussion with council support staff, local age care
providers and a doctor was conducted. The site was demonstrated to the users
in the group session. A short guide was developed and agreed on with the council

175
and was used as a prompt for the group discussion. The discussion focused on
the usefulness of the website for their organisations to support their clients. For
these stakeholders the community members they supported were often more
isolated and needed more health and social support from the council and
community groups. The focus group also explored the key intervention points that
the website needed to support as well as usability features.

3 Findings
3.1 End- User Interview Results
3.1.1 Headline and Written Content
The site was originally headlined as “Active Ageing” (Figure 3). Although this
might be a standard term in policy making and academic settings, it was
unfamiliar to the participants. Most participants found active ageing to be a
“marketing” term that they did not feel had relevance for them. This is illustrating
that familiar policy language may still need to be introduced to the community.
The alternative headline line “Add life to your years” was considered a cliché and
a bit patronising by most of the participants.

“Active ageing” was mostly associated with the idea of physical activity, nearly all
participants added mental activity or learning or creative expression only thinking
about it for longer. In the home page description participants found the “Find a
range of activities to keep your body and brain active” clear and concise.
Although, some participants noted that social elements were missing.

Words like 'seniors', 'older adults', 'retirees' or similar were all firmly rejected. Not
surprisingly and in accordance with literature outlined previously anything that
was associated with the idea of being in some way infirm, incapable or needing
support was rejected. The idea of "over 55s", the current name for the portal is
problematic as it emphasises the age rather than capabilities. Several
participants suggested that they would all qualify as “seniors” believing that they
could apply for a senior’s government concession card in the states of Victoria,
however, this only applies to people over 60 years of age. This highlights the
difficulty of naming a group that the site is targeting without the intended
audience, or parts of it, feeling disenfranchised.

The longer description “Active ageing is about helping people realise their
potential for physical, social, and mental well-being and participate in society,
while providing them with adequate protection, security and care when they need
it” was more generally disliked. Participants commented that it was marketing
speak or overly academic” (Figure 3). Several felt that this description mixed up
the concepts of active ageing and independent living. They questioned whether
the idea to “...realise their potential...” as a retired person was important. They
preferred clear, concise language with an emphasis on active and concise
wording such as “meeting people”, “going to places” “fun”, and “enjoyment.”

176
Figure 3.
Are they having fun? - Original home page

Much of the detailed written content participants liked in terms the brief
descriptions of the events, news and other items. The feedback focused on
highlighting key details like time, place, contact details. The events had more of
the wording base on enjoyment lacking in the headline page for example “Ageless
grace is a fun, seated, exercise program to uplifting music”. Participants were
able to assess quickly if this was of interest to them.

The wording of the activities showed similar results. A wheelchair-based exercise

class, “Ageless grace” (fig. 4) was often listed first in the results in the initial
prototype. One participant suggested that this was “more for people who might
be in an aged care facility” and her interest was to find a new Zumba class.
Although there were Zumba classes available, she indicated she'd already "think
that the website wasn't for her" based on that first result. This was re-enforced as
this same result appeared under physical, mindfulness and social categories.
Other less able participants found this class appealing. This illustrated the
difficulty of providing attractive offerings to such a wide range of ages and
capabilities.

177
 Figure 4. Search Results

3.1.2 Visual design and photographs

The initial prototype website had very few visuals elements. Most participants
made some reference to the overall aesthetic, as being “a bit vanilla”, “lacking
colour and movement”, “boring” or noting that colour had been “banned”. All but
two participants want more pictures as a short cut to understanding what activities
were on offer and what they entailed. This was backed up by the participants
comments about the pictures on the website which they used as a quick guide to
the activities and provided the motivation to read the associated content or move
on. They also indicated that the picture gave them cues as to whether they
identified with the people who appeared in the picture. Participants noted that the
pictures are part of the decision-making process giving a sense of the activity. It
was important that any picture is aligned with the activity. One of the header
pictures showed older adults at a farmer’s market (Figure 3). As one participant
noted “I expect to see pictures that represent active-ageing activities” adding that
pictures defined the term, and another said, “they all look really boring.” Like the
language issues, participants were sensitive to any pictures that equated being
an older adult with being in anyway infirm or incapable.
The prototype site had links to the service provider sites. Participants noted that
most of the other sites used pictures more usefully, but also pointed out where
the picture and the activity did not align. For example, a cycle club where the
bicycles and cyclists were not clearly shown on the picture, instead a Chinese
dragon was shown to represent a place they had visited. Similarly, participants
noted that the Seniorpreneurs group photo did not seem to include anyone over
55 (Figure 5).

Although all the participants were of European heritage, they were disparaging of
photos that lacked diversity or showed clichés. For example, one early photo
showed a group of older adults all Anglo- Saxon heritage, all a similar age range
and in couples (Figure 3). Participants pointed to the lack of cultural diversity, the
stereo type of male/female couples and sedate activity that had little relationship

178
to active ageing. Participants did not like having a male only and no female only
listing in the main drop down.

This was in part due to the obvious unfairness and that they wanted a principle
of inclusion as they searched, the listing name e.g. “men’s shed” being enough
in to determine gender specific events. Several participants noted that some in
the LGTBIQ community may also feel discriminated against.

Figure 5. Picture and word mismatch

3.1.3 Search Interactions

In the original prototype the search only had a dropdown list by category (e.g.
physical, creative, etc) and then options to limit the search based on other criteria
such as operating organisation type. There was no general search engine where
key words could be used. While the intention was to make this an easier process,
this was universally disliked. All participants had used Google search or an
equivalent search engine before, so the limited search was not in keeping with
their expectations. It created unintended consequences as noted above where
items were listed alphabetically with the same results appearing in different
categories. The only option was to page through several pages to view all the
items. In the case of the participant interested in the Zumba class this would have
taken some time which more generally is unsuitable for people who know what
they are looking for.

The revised prototype included a key word-based search engine that was
universally preferred by the participants. From the second round of testing two
types of search mechanisms emerged. Participants used the search function in
directed and exploratory ways. Directed is a ‘Google style’ key word search that
is used to quickly narrow down a search to a small set of items, e.g. “Cycling
Club.” The second search was described by one participant as, “I don’t know what
I’m looking for, but I’ll know it when I see it”. This style of searching was used to
look for new things and find out what was available. One participant noted that
this was especially important for people moving into a new area and who might
be looking for ways to engage socially and need help to find out what is available
in their community.

179
Other important aspects included that the advanced search also had an “apply”
button to activate any selection. Owning to poor positioning of this button most
participants waited for the results to apply automatically and quickly became
frustrated. Similarly, lack of clear navigation to get back to the search start or
home screen had the same result.

3.1.4 Content by Age group

As noted above the detailed content generally was succinct, clear and the short
descriptions were able to give participants enough information to decide whether
to read the long descriptions or not.

Participants did question the notion of segregation by age as one participant

emphasised "I want to go to a philosophy lecture not a philosophy lecture talk for
over 65’s”. Others questioned the idea of event curated for over 55's: what was
the criteria for the event of news item being in or out of that grouping? Was it
specifically only events that over 55s were eligible for, or events and news that
someone thought appropriate for over 55s? The idea that it may only be items
that over 55's were eligible for was very limiting. The idea that someone was
deciding on appropriateness raised the question of on what basis these decisions
were made.

Some participants explained that they need to be aware of school holiday

programs in their capacity as a grandparent, others so they knew when to avoid
crowded libraries over the holidays.

Many participants pointed out that capability across the spectrum of active ageing
was more important than age itself. Pointing to 70+ year old people who were
very fit and active compared to some younger people who were not. Another
example included 60 to 90 years olds playing an Australian Stock Market game
online as part of a Probus Club. Demonstrating technical know-how and the ability
to make sophisticated decisions about stock picks, communicate online and
follow stock related news while playing the trading game.

3.1.5 Technology Use

Only one user had real trouble navigating websites and using the internet in
general. Nearly all participants had PCs and most also had iPads. Nearly half of
the participants conducted the interview session on an iPad or equivalent android
tablet devices. Most had mobile phones but would not access the council's site
using the mobile phone.

Several of the participants had careers that included work in developing, testing
or maintaining websites, or some other technology. Recruitment was on a
volunteer basis and may have led to a technophile biased sample. However, it
would seem to point to a transition between adults who did not use internet
technologies in their working life and those that did. These participants were very
comfortable using technology and able to point to very specific aspects of the
design including, in one case, the misapplication of certain standards at a screen
level where cursor movement using the tab key was incorrectly applied. This

180
being especially important for people who are visually impaired and are more
likely to use tab keys than a mouse.

3.2 Focus Group Results

The practitioners focus group consisted of eight participants from the Council,
respite services, health and aged care providers and a retired GP. The session
was about 50 minutes long and discussed the ways the portal and the Council’s
active ageing agenda may intersect with their organisations and practices. This
group tended to work with older people in the community who needed the most
support.

The idea of key intervention points emerged from a discussion of how the website
could support their activities, shown below (Figure 6).

Figure 6. Key intervention points for older adult

At each of these points people’s social connections and the ability to maintain
them changes. People would “reflect on their own wellbeing” and wonder “what
they would do”. A second grouping of interventions was driven by their children
(Figure 7), possibly triggered by the same crisis or by a perceived crisis.

This expanded the potential audiences to children and the community groups
represented who might be using the website and supporting resources that the
Council provides.

3.3 Implementation
A final report was drafted after the focus group and final round of interviews. The
final high priority changes were made to the system which went live in August
2018. The websites visitor numbers doubled within the first month and the
website was a finalist for the Municipalities Association of Victoria, Customer
Achievement of the Year award.

181
 Figure 7. Key intervention points from children

3.4 Recommendation Summary

The final site was much more acceptable to the users and the overall process
worked well and could from the basis of an activity package for similar
developments (Figure 8). Given more time we would have tried to include more
cultural variation to ensure the site was useful for an emerging demographic of
users with different cultural backgrounds. Starting earlier in the project lifecycle
we may have been able to incorporate more changes. While we considered the
use of phones and iPads, we could have investigated changes in technology use
in emerging demographics.

Figure 8. Suggested activity package for similar projects

4 Discussion
4.1 Emerging context
182
The average expected age of death in Australia is 84.6 years for males and 87.3
years for females. A website for over 55s would need to support active ageing for
an average of 32 years. Similar statistics are observed globally. What is
responsible design for an older future look like with such a range of ages,
capabilities and the number of years where an individual is a target of the system?
Lancet (2018) notes that there are increasing calls for ageing to be classified as
a disease with WHO introducing a new category. Biologically there are no agreed
markers or timing for transition to ageing (The Lancet Diabetes & Endocrinology,
2018). People subjective age can be very different to the actual age, often much
lower and its increasingly common to see older adults performing feats that would
have been unthinkable a generation ago.

The image of a frail older adult who needs support and is technologically
incapable is a strong one. By 2020 the internet will have been mainstreamed in
developed countries for 22 years, smart phone available for 14 years and pads
available for 7 years. Adults who will be turning 60 years of age would have spent
a significant proportion of their working life with access to computers, smart
phones and other technologies either in their workplace or at home. The
assumption of technological incompetence for these types of technology will no
longer be reasonable. This was certainly evident from our participants several of
whom had worked in the computer industry for many years including coding,
testing and content management. This target user group does not want to feel
that the design is for an elderly person, rather the design is tailored to meet their
interests and capabilities. Our results are consistent with the move to
understanding older adults as a diverse grouping in both capabilities and
interests.

4.2 Co-design against Ageism?

The project illustrated the tensions between acknowledging the extra support that
can be required as people age and the ease with which approaching this groups
can be stigmatising and re-enforce ageist ideas. While we can understand the
desire and potential benefits to reach out to such a community, even the act of
doing so and defining a group by age in many ways is stigmatising already. It
would seem odd to have a site targeting the 22 to 49-year olds for example.
Demographic transitions and increasing life expectancies demand interventions
that remove the notion of ageing as one point in time segregation or a cohort that
can be managed based on age alone. We must move to a position of recognising
that ageing does bring changing needs, but those changes are specific to the
individual and highly contextual.

There were some design elements such as the limited search engine, which
suggested a deficit model in the early design process. However, with other
elements such as the use of the term “Active Ageing”, it was not as immediately
apparent that the participants would have a strong negative reaction. São José,
Timonen, Amado, & Santos (2017) argue that seemingly new positive discourses
such as ‘active ageing’ can pose problems. They suggest new discourses that
‘models’ ageing, becomes a model of old age; and that ‘active ageing’ suggests
older adults have to behave in a certain way (being active) to become the
“solution” to the perceived societal “problem” of ageing and longevity. As

183
previously noted, older adults who do not match this social idea of ageing may
face a hostile reaction; we run the risk of replacing a deficit model of infirmity with
an active ageing one.

For most councils’ websites will remain the dominant technology they use to
interact with their local populations. While a website re-design may seem at first
glance a straightforward process, this project illustrates the impact co-design can
still have to counter act bias towards normative ideas and assumptions about
ageing that would have resulted in a poor adoption of the website when launched.
It also enables other stakeholders around the older person to give input into the
design process, picking up important elements of their use and transition points
for clients.

Design must focus on these contextual needs as a continuum, and as a result

they must be addressed via processes of co-design within a framework that
supports systemic innovation. Righi, Sayago, and Blat (2017, p26) propose a
“turn to the community” that invites an adoption of “a situated lens on the social
category of older people by looking at the communities in which they engage on
the basis of their interests, skills, needs, goals, self-identities, and contingencies
of daily life and life transitions.”. This model fits with our research findings. The
co-design with community members reveals these changing patterns of
understanding and context, to which designers must respond to. The focus group
also highlights other pathways and moments in time that are important for the site
to capture. This remains largely unexplored and would be an interesting area of
future research. Transitions such as bereavement and retirement were noted in
the focus group as key points to capture people as they make new decisions
about their life. These are opportunities to provide pathways to activities and
social connection are vital to enable people to create their own goals for ageing
well that will have a long-term impact. Similarly, other potential users such as
family members, council staff and other carers are important stakeholders who
can encourage participation at these transition points.

5 Conclusion
While we speak in this paper of older users, the defining characteristic that drives
the design that was successful by the participants was the one that was appealing
to a wide range of capabilities, interests and ages. The main purpose of the
developed Website is to attract older adults and connect them to suitable
activities, irrespective of their technological capabilities or age. Co-design
enables designers and end-users to bypass stigmatisation by exploring the
values, attitudes, life experiences and ambitions of older people and support
these through design rather than focusing primarily on their physical or cognitive
limitations (see also Edlin-White et al, 2012; Vines et al, 2012).

Co-design then takes a positive stance, in the context of older adults, as being
competent regarding their own needs, a value neutral stance on what those
needs might be and a many pathway stance recognising the variety within a
community. A Living Lab provides a larger mechanism to explore this variety and
the systemic causes of ageism and exclusion in a community. These maybe

184
simple opportunities. However, one participant noted the car parking spaces near
community houses were shorter in their time restrictions than the length of most
of the classes offered. Similarly, making the link between the website, actual
changes in activity uptake and long-term health and happiness are worthy of
further research.

References
Australian Social Trends, 2014, Australian Bureau of Statistics,
https://www.abs.gov.au/ausstats/abs@.nsf/Lookup/4102.0main+features
82014
Bordone, V., Arpino, B., & Rosina, A. (2019). Forever young? An analysis of the
factors influencing perceptions of ageing. Ageing and Society, (May), 1–
25. https://doi.org/10.1017/s0144686x19000084
Bounty, K. (2012). ‘Active ageing’: from empty rhetoric to effective policy tool.
Ageing and Society, (July 2012), 1–22.
https://doi.org/10.1017/S0144686X1200030X
Browning, C. J., Enticott, J. C., Thomas, S. A., & Kendig, H. (2018). Trajectories
of ageing well among older Australians: A 16-year longitudinal study.
Ageing and Society, 38(8), 1581–1602.
https://doi.org/10.1017/S0144686X17000162
Chasteen, A. L., & Cary, L. A. (2015). Connections to Research on Subjective
Aging. Annual Review of Gerontology and Geriatrics, 35, 99–120.
https://doi.org/http://dx.doi.org/10.1891/0198-8794.35.99
Durick, J., Robertson, T., Brereton, M., Vetere, F., & Nansen, B. (2013).
Dispelling ageing myths in technology design. In OzCHI ’13 Proceedings
of the 25th Australian Computer-Human Interaction Conference:
Augmentation (pp. 467–476). https://doi.org/10.1145/2541016.2541040
European Commission, 2012, “Guiding Principles for Active Ageing and
Solidarity between Generations”,
http://register.consilium.europa.eu/doc/srv?l=EN&f=ST%2017468%2020
12%20INIT
González-Domínguez, S., Muñoz, M., Ausín, B., Castellanos, M. A., & Pérez-
Santos, E. (2018). Age-related self-stigma of people over 65 years old:
adaptation of the Internalized Stigma of Mental Illness scale (ISMI) for
use in age-related self-stigma (IS65+) in a Spanish sample. Aging and
Mental Health, 22(2), 250–256.
https://doi.org/10.1080/13607863.2016.1247422
Gore, P., Kingston, A., Johnson, G., Kirkwood, T. and Jagger, C. (2018). New
horizons in the compression of functional decline. Age and Ageing, 47(6),
pp.764-768.
Lancet Diabetes and Endocrinology, 2018, Opening the door to treating ageing
as a disease, https://doi.org/10.1016/S2213-8587(18)30214-6

185
Lindley, Sian; Harper, R. S. A. (2008). Designing for elders: Exploring the
complexity of relationships in later life. Proceedings of the 22nd Annual
Conference of the British HCI Group (HCI 2008), 1, 77–86. Retrieved
from http://research.microsoft.com/apps/pubs/default.aspx?id=102057
Productivity Commission 2005, Economic Implications of an Ageing Australia,
Research Report, Canberra.
Patton, M.Q. (2002). Qualitative research and evaluation methods (3rd edition).
Thousand Oaks, CA: Sage Publications.
Pedell, S., Keirnan, A., Priday, G., Miller, T., Mendoza, A., Lopez-Lorca, A., &
Sterling, L. (2017). Methods for Supporting Older Users in
Communicating Their Emotions at Different Phases of a Living Lab
Project. Technology Innovation Management Review, 7(2), 7–19.
https://doi.org/10.22215/timreview1053
Righi, V., Sayago, S., & Blat, J. (2017). When we talk about older people in HCI,
who are we talking about? Towards a ‘turn to community’ in the design of
technologies for a growing ageing population. International Journal of
Human Computer Studies, 108(June), 15–31.
https://doi.org/10.1016/j.ijhcs.2017.06.005
Rogers, Y, Paay, J, Brereton, M, Vaisutis, K, Marsden, G, Vetere, K. (2014).
Never Too Old : Engaging Retired People Investing the Futues with
MaKey Makey. In CHI 2014 (Vol. 1, p. 183).
https://doi.org/http://dx.doi.org/10.1145/2556288.2557184
São José, J. M. de, Timonen, V., Amado, C. A. F., & Santos, S. P. (2017). A
critique of the Active Ageing Index. Journal of Aging Studies, 40, 49–56.
https://doi.org/10.1016/j.jaging.2017.01.001
Sayago, S. (2019). Perspectives on Human-Computer Interaction Research
with Older People,. Springer International Publishing,.
The Lancet Diabetes & Endocrinology. (2018). Opening the door to treating
ageing as a disease. The Lancet Diabetes and Endocrinology, 6(8), 587.
https://doi.org/10.1016/S2213-8587(18)30214-6
Vines, J., Pritchard, G., Wright, P., Olivier, P., & Brittain, K. (2015). An Age-Old
Problem : Examining the Discourses of Ageing in HCI and Strategies for
Future Research. Tochi, 22(1), 1–27. https://doi.org/10.1145/2696867
W3C, 2008, Web Accessibility for Older Users : A literature review,
https://www.w3.org/TR/wai-age-literature/#what
World Health Organisation (WHO), 2018, Ageing and Health, Key Facts,
https://www.who.int/news-room/fact-sheets/detail/ageing-and-health
Zaidi, A., Gasior, K., Hofmarcher, M.M., Lelkes, O., Marin, B., Rodrigues, R.,
Schmidt, A., Vanhuysse, P. and Zolyomi, E., 2013. Active Ageing Index
2012: Concept, Methodology and Final Results. EC/UNECE, Active
Ageing Index Project, UNECE Grant ECE/GC/2012/003. European
Centre for Social Welfare Policy and Research, Vienna.

186
187
188
 A Case Study of a Living Lab through a Bus
Improvement Committee in the Yeongjong are
of Incheon City
Min-ho Suh*1, Junghyun Park1, Minki Kim1 and Won-Kyun
Joo1

*Corresponding author
1 Korea Institute of Science and Technology Information (KISTI), South Korea

Category: Innovation Paper

Abstract
The concept of the living lab has been developed as a social problem-solving
tool and recently it is being reexamined as a social problem-solving tool for
disasters and urban problems based on the social contributions of science and
technology. In terms of science and technology, the data age, represented by
the fourth industrial revolution, is emphasized such that the living lab is now
based on the support of data analysis and ICT technology. The purpose of this
study is to examine the planning, operation, and meaning of ICT support by
means of a database analysis of a living lab case of a bus improvement
committee in the Yeongjong area of Incheon, Korea. Based on the data analysis,
it was found that evidence-based rational discussions among citizen members
as well as the civil servants enable the living lab to operate on a more ground-
based basis and facilitate problem solving. By systematizing the data analysis
technology, it is expected that the effects of the living lab will accelerate as city
officials and civic organizations share important information.

Keywords: Transportation, Living Lab, Bus, Data Analysis

189
1 Introduction
Living Labs is a rather new research area and phenomena that introduces new
ways of managing innovation processes (Bergvall-Kareborn et al., 2009). In this
paper, living labs can be considered as a tool that can accelerate the innovation
process (Bergvall-Kareborn and Stahlbrost, 2009) even though living labs have
been interpreted differently such as an environment, approach, etc. in many
cases (Wolfert et al., 2010; Schuurman et al., 2011; Ruijer and Meijer, 2019;
Chroneer et al., 2019).

In recent years, Korea has emphasized the use of living-lab tools in solving the
problems associated with safe living of people using science and technology,
and promoting smart cities. The Ministry of Science and ICT announced in May
of 2018 that the second plan for the national public safety issue would illustrate
the use of living labs as a demonstration tool for related developed technologies
(National Science Advisory Council, 2018). Moreover, with regard to the smart
city plan, the concept of a living lab is also considered to be important (Fourth
Industrial Revolution Commission, 2018). Although the concept and promotion
of living labs have long been in existence, the concept and application of a living
lab is emphasized again in science and technology applied fields, which are
closely related to current civic life.

The reexamination of artificial intelligence technology (Google) following the

fourth industrial revolution and the conversion to a data-based science and
technology research paradigm must be global, and this is taking place not only
in Korea but also around the world at present. It is expected that the data-driven
technology that will be used as the digital lubricant of the fourth industrial
revolution (Schwab, 2016) will be combined with artificial intelligence and that
other applications will be the core of national competitiveness in the future.

KISTI, the Korea Institute of Science and Technology Information, is a research

institute concentrating on work in the fields of science and technology information
and data and supercomputing as a government-funded research institute. KISTI
has been providing services such as science and technology information
services, super computing services, and science and technology information
analysis to the Korean scientific and technological community and has been
producing and spreading public technology mainly through projects entrusted by
the central government. However, in order to respond to the recent social
expectations of science and technology, public life is emphasized by the current
government, as are the expectations of contributing to public science and
technology via a disaster response system, diffusion predictions, earthquake
predictions, fine dust predictions, and traffic problem solving efforts. In terms of
beneficiaries, it is expanding the scope of its applications by providing support
for direct problem solving using actual data from local governments KISTI has
signed a MOU with Incheon City, the fourth largest city in Korea. Incheon City
provides actual data from the city, and KISTI has developed a data analysis
solution that can be used by Incheon City to develop data analysis solutions in
the four fields of water immersion, earthquakes, traffic, and fine dust. Specifically,
in the transportation problem solving sector, we are developing solutions with the

190
goal of optimizing data bus routes, providing bus demand analyses, simulation-
based feasibility analyses, and route-optimization calculations.

The Yeongjong area is an island that accounts for approximately 10% of bus
traffic demand in Incheon. It is connected to the mainland, and a bridge to the
Incheon International Airport is nearby, as are rural areas. There are many
complaints about the bus routes, where there are various characteristics.
Incheon City has decided to run a living lab with citizens, municipal councilors,
police, city officers, and other interested parties to improve the bus route
operation in the Yeongjong area. In this living lab, KISTI participated in the data
analysis, contributing to the living lab and gaining experience through a pilot
study of the Yeongjong area in developing an overall traffic solution for Incheon
City.

In this paper, by presenting data obtained from the execution of the

transportation living lab project of Incheon City, the contribution of the data
analysis by KISTI, the progress of lessons and learning and of future plans is
detailed. As data-based living lab progress knowledge, the author makes this
contribution.

2 Planning
The planning meeting for the Incheon City Living Lab started with lectures on
living labs aimed at sharing definitions and examples of living labs, and forming
a consensus of city officials about the necessity of utilizing the living lab tool for
Incheon City. In the city administration, considering citizen participation is a
major burden for city government officials. Although citizens' complaints about
city officials are important clues to identify and take action to solve administrative
problems, and efforts to initialize civic responsibilities are substantial, direct
contact with citizens requires some courage out of concerns for city officials.
Nevertheless, the effect of civic participation is significant. The use of a living lab
is not only a source of direct idea acquisition but also a global trend, and it has
had a promotional effect on the interactivity of city administrations (as opposed
to one direction of administration). Building a consensus on the need for city
officials in a coherent direction is not an easy task, and strong initiative from top
government officials is also important.

The lecturer for the planning meeting is a specialist who has provided advice on
various living labs. She focused on the success factors of a living lab,
emphasizing that one important factor is social economic organization, which
represents citizens and the participation of chairman-level members of
representative civic groups. Hence, a living lab can be successful with the center
of ordinary citizen volunteers, but should also be able to share conclusions with
professional knowledge. The living lab should be operated with public-interest-
based communication to achieve successful goals. In the end, the living lab
requires participants with good qualities.
On the other hand, the pursuit for real benefits is more important than pursuing
an exhibition of city administration. Fortunately, the main decision-makers
among the city officials ordered an efficient operating system that would make

191
the citizens not only gain publicity but also allow them to make substantial
progress toward this goal. First, it should be emphasized that living labs find and
solve complicated social problems such as those experienced by Incheon City
through the direct use of existing expert networks, ICT technology and citizen
ideas. This was interpreted as emphasizing the efficient operation of a living lab
by strengthening the use of existing expert networks. Second, social problems
should be divided into individual problems and meetings, and living labs should
be operated intensively. In other words, specialists and civil joint committees that
are specially designed for each problem would be effective. Third, it was advised
to utilize big data analysis technology and to utilize the support of KISTI. Empathy
and momentum regarding the direction of this top decision-making group have
become the main directions of the promotion of the Incheon City living lab.

Many other experts have provided advice on the successful start of the Incheon
Living Lab, and below we present several noteworthy examples. There was an
opinion that it is important to find problems inherent in Incheon City itself. In other
words, the idea of solving the problem using citizens' creative solutions was
presented. Recognition and discovery of problems unique to the region marked
a successful start of the living lab. There was also an opinion that emphasized
the role of the private enterprise sector. This is very meaningful advice because
it emphasizes the participation and role of private enterprises from the beginning,
as the final realization is carried out by a private enterprise, regardless of the
hard work and solution development by the public domain. Finally, occasionally
we will face situations in which the opinions of the municipal government and
citizens conflict. When it is necessary to discuss the issue, emphasis is on the
involvement of citizen representatives who have been mentioned before with
regard to the public interest. It was the responsibility of responsible citizens to
ensure that the living lab does not become a tool for individual citizens to pursue
their own interests. These preliminary discussions have helped Incheon City to
plan for the use of the concept of the initial living lab as a customized tool for
Incheon City.

Among the characteristics of Incheon City living labs are the will to utilize science
and technology actively and the emphasis on citizen participation that considers
public interests. There are several distinctive strategies related to these
characteristics. First, they are actively seeking help from big data analysis
solutions. Existing living labs were using technology as a means of realizing the
improvement ideas of ordinary citizens. However, this case emphasizes the use
of scientific technology, especially big data analysis techniques, in helping to find
solutions and to discuss various ideas of citizens. Second, the plan is to utilize
living labs for the purpose of demonstrating data analysis solutions and acquiring
ideas for developing services that utilize them. This emphasizes the intention to
service and systematize the knowledge gained when solving the problem. It
considers the enhancement and diffusion of public technology developed
through living labs. Finally, they will still utilize the existing expert network, which
has already been emphasized in the directional orientation as previously
discussed. Even if citizen ideas are emphasized, the use of existing expert
networks will play a major role.

192
In this section, I explained how Incheon City planned the living lab and how the
living lab was implemented with some directions and strategies. It was
emphasized that this effort differs from existing citizen voluntary small living labs,
and it is thought that this approach can be used as a reference when planning
the promotion of living labs initiated by the public.

3 Methodology
Different living labs are being organized for each problem in Incheon City, and
the field will expand in the future. In particular, the transportation sector was
considered first because traffic issues are very popular with the general public,
and KISTI also considered an area where it wanted to develop data analysis
solutions. In Incheon, traffic jams are an important issue in the Yeongjong area,
and public officials considered an improvement committee that included citizens.
This marked the beginning and most difficult mission of the living lab, i.e., the
participation of the three main actors of civil servants, citizens, and experts, all
of whom are important among the members of a living lab. With such
preparations moderately ready, the project was launched in December of 2018.

The Yeongjong Bus Improvement Committee Living Lab was constructed and
operated due to the rapid increase in the resident population and the expansion
of the Incheon International Airport (the opening of Terminal 2). New large-scale
apartment complexes are being constructed and residents are continuously
demanding improvements. It is planned to continue to operate during the
reorganization of the bus routes in the Yeongjong area. A total of 25 members
composed of city officials, experts, representatives of bus transportation
companies, civic groups, resident representatives, police, and city councilors
comprise this committee. The committee is held in principle once a quarter,
though it is also held whenever necessary. The main roles of the committee are
defined as follows. First, consultations about the coordination of local buses and
public buses in the Yeongjong area are held. Second, they collect opinions from
Incheon International Airport about linking and transit routes. Third, they prepare
an efficient route adjustment plan for the Yeongjong area.

The first meeting mainly focused on understanding members' professions and

roles, sharing thoughts about the traffic situation in the Yeongjong area and
devising improvement directions. City officials attempted to improve their
understanding of participating citizens and to reach a citizen consensus on the
situation of city financial support through an explanation of the Semi-Public Bus
Operating System, which was introduced in Korea in large cities in the 2000s. In
addition, there were plans to consider the dispatching interval of bus circulation
routes, an issue of high demand from residents. Moreover, flexible residential
times and the creation of a new bus parking lot in the Yeongjong area were
presented. For new large residential apartment areas, bus route adjustments are
being considered. In addition, there are a number of buses connecting the Seoul
Subway and the Incheon Metropolitan Subway, and city officials announced the
opening of a number of issues, including the activation of public buses and the
discovery of several tourist routes. In the first gathering of the living lab members,
it was very important to understand and share the issues with each other.

193
4 Results and Discussion
A total of ten route reorganization plans were announced at the second
Yeongjong Bus Improvement Committee held in March of 2019, two of which
were already in operation. Through the committee, city officials explained in
detail the advantages and disadvantages citizens would receive through the
changes in the routes, and they listened to the opinions of citizens. Most of the
cases involved routes that had started to move into the new apartment complex,
with changing of existing routes. In such cases, the main concern of residents
representing the excluded station areas was the possibility of the presence of
alternative routes and the inconvenience caused by the changes. Most resident
representatives were already aware that residents' route change demands with
regard to new, large apartment complexes should be resolved through some
adjustment of the given number of routes and vehicles. However, the resident
representatives had to appeal to the committee as much as possible about the
discomfort of their neighbors that would arise due to such changes.

The traditional communication method between city officials and citizens when
discussing transportation affairs was that the city official explained the changes
to the residents without clearly explaining the benefits and losses related to these
adjustments and attempted to persuade citizens with ambiguous explanations,
such as the total cost of the city bus grant. In addition, it is difficult for citizens to
grasp how much profit is obtained through such adjustments, and it is an
outcome which cannot be obtained with only a slight sacrifice. This fact provides
a good indication of the main performance implications of a data analysis. Gains
and losses pertaining to the change plan are reported in real time or on the basis
of the meeting, indicating that the meeting must be held while the city officials
and the citizen representatives are sharing ideas.

Below is an example of the feasibility study of the Yeongjong area bus route
improvement plan by the city government officials on the basis of a data analysis.
These results were not revealed to citizens in the second living lab directly, but
this incurred a considerable amount of controversy, such as in the development
of the forms, meaning that the public officials and KISTI should induce public
debate such that there are no misunderstandings after much discussion.

For example, as shown in Figure 1, the plan from the city official is to change the
203-bus line in the Yeongjong area to satisfy the bus usage demands of newly
constructed large-scale apartment complexes.

194
 Nubdi
: 203,307, 598,304
: 203, 307

203

Nubdi
: 203, 307,
: 203, 307

new

new Apt.

Figure 1: Route change plan example of line No. 203 in Yeongjong area

The important items about the 203-route change plan are the exclusion of the
'Unseo Elem. School' stop and the route to the new apartment complex stop.
Therefore, considering the addition of a new apartment complex stop to such a
change, it is necessary to re-evaluate it. How should one measure and judge the
exclusion effect of the 'Unseo Elem. School'? For this feasibility analysis, the
following analysis is required, and it can be done based on user data through
traffic usage cards.

As shown in Table 1, the Unseo Elementary School (35-270) is bus stop of No.
203, which is the fifth bus stop, and the ride share of No. 203 on this stop is
31.01%. Unseo Elementary School (35-270) does not have a large portion of the
total number of departing passengers at No. 203, and there are many
passengers departing from other lines (No. 598 and No. 304). Particularly,
approximately 48% (453/944 people) of the passengers traveling to the Incheon
International Airport from Unseo Elementary School (35-270) account for 90.96%
of the O-D demand. The remaining part of the O-D demand is covered by No.
203 (17-minute intervals) and No. 307 (36-minute intervals). Therefore, an
alternative means to the Incheon International Airport should be considered as
important

Table 1: Status of Unseo Elem. School (35-270) Bus Stop

195
 Unseo Elem. School
Line 203 Excluded Stop
(35-270, to airport)
Ride Ranking
Ride Sharing of this
of this stop on
5th (944) line on this stop (%) 31.01
this line
Taking off
Taking off Sharing of
Ranking of
this line on this stop
this stop 22th (141) 24.06
(%)
on this line
Top 3 arriving
Incheon Airport
stop from Unseo Station Lotte Mart
T2
this stop
O-D 453 259 136

This line
dependency
of this
O-D (%)

As presented in Table 2, at the Unseo Elementary School (35-282) stop, there

are not many passengers of but No. 203, and there are many passengers who
ride other routes (No. 598). Unseo Elementary School (35-282) is the fifth-ranked
bus stop for No. 203, and the departure share of No. 203 at this stop in
comparison to other lines is 36.44%. Particularly, about 32% (228/707) of the
passengers who travel from Incheon Airport T2 to Unseo Elementary School (35-
282) account for 86.69% of the corresponding O-D demand. No. 307 (36-minute
intervals) covers the remaining 13.31%. Hence, an alternative means of coming
from the Incheon International Airport should be considered as important. In
summary, passengers (average 113.5 persons/day) on the "Unseo Elementary
School - Incheon Airport T2" O-D using the existing No. 203 were required to
take a six-minute walk (400 meters) when using the alternative “Nub-di” station.
However, the merits of enhancing the new apartment passenger demand should
be compared with the lack of an Unseo Elem. School stop in the near future.

KISTI reported the results of the above analysis pertaining to ten route change
plans announced by the second Yeongjong Bus Improvement Committee Living
Lab, and the city officials agreed with the results of the data analysis and hope
to refer to it. However, it is emphasized that the final route decision should take
into consideration not only the data analysis results but also numerous practical
considerations. However, the results of the feasibility study based on the data
analysis are important criteria because they can be the basis of one objective
discussion, showing that data that can support a living lab and demonstrating a
type of relevant ICT support

Table 2: Status of Unseo Elem. School (35-282) Bus Stop

196
 Unseo Elem. School
Line 203 Excluded Stop
(35-282, To Skycity)
Ride Ranking
Ride Sharing of this
of this stop on 24th(142) 28.57
line on this stop (%)
this line
Taking off
Taking off Sharing of
Ranking of this
this line on this stop
stop 5th(707) 36.44
(%)
on this line
Top 3 departing
Incheon Ariport T2 Unseo Station KT airport branch
stop to this stop
O-D 228 170 123

This line
dependency of
this O-
D (%)

In this case study, KISTI supported the Incheon city government's data analysis-
based route improvement review and then decided to develop a traffic living lab
into a citizen-led idea discussion method. This study is ongoing, and it is likely to
take a more developed direction through cooperation with citizens and city
officials in the future. As a short-term goal, the third (scheduled to be held in June
of 2019) committee plans to announce the improved route plan by Incheon City,
with urging by KISTI to refer to the data analysis results actively. In addition, a
conference meeting will be held during the second half of 2019 to encourage
citizen-led improvement ideas to be discussed in the living lab. At the same time,
ICT support will be provided for real-time analysis so that residents can confirm
the effects of the route improvement during the meeting.

5 Conclusions
This study deals with the contributions of science and technology in solving a
national life safety problem in Korea, mainly the application of data-based
technology. Particularly in the case of transportation living labs, the significance
of data-based technologies, which can promote living labs, the process of
persuading stakeholders, and the methodological directions of those data-based
technologies, can facilitate the efforts of living lab operations to share knowledge.
Discussions on improvements based on clear facts during social problem-solving
discussions are very important, and in this case, it can be said that science
technology, especially data analysis technology, supports these efforts. This is
meaningful because it is a clear example of the possibility of leading a data-
foundation- based administration and citizen debates based on data when
attempting to solve social problems. In addition, it is important to reaffirm that the
support of civic-oriented living labs is important, but the participation of social
organizations, civil servants, the police, and municipal councillors is also crucial.
The public use of public administration data and the development of analytical

197
technology that fills the gap between data and citizens can lead to the
development of living labs in the data era. In this sense, in the future it can be
expected that these discussions will be more active regarding the environment,
where researchers are expected to use data actively to understand each other

Acknowledgements
This research was supported by Ministry of Science and ICT, Republic of Korea
(Project No. K-19- L05-C01-S01). Also, we would like to thank Incheon City for
local data support and practical opinions.

References
4th Industrial Revolution Commission. (2018, January). Smart City Strategy for
City Revolution and Future Creative Engine.
Bergvall-Kareborn, B., Hoist, M., and Stahlbrost, A. (2009, January). Concept
Design with a Living Lab Approach. In Proceedings of the 42nd Hawaii
International Conference on System Sciences, IEEE.
Bergvall-Kareborn, B. and Stahlbrost, A. (2009). Living lab: an open and citizen-
centric approach for innovation. International Journal of Innovation and
Regional Development, Vol. 1, pp356-370, DiVA.
Chroneer, D., Stahlbrost, A., and Habibipour, A. (2019, March). Urban living labs:
Towards an Integrated Understanding of their Key Components.
Technology Innovation Management Review, Vol. 9, pp50-62, Carleton
University.
Google. https://deepmind.com/research/alphago/, retrieved at April 1st, 2019.
National Science Advisory Council. (2018. June). The second science and
technology-based national life (society) comprehensive problem solving
plan ('18 ~ '22).
Ruijer, E. and Meijer, A. (2019, February). Open government data as an
Innovation Process: Lessons from a living lab experiment. Public
Performance & Management Review, published online: 15 Feb 2019,
Taylor&Francis Online.
Schuurman, D., Moor, K.D., Marez, L.D., and Evens, T. (2011). A living lab
research approach for mobile TV. Telematics and Informatics, Vol. 28,
pp271-282, Elsevier.
Schwab, K. (2016, January). https://www.weforum.org/agenda/2016/01/the-
fourth-industrial- revolution-what-it-means-and-how-to-respond/, retrieved
at April 1st, 2019
Wolfert, J., Verdouw C.N., Verloop, C.M., and Beulens, A.J.M. (2010, March).
Organizing information integration in agri-food – A method based on a
service-oriented architecture and living lab approach. Computers and
Electronics in Agriculture, Vol. 70, pp389-405, Elsevier.

198
199
 Adapting the Urban Living Lab approach to
marginal contexts and urban regeneration: the
case of Mapping San Siro Lab
Francesca Cognetti1 and Elena Maranghi1

1 Department of Architecture and Urban Studies – Politecnico di Milano, Italy

Category: Innovation Paper

Abstract
The aim of the following paper is to offer a reflection on the role and the
characteristics that an Urban Living Lab could assume in processes of urban
regeneration of marginal and deprived contexts of contemporary cities. The
paper analyses Mapping San Siro case study: an on-going University-promoted
Lab, opened in 2013 in one of the biggest Milanese public housing
neighbourhoods (Milan, Italy). After briefly introducing the potential of Urban
Living Labs in urban regeneration, the authors will contextualize the case study
within the approach, highlighting the most significant points of contact. To
conclude, they will open up the reflection on critical points to be considered when
orienting an Urban Living Lab to the local development of a marginal context.

Keywords: Living Lab, Transportation, ICT Tools, Big data, GIS, Decision-
making

** Although the paper is the result of a collective work of the two authors, paragraphs 1, 2 and 5
should be attributed to Elena Maranghi; paragraphs 3 and 4 to Francesca Cognetti.

200
1 Introduction
In the last decades, in the Italian context, the topic of urban regeneration and
local development of marginal and peripheral areas has expanded its boundaries
and meanings, becoming a more and more complex arena, characterized by the
presence of very diverse actors. Increasing importance was especially gained by
citizens and local organizations, considered able to interpret and bring out local
resources and competencies, essential to develop successful interventions,
especially in an era of scarce availability of funds.

In Italy, the concept of participation – as applied in the Nineties to urban

regeneration – was progressively overcome by the ones of social activation and
social innovation: notions that are questioning the mere information and
consultation to which participation risked to be confined in the widespread
institutional approach to urban policies. As a result, nowadays, local communities
of practice (Wenger, 1998) are assuming a more and more active and direct role
in the transformation of their own environment and in the fulfilment of their social
and cultural needs.

But what happens to especially fragile communities or populations, not always

able to promote or actively participate in local development processes?

It appears urgent for engaged researchers and practitioners in urban

regeneration to question themselves on the development of effective tools that
could support the transformation of local citizens and organizations in truly
empowered actors, able to promote and “control” changes, but also to claim for
effective institutional support.

2 SoHoLab project: questioning Urban Living Lab methodolo-

gies in marginal contex
Currently, it can be observed, indeed, how marginal territories suffer from a lack
of chances in terms of bottom-up promoted regeneration. On the one hand,
because of the difficulties for especially fragile populations to access to certain
languages or tools (among other factors because of a substantial digital divide);
on the other hand, because of a widespread sense of distrust, generated by the
perceived “absence” of competent institutions, which seem to be no longer able
to promote effective policies in such contexts. As a result, the process of
exclusion of these territories is currently worsening.
In the last years, applying the Urban Living Lab (ULL) approach to urban
regeneration has been seen as promising to tackle this issue: existing literature
has especially underlined their potentiality in terms of transformation and
enlargement of urban governance and of empowerment of the different social
actors (Concilio, 2016; Steen & van Bueren, 2017; Nesti, 2018; Naumann et al.,
2018). ULLs are described, indeed, as “cross-boundary objects/arenas” and
knowledge creative contexts (Concilio, ibid.), able to connect stakeholders and
relevant actors at different levels (institutions and the so-called “users”: here,
citizens, dwellers, communities); and, at the same time, capable of fostering
social innovation (Naumann et al., ibid.) through the emphasis given to co-design
and co-creation, as elements able to promote a really participated change.

201
Indeed, as several authors have pointed out so far (Hakkarainen & Hyysalo,
2013; Concilio & De Bonis, 2012; Franz, 2015; Ståhlbröst et al., 2018), even if
considered to be valuable tools in developing smart and innovation strategies in
the urban context, the academic debate on ULLs in urban studies still remains
underdeveloped and unclear, especially when coming to a more socially-centred
orientation of this approach (Franz, ibid.). Moreover, so far ULLs have rarely
addressed deprived and marginalized contexts and superdiverse (Vertovec,
2007) communities, characterized by severe conditions of social, cultural and
economic exclusion.

Started in 2017, the SoHoLab Project “The Regeneration of Large-scale Social

Housing Estates through Living Labs”7 has, indeed, the aim to establish and
evaluate how ULLs could deal with the regeneration of social housing complexes,
focusing on the role of University-promoted Labs8 and aiming «to develop an
action research to effectively deal with the social-spatial exclusion of residents in
underprivileged large-scale social housing estates in Europe through a Living Lab
approach9».

3 The case study: Mapping San Siro Lab

Moving from these general statements and assuming the framework of ULLs as
developed so far, as Politecnico of Milan team we have started to question
ourselves on if and how it was possible to “translate” (Franz, ibid.) the
technologically-centred approach to ULLs to a more socially- oriented one
especially addressed to marginalized contexts, starting from our own on-field
experience. In fact, The SoHoLab project involves as a case-study an already
existing and on- going experience in which we are involved in as researchers:
Mapping San Siro (MSS) Lab. Supported by the Department of Architecture and
Urban Studies and by Polisocial Program (Politecnico of Milan public
engagement program), MSS was started in 2013 as a workshop activity, involving
the participation of a group of students, researchers and teachers, interested in
challenging the negative narratives associated to San Siro neighbourhood, one
of the biggest public housing complexes of the city 10, and in producing shared
and “usable” representations that could effectively trigger local change. Thanks

7
Financed through the JPI URBAN EUROPE CALL 2016 - ERA NET COFUND SMART URBAN FUTURE the
SoHoLab Project (2017 – 2020) involves three Universities: Vrije Universiteit Brussel (international
coordinator), DAStU – Politecnico of Milan and AHTTEP – AUSSER – École Nationale Supérieure
d’Architecture Paris La Villette. The project also includes non-academic partners at national levels. More
information available on the website: www.soholab.org.
8
The approach is developed, tested and refined on the basis of a retrospective evaluation of existing
projects in Paris, of action research in an ongoing LivingLab experience in Milan and a new one in Brussels.
9
From project proposal
10
Located in the North-West part of the city, not far from the city centre, San Siro is composed of about 6.000
housing units and with a population of about 10.000 inhabitants, the neighbourhood is characterized by the
presence of fragile populations and by strong socio-spatial inequalities and intercultural/intergenerational
conflicts (around 50% of the population are immigrants, with about 85 nationalities represented). Despite
being also characterized by the presence of diverse and strongly committed local actors (associations,
cooperatives, groups of inhabitants), San Siro has always been heavily stigmatized in public discourses with
the effect of worsening its exclusion from urban dynamics. More information on the website
www.sansirostories.com (developed by Master in Journalism of Università Cattolica of Milan together with
Mapping San Siro) and on www.mappingsansiro.polimi.it.

202
to the involvement of local actors, the workshop was particularly successful and
a group of researchers decided to continue to work in the neighbourhood. In
2014, MSS obtained from Aler – the Regional Agency for Public Housing of the
Lombardy Region, which owns and manages the housing stock – the possibility
to re-open a vacant shop in the neighbourhood, located on the street level, which
became the headquarter of the group.

It was the beginning of a new phase: inhabiting a space and becoming a locally
rooted actor, on the one hand developing research and teaching activities on
three main topics – home and dwelling conditions; courtyards and public spaces,
non-residential vacant spaces – and, on the other hand, trying to tackle the urgent
issue of promoting participated local change, in a neighbourhood characterized
by abandonment, distrust and inertia. We have defined our presence on the field
as situating (Cognetti & Castelnuovo, 2019): building significant relationships with
the context and gaining an internal perspective to the neighbourhood through a
long and slow rooting process. An aspect which profoundly shapes the other two
dimensions of our practice: inquiry (embedded research) and acting (promoting
participated change).

4 A local research-driven Urban Living Lab?

Even if MSS was not intentionally started as a ULL, but as an action-research
experience, several elements could be identified that it shares with the approach
(among the others see Concilio, ibid.). In particular, elements that – even if
already present in the broader approach – should be taken into particular
consideration in order to steer the ULL approach when dealing with marginalized
contexts.

(1) The centrality of the co-research phase for co-learning and co-design
(inquiry): as broadly known, LLs are based on a co-creation approach that directly
involves the so-called users. To our experience, when coping with marginalized
and fragile contexts, particular attention should be paid to make the phase of co-
research as inclusive as possible: identifying and highlight a shared and co-
constructed vision of problems and resources, especially by significantly linking
scientific knowledge with common knowledge (Dewey, 1938), produced by the
so-called everyday-makers (Bang & Soresen, 1999) as to say dwellers and local
organizations. Co-research is here intended as a mutual learning process (co-
learning) which constantly shapes the phase of co-design. It is particularly
relevant because it allows local actors to mutually acknowledge themselves a
competence and a voice on issues concerning the neighbourhood and its
possible transformation, empowering their ability to act 11. As MSS, we have
practiced this aspect in particular through the development and coordination of
the local network of formal and informal organizations, called Sansheroes (see
Maranghi, 2019), which became able to produce a shared vision on the

11 We refer here to the concepts of “right to research”, proposed by Appadurai in 2006, and of the one
of knowledge as a “condition for development”, Freire, 1970.

203
neighbourhood and a platform of planning, elements that reinforced their capacity
to interact with institutions.

(2) Co-creation as an incremental process to experiment on effective cross-

boundaries arenas (acting): in marginal context – where a relationship of mutual
trust and acknowledgement of the different actors often needs to be re-build – we
have experimented how the co-design of little and incremental pilot projects,
which engages both institutions, local organizations and inhabitants, is a powerful
tool to rebuild a learning-friendly context, fertile to acquire the ability of working
together and generating new forms of governance. As Franz has already pointed
out (ibid.), socially- oriented LL are usually concerned with process-based
innovation rather than with product based one. To our experience when coping
with marginal contexts, the ULL itself – as an arena of collective work – should
be considered as a significant incremental outcome, capable of generating
innovation in the way in which different actors interact. The University as a
promoter of the ULL in this sense plays a key role since its “third position” allows
it to effectively engage institutions, civil society, local actors and city level actors
(Castelnuovo & Cognetti, ibid.).

5 Provisional conclusions
To conclude, we would like to share some open issues that we consider to be
able to influence the “success” of so-called Living Labs in marginalized contexts.
Elements that challenge, on the one hand, the competencies that we should
develop as researchers and practitioners involved in the process, on the other
hand, the possibility to activate practices able to produce a durable and scalable
change in terms of local development/regeneration.

(1) Situating: spatiality and time matter. We have defined situating as the practice
of conducting embedded research through the opening of a space in the field
(specifying, somehow, the concept of geographical embeddedness of ULLs,
Voytenko et al., 2016). The physical space could be considered the core of our
activity since it helps to practice contingency (Karvonen & Van Heur, 2014): a
constant process of co-learning – related to the “here and now” – grounded in
facts, relationships and situations directly experienced and shared with the
research field. We argue that when setting up a ULL in a marginal context opening
a physical space matters: “being local” helps to build trust and reliability – both
towards the institutional level and the local level – and it opens up access to
different forms of knowledge (local one, institutional one, scientific one, etc.), key
elements for urban regeneration. We recognize, however, that such a rooting
process is profoundly demanding in terms of time and possibility to engage in a
deep relationship with the context (Kondo, 2012).

(2) Potentiality and limits of a local scale. To intend the ULL as an incremental
object, questions how not to be “trapped” not only on the local scale but in very
minute issues (referring to San Siro, for instance, the transformation of public
space in a neighbourhood where housing is the most urgent matter). Besides, it
more broadly questions the ULL approach, tackling the matter of how to create a
durable and significant change, even if starting from a very local dimension

204
(Steen & van Bueren, ibid.). In this sense, we are exploring the possibility to
intend a ULL in marginal contexts precisely as a device to engage different actors
– especially the ones usually perceived as distant from the local level – through
the development of pilot projects, constantly giving them the chance to
experiment new configurations of urban regeneration governance (co-producing
knowledge, working together, etc.)

Both points, indeed, question whether do we have to intend local regeneration as

an incremental process and, if so, ULLs as local permanent structures, always
capable of reinventing themselves but at the same time to be stable and durable.

References
Appadurai, A. (2006). The right to research. Globalisation, Societies and
Education, 4(2), 167- 177.
Bang H.P., E. Sørensen E. (1999). Everyday Maker: A New Challenge to
Democratic Governance. Administrative Theory & Praxis, 21 (3), 325-341.
Castelnuovo I., Cognetti F. (2019). Mapping San Siro Lab: experimenting
grounded, interactive and mutual learning for inclusive cities. Transactions
of the Associations of European Schools of Planning, 3, 37-54.
Chronéer, D., Ståhlbröst, A., Habibipour, A. (2018). Towards a unified definition
of Urban Living Labs. Presented at the ISPIM Innovation Conference –
Innovation, The Name of The Game, Stockholm, Sweden on 17-20 June
2018, International Society for Professional Innovation Management
(ISPIM).
Cognetti F., Padovani L. (2017). New meanings for public housing through the
co-production of knowledge. Policies for everyday life in marginal
neighbourhoods. In: Bargelli E., Heitkamp T. (Eds.), New developments in
Southern European housing. Pisa: Pisa University Press.
Concilio, G., De Bonis, L. (2012). Smart Cities & planning in a Living Lab
perspective. In: Campagna M., De Montis A., Isola. F., Lai S., Pira C.,
Zoppi C. (Eds.), Planning Support Tools: Policy Analysis, Implementation
and Evaluation, Proceedings of the VII Int.l Conf. on Informatics and Urban
and Regional Planning INPUT 2012. Milan: Franco Angeli.
Concilio G. (2016). Urban Living Labs: opportunities in and for planning. In:
Concilio G., Rizzo
F. (Eds.), Human Smart Cities Rethinking the Interplay between Design and
Planning. Berlin: Springer.
Dewey, J. (1938). Experience and Education. New York: Macmillan.
Franz, Y. (2015). Designing social living labs in urban research. info, 17 (4), 53-
66. Freire, P. (1970). Pedagogy of the Oppressed. NewYork: Herder &
Herder.

205
Hakkarainen, L., & Hyysalo, S. 2013. How Do We Keep the Living Laboratory
Alive? Learning and Conflicts in Living Lab Collaboration. Technology
Innovation Management Review, 3(12): 16–22.
Karvonen, A., & Heur, B. (2014). Introduction. International Journal of Urban &
Regional Research, 38, 379-392.
Kondo, M.C. (2012). An ethic of love for planning. In: Porter L., Sandercock L. ,
Umemoto K., Bates L. K., Zapata M.A. , Kondo M.C. , Zitcer A., Lake R.W.,
Fonza A., Sletto B. , Erfan A. & Sandercock L. (2012). What's love got to
do with it? Illuminations on loving attachment in planning. Planning Theory
& Practice, 13 (4), 593-627.
Maranghi, E. (2019). Networking collective knowledge to foster change. The case
of Sansheroes network (San Siro, Milan). Presented at the Annual
Conference of the Associations of European Schools of Planning –
Planning for transition, Venice, Italy on 9 -13 July 2019.
Naumann, S., Davis, M., Moore, M., & McCormick, K. (2018). Utilizing Urban
Living Laboratories for Social Innovation. In: Elmqvist R., Bai X.,
Frantzeskaki N., Griffith C., Maddox D., McPhearson T., et al. (Eds.),
Urban Planet: Knowledge towards Sustainable Cities. Cambridge:
Cambridge University Press.
Nesti, G. (2018). Co-production for innovation: the urban living lab experience.
Policy and Society, 37(3), 310-325.
Steen, K., & van Bueren, E. (2017). The Defining Characteristics of Urban Living
Labs.
Technology Innovation Management Review, 7(7), 21-33.
Vertovec, S. (2014). Super-diversity. London and New York: Routledge
Wenger, E. (1998). Communities of practice: learning, meaning and identity. New
York: Cambridge University Press.
Voytenko, Y., McCormick, K., Evans, J., & Schwila, G. (2016). Urban Living Labs
for Sustainability and Low Carbon Cities in Europe: Towards a Research
Agenda. Journal of Cleaner Production, 123, 45–54.

206
207
 Intelligent Living Lab: Supporting data-centric
decision-making using ICT tools
Minki Kim*1, Junghyun Park1, Min-ho Suh1 and Won-Kyun
Joo1

*Corresponding author
1 Korea Institute of Science and Technology Information (KISTI), South Korea

Category: Innovation Paper

Abstract
Many studies have reported a combination of information and communications
technology (ICT) tools to support scientific analyses of living lab concepts which
intend to solve various urban problems, such as urban traffic problems. Similarly,
in this study, the results of a big data analysis focusing on traffic were visualized
in a way that can be easily understood by citizens who lack specific domain
knowledge. The goal was to improve a bus route, which is a typical public
transport facility in a city. In this paper, we introduce the case study of an
intelligent living lab that utilizes visualization ICT tools based on big data related
to traffic.

Keywords: Living Lab, Transportation, ICT Tools, Big data, GIS, Decision-
making

208
1 Introduction
The living lab is a research concept which operates based on public-private
collaborations as a user- centred open-innovation ecosystem (Pallot, Trousse,
Senach, & Scapin, 2010). A living lab is literally defined in various ways, such as
a ‘living laboratory’ or ‘laboratory of daily life’, a ‘laboratory in our town’, and an
‘innovation space with user participation’ (Tang & Hämäläinen, 2012). A living
lab, unlike a common ‘laboratory’ and existing ‘test bed’ businesses, is a system
in which users voluntarily participate and study technological innovations.
Moreover, recently, the meaning of a living lab has been expanded and now can
refer to a concept that enhances governance and sustainability. In addition, living
labs are being used as experimental spaces to shift to a ‘sustainable social and
technological system’. This trend is particularly evident in ICT. In Korea, we see
that ICT tools as a medium of user participation and participation contribute to
lowering entry barriers and creating performance outcomes in various studies
(Seong & Park, 2015). In overseas cases, much early experience was gained in
the European Network of Living Labs (ENoLL), a platform established in 2006 for
ICT-based innovations (ENoLL, 2014). In addition, research has reported the use
of a living lab as a collaborative tool for integrated development with ICT tools to
develop the agriculture industry (Wolfert, Verdouw, Verloop, & Beulens, 2010).
As mentioned above, compared to overseas cases where living labs have been
actively used, Korea has more recently become aware of the importance of
citizen-led living labs.

A living lab differs from a traditional ‘research laboratory’. We consider users as

subjects of research innovation activities, not as subjects of research innovations,
and emphasize testing and demonstrations in real-life situations away from a
closed laboratory. Living labs are rapidly spreading, as evidenced by the growth
of the ENoLL, and they have received much attention from policymakers, a trend
that is expected to continue (Dell'Era & Landoni, 2014).

In this paper, we introduce an example of the application of traffic data

visualization ICT tools implemented by our research institute to a living lab called
the ‘Bus Route Improvement Committee of the Yeongjong-do Area’ to improve
an actual bus route in Incheon Metropolitan City. The purpose of this research on
an intelligent living lab is to support data-centric decision-making using
visualization ICT tools, which process and analyse traffic big data such as smart
card data secured through the cooperation of local governments. In Section 2,
we introduce living lab research cases linked with ICT tools. In Section 3, we
introduce the actual living lab research that combines traffic big data-based ICT
tools to improve a bus route, which is the focus of this research. In Section 4, we
explain the traffic- related big data-based ICT tools in detail. At the end of the
study, we present the lessons learned and the conclusion obtained through this
research and introduce future research plans.

2 Related Work
Living labs are attracting attention as a new type of approach for problems that
require long-term, effective, sustainable innovations by involving users instead of
restricting users and designing their surroundings. This is possible because living

209
labs are suitable for implementing new methods of user- centred research to
promote sustainable lifestyles and product innovation (Liedtke, Jolanta Welfens,
Rohn, & Nordmann, 2012). As a result, in order to ensure policy sustainability for
various issues in the future, the utilization of living labs is expected to become
more widespread.

Recently, different municipalities have built smart cities by networking all

infrastructure units in the city using advanced ICT and big data. A smart city is
defined as a city where citizens can enjoy a convenient and comfortable life after
traffic, environment, and housing problems, and facility inefficiencies associated
with city life have been solved utilizing ICT technologies. Each municipality
actively uses the smart city construction process together with a living lab to solve
various problems in the city. Examples of such smart cities are ‘U-City’ in Songdo,
Incheon, and smart-life cities based on the 3D Internet in Japan (Prendinger, et
al., 2013). In addition, based on the development of information technology
systems that enable measurements of odor emissions and the collection of citizen
feedback, the living lab approach is being implemented to include citizens, public
authorities, and industry and environmental non-governmental organizations
(NGOs) (Reiter, Gronier, & Valoggia, 2014). There is also an interesting study on
a new urban economic model that distributes information gathered and generated
through smart city construction to the marketplace (Cosgrave, Arbuthnot, &
Tryfonas, 2013). Thus, in the future, many studies of smart cities and living labs,
as well as ICT tools to support data- centric scientific analysis, will be conducted
at the basic methodology level in an attempt to solve various urban problems
more efficiently.

3 Case Study: Intelligent Living Lab

In this paper, we introduce the ICT tools implemented by the living lab entitled
‘Bus Route Improvement Committee of the Yeongjong-do Area’. As introduced in
Section 2, Incheon city plans to make the entire city a smart city, starting with
Songdo ‘U-City’. The living lab concept can also be used to solve various urban
problems, such as fine dust, flooding, and earthquakes and can be used to
improve public transportation. In particular, Incheon City and the Korea Institute
of Science and Technology Information (KISTI) signed an MoU in July of 2018 to
demonstrate KISTI’s competence as a data agency and to carry out various
studies to solve urban problems in Incheon City.

In Incheon city, which is expected to overhaul the bus route in 2020, we present
here a case study of a living lab called ‘Bus Route Improvement Committee of
the Yeongjong-do Area’, which is one of various efforts to improve public
transportation routes, a popular topic at present. We are interested in improving
public transportation in Incheon because this city is the third-largest city in Korea,
with a mainly metropolitan population, and because Incheon City has
experienced numerous fiscal deficits since the introduction of the semi-public bus
system in 2009. In the Yeongjong-do area, Incheon International Airport, which
opened a second passenger terminal on January 18, 2018, is located, and many
new apartments have been built according to new city policies with titled such as
‘Yeongjong International City’. As a result, traffic levels in the Yeongjong-do area

210
are expected to increase steadily. Although Yeongjong-do is an island, users of
public transportation can enter Yeongjong-do through only two major bridges (the
Yeongjong Bridge and the Incheon Bridge) which connect the inland to Incheon
City. Related to the various characteristics described above, in Yeongjong-do,
public transportation-related complaints are relatively frequent relative to the rest
of Incheon City. Meanwhile, Incheon City paid 59.5 billion won in 2016 and 106.2
billion won in 2018 after it started to use a semi- public bus system in 2009. As a
result of the increase in subway users, the proportion of bus users was 49.3% in
2016 compared with 43.3% in 2018, and the share of buses out of all public
transportation is steadily decreasing. In general, adjusting transit intervals and
routes to increase citizens’ convenience increases the overall cost of transit
operations. In this study, we utilize a living lab that integrates ICT tools to
undertake bus route optimization in order to improve the convenience of public
transportation and the efficiency of bus routes, considering a trade-off
relationship.

In addition, research is underway in the agency to carry out the bus route
reorganization work and to understand the functions desired by city government
officials and policymakers through interviews and then to implement them on the
web. The web system currently being implemented includes all of the basic
functions of public transportation research, such as an economic analysis
according to changes in departure times, calculations of the bend radius of each
bus route, as well as user-customized functions obtained through interviews with
city government officials in Incheon City. In the future, we believe that this traffic
data analysis based on the web will become the basis for constructing a real- time
public transport control system (Amini, Gerostathopoulos, & Prehofer, 2017).

4 Use ICT Tools to Support the Living Lab

In this study, Tableau (Tableau Software, 2003) was used as an ICT tool to
support the living lab to improve bus routes. In addition, with regard to traffic-
related big data, which Tableau cannot easily handle, we used Python’s NumPy,
Pandas, and other libraries for programming using the Python programming
language with Jupyter Notebook. Additionally, the traffic network is constructed
with traffic data such as bus routes and stops collected through a bus information
system (BIS) and bus management system (BMS) for integrated visualization
using ArcGIS (Esri, 1999).

4.1 Smart Card Data

In this study, we investigated the level of demand for buses in order to implement
the bus route improvement process. For this purpose, we obtained traffic card
data from the Korea Smart Card Company and from Eb Card, which operate
automatic fare collection (AFC) systems. According to earlier work (Song, Eom,
Lee, Min, & Yang, 2015), Seoul, the capital city of Korea, has more than 10 million
transits per day, according to the smart card system. As a result of checking the
traffic card data of Incheon City in this manner, similar to the Seoul case, as
shown in Table 1, the traffic card usage records of nearly 15 million instances per
weekday are shown. The largest difference between the Korea smart card and

211
the Eb Card traffic card data is that Eb Card provides detailed information about
each transfer.

Table 1: Data from the Smart Card Automated Fare Collection Systems

Date Korea Smart Card Eb Card

File size (GB) # of records File size (GB) # of records
Sunday, July 1, 2018 1.06 6,932,275 1.64 5,152,984
Monday, July 2, 2018 2.16 14,084,184 3.22 10,025,347
Tuesday, July 3, 2018 2.3 15,011,778 3.44 10,697,155
Wednesday, July 4, 2018 2.35 15,352,585 3.52 10,949,652
Thursday, July 5, 2018 2.33 15,215,930 3.49 10,870,453
Friday, July 6, 2018 2.44 15,939,027 3.68 11,471,180
Saturday, July 7, 2018 1.8 11,802,249 2.78 8,718,348
Thursday, November 1, 2018 2.38 15,568,227 3.54 11,033,829
Friday, November 2, 2018 2.47 16,154,709 3.71 11,564,515
Saturday, November 3, 2018 1.89 12,447,079 2.94 9,224,146
Sunday, November 4, 2018 1.41 9,265,914 2.2 6,915,418
Monday, November 5, 2018 2.33 15,266,342 3.46 10,777,226
Tuesday, November 6, 2018 2.36 15,464,232 3.51 10,903,731
Wednesday, November 7, 2018 2.37 15,491,086 3.51 10,921,823
Sum 29.65 193,995,617 44.64 139,225,807

Therefore, as shown in Table 1, the file size is large despite the small number of
records. Essentially, the contents to be introduced later in this paper are based
on traffic card demand data as introduced in the current section.

As a result of analysing the traffic card data obtained from Incheon City, it was
found that 99% of users of Incheon Metropolitan City bus users use the
transportation card as opposed to paying cash. In addition, the dropout rate of
the endpoint information of traffic card data was 1~2% of the total data. This is in
contrast to the results of studies in which most AFC systems only record boarding
information (Li, Sun, Jing, & Yang, 2018). Incheon city's traffic card data is better
than Seoul’s traffic card data because Seoul has a single fare system for bus use
only, while Incheon Metropolitan City has a metropolitan unity fare system that
uses distance-proportional rates, as shown in the list below:
● Higher-grade bus: Basic charge within 10km, additional 100 won at every
5km for 10~40km, additional 100 won charged after 40km
● M bus (wide-area express bus): Basic charge within 30km, additional 100
won at every 5km for 30~60km, additional 100 won after 60km
● Subway: Basic charge for 10km, additional 100 won at every 5 km for
10~50km, additional 100 won at every 8km after 50km

4.2 Origin to Destination Matrix (ODM)

In this study, the origin to destination matrix (ODM) shown in Figure 1 was created
by calculating the numbers of passengers at boarding stops and the numbers of
passengers departing at stops according to the traffic card data introduced in
Section 4.1. After evaluating the demand based on the generated ODM, with
different colors for each grade, it was possible visually to grasp at a glance the
usage demand according to each stop along each route. In order to express this,

212
first a set of a boarding stops and a set of departing stations are obtained; when
the number of passengers for each boarding and departure stop pair is combined,
the above-mentioned ODM is generated. Modeling and analysis with a traffic
network based on a geographic information system (GIS), as described in Section
4.3 based on the ODM built here, can also be performed.

Destionation
A B C D E F G H I J K
A
B
C
D
E
F
G
H
I
Origin

J
K

Figure 1: Origin to destination matrix (ODM)

Another important factor when attempting to measure the demand for public
transportation is the level of peak-hour customer demand. It becomes possible to
adjust the allocation of vehicles according to the level of demand for buses when
dealing with different time zones and to grasp the current state of a route to judge
the efficiency of bus route management. Based on this information, citizens’
convenience can be improved by reducing the number of buses on routes with
relatively few passengers at peak times and by placing more buses on routes
with a large number of passengers at such times. In order to make these analyses
possible, in this research, we present the number of passengers according to the
travel time for each route in Figure 2. In this visualization, the 23 routes passing
through Yeongjong-do, Incheon City are expressed in different colors. As a result,
it is possible to understand the peak hours of routes, such as when people go to
work and when they return home, and the number

213
 Figure 2: Number of passengers on each Figure 3: Comparison of usage rates for
route by time frame each route

Figure 4: Tableau online dashboard

of passengers according to the time of each route. This also enables a proper
efficiency analysis of the overall bus operation. Further, as shown in Figure 3,
one can intuitively confirm the share of each route by confirming the utilization
rate in each case, as expressed here by a pie chart. Hence, based on the
information of stops along each route, we determine the number of passengers
on the buses by time zone through passenger information, specifically who
boarded and departed the buses at each stop. This visualization analysis to
determine basic public transport levels represents how a living lab can improve
bus routes in areas when civil servants and those living in the area work together.

214
This can be used to support data-centric scientific decision-making for citizens
who participated in a living lab. Through these ICT tools, we were able to confirm
the claims based on the experience of civil servants and citizens with actual data,
which was used as a tool to verify the validity and usefulness to meet all
concerned parties’ needs. We will deal with the results of using the materials
created here in the living lab in Section 5. Finally, readers reading this article can
also review the traffic demand data configured as shown in Figure 4 through a
dashboard provided through links provided here. The dashboard presented here
is identical to that used by the living lab discussed in this study. Figure 4 shows
the tableau online dashboard as an integrated representation of Figures 1 to 3
described in the above sections. The graphs on the dashboard were designed
with reference to the graphs of web pages provided by Incheon city to the citizens.
In addition, it was customized to visually check the lack of bus data provided by
Incheon city. This Tableau dashboard is designed to be used as reference
material for numerical presentations based on current data on the opinions of
living lab participants. For this purpose, it was designed to be as intuitive as
possible for non-experts to utilize, and experts participated in the actual living labs
to support the use of ICT tools and to explain current situation based on actual
data. This dashboard was used to support the decision making through the
confirmation of facts about the opinions and the status of the opinions after the
experts had heard the opinions of the living lab participants. The dashboard,
which can be checked via the web, supports faceted searches and interactions
with each route and bus stop. These different ICT tools as described here were
leveraged to help make data-centric decisions in the living lab to improve bus
routes.

4.3 Traffic Network Modeling at Yeonjong-do

4.3.1 Yeongjong-do Bus Routes
Incheon Metropolitan City has a web portal service for citizen based on the
ArcGIS platform (Figure 5) (Incheon Metropolitan City, 2014). Most of the area of
Incheon City is within a traffic network, but Yeongjong-do does not provide web
portal service due to regional specificity. Figure 6 shows the integrated traffic and
speed data management screen currently provided by Incheon, showing that the
Yeongjong-do area is excluded (i.e., the island on the left).

In order to analyze the optimal route of public transportation, the network of the
target area (transportation network) is required. The transportation network is
constructed using the navigation network and national standard data based on
GIS. The design of the transportation network must be extendable to reflect the
flows of various modes of transportation, such as taxis, buses, and trains. The
traffic network of Incheon is divided into a road network and a public transport
network considering the characteristics of both, and it is composed of nodes and
links. In the road network, a node represents a major intersection point, and a link
represents a unique road between nodes.

215
 Figure 5: Smart-GIS of web portals at Incheon Metropolitan City

Figure 6: Visualization of traffic volume and speed data of Incheo

In the public transport network, a node represents transport between nodes that
link the main station (bus stop or other stop or station). In order for network
composed of nodes and links to represent real-world networks, links and nodes
must have certain attributes necessary for this analysis. For a road network, for
example, a node indicates an intersection or an end of a road. A link must also
include information about the costs associated with the use of the road (Chae,
2011). Different types of cost information can exist, such as the travel time on the
road and travel distances depending on the application.

216
 Bus stop (b) Link & node

(c) Bus traffic network

Figure 7: Bus traffic network using Incheon regional GIS data

In this study, a new bus network based on the Incheon traffic network is devised
to support GIS for the optimal reorganization of the bus routes in the Yeongjong-
do area. The bus network was established for all bus routes entering Yeongjong-
do using information about bus stops, intersections, and links in the
shapefile(.shp) format provided by Incheon in 2018 (Incheon Metropolitan City,
2011). The bus network consisted of 5402 bus stops, 3093 nodes, and 7774 links,
with a total of 24 bus routes (Figure 7). Bus routes include a range of information,
including station information (station ID), route information (route ID), operation
schedules (timetables), average running speeds (km/hour), and other data. Using
correlations between data points and a network algorithm which determined
shortest paths, optimal bus routes were created. Figure 8 shows the 23 bus

217
routes (Bus routes number: 1, 2, 2-1, 3, 4, 111, 117, 202, 203, 204, 222, 223,
302, 303, 304, 306, 307, 308, 320, 303-1, 310, 330, 598) entering Yeongjong-do
for bus network analysis. An analysis of the transportation network shows that
the bus route network to Yeongjong-do is reasonably operated, but problems
exist on some routes.

Figure 8: 23 Bus routes to Yeongjong-do

In 2018, Yeongjong-do opened a second passenger terminal at Incheon

International Airport, and additional bus routes were added. As a result, a bus
route with a travel distance of 120 km and a travel time of 2 hours was created.
Figure 9 shows the five longest routes out of the newly added bus lines. The five
bus lines seem to have improved convenience on the user side, but the bus
driver's job intensity has greatly increased. This is not only a decline in the quality
of the bus service but also the safety of bus users. These bus routes can improve
the problem by ensuring that the route is divided or that the bus operator takes a
break.

4.3.2 The curvature of Bus Routes based on ODM

Route curvature is one of the most important indicators of route design related to
route optimization. The degree of curvature of a route has a value of 1.0 when
connecting the starting and endpoints of a certain route at the shortest distance,
with this value calculated according to the actual operation of the route. The
curvature is not considered to have an adverse effect on the routing system
unconditionally, but it is considered to be the most important factor when
evaluating a bus route because this is recognized as necessary to improve the
time, ensure a short travel time and to minimize the time cost.

218
Figure 9: Five longest bus routes to Incheon International Airport Terminal 2

219
 Figure 10: OD cost analysis using a network analysis based on ArcGIS pro

As the OD information of actual bus users has released been recent, an OD

matrix-based route bend analysis has been suggested as the most effective
evaluation index (Park, Ha, Kwon, & Oh, 2019).

A combination of the ODM constructed in Section 4.2 and the Yeongjong-do bus
routes were used to analyze the curvature experienced on a per-user basis. In
order to calculate the degree of curvature, a network analysis was conducted
using the origin-destination cost matrix provided by ArcGIS (Figure 10). Figure
11 shows the curvature OD dissolution and curvature distribution calculated by
the network analysis. The average degree of curvature was 1.17 for 14 days, and
most users were not greatly inconvenienced when traveling to Yeongjong-do
using the current bus route. However, on some routes, users may be
inconvenienced with the curvature 2.0 or more. For these routes, it is necessary
to reflect user-based policies such as the addition of new routes to reflect
passenger numbers and changes of bus stop locations based on population
densities.

6 Conclusions
Here, we introduce the concept of the living lab, introduce a living lab which used
ICT tools, present an overview of ICT tools based on traffic data, and describe a
case study conducted by an intelligent living lab. While concluding the paper, we
will also discuss the lessons learned through the actually implemented intelligent
living lab and then introduce future research plans.

First, while carrying out this research, we confirmed that it is a desirable policy
direction to introduce the living lab concept for sustainable policy setting and
troubleshooting. In particular, while investigating various research cases related
to living labs, we were able to confirm that introducing this type of living lab
concept to solve various social problems is a worldwide trend. Similarly, we
identified various research cases that combine ICT tools to support data-centric
decision-making for the general public when they participate in living labs. This
shows that the living lab concept will evolve intelligently. For the bus route
reorganization carried out here, the case of an actual living lab with ICT tools

220
based on traffic-related big data to support scientific decision-making was
explained.

Among the four major urban problems (traffic, flooding, earthquakes, and fine
dust) for which our research team is currently constructing solutions, the most
important issue is fine dust. However, in order to carry out research successfully
on public transport, used by many citizens from the past to the present and the
future, our research team put in a great effort. Furthermore, for Incheon City, the
focal point of this research, cooperation between businesses and our research
team is also succeeding, and we are planning to reorganization bus routes
completely in 2020. Given this situation, the living lab is operated to determine
the optimum points of improvement in the trade-off between convenience for
users and the efficiency of bus operations. The coordination of existing bus routes
has mostly been determined by complaints from citizens who use the buses and
from urban planning events. As a result of rule-of-thumb management, Incheon
City has been increasing the size of the budget, nearly in a snowball manner,
since the introduction of the semi-public bus system. This is done to support the
living lab to improve bus routes, to support traffic-related big data-based ICT
tools, and to provide objectivity to assertions and immediately consider citizen
claims stemming from the data gathered to the greatest extent possible. In
addition, by helping citizens access the web anytime and using big data
visualization ICT tools, this effort is helping citizens given the scientific analysis
conducted to improve their bus routes. While carrying out this research, we were
able to confirm the good aspects of using ICT tools, but we also found that some
items cannot be confirmed using data alone. For example, in the case of Incheon
City, some private buses are operated, but their routes show various utilization
rates, and the rates and times of operation are paradoxical when attempting to
reduce the deficit through route adjustments to reduce traffic and the number of
operating buses. Because the analysis used only data from a fixed period
provided periodically, the seasonality analysis was insufficient. Solving these
problems is limited in that they require much time from the actual traffic data
collection stage. Therefore, using data pipeline tools such as ‘OpenAPI’ to
automate the processes of data collection, processing, and analysis we allow us
to save time when addressing the above items. Based on previous research, the
methodology presented in this paper will be utilized for the total reorganization of
bus routes in Incheon in 2020 and will be expanded further to research on other
forms of public transport, such as subway including buses.

221
 Figure 11: OD flow maps (upper panel) and distribution of curvature (lower panel) to
Yeongjong-do

As the problems to be solved grow, artificial intelligence for traffic route

optimization (AI Solver), which we are continuing to study, together with the
supercomputing resources of KISTI, can be used to cope with this. In addition, in
conjunction with the other urban problems (flooding, earthquakes, and fine dust)
dealt with by our current research team, we are also promoting research on bus
route optimization through data fusion methods. Finally, various urban problems,

222
including the traffic problems discussed here, are problems that occur not only in
Korea but also in the cities of other countries. Accordingly, the goal of the
research that our research team ultimately pursues is for our research to be used
in many different cities all around the world.

Acknowledgements
This research was supported by Ministry of Science and ICT, Republic of Korea
(Project No. K-19- L05-C01-S01). Research is being conducted to solve various
urban problems such as traffic problems through cooperation between Incheon
City and our institute. We would like to thank the local government for their help
giving much assistance for the research on the data-centric solution of social
problems.

References
Amini, S., Gerostathopoulos, I., & Prehofer, C. (2017). Big data analytics
architecture for real-time traffic control. 2017 5th IEEE International
Conference on Models and Technologies for Intelligent Transportation
Systems (MT-ITS) (pp. 710-715). IEEE.
Chae, W. (2011). Analyzing the service effectiveness of the bicycle road plan by
GIS network analysis method. Master's thesis.
Cosgrave, E., Arbuthnot, K., & Tryfonas, T. (2013). Living labs, innovation
districts and information marketplaces: A systems approach for smart
cities. Procedia Computer Science, 16, 668-677.
Dell'Era, C., & Landoni, P. (2014). Living Lab: A methodology between user-
centred design and participatory design. Creativity and Innovation
Management, 23(2), 137-154.
ENoLL. (2014). Retrieved from European Network of Living Labs:
https://enoll.org/
Esri. (1999). ArcGIS Online | Interactive Maps Connecting People, Locations &
Data. Retrieved from ArcGIS: www.arcgis.com
Incheon Metropolitan City. (2011). Incheon Bus Information Management
Systems. Retrieved from http://bus.incheon.go.kr
Incheon Metropolitan City. (2014). Smart-GIS Incheon. Retrieved from
https://imap.incheon.go.kr/ Li, T., Sun, D., Jing, P., & Yang, K. (2018).
Smart card data mining of public transport destination: A
literature review. Information, 9(1), 18.
Liedtke, C., Jolanta Welfens, M., Rohn, H., & Nordmann, J. (2012). LIVING LAB:
user-driven innovation for sustainability. International journal of
sustainability in higher education, 2(13), 106-118.
Pallot, M., Trousse, B., Senach, B., & Scapin, D. (2010). Living lab research
landscape: From user centred design and user experience towards user
cocreation. First European Summer School "Living Labs".

223
Park, M.-C., Ha, T.-J., Kwon, S.-D., & Oh, S.-J. (2019). Improvement of Bus
Route System Considering Route Curvature. JOURNAL OF THE
KOREAN SOCIETY OF CIVIL ENGINEERS (pp. 93-103). Korean Society
of Civil Engineers.
Prendinger, H., Gajananan, K., Zaki, A., Fares, A., Molenaar, R., Urbano, D., . . .
Gomaa, W. (2013). Tokyo virtual living lab: Designing smart cities based
on the 3d internet. IEEE Internet Computing, 30-38.
Reiter, S., Gronier, G., & Valoggia, P. (2014). Citizen involvement in local
environmental governance: A methodology combining human-centred
design and living lab approaches. Electronic Journal of E-Government,
12(2), 108.
Seong, J., & Park, I. (2015). Living Lab as Transition Arena : Case Analysis and
Implication.
Proceedings of the Korea Technology Innovation Society Conference, (pp. 907-
926).
Song, J.-Y., Eom, J., Lee, K., Min, J., & Yang, K. (2015). Public transportation
service evaluations utilizing seoul transportation card data. Procedia
Computer Science. 52, pp. 178-185. Elsevier.
Tableau Software. (2003). Business Intelligence and Analytics Software.
Retrieved from Tableau: https://www.tableau.com/
Tang, T., & Hämäläinen, M. (2012). Living lab methods and tools for fostering
everyday life innovation. 2012 18th International ICE Conference on
Engineering, Technology and Innovation (pp. 1- 8). IEEE.
Wolfert, J., Verdouw, C., Verloop, C., & Beulens, A. (2010). Organizing
information integration in agri-food—A method based on a service-
oriented architecture and living lab approach. Computers and electronics
in agriculture, 70(2), 389-405.

224
225
 Open Innovation Camp (Oic) – A Tool for Solving
Complex Problems Rapidly
Teemu Santonen1*, Julia Nevmerzhitskaya1, Aletta Purola1,
and Harri Haapaniemi1

* Corresponding author
1 Laurea University of Applied Sciences, Finland

Category: Innovation Paper

Abstract
This paper proposes Open Innovation Camp (OIC) concept as a novel
methodological solution for overcoming the constraints on upscaling Living Lab
experiments. OIC is co-creation sprint type of multi-day event grounded on an
open innovation 2.0 principles where a group of carefully selected stakeholders
having diverse but complimentary expertise creates a common understanding of
(a complex societal) challenge and work together to develop in a co-creative
manner user centred concepts and solutions to pre-defined challenges in a set
timeframe. Based on the experience from the OIC implementation it is suggested
that OIC can be an effective tool for overcoming 1) lack of time and financial
resources, 2) unbalanced stakeholder representation, 3) silo effect in co-creation
activities problems. Based on the feedback analysis of 47 OIC participants, it is
suggested that OIC provides the most value, when implemented in the very
beginning of a project. How OIC can help to overcome multi-stakeholder
engagement constrains in Living Lab setting is discussed.

Keywords: innovation camp, open innovation 2.0, design sprint, service design,
design thinking, complex problem, co-creation, quadruple helix

226
1 Introduction
Solving complex problems (Murthy, 2000) – also known as ill structured problems
(Simon, 1973) or wicked problems (Navarro et al. 2008) – requires socio-
technological environments that bring together people with different,
complementary, and often controversial knowledge and skills. Diversity can be
associated to any attribute to indicate that another thing, person, group,
organization, network or ecosystem is different (adapted from Williams and
O’Reilly, 1998). In an innovation process, a novel thinking outside the box can be
boosted when diverse people having complimentary skills and knowledge follows
open innovation principles (Chesbrough, 2006). However, the participant
diversity can also reduce innovation performance due too high level of task
conflicts (also known as cognitive conflicts), which can be defined as perceived
disagreement among group members relating their opinions and ideas (Simons
and Peterson, 2000). In worst case scenario task conflicts are causing
relationship conflicts, (or emotional conflicts), between the group members and
leading to negative impact on group satisfaction, commitment and decision
quality. Thus, effective facilitation and management of the development and
innovation efforts is among the key challenges of any open innovation activity
which is grounded on a multi-stakeholder collaboration including Living Lab
approach.

Service design (SD) (Zomerdijk & Voss 2010) and design thinking (Brown, 2008)
have become a central framework to co-create novel solutions. It is about
planning, developing and innovating product, services and concepts through
specific iterative development processes while utilizing various methods,
techniques and tools. The main purpose of SD is to create a customer-centric
experience that meets the needs and demands of the end-customers but also
fulfils the business objectives. It is argued that through a SD approach, diverse
teams can collaboratively identify needs, ideas, experiences, opportunities and
generate fast prototypes to be tested by the real users and customers. SD helps
to innovate (create new) or improve (existing) services to make them more useful,
usable, desirable for customers and efficient as well as effective for the
organization.

1.1 Objective and structure of this study

The goal of this study is to introduce Open Innovation Camp (OIC) concept, which
in relatively short-timeframe brings together in a controlled manner a diverse set
of actors having complimentary skills to rapidly co-create visions and practical
solutions for complex problems, which can further to be developed and tested
e.g. in series of Living Lab activities.

The structure of this paper follows a constructive action research paradigm

(Cassel and Johnson, 2006) process which is a methodology to develop solutions
to a practically relevant problem by applying theoretical knowledge and
demonstrating the functioning and innovativeness of the suggested solution in
real life (Jaatinen and Lavikka, 2008). The structure follows a process framework
originally proposed by Kasanen et. al. (1993) and refined by Oyegoke (2011) as
follows:

227
 1) Justify the practical relevance of the proposed problem (i.e. the
challenge of solving complex problems in section 1),
2) Present the theoretical connection (i.e. open innovation 2.0,
management of participant diversity and design sprint type of
approaches justified in section 2),
3) Construct the solution (i.e. OIC-concept description as presented in
section 3),
4) Demonstrate that the suggested solution is working (i.e. real-life
implementation of the OIC and collecting feedback from OIC
participants) and
5) Present the research contribution including applicability of the solution
(i.e. discussion and conclusions of this study as presented in the
section 5).

2 Theoretical foundations of Open Innovation Camp (OIC)

concept
2.1 OIC as an Open Innovation 2.0 approach
Theoretical foundations of OIC are grounded on Open innovation approach
(Chesbrough, 2006). Open innovation provides a generic framework for a
distributed innovation process and knowledge flow management across
organizational boundaries by using pecuniary and non-pecuniary mechanisms to
support organization’s business model (Chesbrough and Bogers, 2014).
However, OIC as a tool to solve complex problems needs to go beyond facilitating
collaboration between individual organizations to achieve its’ ambitious goals.
Therefore, OIC leans more on the Open Innovation 2.0 (OI2) framework, which
emphasis ecosystem centric cross-organizational innovation grounded on the
collaboration between quadruple helix actors (Curley & Salmelin 2013).
Quadruple Helix model (Carayannis and Campbell, 2009) describes an
innovation system where government, industry, academia and civil society work
together to co-create the future and drive structural changes far beyond the scope
of what any single organization or person could do alone. As a result, OIC
provides possibilities to have sufficient capability to successfully integrate the
information obtained from the external sources into internal processes which is
crucial for open innovation to be effective (Nonaka, 2007).

2.2 Enhancing creativity by managing participant diversity

Careful management and diverse inclusion of complementary stakeholders is an
important part of OIC. This suggestion is in-line with stakeholder theory
(Freeman, 1984) and stakeholder engagement in an open innovation processes
as suggested by Gould (2012). Importantly, successful stakeholder engagement
in OIC will go beyond the acquisition of specific information from external experts
to “trust and relationship building” and “mutual understanding” among various
innovation ecosystem actors.

Participant diversity management in OIC is based on a model proposed by

Santonen (2016), which includes cultural, organizational, user-driver, cross-
functional, disciplinarity and cross-industry diversity as presented in Figure 1.

228
 Figure 1: Diversity management model (Santonen, 2016)

Cultural diversity is one of the major challenges in the 21st century, which have
a direct impact on the innovation acceptance. In the suggested diversity
management model, cultural diversity reference is two folded. Cultural diversity
refers to a need to recruit participants from different countries in which the market
conditions and code of conduct are differing. Organizational diversity is linked to
Quadruple Helix model, which is the foundation of the OI2 approach as suggested
by Curley & Salmelin (2013). User-driven diversity highlights the need to
understand different kind of end-users, who are expected to use the co-created
solutions. Personas derived from service design methodology (Zomerdijk & Voss
2010) are archetypes of actual users and can be successfully used to verify user-
driven diversity. Cross-functional diversity is related to an idea of a cross-
functional team in which is a group of people with different functional expertise
are working toward a common goal (Kahn 1996). Basically, this referrer to making
sure that OIC includes persons having different job descriptions such as R&D,
marketing or management. Finally, disciplinarity diversity and cross-industry
diversity are referring to involving participants from different scientific disciplines
or industries.

By considering the above participant diversity dimensions and making

consciously decisions who to recruit to OIC, the likelihood of co-creating more
radical innovations should increase. Furthermore, the participant recruitment
challenges within Living Lab projects have been identified as a major constrain
to on upscaling Living Lab experiments (Dijk et al. 2019). Therefore, more
systematic stakeholder management is essential for Living Lab success.

2.3 OIC as a design sprint

229
Methodologically OIC concept belongs to a family of several time-constrained
agile development exercises such as hackathon, design sprint, service jam,
innovation camp, solution camp and entrepreneurship camp. The definitions of
the above terms vary, but typically these approaches can be characterized by
following attributes (Halvari et. al 2019): 1) short time-bounded event, 2) intense
collaboration, 3) competition, 4) collocation, 5) offline: people meet locally, 6)
ideation, experimentation and creativity, 7) teams, 8) pitching/presenting and 9)
recognition. Hackathons are team-based coding or product development
marathons, which originates from the 1960s, but became popular soon after the
turn of the millennium (Halvari et. al 2019). Scientific Hackathons can be used for
deep data analysis as an agile interdisciplinary collaboration between
organizations and researchers (Ghouila, 2018). Hackathons can also be
competitive rather than collaborative events (Richterich, 2017). Design sprints
are co-creation events focusing on testing prototypes. Probably the most popular
Design sprint approach is the five-day model which is inherited from the Google
(Knapp et. al. 2016) and includes idea, build, and launch and learn stages.
Service jams are defined as a type of short-term innovation communities (Römer
et al, 2011), which usually last 48 to 72 hours and bring together thousands of
experts and interested people globally in order to openly work on predefined
challenges, problems, or topics. Most known format today is the Global Service
Jam (GSJ, 2018), which is a volunteer community of service design experts.
Innovation Camps can take different forms. For example, the Aalto Camp for
Societal Innovation (ACSI) is addressing societal challenges via open-ended
challenges while relying on self-organising working approach (Rissola et al,
2018). Innovation camps (also known as solution camps or entrepreneurship
camps) can also be used for educational purposes (Bager 2011). In this type of
an event students with other stakeholders are co-creating solutions for innovation
challenges while learning team building, creativity and innovation skills.

As a result, it is argued that OIC type of events can take many forms depending
on the thematic focus of the event. Based on the above theoretical foundation
following definition for Open Innovation Camp (OIC) is proposed for purpose of
this study:
“Open Innovation Camp (OIC) is co-creation sprint type of multi-day event
grounded on an open innovation 2.0 principles where a group of carefully
selected stakeholders having diverse but complimentary expertise meet
locally and creates a common understanding of (a complex societal)
challenge and work together in teams to develop, present and review in a
co-creative manner user centred concepts and solutions to pre-defined
challenges in a set timeframe”.

3 The key characteristics of Open Innovation Camp concept

3.1 Roles, stakeholders and working groups of the OIC
OIC is a co-creation event, which follows the definition of co-creation as an “act
of collective creativity shared between two or more people” (Sanders et al, 2008).
It means that planning and implementing OIC requires collective effort of different
stakeholders with defined roles. The main roles of OIC are described in Table 1.

230
 Table 2. OIC roles

Role Definition Main tasks during the OIC

The main organizer who is
setting the frame of the OIC and
OIC acting as a mediator between
Orchestrator other involved parties. Similar to
Camp Convener role (See
Rissola et al, 2017, p.57)
Representative of an
organization to which the
challenge is of a strategic
importance, who sets up the
scope of a challenge, and who is
Group motivated, direct interest, and
Owner means for solving the challenge.
Have substantial understanding
of the given challenge. Similar to
a challenge owner, case owner
or product owner used in design
sprints.
Person who facilitate people's
expressions of creativity at all
Facilitator levels (Sanders et al, 2008).
Experts in service design and co-
creation.
People taking part in OIC. OIC
participants are internationally
recognized experts in their field
Participant and end-users who have been
selected based on the participant
diversity management framework
presented in section 2.2
- Overall content and outcomes
planning, coordination and practical
arrangements
- Recruitment of challenge owners,
facilitators and participants
- Being a host and main point of
contact during the OIC
- Create background materials for the
challenge
- Introduce the challenge to the team
during OIC
- Help the team to answer content-
specific questions during creative
process
- Experts in the challenge field.
-
Ensure that creative process is
implemented according to the plan
- Ensure that the right set of service
design tools is used to unleash the
co-creation potential of a diverse
group of experts
- Bring people into the design process
in the ways most suitable to their
ability to participate
- To bring the expertise based on the
role he/she is representing in the OIC
- To share their knowledge with other
and co-create new ideas towards
solving the challenge

The four defined roles can be further divided into subgroups. For example, the
role of an orchestrator can be shared between different stakeholders, one being
responsible for practical arrangements and OIC logistics, and another one for
content and facilitation. Also, the challenge group owner role in case of a
challenge related to a specific business model, or consumer understanding, can
be performed by a researcher or a consultant, whereas group owners of
sectoral/industry-specific challenges are usually represented by business
decision-makers.

During the camp, participants belong to a home group, which composition is

defined by following the participant diversity guidelines. The home group
composition remains the same during the whole OIC event. Two types of home
group are identified: Industry and Thematic home group. Industry group
participants represent various industry ecosystem expert roles in the given
industry. They have profound knowledge on this specific industry field. Thematic

231
group member represents experts who have specialized skills in a specific
thematic field area, which can be used in multiple industry setting.

3.2 The main OIC phases

The preparation and execution of the OIC follows the three stages described
below: the planning phase, the implementation phase, and the communication
phase.

PLANNING PHASE: In the planning phase the topics and scope to be addressed
during the OIC are first collaboratively defined by the OIC Orchestrator and Group
Owners. Once the agreement is achieved, the collection of the background
information and preparation of the starting point materials and challenge
descriptions can start. The Group Owners have the main responsibility in this
process. Based on the OIC scope and availability of the background information,
OIC Orchestrator will define the duration and structure of the camp as well as
select suitable service design tools for the camp in collaboration with the
facilitators. Defining, inviting and recruiting participants based on what
complementary expertise is also an essential part of the planning phase.

To ensure the optimal balance of participants, recruitment process includes

invitation and application processes. An invitation is a process where
orchestrator(s) and group owners sends a personal request to an expert to join
the OIC. Only those experts who are considered as the “world’s leading experts”
within their field and their presence in OIC will provide exceptional value for the
camp, receive an invitation. All other participants is suggested to go through an
application process in which applicants describe their motivation to participate;
their relevant expertise, and argue what would be their potential contribution to
the open innovation camp. To follow participant diversity management practices
described in section 2.2, the desired overall composition of the OIC must be pre-
defined before starting the recruitment process. However, the optimal overall
composition is always depending on the given challenges of the OIC. Finally,
practical event arrangements and budgeting are also carried out in planning
phase.

IMPLEMENTATION PHASE includes carrying out the OIC according to

predefined plan and making adjustment to camp program and tools if needed
during the camp. First, concentrating on creating the team spirit and enabling
creative atmosphere. Next managing design thinking workflow, and collaborative
decision making on selected concepts to be further developed e.g. in Living Labs.
Finally collecting feedback from the participants as well as documenting and
analysing the key contributions and impact of the OIC.

COMMUNICATION PHASE is about sharing the final deliverables with the OIC
participants and to a wider audience. The hopefully positive OIC experience aims
to further engagement of the participants into follow-up activities such R&D,
testing and demonstrations in Living Labs. Also maintaining established
relationships with OIC participants by inviting them into external advisory group
of the possible follow up project.

232
3.3 OIC content structure and daily program
The OIC structure is designed taking into consideration the complexity and
diversity of the challenges to be addressed by a high number of experts who are
not familiar with each other and the context of the challenges. Regardless of the
challenge, the daily programme follows the basic structure presented in Figure 2.

Figure 2. OIC daily structure

PHASE 1: Creating shared understanding and trust: The first day of the OIC
is designated to getting to know each other and building trust among the
participants, in order to generate a secure creative open innovation culture.
These activities include a series of ice breakers and facilitated introductions
among participants (Preziosi, 2006). Alongside getting to know each other, a
shared understanding of the OIC goals, outcomes and vision is co-created which
together are forming the foundation for the shared commitment. The shared
understanding will be used as a benchmarking tool for reviewing the generated
ideas during the final day of OIC as well as later on during follow up Living Lab
activities.

PHASE 2 (to max day 4): The day 2 to until the day 4 (depending on the duration
of the OIC), are grounded on the series of short-term design thinking workshops
in which various working groups are collectively and iteratively developing
solutions for the defined challenges. During the workshop’s participants work in
cross-group teams to co-create and refine the concepts and practical solutions
which are expected to solve the defined challenges. The facilitated workshop are
all about sharing the knowledge and learning from each other while also critically
reviewing the suggested ideas against the shared commitment. The mixed teams
are benefitting from complimentary expertise of diverse stakeholders.

PHASE 3 (day 3 to day 5): The first part of the final day (day 3 to 5 depending
on the duration of OIC) includes fine-tuning, documenting and presenting the
outcome deliverables of the camp to all camp participants. In this process the
original home group compositions can be changed, to dedicate resources where
they are mostly needed. Based on the presentations, a collaborative reviewing of
the developed solutions is conducted via crowd voting system. The group

233
assessment helps to selected and make decisions which concepts should be
further developed after the OIC. Collective selection process also aims to lowers
the resistance in the OIC follow up stages.

A daily program example based on real-life OIC implementation is presented in

Appendix 1. The example consolidates collaboration processes between the
within home groups and between mixed groups and shows how the iterative co-
creation process is evolving during the OIC.

4 Feedback on real-life OIC experiment

As a part of European Commission’s H2020 circular economy programme funded
project, OIC camp developing Circular Economy (CE) solutions for four different
industries and including 80 participants was implemented in Autumn 2018. After
the camp, online feedback survey was sent to all participants resulting 58.6 %
response rate (47/80).

The feedback received from the OIC participants supports our assumption that
an OIC is an excellent tool for rapid stakeholder engagement when addressing
complex societal challenges such as circular economy. Nearly ninety percent
(89.4%) of all respondent had found new contacts initiated by the OIC. All
respondents gained new insights and knowledge while over half of them gained
to great or to very great extent (51.1%). Over third of the respondents (36.2%)
could apply the new knowledge to great or to very great extent to their work. OIC
participants were willing to recommend the camp for others. At the time of the
data collection, eleven respondents (23.4%) had already done it and twenty
(42.5%) would definitely do that. Fifteen respondents (31.9%) would probably
recommend and only one participant said that he/she probably would not
recommend. Astonishing 95.7 percent of respondents would probably or
definitely attend OIC again.

5 Discussion
Based on the experience from the OIC, we believe that it can solve a number of
constrains related to multi-stakeholder engagement in innovation and
development processes. For the discussion purpose we adopted the constrains
to Living Labs identified in SmarterLabs project (Dijk et al, 2019) and clustered
them in 4 major groups, each followed by a suggestion how OIC can address the
constrain.

Lack of time and financial resources problem: OIC helps to address the issue
related to lack of time and financial resources of stakeholders and users to
participate in open innovation process and Living Lab activities. This lack of
resources often creates an issue of certain stakeholder group not participating in
the Living Lab activities. OIC is a from 3 to 5 days event which gathers in one
place diverse stakeholders, thus allowing participants to use the time efficiently
to address complex problems which otherwise require substantial amount of time.
Considered that the participants are located in the same hotel during the Camp
and OIC includes also social activities during the evening time allowing un-formal
discussion in relaxed atmosphere to discuss further bilateral cooperation

234
opportunities. Since the participation in the OIC is pro bono, but the travel and
accommodation expenses are covered by the OIC organizers, such events offer
a greater return on investment (time spent in the Camp) than traditional brokerage
events or co-creation activities.

Unbalanced stakeholder representation: This constrain cluster includes a

number of issues. First, some of the relevant stakeholders do not share the
urgency to discuss the issues at stake and take actions, and hence they are often
left out from the Living Lab activities. Second, often happens so that some
stakeholders, such as academic partners or Living Lab experts, are often given
more weight than other participants in the co-creation activities. OIC can solve
the issues by identifying relevant target groups based on the participant diversity
management approach and make sure that as proper representation of
stakeholders is achieved. Selection and recruitment of these stakeholders are
crucial to the success of an OIC.

Silo effect in co-creation activities: Stakeholders participating in co-creation

activities, are often highly fragmented, which results in so-called silo effects and
lack of cooperation between different types of stakeholders. Also, indirect
stakeholders are excluded from co-creation activities, for example,
representatives of different industry sectors. The matrix structure of an OIC daily
program emphasises close interactions among diverse stakeholders enabling a
seamless workflow between different subgroups, and making sure that
stakeholders from different ecosystems interact with each other. Furthermore, the
cross-group review and reflections are making different stakeholders’ needs and
requirements visible. Finally, the professional facilitators and purposefully
selected co-creations tools enhance the co-creation productivity.

Co-creation is a process not an event: Often co-creation activities are

organized as separate steps in service design process. Experiences and ideas
from stakeholders who participated in earlier stages of co-creation are often
overlooked. Using OIC in the very beginning of the Living Lab planning process
allows for systematic engagement of diverse stakeholders also later in the Living
Lab activities, as OIC creates a feeling of shared understanding and joint
responsibility towards solving the challenges. Towards the end of Living Labs
OIC can be used as a validation tool for developed solutions. It is important to
keep in mind that an OIC is a part of a process, not a separate tool.

6 Conclusions and Recommendations

To conclude, OIC is proposed as a tool to rapidly establish new collaboration
relationships, discover new insights by sharing knowledge and co-creating novel
solutions by diverse set of actors who can apply outcomes of to their work. Thus,
OIC is recommended as a tool especially when there is a need to establish new
innovation and knowledge sharing networks in a situation when actors do not
know each other before hand. However, selecting suitable participants of OIC
and defining fluent workflow across subgroups during the OIC days is demanding
task, which requires careful planning but also flexibility to change plans based on
the daily deliverables if needed. The following recommendations are offered for

235
replicating and upscaling the OIC concept to other types of complex societal
challenges:

OIC is a good tool for solving complex societal challenges rapidly while providing
economic value, however, the tool alone is not efficient without supporting Living
Lab processes. We recommend to use OIC in the very beginning of a project, to
create a sense of shared responsibility among different stakeholders, and a
common understanding of a challenge and possible solutions, and at the end of
a project, as a validation tool for developed solutions.

When using OIC at the very beginning of a living lab project, it has the greatest
likelihood to provide economic value. The early phase of innovation process
(Cooper, 1988) also known as a fuzzy front end (FFE) of innovation (Smith and
Reinertsen, 1991) is important since quality, costs, and timings of the innovative
solution are mostly defined during this stage (Herstatt and Verworn, 2004).
According to Cambridge dictionary, economic value can be defined as the value
of an asset calculated according to its ability to produce income in the future.
Furthermore, Bowman and Ambrosini (2000) referred to multiple resource-based
theory studies and argued that resources in general are assumed to be valuable
and especially if they 1) enable customer needs to be better satisfied, or 2) satisfy
customer needs at lower costs than competitors or 3) enable a firm to conceive
of or implement strategies that improve its efficiency and effectiveness. As stated
in the discussion section, OIC can address multiple the living lab constrains
including 1) lack of time and financial resources problem by using pro bono as
well as short and intensive time period approaches to engage world class experts
with lower costs than operating e.g. via paid consulting contracts, 2) overcoming
unbalanced stakeholder representation and silo effect by applying participant
diversity approach, thus increasing the likelihood for better satisfying the
customer needs by enriching the co-creation process with multiple viewpoints.

OIC offers an efficient solution for engaging stakeholders who do not know each
other. In this sense OIC concept is addressing initiation of new relationships in
complex ecosystems. However, maintaining this relationship is a different
challenge which can be addressed by systematic engagement of an OIC
participants into Living Lab activities and validation process which takes place
after the OIC. This is especially important since among network theorists of
innovation (e.g. Snehota and Hakansson, 1995), organizations are rarely capable
to innovate independently. Some even argue that networks are the main source
of innovation (Von Hippel, 2007). Networks and knowledge as the key
components of the knowledge and networked society are indisputably the core
components of the any business success. Thus, the findings that most of the OIC
respondent had found new contacts initiated by the OIC as well as participants
were able to apply their new insights to their own work, provides strong tools for
knowledge and network driven innovation processes.

OIC is based on pre-defined structure and carefully planned group interaction

between complimentary actors. The success of an OIC is dependent on creating
the matrix structure which enables systematic co-creation process, when results
of one subgroup interaction are reflected and further developed by the following

236
subgroups. Therefore, designing OIC structure that is easy to implement but also
allows seamless interactions between all challenge groups is crucial.

Acknowledgements
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No [776503] for A
circular economy approach for lifecycles of products and services – project
(CIRC4Life). For more information see www.circ4life.eu. The authors gratefully
acknowledge this support and present also our gratitude and appreciation to
CIRC4Life project partners and innovation camp participants.

References
Bager, T., 2011. The camp model for entrepreneurship teaching. International
Entrepreneurship and Management Journal, 7(2), pp.279-296.
Beaulieu, Luce & van Durne, Gabrielle & Arpin, Ml. (2016). Circular Economy: A
Critical Literature Review of Concepts.
Bowman, C. and Ambrosini, V., 2000. Value creation versus value capture:
towards a coherent definition of value in strategy. British journal of
management, 11(1), pp.1-15
British Design Council, 2015, Design methods for developing services, retrieved
from
https://www.designcouncil.org.uk/sites/default/files/asset/document/Desig
n%20methods%20for%20developing%20services.pdf
Brown, T. (2008). Design thinking. Harvard business review, 86(6), 84.
Carayannis, Elias G., and David FJ Campbell, (2009) 'Mode 3'and'Quadruple
Helix': toward a 21st century fractal innovation ecosystem. International
Journal of Technology Management 46.3, 201-234.
Cassell, C. & Johnson, P., (2006), Action research: Explaining the diversity,
Human Relations; Vol. 59, 6; 783-814.
Chesbrough, H., Bogers, M., (2014) Explicating open innovation: clarifying an
emerg-ing paradigm for understanding innovation. In: Chesbrough, H.,
Vanhaverbeke,W., West, J. (Eds.), New Frontiers in Open Innovation.
(Oxford University Press,Oxford,
Chesbrough, H.W., 2006. Open innovation: The new imperative for creating and
profiting from technology. Harvard Business Press.
Chesbrough, Henry. (2012). Open Innovation: Where We’ve Been and Where
We’re Going. Research-Technology Management. 55.
Cooper, R.G. (1988), Predevelopment activities determine new product success,
in: Industrial Marketing Management,Vol.17, No 2,pp. 237-248
Curley, Martin & Salmelin, Bror (2013) Open Innovation 2.O: A New Paradigm.
Dijk, M., da Schio, N., Diethart, M., Höflehner, T., Wlasak, P., Castri, R., Cellina,
F., Boussauw, K., Cassiers, T., Chemin, L., Cörvers, R., de Kraker, J.,

237
 Kemp, R., van Heur, B. (2019). How to anticipate constraints on upscaling
inclusive Living Lab experiments, SmarterLabs project 2016-2019, JPI
Urban Europe
European Commission, 2018, Open Innovation 2.0, retrieved from
https://ec.europa.eu/digital-single-market/en/open-innovation-20
Freeman, R.E. (1984). Strategic Management: A Stakeholder Approach. Pitman,
Boston.
Ghouila, A., Siwo,G., Entfellner, G., Domelevo, J., Panji, S., Button-Simons, K.,
Davis, S., Fadlelmola, F., Ferdig, M., Mulder, N. (2018), Hackathons as a
means of accelerating scientific discoveries and knowledge transfer, April
2018, Genome Research 28(5).
Gould, R (2012) Open Innovation and Stakeholder Engagement, Journal of
Technology Management and Innovation 7(3):1-11
GSJ, 2018, Global Service Jam: About, retrieved from
http://planet.globalservicejam.org/content/about
Herstatt, C. and Verworn, B., 2004. The ‘fuzzy front end’ of innovation. In Bringing
technology and innovation into the boardroom (pp. 347-372). Palgrave
Macmillan, London.
Halvari, S., Suominen, A.H., Jussila, J., Jonsson, V., Bäckman, J., (2019)
Conceptualization of hackathon for innovation management, in the the
proceedings of The XXX ISPIM INNOVATION CONFERENCE -
Celebrating Innovation - 500 Years Since Da Vinci - 16-19 June 2019 -
Florence, Italy, (EDS.) Bitran, I., Conn; S., Gernreich; C., Heber; M.,
Huizingh; K.R.E., Kokshagina; O., Torkkeli; M., Tynnhammar, M.,
Tutkimusraportit – Research Reports
Jaatinen, M., Lavikka, R., (2008), Common Understanding as a Basis for
Coordination, Corporate Communications: An International Journal, Vol.
13 No2., pp.147-167
Kahn, K.B. (1996). “Interdepartmental Integration: A Definition with Implications
for Product Development Performance, Journal of Product Innovation
Management, 13 (2), 137–51.
Kalmykova, Y., Sadagopan, M., Rosado, L., (2018), Circular economy – From
review of theories and practices to development of implementation tools.
Resources, Conservation and Recycling, Volume 135, 2018, pp. 190-201
Kasanen, E., & Lukka, K. (1993). The constructive approach in management
accounting research. Journal of management accounting research, (5),
pp. 243-264.
Kirchherr, J., Reike, D. and Hekkert, M., 2017. Conceptualizing the circular
economy: An analysis of 114 definitions. Resources, Conservation and
Recycling, 127, pp.221-232.
Knapp, J., Zeratsky, J., & Kowitz, B. (2016). Sprint: How to solve big problems
and test new ideas in just five days. Simon and Schuster.

238
Miaskiewicz, T., & Kozar, K. A. (2011). Personas and user-centered design: How
can personas benefit product design processes?. Design studies, 32(5),
417-430.
Murthy, P. N. (2000). Complex societal problem solving: A possible set of
methodological criteria. Systems Research and Behavioral Science, 17(1),
pp. 73-73.
Navarro, J., Hayward, P., & Voros, J. (2008). How to solve a wicked problem?
furniture foresight case study. Foresight: The Journal of Futures Studies,
Strategic Thinking and Policy, 10(2), pp. 11-29.
Nonaka, I. (2007). The knowledge creating company. Har-vard Business Review,
85(7/8), 162-171.
Oyegoke, A., 2011. The constructive research approach in project management
research. International Journal of Managing Projects in Business, 4(4),
pp.573-595.
Preziosi, R. C. (2006). Icebreakers. American Society for Training and
Development.
Richterich, A. (2017). Hacking events: Project development practices and
technology use at hackathons. Convergence: Convergence: The
International Journal of Research into New Media Technologies1–27.
Rissola, Gabriel & Kune, Hank & Martinez, Paolo, 2017. "Innovation Camps
Methodology Handbook: Realising the potential of the Entrepreneurial
Discovery Process for Territorial Innovation and Development”
Römer, M., Thallmaier, S., Hormeß M, Lawrence, A, Habicht, A, 2011, Jams as
emerging practice of innovation communities: The case of the Global
Service Jam 2011, user.tu-berlin.de
Sanders, Elizabeth B.-N. & Stappers, Pieter Jan (2008) Co-creation and the new
landscapes of design, CoDesign, 4:1, 5-18, DOI:
10.1080/15710880701875068
Santonen, T. (2016). Management of diversity in open innovation processes. In
Open Innovation: A Multifaceted Perspective: Part II (pp. 631-658).
Simon, H.A., (1973), The structure of ill structured problems. Artificial intelligence,
Vol. 4. pp. 181-201.
Simons, T.L. and Peterson, R.S., (2000) Task conflict and relationship conflict in
top management teams: The pivotal role of intragroup trust. Journal of
applied psychology, 85(1), p.102.
Smith, P.G. & Reinertsen, D.G. (1991). Developing Products in Half the Time.
NY: Van Nos-trand Reinhold
Snehota, I., and Hakansson, H. (Eds.). Developing relationships in business
networks. (Londres: Routledge, 1995).
Williams, K. Y., and O'Reilly, C. A. (1998) Demography and diversity in
organizations: A review of 40 years of research. Research in
organizational behavior, 20, 77-140.

239
Von Hippel, E. (2007) The sources of innovation (111-120). Gabler.
Zomerdijk, L.G. and Voss, C.A., 2010. Service design for experience-centric
services. Journal of Service Research, 13(1), pp.67-82.

240
Appendix 1: OPEN INNOVATION CAMP (OIC) program example
for Circular Economy

241
Sustainable
Living Lab
Processes,
Business Models
And Goals
242
243
 Building a platform of social entrepreneurship
and living together
Athanasios Priftis*1, Leonor Afonso2, Theo Bondolfi3 and
Jean-Philippe Trabichet4

* Corresponding author
1 University
of Applied Sciences in Switzerland (HES-SO) / Ecopol Living Lab,
Switzerland
2 Ynternet.org, Switzerland
3
Ecopol Living Lab
4 University of Applied Sciences in Switzerland (HES-SO), Switzerland

Category: Research-in-progress

Abstract
The goal of this paper is to present the initial steps of a web / mobile application
of co-living and social entrepreneurship to be used in European ecovillages. The
application is set to improve existing co-living conditions through more
collaboration between its members, thus contributing to more stability and better
long-term relations. The main hypothesis is that if we manage to decode and re-
introduce co-living activities, already taking place in ecovillages and
ecoquartiers, in a clear, open and collaborative way, then we can stimulate more
entrepreneurship in between communities, as well as other actors. In order to
test and implement the first version of this application and build on an open and
collaborative approach, we participated in a Social Hackathon (2018) presenting
our concept for establishing a prototype. The results of this effort are included in
this paper. Finally, our initial deployment target public will be one network of
ecovillages (Ecopol - Smala) based in Switzerland.

Keywords: Living Lab as a service, Living Labs, Business model, Design-driven

innovation

244
1 State of the art and research questions
Co-living and sustainable communities is an issue of intense research interest.
Ecovillages incorporate a variety of ways of living in community with others,
providing new departures in personal, social, and ecological living (Bang, 2005).
While technologies themselves cannot address the societal challenges (Bierens
de Haan, 2006) in ecovillages, they consist of an inevitable playground,
particularly when they are coupled with collaborative skills and social
entrepreneurship opportunities. Reuse and improvement of existing process in a
community can be a form of innovation and extend to include frameworks,
processes, and policies (Waugaman, 2016). However, adaptation of communities
to new technologies can be a quite difficult task. Especially, when it comes to
using new application and tools that go beyond the existing habits there can be
significant resistance. Nathan (2008) provides as with an interesting perspective
and an example of a digital technology paradigm that has resisted adaptation. He
states a situation of a community where all members have access to and check
email at least once a week in order to receive critical information; business
meeting agendas, proposals, and minutes. According to a key member of the
steering committee, “email is much more efficient”. Yet during each meeting there
are members who have not received the information because: (1) email was not
addressed correctly, (2) attachments were missing, or (3) email was not read by
recipient before meeting.

This perspective demonstrates that specific applications have a clear role to play
when it comes to organising every day or longer-term activities in communities of
people that share similar living habits. Particularly, when it comes to multiple
tasks with different context, participants and level of complexity. This is why this
project adopts the methodological approach of collaborative action research
(Somekh, 2006), which requires a feedback loop that links the processes of
planning, acting, observing and reflecting throughout the project cycle.
Methodologically this leads us to the development of a mixed methods research
plan, whereby different data collection tools and the data resulting from them
(web analytics, rich data resulting from interviews and focus groups with
stakeholders) are used in a complementary manner during different phases of
the project. Our evaluation will thus address the following broad themes: How do
people perform certain tasks in a community? Can internet applications help them
in their organisation and implementation? How do communities understand
collaboration into their ongoing online and offline practices? What about privacy
in and between communities? How should the application help people interact
differently with it? What kind of objectives should it serve? What are the forms of
socio-technical innovation produced during the use?

2 Objectives and methodology

Following the work of Daly (2015) there is a need for visioning process of the
process in an ecovillage providing a means of raising many practices and
elements, particularly meanings, from the practical to the discursive
consciousness of all members of the community. Bringing daily habits, or
elements of practice, into a discursive consciousness is a crucial step in creating
pro-environmental change, and one that should involve a social exploration of

245
new alternatives at a group or community level (Kitchen, 2009). Our application
seeks to move between the social and the technological, proposing four (4) major
objectives: a) Better organise existing activities in communities, b) create a
platform based on (social) co-living tasks and results, c) identify and promote
social entrepreneurship opportunities and d) improve and promote intentional
communication and strengthen the bond between people involved in the
ecovillages.

The main socio-technical characteristics of the application as set in collaboration

with the Ecopol – Smala community can be described as follows (Bondolfi, 2005):
a) free and open source development and licensing, b) open participation with
the invitation of various stakeholders including inhabitants, visitors, researchers
and policy makers to review its deployment, c) respecting user privacy using
cryptography for personal data and d) controlling published metadata with explicit
user authorization and decentralised database architecture that secures its long
term sustainability and community oriented approach. The application itself
should be modular and transformable to a social entrepreneurship market where
participants agree on the data aggregated and published at a central space.
These data can be communicated and connected to other community platforms,
matching supply and demand in regional, national and international level.

More specifically, we contacted five (5) interviews with the coordinating team of
the Ecovillage, including habitants and selected members of the co-working
space. Our questions were structured around two areas leaving together and
opportunities to work together. Following several working sessions during 2018
within the actors of the Smala – Ecopol ecovillage, we came up with specific
proposed, initial, functions that are described below. Their listing serves as a way
to explain to participants the potential use of our application for:
1. Preparation, animation, and follow-up of co-inhabitants/co-operators
reunions
2. Attribution of responsibilities within workgroups, satisfaction feedback
from the beneficiaries of the workgroup’s services
3. Management of the resources acquired through common budget (rooms,
furniture, equipment, shared spaces etc)
4. Online buy and sell possibilities connected to local networks, coordinated
by secretaries/facilitators/delivery people
5. Satisfaction indicators for services provided by the community members
to the members (cleaning, garden, personal tidying, maintenance)
6. Online support and documentation of various checklists, online-based,
request forms.

Our main assumption is that the tasks and needs deriving from the above
functions will be covered from the communities themselves, while creating
opportunities for larger partnerships with other communities, such as SMEs,
entrepreneurs and activists. Our initial deployment target public for the prototype
version of the application is a specific network of ecovillages (Smala - Ecopol).
The plan of action consists of: a) co-designing the application by assigning
concrete activities, roles, logistics, evaluations, services, based on its early
prototype described above, b) validating the functions and evaluate its results in

246
specific pilots, c) measuring the entrepreneurial potential within the selected
communities and d) promoting, at a later stage, the results within european and
international ecovillages.

Testing and deployment of the prototype could lead to the appropriation of the
application as a collective, co-working platform. Following Silvestro’s
conclusions, the ecovillage is intended to create from scratch a micro-society
where each member will be able to discuss and voluntarily integrate the proposed
social contract (Silvestro 2005). Further, the platform should organise groups and
data in a way that will facilitate exchanges and transactions between the
members of every group (community) but also at intra community lever.

3 Co-designing and co-developing the application

As already highlighted, our approach comes with a collaborative impetus focusing
on creating a positive space and experience for this to happen. We look to boost
collaboration within communities and promote entrepreneurship opportunities in
order to make co-living possible. Our collaborative approach is twofold: on the
one hand, co-design the application with members of the Ecopol - Smala Living
Lab through focused interviews described above and, on the other, present these
requirements to a wider - open community both for their review and development
(Social Hackathon).

With the first wave of requirements listed above, we decided to test our
collaborative approach openly from the very start of the implementation of the
application and participate to the third edition of the Social Hackathon Umbria
(SHU, 2018). The Hackathon was focusing on unveiling how digital competence,
sense of initiative and entrepreneurship represent some of the most required
competences by the labour market and, therefore, their development should be
strongly promoted by the European Union for citizens of all ages and origins. The
Ynternet.org - Smala team actually won the b-work challenge of the SHU 2018
and received the prize for the “Best Digital Innovator for Entrepreneurship”.
During the 48 hours Hackathon, teams choose to produce a pitch, a prototype or
a product to be presented in front of an international jury. Ynternet.org
participated on the B- WORK challenge.

The Social Hackathon Umbria collaborative process can be described as follows.

Each team was composed by a member of the organisation who submitted the
idea (the team leader - Ecopol Smala Living Lab in our case), three (3) hackers -
coding specialists, and four (4) support team members.

The roles in the Ynternet.org team were easily defined, which helped the
workflow. The “hackers” worked on different parts of the app: while one
constructed the “brain” of the app, a graphic designer gave it a face, while the
third one made it compatible with portable devices (android and IOS). The
support team members had a crucial role in the brainstorm phases and in the
development of the content for the final product, while the team leader was
responsible for presentations, delivering the pitches, meeting with the audience
and answering to all brand/product representations needs.

247
We started out this process with an open exchange of ideas with the team. The
team leader took the time to go over the first draft of the idea, the specific and
concrete needs the app is trying to answer, and the main characteristics of “living
and working in an ecovillage”. Each team member had the chance to question
and contribute with their own ideas. This was a very critical moment as it was the
first time the product idea was submitted to a brainstorm group exercise. This
moment allowed to redefine the product and at the same time it became more
realistic and doable in the time we had available (48h).

After few hours of brainstorming, the idea was presented to a panel of

experienced members of the jury. A seven-minute pitch was prepared and the
jury gave their first feedback. This initial feedback from the jury allowed the team
to better understand the criteria under judgement and better adapt to the
requests. Frequent team meetings took place in order to distribute workload and
tasks, to find solutions for problems and to promote the flow of ideas and
emotions between the members of the group.

During the two days in which the app was developed, members of the jury,
audience and other important stakeholders including local politicians and
members of other international organisations were involved by giving targeted
feedback. They were also encouraged to visit the work space of the teams and
check the work in progress. This moment resulted to a rich feedback with
concrete questions and specifications. The team extrapolated new needs and
new solutions to those needs enriching the final product.

4 Initial results
In a nutshell, we competed against one other international team to build a mobile
application promoting better communication and entrepreneurship attitude in
ecovillages all over the world. As an overall evaluation of the Hackathon process,
we consider that the collaborative methodology with the clear time frame and
moments of feedback/pitching are an extremely useful environment for
development of creativity and problem solving of very real issues of our 21st
century society. Below, we present a more detailed description of the results of
the common work between our team, developers and activists in place.

248
 Figure 1. My community

In this section, the user becomes member of her ecovillage (“Ecovillagers”) with
access to the private part of the ecovillage discussions and tasks lists and forums.
Members are also able to see public posts on their ecovillage and others
ecovillages. Friends of the ecovillage are all the other stakeholders of the
ecovillage that want to participate in the PUBLIC discussions of the ecovillage
(meaning that “ecovillagers” can also be “friends of the ecovillage” if they are
interested in following other ecovillages besides their own ecovillage).

249
 Figure 2. My community, my village

In this section, we can navigate through the different categories of discussions.

This is an organisation of the discussions more or less by theme (to facilitate the
organisation of the discussions). Examples of other themes are the maintenance
of the common infrastructure, common budgets, rules of cohabitation and
discussion in the topics, parties and events, projects of the ecovillage and others.
Posts come with a set of metadata (description of the post, body of the post) and
number of tasks are identified in the post. Different answers from other members
of the ecovillage are possible and new tasks are added to the list of things to
prepare in the checklist.

250
 Figure 3. The Checklist

In the checklist section in the example, the author of the task agrees to do the
task but needs help from other people. She adds a description of the task (small
text, plus costs, timeline to do the task) and check the people who said they could
help with this task. The colour code selected includes:
• Vivid Green: task was completed with success.
• Bluish Green: Task is open. It means that it still needs people to volunteer.
• Red: the task is now closed and it was not completed successfully.
• Grey: task was cancelled or deleted.

This colour scheme allows an easy and informal diagnosis of the ecovillage’s
activities.

251
 Figure 4. Events and Global

If needed a Post can change its status from PRIVATE to PUBLIC (and vice
versa), in order to reach a larger audience and become an Event to promote an
activity. In this case, the post will be visible in the “Global” section of the
application. In image 4, we can see all the PUBLIC posts of the different
ecovillages we follow as “Friend of the ecovillage”. We can easily see the number
of new notifications (unopened new answers) in each Topic.

5 Conclusions and next steps

Following these initial results, the team is spending time in 2019 to secure the
budget for implementing the code and the process of the proposed application.
Our deployment target public will be the network of ecovillages of Smala - Ecopol
with a plan of action that consists of:
• Continuously co-designing and testing the next version of the application
by assigning concrete activities, roles, logistics, evaluations, services to
the Smala - Ecopol participants.
• Validating the functions and evaluate its results in specific workshops.
• Measuring the entrepreneurial potential within the selected communities.
• Finding collaboration and funding opportunities to improve, develop and
diffuse the application.

252
We remain confident that our application will be in full production and used in the
Ecopol Smala Living Lab during 2019 - 2020 giving us new insights for this work.

References
Bang, J. M. (2005). Ecovillages. A Practical Guide to Sustainable Communities.
Edinburgh, Floris Books.
Bierens de Haan, C. (2006). Entre écovillages et projets d’architectes, les
écoquartiers. Urbanisme, (348), 41-44.
Bondolfi, T. (2016). Ecopol : Labels et services pour les écovillages. 324p. Smala
& Ynternet.org. Collection eCulture.
Daly, M (2015). Practicing Sustainability: Lessons from a Sustainable Cohousing
Community. In State of Australian Cities Conference 2015: Refereed
Proceedings, edited by Paul Burton and Heather Shearer. State of
Australian Cities Research Network.
Dawson J. (2008). How Ecovillages Can Grow Sustainable Local Economies.
Communities n°133, 2006. pp. 56-61.
Imre K. (2009). Ecovillages: In Vitro Sustainability. World Futures, vol. 65 (5-6),
pp. 365-371.
Kitchen, L. and Marsden, T. (2011). Constructing Sustainable Communities: a
theoretical exploration of the bio-economy and eco-economy paradigms.
Local Environment: The International Journal of Justice and Sustainability,
16(8), 753-769.
Nathan L. (2008). Ecovillages, values, and interactive technology: balancing
sustainability with daily life in 21st century america. CHI '08 Extended
Abstracts on Human Factors in Computing Systems, pp. 3723-3728.
Silvestro M. (2005). Les écovillages comme stratégie holiste de développement
durable et d’économie sociale. Centre de recherche sur les innovations
sociales, UQAM.
Social Hackathon Umbria (2018). Grande successo per la terza edizione 2018.
Retrieved from http://www.socialhackathonumbria.info/shu2018-grande-
successo-per-la-terza-edizione/
Somekh B. (2010). The Collaborative Action Research Network: 30 years of
agency in developing educational action research. Educational Action
Research, (18)1, 103-121.
Waugaman A. (2016). From Principle to Practice : Implementing the Principles
for Digital Development. Washington, DC: The Principles for Digital
Development Working Group, 76p.

253
 Business model review for Living Labs:
Exploring business challenges and success
factors of European Living Labs
Justus von Geibler1*, Julius Piwowar1 and Linda Weber1

*Corresponding Author
1 Wuppertal Institute for Climate Environment and Energy,
Division Sustainable Production and Consumption, Germany

Category: Research-in-progress

Abstract
Living Labs offer an open-innovation infrastructure for co-creation and product
testing and gained increasing attention with regard to their potential to support
sustainable innovation. However, many Living Labs face the challenge of
financing their services, especially, when the business models focus on solving
wicked problems such as urban transition and the future of health. Based on a
desktop research and four qualitative interviews with active Living Labs, this
paper explored experiences of Living Labs and their business model challenges
as well as main future success factors.
The findings demonstrate the need and opportunities to transition from public
towards private funding, e.g., to clarify the Living Lab-as-a-service beyond
technology showroom and innovation workshops; to be distinct from traditional
R&D tools and consultancies and to place services within established innovation
funding schemes, e.g., start-up vouchers. Furthermore, the results indicate
challenges according to balancing the innovation process of flexibility and
standardisation. Although iterative and agile processes are core values of the
Living Lab, the innovation process needs some standardisation to allow
efficiency gains and to comply with public regulations, e.g., by structuring the
process and to include descriptions where and how to use Living Lab meth-
odology as well as to clarify linkages to other established design approaches.
The paper concludes that future research needs to better understand the
linkages between entrepreneurship and design research, e.g., towards design-
driven innovation and user experience design.

Keywords: Living Lab as a service, Living Labs, business model, design-driven

innovation

254
1 Introduction
1.1. Living Labs for innovation development
Living Labs have been developed as a new approach for innovation development
and (Greve et al., 2016; Ogonowski et al., 2015, Bódi et al., 2015; Ståhlbröst,
2013) gained increasing attention with regard to their potential to support
sustainable innovation (Keyson et al., 2016; Ley et al., 2015; Liedtke et al., 2015).

According to Brown (2009), innovations must solve the three key dimensions of
desirability (what makes sense to people and for people), feasibility (what is
functionally possible within the foreseeable future), and viability (what is likely to
become part of a sustainable business model). Living Labs are a promising
approach to address these three key innovation dimensions by placing people
and users at the centre of the innovation process; users are actively engaged in
the development process e.g. seniors interact and test innovative health solutions
in their own homes (Bamidis et al., 2017; Konstantinidis et al., 2016). This user-
oriented approach is intended to increase acceptance and market success in the
event of fundamental innovations or high market and technology uncertainty
(Clausen et al., 2011; Geibler, Piwowar, Greven, 2019).

Hence, Living Labs can be described as user-centred, open innovation

ecosystems based on a systematic user co-creation approach, integrating
research and innovation processes in real life communities and settings (EnoLL,
2019). Thereby, Living Labs provide an extensive open-innovation infrastructure
and thus can be an effective approach of wicked problem solving and to address
sustainability challenges (Liedtke et al., 2015; Geibler et al., 2014; Thienen et al.,
2014). Wicked problems (e.g. Rittel and Webber, 1973) are not defined by a clear
process for solutions (e.g. as tame problems in mathematics) but information and
systems are confusing involving many decision makers and clients with
conflicting values (Churchman, 1967).

However, the economic sustainability of Living Labs still presents a critical issue
due to their lack of continuity once the initial public funding has depleted (Grezes
et al., 2013; Burbridge et al., 2017). There are only few success cases of turning
Living Lab research into usable new products and services and uncertainty
remains on what Living Labs actually do and contribute (Katzy, 2012). Mulvenna
et al. (2010) refer to this issue as the “Achilles heel of living labs” and affiliate it
to the absence of explicit business models and profitable partnerships. Yet, if
Living Labs focus on realising commercial benefit of R&D and user-centric
processes, they have a higher potential to become self-sufficient (Mulvenna et
al., 2013). Katzy (2012) states that Living Labs can generate revenue from
investors such as venture capitalists or industrial firms by providing opportunities
to create their sustainable business model. That way, not only flows of information
and benefits, but also flows of money can be identified to capture the value of the
societal engagement with Living Lab stakeholders (Grezes et al., 2013). Hence,
Mulvenna et al (2010) propose the combination of seed capital models, such as
Y-Combinator or HackFwd, with Living Lab concepts, which produces a hybrid,
combined engagement model, comprising the four-stages of ideation, co-
creation, seed capital, and evaluation. Rits et al. (2015) argue that business
model research should be included in the "Living Lab as a service" concept. So

255
far, only few studies have focused on the self-sufficiency of Living Labs.
Nevertheless, there is a substantial need to address this challenge (Grezes et al.,
2013).

Against this background, this paper reviews business models of European Living
Labs based on qualitative interviews. The research question is: What business
challenges do Living Labs face and which success factors are relevant for their
(future) revenue streams?

2 Research Methodology
In order to identify business challenges and experiences of established Living
Labs, the research process involved two main steps combining desktop research
and inter- views.

Step 1: Desktop research: Screening and selecting Living Labs for

interviews
Based on the European Network of Living Labs (ENoLL, 2017) and the Building
Tech- nology Accelerator - Living Lab Network (BTA, 2017) different Living Labs
were iden- tified based on two main criteria:
a) The Living Lab is active in wicked problem solving and sustainability, e.g.,
urban transition or future of health.
b) The Living Lab provides a website with information on a service offer and
general characteristics (see Table 1).

The selected Living Labs were then screened based on general characteristics
(see Table 1) and evaluated based on their service offerings (see Geibler,
Piwowar and Greven, 2018 or table 2). The evaluation criteria were: (++) services
are very visible on the website; (+) visible on the website; (-) not visible on the
website. In total, 18 Living Labs were selected.

Table 1. Summary of characteristics for the screening of Living Labs in Europe

Characteristics Examples
Name / Location Smart Kalasatama Living Lab Helsinki, Finland
Activity yes / no
Institutionalisation project based / institutionalised
Living Lab Driver Industry / Research/ University; Public Institution
Innovation theme / area of expertise Office Lab, Health Lab, Mobility Lab, Retail Lab etc.
Technology infrastructure and user
3d printer, physiological sensors, cameras; wearables etc.
pool

256
 Table 2. Living Lab service characteristics for the screening (based on Geibler, Piwowar
and Greven, 2018)

Service segment Description

Showroom Support to collect user feedback for prototypes in showrooms.

Support to analyse target user groups with quantitative and qualitative
User studies
research methods.
Business model Support to generate and develop business ideas (models), e.g. within a
development workshop setting (business model ideation).
Stakeholder
net- working Support to connect and engage with relevant stakeholders.
and brokerage

Co-design Support to co-design innovative product concepts with relevant stakeholders

and potential users.
Support to co-design prototypes with relevant stakeholders and
Co-prototype
potential users.
UX testing and Support to design, test and evaluate the user experience of
evaluation products and services.
Motivational Support to design innovative product concepts, which consider intended
design user behaviour and the motivation of users (e.g. by gamification principles).
Sustainability
assessment Support to analyse sustainability potentials of innovative product concepts.
and evaluation

Step 2: Exploring different business models of Living Labs based on

interviews
In order to further explore the business models of Living Labs expert interviews
were carried out. The selection of the interview partner was based on the results
of desktop research and availability:
a) Coverage of comprehensive Living Lab services;
b) Coverage of different Living Lab drivers (industry, research/ university, other
public institutions);
c) Representing a geographical spread (North, South, East, West Europe);
d) Availability of a representative for an interview at the Open Living Days 2017 in
Krakow.

A total of four interviews were carried out in Krakow (August 2017) and the
interviewees were experts and practitioners in Living Labs (see Table 3). The
interviews were held and recorded in English with a duration of 45 to 60 minutes.

257
 Table 3. Overview of Living Labs represented by interview partners12

Code Living Lab Affiliation City, Country Homepage

Smart Kalasatama
[1] Helsinki, Finland www.forumvirium.fi/en
Living Lab
Ghent and Antwerp,
Imec Living Lab www.imec-int.com/en/livinglabs
[2] Belgium

Krakow Technology Park www.kpt.krakow.pl/en/technology-

[3] Living Lab Krakow, Poland park/livinglab

Active and Healthy Ageing

Thessaloniki, Greece www.aha-livinglabs.com
[4] Living Lab

The exploration of business models and experiences utilised a semi-structured

interview approach with guiding questions to discuss in an open manner and to
allow for a detailed exploration (Mayring, 2002). The development of the interview
questionnaire (see Annex 2) is based on the approach of the Business Model
Canvas (BMC; Oster- walder and Pigneur, 2010), which has become one de facto
standard for business model development. It defines a business model as the
rationale of how an organisation creates, delivers, captures value, and provides
a practitioner's tool to operationalise this. The interviews were transcribed and
analysed through a qualitative content analysis (Mayer, 2015) with five pre-
defined categories based on the interview guide:
• Background (goals and organisational structures)
• Main customers
• Service offerings
• Revenue streams and costs (public / private funding)
• Lessons learnt (elements of success, challenges)

Structure of the paper

The remaining part of the paper is structured in the following way: The results
section provides the results of the Living lab screening (desktop research) and
the interviews focussing on the Living Lab business model. The discussion
section summarises on success factors for a Living Lab business model. The
report ends with a conclusion on the Living Lab business model.

3 Results
Screening of LL and service evaluation (Step 1)
In total 18 Living Labs were identified and evaluated based on their general
characteristics (see Annex 1) and their services (Table 4).

Table 4. Services of identified European Living Labs (Living Lab selected for interviews are
displayed in bold letters)

12 Names of the interviewees are kept anonymous as their confidentiality has been agreed upon. The
codes are used in the remaining paper to refer to the related interview.

258
 Co-Prototype and
Business model

networking and

Sustainability
development

development
User studies

Motivational
Stakeholder

UX Testing/
technology
Co -Design
Showroom

Evaluation
brokerage

Creativity/
No Living Lab, city,

service
Design
country

De andere Markt,
1. Belgium
- - + + ++ ++ - - -

Estonian Smart City

2. LaB, Estonia - - + + - - + - -

3. DOLL, Denmark + + - - - - + - +

Adaptive Governance
4. Lab, Ireland + + - - ++ + + + +

5. Dublin Living Lab, + ++ + + - - + - -

Ireland

Urban Management
6. Fieldlabs, Amsterdam - ++ - - ++ ++ ++ - ++

imec.livinglabs,
7. Belgium + ++ ++ - ++ ++ ++ - -

8. Botnia Living Lab, - - - - ++ ++ ++ - -

Sweden

9. Basaksehir Living Lab, ++ ++ ++ ++ ++ ++ + - -

Turkey
10. Bristol Living Lab, UK - + ++ - + + ++ + +

11. NEST, Switzerland ++ ++ - - + + ++ + ++

12. Green Living Lab, ++ - ++ + - - - + +

Netherlands
13. Marconia, Netherlands + - - ++ - - + + +
City of the Future
14. Living Lab, Italy - + - ++ ++ ++ ++ + +

15. EVOMOBILE, Italy + + - - + + ++ ++

Smart Kalasatama
16. Living Lab, Finnland ++ ++ ++ ++ ++ ++ ++ - +
Krakow Technology
17. Park, Poland - ++ ++ ++ ++ ++ ++ + -

18. THESS, Greece ++ ++ - ++ ++ ++ ++ ++ -

Note: To assess the services offered, the websites were screened for direct (e.g., “we provide
user tests in simulated environments”/ “a network of…”) and indirect (e.g., “we help you to
innovate” /” bring you together with your clients”) information. As a result, the services are very
visible on the website (++), visible on the website (+), not visible on the website (-)

Experiences on business model (Step 2)

259
This section describes the results of the interview regarding the following topics:
background (organisational setting); revenue streams; main customers (target
group); service offerings and lessons learnt (elements of success, challenges).

Organisational settings
The Living Labs represented by the interviewees are embedded in different
organisational settings. The Living Labs are run as part of an innovation company
[1]; non-profit research institute [2]; a public-private-partnership of a business
innovation centre and a city municipal office [3]; and a unit of a university [4] (see
Table 5).

Table 5. Overview of organisational settings

Living Lab Name Organisational setting

Smart Kalasatama is a district based Living Lab coordinated by the

City of Helsinki’s innovation company Forum Virium Helsinki.
Kalasatama is a model district for smart urban development and a
pioneer regarding the climate goals of the City of Helsinki. In 2014-
Smart Kalasatama Liv- 2018, Kalasatama was developed as part of the Six City Strategy,
ing Lab, which develops open and smart city services in the areas of the six
Helsinki, Finnland largest cities in Finland (Helsinki, Espoo, Vantaa, Tampere, Oulu
and Turku). The project has attracted over a hundred companies to
engage in collaboration with the City and each other while also
raising major national and international interest. The project was
granted follow-up funding for 2018–2020 from the City of Helsinki
Innovation Fund.
The Imec Living Lab is a business unit of the non-profit research
imec.livinglabs Ghent institute Imec. It is a research and innovation hub in nano electronics
and Antwerp, Belgium and digital technologies. The Living Lab unit includes seven people
(user research) and cooperates with experts from the business
model unit to fit user research into the business model logic.

The Kraków Living Lab was developed in 2013 as a joint project of

the business innovation centre “Krakow Technology Park” (KTP)
Krakow Living Lab, and the Municipal Office of Krakow. The development was supported
Krakow, Poland by the strategic partner Forum Virium Helsinki and the advisor Jarmo
Eskelinen. The initial touch point of KTP to Living Lab methodology
were activities in smart city innovation.

Healthy Ageing Living ThessAHALL was founded in 2014 in context of a EU funded project
Lab (ThessAHALL), on smart homes for elderly people and independent living. It is
Thessaloniki, Greece governed by the University of Thessaloniki, Laboratory of Medical
Physics.

260
Revenue streams
Living Lab revenue models are coming from public and business funding (see
Table 6). The interview results showed that public funding is predominant [1; 3;
4], but future strategies also aim for an increase of private funding options.

Table 6. Different revenue models in Living Labs

Revenue model Description

EU (e.g. FP7, H2020, Interreg), national, local institutions (e.g. subsidising
innovation vouchers) provide funding for Living Lab activities based on
Public funding
public objectives, e.g., promoting sustainability, performing research,
(EU, national, local) industrial competitiveness [1; 3].
Start-ups and SMEs contract the Living Lab for innovation services and thus
Business funding receive revenues [1; 2; 3; 4].
A hosting partner can realise revenues by, e.g., setting up a showroom to
Hosting partner demonstrate their (future) products or organising visitor tours [1].
Scientific partners take part in research activities in the Living Lab
Scientific partner infrastructure. These partners can host a full, partial location or be involved
in the research performed at other locations [4].
Self-funded Revenues are based on self-funded projects e.g. LLMCare (2017), FitForAll
projects (2017) [4].
Organisations can become a member e.g. with a memorandum of
understanding. Anybody can become a member, as long as they are
involved in the research project objectives and thus could participate in
Membership events and receive information [3]. Non of the interviewed LL charges a
membership fee.

Private funding is partly generated by positioning Living Lab Services within a

traditional innovation scheme (accelerator, incubator) and as the logic of start-up
vouchers. Thereby the Living Lab services are integrated in the language of
business thinking.

“The approximate proportions of the revenues are coming from public (90 per-
cent) and private sources (10 percent). The aim is to shift the models to 50/50 in
the next 2- 3 years.” [4].

“In future there will be an increase of private money for Living Lab services, but
public money will remain the most important revenue stream” [3].

“The vouchers are funded by regional authorities and directed to start-ups, which
are demanding small scale RD services. Companies only pay between 10 to 40
percent of the actual value of the voucher”. [3].

“The start-ups vouchers cover business and innovation development services

worth max. 10.000 Euro and are released by public and private organisations.
Interested start-ups are able to buy the voucher but get 70% discount of the
vouchers value” [2].

Funding challenges

261
Nevertheless, the financial sustainability of Living Lab services is still very
challenging:
“The share of private funding for Living Lab services will increase but the services
will never be economical feasible”. [3]

None of the interviewees utilise industry sponsors or make use of shares of start-
ups.

“We have a company partner but without membership fee. As long as we have
public funding we can't take any money” [1].

“The Living Lab does not have any ambition to buy shares of start-ups because
the value of the service is below the amount to take shares from and public funding
schemes we use do not allow us to take shares e.g. as other accelerators, which
involve equity seed funding for their programs” [3].

Target group
The Living Labs’ main target groups are start-ups [1; 2; 3], a limited number of
larger organisations [1; 2; 3] and the public sector [2]. Furthermore, customers
are linked to the research community [4] and patient association, which can be
both customers and participants.

Start-ups as customers may differ in their motivation depending on their project

background:

“We are working with start-ups in internal and external projects. In internal projects
the Imec incubation team hires us, the Living Lab staff, and in external projects
start-ups contact us directly. Mostly the start-ups from external projects are more
motivated, active, and co-creative.” [2].

Furthermore, start-ups were characterised as the main target group because of

their lean corporate structure and innovation culture:

“Start-ups have a quick business development cycle because there is mainly just
one contact person. Furthermore, the Imec Living Lab experienced that start-ups
are very open, critical, and demanding. They are asking a lot of questions and
expect quick responses. These attitudes are beneficial for an innovation process”
[2].

In contrast, public organisations such as hospitals were characterised as very

challenging because of the decision-making processes:

“Greek hospitals receive funding by the ministry of health; they are not allowed to
decide on the money and a Living Lab collaboration - it’s a central strategy. It
takes a lot of effort to convince policy makers and government for Living Lab ser-
vices in hospitals. Therefore, strategic decisions are primary directed to the
private sector, who easier understands the benefits of the Living Lab” [4].

Service offerings

262
The interviewees point out the following unique selling points (USP) of a Living
Lab:
• Matchmaking with contacts of the innovation field, e.g. access to an
(international) partner network of research institutions and big
organisations and city representatives, city data [1].
• Understanding the usage context and access to space for real-world
experiments, e.g. in contrast to (virtual) simulation under strict lab
conditions and forecasting methods. This includes user studies, e.g. to
better understand user habits and user practices in daily life as well as user
tests with prototypes in the real-life setting (e.g. in private households or in
a factory) [2; 3].
• Direct contact to potential users and access to group dynamics of co-
creation workshops [2], e.g. in contrast to homogenous focus groups in
traditional market re- search.
• Providing a neutral innovation ecosystem, e.g. in contrast to competing
accelerator programmes of large private organisations [1].
• Accelerating innovation projects due to agile and experimental processes
“to fail fast and to learn fast, e.g. in contrast to traditional structures of larger
organisations [1].
• Providing collaborative consultancy activities and research insights to
create a solution together and not isolated, e.g. in contrast to traditional
consultancies where one asks for the problem and then provides the
solution or advice [2].

The services are operationalised in different ways and service packages, e.g.:
• Living Lab services as a start-up programme, e.g. Agile Piloting13. This
service lasts for 1-6 months [3] or 1-12 months [1] and aims to accelerate
innovation projects. The investment for this service is comparably small
with max. 8.000 Euros [1] or 10.000 Euros to 40.000 Euros. It covers e.g. one
to two iterations and three co- creation sessions [3].
• Living Lab services fit to other innovation services and schemes, e.g.
“innovation challenges” [1] or innovation accelerator programmes [2]. A
start-up challenge in- volves innovation development support (e.g. over 12
months14) and the final prize (e.g. 100.000 Euros). The innovation
development is partly supported by the Living Lab services (max. 10.000
Euros; six months).
• Living Lab services cover a certification system of a testing process, e.g.
to offer infrastructure for clinical trials and thus providing certification for
smart health devices, which cannot be provided by individual
organisations [4].

Challenge: communication and market positioning

13 Forum Virium Helsinki. Agile Piloting Drives Innovation in Smart Kalasatama (2018, November 22)
Retrieved from https://forumvirium.fi/en/agile-piloting-drives-innovation/
14 Nordic Innovation. (2017, November 8). The Nordic Independent Living Challenge.
Retrieved from http://www.realchallenge.info/

263
Interviewees point out that they are challenged to appropriately communicate
their added value and market position, e.g., Living Lab services compete with
traditional research and development schemes as well as tools from universities.
“Discussion on the value of Living Lab services with the regional authority was
not easy because it meant to agree on equivalent value of traditional, university
RD service. The regional authorities believed that Living Lab services are quite
soft compared to the university services” [3].

Furthermore, the understanding and communication of the lab approach can be

quite a challenge:

“It is easier to communicate lab services at a physical “high tech house" in the
university, but it does not cover the unique value of the Living Lab” [4].

Challenge: political regulations

The Living Lab services are dependent on highly flexible processes to, e.g., co-
create, to allow for iterative innovation tests [3] and to be open for the unexpected
[2]. However, political structures and laws can be barriers for such experimental
activities in the real-world.

“Unfortunately, under the inflexible polish public procurement law it is impossible

to iterate and to co-create in deep tech in energy or mobility” [3]. The innovator
needs to specify its intentions at the beginning and in detail and is involved in a
call for tenders”. [3]

Challenge: innovation process and methodology

The Living Lab process is flexible and open but also needs structure and some
standardisation to allow efficiency gains, e.g. to allow for a shared understanding,
direction and orientation in the team.

“I would develop a process list on how the methodology of the Living Lab works
and follow that from the beginning” [4]

4 Discussing success factors for a Living Lab business model

Based on the results presented, we derive potential success factors for viable
business models of Living Labs and discuss practical implications.

1. Communicating a clear added value (USP) to funding partners and target

groups. Living Labs could communicate their services beyond the provision of a
pure technology infrastructure (“high-tech house”) to make their added value
clearer. For example, Living Labs could illustrate the relevance of a thorough
analysis of the real-world context with cases of prototype tests in private
households or in factories. Furthermore, the demonstration of co-creation
sessions engaging non-homogeneous user groups (technology lovers, critical
users etc.) could indicate the benefit of lively and creative debates between the
innovator and users.

2. Better understanding of potential customers (and start-ups). The Living

Lab founders could spend more time to understand the needs of the market. This

264
could involve interviews or meetings with potential customers, e.g. at
conferences. Potential customers are for example start-ups since they have a
quick business development cycle and often lack of own R&D facilities. In this way
Living Lab services could be better adapted to specific start-up needs, e.g.
affordability (max. 10.000 Euros / 6-12 months).

3. Linking to funding schemes of third parties, e.g. through start-up

vouchers. To increase financial stability Living Labs could offer their services
aligned to existing funding models of third parties. Examples might be innovation
or start-up vouchers, offered in accelerator or start-up programmes of public or
private funding organisations. In this way, Living Lab services could be more
recognised and co-financed by third parties. Furthermore, Living Labs could get
better access to public funding sources by clearly defining indicators on how their
activities are supporting public policy objectives, e.g. supporting the development
of sustainability business models.

4. Building a start-up success story. To become recognised, the impact of

Living Lab activities should be visible and provide a substantial contribution to the
success of an organisation, e.g., a Living Lab helps a start-up to gain their first
users and to increase employees from two to forty within two years. Based on a
good start-up success story, the Living Lab can attract other customer segments
such as SMEs, larger organisations and the public sector.

5. Multidisciplinary team: Living Lab teams trust their Living Lab methodology
with a hands-on mentality. The Living Lab experts (user researchers) are part of
a multidisciplinary team including business model experts and technology
experts, so that there are no technical or disciplinary silos but reasoning from and
with a users’ perspective (see Figure 1).

6. Involving existing design and innovation approaches: Although the Living

Lab approach builds on a creative and highly flexible process, there is the need
for standardised processes to allow efficiency gains. Established design-driven
approaches could be linked to the Living Lab approach to enable appropriate
structures, e.g. human centred design (Norman, 2013, see Figure 2), or visions
in product design (Dijk and Hekkert, 2014). Furthermore, the understanding of
connections and relationships of design and innovation tools could further
structure the analytical and creative processes, as for example suggested by
Echternach et al. (2016).

7 Convincing regulation authorities for flexible innovation experiments in

real life: Living Lab services are dependent on highly flexible and iterative
processes, e.g. to co-create and experiment with innovations (“to fail fast and to
learn fast”). Regulations, public procurement law and other political conditions
could better allow for flexible processes and possibilities to rearrange parameters.
These adaptive capacities facilitate creativity and a process of learning by doing;
being open for the unexpected.

265
Figure 1. Living Lab competencies and way of reasoning based on design thinking innovation
model (adopted from Brown, 2009).

Figure 2. Structure of the iterative process as well as user and expert involvement (based on
Norman, 2013 and Meurer et al., 2015) Note: the involvement illustrated with the different
boxes (light/ dark grey) is symbolic and does not represent results.

266
5 Conclusion and future research
Based on a desktop research and four qualitative interviews with active Living
Labs, this paper explored experiences of Living Labs and their business model
challenges. Furthermore, potential success factors are suggested and discussed.
Due to the limited number of interviews the results and suggested success factors
should be interpreted with care. Still, a number of conclusions can be drawn.

The findings demonstrate that the Living Labs represented in the interviews
search for new funding opportunities coming from private sources. To support
this transition, Living Labs should communicate the Living Lab-as-a-service more
clearly. Thereby the Living Lab can be better positioned on the market and
distinguish itself from traditional R&D tools or consultancies. Furthermore, Living
Lab business models could benefit from offering services directed to start-ups
because of their agile innovation culture and their lack of own R&D facilities. In
addition, collaborations with start-ups could demonstrate the impact and
substantial role of the Living Lab service, e.g. to acquire first users. Accordingly,
success stories could make the Living Lab activities visible and meaningful, which
also helps to attract other customer segments such as SMEs, larger
organisations and the public sector.

Finally, Living Labs are challenged with balancing flexibility and standardisation
regarding the innovation process. Standardised processes could promote
efficiency gains and compliance with regulations. The findings suggest following
a structured Living Lab process with descriptions on where and how to use Living
Lab methodology. In addition, this could involve linkages and methodologies
coming from known, approved design approaches, e.g., human-centred design,
design thinking and user experience design.

In conclusion, future research could deepen the results based on a broader

empirical basis, and test the hypotheses based on more interviews and more
Living Labs analysed. Additionally, linkages between entrepreneurship and
design research could be explored e.g. to support design-driven innovations and
strengthen Europe’s competitiveness and promote collaborative learning.

References
Bamidis PD, Konstantinidis EI, Billis AS, & Siountas A. (2017). From e-homes to
living labs: founding and organising the Greek active and healthy ageing living
lab (Thess-AHALL) and its networked services. Hell J Nucl Med Suppl, 20(2):
112-125.
Binder, T., Brandt, E., Halse, J., Foverskov, M., Olander, S., & Yndigegn, S.
(2011). Living the (co- design) Lab. Nordes, (4).
Bódi, Z., Garatea, J., García Robles, A., & Schuurman, D. (Eds.) (2015). Living
Lab Services for Business Support and Internationalisation. ENoLL.
Brown, T. (2009). Change by design: How design thinking Transforms
Organizations and Inspires Inno- vation. HarperBusiness.

267
BTA. (2017). EIT Climate-KIC: Building Technologies Accelerator. Retrieved from
https://bta.climate- kic.org.
Burbridge, M., Morrison, M. G, van Rijn, M., Silvester, S.; Keyson, D., Virdee, L.,
Baedeker, C., Liedtke.
C. (2017). Business Models for Sustainability in Living Labs. In: Keyson, D. V.,
Guerra-Santin, O., & Lockton, D. (Eds.). Living Labs: Design and Assessment
of Sustainable Living. 391-403. Springer, Cham.
Churchman, C. W. (1967). Wicked problems. Management Science, 14 (4), B-
141-B-142.
Clausen, J., Fichter, K., & Winter, W. 2011. Theoretische Grundlagen für die
Erklärung von Diffusions- verläufen von Nachhaltigkeitsinnovationen –
Grundlagenstudie. Verbundvorhaben im Rahmen der BMBF
Bekanntmachung „Innovationspolitische Handlungsfelder für die
nachhaltige Ent- wicklung“ im Rahmen der Innovations- und Technikanalyse.
Borderstep Institut für Innovation und Nachhaltigkeit gemeinnützige GmbH,
Berlin.
Echternacht, L., Geibler, J. v., Stadler, K., Behrend, J., & Meurer, J. (2016).
Methoden im Living Lab: Unterstützung der Nutzerintegration in offenen
Innovationsprozessen (Entwurf Methoden-hand- buch). Arbeitspapier im
Arbeitspaket 2 (AS 2.2) des INNOLAB Projekts. Wuppertal Insti-tut für Klima,
Umwelt, Energie, Wuppertal.
ENoLL. (2019). What are Living Labs. Retrieved from https://enoll.org/about-
us/what-are-living-labs/ EnOLL. (2017). Network Living Labs. Retrieved from
https://enoll.org/network/living-labs/
European Commission (2015). Implementing an Action Plan for Design-Driven
Innovation. Commission Staff Working Document.
FitForAll (2017). Online at http://www.fitforall.gr/play/app (available online at
2017).
Forum Virium Helsinki. Agile Piloting Drives Innovation in Smart Kalasatama
(2018, November 22) Retrieved from https://forumvirium.fi/en/agile-piloting-
drives-innovation/
Geibler, J. v., Erdmann, L., Liedtke, C., Rohn, H., Stabe, M., Berner, S., ... &
Kennedy, K. (2014). Ex- ploring the potential of a German Living Lab research
infrastructure for the development of low resource products and services.
Resources, 3(3), 575-598.
Geibler, J. v., Erdmann, L., Dönitz, E., Stadler, K., & Zern, R. (2018). Roadmap
Living Labs für eine Green Economy 2030. Kurzfassung. Broschüre zum
Arbeitspaket 7 (AP 7.4) im INNOLAB Pro- jekt: „Living Labs in der Green
Economy: Realweltliche Innovationsräume für Nutzerintegration und
Nachhaltigkeit“. Wuppertal Institut für Klima, Umwelt, Energie, Wuppertal.
Geibler, J. v., Piwowar, J., & Greven, A. 2019. The SDG-Check: Guiding Open
Innovation towards Sus- tainable Development Goals. Technology Innovation
Management Review, 9(3): 20–37. http://doi.org/10.22215/timreview/1222

268
Greve, K., Martinez, V., Jonas, J., Neely, A., & Möslein, K. (2016). Facilitating co-
creation in living labs: The JOSEPHS study. 23rd EurOMA Conference.
Trondheim, Norway.
Grezes, V., Fulgencio, H., & Perruchoud, A. (2013). Embedding business model
for sustainable collab- orative innovation in African Living Labs. 2013 IST-
Africa Conference & Exhibition (pp. 1-9). IEEE.
Hassenzahl, M. (2010). Experience Design: Technology for All the Right
Reasons. Morgan and Claypool Publishers.
Hekkert, P., & Dijk, M. V. (2014). ViP (Vision in Design).
IDEO. (2017, November). Our Approach: Design Thinking. Retrieved from
http://www.ideo.com/about Katzy, B. (2012). Designing Viable Business
Models for Living Labs. Technology Innovation Management Review. 19-24.
Keyson, D. V., Guerra-Santin, O., & Lockton, D. (Eds.). (2016). Living Labs:
Design and Assessment of Sustainable Living. Springer.
Konstantinidis, E. I., Billis, A., Bratsas, C., Siountas, A., & Bamidis, P. D. (2016).
Thessaloniki Active and Healthy Ageing Living Lab: the roadmap from a
specific project to a living lab towards open- ness. In Proceedings of the 9th
ACM International Conference on PErvasive Technologies Re- lated to
Assistive Environments (p. 73). ACM.
Liedtke, C., Baedecker, C., Hasselkuß, M., Rohn, H., & Grinewitschus, V. (2015).
User-integrated inno- vation in Sustainable LivingLabs: an experimental
infrastructure for researching and de-veloping sustainable product service
systems. Journal of Cleaner Production, 97: 106 – 116. DOI:
10.1016/j.jclepro.2014.04.070
LLMCare (2017) Online at http://www.llmcare.gr (available online at 2017).
Mayer, I. (2015). Qualitative research with a focus on qualitative data analysis.
International Journal of Sales, Retailing & Marketing, 4(9), 53-67.
Mulvenna, M., Bergvall-Kåreborn, B., Wallace, J., Galbraith, B., & Martin, S.
(2010). Living labs as en- gagement models for innovation. In eChallenges e-
2010 Conference (pp. 1-11). IEEE.
Mayring, Philipp (2002). Einführung in die qualitative Sozialforschung, Eine
Anleitung zu qualitativem- Denken (5th ed.). Weinheim: Beltz.
Norman, D. A. (2013). The Design of Everyday Things. Human Factors and
Ergonomics in Manufactur- ing (Vol. 16). https://doi.org/10.1002/hfm.20127.
Norman, D. A., & Verganti, R. (2014). Incremental and radical innovation: Design
research vs. technol- ogy and meaning change. Design Issues. 30 (1), 78-96.
Nordic Innovation. (2017, November 8). The Nordic Independent Living
Challenge. Retrieved from http://www.realchallenge.info/
Ogonowski, C., Jakobi, T., Stevens, G., & Meurer, J. (2015). Living Lab As A
Service: Das Living Lab als Dienstleistungsbaukasten zur Nutzer-zentrierten
Entwicklung und Evaluation innovativer Smart Home Lösungen. In Mensch &
Computer Workshopband (pp. 701-711).

269
Osterwalder, A., & Pigneur, Y. (2010). Business model generation: a handbook
for visionaries, game changers, and challengers. John Wiley & Sons.
Rits, O., Schuurman, D., & Ballon, P. (2015). Exploring the benefits of integrating
business model re- search within living lab projects. Technology Innovation
Management Review, 5(12), 19-27.
Rittel, Horst W. J., & Webber, Melvin M. (1973). Dilemmas in a General Theory
of Planning. Policy Sciences. 4: 155–169.
Ståhlbröst, A. (2013). A living lab as a service: creating value for micro-enterprises
through collaboration and innovation. Technology Innovation Management
Review, 3(11).
Thienen, J. P. A. von, Meinel, C. & Nicolai, C. (2014). How design thinking tools
help to solve wicked problems. In H. Plattner, C. Meinel and L. Leifer (eds.),
Design thinking research. Building inno- vation eco-systems (97-102). Berlin:
Springer.

270
Annex 1: Selected leading Living Labs in Europe
LL
Driver

No Name Area of expertise Technical Infrastructure

Other Public
Institutions
and user pool

Research
Industry
transition
Urban
De andere Markt,
1 Genk, Belgium Unemployment, future of Mobile printing press ++ X
work
Estonian Smart City Smart city, energy,
2 Lab, Tartu, Estonia health, social welfare 300 test users ++
ICT
DOLL, Glostrup, Den-
3 mark Smart city, lighting None/ city grid ++ X

Adaptive Governance
4 Lab, Limerick, Ireland Public spaces information not available ++ X

Dublin Living Lab, Clean municipal tech

5 Dublin, Ireland (lighting, EV, buildings, information not available ++
waste, water)
Urban Management
6 Fieldlabs, Amster- Resilient communities Field labs in different ++ X
dam, Netherlands city districts

imec. livinglabs, Ghent,

7 Antwerp, Bel- gium Innovative digital 21.000 test users + X
solutions

Botnia Living Lab; Lu-

8 leå, Sweden Future ICT information not available + X

Basaksehir Living Lab, 3D printer, design

9 Istanbul, Turkey Smart living, smart home table, solar, fiber optic ++ X
internet
Bristol Living Lab,
10 Bristol, UK Digital needs for local 5500 households + X
citizens
NEST, Dübendorf, Modular building for
11 Switzerland Smart living & office testing innovations built ++ X
on-site
Green Living Lab,
12 Amsterdam, Nether- Healthy urban living Biomeiler, ecological toi- ++ X
lands lets, outdoor research
lab
Marconia, Rotterdam, Architecture,
13 Netherlands sustainable water, information not available ++ X
energy, waste systems
City of the Future Liv-
14 ing Lab, Milan, Italy eHealth, smart living information not available + X

EVOMOBILE, Pa- Several charging

15 Electric mobility stations, electric vehicles ++ X
terna, Italy
Smart Kalasatama Small Lab, Field lab in
16 Living Lab, Helsinki, Smart city innovation city district; engaged ++ x
Finland 800 users

KTP Living Lab, Kra- Multimedia and field lab,

17 kow, Poland Smart city, creative in city districts; in + x
industry in- novation industries

AHA Living Lab

18 Thessaloniki, Health innovation, Health lab and field - x
Greece Smart home for the labs in private
elderly households

271
Annex 2: Interview questionnaire

Thank you for taking the time to for this interview. The aim of this interview is to
explore the different business models of living labs in Europe. The interview is
conducted within a publicly funded project of the "Knowledge and Innovation
Community on Climate" (Climate-KIC).

I. PORTRAIT: LIVING LAB GOAL AND ORGANISATIONAL SETTING

Please describe in a few sentences your Living Lab and your role.
a) What are the specific goals and objectives of the Living Lab? Is sustainability
an objective? What type of innovations do you mainly support (incremental,
radical disruptive)?
b) What is the legal entity and organizational structure? What are related benefits
and challenges? Who was the founder of the Living Lab?
c) What is the founding year, size (m²) and number of employees?

II. LIVING LAB VALUE: SERVICES AND REVENUES:

Please describe your services and revenues.
a) What are your main service and technological equipment offerings to your
clients? Are there plans for future services?
b) Who can make use of the Living Lab services? (Which pricing models do you
have?)
c) Is membership needed? Does the fee vary between members?
d) What are the main sources of revenue / funding for the Living Lab?
e) What approximate proportions of the budget arrive from public (national, sub-
national, international (e.g. EU) private and other sources? What are
improvements to for this ratio?

III. COSTS: MAIN BARRIERS FOR SUCCESS

Please describe your main barriers of the LL business model.
a) What are your main three advices for LL founders?
b) If you would open a LL again what three things you would do differently today?
c) What are your main costs/ investments? What are improvements would you
like to improve costs? Innovation pipeline not filled?

272
 Facilitate innovation and collective intelligence
through play
Yves Zieba1 and Isis Gouédard1

1 IsYnnov, Switzerland

Category: Innovation Papers

Abstract
While companies, universities, citizen and governments become aware about
the Sustainable Development Goals, they are confronted with challenges as well.
How to set the goals, how to agree on priorities, how to convince everyone, how
to mobilise employees or advocates? That is where our specifically designed
methods and gamified tools help stakeholders turn SDGs intentions into action
plans. This article relates our experience and gives a taste of our magic recipe
and ingredients. Simple game rules and an inclusive climate of trust; openly
oriented towards co-construction. These basic principles are the fundamentals
of our innovative, inclusive and participatory approach based on the
establishment of a permanent dialogue between populations and technical
agents, on mutual respect and the principle of partnership, as well as on the
recognition of local know-how. We use the concept of boundary objects as a
foundation for the pursuit of a common goal and help to minimize or avoid
conflicts. Our playful approach combined with our games made of natural
materials allows us to highlight the multi-sensory dimension of the experience
we offer. By stimulating all the senses, everything makes sense!

Keywords: Boundary objects, Collective innovation, Collective intelligence,

Digital fabrication revolution, Sustainable Development Goals, Inclusive and
participatory approach, Living Lab tools, Natural Resources, Economic and
social value creation, Mission Innovation, Gamified tools and technics, Inclusive
mindset, Co-creation and co-construction, Quadruple helix, Multi-sensory
experience

273
1 Introduction
Our experience and knowledge are shared from Geneva, Switzerland. Our dream:
to nurture collective intelligence in the service of the Sustainable Development
Goals on a daily basis!

1.1 The SDGs (Sustainable Development Goals) have been gamified!

We are entrepreneurs passionate about human interaction, agility (defined as
groups of practices for managing and implementing projects with high stakeholder
involvement and responsiveness), collective intelligence and innovation. We have
partnered to embark on the adventure of facilitating co-creation, change and
collective innovation in Geneva, Switzerland.

Collective intelligence, according to Malone & al (2010), is built slowly and is

experiential. As humble practitioners, we strive to create the conditions for others
to let themselves be transformed. Applying Living Lab approaches and tools
(Dubé & al, 2014) combined with play and boundary objects, we help them to
accept to open themselves to questions, their own and others'. New individual and
collective benchmarks are emerging, creating a fertile ground for innovation. We
apply our collective intelligence methods to innovative projects through an
approach that allows us to bring out new, sustainable solutions that are better
adapted to the complexity and constant changes of our world.

We are co-constructing serious games aimed at facilitating communication,

creativity and expression to help unleash the power of collective intelligence to
co-create the future. They allow the serious dimension to become more attractive
by providing interactivity, a set of rules and some playful objectives. That’s the
reason why we oriented our action, our resources and our methods on games, on
the pleasure of playful interaction that allows for the construction of shared
visions, group animation, team facilitation, governance and deriving action plans.
For further reading, Rieber & al (1998) gave an interesting insight on the value of
serious play.

Both deeply convinced by the necessity to act on the UN's Sustainable

Development Goals (SDGs:
https://www.un.org/sustainabledevelopment/fr/objectifs-de-developpement-
durable/), we apply our methods of facilitating collective innovation through games
to opening up reflections on these themes. Our objectives range from raising
awareness only to supporting the institution to identify which SDGs are most
relevant in its context and deriving a roadmap that can be activated and supported
by its management to take concrete actions.

The first element on which our games are based on the SDGs is a 20-sided dice,
an icosahedron, which we co-created (Payne & al, 2008) in partnership with SDG
Solution Space and published under a Creative Commons license. 17 of the faces
correspond to the SDGs and 3 faces are customizable according to the needs of
the game / institution. In particular, we use it to launch the debate and raise
awareness of the SDGs among stakeholders and it allows us to gently approach
the serious playful world we want to bring to life.

274
 Figure 1.

Then, we deploy our range of games and methodological know-how according to

the context and can also co-construct the experience with our clients when it
makes sense.

Our success factors? Simple game rules and an inclusive climate of trust;
openly oriented towards co-construction.

In this context, using transposition exercises or games calling on the participants'

personal experiences, we encourage the use of "I" aiming to focus on the identity
and values of the stakeholders.

These basic principles are the fundamentals of our innovative, inclusive and
participatory approach defined as a dynamic process in the sense that it is
evolving in time given the local specificities and conditions. It is based on people's
knowledge and perception of their environment and the interaction of the various
elements involved in the management of its specificities. The participatory
approach is based on the establishment of a permanent dialogue between
populations and technical agents, on mutual respect and the principle of
partnership, as well as on the recognition of local know-how.

We also use the concept of boundary objects which are defined as a tool for
mutual understanding and cooperation that allows different heterogeneous social
worlds to communicate/collaborate through reconciliation around the same
concepts in different worlds. We use them as a foundation for the pursuit of a
common goal and help to minimize or avoid conflicts. This term would have first
appeared in a study by Suzan Leigh Star and James Griesemer (1989).

Our usage of boundary objects in innovation games fosters connexions between

participants and facilitate the emergence of collective intelligence. They are
mostly made of natural materials, locally produced or even created by the
participants themselves!

We are sometimes asked why we have chosen to produce games with objects
locally. It is true that in the digital age, it would have been convenient to create
online games or smartphone applications. We prefer the use of objects because

275
they allow several people to play together to achieve the same objective: whether
it is a pawn, a voting token or an origami made by the participants themselves,
the time spent appropriating, or even designing, the object makes it possible to
initiate dialogue and to propose a simultaneous approach at the physical and
intellectual levels.

To facilitate debates and unleash collective intelligence, to get everyone to

express themselves and capture as many good ideas as possible from a group,
there are various methods. But through boundary objects, stakeholders have the
possibility to pass the object and therefore symbolically the focus / right to speak,
which invites each and every one to participate and express themselves in a soft
and efficient way. Just like the “talking stick” in holocracy (governance
organization system based on the formalized implementation of collective
intelligence), this allows us, as facilitators, to involve everyone, even the most shy
or resistant who would tend to hide at the back of the room!

What else does our approach bring? By stimulating all the senses, everything
makes sense!

Our playful approach combined with our games made of natural materials allows
us to highlight the multi-sensory dimension of the experience we offer. Indeed,
touch is one of the senses that we generally stimulate little, and the physical body
allows us to deeply anchor experiences. A form of communication that requires
both hemispheres is richer in codes and interpretations. Experiencing concepts
through the body in addition to verbal activities refers to all the structures of the
brain and makes communication more obvious.

Figure 2.

In the field of the transmission/assimilation of knowledge, we draw inspiration from

the work of Maria Montessori (https://www.montessori-france.asso.fr/page/
167406-maria-montessori-et-sa- vision-de-l-enfant) which, like so many other
works, tends to prove that teachings are more easily and deeply assimilated when

276
the body is associated with reason. If we add the noise of the machines, the
odours of smoke or chemical treatment (non-toxic!), several senses are also
stimulated when passing through a Fablab and the experience remains in our
memory longer.

1.2 Two levels of interaction for the most participatory governance possible
Because we want to capture information from people from a wide variety of
backgrounds, with different skills, going through the object allows us to create a
link.
With our games, we act on several levels. We first call on volunteers to give them
the freedom to express themselves. This is our first level. Generally, many
participants take this opportunity to express an idea, share a positive or negative
experience. At the second level, we invite participants to vote or rank the ideas,
opinions or opinions expressed; this helps to keep everyone's attention and detect
common features between the votes, in order to provide a first interpretation of
the results. We also offer fewer objects than there are participants to observe the
social interactions that are taking place.

From the feedback we have gathered so far, what comes back frequently is the
satisfaction of having listened to their peers and of having been able to express
themselves / to be listened to / to interact in community. The empathy thus
facilitated makes it possible to forge a bond and initiate a dialogue of the type "and
you, how did you cope with this situation? ".

This is what we experienced during the Open Living Lab Days in August 2018 in
Geneva. We proposed two games around the Sustainable Development Goals,
in particular SDG #5: "gender equality" and SDG #11: "sustainable cities and
communities".

With the first, in the form of a traditional adapted "Snakes and Ladders", we
opened the debate on opportunities and constraints related to gender equality.
Despite the short time available and the diversity of the profiles present, the
examples were profound and personal and we discovered, for example, that
women and men diverged in their prioritization of the elements identified when we
gave them a limited number of tokens to use to support an idea.

Thanks to our approach, connections were forged, ideas unleashed and concrete
conclusions drawn. The second, oriented as a change management exercise,
aimed to open up reflection and sharing of experience on the mobility issues of
the future. Participants from all over the world were able to share in a fun way
while becoming aware of the difficulty of taking the necessary actions to achieve
a goal when one has invested body and soul in a personal construction project,
however simple it may be.

277
 Figure 3.

The approach raised deep reflections associated with loud laughter!

1.1 Our contribution to the SDGs on a daily basis

Finally, to have a real and positive impact in ecological, economic and societal
terms (Allingham & al, 1975), we have chosen to make our games using local
materials and natural resources. In this way, we also hope to inspire the
communities with whom we interact in the context of collective intelligence.

That's why we naturally turned to the nearest Fablabs : a contraction of the words
Fabrication (Manufacturing) and Laboratory (Experiments), the Fablab
(Gershenfeld, 2012) is an affordable place, open to the public, providing its users
with the technical, technological and human resources (machines, tools, software,
processes, know-how, mentors) necessary for the design, optimization and repair
of all kinds of objects.

A Fablab is also and above all a place of sharing, where members are in turn
beneficiaries and contributors, where experiences are shared in order to optimize
the global potential for innovation.

2 Conclusion
It is with great joy that we have, in these few lines, shared some elements of our
dream: to nurture collective intelligence in the service of the Sustainable
Development Goals on a daily basis.

Through our Living Lab approaches combined with specifically designed methods
and gamified tools, we help stakeholders turn SDGs intentions into action plans.
Simple game rules and an inclusive climate of trust; openly oriented towards co-
construction; are the fundamentals of our innovative, inclusive and participatory

278
approach. Moreover our games made of natural materials allows us to highlight
the multi-sensory dimension of the experience we offer.

We are starting locally with the ambition to scale globally!

References
Dubé, P., Sarrailh, J., Billebaud, C., Grillet, C., Zingraff V. Kostecki, I. (2014). Le
livre blanc des Living Labs. Edition SAT Montreal.
Suzan Leigh Star and James Griesemer (1989). Social Studies of Science, Vol.
19, No. 3 (Aug., 1989), pp. 387-420. Sage Publications.
Neil Gershenfeld, (2012), How to Make Almost Anything, The Digital Fabrication
Revolution, Vol. 91, No.6 (Nov., 2012), Foreign Affairs.
Thomas W. Malone, Robert Laubacher and Chrysanthos Dellarocas (2010), The
collective intelligence genome, Vol.51, No 3, MIT Sloan Management
Review.
Rieber, L. P., Smith, L., & Noah, D. (1998). The value of serious play. Educational
Technology, 38(6), 29-37.
Allingham, M.G. Zeitschr. f. Nationalökonomie (1975) 35: 293. Economic power
and values of games https://doi.org/10.1007/BF01284617.
Payne, A.F., Storbacka, K. & Frow, P. J. of the Acad. Mark. Sci. (2008) 36: 83.
Managing the co- creation of value. https://doi.org/10.1007/s11747-007-
0070-0.

279
 Launch Process of a Living Lab and Required
Leadership for Practitioners
Masataka Mori1 and Kyosuke Sakakura2

1
Miratuku, Japan
2 Tokyo City University, Japan

Category: Full Research

Abstract
This research aims to clarify the process to set up living labs and required
leadership for its practitioners regardless of social condition each country or
community has. Using Forum Virium Helsinki in Finland, High Tech Campus
Eindhoven in the Netherlands and Living Labs Taiwan as a case study, data
were collected through web surveys and interviews and analysed with Grounded
Theory Approach. As a result, research shows three phases of launch process
(launch, foundation and involvement) and eight components （ theme setting,
ecosystem formation, co-creation approach, funding and framework, places and
opportunities, media and transmission, citizen-based projects and leadership are
essential to set up a living lab, and five principles with sixteen actions（co-
creation, empowerment, exploration, open and fair, and reflection）are required
for leaders, which we call “orchestratorship”.

Keywords: process of launch, launching, components, leadership, orchestration

280
1 Introduction
Historically, Japan has achieved economic growth and social formation through
industrial development. As Japan does not have abundant natural resources, it
has achieved economic development by importing the raw material, processing
the products and exporting them to abroad in the textile products, steel and
automotive industries. Traditionally, these manufacturing companies have been
closed to avoid the leaks of ideas and technology.

Along with the transition from the industrial society facing the limit of closed
development, there has appeared the development and creation beyond the
organisational frameworks. This phenomenon is the open innovation advocated
by Henry Chesbrough (Chesbrough, 2003, xxiv). Companies focus on making
the relationship with other stakeholders such as creating innovation hubs,
hosting acceleration events or using co-working spaces as the satellite office.
This dynamic movement is also being focused by the Ministry of Economy, Trade
and Industry as a driver for national growth (Open Innovation White Paper,
2018).

With rising expectations for open innovation, living labs have also attracted much
attention from many industries. Since the appearance of first living lab in Japan
a few years ago, there are approximately 44 living labs at present. In 2017,
Future Centre Alliance Japan, as a flagship organisation to accelerate cross-
border knowledge creation, and European Network of Living Lab formed a
partnership, which helped to promote living lab activities in Japan (Future Centre
Alliance Japan, 2019).

However, there are still many challenges in launching a living lab in Japan such
as the characteristics of Japan as an industrial country, the national character of
the people and tolerance to open innovation. One major characteristic is that
Europe and Japan have different social condition, historical background and
cultural contexts. Although many of the researches has been done for living labs,
there are many studies dealing with cases and social situations in Europe as
research and practice are leading in Europe. Therefore, it is necessary to do the
research that can be diverted into other regions without being too dependent on
the European context.

This research aims to identify the phases of process and components to launch
a living lab and the leadership required for a leader to orchestrate the ecosystem.
Focusing on Finland, the Netherlands and Taiwan as a case study, we have
collected data by web-based survey and interview and analysed data by
Grounded Theory Approach.

One outcome is three phases of launch process and eight components to launch
living labs. The other outcome is the five principles and sixteen actions required
for a leader who orchestrates the ecosystem. Still, since this research has only
started two years ago, we introduce the future avenue of this research to
conclude this paper.

281
This research has just started. The purpose of this research is to conduct case
studies mainly in Europe, and to create a common framework that does not
depend too much on culture and social background. We believe that adding
culture and social background is essential for launching a living lab in various
places. Therefore, the next step is to apply a framework in the context of Japan.

2 Previous research
Since ENoLL was founded in November 2006, many studies have been
conducted to support the better and deeper understanding of the concept.
ENoLL Projects Portfolio published by ENoLL in 2018 introduces the six common
features of a living lab: active user involvement, orchestration, real-life setting,
multi-stakeholder participation, multi-method approach and co-creation
(PROJECT PORTFOLIO 2018 - 2019, 2019). Another research based on the
article searching of 851 published documents illustrates the eight facets of living
labs such as real-life environment, challenges or sustainability (Hossain,
Leminen & Westerlund, 2018).

The question that still remains is why it is not easy to launch living labs in Japan.
We have done the previous research and reached to the hypothesis that many
researches done so far mainly deal with the cases based on social situation,
historical background, cultural context, and natural environment in Europe. It
could be said that the development of living labs has progressed with such
conditions of Europe as the standard since living labs were first born in Sweden.

Furthermore, there is no research focusing on launch and development

processes. This is probably due to the fact that information sharing and
discussions on the launch and development of living labs are taking place at
events held at Open Living Lab Days and elsewhere, and there is no need to
conduct research in Europe.

The development of living labs has reached a turning point that it has been
spreading from Europe to other regions and scaling up from local to global. This
innovative method of working with various stakeholders to develop products or
services is expected to be more increasingly needed in many areas in the future.
Therefore, it is a preferable situation that living labs can be launched without
being too dependent on the social situation and culture background each
community has. There may be may social characteristics or cultural features
understood as common in Europe but not in different situation, which often
happens when it comes to the launch of living lab.

3 Methodology
3.1 Research question
In order to support the launch of living lab, two research questions have been
set up. The first focused on the necessary components to launch a living lab in
the chronological order. It would be possible to launch and grow a living lab more
smoothly if the practitioners know the necessary components and efforts in line
with the time axis. The second is the required leadership for practitioners.

282
Harmony with ecosystem stakeholders is one of the key features of living labs
so that focus is put on the leadership required for orchestration.

3.2 Case selection

It is essential to select countries with different backgrounds in order to
understand the components needed for a launch that are not too dependent on
social context or cultural context. Therefore, are chosen Finland from Nordic
country as the origin of a living lab, the Netherlands from Europe as the next
wave and Taiwan from Asia as the developing region for a living lab. Considering
the exposure and external evaluation of ENoLL and other paper, each living lab
is selected: Forum Virium Helsinki from Finland, High Tech Campus Eindhoven
from the Netherlands and Living Labs Taiwan from Taiwan.

Forum Virium Helsinki is the City of Helsinki innovation company with the mission
of making Helsinki the most functional smart city in the world ("What does Forum
Virium Helsinki mean to the city? - Forum Virium Helsinki", 2019). One of the
remarkable projects is called Smart Kalasatama started in 2013 to re-create
Kalasatama as the model district of smart city development. Citizens, companies
and public sectors co-creates the agile projects to realise a smart city in
Kalasatama utilising the living lab as methodology.

High Tech Campus Eindhoven is the campus located in Eindhoven, the

Netherlands where more than 185 companies and institutes, and 12.000
researchers, developers and entrepreneurs work on developing future
technologies and products. Started as the core laboratory of Phillips for its
national R&D activities, it was sold to Ramphastos Investments from Philips so
that the Campus has been utilised to accelerate the open innovation beyond the
borders of companies, research institutes, citizens or academia ("High Tech
Campus Eindhoven: Campus History", 2019).

Living Labs Taiwan is one of the projects carried out by the Institute for
Information Industry (III) in Taiwan which was established in 1979 as a Non-
Governmental Organisation (NGO) under the partnership of public and private
sectors. The project is called “Integration of Wearable Devices and Personal
Health Records”. It aims to enable senior citizens to manage their health
condition more easily by themselves with ComCare platform, a service that
combines wearable devices and IoT. This project was awarded by ENoLL for the
second year in a row in 2006 and 2007 ("Taiwan III Living Lab", 2019).

3.3 Data collection

Data collection was mainly conducted by web survey and interviews. First, we
collected the basic information, activities and events of each living lab from
websites, journals, papers or web articles. Gathered information is tagged
considering the contents such as citizen participation or media exposure, and is
put in spreadsheet in chronological order. Second, we have conducted a semi-
structured interview based on gathered information by web survey. In the
interview, we asked about the parts that could not be obtained in the web survey
and the parts that should be understood in detail. Questions for components and
process for launching a living lab focus on actions and events that are impactful

283
and inevitable for each living lab such. For the questions of leadership, the focus
was put the significant values or actions leaders should take such.

Selection of interviewee was limited to the person who started up, or who has
been involved since the start up. For the components and process, it is essential
that interviewee can look back on history and process to share the most effective
initiatives and actions that provided values. For the leadership, it is important to
gain the values, experiences, and tacit knowledge that only practitioner who have
launched or operated a living lab must have.

We used GTA (Grounded Theory Approach) for the analysis of this research.
GTA is one of the analytical methods such as qualitative survey data in social
surveys and was invented by Glaser and Strauss in 1967. Data collected from
web surveys and transcripts of each interview were divided into short sentences
as elements, considering the contexts, and were labeled and grouped to
represent the phenomena (Glaser & Strauss, 1967).

4 Process and Components

The three key components of the Smart Kalasatama Project, run by Forum
Virium Helsinki, are introduction of scheme, vision making with citizens and place
making for stakeholders. In particular, innovators club and developer’s club as
place making has very much contributed to the progress of this project and Smart
Kalasatama. The Smart Kalasatama Innovators Club is held four times a year,
where participants regularly share news and create their own project, which help
build a community for passionate stakeholders. Kalasatama Developers Club
aims to support further connection between different actors in the ecosystem.
Small amount of budget from 1000 to 8000 euros can be provided for
entrepreneurs to help their product development and agile testing. This
community accelerates the passionate entrepreneurs, developers, designers or
artists who can contribute to the area by creating product, which is interlinked
with Smart Kalasatama Innovators Club ("Smart Kalasatama initiative enters a
new phase - Forum Virium Helsinki", 2015).

Having three components of place making, citizen participation and creating an

ecosystem, High Tech Campus Eindhoven emphasised the significance of the
last one the most. One remarkable contribution is the establishment of Holst
Centre as an independent R&D centre in 2005 under the cooperation of imec,
TNO, and local, regional and national governments (TNO, 2019). Since its
establishment, Holst Centre has become the physical space for researchers and
academia to conduct research and development, which enables them to be a
part of an ecosystem. Moreover, this research facility is so supported by local,
regional and national governments that it strengthened the relationship between
the Campus and the public sector. Another one is an award winning of Intelligent
Community of the Year 2011 for its remarkable development of the Campus as
an open innovation platform despite the financial crisis ("Eindhoven", 2019). This
global award has spread the name of the Campus not only to the Europe but
also to other parts of the world, which has attracted more entrepreneurs,
researchers, startups or medias from all over the world. What is more, this award

284
has created the bridge between the Chinese investment and the Dutch high-tech
ecosystem through holding regular events in China ("HighTech Connect China",
2018).

Living Labs Taiwan has three components for place making, co-creation and
citizen-based project. Although it has the strong relationship with national
government, it cherishes the community of citizens in Taipei, Taiwan, which has
rooted its historical background. It used to be colonised by Japan for 50 years
from 1895 to 1945 and then became a part of China. During this period, social
system such as education and public transportation has organised rapidly, and
the capital investment from Japan also made the economy grow significantly.
However, in 1945 when World War II ended, Taiwan became a part of China,
from which social friction has appeared such as the issue of independence of the
nation and freedom of speech (Jara Pallana, 2016). Therefore, citizens have
strong intention and motivation to create their own area by themselves so that
citizen-based projects and co- creation involving citizens works very well for
Living Labs Taiwan.

Figure 1.
5 Three phases of process: launch, foundation, involvement
In the launch phase, it is essential to set the direction of living labs. There are
many ways to begin with such as the introduction of urban planning schemes
and the hearing of citizens, but it is important to prepare the condition in that
different stakeholders are able to run in the same direction. For example, in the
case of High-Tech Campus Eindhoven, the fact that the Campus has been open
to other research institutes and startups is very important. It issued a message

285
that they have a will to achieve open innovation across the region beyond the
boundaries of the organisation.

The essential part of this foundation phase is to create a foundation that will play
a central role in managing a living lab. Such central core of a living lab can be
generated by creating an ecosystem in which stakeholders from various fields
and sectors participate or a physical space where citizens can gather freely.
Smart Kalasatama Innovators Club and Kalasatama Developers Club is the good
example to show its impact, as mentioned above. These clubs have created the
opportunities for stakeholders in an ecosystem to gather regularly for exchanging
information or creation of project. It is inevitable to empower such passionate
stakeholders including citizens to do what they can do as a part of the project.

In the involvement phase, it aims to expand and accelerate the activities of living
labs. Exposure to the media, agile project with citizens and creation of public
facilities will create a greater impact by involving more people who have never
had a chance or are uninterested to be part. Living Labs Taiwan, they distributed
300 tablets to elderly people for use as a test bed for a project to support elderly
people's healthcare. Initially, it took half a year to distribute only 100, but then
information was spread by user's word of mouth, and it was possible to distribute
300 at a stretch. This activity showed the existence of Living Labs Taiwan to the
public and citizens started to have an understanding toward their projects.

Three phases of launch, foundation and involvement circulate. To enter into

involvement phase does not mean the end of going in but it come back to the
launch phase to start new projects and small initiatives from there. By circulating
and spiraling up, it can be expected to move forward more smoothly than in the
first flow.

5.1 Theme setting

Theme setting means the same to indicate the vision: what kind social issues the
living lab aims to solve or what kind of future the living lab aims to create. Of
course, stakeholders satisfy their own interests and earn benefits, but if they do
not agree with the vision, it is expectable they will soon withdraw from
participation. By creating a clear vision, relevant stakeholders can gain access
to the core and an ecosystem of high purity is formed, which can increase
sustainability. In the Smart Kalasatama Project, the vision is defined to be free
up one hour of extra time for its inhabitants every day ("Smart Kalasatama
attracts innovation tourists to Helsinki - Forum Virium Helsinki", 2017). It is a
good example of presenting an attractive and easy-to-understand vision for each
stakeholder, particularly citizens. For discovering the clear vision, it is also
effective to find the similar approach or the living lab as a benchmark, and to
name the project in order to enable stakeholders to have attachment.

5.2 Ecosystem formulation

Ecosystem is the basic component of a living lab. Particularly as proposed by
Quadruplex Helix Model, the four actors are indispensable as the basis of the
ecosystem; academia, civil society, company and the government (Carayannis
& Campbell, 2009). The idea of an ecosystem may be common and may not be

286
difficult to create in Europe, particularly in Nordic countries where living lab was
first adapted However, in a society such as Japan, where there is still little
interaction between industries, and a vertically closed organisational system is
still majority, the concept of ecosystem becomes the key for building up the living
lab. Diversity of citizens and stakeholders is the source of innovation in living
labs.

5.3 Co-creation approach

Co-creation is the very heart of living labs as well as proposed by ENoLL.
Stakeholders’ participation is essential for the realisation of the vision through
living labs. Based on the ideas and needs of citizens as end users who well
understand the area or community, living labs shows its value by co-creating with
universities, companies, and governments that can facilitate the realisation of
research and innovation at the same time. There are also many studies in the
field of co-creation such as The Handbook for Co-creation published by city of
Espoo illustrating well-organised method of co- creation ("Handbook for Co-
creation", 2019).

5.4 Funding and Framework

One of the major differences between Europe and the rest of the world is scheme
and framework. As long as the concept of open innovation and a living lab is still
minority, there are very limited grants or open call to projects to support a living
lab from the government or foundation. The concept has not yet been so
generalised that the role of external evaluation or mass media that support the
project is very large. Furthermore, the need to build frameworks and systems
that encourage participation is growing, as citizens and other stakeholders are
not familiar with it.

5.4 Places and Opportunities

It has a great meaning in creating places and opportunities for stakeholders to
meet regularly. In particular, through the active try in the provided places and
opportunities, citizens and stakeholders can become the active participant from
a mere a mere audience to the living lab. MindLab, which was created with the
aim of co-creating policy with the public in Denmark, also regularly held events
and meetups for anyone wish to participate. One of the goals was to involve
more people and enrich the ecosystem, but it focused more on a point that each
citizen would be the active creator in the lab ("MindLab event", 2016).

5.6 Media and Transmission

In order to promote the circulation of the ecosystem, it is essential to advertise
the mission and activities of a living lab to a wider audience. Media exposure and
transmission play a major role in achieving such purpose. Open Living Lab Days
is one of the good examples. By participating in the Open Living Lab Days held
by ENoLL every year, participants from a living lab can widely tell the vision and
activities of the living lab not only by communicating with other participants but
also by holding the workshop or displaying the poster ("Call for Workshops",
2019). Having the global connection with researchers and practitioners as well
as domestic ones can enhance the impact of each living lab.

287
5.7 Citizen-based projects
Citizens who are active in a living lab activity are the source of innovation. It is
crucial to support those who have the passion for doing the project for the city or
community. This is based on the idea of lead-user innovation that users know
the most what they really need, advocated by Eric von Hippel (Hippel, 1986).
Forum Virium Helsinki provide the open call for the enthusiastic people to submit
their proposal of the project under the theme and provide a certain amount of
grand to carry on the project if it is accepted ("Open call arkistot - Forum Virium
Helsinki", 2019).

5.8 Leadership
Leadership is one of the essential elements of living labs, but in Europe it has a
different way of understanding. It is the image of general leadership that goes to
the vision and purpose, and lead the stakeholders, but in Europe, the emphasis
is on the harmonisation of stakeholders with different historical backgrounds and
values that exist in the ecosystem. Listen carefully to the ideas and ideas
possessed by local residents and citizens, cherish the diversity that is said to be
the source of innovation, and work toward your vision.

6 Orchestratorship
Another focus is put on the leadership required for practitioners of a living lab.
This section begins with the brief summary of each interview and then describes
5 principles and 16 actions that are identified by GTA analysis. In honor of
Kaisa’s expression in her interview, we call the required leadership for
practitioners of the living lab as “orchestratorship”. 5 principles and 16 actions
that compose the orchestrator ship are described below.

High Tech Campus Eindhoven

At High Tech Campus Eindhoven, Mr. Cees Admiraal, Business Director,
cooperated with the interview. Having joined Phillips as a researcher, he was
involved in launching High Tech Campus Eindhoven in particular for creating an
ecosystem with professors or making a vision with stakeholders. He mentioned
the significance of knowing what you do not know through listening to the
professionals for becoming a better leader. On the other hand, it was added that
it was also important to hear the voice of the younger generation who had new
values. Open your mind by exploring these new opportunities is significant action
for leaders.

Forum Virium Helsinki

From Forum Virium Helsinki, Kaisa Spilling, Development Manager of Smart
Kalasatama joined our interview session. In the interview she had repeated the
word “orchestration” over and over. It is necessary for leaders to harmonise
stakeholders in the ecosystem who have the different values and ideas. Thinking
about what their value is and providing value in order to get them involved in the
project. Building relationships of trust, delivering value to each stakeholder and
orchestration for greater impact are the actions that living lab leaders need.

Living Labs Taiwan

288
From Living Labs Taiwan, Ms. Belinda Chen has answered our interview in
Taipei, Taiwan. She is the director of Living Labs Taiwan and also is the deputy
director of Institute for Information Industry (III) which manage Living Labs
Taiwan. She is always thinking about what a living lab can offer for the future in
five to ten years when conducting a testbed with citizens as end-users based on
joint research with a private company, which is one of her duties. What she
cherishes as a leader was to understand that it is a matter of course that many
failures occur in the living lab as a field of demonstration experiments. It is more
important to learn what you can learn from the mistakes, she added.

Figure 2.

289
6.1 Co-creation
The first principle is to promote co-creation, which is one of the essential
principles for living lab practitioners, as highlighted by the eight main components
and by ENoLL. There are two action that are proposed for promoting co-creation.
The first is to work with multiple stakeholder people with different background in
ways of thinking, in nationality, in history and in future. By respecting the opinions
of stakeholders who have different values, ideas, and ideas for the future that
exist in the ecosystem, it is possible to create unexpected innovations from living
labs. It is essential for leaders to be able to respect for differences and diversity
in an era of accelerating globalization and increasing mobility of human
resources. The second is to have experience of cooperation at all levels from
policy making to working in the field. In order to carry out demonstration
experiments involving the region and the subsequent social implementation, not
only cooperation with citizens and end users at the field level, but also higher-
level experiences such as policy making are required. Belinda from Living Labs
Taiwan focuses on not losing the opportunity to actually interact with citizens and
end users even while working with the government or public sectors.

6.2 Empowerment
The second principle is empowerment. Enabling citizens and other stakeholders
to be able to actively participate in the project instead of being involved will
increase the sustainability of living lab with the higher motivation of them and
leads to a greater impact. Leaders needs to empower stakeholders in the
ecosystem by sharing their own energy and vision to support their proactive
efforts. To be more specific, there are four actions in this principle. The most
representative action is to have the social and emotional skills. It is very important
to read social change and think about the next trend in order to produce a larger
impact. On that basis, understanding what an individual needs and what kind of
emotion it has can empower the individual in a way that is good for both the
individual and society. To Influence the young generation with motivation and
capability to want to challenge is also important. Influencing young people with
passion and capability also leads to better ecosystem metabolism. As the pace
of change in society is increasing at an accelerating pace, the younger
generation is the driving force for the creation of new values. In High Tech
Campus Eindhoven, Cees Admiraal is consciously making acceleration events
or pitch contests for young researchers or entrepreneurs to challenge.

6.3 Exploration
The third principle is exploration. It is important for the leaders themselves to be
inquisitive and to respect new values, ideas and whatever they have not been
encountered before, which fosters a culture that respects new things as a whole.
For this principle, three action are provided. The most representative action is to
listen to and learn from the professionals and the young. It always comes from
young generations who live in the new culture or professionals who know the
cutting edge. The leaders are required to listen to them to encounter whatever
you may not know. Cees Admiraal from High Tech Campus Eindhoven also
make it a practice to talk to professionals on the Campus or to younger
generations in the organisation when he faces the challenges.

290
6.4 Open and fair
The fourth principle is open and fair. In order to let more stakeholders to be a
part of the project, it should be open enough that anyone can participate in
projects and discussion, and reasonable enough that participants can
understand the progress and decision process. In Smart Kalasatama Project,
only the elected members can attend Smart Kalasatama Innovators Club but
many of the opportunities are open to anyone who has the interests and
motivation to be part of it. There are four actions for this principle. The most
representative action is to show project process, structure and progress. Even if
people can get into part of the discussion or project but are not provided the
mechanics of decision- making, they are not enabled to participate but exploited.
Mads Bonde Clausen of MindLab explains this situation that exploitation of
citizens’ participation and motivation does happen for the participatory design
project or co-creation process unless leaders pay attention. It requires certain
costs to share the process, structure and progress but by sharing these helps to
foster the independence of the participants.

6.5 Reflection
The last principle is reflection. By looking back on your own efforts, you will be
able to see if you are approaching the future you want to go both as an individual
and as an organization. Three actions are required to do for this principle. One
is that the stakeholders are diverse, the value provided is also diverse. If you
emphasize only one actor, it may happen that you cannot provide value to other
actors, so it is necessary to reflect on the value provided. The second is that
although the ecosystem is the foundation of the living lab, it is also important that
the leader in charge of the center review the behavior is. The last is Have the
long-term view based on new innovative behavior. While living lab efforts can
have greater impact, they cannot always deliver immediate value. Based on the
new principles of behaviour, it is essential to think things in the long run.

7 Conclusion
Living labs are spreading mainly in Europe, but social background and cultural
context are different from other part of the world such as Japan, with the history
of industrial development. Previous research shows there are many researches
introducing the components and features for living labs, but these usually deal
with case study of European living lab, which cannot be directly diverted into
other countries with different conditions. Research turned out to show that there
are three main phases of launch, foundation and involvement and eight main
components of theme setting, ecosystem formation, co-creation approach,
funding and framework, places and opportunities, media and transmission,
citizen-based projects and leadership. There are also actions to be taken in each
phase so that potential users can use it as guidelines understanding in which
phase they are in. Another outcome is the leadership required of living lab
practitioners. We conducted interview with the practitioners in Finland, the
Netherlands and Taiwan and found five principles and 16 principles of action,
which we call orchestratorship.

291
The first future avenue is to find a blind spot. This time, it is assumed that there
are important components that have not been found yet and actions that are
required of the leader, because research results were generated from interviews
with the limited number of practitioners. The next development is to increase the
number of interviewees and explore the blind spots. The second is to increase
the resolution of research. Although we could have identified the framework
through this research, the purpose of this research is to support practitioners so
that we aim to bring it into a state that is easy for practitioners to use. The last is
to validate the research. By using research output together with researchers and
practitioners, we aim to produce more valuable research by making
improvements.

In particular, in this research, although the framework was produced based on

the successful cases in Europe, verification of its usefulness is still insufficient.
In Japan, there are many close approaches that are not called living labs. For
example, we feel great potential in living a lab on the theme of utilization of
forests and mountains, which the administration of Shiojiri City, Nagano
Prefecture is working on. As the next phase, we would like to apply the
framework and principles created from this research to the context of Japan and
create a mechanism that this framework can be applied to each region. Since
this research has just started two years ago, we would like to work on these
future avenues and co-create this research with researchers and practitioners
around the world.

References
Call for Workshops. (2019). Retrieved from https://openlivinglabdays.com/call-for-
workshops/
Carayannis, E., & Campbell, D. (2009). 'Mode 3' and 'Quadruple Helix': toward a
21st century fractal innovation ecosystem. International Journal Of
Technology Management, 46(3/4), 201. doi: 10.1504/ijtm.2009.023374.
Chesbrough, H. (2006). Open innovation (1st ed., p. xxiv). Boston: Harvard
Business Review Press. Economic and Industrial Survey Committee of
Japan. (2018). Open Innovation White Paper. Tokyo. Eindhoven. (2019).
Retrieved from https://www.intelligentcommunity.org/eindhoven.
European Network of Living Labs. (2019). PROJECT PORTFOLIO 2018 - 2019
[Ebook] (1st ed., p. 5). Brussels. Retrieved from
https://ja.scribd.com/document/394441232/ENoLL-Projects-Portfolio.
Forum Virium Helsinki - European Commission. (2019). Retrieved from
https://ec.europa.eu/growth/tools-databases/regional-innovation-
monitor/organisation/helsinki- uusimaa/forum-virium-helsinki.
Future Center Alliance Japan. (2019). Notice of partnership agreement with
ENoLL. Retrieved from https://www.futurecenteralliance-
japan.org/information/news/590.
Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago:
Aldine Pub. Co.

292
Handbook for Co-creation. (2019). Retrieved from
https://www.espoo.fi/materiaalit/espoon_kaupunki/verkkolehti/handbook-
for- cocreation/html5/index.html?page=1&noflash.
High Tech Campus Eindhoven: Campus History. (2019). Retrieved from
https://www.hightechcampus.com/who-we-are/campus-history.
HighTech Connect China. (2018). Retrieved from
https://www.hightechxl.com/hightech-connect- china/en.
Hippel, E. (1986). Lead users: a source of novel product concepts. [S.l.]: [s.n.].
Hossain, M., Leminen, S., & Westerlund, M. (2018). A Systematic Review of
Living Lab Literature. SSRN Electronic Journal. doi:
10.2139/ssrn.3307055.
Intelligent Community of the Year. (2011). Retrieved from
https://www.intelligentcommunity.org/ic_of_year.
Jara Pallana, M. (2016). Taiwan after 70 years: How Japan's colonialism still have
an impact in modern day Taiwan. Academia Edu. Retrieved from
https://www.academia.edu/28565009/Taiwan_after_70_years_How_Jap
ans_colonialism_still_have_
an_impact_in_modern_day_Taiwan?auto=download.
MindLab event. (2016). Retrieved from http://storbritannien.um.dk/en/news/
newsdisplaypage/?newsid=48ad5889-b3d9-4c8f-84cb- c2d2448fd08f
Open call arkistot - Forum Virium Helsinki. (2019). Retrieved from
https://forumvirium.fi/en/tag/open-call-en/
Pentland, W. (2013). World's 15 Most Inventive Cities. Forbes. Retrieved from
https://www.forbes.com/sites/williampentland/2013/07/09/worlds-15-
most-inventive- cities/#414b6e34ec5d.
Smart Kalasatama attracts innovation tourists to Helsinki - Forum Virium Helsinki.
(2017). Retrieved from https://forumvirium.fi/en/smart-kalasatama-
attracts-record-numbers-innovation- tourists-helsinki/
Smart Kalasatama initiative enters a new phase - Forum Virium Helsinki. (2015).
Retrieved from https://forumvirium.fi/en/smart-kalasatama-initiative-
enters-a-new-phase/
Taiwan III Living Lab. (2019). Retrieved from http://www.livinglabs.com.tw/
en/index.php.
TNO, H. (2019). Holst Centre in a nutshell. Retrieved from
https://www.holstcentre.com/about-holst-centre/holst-centre-in-a-
nutshell/
What does Forum Virium Helsinki mean to the city? - Forum Virium Helsinki.
(2019). Retrieved from https://forumvirium.fi/en/introduction/what-does-
forum-virium-helsinki-mean-to-the-city/

293
294
 Living Labs need sustainable revenue
models: The Funding Mix Framework to
bridge the gap
between theory and practice
Edoardo Gualandi*1 and Flavia Fini2

*Corresponding author
1Eindhoven University of Technology, The Netherlands
2 Alma Mater, Italy

Category: Innovation Papers

Abstract
Living Lab (LL) represents an emerging innovation methodology which has the
potential to bring together different actors in a collaborative process to develop
solutions to diffuse social problems. Nevertheless, a substantial number of Living
Labs struggle to translate the value created into a sustainable revenue model
and, thus, they often present an unintended temporary nature. Research about
Living Labs is primarily focused on theoretical and methodological aspects, while
good practices, especially for what concerns funding, revenue and business
modelling, are still under-researched. In this paper, we analyze good practices
and critical problems of six LLs from across Europe. Then we apply the
previously developed Funding Mix Framework to understand if it can be
considered valuable support for LLs to develop a more sustainable revenue
model, ensure long term viability and scale up their operations

Keywords: Living Lab, financial sustainability, long-term viability, social value,

social challenges, business model, revenue model

295
1 Introduction and objectives
Deeply rooted in real-life environments, Living Lab (LL) is either part of, or
constitutes an Innovation Network of people, private firms and public institutions.
LL is a methodology based on knowledge and observation, and guided by a
practice-driven approach; these elements combined, in the form of innovation
projects, concur in the realization and implementation of innovative solutions that
are user- or community-driven, co-created by the customers, and tested and
validated in real-life settings (Gualandi, 2018). Indeed, LLs have a great potential
to serve their community and develop innovations which can solve diffused social
problems and improve the life of the citizens. In order to concretely have an
impact on society, LL’s operations must be ensured for an adequate time:
financial sustainability is fundamental for a LL to be viable in the long-term and
eventually scale up its operations. Although that, a substantial number of LLs
struggle to translate the value created into a sustainable revenue model and,
thus, LLs often present an unintended temporary nature since they stop their
activities when the funding ends (Katzy, 2012). Despite that, most studies focus
on theoretical and methodological aspects of LLs, while good funding practices
are still under-researched and there is a lack of concrete research on viable
revenue models for LLs.

Therefore, in this paper, we first analyze the value creation process for LLs
through a systematic literature review, with a prevailing focus on the social value
which makes LLs a promising methodology to address the social and
environmental challenges of our time. Then, we explain the problem of financial
sustainability, which often precludes to the LLs the possibility to concretely better
society. Then, we present the Funding Mix Framework (FMF), a tool developed
in previous studies, which can be considered a practical support to design a
sustainable funding model. Then, taking the FMF as a reference model, we
analyze six LLs from Italy, Spain, Serbia, Slovenia and the Netherlands. The
results of the case study provide concrete insights over good financing practices
and common problems. In the conclusions, on the one hand, we have some
confirmation over the suitability of the FMF in developing a self-sustainable
funding model and, on the other hand, we reflect on the implications of the
findings for LL’s long-term viability and scalability.

2 Literature review
The last decades marked a radical shift from the traditional conceptions of
organization and market. Many firms collaborate with customers to improve the
effectiveness of product development which often results in considerable benefits
(Nijssen et al., 2012). Furthermore, customers are valuable sources of product
and service innovation (von Hippel, 2005). Moreover, the need for always new
and complex products and services paved the way for many organizations to
draw from a broad set of external sources of knowledge: users, customers,
suppliers, partners and competing enterprises, universities, research centres and
governmental institutions can concur to a shared objective. It is based on these
premises that Chesbrough’s (2006) developed the concept of Open Innovation.
Living Lab is an emergent methodology that has the potential to fulfil the role of

296
bridging the gap between User-Innovation and Open-Innovation (Schuurman,
2015).

2.1 Living Lab and value creation

According to ENoLL(https://enoll.org/about-us/) LLs are “user-centred, open
innovation ecosystems based on a systematic user co-creation approach
integrating research and innovation processes in real-life communities and
settings”. The value created by the LL varies in terms of nature and affected
actors. Hence, to better understand and direct the activities of a LL, can be helpful
to adopt the classification between economic (transactional), business (company)
and public (social) value (Gualandi & Leonardi, 2018; Gualandi, 2018).
Transactional value covers aspects that are highly tangible from different
stakeholders’ perspectives such as the growth of the companies that collaborate,
an increased competitive advantage or the extent to which new business are
generated and survive (Baccarne et al., 2014). LL’s innovation process delivers
transactional to the whole value chain in the form of better products and services.
(Ståhlbröst & Holst, 2012). Company value is an extension of economic value
and includes other forms of value such as employee value, customer value,
supplier value, managerial value, and societal value. (Bergvall-Kåreborn et al.,
2009). Finally, social value is one peculiar characteristic that differentiates LL
from other innovation methodologies and makes it a relevant innovation
methodology for the future due to its ability to support Sustainable Development
Goals. The LL serves as a connection between research, citizens and the actual
living environment (Franz, 2014) and works as an innovation intermediary and
aggregators of external inputs that are then translated into requirements and
design parameters for valuable social innovation (Mention & Torkkeli, 2015;
Almirall & Wareham, 2011); researchers, businesses, and users explore novel
manners of contextualizing knowledge and technologies in real-life to result in a
concrete impact on society (Pierson & Lievens, 2005); by the inclusion of the user
at the centre of the development process, it is possible to meet the specific needs
and aspirations of local contexts with the proposed solutions (ENoLL, 2016).
Moreover, per Ståhlbröst (2012) the innovation processes supported by LL must
address environmental and social issues while considering the economic impact
and LL also takes responsibility of environmental, social, and economic effect
(Bergvall-Kåreborn et al., 2009) and pursue societal improvements (Leminen et
al., 2012). LL offers a mechanism to support stakeholders’ collaboration and
generation of innovation outcomes in social environments, ranging from
improving everyday living conditions to systematic citizens innovation (Leminen
& Westerlund, 2015). Indeed, LLs dedicated to social value can effectively
increase cohesion in society (Schuurman et al., 2016) or improve users’
behaviour with the respect of issues like environmental awareness (Ståhlbröst &
Holst, 2016).

2.2 Financing strategies: the struggle for a revenue model

Social value should be the core of any LL projects but it is also the hardest to
obtain: long-term viability is necessary to have a concrete impact on society since

297
LLs require continuous funding and a sustainable financing model to support and
scale up their innovation (Guzman et al., 2013; Evans et al., 2015). Nevertheless,
LLs are often not financially sustainable and struggle to translate the value
created into a sustainable business model (Brankaert et al., 2014, Katzy, 2012).
Hence, a significant number of LLs presents an unintended temporary nature
(Leminen et al., 2012) and financial sustainability seems to be the key condition
for LLs to become permanent and remain operational in the long-term (Veeckman
et al., 2013). Moreover, the ability to implement solutions with a concrete social
impact must be supported and ensured during the entire life of a LL.

Several researchers found out that most of LLs that are adequately funded
primarily rely on public grants and subsidies (i.e. Brankaert, den Ouden, &
Grotenhuis, 2014; Wu, 2012). Even if this is a feasible financing option in the
short-term, it does not ensure the viability in the long-term since many of these
LLs stop their activities when the funding ends. A potential cause of this struggle
can be linked to the fact that many LLs do not make systematic use of business
modelling techniques for themselves (Mastelic et al., 2015). In addition, often
common business modelling tools such as the Value Proposition Builder (Barnes
et al., 2009), the Value Proposition Canvas (Osterwalder et al., 2015), and the
People Value Canvas (Wildevuur et al., 2013) are not adequate to consider
certain peculiar characteristics of the LL (Äyväri & Jyrämä, 2017; Schuurman et
al., 2019).

Finally, LL theory lacks solutions and concrete financing strategies: there is only
a limited amount of literature available that combines LLs with business models
(Hossaina et al., 2019). Being the challenge of financial sustainability of great
importance in the success of LL projects and a key condition for the generation
of social value, the lack of concrete instruments to define a viable revenue model
is a critical deficiency in LL research (Gualandi, 2018). In line with that, in a
previous study (Gualandi & Romme, 2019), we proposed the Funding Mix
Framework as a concrete tool to better identify potential revenues and sources
of funding which can be potentially exploited by any LL in order to reach financial
self-sustainability, long-term viability and thus to scale up their operations. The
FMF represents a practitioner-oriented tool that can help to bridge the gap
between theory and practice, and that can improve the chances of a LL to
successfully deliver social value to its stakeholders and to the citizens

2.3 Funding Mix Framework: a novel revenue model

In line with that, in a previous study (Gualandi & Romme, 2019), we proposed the
FMF as a concrete tool to better identify potential revenues and sources of
funding which can be potentially exploited by any LL in order to reach financial
self-sustainability, long-term viability and thus to scale up their operations. The
FMF represents a practitioner-oriented tool that can help to bridge the gap
between theory and practice, and that can improve the chances of any LL to
successfully deliver social value to its stakeholders and to the citizens.

According to the FMF, LLs can draw on different funding options, which we
classify as pay per service (PPS), subsidies (SUB), out of network funds (ONF)

298
and cross-financing (CRF). In the next paragraphs, we explain more in detail the
four categories.

Figure 1. Funding Mix Framework

2.3.1 Pay per service

Pay per service is the most immediate monetary recognition of the services
offered by the LL. For this reason, it can be considered the financial return for the
transactional value generated. Hence, transactional value is primarily delivered
to business partners which turn to the LL for the development or improvement of
commercial products and services. Indeed, the source of PPS is mostly private.
In rare occasions, PPS can be related to company and social value. In these
cases, the source can shift partially toward the public sector. The stakeholders
that contribute with PPS belong to the LL’s the network. Finally, PPS is a funding
option at a project level: indeed, PPS relates to the services provided by the LL
in the context of a specific project.

2.3.2 Subsidies
Subsidies are the most frequent funding option connected to social and company
value and are ensured by the strategic partners. In fact, social and company value
is mostly recognized by actors committed to a long-lasting relationship, in which
the interest is not limited to projects but aims at the development of shared goals
and objectives. Social value is delivered to the citizens and to stakeholders from
public sector and education, which compensate the LL in the form of subsidies.
In addition, public sector, education and businesses are the main recipients of
company value, if involved in long-lasting relationships and compensate the LL
also with subsidies. Hence, SUB is a funding option that mainly relies on public
sources. Finally, we noticed that SUB is a funding option linked with the entire

299
innovation process and operations of the LL and is provided by stakeholders and
actors belonging to the network.

2.3.3 Out of network funds

LL’s mission is generally in line with the Sustainable Development Goals. For this
reason, LLs have the possibility to gather important funds by systematically
applying for European Community, national and regional calls. LL’s projects are
often compatible with public policies, and open calls are good options to finance
the creation of public value. The funds are made available primarily by public
bodies and, thus, are mostly coming from public sources. These calls can also
be thrown by private entities, like banks and cooperatives, but is a less common
situation. The organizations issuing the funds are not directly involved in the LL
network, but they are only responsible to grant funds based on predefined criteria.
Finally, the funds are a support for the mission of the LL and, thus, relates mostly
to the strategic level.

2.3.4 Cross-financing
Differently from PPS, SUB and ONF, this financing option is not linked to the
activities of the LL, neither contribute to the network. In fact, cross-financing is
rather an alternative way to profit from the LL’s assets, such as the physical
location (i.e. the LL can sublet permanently part of its space to a bar or to a co-
working office, or temporarily to events, conferences, meetings) or the
complementary equipment (i.e. the LL can lease printers, software etc.). The
source of CRF is almost exclusively private and completely external to LL
activities.

3 Research questions
The objective of this paper is to better interpret the FMF developed in the previous
studies to have a deeper understanding of how it can concretely support any LL
in exploiting the full potential of its network and activities.

Therefore, the empirical research will be guided by the following research

questions:
• How do LLs exploit the different funding options proposed by the Funding
Mix Framework and how can a balanced approach help Living Labs to
become financially self-sustainable?
• How can the Funding Mix Framework Support Living Labs’ innovation
projects in order to have a greater impact on society and aim at scalability?

The hypothesis is that any LL can have greater chances of success in generating
social value, initiating virtuous processes that can be scaled up or replicated if
they correctly approach their business and revenue model. In particular, we
hypothesize that LLs that address the different funding options presented in the
FMF in a balanced way.

Only in this way, LLs can succeed in having a substantial impact over society.

300
4 Methodology
4.1 Case study design
To answer the research question, we adopt a multiperspectival methodological
approach both in regards to the case selection process (§ 4.2) and the data
collection and analysis (§ 4.3). This method is deemed the most appropriate given
the diverse case studies intended to be analysed, and the most effective in giving
to the results a more compelling nature and thus ensure higher robustness, which
is a fundamental measure of the quality of the research design. This has been
done in accordance with Yin (2003) which states that the choice of the number
has a strong relationship with the purpose of the investigation and thus it is to be
defined by the practitioner coherently with the adopted replication logic. In fact,
the cases can be selected in such a way that they either predict similar results or
provide contrasting results but for predictable reasons. Therefore, in an
exploratory study like the current one, we decided to have to have multiple
perspectives and examples, but, at the same time, we still wanted to include not
an extremely large sample so that we could still explore them in detail in a
qualitative way. Accordingly, we designed the case study to involve between 5
and 10 cases.

4.2 Case selection

In order to select the cases, around 50 Living Labs from all over Europe were
analyzed based on different sources of information (websites, newspaper
articles, academical studies) to preliminarily understand their positioning about
orientation and source of funding. Then, 24 Living Labs that were considered
consistent with the research purposes were contacted. Only a limited number of
them replied, and among the respondents several were not available for research
purposes or were no longer active. Three cases, Stratumseind Living Lab (SLL)
in Eindhoven (the Netherlands), Amsterdam Fieldlabs (AFL) in Amsterdam (the
Netherlands) and Textile&Clothing Living Lab (TECLA) in Palermo (Italy) were
already involved in previous studies thus we iterated the analysis in the light of
an expanded theoretical framework and through new perspectives. To further
expand the previous studies, we included three additional cases: ORbITaLA in
Maribor (Slovenia), PA4ALL in Novi Sad (Serbia) and BIRD Living Lab in Bilbao
(Spain) are completely new additions to this research. The six cases are very
different from each other under many points of view: nevertheless, we do not
consider this aspect a limitation for the current research but rather an opportunity:
in fact, the previous studies suggested that the FMF can be suitable for
heterogeneous LL. Indeed, we selected the six cases in a fashion that could
ensure the broader view as possible over different casistics of LLs. Therefore, in
this case, we present the insights about LLs active in different fields: from textile
to open data, from social policies to natural environments and farming, from urban
contexts to rural areas. At the same time, among the potential candidates we
selected LLs in different phases of their life - from the ones in their start-up phase
to ones that are already considered well-established institutions in the context
they operate in.

4.3 Data collection and analysis

In regard to data collection and its analytical process, we employ a wide array of
data. Interviews are the primary source of data collection. Interviews are often

301
used since they allow the researcher to retrieve experiences, behaviors, opinions,
values, feelings, factual knowledge, and personal background (Esterberg, 2002).
When possible, we opted for semi-structured interviews, which means that we
began with few preset questions to then follow the respondent’s tangent of
thoughts. That seems an adequate choice in an exploratory study in which the
research direction is not completely defined upfront and new unexpected
information can raise. It also seems the most appropriate methodological tool in
order to balance the potential disparity between the researchers’ positionality and
expectations in regards to the outcome of the data collection and our
interlocutors’ interpretation and personal analysis of the data which, in this way,
is not constrained and orientated by a strict set of questions. Furthermore, we
employed also a heterogeneous set of complementary sources of information and
methods. The three Living Labs provided us with additional study material,
ranging from handbooks to the official applications to ENoLL, from video-
interviews of important partners to official PowerPoint presentations. In certain
cases, in situ research was not possible and thus we involved the manager in a
detailed survey and digital interviews covering several meaningful aspects of their
LLs still respecting the semi-structured and open-ended question approach.

In order to analyze the collected information, we combined ex-ante and ex-post

coding, and content analysis to process the entire set of collected data. The
second approach to coding is particularly adequate in these cases where the
interviews’ questions are mainly open-ended, and thus it is harder to predict the
data. Finally, we employed thematic content analysis since it is an adequate
method to represent sentences and topics referring to the questions of the
research.

4.4 Quality of research

We can state that the research and the resulting models present a good degree
of quality of research thanks to the employment of heterogeneous sources of
information and by systematic triangulation of the results. The nature of the
research, based on qualitative methods and on an exploratory approach, is a limit
to the external validity of the results. At the same time in situ researches and
cross-data analysis increase the internal validity of the approach.

This is in line with the broader objective of this study - to develop a coherent
framework gathering several aspects which are not investigated in a systematic
manner in current literature and favour the path to future researchers about
promising novel approaches rather than developing robust models.

5 Observations and results of the case study

Orbitala is a LL founded by the Maribor Development Agency with the objective
to guarantee better connectivity between research and SMEs and perform
research for industry or public administration around meaningful local challenges.
The LL has a strong connection with the public administration and the business
sector while the education sector is almost absent. ORbITaLA’s financiers, both
from inside and outside its network, are mainly involved on a project level and
provide funding only for specific projects. For this reason, the Living Lab is not

302
able to perform its activities on a regular basis and is often forced to operate
below the expected standards. The various innovation projects often depend on
a single source and the generation of a concrete impact over society is dependent
on the objective of each project, which is often determined by the main partner.
The approach to funding is often improvised and the LL is far from a concrete
financial self-sustainability. Despite that, also thanks to EU funds ORbITaLA ran
several successful projects which realized transactional, company and public
value for the main partners involved and which had a concrete impact on society.

The municipality of Eindhoven developed various LL initiatives. SLL is one of the

most successful examples. The mission of SLL is to support companies and
institutions in the development of innovative products, services and policies which
can foster the economic environment of the region and improve the quality of life
in the neighbourhood. SLL has a very strong connection with public bodies, with
the many tech companies in the area and with several educational institutes. The
LL generates consistent company value and transactional value especially for the
businesses and for the public administration, while the social value is rather
marginal. The LL developed very tight relationships within the network and counts
on important funding both on project and strategic level. Therefore, SLL is able
to perform its activities regularly, respecting very high standards. The approach
to funding is highly structured, systematic and relies on multiple sources.
Nevertheless, the local government and other public organizations are the
stakeholders that provide the most substantial part of the funding. In conclusion,
SLL has a sustainable revenue model which ensured long-term viability to the LL.

The municipality of Amsterdam, together with the Amsterdam University of

Applied Sciences (AUAS) formalized a knowledge alliance and initiated three LL
initiatives, the “Fieldlabs”, in areas at the edge of the city centre to address
diffused social issues. In 2017, AFL was able to upscale its operation to a
metropolitan perspective. The public administration and AUAS are the two main
strategic partners and, in many projects, they are the only financers. At the
moment AFL regularly performs its activities respecting very high standards but
its long-term viability is at stake: in fact, it is strongly dependent on two main
sources and thus cannot guarantee viability if the funding ends. In addition, AFL
mostly generates social value while transactional value is neglectable. Moreover,
AFL does not have a structured approach to the revenue model and the
generation of revenues from different sources is not contemplated by the board.
Nevertheless, AFL developed a trustful relationship with its main partners and the
social value that generates is highly recognized also by the public opinion and
thus represent a project legitimation.

PA4ALL focuses on precision agriculture and was founded in 2012 by the

BioSense institute in Novi Sad as a meeting place for heterogeneous
stakeholders in the agrifood sector. PA4ALL has a strong connection with the
business sector which is often involved on a strategic level and is the main
receiver of value. Despite that, the funding from this sector is rather scarce. Also,
the education sector is often involved as a strategic partner and the amount of
funding is more substantial. The public administration is mostly involved on a
project level and the funding is marginal. Indeed, international calls represent the

303
primary source of financing. PA4ALL has a structured and systematic approach
to funding which ensures a good degree of financial self-sustainability and long-
term viability. Nevertheless, PA4ALL does not contemplate cross-financing. In
conclusion, PA4ALL, which was the first precision agriculture lab in Europe, has
a strong revenue model which allowed it to have a concrete impact on society
with the creation of new businesses, the establishment of synergies and raising
the public awareness around relevant topics

BIRD is situated in the Biosphere Reserve of Urdaibai in the Basque Country, an

environmentally protected area with high industrial development constraints. The
LL is the result of an initiative started in 2009 and is now owned by the Cluster
GAIA the association of technology firms of the region. The LL connects
technology and smart economy with relevant social issues especially in the field
of sustainable ecology and biodiversity. BIRD involves partners across the
quadruple helix, especially from the business sector which are also the main
receivers of value together with the education sector. The LL has a structured
and systematic approach to business modelling but is mainly able to ensure
funding at a project level and thus only certain projects are adequately funded.
Also for BIRD, national and EU calls are fundamental. In addition, the LL exploits
several alternative ways of funding: workshops, colocation and events are just
some of the opportunities that the lab exploits outside its core activities.

TECLA is a physical space in Palermo, that encourages to discuss ideas and

projects, meet partners, develop cooperation methodologies where textile and
clothing manufacture meets technologies and advanced multimedia tools.
TECLA is hosted by Consorzio ARCA, a university business incubator. Local
authorities are not deeply involved, and the public sector is mostly represented
by the European institutions involved in the Horizon2020. Some education
institutes and small businesses are involved but only marginally and in specific
projects. Despite the efforts of the LL to structurally and systematically approach
the revenue model, the amount of funding from the network of the LL is not
substantial and thus TECLA is not always able to perform its activities according
to the planned standard and some projects cannot be executed. The main source
of financing is EU calls which allows TECLA to take part in European initiatives.
In addition, TECLA shares its space with other realities such as a coworking and
a restaurant. Nevertheless, the amount of funding is not enough to perform on a
constant basis, hence it is not always possible to have the desired social impact.

6 Discussion
The case study shows that financial self-sustainability is still perceived as a
challenge by the LLs which address it in different ways: while in some cases the
approach is systematic and well-structured, in others is rather improvised and
circumstantial. Moreover, the employed funding options are various and the six
cases have different approaches which suggest that they do not refer to a
common paradigm. Nevertheless, the empirical evidence suggests that the
Funding Mix Framework adequately represents the full range of financing options
available for the analyzed LLs and thus it can be interesting to analyze their
funding strategy accordingly.

304
Four LLs can be considered financially self-sustainable, even though not every
case employs all four options. These four LLs are able to perform their activities
on a regular basis, to respect the expected standard and almost all their projects
are ran until the end with successful results: (1) BIRD is the only LL to make use
of PPS, SUB, ONF and CRF and in fact it evolved over the past ten years, it was
scaled up, and was framed in different initiatives. (2) Despite SLL only counts on
PPS and SUB, it established very strong relationships with strategic partners from
public and education sectors and with many businesses from the vivid high-tech
field. (2) PA4ALL does not employ CRF but thanks to a balanced mix of the other
three options, together with the broad national and international network, ran
many projects with success. Nevertheless, the LL’s activities are performed
regularly but not on a daily base: thus, has a different approach: it mostly counts
on SUB from two main partners and does not have a structured approach to
financing. This strategy has been successful in the last years and AFL was able
to scale up its operations. At the same time, it is a factor of risk: long-term viability
might be at stake if one of the two main stakeholders retire. On the contrary,
BIRD, PA4ALL and SLL can count on a broad and heterogeneous network which
makes them not only self-sustainable in their routine activities but also eventually
viable in the long-term.

The remaining two cases present some similarities: they are not able to perform
their activities on a regular basis but they only function occasionally. The staff of
the two LLs is mainly constituted by the two founders which are employed by the
institution owning the LLs. Moreover, the prevailing funding option is represented
by ONF: the LLs are activated in the context of European projects and their
budget is mostly provided by EU calls. These are still interesting cases: in fact,
they are far from a financial self-sustainability which can allow routine operations
and activities. Nevertheless, thanks to their ability to systematically apply to
international calls they have the potential to be viable in the long-term. This is
primarily suggested by ORbITaLA which has been active already for seven years.
In contrast, TECLA is still in a start-up phase and is still fine-tuning its business
model.

Another interesting aspect emerging from the case study is related to the
geographical location of the six LLs: there are hints which suggest the existence
of very different approaches between economically depressed areas, mostly in
the South of Europe, and the wealthier countries in the North. Indeed, the cases
from the Netherlands are the only ones that do not need to apply to EU calls to
finance their activities: in these cases, they are capable to obtain substantial
financial support by the public administrations and by education institutes, while
in the other cases the involvement of the institutions is rather marginal when not
completely absent.

Furthermore, it is interesting to notice the importance of having a diversified set

of financers and sources of funding: even if it is possible to be financially self-
sustainable for several years only counting on one or few main partners while
pursuing social value (i.e. AFL), we also noticed that when a LL rely on too few
partners it tends to become strongly dependent on him, not only economically,

305
but also in the direction that the LL’s projects take. In addition, the viability of such
LLs is at stake in the moment that an important partner resigns or stop providing
funds. Therefore, having a broad and heterogeneous set of financers is not fully
necessary but it might be an important advantage in achieving a stable long-term
viability.

Finally, we could have some interesting insights over the social impact of the six
LLs. AFL’s stakeholders are primarily interested in the social value and thus the
social impact, which is the main purpose of the LL, is an adequate project
legitimation. Also, for BIRD, social value is a central aspect and thanks to the EU
funds the LL is able to spread the environmental awareness across the entire
network and direct innovation towards shared sustainable goals. PA4ALL, also
thanks to EU funds, delivers social value in its community through sustainable
rural development and through the creation of new businesses fostering
employment. A similar approach is adopted by TECLA in the textile field but, since
it is not able to perform regularly, the achieved social impact is far from the
objectives. This is a problem that also ORbITaLA is experiencing: in fact, being
forced to operate occasionally only in specific projects the generation of social
value is strongly dependent on the financers: some project has a concrete impact
on society, but in certain cases the financer is rather interested in transactional
and company value and social value remains marginal. Finally, SLL is strongly
business oriented and only rarely involves actively the citizens in its processes.
Therefore, social impact in the short term is mostly limited to an increase in safety
and security in the nightlife of the neighbourhood. Nevertheless, the LL is actively
involved in the definition of state-of-the-art open data policies whose social value
will eventually materialize in a later time. In conclusion, participation in EU
projects and a tied relationship with institutions and education centres favour the
realization of social value. On the contrary, a too marked business orientation or
occasional project orientation are impediments for adequate attention to social
value.

7 Conclusions
We believe that this paper represents an important contribution to the state-of-
the-art of LL both from a theoretical and practical perspective. We put in the
spotlight several deficiencies in the current literature which represent an obstacle
for the definition of virtuous practices financial self-sustainability. In line with
previous studies (Brankaert et al., 2014; Leminen et al. 2012; Ståhlbröst, 2012;
Schuurman, 2015; Gualandi & Romme, 2019), the empirical research showed
that a structured approach to business and revenue modelling can increase the
possibilities for a LL to become financially self-sustainable and viable in the long-
term.

The research shows two main tendencies in the definition of a sustainable

financing model: (1) some LLs confirmed the idea of Niitamo et al. (2006) and
strongly rely on their network and secure long-term funding establishing strong
relationships with important stakeholders, while it emerged that (2) other cases
mainly count on EU calls and operates in the context of European projects. Both
directions come with consequences:

306
In the first case, LLs are capable to finance regularly their activities but they risk
that the interest of the partners prevails over the mission of the LL which becomes
a problem when the network has a strong business orientation which may prevent
the LL to pursue actively social value. Moreover, when a LL counts on a limited
number of financers, also long-term viability is at stake: if one of the main partners
stop financing the LL, financial self-sustainability cannot be ensured.

In the second case, LLs are more independent in pursuing their goals, and if they
systematically apply to EU funds they can be viable in the long term.
Nevertheless, LLs that rely on European calls often cannot give continuity to their
activities and operate occasionally with a prevailing project orientation. In this
way, it is harder to have a concrete impact on society.

The research suggests that an optimal strategy for a LL is to place at the middle
between these two diverging approaches: indeed, the FMF seems to be a
valuable instrument to be employed by LLs in defining the most suitable funding
mix. In fact, all the funding options employed by the LLs participating in the case
study can be framed in the FMF. Nevertheless, we suggest that there is not a
financing model which is optimal for any LLs, but it depends on several conditions
of the LL such as the mission, the field of application, the local context, the
maturity of the institutions, the innovativeness of the network.

Moreover, the empirical research confirmed that when they are not financially
self-sustainable, LLs struggle in generating the expected social impact. At the
same time, we discovered that most LLs do not fully exploit all the possible
sources of financing identified by the FMF. Therefore, LLs could use the FMF to
identify new funding options which can eventually increase their impact on
society. Finally, being the scalability of a LL connected with its self-sustainability
and long-term viability, the FMF can also help LLs to scale up their operations.
This aspect is under-researched in current LL theory. Indeed, this point is truly
based on in-field observation and on reasoning: indeed, it seems clear from the
analysed cases that financial self-sustainability does not necessarily imply the
possibility to scale up the LL, but if a LL is not financially self-sustainable then it
does not represent a promising virtuous model to be replicated or scaled up.

8 Limitations and future work

Even though the Funding Mix Framework seems to be a promising tool to reduce
the gap between theory and practice and valuable support to fine-tune an
adequate business and revenue model including a full range of complementary
sources of funding, this framework needs further validation with more quantitative
studies and with continuous observation of LL cases. Moreover, the paper puts
in the spotlight the deficiencies in current LL research: the topic of long-term
viability needs to be analysed and explored broadly. For further research in this
direction, it seems also necessary to look at the specifics of LLs’ activities from a
geographical point of view, examining whether the presence of a LL in a certain
territory can impact its practice and its ability to achieve a long-term and self-
sustainable revenue model. Therefore, we advocate for substantial and

307
systematic research around this topic to develop a shared understanding and a
reference theory.

References
Almirall, E., & Wareham, J. (2011). Living labs: Arbiters of mid-and ground-level
innovation. Technology Analysis & Strategic Management, 23(1), 87-102.
https://doi.org/10.1080/09537325.2011.537110.
Äyväri, A., & Jyrämä, A. 2017. Rethinking Value Proposition Tools for Living Labs.
Journal of Service Theory and Practice, 27(5): 1024–1039.
https://doi.org/10.1108/JSTP-09-2015-0205.
Baccarne, B., Mechant, P., & Schuurman, D. (2014). Empowered cities? An
analysis of the structure and generated value of the smart city Ghent. In
Smart City (pp. 157-182). Springer, Cham.
https://doi.org/10.1007/978-3-319-06160-3_8.
Barnes, C., Blake, H. and Pinder, D. (2009). Creating & delivering your value
proposition. London: Kogan Page.
Bergvall-Kåreborn, B., Eriksson, C. I., Ståhlbröst, A., & Svensson, J. (2009). A
milieu for innovation: defining living labs. In Proceedings of the 2nd ISPIM
innovation symposium: Simulating recovery - the Role of innovation
management, New York City, USA.
Brankaert, R., den Ouden, E., & Grotenhuis, F. (2014). Identifying differences
between living labs - Learning from practice. In ISPIM Conference
Proceedings (p. 1). Dublin, Ireland.
Chesbrough, H. W. (2006). Open innovation: The new imperative for creating and
profiting from technology.
Boston, Massachusetts: Harvard Business Press.
ENoLL (2016). Introducing ENoLL and its Living Lab Community. Retrieved from
https://issuu.com/enoll/docs/enoll-print.
Evans, J., Jones, R., Karvonen, A., Millard, L., & Wendler, J. (2015). Living labs
and co-production: university campuses as platforms for sustainability
science. In Current Opinion in Environmental Sustainability, 16, 1-6.
Franz, Y. (2014, September). Chances and challenges for social urban living labs
in urban research. In Conference Proceedings of Open Living Lab Days
(pp. 105-114). Amsterdam, the Netherlands.
Gualandi, E. (2018) Models for Living Lab Financial Sustainability: a Dutch-Italian
Case Study. Master Dissertation. Eindhoven University of Technology, the
Netherlands.
Gualandi, E. & Leonardi, L. (2018) Models for Living Lab’s sustainability:
Evidences from Italy and the Netherlands. Proceedings of Open Living Lab
Days (pp. 134-149). Geneva, Switzerland.

308
Gualandi, E., & Romme, A. G. L. (2019). How to make living labs more financially
sustainable? Case studies in Italy and the Netherlands. Engineering
Management Research, 8(1), 11-19.
Guzmán, J. G., del Carpio, A. F., Colomo-Palacios, R., & de Diego, M. V. (2013).
Living labs for user-driven innovation: A process reference model.
Research-Technology Management, 56(3), 29-39.
https://doi.org/10.5437/08956308X5603087
Hossain, M., Leminen, S., & Westerlund, M. (2019). A systematic review of living
lab literature. Journal of Cleaner Production, 213, 976-988.
Katzy, B. (2012) Designing Viable Business Models for Living Labs. Journal of
Organizational Virtualness. 2(9), 19-24.
https://doi.org/10.22215/timreview/604
Leminen, S., Westerlund, M., & Kortelainen, M.J. (2012). A recipe for innovation
through living lab networks.
Proceedings of ISPIM Conference, Barcelona, Spain.
Leminen, S., & Westerlund, M. (2015). Cities as labs: Towards collaborative
innovation in cities. In P. Lappalainen,
M. Markkula, & H. Kune, Orchestrating Regional Innovation Ecosystems–Espoo
Innovation Garden, Aalto, Finland: Otavan Kirjapaino Oy.
Mastelic, R., Sahakian, M. & Bonazzi, R. (2015), How to keep a living lab alive?
In P. Ballon, & D. Schuuman.
Living Labs: Concepts, Tools and Cases, 17(4), 12-25.
https://doi.org/10.1108/info-01-2015-0012
Mention, A.-L., Torkkeli, M. (2015) Open Innovation: A Multifaceted Perspective.
Singapore, World Scientific Publishing Company.
Nijssen, E., Hillebrand, B., De Jong, J.P.J., & Kemp, R. (2012) Strategic value
assessment and explorative learning opportunities with customers.
Journal of Product Innovation Management, 29, 91-102.
https://doi.org/10.1111/j.1540-5885.2012.00960.x.
Niitamo, V. P., Kulkki, S., Eriksson, M., & Hribernik, K.A. (2006). State-of-the-art
and good practice in the field of living labs. Proceedings of the 12th
International Conference on Concurrent Enterprising: Innovative Products
and Services through Collaborative Networks (pp. 349-357). Milan, Italy.
https://doi.org/10.1109/ICE.2006.7477081.
Osterwalder, A., Pigneur, Y., Bernarda, G., & Smith, A. (2015). Value Proposition
Design: How to Create Products and Services Customers Want.
Chichester, UK: John Wiley & Sons.
Pierson, J., & Lievens, B. (2005) Configuring living labs for a ‘thick’ understanding
of innovation. Proceedings of Epic 2005 (Ethnographic Praxis in Industry
Conference). Redmond, Washington.
Schuurman, D. (2015) Bridging the Gap Between Open and User Innovation?
Exploring the Value of Living Labs as a Means to Structure User

309
 Contribution and Manage Distributed Innovation. Doctoral Dissertation,
Ghent University / Vrije Universiteit Brussel, Belgium.
Schuurman, D., Ballon, P., Baccarne, B., De Marez, L., & Veeckman, C. (2016)
Living Labs as Open Innovation Systems for Knowledge Exchange:
Solutions for Sustainable Innovation Development. International Journal
of Business Innovation and Research, 10(2-3), 322-340.
https://doi.org/10.1504/IJBIR.2016.074832.
Schuurman, D., Herregodts, A.-L., Georges, A., & Rits, O. 2019. Innovation
Management in Living Lab Projects: The Innovatrix Framework.
Technology Innovation Management Review, 9(3): 63–73.
http://doi.org/10.22215/timreview/1225.
Ståhlbröst, A. (2012) A Set of Key-Principles to Assess the Impact of Living Labs.
International Journal of Product Development, 17(1-2), 60-75.
https://doi.org/10.1504/IJPD.2012.051154.
Ståhlbröst, A., & Holst, M. (2012). The Living Lab Methodology Handbook. Luleå,
Sweden: Luleå University of Technology Press.
Ståhlbröst, A. & Holst, M. (2016) Living lab: Stimulating adoption of smart city
innovations. Proceedings of Open Living Lab Days (pp. 145-162).
Montreal, Canada.
Veeckman, K., Schuurman, D., Leminen, S., Lievens, B., & Westerlund, M.
(2013) Linking characteristics and their outcomes in living labs: A Flemish-
Finnish case study. In Proceedings of the XXIV ISPIM Conference –
Innovating in Global Markets: Challenges for Sustainable Growth.
Helsinki, Finland.
Von Hippel, E. (2005) Democratizing Innovation. Cambridge, Massachusetts:
MIT Press. https://doi.org/10.7551/mitpress/2333.001.0001
Wildevuur, S., van Dijk, D., Hammer-Jacobsen, T., Bjerre, M., Äyväri, A., & Lund,
J. 2013. Connect – Design for an Empathic Society. Amsterdam: BIS
Publishers.
Wu, H.Y. (2012). An empirical study of UK living labs. In Proceedings of
International Association for Management of Technology IAMOT, pp. 1-
21.
Yin, R. K. (2003). Case study research design and methods. Applied social
research methods, Second edition, Sage, Newbury Park, California.

310
311
 Sustainable Person-centred Living Lab for
regional management as extension of Japanese
dementia care activities
Atsunobu Kimura*1, Mizue Hayashi*1, Fumiya Akasaka1,
Masayuki Ihara1

*Corresponding authors
1 NTT Service Evolution Laboratories, Japan

Category: Research-in-progress

Abstract
One of the difficulties of Living Labs (LLs) is ensuring their sustainability. Our
research focuses on creative person-centred care activities on dementia for 19
years in Omuta city, Japan. We analysed their sustainable co-creation activities
and extracted 3 key functions (pursuing regional philosophy, sharing the
philosophy with neighbours and co-creating activities with the neighbours). This
paper proposes sustainable LL as regional management method to utilize those
3 key functions. To societally implement the sustainable LL based on Omuta’s
philosophy of person-centred (person-centred LL), Omuta future co-creation
centre was established in collaboration with Omuta local municipality. The centre
and Omuta local municipality tackle comprehensive regional management
through sustainable LL activities.

Keywords: Philosophy of person-centred, Living Lab, regional management, co-

creation, sustainability

312
1 Introduction
Digital technologies like AI, IoT and social robots are rapidly maturing and they
are expected to achieve innovation (D'Emidio, 2015). About society, quality of life
or well-being topics are receiving attention within Sustainable Development
Goals (SDGs) (United Nations, 2015). Against this backdrop, we have to rethink
the Socio-technical system (STS) (Trist, 1951) which refers to the tight fusion of
technical systems and social systems where it becomes impossible to design and
operate separately. For reestablishing a clear relationship between technology
and society, LLs (ENoLL, 2016, Leminen, 2015) offer the possibility of
establishing a partnership of equality between citizens, companies, and local
municipality throughout the entire participatory design process.

A systematic review (Hossain, 2019) of 114 academic papers on LLs indicates

that one of the big issues is “sustainability”. It says that despite advancements in
collaborative innovation for smart cities, few studies address sustainable
development.

Our research focuses on creative dementia care activities on dementia in Omuta

city, Japan. Because Omuta has a public-private organization whose member
created cutting-edge activities collaborating with citizens, the local municipality,
and companies in 19 years over normal care activities. That’s why Omuta is
called holly cite of dementia in Japan.

This paper systematically analyzes their activities and extracted 3 key functions
which contribute sustainable cutting-edge activities.

2 Analysis of Welfare Activities in Omuta

Japanese ministry’s report (Japanese ministry, 2018) says problems of
sustainable operation of reginal management organization are following; lack of
members, lack of activities for regional problem, less relationship with local
municipality and less budget.

With reference to this report, this chapter introduces key viewpoints for
sustainable co-creation activities. Reginal philosophy is added on those topics,
because all activities and plans are based on Omuta’s philosophy of person-
centred case and members regard the philosophy is the most important attitude
and mind-set.

2.1 Pursuing Reginal Philosophy

Problem: If there is no core philosophy, the actions of people in various situations
become unfocused so that of activity sustainability is lost. That’s why most
organizations and companies create their own mantras called vision, mission, or
question. Moreover, some of them change the vison frequently to encourage
independence of individuals and/or organizations.

Function: To pursue issues for the age to come, it is important that the
philosophy provides a way for citizens and concerned individuals to establish their
thoughts or culture based on regional history and to deepen and broaden them.

313
In the case of Omuta, they have, over the last 19 years of person-centred care
(Kitwood, 1997), established the philosophy of person-centred activities.

The philosophy of Omuta welfare started in 1997 based on the methodology of

person-centred care (Nakajima, 2009). Over 20 years, Omuta welfare practices
were refined and deepened which has clarified the philosophy; “person-centred”.
Person-centred is a concept that captures the attributes of human existence such
as "inconsequential", "irrational" and “limitless capability (Sen, 1992)”. Based on
the assumption that a human's life is built upon social connections developed
through knowing, meeting, linking etc., the good life consists of a limitless variety
of capabilities elicited by psychological environment (problem consciousness,
curiosity or etc.) and social connection. (Figure 1)

Figure 1. Conceptual diagram of person-centred

Example from Omuta: Here is a story of two women. One senior woman with
dementia and the other who had mental illness, both entered hospital at the same
time. Both had no family to support them, so the Japanese welfare system would
normally send both to “closed” care facilities after medical treatment. But social
workers in Omuta felt this would not be the best life. To change this pattern, social
workers tried to know and follow each person’s personality, narrative and
humanity by talking with them, their family and their neighbors and also watching
their home and neighborhood. Common wishes are "wanna go home" and
"wanna meet my child". This is, however, often impossible because their home is
too old and dangerous. Sometimes they fail to recognize their children. To explore
better care solutions, the social worker developed the new approach of house
sharing by the two of them (Figure 2). This example illustrates the person-centred
philosophy in action.

314
 Figure 2. Omuta practice of house sharing

2.2 Sharing the Philosophy with Neighbors

Problem: It is impossible to maintain activities sustainably with few members. To
achieve long term success, it is necessary to get new members shared person-
centred care philosophy.

Function: Practical Leaning provides opportunities for welfare professionals,

citizen, local municipality staff and company members to learn and deepen their
understanding of life based on the practical person-centred philosophy.

Example from Omuta: In Omuta, the above-mentioned public-private council

provides dementia coordinator training program. The program is designed to
create experts that understand "person-centred care" deeply, support the dignity
of dementia suffers in daily life and promote community renovation focusing on
the suffers and their families.

2.3 Co-creating Activities with the Neighbors

Problem: Before yielding activities for regional problem, it is important to foster
close relationships which push forward members to co-create something. If there
is no such relationship, co-creation will fail or fake co-creation will take place.

Another problem is the cost of fostering such relationships. If a company tries to

form deep relationships; it is emphasizing activities that are not directly related to
earning money. The company finds it easy to stop the volunteer or CSR
(corporate social responsibility) activities unless their commercial activities are
strongly connected to developing the relationship.

Function: To help close relationship, social workers (SWs) coordinate citizen,

companies and local municipality. In daily task, SWs have opportunities to talk
with people experiencing troubles and to network with citizen, companies and
local municipality to support the people who have troubles. SWs’ activities are
already co-creation and also new partnerships are yield through their activities.

315
This can be said co-creation in daily life which is one of the ideal styles of LL to
provide a win-win opportunity for citizens and local municipalities / companies. In
this approach, partnership means a relationship to co-create better life and social
environment of citizen, not just flat relationship based on workshops or interviews.
In some cases, user’s role is simply to live as is and to discuss with company staff
about usual activities or reminiscences. Through this communication, company
staffs refocus on way of life based on the person-centred philosophy. In other
cases, user might talk about his/her inner thoughts only to special people such
as family, friends or favored welfare professionals. Collaboration with such people
is important in deepening person-centred partnerships.

Example from Omuta: Local municipalities are required to establish a total

welfare inquiry counter named the local integrated support center. At the support
center, welfare professionals hear citizen’s daily life problems and support them
or connect them to the resources required.

Based on this relationship, they co-create new values with citizens and other
stakeholders. One example is the collaboration between a car dealer and a day-
care facility. One day-care facility was attempting to help a senior user but he
lacked motivation to undertake rehabilitation, he did, however, love cars. He also
knew that a local car dealer was suffering a staff shortage. Though discussion
among the parties, the senior user rehabilitated himself by washing cars at the
car dealer with a helper from the day-care facility (Figure 3).

Figure 3. Omuta practice of car washing for rehabilitation on article from a newspaper

316
2.4 Relationship with local municipality
Example from Omuta: In 2000, a public-private organization named Long-term
care service provider council was jointly established by Omuta city and private
long-term care service providers in Omuta. Activities collaborating with citizens,
the welfare department of Omuta city and those private providers were promoted.
Such formation was key point to pursuit activities based on philosophy.

2.5 Financing
Example from Omuta: The private-public organization is financed by
membership fees paid by private service providers. Omuta city pays outsourcing
expenses to a private-public organization as operation fees of activities. The
dementia coordinator training is available at cost to other city’s care
professionals. Omuta city and this organization also secure some funds from the
MHLW for national model projects that explore future care systems and the social
impact bond schemes.

3 Model for Sustainable Activities

Omuta’s cutting-edge co-creation activities have been continued for over 19
years and we found they have a good mechanism for ensuring sustainability. By
analyzing the previous chapter, we develop a model for sustainable activities with
3 key functions, see Figure 4.

The three key functions for sustainable activities are interlinked. Pursuing
regional philosophies yields educational concepts for value creation. Sharing
philosophies increase the number of neighbors who will join co-creation activities.
Then co-creation activities provide opportunities to educate the neighbors in
practical matters. The educational opportunities ensure that neighbors can
rethink the attitudes induced by uncomfortable feelings.

Moreover, relationship with local municipality and financing will be considered

when concreate organizations are established.

Figure 4. Model for sustainable activities

317
This model was made from analysis of care activities but is useful for operation
of co-creation activities like LLs. We expect this model will be very effective in-
service design, for example making products, service development, policy
making and community design, and those activities can restructure society based
on each philosophy and resolve future social issues.

4 Sustainable Person-Centred Living Lab

To collaborate in not only welfare field but also industry field, like healthcare,
education, transportation or housing services, among local municipal entities,
citizen communities, and private companies, Omuta future co-creation center
was established as a private company. The center and Omuta city aim to regional
management utilizing the model described above based on philosophy of person-
centred. We call it the person-centred LL.

Figure 5. Formation of person-centred Living Lab

Formation of the person-centred LL is shown in Figure 5. One of the key players

is intermediator who is a challenge owner from regional point of view bridging
company and region and a facilitator. The welfare professional is also key player
as he/she has close and long-term relationship with citizens and knows the
citizens’ life histories or motivation in life.

4.1 Person-centred cultural meeting at 1st layer

The first function is to pursuit the philosophy that underlays all activities. In the
VUCA world (Stiehm and Townsend, 2002), the design process starts with
questions. We have to think about what kind of society we want to make or what
kind of life we hope for. Omuta’s philosophy; person-centred has been discussed
in the welfare field, and it is hoped that it will be deepened through discussion
with various types of professions; humanities, sociology, engineering and so on.

Person-centred design training program at 2nd layer

The second function is to share the person-centred philosophy with neighbors
and members, like local children and adults, municipality staffs and staffs of

318
companies. Omuta was the first to start dementia coordinator training in the
welfare field, and last year we created a training program for the industry field as
a LL education program under the Ministry of Economy, Trade and Industry
(METI). Figure 6 shows the mind-set and process of person-centred design.
These activities aim to create new services based on the person-centred
philosophy.

Figure 6. Mind-set and process of person-centred design

4.2 Co-creation project at 3rd layer

The 3rd function is to lay out situations for empowerment of activities with the
neighbors. Co-creation opportunities should not be temporary and exclusive like
workshop events restricted to certain groups of people. To generate fundamental
partnership, in some cases official formation helps people to co-create and in
other cases unofficial relationship helps people to co-create. Some citizens like
dementia suffers notice their problems in daily life and share them with welfare
professionals who have close relationships. We think this is a kind of co-creation.
Through such daily dementia care, the person-centred LL accumulates the
problems of citizens and regional social problems.

The facilitator uses the close relationship with citizens and accumulated problems
to coordinate with companies depending on their purpose and resources. This
last activity is similar to the traditional LL.

4.3 Relationship with local municipality

The Center has partnership with local municipality. For examples, both of them
collaborate for SDGs and were selected as SDGs future city in Japanese Cabinet
office in 2019. And also they tackle clearance gap social issues, like habitation,
transportation or education, which are beyond the jurisdiction of each department
of the local municipality. The center member and staffs of local municipality build
a team and iterate try and error processes.

4.4 Financing
About financing, the person-centred LL elucidates and informs new values based
on the person-centred approach to outside stakeholders, and also obtains
financial and human support from outside. For example, companies will pay
Omuta future co-creation center to assist in the creation of new services in
person-centred LL. Municipality staff will pay for training in the Omuta future co-
creation center to learn mind-set and process of the person-centred approach.
Inside of the region, local municipalities can cut expenditures needed to resolve

319
social issues like social impact bond (Nikki, 2014). In the case of Omuta city, they
trying to not only cut the cost of caring but also to maintain the quality of caring
by empowering regional actors who provide informal care services.

5 Summary
This paper presented a model for sustainable activities analyzed of Omuta
welfare activities and a person-centred LL applied by the model to LL. Omuta
future co-creation center was launched to operate activities of person-centred LL
and to realize open innovation with light foot-work over sectionalism. The center
has already started LL project with citizens, companies and the municipality and
will develop new services series based on person-centred philosophy. In future
those person-centred services, policies and communities are created in Omuta
city, our ideal concept of person-centred City will be realized in which people live
as he/she is, to elicit of their capabilities within their social connections.

References
Bergvall-Kareborn, B., et al. (2009). Concept design with a living lab approach,
Hawaii International Conference on System Sciences.
D'Emidio, T., et al. (2015). Service innovation in a digital world. McKinsey
Quarterly.
European Network of Living Labs, (2016). Introducing ENoLL and its Living Lab
community.
Folstad, A. (2008). Living Labs for Innovation and Development of Information
and Communication Technology, The Electronic J. Virtual Org. and
Networks, 10.
Hossain, M. et al. (2019). A Systematic Review of Living Lab Literature. Journal
of Cleaner Production.
Kitwood, T., (1997). Dementia Reconsidered, Open Univ. Press.
Lasher, D. R, et al. (1991). USAA-IBM partnerships in Information technology,
MIS Quarterly, 15(4).
Leminen, S. (2015). What are living labs?, Tech. Innov. Mgmt. Rev., 5(9).
Nakajima, T.,(2009) Challenge of Omuta city: Toward the city in which people
with dementia live the way he/she is, National community life support
center.
Nikki, B., (2014), Social Impact Bonds in Emerging and Developing Countries,
AV Akademikerverla.
Trist, E. L. et al. (1951) Some social and psychological consequences of the
Longwall Method of coal-getting. Human Relations, 4(1).
United Nations, (2015). Transforming our world: the 2030 Agenda for
Sustainable Development, General Assembly.
Sen, A., (1992). Inequality Reexamined, Harvard Univ. Press.

320
Stiehm, J and Townsend, N (2002). The U.S. Army War College: Military
Education in a Democracy. Temple Univ. Press.
Ministry of Internal Affairs and Communications (2018). Investigation research
report on formation of Regional operation organization and sustainable
operation (in Japanese).

321
 The value of participatory approaches in
developing energy services
Joelle Mastelic*1 and Stéphane Genoud1

*Corresponding author
1 Switzerland University of Applied Sciences Western Switzerland, Switzerland

Category: Innovation Paper

Abstract
How can stakeholders be involved in the development of energy services to
increase energy efficiency? What is the optimal process for engagement? This
is what has been tested in the EnergyLiving Lab, which focuses on energy
efficiency and the development of renewable energy. This innovation paper is
based on several applied research projects. Its objective is to disseminate
research results of a PhD thesis (Mastelic, 2019). The advantages of the Living
Lab method for developing energy services are highlighted. The main steps of
the Living Lab Integrative Process are summarised in a checklist for
professionals and includes: (1) Selection of a practice, (2) Identification of
barriers, (3) Integration of stakeholders, (4) Development of a pilot, (5)
Measurement of results, Communication and dissemination. In conclusion, this
vulgarisation article facilitates the transition from the local to the global scale by
encouraging the development of Living Lab mode initiatives in the energy sector.

Keywords: Living Labs, Energy Services, Social Marketing

322
1 The context: Energy services and stakeholders
Although the Living Lab method has been in use for a decade, it has not been
widely explored in the field of energy. Some papers describe the use of the
method for energy savings, such as in Krogstie et al. (2013). The Energy Living
Lab presented in this innovation paper has been developing projects since 2014
specifically in the field of energy efficiency and the promotion of renewable
energies. The key question is to know what value co-design methods bring to a
field that is still very much driven by technological innovations.

This article proposes approaches for integrating stakeholders in the development

of energy services using the Living Lab method.

1.1 What is an energy service?

A 2017 study by Michael James Fell indicates that only 0.5% of the 185 scientific
articles analyzed by two major energy journals mention “energy service,” and only
10% of these studies define what an energy service is. Clarification has,
therefore, been required, and Fell proposes the following definition: “Energy
services are those functions performed with energy that are means of obtaining
or facilitating the desired services or end states.” Lighting, for example, is an
energy service that can be produced using different primary energy sources and
leads to a desired end state: illumination in the home or office. How does this
service create value? According to marketing researchers Vargo and Lusch,
value creation occurs when the service is consumed by the customer (2004). A
watt lost during transport does not create any value. In marketing, we are
considering the consumer as a co-creator of value. It is, therefore, always a value
perceived by the consumer.

It is sometimes thought that consumers are not rational, for example, when they
leave windows open in winter. From their point of view, their actions are rational
because otherwise they would not behave in this way. For a specialist, it is often
difficult to put oneself in the consumer’s shoes and understand these types of
practices that are detrimental to energy efficiency (EE) and the environment.

1.2 The intangibility of energy services

This perceived value causes problems for energy service providers. Indeed, the
value of services is often not perceived. Users expect to benefit from services
such as heating, lighting, and ventilation. They realize the value of such services
only when they experience poor quality or an interruption in service. This is what
was measured in our previous project in a sustainable neighborhood (Mastelic et
al., 2016). When the heating system fails in the middle of winter, the value of the
energy service is realized by its absence. How, then, can we raise awareness
among users of the value of energy services when everything is
working well? It is a challenge to make energy tangible, to make it visible.
Today, we are witnessing a decoupling between primary energy and “the desired
end state”. During our grandparents’ time, people were well aware of the primary
energy needed because they had to feed the fire to warm their homes. They had
to cut wood, carry it, dry it, and manage the supply to the furnace or woodstove
so as not to let the temperature drop. There was a direct and tangible link between
primary energy and the desired end state. And what our grandparents

323
experienced 100 years ago, our African neighbors still experience every day. In
Europe today, it is mostly an automatic system that does the work for us, and as
a result, most of us have lost this link with primary energy.

1.3 Raising user awareness

Energy planners then consider solutions to make the population aware of energy
savings. Appealing to users’ common sense, they think they can solve the
problem by carrying out educational actions. Unfortunately, for the past two
decades, social marketing scholars haveindicated that, although necessary,
awareness and education are not enough to get users to take the desired actions
(McKenzie-Mohr, 2000). It is also difficult to produce lasting effects over time, and
this type of intervention must be repeated. Admittedly, mentalities have changed,
and today, a larger segment of the population declares that it wishes to act in
favor of the environment. Attention must be paid to the potential gap between
attitudes, intentions, and actions (Kollmuss and Agyeman, 2002). In a sustainable
district that we studied and according to the results of the tree of correlations
between the satisfaction of living in the district and the various services provided,
energy was not among the priorities; they were looking first for satisfactory social
relations, an attractive place to live, and a location (Mastelic et al., 2016).

1.4 Limited perceived control

The power to act personally is often underestimated by stakeholders. Most
people think there are other, more competent people who will act to increase EE.
The problem is not being addressed, as demonstrated by our study on the
development of an energy management system (Mastelic et al., 2017). If no EE
targets are set by management, employees will not take the ownership of the
challenge and leave it to the management team. If employees (and especially
custodians) are not measured annually on their energy performance, then why
should they act? In addition, stakeholders often do not have much knowledge
about energy; they have low “energy literacy.” The notion of kW/h is highly
abstract for them, and they don’t know how much it can represent. Most of the
people interviewed in past studies do not know how much they spend on energy,
and even if they do, the percentage is a relatively low one based on their total
budget (about the price of a cup of coffee a day for electricity in a household) and,
thus, provides little incentive for EE actions. Subsidies can, to some extent,
provide an incentive for action.

1.5 Automation of energy services

EE specialists, therefore, imagine increasingly complex technical systems that
can solve problems. These include automatic lighting with presence sensors,
regulation of temperature control in buildings according to theoretical occupancy
data, and the use of windows that prevent opening. They use technical and
economic models to implement them and sometimes forget that these systems
will interact with users. Indeed, as proposed by Geels, they are socio- technical
systems composed of rules, technical artefacts, and human actors (2004). User
are sometimes forgotten or relegated to the end of the process to test the final
service, thought of in a patriarchal way as: “We know what is good for you”. Users
often do not oppose active resistance in this phase as they have a low technical
knowledge and tend to delegate ownership of the challenge to experts.

324
The problem is that, if the technical solution and/or the rules put into place
(system regulation, laws...) do not meet their requirements, users will find
inventive ways to try to circumvent the system (bypass use). When specialists
encounter blockages, researchers in social sciences are often asked for
interventions that promote “social acceptance,” a kind of magic wand that would
be used to accept technical solutions which do not work optimally. Unfortunately,
it is often too late to change the technical artefacts, and only small adjustments
can be proposed. It is, indeed, at the start that action should be taken.

2 Ways to promote stakeholder engagement

How to engage stakeholders in a co-design process to co-develop energy
services? This is what has been tested in the phd thesis (Mastelic, 2019). The
following section gives recommendations for professionals based on four years
of research.

2.1 Transformative research and quasi-experimentation

When we want to act on energy services, we act on a complex system. In such a
system, the relationships between variables are not linear, and the preferred way
to test whether or not an intervention has an effect is through experimentation
(Kurtz and Snowden, 2003). This is referred to as quasi-experimentation because
we do not have a random sample of the population.
In addition, we work in the field, in situ, and therefore, we cannot control all the
variables in the system. The effects of certain variables, such as the influence of
weather on energy consumption, must therefore be considered.
We are attempting to move from one energy system to another, so we want to
change the reality we observe. This is not considered acceptable in all scientific
disciplines. We are in a constructivist epistemological paradigm: the researcher
is not neutral; we conduct action research that is called “transformative” because
it transforms the observed reality (Schneidewind, 2016). The author tested the
Living Lab approach to co-design energy services with stakeholders.

2.1 What is a Living Lab?

The Living Lab (LL) approach is relatively new. It has been in place for about ten
years at the European level, but few experiments exist in the field of energy
services. Research on the “Living Lab” phenomenon is, therefore, in its infancy.
Definitions abound, but it remains difficult to capture the complexity of the
phenomenon. We propose our own definition in the thesis:

“A Living Lab is an innovation intermediary, which orchestrates an

ecosystem of actors in a specific region. Its goal is to co-design products
and services, on an iterative way, with key stakeholders in a public private
people partnership and in a real-life setting. One of the outcomes of this co-
design process is the co-creation of social value (benefit). To achieve its
objectives, the Living Lab mobilises existing innovation tools and methods
or develop new ones.” (Mastelic, 2019).

325
What differentiates the Living Lab from other participatory methods is the
combination of several factors, listed in the definition and detailed below:
1) An ecosystem of stakeholders: This laboratory emulates a partnership
between public authorities, companies, citizens, and academics. The
Living Lab manager acts as a catalyst to build a common vision, provide
methods, coordinate experiments, and measure results. To select the key
stakeholders, we use the power/interest matrix, detailed in another article
(Mastelic, 2017).
2) Co-designing solutions: The prefix co- means “with.” We do not develop
solutions for users but rather with users.
3) An in situ environment: Research does not take place in a laboratory but
in the place where energy is consumed or produced; it adapts to different
contexts.
4) A societal-improvement objective: A strategy for individual well-being is
not developed but rather societal well-being is the aim.

2.3 Living Lab for energy services

The Energy Living Lab was created in 2014 to support the development of field
interventions and to help achieve the objectives of the Swiss Federal Council’s
Strategy 2050. It operates in two main sectors: increasing energy efficiency
(particularly in the built environment) and the diffusion of renewable energies.
After an initial test in the Chablais region, the Living Lab increased its
interventions in a portfolio of projects supported by public and private funds (work
on the energy performance gap, the dissemination of renewable energies in a
region, the dissemination of photovoltaics in French-speaking Switzerland, the
deployment of district heating systems, etc.). Some of these interventions will
illustrate the purpose and feedback below.

2.4 The process developed by the Living Lab

The process is based on “Community-Based Social Marketing” as proposed by
McKenzie- Mohr (2000) and integrates knowledge of social marketing and social
psychology. Marketing gives particular importance to the choice of targets for
interventions. You can’t talk to “everyone” because it’s the most effective way to
avoid addressing anyone. Social marketing, then, focuses on barriers to pro-
environmental or pro-social action. Understanding barriers is key to
understanding why EE interventions fail. Measuring results is also central in
marketing because, in order to achieve objectives, it is necessary to know how to
determine performance indicators and how to measure them. As part of the
research, we combined social marketing and the Living Lab approach; this
involves analysing and integrating key stakeholders from the beginning of the
value chain. Non-specialist actors work alongside experts to co-design a solution
(product/service/action plan). It is also important to go into the field quickly to test
the proposed solution under development and then return to development
iteratively (agile methods). More information on social marketing can be found in
the thesis (Mastelic, 2019):
1) Selecting a practice. An analysis of existing data is conducted to
determine which practice(s) we want to act on. In the sustainable district
studied, for example, the focus was on heating and mobility. We attempt
to take a step back and choose according to the context and data rather

326
 than choosing a field of use a priori. The PESTEL model (political,
economic, social, technological, ecological, legal) can also be used to
understand the complexity of the usage context.

2) Integrating stakeholders. Stakeholders are listed, and then a matrix is

used to classify them, in this case, the power/interest matrix (Eden &
Ackermann 1998, in Bryson, 2004). We look at who has the power and
who has the interest in changing the service, for example, in the case of
heating. If citizens are not the ones with the power to make an impact, why
focus on them? (Mastelic et al, 2017). We place our research hypotheses
a priori in this matrix because we do not know a priori who has the power
and interest. Efforts will also be made to integrate four types of
stakeholders, the Quadruple Helix model (Carayannis & Campbell, 2012):
academics, public authorities, companies, and citizens/users of the energy
service.

3) Identifying the Barriers. Qualitative interviews will then be conducted

with each of the key stakeholder groups based on the matrix, in order to
better understand their perceptions regarding the level of power and
interest, their motivations, and the barriers. In one of the fields studied,
even the director of the school thought he did not have the power to
change the situation. There was no energy-saving objective. Who does
have the power if the director doesn’t think he holds it? (Mastelic et al.,
2017). There is often a lack of ownership of the challenge. In LLs, the first
step is to encourage stakeholders to take ownership of the challenge. This
is achieved, for example, by asking the director in front of the stakeholder
assembly: “What is your goal for energy efficiency?” A public and voluntary
commitment is necessary. This allows better ownership of the project and
future results (Cialdini, 2001). Once we have completed this stakeholders’
analysis, we compare our research hypotheses with what stakeholders
have mentioned. We redraft a power/interest matrix in relation to their
perspectives and see if there are any differences. We also analyse all the
other barriers to action such as, for example, lack of cash flow, low
motivation, and technical ignorance.

4) Co-designing the solution. We then collaborate with stakeholders to co-

design a common vision. We have had cases where companies have seen
municipalities as actors who are there only to place obstacles in their way.
Conversely, sometimes municipalities see companies as interested only
in their profit margins. If we do not offer this space for dialogue between
actors to correct certain biases, they cannot co-develop a common vision.
Qualitative interviews are always conducted first to gain a clear
understanding of individual barriers to change and then to moderate this
type of multi-stakeholder process that involves building trust between
stakeholders, we have developed a model to build trust among
stakeholders, detailed in anther article (Dupont et al., 2018). Simply
bringing people to the table and providing an environment that fosters trust
and the development of a common vocabulary often helps to move the

327
 process forward. Many tools exist to co-design energy services and are
presented in other works by the same authors.

5) Piloting an experiment. Primary importance is given to testing co-

designed solutions in a real-world situation with authentic users. Agile
methods facilitate regular trips back and forth between the experimental
field and theory. Today, we see the emergence of laboratories in which
researchers live and test new products and services. Although these
experiences are similar to a real-world environment, they do not convey
the actual and authentic conditions of use, including users’ knowledge.
Thus, barriers may be overlooked related to technology adoption, culture,
lack of time, and a range of social factors that will not be reflected in the
test environment.

6) Evaluating performance. A measurement and verification plan is

proposed before the pilot. EE standards can be referred to, for example
IPMVP14. A large volume of data is now available, such as energy
consumption from smart meters. To interpret the results, it is often useful
to collect sociodemographic data using surveys and to cross-reference
them with consumption data to provide analyses by consumers’ clusters
(Cimmino et al, 2016).

3 Impacts of Living Labs on energy services

The Living Lab approach, tested in several research projects or
mandates, is beneficial from many perspectives, as presented below.

3.1 Reducing barriers to change

The approach provides a better understanding of barriers to action from different
perspectives. This focus on barriers is similar to the social marketing methods. A
new and “naive” look at the energy challenge is taken. Indeed, barriers are not
always found where we expect to find them. An example is the development of
district heating system (DHS) in an article from Previdoli and her colleagues
(2015). The contracting authority thought that prospects could raise economic
barriers, as the installation of the network has high initial costs. However, after
stakeholder analyses, a different and unexpected barrier emerged: resistance to
change from the environmental service. For years, the service had required the
switch from oil to gas. With the arrival of the DHS, a new argument had to be
developed to convince employees in contact with prospective users and then the
prospects themselves. Without involving all key stakeholders and focusing only
on users, this important barrier to deployment could have been missed.

3.2 Development of a common vision

An ecosystem of actors is integrated into the reflection from the very beginning
of the projects. The first step is to network the partners and define a common
vision. Interactions and mediation promote trust-building and knowledge-sharing

14 International Performance Measurement and Verification Protocol

328
(Dupont et al, 2018). The LL method makes it possible to build bridges between
actors and between disciplines.

An example is the municipality of Saint-Martin. The president of the village tested

the LL method to develop renewable energies in the village. School children,
aged 12 years old, teamed up with engineering and economics students from the
university to propose a renewable energy development plan for the municipality.
The best project was selected by experts and presented by the young people to
the primary assembly. The latter accepted the budget for the preliminary studies
for the deployment of the plan in the village by local companies.

3.3. Increased perceived control by change agents

Stakeholders sometimes feel that they cannot act. The example of the
hummingbird, cited by Pierre Rabhi, comes to mind. The little bird is the only
animal who attempts to extinguish a raging forest fire by filling his beak with water
and spraying it. The other animals tell him not to bother as he won’t make a
difference, and the tiny bird answers, “Maybe so, but I’m going to try.” In the
energy field, stakeholders also need examples, support, and reassurance to give
them confidence that they can act and have an impact. Our project in LL mode
tested the TupperWatt evenings, workshops that bring neighbours together
around the theme of EE. An expert attend and presents technical solutions. The
participants can freely express the obstacles they foresee, and the exchange of
experiences is rich in learning. EE materials are then offered for sale, and
participants are given a “good practice” guide and a small gift. In this way, they
become aware that they can act at their own level.

3.4 Changing the role of users

Users become active co-designers of the energy services and even sometimes
prosumers. Users have often lost their link with primary energy; they have
become passive consumers of automated services. As part of the LL’s activities,
we are trying to give them back an active role as co-designers alongside
specialists. For example, we have developed a serious game: the poker design
(initially proposed by Cité du Design from St-Etienne). After conducting qualitative
interviews with stakeholders, personas, a kind of stereotype of the system’s
actors, were created. For example, there is the energy distributor, the elderly
person in subsidized housing, the employee of the real estate agency, etc. We
then developed three types of cards: (1) persona, (2) actions, (3) uses.
Stakeholders were able to combine the cards to develop an EE plan for their
neighbourhood. For example, the “energy distributor” card is chosen, which
“encourages” the “inhabitants” to “install a HYDRAO shower head to save water.”
The persona allows users to take a step back from their own practices and to
avoid guilt. The game stimulates discussions and the co-design of solutions to be
implemented in the neighbourhood.

3.5 Increasing the social acceptance of systems

When stakeholders come up with ideas to improve energy services, they express
their own needs. Co-designed solutions are closer to these needs. They are all
the more easily adopted by the actors, even if they did not participate in the co-
design process. In the example of Saint- Martin cited above, the whole village is

329
behind the projects of mini hydraulic turbines in drinking water systems, joint
tendering for solar panels on public and private roofs, and DHS in the village. The
assembly unanimously voted the budgets to implement the plan developed by
the children and students, and local companies are working on it.

3.6 Reconnection to primary energy

When working with participatory methods, actors re-examine questions that they
had omitted, such as the source of primary energy (“Where does the energy I put
in my car or in my heating system come from, and who benefits from the money
I spend to purchase it?”). Several illustrations of this phenomenon can be cited.
During the qualitative interviews conducted to understand the motivations and
barriers of the actors to the deployment of a DHS, many understood the need to
valorize the energy from waste rather than importing fuel oil (article by the same
author). The mental images communicated are often more meaningful than the
words: “Imagine the distance that fuel oil must travel in tankers and then by road
to get here.” The argument of local energy speaks to many and favors the
circular economy: the forester who feeds the DHS with wood waste from the local
forest reinvests his salary locally.

3.7 Piloting the co-designed service

The quasi-experiments used in the LL method allow the solutions to be tested in
situ to see the results of the system. This allows quick round-trips between the
field and R&D to adapt the energy service. For example, one idea that emerged
several times in the co-design workshops was to develop a large red button for
households to turn off all sources of unnecessary electricity consumption when
leaving the house. This is a simple idea based on a common need and could be
installed by default in senior citizens’ housing, for example. Its implementation is
more complex, however: Do we really want to cut everything off? This requires
prior testing in a real-world setting, which is the case in the studied sustainable
neighbourhood.

4 Conclusions
How can we achieve the energy transition while engaging key stakeholders? The
share of household energy consumption is very high, accounting for about half of
the total consumption. It is, therefore, illusory to imagine an energy transition
without citizen. It is also completely unrealistic to imagine that, by giving them
information only, this could be enough to get household users to drastically
change their consumption practices. Although they represent about half of the
energy consumption, these consumptions are highly diffuse, with very varied
uses. It is, therefore, difficult to establish a cost-effective economic process to
help them reduce their consumption because, apart from communication
solutions, which have demonstrated their limits, the time required to help them
achieve an energy transition within the expected time frame generates significant
consulting and support costs. This transition will have to be rapid and will be very
difficult to achieve without effective participatory approaches. LL’s approaches
are perfectly aligned with this spirit, as co-designed solutions are more easily
accepted. Many indicators demonstrate the awareness of the population—and
young people, in particular—of the need to reduce our impacts on the

330
environment. This is very encouraging and adds one more reason to help
redefine market rules. We must accelerate the energy transition, for example, in
the field of construction where, with a rate of 1% per year of renovation of existing
buildings, it would take us 100 years to refurbish our real estate assets. We
obviously don’t have that time available. Today, a large part of the activities
conducted by economic actors i will have to be refocused on actions around the
energy transition. This is also true for universities and the institutions that finance
them. They should contribute with a managerial impact to the reduction of CO2
emissions and the production of renewable energies, not only through theoretical
contributions. There are currently many solutions on the market that are energy
efficient, both technically and financially. The deployment of these solutions
should be supported with as many resources as fundamental research on
technological solutions. Clearly, technology must continue to evolve, but in
today’s society we need much more action around stakeholders’ participation,
who will ultimately decide whether or not to join the collective effort. This is
certainly the main benefit of “action research” to promote the energy transition
that really contributes, here and now, to increasing the production of renewable
energies and reducing CO2 emissions.

5 Checklist for getting started with the “Living Lab Integrative

Process”
The Living Lab Integrative Process is explained step by step in the following
checklist. The aim is to transmit the standardised method to professionals and
researchers wanting to experiment it.

Figure 1. Living Lab Integrative Process

1) Selecting a practice
Study the available data on your energy service. What are the practices of the
actors that have a strong impact (positive or negative) on the efficiency of your
energy service? Select between 1 and 3 practices (e.g., taking a bath instead of
a shower, leaving windows open, changing the temperature set point, allocating
charges to the residential surface...). Try defining the “roots” of the problem and
not only the symptoms.

2) Integrating stakeholders
Make a list of stakeholders who have influence over your energy service. Try to
place them on the power/interest matrix (your own assumptions): (e.g., the
commune’s energy delegate, the building janitor, the end users, the financiers of
the solution, the energy distributor...). (Eden & Ackermann 1998, in Bryson,
2004).

331
 Subjects Players

HIGH
Level of Interest Crowd Context Setters

LOW
LOW HIGH
Level of Power

Figure 2. Power-Interest Matrix, adapted from Eden & Ackermann in Bryson 2004.

3) Identifying the barriers

Interview the key players individually (box: Keep satisfied, manage closely, and
Keep informed). Are your assumptions true? What are the barriers and levers of
action of these actors toward efficiency?

4) Co-designing the solution

Then bring together the key players (e.g.: workshops, world cafés, BarCamps).
Ensure that you invite the four types of actors: companies, public authorities,
citizens/users, academics. Work toward developing a common vision and shared
objectives for your energy service. Co-develop the solution WITH users, not FOR
users (using design service, design thinking, crowdsourcing, etc.). Adapt your
vocabulary to an audience with a low level of energy knowledge.

332
5) Piloting an experiment
Test the co-designed solution in the field and not in the offices! Collect feedback
to improve your energy service (interviews, ethnographies). Perform as many
iterations as necessary without waiting for a final prototype (agile methods).

6) Evaluating performance
Establish the measurement and verification plan before the pilot (e.g., IPMVP)
and evaluate the results regularly. Triangulate the data to verify your conclusions
(qualitative/quantitative, simulation/real consumption data etc.).

7) Communicating results and replication

Communicate the results of your project to all stakeholders and celebrate
success with them (media communication, end-of-project event, etc.). Share your
success to allow others to replicate it (open innovation, open science).

References
Bryson, J. M. (2004). What to do when Stakeholders matter. Public Management
Review, 6(1), 21-53. https://doi.org/10.1080/14719030410001675722.
Carayannis, E. G., Campbell, D. F. J. (2012), Mode 3 Knowledge Production in
Quadruple Helix Innovation Systems. New York, NY: Springer New York.
doi: 10.1007/978-1- 4614-2062-0_1.
Cialdini, R. (2001), Harnessing the Science of Persuasion, Harvard Business
Review, 79(9), 72-81.
Cimmino, F., Mastelic, J., & Genoud, S. (2016, September). Multi-method
approach to compare the socio-demographic typology of residents and
clusters of electricity load curves in a Swiss sustainable neighbourhood.
In 2016 ENTRENOVA Conference Proceedings.
Dupont, L., Mastelic, J., Nyffeler, N., Latrille, S., Seuillet, E. (2018), Living Lab as
a Support to Trust for Co-creation of Value: Application to the Consumer
Energy Market, Journal of Innovation Economics and Management,
DeBoeck Superior.
Fell, M. J. (2017). Energy services: A conceptual review. Energy Research &
Social Science, 27, 129–140. https://doi.org/10.1016/j.erss.2017.02.010.
Geels, F.-W. (2004). From Sectoral Systems of Innovation to Socio-Technical
Systems: Insights about Dynamics and Change from Sociology and
Institutional Theory. Research Policy, 33(6), 897–920.
https://doi.org/10.1016/j.respol.2004.01.015.
Kollmuss, A., & Agyeman, J. (2002). Mind the gap: why do people act
environmentally and what are the barriers to pro-environmental behavior?
Environmental education research, 8(3), 239-260.
https://doi.org/10.1080/13504620220145401.
Krogstie, J., Stålbrøst, A., Holst, M., Gudmundsdottir, A., Olesen, A., Braskus, L.,
... & Kulseng, L. (2013). Using a Living Lab methodology for developing
energy savings solutions.

333
Mastelic, J., (2019), Stakeholders’ engagement in the co-design of energy
conservation interventions: The case of the Energy Living Lab, doctoral
thesis, University of Lausanne.
Mastelic, J., Genoud, S., Cimmino, F.M., Previdoli, D., Fragnière, E., (2016),
Perceived value of energy efficiency technologies in a sustainable
neighborhood: an empirical enquiry from the Energy Living Lab,
Conference Proceedings, Open Living Lab Days 2016, Montreal.
Mastelic, J., Emery, L., Previdoli, D., Papilloud, L., Cimmino, F., & Genoud, S.
(2017). Energy management in a public building: A case study co-
designing the building energy management system. In 2017 International
Conference on Engineering, Technology and Innovation (ICE/ITMC), pp.
1517-1523, IEEE, DOI 10.1109/ICE.2017.8280062Rabhi, P. (2006), La
part du colibri. L’espèce humaine face à son devenir, Ed. MiKros.

334
335
Theoretical &
Methodological
Living Lab
Challenges
336
337
 Blockchain, a promising way for scaling up co-
creation of innovation from local to global
Eric Seulliet1

1 La Fabrique du Futur & Co., France

Category: Innovation Paper

Abstract
It is difficult to mobilize co-creating users over time: lack of motivation to
contribute, difficulty in capitalizing on contributions. The outcomes of traditional
co-creation processes are therefore often limited in scope. The use of blockchain
is one way to overcome these limitations and scale up. By allowing contributors
to be recognized or even remunerated, the blockchain produces a "nudge" effect
thanks to the climate of trust it creates. Thanks to the traceability and
capitalization of contributions, the blockchain also makes it possible to make the
most of them by promoting their recombinations. Finally, the blockchain allows
new approaches to intellectual property to be unleashed and new types of
organizations (DAO) based on sharing to emerge.

Keywords: Blockchain, Nudge, Trust, DAO, Collective Intelligence, Motivation,

Engagement

338
1 Introduction
The interest of co-creation is increasingly recognized in innovation approaches.
Co-creation makes it possible to optimize and enrich these approaches at all
stages of the innovation process.

At the ideation stage, co-creation makes it possible to produce more ideas and,
above all, it generates a biodiversity of creative ideas, these coming from a
plurality of actors. The observation of various situations and practices in real life
contexts, which are the very basis of living labs approaches, also makes it
possible to detect more emerging uses.

Then, when it comes to experimenting with creative approaches, the multiplicity

of actors allows faster iterations and generates more relevant results.

In the development phase of desirable and viable solutions, co-creation makes it

possible to build a constructive climate through the mobilization of the collective
intelligence of stakeholders.

Thus, co-creation is the intrinsic value of the Living Lab movement and is its
fundamental characteristic.

However, co-creation processes encounter a number of limitations that prevent

them from having a greater impact. To sum up, one could say that the challenges
of co-creation are both of a quantitative and qualitative nature.

This double glass ceiling that co-creation faces must absolutely be crossed if we
want the actions of living labs to be more recognized and that living labs move
from a sympathetic consideration to a real recognition. It is at this price that we
can hope to ensure that the results of co-creation have a real impact, much
broader than those often too limited in traditional communities of co-creators.

Our paper proposes to share, modestly, our insights on the issue based on our
own research, our experience, and our exchanges with professionals. Thus, our
conviction is that the blockchain can be a formidable lever to boost co-creation
and open innovation and boost it from an artisanal stage to a larger scale.

Our paper presents concrete examples and we also present a project in the
process of being launched as well as our thoughts on future directions for
blockchain in a context of co- creation.

2 The limits of co-creation

Co-creation, as mentioned above, faces two kinds of limitations:

Quantitative limitations
Co-creation approaches encounter a quantitative limitation as it is often materially
difficult to gather a large number of contributors.

339
The scope of the innovations produced also seems limited, since contributors are
generally recruited from a limited circle of users, who are often endogenous in
nature.

If we really wanted to scale up co-creation, it is necessary to mobilize many more

co-creators.

Qualitative limitations
A major problem in a co-creation process is to be able to sufficiently and
sustainably mobilize the participants in the process. Several factors can explain
these difficulties in mobilizing and involving co-creators.

There are above all psychological difficulties:

These difficulties are above all problems of envy, jealousy and competition
between individuals, which tend to lead to a natural tendency for everyone to
overvalue their contributions and to minimize those of others. Distrust, jealousy,
denigration that can combine with this lack of discernment lead to withholding of
information and reluctance to cooperate. The difficulty in measuring, and
therefore valuing, the various contributions and their impacts is another obstacle.

Another important factor to be considered is the issue of intellectual property.

Current legal solutions (patents, NDAs, etc.) do not offer a satisfactory answer,
particularly in the case of multiple contributors from different organizations,
particularly for large-scale projects or projects that address societal issues.
Participants in a co-creation process may be frustrated by having to abandon any
claims in this area and feel that they are deprived of the opportunity to value their
contributions. Therefore, this is undeniably a barrier to their motivation to get
involved. These situations can generate inertia and losses leading in the first
place to a lack of overall efficiency.

Human beings also need a minimum of personal recognition. They may be willing
to engage in a collective process, for example in an ideation sequence based on
brainstorming to collectively produce creative ideas, but their full engagement will
quickly be limited if they feel that their contributions are not being credited to them.
The saying "let's repay to Caesar what belongs to Caesar" is most valid in this
case.

Organizational and logistical difficulties:

How can co-creation approaches be optimized to make the most of them? How
can we best capitalize on each other's contributions in such a way as not to
"reinvent the wheel" but on the contrary to ensure that we build real avenues for
innovation? How to effectively sort contributions and prioritize them? How to
evaluate them? How to organize productive group meetings? What are the most
effective methods and means? What are the most appropriate environments and
contexts?

These are all questions that do not easily find answers in the usual co-creation
processes.

340
Nor does classical co-creation seem suitable for producing radical innovations
because it is based above all on everyday life experiences. Moreover, it is not
easy to prioritize and structure the insights collected because they are delivered
in a disorderly manner.
In practice, it is also complicated to recruit co-creators, it is difficult to motivate
them over time..... In addition, the whole process is rather slow because of the
necessary iterations that it requires. The process must also be organized and
animated, which requires human resources and tools.

Living labs, for which the issue of user involvement is particularly important, seem
to be increasingly aware of the need to find ways to take the co-creation of
innovation to a more advanced stage. It seems that the solutions can be
combined around two main axes:

The nudge axis:

In concrete terms, nudge consists in indirectly encouraging individuals (citizens,
clients, patients, etc.) to adopt new behaviors considered responsible and
beneficial for themselves and the community, by modifying the context of their
decision-making, without forcing them. This concept of nudge was popularized
by economist Richard H. Thaler, who received the Nobel Prize for Economics in
2017.

This concept of nudge obviously applies perfectly to the issue of the involvement
and motivation of co-creators. Two examples can be mentioned in this regard
from the latest OpenLivingLab Days in Geneva in 2018:
• A workshop entitled "Involving end-users: intrinsic or extrinsic rewarding"
clearly questioned what motivates participants in a co-creation group. The
outcome stated: “the discussion was interactive and very vivid. The biggest
motivation for end-users to participate is intrinsic (acknowledgment,
contribution, ownership, social contact), but extrinsic rewarding must be
taken into consideration because it helps to get a good mixture of
participants. A monetary incentive (voucher, cinematicket, freelunch) can
be an important trigger for some target groups”.

• Another workshop entitled "Technology, Goal setting and behavioral

nudges: 1000 ways to save energy" focused on the question of the
incentives needed to motivate residents and building users to save energy.

The trust axis

It all boils down to the question of trust: to co-create and co-innovate with
complete serenity, it is essential to establish a climate of trust in the first place.
The workshop on the search for energy savings mentioned above was extended
by a research work that gave rise to an academic paper entitled “Living lab as a
support to trust for co-creation of value: application to the consumer energy
market15” which highlighted this question of trust: “The notions of trust and
contract are strong issues in Living Labs. Indeed, Living Labs as innovation

15
https://www.cairn.info/revue-journal-of-innovation-economics-2019-1-page-53.htm

341
networks require the establishment of strong links of cooperation and
collaboration.../... In addition, the mobilization of actors is generally based on
mutual trust, which must be built and nurtured”. This research work went as far
as developing a tool to assess trust in a living lab process (Co-coon Matrix).

Trust is a prerequisite for individuals to engage together and combat the natural
tendencies of human beings to put forward their own point of view. This need of
trust plays out on several levels:
• Trust in contractors and other stakeholders
• Trust between group members, especially in the case of larger groups,
with concentric circles of participants
• Trust in external partners and contributors
• Trust in the facilitation methods and teams

Trust in the tools used in co-creation processes: a certain reluctance to use open
innovation tools due to the lack of trust that contributors may feel towards current
co- creation and open innovation platforms, particularly with regard to the
protection of their personal data.

3 The blockchain to boost co-creation

In this second part of our paper we will share our deep conviction that the
blockchain is an extremely promising path for co-creation. We are convinced that
the blockchain is an extraordinary opportunity for co-creation both qualitatively
(better results) and quantitatively (possibility of extending the reach, finally
allowing it to scale up from a confidential to an extended stage). We will support
our point by giving examples and presenting a concrete project. We will also
indicate what implementation precautions have to be taken and we will present
our vision for future developments that can be anticipated.

What is blockchain?
The blockchain is a transparent and secure information storage and transmission
technology, operating without a central control body. By extension, a blockchain
(sometimes called DLT "Distributed Ledger Technology ") is a database that
contains the history of all the exchanges made between its users since its
creation. This database is secure and distributed: it is shared by its various users,
without intermediaries, which allows everyone to check the validity of the chain.

While its first applications were limited to the field of fintechs. Bitcoin is the most
well-known use case of blockchain. It was created in 2008 by an unknown person
whose pseudonym is Satoshi Nakamoto. It refers to both a secure and
anonymous payment protocol and a cryptocurrency. Blockchain is now finding
new uses in many sectors: energy, health, logistics, intellectual property, etc. As
far as intangible capital is concerned, one area that seems particularly promising
for the blockchain is that of co-creation and open innovation.

Blockchain and co-creation

The blockchain was initially linked to monetary transactions in the field of crypto-
currencies. But what is interesting is that by extrapolation it can be applied to all

342
kinds of transactions. But what is an innovation process if not transactions of
ideas, suggestions, creative and inventive contributions? By broadening the
concept of innovation, it can also be described as transactions in intangible
assets.

- Therefore, all the advantages of the blockchain can be applied to co-

creation:
- Traceability of the co-creator’s contributions: this makes it possible to
identify precisely who was at the origin of what and also to capitalize them.
- Transparency allows for the recognition of the authorship of ideas and
contributions
- Authenticity of transactions
- The capitalization of contributions has an essential effect: this
capitalization makes it possible to build new paths through the remixing of
innovative ideas and tracks. It is a powerful lever for creativity and
inventiveness.
- The interest of the blockchain is that it makes it possible to go beyond the
debate mentioned above about co-creation (quantitative improvements vs.
qualitative improvements): by making it possible to capitalize on ideas and
contributions, the blockchain makes it possible to avoid "reinventing the
wheel", by constantly remixing them, it constantly enriches them, thus
arousing the desire to push them even further. At the same time, by
allowing everyone to express themselves, the blockchain maximizes
contributions. Quality of contributions x quantity of contributions results in
a virtuous spiral of creativity and inventiveness!

The Nudge and Trust axes have been mentioned above. It is striking to note that
the blockchain allows precisely major contributions in these two areas:

Trust:
Blockchain is a technology that has intrinsic virtues of transparency and equity. It
generates a spirit of collective sharing and intelligence as well as a sense of
belonging to a community. It thus promotes the reunion of individuals who share
common values and are oriented towards the same goal. All these qualities make
the blockchain an ideal instrument for building trust.
Moreover, by protecting intangible assets, the blockchain makes it possible to
secure innovators, thus increasing their trust in the system, which in turn
encourages their involvement.

343
Nudge:
By ensuring the traceability of contributions, the blockchain enables to know who
was at the origin of the value creation in a co-creation process. And even if this
is a collective process, it ensures that each contributor is truly recognized. And
when we know that individual recognition is a source of motivation, we see all the
potential that blockchain can bring to boost co-creation.

But of course, blockchain can allow this nudge effect to go far beyond simply
distinguishing individuals for their personal contributions. The blockchain may, for
example, include a scoring and voting mechanism to assess individuals for their
contributions by their peers. The blockchain can also go further in encouraging
by setting up a system of remuneration for contributors through tokens.

These incentives make it possible to create a sound emulation between

contributors in a spirit of cooperation.

Therefore, with these trust and nudge characteristics, it is not surprising that
blockchain can be considered as the technology with the most potential to
advance the co-creation of innovation.

The blockchain can considerably increase the business value for co-creation
approaches through its intrinsic added values: more inventiveness and creativity
are generated while there are fewer losses thanks to the traceability of these
ingredients, possibility to involve more participants.

Blockchain, co-creation and Living Labs

Several initiatives have taken place in recent years under the impetus of a living
lab around the role of the blockchain in the field of co-creation and open
innovation, evidencing that the subject of co-creation and blockchain is a brand-
new research domain. Among these initiatives, we can mention:
• The creation in July 2016 of a LinkedIn group entitled "Blockchain, open
innovation and co- creation"16. To date, it has more than 1140 members
and generates extensive exchanges, which shows the interest of the
subject.
• Conferences such as the three editions of the Blockchain Agora17 event
held in Paris in December 2016, December 2017 and November 2018,
which brought together more than 120 people each time. Each event had
a particular angle related to the general theme of blockchain and co-
creation: the 2016 edition focused on coopetition, the 2017 edition on the
empowerment of individuals through blockchain, the 2018 edition on the
positive and societal impact of blockchain.
• Workshops such as those held during the last two OpenLivingLab Days
(first one in Krakow in 2017 which was an introduction to the blockchain,
the second one in 2018 in Geneva, in a more elaborate and concrete form
which generated great interest), or during The Journée Innovation in May

16
https://www.linkedin.com/groups/8561295/
17
www.blockchain-agora.com

344
 2018 in Paris on the theme of innovation ecosystems, which brought
together several living labs.
• Publication of articles dedicated to this theme, notably for Harvard
Business Review France.

These initiatives were diverse but had the common objective of arousing interest
in the new field of blockchain and co-creation, identifying experiments, identifying
actors, creating partnerships in this field, evaluating the potential of blockchain
for co-creation. They also generated a lot of reactions and testimonials which
have been quite useful for digging the subject of co-creation and blockchain.

Several projects have been identified in this specific field. These projects range
from the simplest (identification of contributors' contributions) to more ambitious
projects (real co- creation approaches).

A benchmark that we conducted at the beginning of 2019 showed that many open
innovation and co-creation platforms based on blockchain have recently
emerged, such as Connecty18, Tribute 19, Ideation20, Crowdholding21, ISH22,
Kakushin23, ValYooTrust, ...

It is not the purpose of this paper to detail these projects. However, it should be
noted that they each have their own specificities in terms of angles (some are
more oriented towards research, others towards innovation), fields of activity,
organization, functioning, etc.

The ValYooTrust project

Among projects mentioned above, the ValYooTrust project is probably the most
advanced. Built around a blockchain, ValYooTrust was born from the meeting at
the end of 2017 of three actors: a living lab, a major academic institution (Institut
Mines Télécom24), and an entity of the French Ministry of Defense. As its name
may imply, ValYooTrust places trust at the heart of its project, with the objective
that everyone can enhance their own innovative contributions.

ValYooTrust is distinguished by several advantages:

• Its scale due to its international dimension
• Its polymorphic side (in addition to being a platform for co-creating
innovative projects, ValYooTrust is also a market place for intangible
assets and a "phygital" incubator (physical and digital)
• Its multi-stakeholder and interdisciplinary side because ValYooTrust
brings together diversified communities of innovation stakeholders: start-
ups, SMEs, large groups, public institutions, but also any individual wishing

18
https://www.connecty.io/
19
https://www.tribute.coop/
20
https://ideation.com/
21
https://www.crowdholding.com/
22
https://ish.earth/
23
https://www.kakushin.tech/
24
www.imt.fr

345
 to develop their own intangible assets (researchers, creators, inventors,
authors, knowledge producers, etc.).
• Its multidisciplinary dimension: ValYooTrust is interested in various
domains like business innovation but also R&D, education, health... for
example, in the field of health ValYooTrust brings together communities of
patients on the assumption that they can be experts in their own
pathologies and therefore able to indicate avenues for innovation.
• Its disruptive side thanks to several innovations: a sophisticated system of
tokens (ValYooCoins) allowing to vote for the best projects and to value
them, new mechanisms of mobilization of communities gathered by
ValYooTrust, the use of artificial intelligence which allows to propose an
advanced "mix and match" engine to put in contact co-creators
• Its mixed human/virtual operation: alongside automated mechanisms
(based on blockchain, smart data, AI,..) ValYooTrust leaves a place to
human interventions. These interventions consist in complementary
evaluations of contributions by experts and mentors. The human side is
also brought by a JuryGreen™ mechanism, based on assessment by legal
experts giving a legal basis to transactions. Another major interest of this
JuryGreen™ blockchain is to generate a reduced carbon footprint,
reducing the blockchain's energy consumption problem.
• Its innovative and viable business model: registration on the platform is
free for individual entrepreneurs, startups, citizens, so a critical mass of
members can be quickly reached; financial resources come from the major
players thanks to a triple source of income: the availability of the platform
in white label, memberships for registration on the platform, transaction-
based and results-based fees.
• Its universal ambition: ValYooTrust has goals that go far beyond simple
business innovation. The project has objectives to promote initiatives in
the cultural, artistic, societal, philanthropic and other fields that are likely
to attract as many people as possible. Beyond a co-innovation platform,
ValYooTrust is a crowd innovation platform.

ValYooTrust will be officially launched in the fall of 2019.

The ValYooTrust project delivers a perfect example of added value for all
stakeholders thanks to the blockchain:
• Living labs and similar organizations engaged in collaborative innovation
achieve more results (see paragraph on business value) and thus will gain
more credibility and global recognition
• It is also easier for them to manage the full process thanks to the
traceability of contributions
• Participants are more motivated and can get gratification and even
monetary rewards
• Other stakeholders can be reassured on the living labs they supervise or
partner with as thanks to the blockchain they have tools to measure and
assess their effectiveness.
• On a broader scale, the whole civil society will of course benefit from
added creativity and innovation

346
4 The great future prospects provided by the Blockchain
The blockchain will open up new horizons in many areas. We can even talk about
paradigmatic revolutions.

IP Renewal
As individuals are encouraged to become more innovative and creative, it is clear
that regulation as we know it today in the field of intellectual property is no longer
necessary and will change profoundly. The current patent system, while it has
been an engine of innovation and growth in the post-war period, is now reaching
its limits: unbearable costs and delays in filing and approval for small structures,
the "tragedy of anti-communities" that prevents the use of basic research when it
comes from a holding entity, too many patents and annuity research...

While it is currently difficult to envisage immediate use of the blockchain in this

field (because it concerns a legal framework that can be changed very slowly),
its adoption will pave the way for a less costly, faster, automated (proof-of-
invention) patenting process, the arbitration of which will depend on an algorithm
and no longer on a judge.

On the other hand, it can be argued that tokenization via blockchain has been
described as a new form of patent system, which would increase trust among
competing firms, stimulate cooperation and eventually, further support open
innovation. In that respect a solution comes from Nalebuff and Stiglitz (1983),
who suggested that prizes should be used to reward great ideas. According to
the Global Intellectual Property Center (2009), prizes are better at proving a
concept than bringing concrete, useful technologies into existence. Accordingly,
tokenization could be used to assign prizes in a web-based idea competition.

Empowerment of individuals
A major revolution brought by the blockchain is certainly to empower everyone.
The blockchain is thus a formidable lever of inclusiveness as well as a social
elevator: everyone can express himself, put forward his ideas and obtain rewards
(not only moral but potentially in sound and stumbling cash). You don't need to
be well-born, nor to have prestigious diplomas or privileged access to
technological tools to be an innovator. This extension of opportunities opens up
huge opportunities for an extensive deployment of innovation and thus for a much
broader creation of value that can benefit everyone.

Scaling Up
The blockchain allows to create virtual communities. These can be very large and
extend ubiquitously to the four corners of the world. In addition, the acceleration
of the production of intangible assets and knowledge in the world is generating
an exponential need for "matching" between supply and demand. The blockchain
can accompany and multiply this growth in innovation by taking it to a much larger
scale.

347
The great strength of the blockchain is that it reproduces a natural mechanism
and develops organically. In biology, stigmergy is an indirect coordination
mechanism between agents. The principle is that the trace left in the environment
by the initial action stimulates a subsequent action by the same or a different
agent. In this way, successive actions tend to be reinforced and thus lead to the
spontaneous emergence of coherent, apparently systematic activity. Thanks to
the stigmergical processes it induces, the blockchain makes it possible to
massively mobilize collective intelligence, to introduce a spirit of sharing, to
mutualize the contributions of community members, to generate a broader
cooperation that is more natural, fairer and more motivating. By managing the
individual reputation of community members, the blockchain also creates a sound
emulation between co-creators. Finally, by combining the principle of cooperation
with the co-creation of innovation, it creates a new paradigm, deploying more
efficient, productive and ethical innovation.

New forms of organization and social impact

Beyond allowing this individual empowerment, the spirit of the blockchain is
fundamentally based on collective values: those of sharing, cooperation, and the
creation of common goods. In the words of French entrepreneur Gilles Babinet,
currently the digital champion of France at the European Commission, the
blockchain can help to "horizontalize" the world.

With the enormous prospects that it brings the blockchain is much more than a
technology that will boost the co-creation of innovation. The blockchain paves the
way for a new economy that some call crypto-economy, based in particular on
peer-to-peer transactions without confiscating the value created through
intermediaries. In this approach, the value is fairly distributed with minimum
resources being allocated to everyone and the added value being shared
between those who created it. But it is clear that beyond these economic
challenges, these new schemes will make it possible to bring about new
decentralized, more democratic and more ethical social organizations such as
DAOs (Decentralized Autonomous Organizations): these are organizations
whose governance rules are automated and immutable and transparent in a
blockchain. Aragon25 is a good example of a prototype platform based on the
DAO principle.

The DNA of these new types of decentralized organizations naturally tends to be

oriented towards the common good. Organizations like Positive Blockchain 26 are
already working in this direction by identifying blockchain initiatives with a real
social impact.

Required developments
But let's face it, as it stands, the blockchain is not free of shortcomings, of a
technological, practical and societal nature.

25
https://aragon.org/
26
www.positiveblockchain.io

348
In particular, the issue of the enormous computing power required, which
generates costs, low yields and slow transactions, will have to be resolved. One
approach initiated by the ValYooTrust project with its blockchain JuryGreen™
consists in reintroducing a human factor into certain mechanisms of the
blockchain, thus reducing its energy consumption.

An economic challenge is to find practical applications: the blockchain is now

entering a paradoxical phase. It represents a great dream, it makes everything
seem possible and its fields of applications are potentially unlimited. But on the
other hand, it is now time to sell something more than dreams. Companies expect
concrete, down to earth applications, they want to see and use the imagined
business cases which can be based on PoCs (Proofs of concepts) that really
work and not only on paper. And even, beyond these experiments it is necessary
to go further by transforming the pilot tests into a profitable business model. This
business model must lead, for the most ambitious projects, to the digital
transformation of companies which have seized blockchain’s disruptive
opportunities and apply it to their strategic business areas.

Technologically, the challenge for the blockchain is to be able to grasp ideas and
co-creations and to transcribe them correctly. As much as standardized data from
bitcoin transactions are easy to enter in chained blocks, it becomes more
complicated when it comes to multidimensional content with strong qualitative
and therefore subjective components. There is also the challenge to bring
together the innovators best able to collaborate. This is where matching solutions
come into play, such as artificial intelligence and smart data. When it comes to
developing POCs and mock ups, virtual reality is a solution that can bring great
added value to a co-creation platform.

Another issue is the security and reliability of the system. Recent events have
shown that the blockchain is not free of loopholes. Should we not fear that groups
of individuals will set up a coalition to influence or divert the added value of
"blockchained" contributions to their benefit?

In the end, provided that individuals are not chained to a technology, however
promising it may be, and that care is taken to give primacy to human beings and
ethics, the paths of the blockchain certainly deserve to be explored by those who
believe in the virtues of co-creation.

European perspectives
We are well aware that these reflections are only the beginning of a vast field of
research and experimentation in which it would be essential for ENoLL to take its
part in evaluating how the living labs could have a new impetus thanks to the
blockchain.

Of course, many questions are raised by the introduction of blockchain in a co-

creation process:
• How to secure the technical feasibility and security of the whole process
• If participants are individually recognized and even financially rewarded,
won’t living labs lose a sense of collective spirit?

349
 • How to train people to use blockchain methods and tools? Won’t it
constitute a technical obstacle and thus introduce a barrier to the
participation of people without sufficient technical background?
• How to cope with the footprint issue of the blockchain?

We can see that this field with its enormous stakes is attracting increasing
attention from Europe. Several recent or current initiatives can be mentioned: the
projects #Blockchain4EU "Blockchain for industrial transformation"27, the prize
Blockchains for Social Good28, The European Union Blockchain Observatory and
Forum29 or the #DLT4Good "Co-creating a European Ecosystem of Distributed
Ledger Technologies (DLTs) for Social and Public Good"30.

As stated on the website of the Commission for this latter project "Distributed
Ledger Technologies (DLTs), the most well-known being Blockchain, are one of
the emerging technologies foreseen to have a deep and broad impact over the
next ten years. Looking in particular into public or third sectors and other civil
society organizations, DLTs are expected to transform how such sectors and
organizations operate or connect with citizens, businesses and other
stakeholders".

The sustainability of the concept of co-creation via the blockchain is directly

derived from the business value, as well as the stakeholder and social impact
brought by the blockchain. In addition, it must be noted that governments and
administrations in many countries are eager to support blockchain-based
projects. This is also notably the case at the European level, as described in the
above paragraph. Of course, this public support will benefit initiatives like co-
creation and blockchain and strongly contribute to its sustainability.

5 Conclusion
We wanted to show in this paper that the blockchain could be a lever for the future
of the co- creation of innovation thanks to two major ingredients: the nudge effect
and confidence building. Thanks to these, the blockchain makes it possible to
overcome the shortcomings of co-creation. In doing so, it opens up exciting
opportunities for living labs. It extends their influence far beyond the local to the
global and universal. It can help them evolve into more transparent, horizontal
and ethical organizations. It allows individuals to be re-invigorated and, by
allowing them to truly flourish, it creates the conditions to put the human being,
rather than the user, at the heart of innovation processes. And this is for the
betterment of both the individual condition and the collective good and progress.

References
Dupont L., Mastelic J., Nyffeler N., Latrille S., Seulliet E., (2019, January) Journal
of Innovation Economics & Management (pages 53 – 78), Living lab as a

27
https://blogs.ec.europa.eu/eupolicylab/portfolio/
28
https://ec.europa.eu/research/eic/index.cfm?pg=prizes_blockchains
29
https://www.eublockchainforum.eu/
30
https://blogs.ec.europa.eu/eupolicylab/portfolios/dlt4good/

350
 support to trust for co-creation of value: application to the consumer
energy market

Seulliet E. (2016, July 17) Open innovation, co-creation: why blockchain is a

small revolution, retrieved from https://medium.com/@ericseulliet/open-
innovation-co-creation-why-blockchain-is-a-small-revolution-
73e7d0b480d5

Shavit D., Seulliet E., (2017, December 2) The empowerment of people thanks
to the Blockchain in 7 points, retrieved from
https://medium.com/@ericseulliet/the-empowerment-of-people-thanks-to-
the-blockchain-in-7-points-e5ccb345905e

Duvaut P., Seulliet E., Shavit D. (2018, February 16) Reinventing co-creation
thanks to the blockchain, retrieved from
https://www.linkedin.com/pulse/reinventing-co-creation-thanks-
blockchain-eric-seulliet/

Duvaut P., Seulliet E., (2018, November 29) Blockchain, a technology that also
protects and promotes your intangible assets, retrieved from
https://medium.com/@ericseulliet/blockchain-a-technology-that-also-
protects-and-promotes-your-intangible-assets-20fc9154e885

Duvaut P., Joly L., Seulliet E., Solani S. (2019, July 16) Libérer la propriété
intellectuelle grâce à la blockchain, retrieved from
https://www.hbrfrance.fr/chroniques-experts/2019/07/27030-liberer-la-
propriete-intellectuelle-grace-a-la-blockchain

351
Cross-cultural Differences in Living Lab Research
Nele A.J. De Witte*1, Ingrid Adriaensen*1, Leen Broeckx1,
Vicky Van Der Auwera1 and Tom Van Daele*2

*Corresponding authors
1 LiCalab/Thomas More University of Applied Sciences, Belgium
2 Expertise Unit Psychology, Thomas More University of Applied Sciences,

Belgium

Category: Full Research

Abstract
Cross-border collaboration is an important part of living lab research, as
circumstances and requirements for services and products can vary greatly
depending on the region in which they are introduced. While cross-cultural
differences can be of interest for these studies, they can also be confounding
factors for data collection and analysis. Dissimilarities in the recruitment and in
the participation of end users in different regions could influence the outcomes
of international studies with multiple implementation sites across countries.
Therefore, the current survey study investigates awareness of such cross-
cultural differences. The sample consists of 36 living labs from 20 countries.
Results show that regional differences are reported in terms of participants’
motivation for participation and the impact of gender, age, professional status,
and socio-economic status on participants’ contribution. Additionally, regional
differences influence whether a moderator should take the role of a facilitator or
a more dominant guide of the process during group sessions. Implementing well-
chosen strategies for recruitment, for grouping, and for supporting equal
contribution in sessions could improve the quality of international living lab
research, while still maintaining sufficient standardisation.

Keywords: Cross-border research, Group dynamics, Moderator, Recruitment,

Living Labs

352
1 Introduction
Living lab organisations are flourishing across Europe and there is a noticeable
rise of these labs in the rest of the world as well (Ballon, Van Hoed, & Schuurman,
2018). Cross-border collaboration is an important part of living lab research, as
circumstances and requirements for services and products can vary depending
on the region in which they are introduced. Obtaining relevant information on local
preferences and customs in respect to a service or design can be of great
importance for successful dissemination across borders. However, dissimilarities
in the recruitment and in the participation of end users in different regions could
influence the outcomes of international studies (e.g., Im, Page, Lin, Tsai, &
Cheng, 2004). Therefore, it is important to investigate the influence of such cross-
cultural differences and adjust study protocols accordingly.

There is an increase in organisations that describe themselves as living labs.

Governmental and international bodies (such as the European Union) are
promoting living lab methodology for innovation (Ballon, Van Hoed, &
Schuurman, 2018). While different interpretations exist, living labs can generally
be defined as open innovation systems where end users and other stakeholders
are involved in the exploration, co-creation and evaluation of solutions in realistic
circumstances (Ballon et al., 2018). Living labs can improve the understanding of
factors contributing to the success of innovations in different social,
environmental and cultural contexts (Mulder & Stappers, 2009). Developers
making use of living labs have been shown to benefit of such research in terms
of gaining new insights, being able to test product-market fit and achieving
positive economic effects (Ballon et al., 2018).

There is a growing need for international living lab research since the cultural,
professional and legal context of specific regions can have important implications
for the products and services of interest. International cooperation between living
labs provides benefits not only for the innovations that are being developed.
Living labs benefit equally, since this provides them with opportunities to learn
from one another, exchange best practices and harmonize ideas (Mulder &
Stappers, 2009). While cultural differences are of interest for the further
development and implementation of innovation, they could however also
influence data collection and be confounding variables for study outcomes.
Characteristics that can be associated with culture (e.g. socioeconomic or
educational status) could threaten the validity and reliability of cross-cultural
studies (Im, Page, Lin, Tsai, & Cheng, 2004). Moreover, methodological
approaches that work well in one country or context might not necessarily be
appropriate in a different context.

Group sessions (e.g., co-creation/co-designing sessions, focus groups) are

commonly used in living lab research since they are considered to be a culturally
sensitive methodology in which group interaction is key to gaining insight into
diverging and converging opinions (Greenwood, Ellmers, & Holley, 2014;
Liamputtong, 2011). However, group dynamics could also hinder data collection
in some conditions. Not all participants might actively contribute to the discussion,
which could be due to the social context and to the characteristics of the peers
that are present (Liamputtong, 2011). Individuals with certain personality traits or

353
social standing can (unintendedly) intimidate group members and hereby
influence the outcomes of the session. Guidelines recommend that participants
of group sessions are homogenous in personal characteristics and background,
but studies investigating the effect of heterogeneity on outcomes in group
sessions are sparse (Greenwood et al., 2014). Additionally, it is not always
feasible to cluster individuals based on all potentially relevant characteristics,
such as gender, age, ethnicity and socioeconomic status (SES), the latter of
which is often measured as a combination of education, income and occupation.

An important benefit of cross-border living lab collaboration as opposed to other,

more traditional, cross-cultural research is that each living lab executes the study
in its own region. Experienced moderators and researchers speak the local
language and are aware of preferred communication styles. However, the use of
standardized protocols for recruitment and study execution might not always be
optimal for their specific context. Greenwood et al. (2014) investigated ethnical
diversity in focus groups and observed that more heterogeneous groups
experienced more difficulty than homogenous groups when discussing ethnic and
cultural differences. However, no effects were found on other topics (i.e., being a
caregiver). Xie et al. (2012) reported that combining individuals from different age
groups (children and older adults) in a co-design study had implications for the
preferred methodology (e.g., using sticky notes or making drawings) and the
organisation of the sessions (e.g. group size, timing and activities during breaks).
Older individuals could also be considered more important figures, especially in
non-Western cultures (Halcomb, Gholizadeh, Digiacomo, Phillips, & Davidson,
2007). Therefore, younger individuals might not consider it to be appropriate to
voice differing opinions, which is problematic for unbiased data collection. Gender
could have a similar effect in some contexts (Halcomb et al., 2007).

As different international contexts might require adjusted methodological

approaches, such flexibility should be included in international research
protocols. However, in order to be able to do so, we first need to be aware of
which differences exist and might be of importance. Living labs use a broad
repertoire of methodological approaches (e.g., Ballon et al., 2018) and mapping
all of the specific approaches or designs is beyond the scope of the current study.

The current study aims to investigate which regional differences can influence the
participation in living lab research in different regions across the world. The aim
is rather to investigate the perceived impact of such differences, both in group
contexts (e.g., focus groups or group co-designing sessions) and in individual
settings (e.g., interviews, human factor studies), across different countries. The
current study intents to map to what extent local contexts are considered when
organizing and moderating living lab research and whether the living labs expect
potential differences to have an impact on study outcome.

2 Methods
2.1 Recruitment
Organisations performing living lab research were invited to complete an online
survey, through personally addressed e-mails, social media, and through

354
networks, including the European Network of Living Labs (ENoLL). Although we
strived for equal representation of different regions across the world, living labs
are less common and more difficult to reach outside of Europe. Data was
collected between December 2018 and April 2019. All participants provided
informed consent.

2.2 Survey
A survey assessing cross-cultural differences in the participation of end users in
living lab research was designed. Firstly, the survey inquired about the
recruitment process in the specific region of the participating living lab. Secondly,
local differences in study participation depending on gender, age, professional
background and SES were assessed. Finally, the participants reported on the
use of inclusion strategies and the role of the moderator, and reflected on
potential cross-cultural differences. The survey consisted of multiple choice and
open-ended questions. Completing the survey took about 35 minutes.

2.3 Analyses
Frequency analyses were used to compare the responses of different regions.
Additionally, thematic qualitative analyses were used to gain more in-depth
insight into cross-cultural differences.

3 Results
3.1 Description of living labs
In total, 36 living labs of 20 countries participated in the survey (Table 1). The
majority of these living labs are situated in European countries (n=28) and are
specialized in the area of Health & Care and/or the Silver Economy. No living labs
from Africa and Eastern Europe could be included. All living labs, apart from one
Danish living lab, completed the entire survey. One living lab identified as being
both French and Spanish. Its inputs are, therefore, included in both the Western
and Southern European regions. Due to a small sample size, findings in countries
outside of Europe should be interpreted with care.

All living labs perform group sessions (e.g., co-creation sessions or focus
groups). On average, they organize 26 of such sessions annually (M = 25.5 SD
= 44.94), however this ranges from 1 session to about 225 sessions per living lab
per year. The living labs perform about 38 individual sessions (e.g., interviews,
human factors studies) annually (M = 38.45, SD = 73.77). Again, differences
between labs are large with two living labs performing no individual sessions at
all and others performing up to 300 sessions per year. Additionally, living labs
perform other activities, such as stakeholder meetings (M = 17.09, SD = 16.47
annually), survey studies (M = 7.59, SD = 8.78 annually), and other activities such
as innovation labs, community building, business model development, live
sessions, and boot camps. There is a large range in the number of participants
that are being included on a yearly basis. Most living labs include between 10
and 500 participants (M = 156.76, SD = 126.14, n = 27). Six living labs include
more than 500 participants each year, specifically in Colombia (n = 1000
participants), UK (n = 1300 participants and n = 6892 participants), Finland (n =

355
2000 participants), Belgium (n = 5000 participants), and Canada (n = 15000
participants).

Table 1. Participating countries organised according to the United Nations Geoscheme of

geographical regions of the world.

Region Subregion Country Participating

living labs
Europe Western Europe Austria 1
Belgium 3
France 5a
Germany 1
Switzerland 1
Netherlands 4
Northern Europe Denmark 1
Finland 2
Ireland 1
United Kingdom 5
Southern Europe Italy 2
Spain 3a
Asia Eastern Asia China 1
Taiwan 1
Southern Asia India 1
Western Asia Turkey 1
Americas Central America México 1
Northern America Canada 1
Southern America Colombia 1
Oceania Australia & New Australia 1
Zealand
Notes. a One living lab indicated that it was operational in both Spain and France.

3.2 Recruitment of participants

Participating living labs were asked to report which communication channels they
use for recruitment and which mode of recruitment was most effective in their
region (Figure 1). Living labs most commonly recruit through other organisations,
such as patient organisations, professional associations, or collaborating
partners and developers. It is the most effective mode of recruitment according
to 13 living labs, mostly because these other organisations have an existing
relationship with - as well as existing knowledge of - the target group. Additionally,
recruitment through the work-related or through the personal environment of the
living lab (e.g. previous contacts, staff & patients of the hospital) is regularly used
and eight living labs find it an effective strategy. This approach has the benefit
that an relationship with end users already exists, which contributes to availability
and motivation. While a mailing list and panel are common ways of recruitment,
they are rarely the preferred means. Few regional differences exist with regard to

356
the communication channels that are being used or preferred. However, online
advertisement (e.g. on fora or social media) is solely practised in Northern Europe
(5/9 living labs) and in Western Europe (by 5/14 living labs).

Through other organisations

Personal environment

Newsletter/mailing list of living lab

Maintaining an active panel

Website of living lab

Advertisement online

Printed advertisement

Recruitment agency

Direct & face to face contact

Word of mouth

Marketing agency

Multiple channels (non-specific)

0 5 10 15 20 25 30 35
Organisations that use this channel
Organisations that report the channel being most effective in their region

Figure 1. Frequencies of the communication channels used to recruit participants and the
communication channels that were reported to be most effective in a region. Living labs were
able to report multiple communication channels.

On average, attrition rate is around 15% in the participating living labs (M = 14.88,
SD = 10.31). Estimates were somewhat lower in Northern and Central America
(8% and 2% respectively) and Australia (5%), as compared to European (16%)
or Asian countries (16%). Categories for motivation for participation were defined
by thematic analysis. The values presented in Table 2 indicate the number of
living labs reporting this motivational principle to be present. Multiple driving
principles could be reported per living lab. In general, participation in living lab
research seems to be most strongly driven by an intrinsic motivation for personal
reward, such as gaining appreciation and knowledge. Other motivating factors
are gaining material rewards, contributing to helping others or the society, being
interested in the specific product or application, or having social contact. Results
suggest that personal reward is predominantly important in Europe and Australia,
while external (monetary) reward is more important in Central America and
Eastern Asia (Table 2). Turkey (Western Asia) appears to be situated more
closely to the European values in the field of motivating factors. Differences
between specific countries within one region are limited.

357
Table 2. Motivation for participation. The five middle columns give an overview of the number
of living labs reporting different principles, which motivate participants in their research
studies. The final column on the right represents the extent to which an external reward is
needed to motivate participants on a scale from 0 (not needed at all) to 10 (participation
solely for reward).

Region Reported motivating principles External

reward
needed
Persona External Contribution Interest in Social Mean (SD)
l reward reward innovation contact
Central 1 8.00
America
Northern 5.00
Americaa
Southern 1 1 3.00
America
Eastern Asia 2 1 1 8.00 (1.41)
Southern 1 1 5.00
Asia
Western Asia 1 1 1 1.00
Australia 1 1 3.00
Northern 6 6 3 5 1 3.67 (1.94)
Europe
Southern 3 1 2 2.00 (2.00)
Europe
Western 9 5 6 3 2 4.73 (2.69)
Europe
All living labs 20 16 13 12b 4 3.83b (2.40)
Note. a The participating living lab (from Canada) did not report specific motivating factors.
b While the input of the French-Spanish living lab was included in the results of both southern
and Western Europe, it was only counted once in the entire sample.

Living labs report that differences between target populations and study designs
relate to whether external rewards are necessary for participation. External
rewards appear more important for professionals (with limited time available) or
for individuals recruited by recruitment agencies. Participants with an intellectual
interest in the topic and patients who can gain from the innovation have a lower
need for external motivators. While a Dutch living lab experienced that older
people are more intrinsically motivated (which they suggested was possibly
because they have more time for participation than their younger counterparts
do), a living lab from the United Kingdom (UK) reported that reimbursement of
costs was relevant for the elderly. Apart from cross-cultural differences, external
factors can also have implications for remunerations. Studies that require more
investment from participants (in time or due to experiencing some discomfort) are
more likely to need external rewards. Additionally, rewards can be influenced by
project budgets and whether the innovation is for public design as opposed to a
private enterprise or academic research.

3.3 Group selection

358
Many living lab activities consist of group sessions (e.g., focus group, co-creation
sessions). Group sizes can vary depending on the innovation of interest, target
population, and study design. However, living labs indicate that they generally
consist of about 10 participants (M = 10.54, SD = 5.68). These groups can contain
individuals differing in gender, age, professional status, and SES. Figure 2
provides an overview to what extent living labs indicated that these factors would
influence contribution in living lab research. Contribution was defined as actively
collaborating in the process, providing input, and formulating ideas in the session.

More than half of living labs (n = 20) report that they do not consider gender to
be of influence to study contribution, while about a quarter does believe it could
have an impact (n = 8). Gender is mainly reported to have an impact in Austria,
India, Mexico, and the UK. However, only India and one Italian living lab organize
separate sessions for male and female participants. Living labs were asked to
estimate the contribution of male and female participants in three conditions:
mixed gender group session, separate group sessions per gender, and individual
sessions. Although living labs find it difficult to make this assessment, the majority
of labs believes that both genders contribute evenly in all three conditions (mixed:
n = 19; separate: n = 13; individual: n = 22). The remaining living labs mostly
report that female participants contribute somewhat to a lot more actively (mixed:
n = 5; separate: n = 7; individual: n = 5). Three living labs (from Austria, Germany,
and the UK) report that male participants would contribute more actively in mixed
group sessions, but that organizing separate group sessions would results in
increased contribution from female participants (n = 2) or equal contribution of
both genders (n = 1).

While age is not considered to have an impact on contributions to living lab

research according to the majority of living labs in France and Australia, most
living labs from the other countries do believe that age could be of influence on
living lab research. Again, assessing contribution depending on age is considered
difficult (do not know: n = 12 to n = 13 in the three conditions). The majority of
labs believe that contribution is not influenced by age in all three conditions
(mixed: n = 12; separate: n = 18; individual: n = 19). About half of the remaining
living labs report that older participants contribute somewhat to a lot more actively
(mixed: n = 6; separate: n = 3; individual: n = 3) while the other half beliefs that
younger participants contribute somewhat to a lot more actively (mixed: n = 5;
separate: n = 3; individual: n= 2). In Turkey (Western Asia), older participants
are a lot more active in mixed group sessions, while older and younger
participants contribute equally when separated by age in group or individual
sessions. Different age groups are consequently always included in separate
sessions in this country. Seven other living labs (from Austria, China, India, Italy,
Mexico, Spain/France, and the UK) also try to organise separate sessions for
different age groups when possible. The majority of European labs include
different age groups in 1 session. Some labs are also specifically focussed on
research in certain age groups (e.g., elderly) and therefore already have a
homogeneous sample in relation to age.

359
 (A) Gender (B) Age
16
14 16

number of livng labs

Number of living labs

14
12 12
10 10
8 8
6
6 4
4 2
0
2
Of Might be Not of Do not
0 influence of influence know
Of Might be of Not of Do not influence
influence influence influence know

Europe Asia Australia The Americas Europe Asia Australia The Americas

(C) Professional status (D) Socio-economic status

16 16
Number of living labs

14 14
Number of living labs

12
12
10
10 8
8 6
4
6 2
4 0
2 Of Might be Not of Do not
influence of influence know
0 influence
Of Might be of Not of Do not
influence influence influence know Europe Asia Australia The Americas

Europe Asia Australia The Americas

Figure 2. These graphs represent to what extent living labs believe personal characteristics,
specifically gender (panel A), age (panel B), professional status (panel C), and socio-
economic status (panel D), would influence active contribution in living lab research.

360
The vast majority of living labs report that professional status, which refers to
whether participants are end users/patients or professionals (e.g., hospital staff),
definitely (n = 22) or potentially (n = 5) influences study contribution.
Nevertheless, only three living labs (all European) try to organise separate
sessions for participants with a different background, while 22 living labs include
them in the same session. Overall, most living labs consider professionals and
end users to contribute evenly in mixed group sessions (n = 14), separate group
sessions (n = 14), and individual sessions (n = 13). However, there were also
many living labs that were not sure how professional status influenced the
contribution in these different sessions (n = 9, n = 13, and n = 11 of the sample,
respectively). Of the living labs that did observe a difference in contribution, it is
mostly the case that professionals contribute somewhat to a lot more actively
than end users do (n = 10, n = 6, and n = 8 of the sample, respectively).

Finally, estimating the influence of SES can be more difficult as compared to

gender and age, since it is harder to observe and not always documented or even
disclosed. Participants with a high SES have a good income and perform at a
high educational/professional level. Participants with high and low SES are
equally likely to participate in research according to 14 living labs. The majority
of Belgian and Dutch living labs and about half of living labs from the UK believe
individuals with high SES are more likely to participate as compared to low SES
individuals. On the other hand, the Turkish, Colombian and one other UK living
lab indicate that individuals with low SES are more likely to participate. The
remaining living labs were not sure about the representation of SES in their
samples. Opinions were also divided regarding the influence of SES on study
contribution, with most participants reporting that it could potentially have an
impact (Figure 2). SES is estimated to have an impact on contribution by seven
living labs, representing the majority of living labs from the UK, Taiwan, and
Turkey as well as half of the Dutch living labs. Seven other living labs,
representing Mexico, Colombia, Finland, Germany, Ireland, Italy, and the UK, do
not generally believe SES to have an impact on contribution in the sessions.
Participants with different SES are often represented in one group, with only three
living labs (in Belgium, China, and the Netherlands) trying to organize separate
sessions if possible. Participants with differing SES are often included in one
group due to the fact that SES is often not known (beforehand) or not of interest
for the study. The majority of participants were not sure how SES influences the
contribution in groups with mixed SES (n = 19), separate groups (n = 23) or
individual sessions (n = 21). All remaining living labs report equal contribution or
more contribution of high SES participants (representing living labs from Belgium,
China, Turkey, the Netherlands, and the UK).

Living labs were able to report other individual characteristics that could influence
participation or contribution to living lab research. Five labs indicated that
geographical living situation is of importance. Not living nearby the living lab
and/or lacking mobility impedes participation and environmental characteristics
(e.g., living in a town vs. rural living) could also influence participation. Secondly,
the match between the product and participant (in terms of interest, experience,
but also having the feeling that they can have an impact) was deemed important
by five living labs. Health can influence participation in two ways. On the one

361
hand, health problems and disability could lower participation rate (through lack
of access or lack of ability to interact with innovations; n = 3). On the other hand,
the health situation could increase the relevance and lead to stronger contribution
(n = 1). The economic situation could also have an impact (n = 3), both in relation
to availability for research (higher workload means lower availability) as well as
when it comes to attitude (e.g., more negativity in unemployed participants). Next,
ethnicity appears to be important in some regions. In the UK, an
underrepresentation was observed of individuals of black and minority ethnicity
(n = 2). Having a migratory background is reported to have an impact in Turkey.
Relatedly, language (e.g., not being a native speaker) can influence contribution
in the sessions (n = 2). Other factors influencing participation in research or
contribution to the sessions are personality characteristics such as technological
competence (n = 3), extraversion (n = 2), position in a group (in studies with
employees of a company; n = 2), previous living lab experiences (n = 1), and
early adopters being overrepresented in research (n = 1).

3.4 Use of strategies

When living labs experience that they are not reaching certain participant groups
in the recruitment process or experience differences in the contribution of
individuals in sessions, they can use different strategies to increase equal
representation in contribution. In the recruitment process, the majority of living
labs from all regions sometimes (n = 22) or always (n = 5) apply strategies to
reach populations that will otherwise be less likely to be represented in the
sample. The most common way to reach these populations is through
intermediary organisations or individuals (n = 9, mostly used in Belgium, the
Netherlands and the UK), by contacting individuals the labs have previously
worked with or are in their database (n = 3, used in France and Turkey), or by
carefully selecting one or more communication channels to reach the targeted
sample (n = 4). Other strategies consist of following up on recruitment and
adjusting strategies to target underrepresented groups (n = 2), being embedded
in a disadvantaged community (n = 1), or optimizing the practical circumstances,
i.e., easy-to-reach venue, time outside or rush hour, ability to bring carer (n = 1).
During group sessions, almost all living labs sometimes (n = 16) or always (n =
16) use specific strategies to make sure that all participants are equally involved
and their opinions are equally heard. Table 3 gives an overview of different
strategies. The living labs often combine multiple strategies. The most commonly
used strategies consist of directly addressing individuals, using different modes
of communication within sessions (e.g., writing individually on sticky notes
combined with oral group discussions), splitting up into smaller groups, and
having a skilled facilitator to conduct the sessions.

362
Table 3. Strategies to promote different individuals (or groups) to be equally represented in
the output of a session. Strategies were based on thematic analyses.

Number of living labs

Strategy applying strategy
Directly and personally addressing individual participants 10
Combining different modes of communication: writing,
verbal communication, constructing models, etc. 6
Implementation of good facilitation/moderation methods 5
Temporarily splitting the group up into smaller groups or
duos 5
Doing a round of the table 4
Increasing participants’ comfort: selection of room,
planning breaks, using icebreakers & games, etc. 4
Giving clear instructions that everybody needs to be
heard 2
Techniques of service design 2
Appropriate language & communication skills 2
Careful preparation of sessions according to needs 2
Using tools to construct a non-hierarchical environment 1
Promoting trust (by including a psychologist) 1

3.5 Moderator selection & involvement

As mentioned above, a moderator (also called facilitator) has a large impact on
the course of a session. However, there could be cross-cultural differences in
how directive a moderator should be for optimal outcome. There is a large
consensus that the role of a moderator should be somewhat flexible, depending
on the characteristics of the product, study, and sample. A moderator should
generally be able to create an environment that puts participants at ease, but also
ensure that all research questions are answered. That being said, there are some
local differences. Predominantly in the UK, Netherlands, Germany, Colombia,
Switzerland, and Mexico, the moderator is seen as a facilitator and participants
should take the lead. The Indian living lab goes even further by calling the
moderator a “co-participant”. However, living labs from Taiwan, Italy, Turkey, and
Ireland believe the moderator should be a director of the session and guard the
process, a role that is somewhat more authoritative.

More than half of living labs regularly (n = 15) or sometimes (n = 4) use diverse
moderators in different studies or sessions. Moderator selection occurs mainly
based on the type of session or research, which might require specific skills or
knowledge. Additionally, it could vary based on group characteristics (such as
age) or practical reasons (such as availability). The majority of living labs (n = 21)
report that the sample does not influence which moderator is being used. Four
living labs (from Belgium, China, Italy, and India) try to match the moderator with
the sample (e.g. a young moderator with a young sample), while two living labs
(from Colombia and the UK) try to use a complementary moderator (e.g., an older
moderator with a young sample). Twelve living labs consistently use the same
moderator and report the following reasons: practical constraints (e.g. not having

363
other moderators available), the assumption that a moderator should have
sufficient skills to moderate all kinds of sessions, and using the same moderator
to build a relationship of trust with the participants.

3.6 Other international differences

When living labs were asked whether participants of their own region differed
from those in other regions of the world, eight living labs reported differences in
culture that were not further specified (Mexico, Colombia, Ireland, the UK, Italy (n
= 2), Spain (n = 2), Switzerland, and Spain/France). Additionally, some other
specific participant characteristics were reported. Two Dutch living labs reported
that their participants are often more direct in their input in the sessions. One of
these labs also reported that Dutch people are more likely to adopt new
innovations. The two Finnish living labs report greater commitment to research in
their country, possibly due to greater equality between citizens. A difference
between Australia and the largest part of Europe is that there is more rural living
(and the associated problems) in Australia. In Austria, the urban/rural structure is
reported to be different from other countries (without further specification). In the
UK, greater privacy concerns have been reported (n = 1), as well as differences
in the use of technology in age groups (n = 1; not further specified). Apart from
these differences between countries in participant characteristics, the location
and approach of the specific living lab could influence how individuals behave in
sessions. A Belgian living lab reported that their panel approach resulted in the
development of a personal bond with participants. A German living lab reported
that due to their location in a university city, they had a strong representation of
individuals high in SES and educational level.

Participating living labs also reported on other cultural differences that were not
necessarily important in their own region, but could be relevant in other parts of
the world. Firstly, policies and business models could influence the input in living
lab research. This specifically concerns differences in the social welfare system
and how secure individuals consequently feel, differences in corporate structure
(hierarchy), and the (political/historical) openness of society and related
openness towards innovation. Other differences could be based on culture and
religion, migration or the geographical location of individuals (within countries).
Secondly, potentially important cross-cultural differences consist of health,
personal and professional beliefs and the existence of trust between participants
and researchers. Anecdotal evidence of a Dutch living lab also hints to some
underlying cross-cultural differences. They experienced that, as compared to the
direct, open and critical stance of participants in the Netherlands, participants in
other countries are more polite and hold back harsh criticisms (e.g., in the UK) or
appear to be more sensitive to social desirability (e.g., in Taiwan). However, these
differences were not reported by the other (respective) countries.

4Conclusion
Cross-border research can deliver highly relevant information for developers and
allow living labs to exchange best practices and ideas. Although cross-cultural
differences could be of interest to developers, it could also be a confounding
factor for data analysis. Differing social norms and personal status related to

364
individual characteristics could imply that the group composition needed for open
communication is dissimilar between countries. The extent to which regions are
aware of differences in individual characteristics varies greatly and awareness
does not always lead to changes in study designs. A first relevant individual
characteristic is gender. Several countries are aware of differences in this area,
which are mostly reflected in female participants contributing more in the
sessions. Secondly, a vast majority of living labs reported age to have an effect
on living lab contribution. Although European living labs mostly combine different
age groups in one session, constructing separate groups depending on age does
appear recommended in multiple countries in- and outside of Europe, especially
in Turkey. Professional status and SES can also influence contribution according
to the vast majority of participating living labs. Results suggest that professionals
might contribute more than patients, but nevertheless participants with a different
professional background are mostly included in the same session.

The findings suggest that organizing separate group sessions and using specific
strategies to support equal representation in study outcomes is warranted.
Participating living labs provided several useful strategies to promote
participation of individuals with diverse background characteristics. Regularly
implemented recruitment strategies consist of using intermediary organisations,
using known participants, and combining multiple communication channels.
Strategies to promote equal contribution in sessions include directly addressing
individuals, combining different communication modalities, and having a skilled
moderator. However, the definition of a “skilled moderator” can differ.
Participating living labs generally agree that a moderator should be able to create
an open and welcoming environment, but also ensure that the procedures are
being followed. Nevertheless, some countries report the need for a more
facilitating stance (e.g., India, Netherlands) while a more authoritative stance
might be required in others (e.g., Taiwan, Italy).

The current self-report study shows that awareness of cross-cultural differences

differs between regions. Further cross-border research is needed to confirm the
effect of such differences. However, the current study indicates that cross-cultural
differences should be a topic of discussion in cross-border living lab
collaborations. Without the necessary methodological considerations, the results
of a study might represent the whole population of a region or might not even
reflect the views of all individuals present in a session. Being aware of cross-
cultural differences and consciously implementing strategies to promote equal
representations as much as possible can improve living lab research. Study
protocols could suggest good practice strategies for promoting equal
representation independently of the included regions. Additionally, when
including certain regions or sensitive topics, it might be advisable to separate
individuals in different groups based on individual characteristics (while
maintaining sufficient standardisation for cross-border comparisons).

References

365
Ballon, P., Van Hoed, M., & Schuurman, D. (2018). The effectiveness of involving
users in digital innovation: Measuring the impact of living labs. Telematics
and Informatics, 35, 1201-1214. https://doi.org/10.1016/j.tele.2018.02.003
Greenwood, N., Ellmers, T., & Holley, J. (2014). The influence of ethnic group
composition on focus group discussions. BMC Medical Research
Methodology, 14(1), 1–13. https://doi.org/10.1186/1471-2288-14-107.
Halcomb, E. J., Gholizadeh, L., Digiacomo, M., Phillips, J., & Davidson, P. M.
(2007). Literature review : considerations in undertaking focus group
research with culturally and linguistically diverse groups. Journal of Clinical
Nursing, 16, 1000-1011. https://doi.org/10.1111/j.1365-
2702.2006.01760.x.
Im, E., Page, R., Lin, L., Tsai, H., & Cheng, C. (2004). Rigor in cross-cultural
nursing research. International Journal of Nursing Studies, 41, 891–
899https://doi.org/10.1016/j.ijnurstu.2004.04.003.
Mulder, I., & Stappers, P. J. (2009). Co-creating in Practice: Results and
Challenges. In Proceedings of the 15th International Conference of
Concurrent Enterprising: ICE 2009 (pp. 1–8). Leiden, The Netherlands:
Centre for Concurrent Enterprise: Nottingham, UK.

366
367
 Developing a quasi-experimental research design
framework using analogue observation to
evaluate the performance of a Living Lab output
Benjamin Nanchen*1, Emmanuel Fragnière1, Patrick
Kuonen1, Joelle Mastelic1, Randolf Ramseyer1 and Henk
Verloo1

*Corresponding author
1 Living Lab Handicap, University of Applied Science Western Switzerland,
Sierre, Switzerland

Category: Research-in-progress

Abstract
Since its introduction in the nineties, the concept of Living Lab has evolved from
a space where technological innovations is tested directly by users for an
innovation eco-system. The creation of the European Network of Living Labs
(ENoLL) in 2006 characterizes this evolution. Although Living Lab are well
disseminated around the world, there is a lack of consensus on how a Living Lab
should be organized (macro level), which types of projects are considered as
Living Lab projects (meso-level) and which methods should be used (micro level)
(Schuurman, et al., 2015). Furthermore, Living Lab need tools to evaluate the
performance of its output (Schuurman, et al., 2015). Therefore, we developed
here a quasi-experimental research design framework using analogue
observation to evaluate the performance of a Living Lab output. In this paper, we
illustrate how to operationalize this research design framework in a case study
that aims at introducing Autonomous Vehicles (AVs) in a city in Switzerland.

Keywords: Living Lab, Quasi-Experiment, Analogue observation, R&D for

Services, co- design, empowerment

368
1 Introduction
The concept of Living Lab was introduced in the nineties at MIT in the US.
Originally, a Living Lab was a space where technological innovations used to be
tested directly by users (Dvarioniene, et al., 2015). Thanks to these
infrastructures, researchers had the opportunities to collect data about the use
of their novel technologies as well as users’ perceptions. Since this first
experience, the concept of Living Lab has evolved to an innovation eco-system
and is now an essential trend in innovation development. This evolution is
characterized by the creation of the European Network of Living Labs (ENoLL)
in 2006. Today, ENoLL counts more than 150 active Living Labs worldwide 31.

As Schuurmann (2015) defined it, three levels characterized a Living Lab: 1)

macro level, the ecosystem, 2) meso-level, the project and 3) micro level, the
tools. Another characteristic of a Living Lab is the involvement of stakeholders in
all categories, as shown in the quadruple helix: users, companies, academics
and public authorities 32. Finally, the user involvement during the whole co-design
process is the most important characteristic of a Living Lab (Schuurman, et al.,
2015).

Although the Living Lab concept is well disseminated around the world, there is
a lack of consensus on how a Living Lab should be organized (macro level),
which types of projects are considered as Living Lab projects (meso-level) and
which tools are used (micro level) (Schuurman, et al., 2015). Furthermore, the
Living Lab initiative need tools to evaluate/assess the performance of its output
(Schuurman, et al., 2015).

A Living Lab approach is also described as “an Open Innovation ecosystem”

were users are involved in the R&D process (Pallot, et al., 2010). R&D projects
are conducted in a laboratory (in vitro). For their part, Living Lab projects are
conducted in natural environments (in situ). Therefore, we propose to structure
Living Lab projects (meso-level) as a Research and Development (R&D)
function. In addition, given that one of the six perspectives typifying a living lab
is service creation (Mulder, et al., 2008), we will use the concept of R&D for
services. As R&D projects use experiment to support, validate or refute a
hypothesis, we propose a quasi-experimental research design framework using
analogue observation (micro-level) to assess causal impacts related to an
intervention by using empirical testing protocols. This tool enables to evaluate
the performance of a Living Lab output.

This paper is organized as follows. In Section 2, we present a literature review

related to the evolution of Living Lab (macro level), R&D for services (meso level)
and evaluation methods (micro level). In Section 3, we describe our
methodological development of a quasi- experimental research design
framework using analogue observation to evaluate the performance of a Living

31
https://enoll.org/about-us/
32 ENoLL Application Guidelines - 13th Wave on https://fr.scribd.com/document/397044439/ENoLL-
Application-Guidelines-13th-Wave

369
Lab output. In Section 4, we illustrate how to operationalize this quasi-
experimental research design framework. The presented case study is based on
a project who aims at introducing Autonomous Vehicles (AVs) in a city in
Switzerland. As an example, we will test the hypothesis that the use of
Autonomous Vehicles (AVs) improve the mobility of people with disabilities.
Finally, in section 5, we discuss our proposition and provide directions for further
development.

2 Literature review
2.1 The evolution of Living Lab and the need for methodology to assess
performance
Since the Helsinki Manifesto (2006), the European Living Labs start to structure
their innovation efforts and methodologies (Mulder, et al., 2008). Mulder and her
colleagues first try to harmonize the methods and tools used in Living Labs. As
in other methodologies for innovation like Design Thinking and Service Design,
there is not one way to foster innovation (Brown & Katz, 2011; Schneider &
Stickdorn, 2011; Fragnière, et al.,2012), but an agreement on several principles
like, the end-users participation in each phase of the co-design process, the
stakeholder involvement (Mastelic, 2019) and the goal to foster a better society.
However, there is a need to find appropriate methods to assess the performance
of a Living Lab, especially of its output (Schuurman, et al., 2015). The Living Lab
harmonization cube does not propose explicitly methods and tools to assess the
performance of a Living Lab output (Mulder, et al., 2008).

2.2 R&D for services – A way to structure and measure innovation

In the Oslo Manual, innovation is defined as “a new or improved product or
process (or combination thereof) that differs significantly from the unit’s previous
products or processes and that has been made available to potential users
(product) or brought into use by the unit (process)” (OECD, 2018, p. 20). A key
tenet of the Oslo Manual is that innovation can and should be measured.
Therefore, it “provides guidelines for collecting and interpreting data on
innovation. […]” (OECD, 2018, pp. 19, 20).

Pallot, et al. (2010) described the Living Lab approach as “an Open Innovation
ecosystem” were users are involved in the R&D process. A Living Lab can be
seen as a “Living Laboratory”, at the level of a region, in which users participate
in the development of innovative goods and services (co-design). Its main goal
is to explore the insight, the salient features valued by a specific population and
to co-create value together. It is also a test environment, open and benefiting
from technological and methodological tools. It is, therefore, an ecosystem
allowing a participatory process, using appropriate tools and methodologies
(Liedtke et al., 2012).

The Frascati Manual (2015, p. 28) defined R&D as “creative and systematic work
undertaken in order to increase the stock of knowledge – including knowledge of
humankind, culture and society – and to devise new applications of available
knowledge.” The new edition of the Frascati Manual put a greater emphasis on
the R&D in the social sciences, humanities and the arts (2015, p. 44). However,

370
the R&D for services process is not formally organized as in other industries
(Miles 2007, Sundbo 1997). Sawatani and Fujigaki (2015, p. 166-168), propose
a Service R&D Model based on the Service-Dominant logic and extended
Moeller’s model where “service processes are divided into facilities,
transformation and usage with three spheres, such as R&D, value co-creation
and site”.

“The classic process of manufacturing innovation follows three steps in a linear

manner: research, R&D and finally manufacturing.” This process cannot be
strictly followed for service innovation because service “raw material” is
knowledge (explicit and implicit) and customer is also co-producer. In addition,
“R&D for production is based on the “make-to-stock” principle. The goods are
produced in batches, then stored and finally sold in the markets.” (Fragnière, et
al. 2018). This is not possible for services, because services are perishable,
instantaneous (Lovelock & Gummesson. 2004) and co-produced, e.g. the
learning and practice of music instrument is not possible without the participation
of the professor and the student.

The iterative Living Lab process – 1. co-creation, 2. exploration, 3.

experimentation and 4. evaluation - (Task, et al., 2017) differs from the R&D
process only in the co-creation component, “which breaks the linearity of the
process and involves every stakeholder taking part in the innovation” (Fragnière,
et al. 2018). “Exploration, experimentation and evaluation have the same similar
roles and purposes as the identification of needs, prototyping and testing in the
classical R&D manufacturing process”. Fragnière, et al. (2018) propose a model
that combines the Living Lab process with the classical R&D process (see Figure
1).

To structure Living Lab projects (meso-level) as a R&D function could contribute

to a better comprehension and acceptance by all stakeholders. Indeed, security,
safety, compliance or social acceptance are attributes that this rigorous and
systematic (i.e. scientific) process can bring to this innovation ecosystem.
Furthermore, this scientific framework facilitates the use of scientific procedures
like quasi-experiment. Given that, one of the six perspectives typifying a living
lab is service creation (Mulder, et al., 2008), we propose to use the concept of
R&D for services.

371
 Figure 1. Integrating Living Lab methods in the more formal R&D process

R&DR

Evaluation

2.3 Evaluation Methods for Living Lab outputs

Impact evaluation relates to the analysis of the causal effect of an intervention.
It is linked to the notion of counterfactual analysis. The outcome has to be
compared with a sample without intervention (Ballon, et al., 2018) and the
developed intervention must create value for its intended users (Ståhlbröst,
2012). In this subsection, we will list some evaluation methods previously used
for Living Lab outputs.

In the field of Information & Communication Technologies and smart home

technologies, many research projects run experiments in Living Lab
environments. In these physical spaces (room, building or district) artefacts
(devices, technologies or services) are tested by users with or without their direct
intervention (Bendavid, et. al, 2012; Budweg, et al., 2012; Buhl, et al, 2017;
Flammini, et al., 2018; Perentis, et al., 2017; Schuurman, et al, 2011;). With the
emergence of user-centered design in the late 1980, “Usability evaluation with
real users became a key part of product development” (Nielsen, 1994).
Qualitative and quantitative data collection techniques are used to assess the
effectiveness of technologies (Bassoppo-Moyo, T. C.,2010).

To assess the users’ acceptance of a Zero Emission Building in the Trondheim

Living Lab, Korsnes, et al. (2017) used a quasi-experiment design, qualified by
the authors as a variant of a qualitative experiment. The authors collected data
through a mix-method approach in which qualitative data is complemented by
quantitative data like the measurement of energy consumption or the outside and
inside temperature.

372
3 Methodological Development
3.1 Quasi-experiment

An experiment is, according to William Ralph Inge, “a test under controlled

conditions that is made to demonstrate a known truth, examine the validity of a
hypothesis, or determine the efficacy of something previously untried” (Shadish,
et al., 2002).

Schuurman et al. (2013) propose a quasi-experimental approach for Living Lab

projects. To this end, they include a pre-measurement, an intervention (a real-
life experiment) and a post- measurement. For them, evaluation phase can be
seen as “the assessment of the impact of the experiment with regards to the
current state in order to iterate the future state” (Task, et al., 2017). It enables to
generate a “post-measurement of the intervention and compare it to the ‘pre-
measurement’ benchmark, illustrating the potential impact and added value
created by the innovation” (Task, et al., 2017).

3.2 Naturalistic and analogue observation

“Naturalistic inquiry differs from conventional science in minimizing constraints
on antecedent conditions (controls) and on output (dependent variables).
Naturalistic inquiry is phenomenological rather than positivists” (Guba, 1978).
When the experimental approach is implausible, it offers an alternative for
researchers. In a naturalistic experiment, data is collected under natural
conditions, i.e. not in the lab (Guba, 1978).

The naturalistic observation relates here mainly to the field of human ethology,
especially for the case of experiments (Eibl-Eiblsfeldt, 1989). Actually, the
ethological experiment falls into the category of quasi-experiment. It means that
we are not able to control all the variables present in the experiment, as it is the
case with pure laboratory experiments. Our goal is not to generalize findings but
rather to discover new behavioural patterns. There is also no standardized
approach. In a Living Lab setting, Fragnière et al. (2017) used naturalistic
observation and ethogram to test their hypothesis that queue structuring can
have a positive impact on wait time perception. In a Living Lab context, analogue
observation could be more appropriated (Norton & Hope, 2001).

3.3 A quasi-experimental research design framework using naturalistic to

evaluate the performance of a Living Lab output
As the Living Lab output is no more than a set of hypotheses addressing the
ways in which users will interact with the service or the object, it can be evaluated
or tested through a quasi- experiment with a control group and an experimental
group. In a quasi-experiment, participants are not randomly assigned to a case
(Shadish et al., 2002), after which the results of the two groups are compared
(Campbell & Stanley, 2015). Knemeyer and Naylor (2011) have identified the
necessary conditions for quasi-experiments to establish the causality of two
tested variables. First, with all other things being equal, solely the independent
variable is changed, and second, the independent variable might or not affect
the participants, and thus the dependent variable might also change.

373
We can, however, describe it as a process containing the four following main
steps:
• Hypothesis formulation. We start by formulating a hypothesis related to a
given human behaviour. This hypothesis is the answer to the stated
research question. In a Living Lab process, the hypotheses are formulated
during the co-creation and the exploration phase.
• Hypothesis “operationalization”. We then “operationalize” the hypothesis
through a simulation of the ecosystem under study to obtain a prediction
of it. This correspond to the experimentation phase of the Living Lab
process.
• Data collection and analysis. We collect and analyse the data of the
experiment (e.g. a simulation of passengers waiting at airport security
gates). This is the evaluation phase.
• Conclusion. We compare the statistical results to the prediction and also
to other findings in the literature to assess the validity of the hypothesis.

4 The operationalization of the quasi-experimental research

design framework - A case study
This case study illustrates how to operationalize the quasi-experimental research
design framework. Based on a project who aims at introducing Autonomous
Vehicles (AVs) in a city in Switzerland, we illustrate how quasi-experiment could
be designed according to our R&D service approach to be able to test hypothesis
related to use of Autonomous Vehicles (AVs) for improving the mobility of people
with disabilities and reduced mobility. This case study shows how the quasi-
experimental research design framework we develop would structure such a
problematic. We focus here solely on the aspect of the framework and not on the
statistical tests of the quasi-experiment.

4.1 Context of the case study

4.1.1 The challenge of autonomy of people with disabilities and reduced mobility
One of the numerous challenges faced by people with disabilities and reduced
mobility is the accessibility of public space. Improving the mobility of people with
disabilities give them autonomy and reduce the risk of social isolation (Simplican
et al., 2015). The mobility of people with disabilities is much more than a
technological challenge. Indeed, regarding our ageing society, it is a necessary
and crucial democratic debate and a process of empowerment (Lord &
Hutchison, 1993).

“In the mobility sector, a large number of new technologies such as autonomous
vehicles (AVs) and services are emerging. AVs involve not only passengers, but
also authorities, manufacturers, public transportation companies, law
enforcement officials, drivers, pedestrians and merchants” (Ramseyer, et al.,
2019). For people with disabilities, the introduction of autonomous vehicles (AVs)
as public transportation represents a challenge. The objective is to deliver a
much better quality of life by identifying and providing opportunities for better
social inclusion. Furthermore, it will be necessary to better define the city's
general traffic plan and the appropriate layout of pedestrian streets tacking into
account the need of people with disabilities while integrating these new means

374
of transportation. Therefore, it is important to design transportation modes that
consider the needs of people with disabilities.

For a successful implementation, the service provided by a new technology must

be a relevant problem solving that changes the users’ life. As such, new services
must be pre-tested and validated by users not only on performance measures
but also on perceptions. Therefore, it is essential to consider the factors that will
affect the adoption and user acceptance of these novel systems and to integrate
as well the felt needs of people with disabilities (McCreadie & Tinker, 2005).

4.1.2 A Swiss Living Lab dedicated to handicap

In Switzerland, about 1’300’000 people find themselves with a handicap33.
Switzerland as adopted in 2014 the Convention on the Rights of Persons with
Disabilities34. This convention is an international agreement that in principle
guarantees people with disabilities the equality of experience in all areas of
citizenship (Darcy, 2012). Furthermore, the laws of the Swiss Confederation
dictate equal opportunities for all. Particularly, the Federal Act of 13 December
2002 on the elimination of discrimination affecting persons with disabilities
(Disabled Persons Equality Act - LHand) provides remedies and rights of legal
action to make it easier for individuals with disabilities to assert their rights.

In 2018, the HES-SO Valais-Wallis and the Innovation Centre for Assistive
Technologies (IATLab) founded the Living Lab Handicap (LLH) with the
collaboration of ASA Handicap Mental and the Foundation for Research in
Favour of People with Disabilities (FRH). This initiative connects people with
disabilities, their families and caregivers, with scientists, companies, public
authorities and all the other stakeholders (care institutions, charitable
associations, etc.) interested in collaborating in the field of disability and the co-
design of innovative solutions. “Nothing About Us Without Us”35. This sentence
perfectly illustrates the vision of this Swiss Living Lab. Being at the center of the
co-design process and the vision of empowerment, participants will elaborate
with all the stakeholders detailed scripts in order to co-design adequate and
useful products.

4.1.3 Autonomous vehicles on public road in Switzerland

In the city of Sion in Switzerland, AVs are in function since 2016 on public road
(Eden, 2017). The pilot project of AVs begins in the summer of 2016 in the old
city of Sion. In 2019, two routes are in function. The first conducts passenger
from the rail station to the old town. The second drives in the old town. The AVs
are in function from Wednesday to Sunday from 7 to 10 am and from 1 to 6 pm.
The goal of this pilot project is to understand if AVs could offer new services and
forms of mobility in regions currently deprived of public transport and to test if the
introduction of autonomous shuttles in the public space is technically and
operationally feasible, while offering added-value to customers.

33 https://www.bfs.admin.ch/bfs/fr/home/statistiques/sante/etat-sante.assetdetail.7347551.html
34
www.humanrights.ch
35
https://en.wikipedia.org/wiki/Nothing_About_Us_Without_Us

375
4.2 Quasi experiment framework implementation
4.2.1 Hypothesis formulation
People suffering from reduced mobility primarily use wheelchairs. According to
the World Health Organization (WHO), the prevalence of disability ranges
between 12 - 18% of the total population and the majority of people with mobility
issues use wheelchairs 36. Our quasi- experiment tests the hypothesis that the
use of Autonomous Vehicles (AVs) improves the mobility of people with
disabilities.

4.2.2 Hypothesis “operationalization”

To operationalize this hypothesis – the use of Autonomous Vehicles (AVs)
improves the mobility of people with disabilities – we recruited three users with
reduced mobility who used wheelchair. The three users had to use the AV in the
Sionold town route.

4.2.3 Data collection and analysis

In the AV, only one user in wheelchair can embark at the same time. The
experiment was run three times. Each time, two same observers collecting data
during the whole experiment accompanied the three different users. The
collected data are pictures, movies, observation notes and phenomenological
quotes. Then, this data are analysed with the help of software like RQDA and
NVIVO.

4.2.4 Conclusion
Even if this case study is solely used to illustrate how to operationalize the quasi-
experimental research design framework, the collected data are interesting.
Indeed, without the help of a third person, it is impossible for a people with
disabilities to use the AV. The ramp to embark and disembark has to be installed
manually (this feature will be automatized in the next generation of AVs) and is
too steep. Furthermore, in many buses stops it was impossible for the user to
embark or disembark due to the lack of space between the bus and an obstacle
(e.g. building). Consequently, here based on these results, we are not able to
confirm the hypothesis that the use of Autonomous Vehicles (AVs) improves the
mobility of people with disabilities. However, our goal here was just to show how
our R&D for services framework can be operationalized in a handicapped Living
Lab to ultimately test the validity and performance of new research hypotheses.

5 Discussion
Even if Living Lab is now an essential trend in innovation development, the
initiative needs tools to evaluate the performance of its output (Schuurman, et
al., 2015). In this paper, we proposed to structure Living Lab projects (meso-
level) as an R&D for services. This rigorous and systematic framework permits
the scientific validation of operationalized hypotheses through quasi-experiment
protocols. In particular, we developed a quasi-experimental research design

36 WHO. (2017b). World report on disability. http://www.who.int/disabilities/world_report/2011/report/en/

376
framework using analogue observation (micro-level) for the evaluation of a Living
Lab output performance.

In the case study, we illustrated how to operationalize the quasi-experimental

research design framework. Our quasi-experiment was designed to test
hypothesis that the use of Autonomous Vehicles (AVs) improves the mobility of
people with disabilities and reduce mobility. Even if this case study is solely used
to illustrate how the quasi-experimental research design framework we develop
would structure such a problematic, results here could not confirm our starting
hypothesis.

For a Living Lab manager, this quasi-experimental research design framework

using analogue observation do not require much efforts and specific
competences (e.g. statistics). However, it is efficient and answers an important
question: do the living lab output solve a problem for the users and improve their
situation? Furthermore, the observation data collected during the quasi-
experiment also represents insights for a new co-creation iteration.

Even if these results based on qualitative data are interesting, the use of
quantitative data will be complementary. In future work, we will use quantitative
data collection techniques to reinforce our quasi-experimental research design
framework in order to enhance our results through statistical testing.

References
Ballon, P., Van Hoed, M., & Schuurman, D. (2018). The effectiveness of involving
users in digital innovation: Measuring the impact of Living Labs. Telematics
and Informatics, 35(5), 1201–1214.
https://doi.org/10.1016/j.tele.2018.02.003
Bendavid, Y., & Cassivi, L. (2012). A ‘Living Laboratory’environment for exploring
innovative RFID-enabled supply chain management models. International
Journal of Product Development, 17(1-2), 94-118.
Brown, T., & Katz, B. (2011). Change by design. Journal of product innovation
management, 28(3), 381-383.
Budweg, S., Lewkowicz, M., Müller, C., & Schering, S. (2012). Fostering Social
Interaction in AAL: Methodological reflections on the coupling of real
household Living Lab and SmartHome approaches. I-Com Zeitschrift Für
Interaktive Und Kooperative Medien, 11(3), 30–35.
https://doi.org/10.1524/icom.2012.0035
Buhl, J., Hasselkuß, M., Suski, P., & Berg, H. (2017). Automating behavior? : An
experimental Living Lab study on the effect of smart home systems and
traffic light feedback on heating energy consumption. Retrieved from
https://epub.wupperinst.org/frontdoor/index/index/docId/6770.
Campbell, D. T., & Stanley, J. C. (2015). Experimental and quasi-experimental
designs for research. Ravenio Books.

377
Darcy, S., 2012. Disability, Access, and Inclusion in the Event Industry: A Call for
Inclusive Event Research. Event Management 16, 259–265.
doi:10.3727/152599512X13461660017475
Dvarioniene, J., Gurauskiene, I., Gecevicius, G., Trummer, D. R., Selada, C.,
Marques, I., & Cosmi, C. (2015). Stakeholders involvement for energy
conscious communities: The Energy Labs experience in 10 European
communities. Renewable Energy, 75, 512–518.
https://doi.org/10.1016/j.renene.2014.10.017
Eden, G., Nanchen, B., Ramseyer, R., & Evéquoz, F. (2017). On the Road with
an Autonomous Passenger Shuttle: Integration in Public Spaces.
Proceedings of the 2017 CHI Conference Extended Abstracts on Human
Factors in Computing Systems, 1569–1576.
https://doi.org/10.1145/3027063.3053126
Eibl-Eiblsfeldt I. (1989). Human ethology. Aldine de Gruyter, NY.
Flammini, A., Pasetti, M., Rinaldi, S., Bellagente, P., Ciribini, A. C., Tagliabue, L.
C., … Pedrazzi, G. (2018). A Living Lab and Testing Infrastructure for the
Development of Innovative Smart Energy Solutions: the eLUX Laboratory
of the University of Brescia. 2018 AEIT International Annual Conference,
1–6. https://doi.org/10.23919/AEIT.2018.8577329.
Fragnière, E., Nanchen, B., & Sitten, M. (2012). Performing service design
experiments using ethnomethodology and theatre-based reenactment: a
Swiss ski resort case study. Service Science, 4(2), 89-100.
Fragnière, E., Cimmino, F., Emery, L., Héritier, M., Kambly, M., Nanchen, B., &
Ramseyer, R. (2017). An ethological experiment to improve airport security
gate process reliability: Understanding time perception and personal
awareness of older adult travelers. 2017 2nd International Conference on
System Reliability and Safety (ICSRS), 68–73.
https://doi.org/10.1109/ICSRS.2017.8272799
Fragnière, E., Nanchen, B., Héritier, M., Ramseyer, R., Dubosson, M., & Mastelic,
J. (2018). Developing a mutualized R&D for network organizations based
on Living Lab methods: the transformation of Sion military airport into a
commercial airport. Zenodo. http://doi.org/10.5281/zenodo.1434966
Guba, E. G. (1978). Toward a Methodology of Naturalistic Inquiry in Educational
Evaluation. CSE Monograph Series in Evaluation, 8. Center for the Study
of Evaluation, Graduate School of Education, University of California, Los
Angeles, California 90024.
Knemeyer, A. M., & Naylor, R. W. (2011). Using behavioral experiments to
expand our horizons and deepen our understanding of logistics and supply
chain decision making. Journal of Business Logistics, 32, 296-302.
Liedtke, C., Welfens, M.J., Rohn, H., Nordmann, J. (2012). Living Lab: User-
Driven Innovation for Sustainability. International Journal of Sustainability
in Higher Education, 132, 106-118.

378
Lord, J., & Hutchison, P. (1993). The Process of Empowerment: Implications for
Theory and Practice. Canadian Journal of Community Mental Health,
12(1), 5–22. https://doi.org/10.7870/cjcmh-1993-0001
Lovelock, C., & Gummesson, E. (2004). Whither Services Marketing?: In Search
of a New Paradigm and Fresh Perspectives. Journal of Service Research,
7(1), 20–41. https://doi.org/10.1177/1094670504266131.
Mastelic, J. (2019), Stakeholders’ engagement in the co-design of energy
conservation interventions: The case of the Energy Living Lab, Doctoral
Thesis, University of Lausanne.
McCreadie, C., & Tinker, A. (2005). The acceptability of assistive technology to
older people. Ageing & Society, 25(1), 91–110.
Miles I., (2007). Research and development (R&D) beyond manufacturing: the
strange case of services R&D. R&D Manag, 37, 249–268.
Mulder, I., Velthausz, D., & Kriens, M. (2008). The Living Labs harmonization
cube: Communicating Living Lab’s essentials. The Electronic Journal for
Virtual Organizations and Networks, 10, 1-14.
Nielsen, J. (1994). Usability engineering. Elsevier.
Norton, P. J., & Hope, D. A. (2001). Analogue observational methods in the
assessment of social functioning in adults. Psychological Assessment,
13(1), 59–72. https://doi.org/10.1037/1040-3590.13.1.59
OECD. (2015). Frascati Manual 2015: Guidelines for Collecting and Reporting
Data on Research and Experimental Development, The Measurement of
Scientific, Technological and Innovation Activities, OECD Publishing,
Paris. DOI: http://dx.doi.org/10.1787/9789264239012- en
OECD. (2018). Oslo Manual 2018. Guidelines for collecting, reporting and using
data on innovation.
Pallot, M., Trousse, B., Senach, B., & Scapin, D. (2010). Living Lab Research
Landscape: From User Centred Design and User Experience towards User
Co-creation. Presented at the First European Summer School « Living
Labs », Paris.
Perentis, C., Vescovi, M., Leonardi, C., Moiso, C., Musolesi, M., Pianesi, F., &
Lepri, B. (2017). Anonymous or Not? Understanding the Factors Affecting
Personal Mobile Data Disclosure. ACM Trans. Internet Technol., 17(2),
13:1–13:19. https://doi.org/10.1145/3017431.
Ramseyer, R., Cimmino, F., Emery, L., Grèzes, S., Grèzes, V., Nanchen, B., ... &
Fragnière.
E. (2019, April). Using Phenomenology to Assess Risk Perception of a New
Technology in Public Transportation the Case of the Autonomous Vehicles
as Mobility as a Service (MaaS) in Switzerland. In 2018 3rd International
Conference on System Reliability and Safety (ICSRS) (pp. 289-293). IEEE.
Shadish, W.R. Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-
experimental designs for generalized causal inference. Wadsworth
Cengage learning, Boston.

379
Sawatani Y., Fujigaki Y. (2015). Service R&D Program Design Aiming at Service
Innovation, in : Service Systems Science, Translational Systems Sciences,
2, 163-174.
Schneider, J., & Stickdorn, M. (2011). This is service design thinking: basics,
tools, cases. Wiley.
Schuurman, D., De Moor, K., De Marez, L., & Evens, T. (2011). A Living Lab
research approach for mobile TV. Telematics and Informatics, 28(4), 271–
282. https://doi.org/10.1016/j.tele.2010.11.004.
Schuurman, D., Baccarne, B., Kawsar, F., Seys, C., Veeckman, C., De Marez, L.,
& Ballon, P. (2013). Living Labs as quasi-experiments: results from the
Flemish LeYLab. In XXIV ISPIM Conference: Innovating in Global Markets:
Challenges for Sustainable Growth.
Schuurman, D., & De Marez, L. (2015). Living Labs: a structured approach for
implementing open and user innovation. In 13th Annual Open and User
Innovation Conference.
Schuurman, D., De Marez, L., & Ballon, P. (2015). Living Labs: a systematic
literature review. Open Living Lab Days 2015, Proceedings. Presented at
the Open Living Lab Days 2015. Retrieved from
http://hdl.handle.net/1854/LU-7026155.
Simplican, S.C., Leader, G., Kosciulek, J., Leahy, M., 2015. Defining social
inclusion of people with intellectual and developmental disabilities: An
ecological model of social networks and community participation.
Research in Developmental Disabilities 38, 18–29.
doi:10.1016/j.ridd.2014.10.008.
Ståhlbröst, A. (2012). A set of key-principles to asses the impact of Living Labs.
International Journal of Product Development, 17(1–2), 60–75.
Sundbo J. (1997). Management of innovation in services. Serv Indust J, 17(3),
432–455.

Task, T., Malmberg, K., Vaittinen, I., Ståhlbröst, A., LTU, A. S., Breuer, J., &
Carter, D. (2017). D2. 2: Living Labs Methdology Handbook.
Wacker, J. G. (1998). A definition of theory: Research guidelines for different
theory building.

380
381
 Identifying Living Lab orchestrators’ individual‐
level skills
Anne Äyväri*1, Tuija Hirvikoski1 and Heidi Uitto1

* Corresponding author
1 Laurea University of Applied Sciences, Finland

Category: Research-in-progress

Abstract
This paper sheds light on the individual skills needed to orchestrate open Living
Labs networks and activities. Since orchestrators (also called mediators) are
people working on the interface of the macro, meso and micro levels of Living
Labs, and in between various stakeholders such as universities, organizations,
NGOs and citizens, specific skill sets are needed in order to enhance
inclusiveness, balance, and communication among the different parties and to
improve the sustainability of the Living Labs’ projects according to the
responsible research and innovation principles. Based on the literature, the skills
are classified in three partially overlapping bundles: first, skills in building
relationships, networks and ecosystems; second, skills in maintaining them; and
finally, skills in executing multistakeholder innovation processes. As a summary
of the literature review, a preliminary framework of orchestrator skills is
presented.

Keywords: Living Lab, multistakeholder co‐creation, orchestrator, skills

382
1 Introduction
Since 2011, the European research and innovation policy has emphasized
Responsible Research and Innovation (RRI) (von Schomberg, 2013; von
Schomberg, 2019) as a core criterion for public funding for research,
development and innovation (RDI) activities. RRI emphasizes the inclusion of all
societal stakeholders, including citizens, throughout the innovation process. This
perception is coherent with the open Living Labs approach, in which companies,
public players, universities, the civic society, and citizens interact and co‐learn
by co‐creating knowledge and adding value in order to unleash innovation given
the ability to laterally communicate with all the stakeholders. According to the
European Network of Living Labs (2019), Living Labs (LLs) are defined as user‐
centered, open innovation ecosystems that are based on a systematic user co‐
creation approach, integrating research, and innovation processes in real life
communities and settings.

Living Labs have been widely scrutinized as open innovation networks.

Consequently, a concept of multistakeholder Living Lab collaboration has
evolved over time. However, research has also pointed out challenges and
paradoxes that occur while dealing with open innovation networks. According to
Klerkx and Aarts (2013), the following three challenges and paradoxes require a
network orchestration of the participating actors. First, balancing new
relationships and existing relationships i.e. exploiting weak ties and fostering
strong ties; second, determining the most appropriate way of interacting with
other organizations, and third, balancing informal and formal relationships. We
maintain that determining and enhancing the right skill set of the orchestrator is
critical in order to solve these grievances pointed out by Klerkx and Aarts (2013).

Living Labs are seen as intermediary organizations (Almirall & Wareham, 2011)
and the Living Lab methodology emphasizes competences to facilitate co‐
creation processes within those organizations (Dell'Era & Landoni, 2014).
Likewise, the demand for skilled co‐creation facilitators, such as T‐shaped
innovators, has been widely recognized for their ability to enhance innovation
within industries (Barile, Saviano & Simone, 2015; Demirkan & Spohrer, 2015).

However, the results of recent systematic reviews on Living Lab literature and
studies (Habibipour, 2018; McLoughlin, 2018; Westerlund et al, 2018) imply that
extant literature on individual and organizational level competences needed in
orchestrating the Living Lab networks and implementing activities is very scarce.
Thus, to address the knowledge gap, the aim of this paper is to identify individual‐
level skills needed to orchestrate Living Lab networks and ecosystems and to
execute Living Lab projects. In other words, this paper focuses on the individuals’
skills related to open Living Labs activities, where the ability to motivate and
include various actors in the network and co‐creation activities are vital. The
collaboration among various stakeholders is seen as a key success factor in joint
problem solving and the innovation process based on RRI (Gray & Purdy, 2018,
p.5; Lusch, Vargo & Tanniru, 2010; Von Schomberg, 2013; Von Schomberg,
2019). We acknowledge the importance of organizational capabilities (see more
e.g. Kazadi, Lievens & Mahr, 2016; Schuurman, 2015, p.314; Vontas &

383
Protogeros, 2009); even though that is not the focus of this paper (see the
Research design section below).

Due to the knowledge gap in Living Lab literature, we widened our literature
search to cover the following themes: innovation ecosystem & orchestration
skills, multistakeholder co‐creation skills, networking skills, mediating skills (in
the context of a multidisciplinary project), facilitating skills, and T‐shaped
innovator skills. Following the three‐layered Living Lab model by Schuurman
(2015, p.316), we aimed at identifying individual‐level skills at macro (Living Lab
constellation), meso (Living Lab innovation project), and micro level (Living Lab
methodology).

In the next section, we discuss the skills identified in the above‐mentioned

streams of literature. The section concludes with a preliminary framework of the
Living Lab orchestrators’ individual‐level skills. Thereafter, we present the
research design of the larger study; this paper being the first step in the study.
The article concludes with a short summary and discussion on potential
contributions of the preliminary framework.

2 Literature review
This paper presents the Living Lab orchestrators’ skills in three, partially
overlapping bundles (Figure 1): first, skills in building relationships, networks,
and ecosystems; second, skills in maintaining them; and finally, skills in
executing multistakeholder innovation processes. It is emphasized that the skills
presented in one bundle, e.g. maintaining relationships, networks, and
ecosystems, are not exclusively needed in the particular task area, but they
might also support successful execution of Living Lab projects.

Skills in

Figure 1. Living Lab orchestrators’ skills categorized according to the three task areas
2.1 Skills in building relationships, networks, and eco‐systems

384
Individuals’ skills in building relationships, networks, and eco‐systems in Living
Lab constellations are categorized in two bundles of skills: visioning skills and
networking skills.

Visioning skills are needed to outline challenges and opportunities for

multistakeholder innovation processes; the vision guides decisions on which
opportunities should be seized and who should be contacted and selected for
partnership (Gray & Purdy, 2018; Jyrämä & Äyväri, 2005; Spekman, Isabella &
MacAvoy, 2000; Äyväri & Jyrämä, 2007). In addition, visioning skills are
important in fostering a willingness to see opportunities or competitive
advantages and interdependences in multistakeholder partnerships (Gray &
Purdy, 2018; Ritala, Armila & Blomqvist, 2009). Entrepreneurial skills e.g.
sensing and discovering new opportunities, are similar to visioning skills (Ritala,
Armila & Blomqvist, 2009). Wishful thinking and open‐minded gifts are horizontal
capabilities characterizing T‐shaped innovators. The former refers to “a thought
that emerges from a wish and creates a new opportunity… and is capable of
triggering new future scenarios” (Barile, Saviano & Simone, 2015, p.1188). An
ability to be open‐minded refers to exploring unknown phenomena, seeking new
experiences, and looking at the same phenomena through a different light and
with different perspectives (Barilo, Saviano & Simone, 2015).

Individual level networking skills have been identified in several studies (e.g.
Kazadi, Lievens & Mahr, 2016; Ritter & Gemünden, 2003; Äyväri & Jyrämä,
2007) and been acknowledged for consisting of complex routines and individual
skills (Kazadi, Lievens & Mahr, 2016). Contact seeking abilities (Äyväri & Jyrämä
2007) as well as an ability to use one’s own contacts (Kazadi, Lievens & Mahr,
2016; Äyväri & Jyrämä, 2007) and the partners’ contacts (Äyväri & Jyrämä 2007)
to identify potential new partners have been identified in previous studies.

The Living Labs as innovation networks combine dispersed resources,

knowledge, and capabilities (Kazaki, Lievens & Mahr, 2016) in public‐private‐
people partnerships (Westerlund & Leminen, 2011). In multistakeholder co‐
creation ecosystems (as Living Labs have been described by Westerlund &
Leminen, 2011), all actors are resource‐integrators that collectively co‐create the
shared value (Pera, Occhiocupo & Clarke, 2016). Hence, the following skills are
needed when building relationships: stakeholder competence mapping skills
(Kazadi, Lievens & Mahr, 2016), an ability to identify the needs of one’s own
organization and then inform other actors of those needs (Äyväri & Jyrämä,
2007), and skills in diverse recognition and ensuring inclusiveness (Azadegan &
Kolfschoten, 2014). Furthermore, an ability to understand certain stakeholders’
interests and aims and take them into consideration when communicating them
to other potential collaborators has been identified by Jyrämä and Äyväri (2015).
Personal influencing and motivating skills have also been emphasized when
inviting others to join multistakeholder innovation processes (Ritala, Armila &
Blomqvist, 2009). Another aspect in networking is brokering or bridging (Äyväri,
Jyrämä & Hirvikoski, 2018), i.e. supporting and enhancing relationship building
among different actors within one’s own network by sharing contact information
and introducing actors to each other.

385
2.2 Skills in maintaining relationships, networks, and eco‐systems
Next, we discuss the individual‐level skills of Living Lab orchestrators in
maintaining relationships, networks, and eco‐systems. However, we wish to
emphasize that the following skills also support relationship building and
executing Living Lab projects.

Communication and negotiation skills are prerequisites for fruitful interaction and
collaboration. Besides information sharing, communication skills (Azadegan &
Kolfschoten, 2014; Jyrämä & Äyväri, 2015; McFadzean, 2002; Ritala, Armila &
Blomqvist, 2009) are needed in building shared interpretations and in the framing
and reframing of meaning making (Gray & Purdy, 2018, p.197‐198; Purdy, Ansari
& Gray, 2017; see also Pearce, 2008). Negotiation skills refer to the
orchestrator’s capabilities of taking the partners’ interests into consideration
(Barile, Saviono & Simone, 2015; Äyväri & Jyrämä, 2007) and to learn from
tension between stakeholder interests in order to mediate those differences
(Pera, Occhiocupo & Clarke, 2016). In addition, they include balancing skills
(Ritala, Armila and Blomqvist, 2009) to maintain an equal distribution of power
among different actors (Gray & Purdy, 2018, p.183). In multistakeholder
innovation processes, conflicts can seldom be avoided, hence, conflict
management skills (Azadegan & Kolfschoten, 2014; Gray & Purdy, 2018, p. 87;
IAF, 2019) are vital for maintaining relationships in ecosystems.

Social skills (Ritala, Armila & Blomqvist, 2009; Ritter & Gemünden, 2003; Äyväri
& Jyrämä, 2007) are linked to communication and negotiation skills. In the
context of Living Labs, social skills are especially related to creating an empathic
and trusting atmosphere, they include an ability to sense other actors’ feelings,
an ability to show empathy, and listening skills (Barile, Saviano & Simone, 2015;
Jyrämä & Äyväri, 2015). Social skills refer to social flexibility (Pera, Occhiocupo
& Clarke, 2016; Äyväri & Jyrämä, 2007) supporting adaption to diverse
backgrounds (Pera, Occhiocupo & Clarke, 2016) and building an appreciation
among cross sector partners (Gray & Purdy, 2018, p.87). Social skills help to
maintain shared ownership and consensus building in collaboration, thus
increasing commitment in partnerships (Gray & Purdy, 2018, p.183).

Coordination skills are needed when coordinating the activities of the actors in
innovation networks and eco‐systems and when systematizing the routines
linked to coordination (Äyväri & Jyrämä, 2007). When orchestrating and
coordinating the multistakeholder collaboration, skills in lateral thinking involving
re‐ combining and re‐setting (Barile, Saviano & Simone, 2015) might become
useful. Creativity and innovation capabilities to create and sustain a participatory
environment for the ecosystem (Jyrämä & Äyväri, 2015) and to find original
solutions (Pera, Occhiocupo, Clarke, 2016) that are effective, are needed to
ensure the actors’ willingness to continue with collaborative actions.

When considering the Living‐Lab‐as‐service approach, client‐centric service

modification skills might be needed to ensure relationship continuity (cf. Äyväri
& Jyrämä, 2007). Finally, we propose that Living Lab orchestrators need to have
an ability to manage time (Äyväri & Jyrämä, 2007) in order to reserve enough

386
time for nurturing the relationships, thus supporting building trust and
commitment among network actors.

2.3 Skills in executing living lab projects

When executing multistakeholder innovation processes, Living Lab orchestrators
need project management skills consisting of e.g. planning skills, human
resource management skills, skills related to funding and financial issues (see
literature review on project management skills in Hwang & Ng, 2013). Extant
studies on project management skills indicate that they cover a wide array of
both management and leadership skills, and some of them are context specific.
In Living Lab projects, we wish to emphasize the importance of the LL context
specific skills (to be discussed next), hence they are not included in the bundle
of project management skills.

Collaborative activities and co‐creation in group settings are the core of Living
Labs. Therefore, facilitating skills are necessary in managing group processes.
Azadegan and Kolfschoten (2014) have developed an assessment framework
for practicing facilitators. One of the starting points of their study is the list of core
facilitator competencies provided by International Association of Facilitators
(2019). A competent facilitator plans appropriate group processes, evokes group
creativity, facilitates group self‐awareness of the task, and guides the group to
consensus and desired outcomes (Azadegan & Kolfschoten 2014, International
Association of Facilitators 2019). It has been suggested that in addition to skills
in group dynamics, skills in problem‐solving and decision‐making processes are
part of the facilitators’ competency (McFadzean 2002).

In the Living Lab literature, learning, knowledge‐creation and knowledge transfer

have been described as Living Lab activities (e.g. Hakkarainen & Hyysalo, 2016,
Voytenko et al., 2016). A Living Lab project manager supports all the other
actors’ efforts in learning, knowledge co‐creation and transfer with his or her
pedagogical skills to build learning spaces for creating shared vocabulary and
shared meanings (Jyrämä & Äyväri, 2015). Furthermore, a skilful orchestrator
possesses an ability to share one’s own knowledge, is willing to learn from others
(Äyväri & Jyrämä, 2007), and open to new ideas originating from other actors
(Pera, Occhiocupo & Clarke, 2016). Learning and reflection skills (Jyrämä &
Äyväri, 2015), knowledge‐ seeking capabilities (Barile, Saviano & Simone,
2015), and employee level absorptive capacity (Kazadi, Lievens & Mahr, 2016)
refer to competencies in personal growth and development (McFadzean, 2002,
see also IAF, 2019).

According to Schuurman’s three‐layered model of Living Labs (2015, p.317),

different research steps are taken to generate user input and contribution on the
micro level. User involvement methods and tools are provided by user innovation
research (Schuurman & de Marez, 2015); hence the skills in this area are
needed. In addition, research skills are needed in making conclusions based on
the context and the opportunities available (Jyrämä & Äyväri, 2015). Besides
research skills, a good command of design methods and tools is a valuable asset
when executing Living Lab projects with multiple stakeholders, including users.

387
Evaluation is just one of the tasks for the Living Lab project managers. For
example, McCormick (2016) maintains that “evaluation of the actions and
impacts of Urban Living Labs is important to feedback the results, and revisit and
refine the goals and visions over time. Evaluation underpins the ability of ULLs
to facilitate formalized learning amongst the participants”. Thus, learning and
evaluation skills are intertwined.

Depending on the context of the innovation co‐created, tested, or validated

during the execution of a Living Lab project, skills related to business model
development and commercialization or skills related to upscaling or
mainstreaming (in the context of social innovation) might also be needed.

Figure 2 summarizes the skills described above as a preliminary framework of

the Living Lab orchestrators’ individual‐level skills.

Building relationships, Maintaining Executing Living Lab

networks, and relationships, networks projects (meso and
ecosystems (macro and ecosystems (macro micro level)
and meso level) and meso level)

Visioning skills Communication skills Project management

Networking skills
Negotiation skills
Social skills
Coordination skills
Innovation and lateral
thinking skills
Client-centric service
modification skills
Ability to manage time
skills
Facilitation skills
Learning and knowledge
co-creation and transfer
skills
Research skills
Design skills (tools,
methods)
Evaluation skills
Commercialization and
upscaling skills

Figure 2. A preliminary framework of the Living Lab orchestrators’ individual‐level skills

3 Research design
The literature review presented in this paper is the first step in an upcoming larger
study on individual‐level skills and organizational capabilities needed in
orchestrating Living Lab networks and ecosystems. The study is part of a project
called Co‐creation Orchestration (CCO), an evidence‐based governance model
enhancing the ecosystem’s value in co‐creation, knowledge transfer, and
business development funded by the Finnish Ministry of Education and Culture
and Laurea University of Applied Sciences. The project will run until the end of
2021.

388
We will proceed to gather insights on individual‐level skills through thematic
interviews and interactive workshops with experienced Living Lab orchestrators
or mediators. We will use abductive reasoning (Shank, 2002, p.119) to refine the
preliminary framework (Figure 2). After finalizing the qualitative phase of the
study, we will conduct a survey aiming for results that are more generalizable.

A similar kind of multi‐staged research process will be conducted in order to

identify the organizational capabilities needed to orchestrate multistakeholder
innovation ecosystems.

4 Discussion
Addressing the challenges that have emerged from the existing Living Labs
experiences, this paper has classified and described skills that are vital for
orchestrating Living Lab networks and activities, with a focus on the
orchestrator’s role. The paper takes into consideration the sustainability of the
ecosystems and also addresses the skills needed in navigating the networks in
the long run. The individual‐level skills are classified into three partially
overlapping bundles: first, skills in building relationships, networks and
ecosystems; second, skills in maintaining them; and finally, skills in executing
multistakeholder innovation processes. The findings of this paper are beneficial
for RDI and HR strategy building within all types of organizations and
multistakeholder projects.

Acknowledgements
The authors are thankful for the financial support from The Ministry of Education
and Culture (Finland) and Laurea University of Applied Sciences in conducting
the study.

389
References
Almirall, E. & Wareham, J. (2011). Living labs: arbiters of mid‐ and ground‐level
innovation. Technology Analyses & Strategy Management, 23(1), 87-102.
http://dx.doi.org/10.1080/09537325.2011.537110.
Azadegan, A. and Kolfschoten, G. (2014). An assessment framework for
practicing fasilitator. Group Decision and Negotiation, 23(5), 1013-1045.
Barile, S., Saviano, M. &Simone, C. (2015). Service economy, knowledge, and
the need for T‐shaped innovators. World Wide Web, 18(4), 1177-1197.
Dell'Era, C. & Landoni, P. (2014). Living Lab: A Methodology between User‐
Centred Design and Participatory Design. Creativity and Innovation
Management, 23(2), 137‐154.
Demirkan, H. & Spohrer, J. (2015). T‐shaped innovators: Identifying the right
talent to support service innovation. Research‐Technology Management,
58(5), 12 15.
European Network of Living Labs (2019). What are Living Labs?
https://enoll.org/about‐us/
Habibipour, A. (2018). Living Lab research: A state‐of‐the‐art review and steps
towards a research agenda. In Research and Innovation Conference
Proceedings 2018. Open Living Lab Days. Brussels: European Network of
Living Labs. 432‒447.
Hakkarainen, L. & Hyysalo, S. (2016). The Evolution of Intermediary Activities:
Broadening the Concept of Facilitation in Living Labs. Technology
Innovation Management Review, 6(1), 45-58.
https://timreview.ca/article/960
Hwang, B‐G. & Ng, W. J. (2013). Project management knowledge and skills for
green construction: Overcoming challenges. International Journal of
Project Management, 31(2), 272-284.
https://doi.org/10.1016/j.ijproman.2012.05.004
IAF, International Association of Facilitators. (2015). Core Facilitator
Competences. https://www.iaf‐
world.org/site/sites/default/files/IAF%20Core%20Competencies_1.pdf
(Retrieved 23.4.2019).
Jyrämä, A. & Äyväri, A. (2015). Art encountering society; identifying the skills. The
paper was presented at AIMAC 2015, Aix‐en‐Provence & Marseille,
France, June 28th – July 1st 2015, organized by The International
Association of Arts and Cultural Management, Aix‐Marseille University and
Kedge Business School.
Kazadi, K., Lievens, A. & Mahr, D. (2016). Stakeholder co‐creation during the
innovation process: Identifying capabilities for knowledge creation among
multiple stakeholders. Journal of Business Research, 69(2), 525-540.
Klerkx, L. & Aarts, N. (2013). The interaction of multiple champions in
orchestrating innovation networks: Conflicts and complementarities.
Technovation, 33(6), 193-210.

390
Lusch, R. F., Vargo, S. L., & Tanniru, M. (2010). Service, value networks and
learning. Journal of the Academy of Marketing Science, 38(1), 19–31.
McCormick, K. (2016). GUST Policy Brief: Urban Living Labs. GUST Project.
www.urbanlivinglabs.net
McFadzean, E.(2002). Developing and supporting creative problem solving
teams: part 2 facilitator competences. Management Decision, 40(6),
537 551.
McLoughlin, S., Maccani, G., Prendergast, D. & Donnellan, B. (2018). Living labs:
A bibliometric analyses. Paper presented at the 51st Hawaii International
Conference on System Sciences (HICSS), January 2 6, 2018, Hawaii,
USA. http://hdl.handle.net/10125/50452
Pera, R., Occhiocupo, N. & Clarke, J. (2016). Motives and resources for value co‐
creation in a multi‐ stakeholder ecosystem: A managerial perspective.
Journal of Business Research, 69(10), 4033-4041.
Pearce, W. B. (2008). Toward a new repertoire of communication skills for leaders
and managers. The Quality Management Forum, 34(4), 4-7.
Purdy, J., Ansari, S., & Gray, B. (2017). Are Logics Enough? Framing as an
Alternative Tool for Understanding Institutional Meaning Making. Journal
of Management Inquiry. https://doi.org/10.1177/1056492617724233
Purdy, J., & Gray, B. (2018). Collaborating for Our Future. Multistakeholder
Partnership for Solving Complex Problems. New York: Oxford University
Press.
Ritala, P., Armila, L. & Blomqvist, K. (2009). Innovation orchestration capability
defining the organizational and individual level determinants. International
Journal of Innovation Management, 13(4), 569 591.
Ritter, T. & Gemünden, H. G. (2003). Network Competence: Its impact on
innovation success and its antecedents. Journal of Business Research, 56
(9), 747 755.
Schuurman, D. (2015). Bridging the gap between Open and User Innovation?
Exploring the value of Living Labs as a means to structure user contribution
and manage distributed innovation. Doctoral dissertation. Ghent
University: Ghent, Belgium.
Schuurman, D. & De Marez, L. (2015). Living labs: a structured approach for
implementing open and user innovation. 13th Annual Open and User
Innovation Conference, Proceedings. Presented at the 13th Annual Open
and User Innovation Conference. http://hdl.handle.net/1854/LU‐6888241
Shank, G. D. (2002). Qualitative Research. A Personal Skills Approach. Upper
Saddler River, New Jersey, USA: Pearson Education Inc.
Spekman, R. E., Isabella, L. A. and MacAvoy, T. C. (2000). Alliance competence:
maximizing the value of your partnership. Upper Saddle River, New Jersey,
USA: Pearson Education Inc.

391
Strauss, D.A. (1999). Designing a consensus building process using a graphic
road map. In L. Susskind, S. McKearnan, and J. Thomas‐Larmer (eds.).
Consensus Building Handbook, 137 168. Thousand Oaks, CA: Sage.
Thorton, P., H., Ocasio, W., & Lounsbury, M. (2012). The Institutional Logics
Perspective: A New Approach to Culture, Stucture and Process. New York:
Oxford University Press.
von Schomberg, R. (2013). A vision of responsible innovation. In Owen, R.,
Heintz, M. & Bessant, J. (eds.). Responsible innovation. London, UK: John
Wiley.
von Schomberg, R. (2019). Why Responsible Innovation. In von Schomberg, R &
J. Hankins (eds.). The International Handbook on Responsible Innovation.
A Global Resource. Cheltenham: Edward Elgar Publishing. Forthcoming.
Vontas, A. & Protogenos, N. (2009). Evaluating Living Labs core competences
and assets. Paper presented in 3rd IEEE International Conference on
Digital Ecosystems and Technologies.
Voytenko, Y., McCormick, K., Evans, J. & Schwila, G. (2016). Urban living labs
for sustainability and low carbon cities in Europe: Towards a research
agenda. Journal of Cleaner Production, 123 (1), 45-54.
https://doi.org/10.1016/j.jclepro.2015.08.053
Westerlund, M. & Leminen, S. (2011). Managing the challenges of becoming an
open innovation company: Experiences from living labs. Technology
Innovation Management Review, 1(1), 19-25.
http://doi.org/10.22215/timreview/489
Westerlund, M., Leminen, S. & Rajahonka, M. (2018). A topic modelling analysis
of living labs research.
Technology Innovation Management Review, 8 (7), 40-51.
http://doi.org/10.22215/timreview/1170
Äyväri, A. & Jyrämä, A. (2007). The networking abilities of craft entrepreneurs. In
P. Naudé, J. Zolkiewski and S. Henneberg (eds). 23rd IMP Conference.
Exploiting the b2b Knowledge Network: New Perspectives and Core
Concepts. Manchester Business School, 30.8−1.9.2007. Manchester
Business School, Manchester, UK.
https://www.impgroup.org/uploads/papers/5827.pdf
Äyväri, A., Jyrämä, A. & Hirvikoski, T. (2018). The Circle of Mediators: Towards a
governance model for tackling sustainability challenges in a city. In
Research and Innovation Conference Proceedings 2018. Open Living Lab
Days. Brussels: European Network of Living Labs. 287‒305.
https://doi.org/10.5281/zenodo.1434951.

392
393
 Improving Quality in Higher Education by using
Living Lab Methods
Karin Axelsson*1, Yvonne Eriksson1 and Anders Berglund1

*Corresponding author
1 School of Innovation, Design and Engineering MDH Living Lab@ IPR
Mälardalen University, Eskilstuna, Sweden.

Category: Full Research

Abstract
This conceptual paper presents a Living Lab model of how university, society
and organisation may strengthen the co-creation capacity on regional and
international level to improve quality in higher education. Our conclusions are
that successful co-creation between universities and society/organizations is
built on mutual contribution, knowledge sharing as well as engagement from all
involved. Here, living lab methods can help improve quality. Further, from a
university perspective, a challenge is to keep the engagement in the project from
all involved, and to assure an equal status between stakeholders. This calls for
a skill in how to conduct project in co-creation with several partners. A skill that
has to be taught to students as well as to teachers and researchers.

Keywords: Lifelong learning, Co-creation, Living Lab, Higher education, User

involvement

394
1 Introduction
Digitalization is expected to affect all parts of society including the role of
universities; it is not only young people that are expected in higher education. The
requirements in relation to the changes in organizations (industry, services, public
sector) leads to a need of higher education for people already establish on labour
market. The universities need to adapt to the requirement of lifelong learning.
One such platform constitutes of Living Labs that has risen from the urgency not
only to collaborate for short time cycle improvements but also to sustain long-
term societal impact. This paper presents a Living Lab model of how university,
society and organisation may strengthen the co-creation capacity on regional and
international level. The paper is foremost of conceptual nature, using theories to
describe how co-creation efforts can improve quality. However, it further builds
on many years of practical co-creation experience from workshops, activities,
projects and research between the Mälardalen University research environment
IPR (Innovation and Product Realisation), which hosts the Living Lab
environment, and external stakeholders.

1.1 Background
The pressure on companies to enhance their growth through innovation and
digitalization is extremely high. One key element of today’s intensified rivalry
between companies is found in the competencies and skills utilization that exists
in each employee. The access to new knowledge is no longer important it is
fundamental. The admittance to high-qualitative knowledge environments and
different forms of collaboration have become more important than ever before. In
this context, Higher Education Institutions (HEIs) have been very successful in
different triple-helix collaboration platforms that has proven fruitful over time. One
such platform is constituted by Living Labs that has risen from the urgency not
only to collaborate for short time cycle improvements but to sustain long-term
societal impact. In Sweden, this collaborative, or co-creative, role for HEIs is
enshrined in the Higher Education Act (Högskolelag, 1992:1434), and after an
additional amendment regulation from 2009 the wording now being both to
collaborate with its surrounding society, inform of the HEIs operations and
promote that research results developed within the organisation become useful,
for instance in solving societal challenges. On a national level, the recognition of
collaboration is further strengthened by a new Governmental initiative rewarding
HEIs competencies and results from co-creation.

Even if public outreach can take various forms in practice, most Swedish HEIs,
express the goal as to co-create knowledge and economic and social growth in
partnership. Co-creation is performed with surrounding industry, public
authorities and non-profit organisations. At Mälardalen University being active in
collaboration lies in the overall strategy and vision of the University, further
claiming a market position in the Swedish academic landscape based on its co-
creation excellence. Here the University’s Living Lab plays a vital role to both
keep the position and continuously progress as regard to co-creation. Further, in
relation to the University’s students, there is an explicit goal that all students
examined from MDH should have both practical and theoretical knowledge of
collaboration and co-production. This since working in co-creating contexts can
help students develop and exercise important soft skills, besides the facts and

395
knowledge received in education. Hereby preparing the students for expectations
and requirements from employers in their future working life.

The work of the Lab included this study - the MDH Living Lab@IPR - position
itself as a milieu for co-creation, where the focus is on developing innovative and
collaboration methods and models. Thus, in relation to Nesti’s (2017) research,
the MDH Living Lab@IPR is both a tool for or co-creation and co-producing
knowledge and innovation between academy, business and public
administrations, at the same time is co-creating tools, methods and models for
improving co-creation as such.

However, as pointed out by Hughes (2014), creating an effective and

collaborative research culture is highly challenging. Even though the collaborative
assignment has been both stimulated and imposed on Universities all over
Europe, and work through for instance Triple Helix constellations have enhanced,
there is still room for improvement. Amongst other things, developing better
working procedures and ways of understanding each other in order to evolve
more effective and innovative co-creative processes. Thus, Hughes (2014)
suggests the different parties need to become better on addressing issues related
to cooperation during the process and reach a better consensus, which in the end
leads to a better result. Furthermore, according to Nesti (2017) and Fuglsang and
Vorre Hansen (2018), there are some apparent challenges and issues concerning
co-creation, posing a gap within the Living Lab literature. They propose it is
unclear how co-creation and user participation is really organized. Further, users
seem not to have strong decision power.

2 Literature Review
There are many definitions of Livings labs flourishing. Bergvall-Kåreborn et al.
(2009, p.1) propose that ‘a Living Lab is a user-centric innovation milieu built on
every-day practice and research, with an approach that facilitates user influence
in open and distributed innovation processes engaging all relevant partners in
real-life contexts, aiming to create sustainable values. Ballon et al. (2005 p.3)
rather suggest ‘an experiment environment in which technology is given shape in
real life contexts and in which (end) users are considered “co- producers”’.
Fuglsang and Vorre Hansen (2018) side with Nesti (2017) suggesting viewing
Living Labs in a broader context of co-production and co-creation, in relation to
this, Hagy et al. (2016) describe Living Labs as places for open innovation where
co-creation is a theoretical methodology, which is transdisciplinary, generating
knowledge by addressing real-life issues.

Living Labs are according to Almirall et al. (2012) motivated by primarily two
ideas; to involve users as coequal co-creators and to experiment in real-world
settings. The user involvement can occur in different ways. Some well-known
approaches are provided by von Hippel (1986) positioning the user as the
innovator, or main creator, directly involved in the product creation process, or by
Chesbrough’s (2012) thoughts on open innovation where firms can and should
use external ideas alongside the internal to deliver better innovations. Howe
(2009), approach the end-user through crowdsourcing allowing the power of the

396
crowd to contribute to development, and Brown (2008) position the users as
participants and co- creators in a design thinking process.

In many Living Lab contexts users’ needs, knowledge and experience provide
external ideas as a resource for innovation (Furestein et al., 2008; Fuglsang &
Vorre Hansen, 2018; Ståhlbröst, 2008), with which different stakeholders such as
researchers, companies, users and public organisations, develop, test or
evaluate new products and services, working together in an innovation process
in real-life settings (Bergvall-Kåreborn et al. ,2009). Thus, many different
organisations and individuals can participate in Living Lab activities. Leminen et
al. (2017) claim typical stakeholders are institutions, organisations, research units
at universities as well as suppliers, customer and users. As pointed out by
Hughes (2014), co- creation is generally perceived as a process where two or
more partners collaborate to create value for themselves and/or others. Thus, the
assumption is that these parties, or others, will benefit from the collaboration.
However, to jointly decide what benefits to focus on and, if conflicting, reach a
consensus, is not necessarily an easy task. In relation to this, Fuglsang and Vorre
Hansen (2018) claim there is a lack of clarification concerning the Living Labs
role and for whom they are to create value. This since, as shown by Nesti (2017),
even if the Labs often are financed through public funds these organisations are
not necessary the primary recipient of the value. Fuglsang and Vorre Hansen
(2018) mean it is not clear if the labs are expected to create knowledge and
benefit for the public good or profit for the participating actors.

So, what prerequisites are important for cooperation? Huxham and Vangen
(2005) propose explicit leadership. Andersson et al. (2011) agree with this adding
communication, commitment as well as rules and regulations. Inherently, there
are risks involved in cooperative activities such as not knowing how the other
parts will act or respond.

Therefore, Tillmar and Lindkvist (2007) suggest also trust between the partners
is necessary. The organisation needs to have cooperative skills on all levels in
an organisation, such as at the strategic, top-management and operative level
(Sullivan & Skelcher, 2002; Barnes & Sullivan, 2002). An organisation’s operative
level is formed around structures and processes that bring about collaborative
activities in practice. Since co-creation and collaboration are dominated by social
activities between people, it is becoming increasingly important to strengthen co-
creating skills among co-creating participants independent of level within the
organisation (Berglund & Bernhard, 2015).

2.1 Higher education – learnings from living lab methods

Research methods used in living lab contexts require a mutual understanding
among the involved in the project regarding e.g. how to conduct the project, and
what role the participants have. Thus, this is important to follow up during the
entire project (Dell’Era & Landiano, 2014).

Living lab methods derive from the life lab theories that not only emphasises user
perspective and involvement, but also includes situated action. Users are affected
by the context and the situation and situated action is defined as the interaction

397
between the situation and socio-cultural aspects. These are dynamic and could
change over time (Vera & Simon, 1993).

Like situated action, phenomenology takes the experience of situations into

consideration. According to phenomenology the context emerges in interaction
between individuals, groups, organization and the situation. The context is
therefore not something that surround a user, it is occur in interaction between
individuals, his/her socio-cultural background, position and the situation. There
are three levels of situated action. Firstly, the sociocultural level that could be
described as a result of the interaction between structures or cultural norms that
are expected from individuals and groups. Secondly, the situation that emerge in
a context. How this context is understood/interpreted is dependent on the
individuals’ aims and goals. This is also dependent on the sociocultural level.
Thirdly, in order to act the individuals, use accessible tools and artefacts. The
third level’s prototyping concerns in particular an intersection in expressions of
internal and external thinking (Berglund & Leifer, 2013). These levels and actions
are also affected by earlier situations and experience as well as the sociocultural
level (Vera & Simon, 1993).

The focus on real-life situation challenges, inclusion of different stakeholders,

cooperation and co-creation and a multidisciplinary approach are important
features of the Living Lab. In addition, the notions and goals of the Living Lab are
similar and applicable to the entrepreneurial learning format (Axelsson et al.,
2018; Jones & Iredale, 2010; Leffler, 2014). Therefore, the Living Lab milieu can
provide a fruitful environment for students practicing and developing
entrepreneurial, or more suitable in the Living Lab context, co-creation skills. In
so doing, the Living Lab activities can improve the quality of education and help
students be better prepared for their future working life and professionals their
life-long learning.

3 The case of Premium - an example of adaption for lifelong

learning
This conceptual paper builds on results from many years of co-creation activities
and research in relation to the MDH Living Lab@IPR. This section describes
experience from a competence development program for professionals, planned
and executed in co-creation within this Living Lab context.

Sweden’s prosperity is built on innovative and successful export companies

repeatedly managing to renew and reorganize production and products to keep
pace with changing markets. The need to strengthen companies' competitiveness
concerns transforming knowledge intensive activities, and how to utilize the
opportunities once more connected and automated, internally and externally. The
importance of knowledge intensive product realization, regionally, internationally
is confirmed by OECD which has identified a need for increased knowledge
content in the products that are manufactured, as well as in the processes used
to produce them.

398
Co-creation, or in this context more suitable term ‘co-production’, also incorporate
that regional industrial partners has granted a significant multimillion co-creation
project named PREMIUM. Funded by the Swedish Knowledge Foundation
(KKS), PREMIUM is the sustainable competence development program that put
cutting edge technologies and learning in the forefront for the impact follows the
Industry 4.0 expansion. It is a Manufacturing Innovation education program that
target professionals to specifically co- produced courses in-between industrial
stakeholders and the local university. Led by Mälardalen University (MDH) the
project also involves Jönköping University (JU) as academic co-producing
partner together with two strong industrial clusters. It is the interaction and
knowledge sharing between these nodes that has initiated the spark that now has
begun to glow. Based on the urgency and needs uncovered, co-production to
transform modern manufacturing stand to set off true magic in the time to come.
The purpose is equally straightforward as it is complex and far from easy to solve:
To strengthen Sweden’s competitiveness and increase industry 4.0 production
transformation and efficiencies.

The use of shorter course packages increases flexibility and modularity for
external stakeholders such as professionals, which help to benefit from education
while working in parallel. Another approach on the same issue is that
professionals need to be able to participate in more comprehensive
courses/course packages in order to really focus on skills development. This
development focuses on how MDH's education offering is structured and the
balance between programs, independent courses and possible completely new
forms of education. Therefore, the PREMIUM project will play an important role
in MDH's development to position itself as an institution that supports specialist’s
knowledge while supporting lifelong learning through a new target group mostly
overlooked in previous initiatives. The stakes are high and so is the extensive
offer that in total presents 21 completely new courses provided in with flexible
learning format to reassure best utilization of different digital tools.

The changing labour market increases the need for skills development in working
life. The shifting need from industry also places new demands on the flexibility of
education and results in a more diversified student group involving both traditional
program students and working students. The basis for establishing a need-based
approach to the exploration of new forms of education, to a large extent
independent of time and space, put pressure on new flexible course designs.

Important success factors in the co-creation phases of the PREMIUM program is

for instance to start the co-creation in an early phase, to get all perspectives,
needs and demands on the table. A clear value for each stakeholder is discussed
and visualised. This is an ongoing process, with multiple meetings providing a
foundation for the future cooperation. Further, driven by the focus on impact
following the Industry 4.0 expansion, the design is important to both fit the
acquirements of the academy as well as the practical realities for the industrial
partners. During the implementation and execution of the programme mutual
trust, openness to innovation, continuous change and an experimental approach
has been vital. In addition, a discussion on how to, and when, to disseminate
knowledge and potential results from the co-creation is important, since the

399
different stakeholders have different organizations, law, regulations and steering
to consider.

4 A Living Lab co-creation model for Higher Education

Based on the result of many years of co-production between university and
external, MDH Living Lab@IPR see the benefits of starting form a holistic
perspective. Challenges are defined jointly among stakeholders, and then these
are explored, exploited and evaluated together in an iterative process. It consists
of a stakeholder-triangle including the university, external organizations and the
society. For MDH Living Lab@IPR users are mostly employees in companies. In
order to improve quality in higher education by a strengthened co-creation
through using Living Lab methods there is a need for developing, testing and
evaluating new models and methods for co-creating knowledge which are
developed in a real-life context.

In an attempt to do so, building on research and insights and experiences from

previous co- creation projects and activities at the MDH Living Lab @IPR,
especially the above described PREMIUM programme, a tentative Living Lab co-
creation model for higher education is proposed. This consists of a proposed
cyclic approach of the co-creation process, as well as an additional 5-phase
analysis approach.

To briefly explain the different parts in the model: Exploration is an early phase
need-finding approach of involving a cascade of multiple perspectives and
stakeholders. Conceive is the process of defining purpose and needs. Design is
the creation and implementation plan of what is to be. Implement is the realisation
of plans, including manufacturing, testing and validation. Operate is about
delivering value over time on the basis of current status. Disseminate is the co-
creation efforts to increase level of influence and involvement, new partnerships
and the communication of visions, goals, activities and foremost achievements.

The inner circle is modified after the Deming cycle (e.g. Moen & Norman, 2006;
Langley et al, 2009) and from our view applied from the operational level in
contrast to the overarching strategic level that is concerns explore yet from
another level. For operations plan concerns activities, do means conducting what
has been planned. Improve is the development that is done through iterations
that leads to refinement and learning. This quality improvement model has been
successfully adopted in other university and industry co-creating initiatives in
recent time (Borys et al, 2012).

400
 Figure 1. Proposed cyclic approach of the co-creation process by MDH Living Lab@IPR.

In addition, in order to achieve a deeper learning from and future improvement of

the work, in relation to the model we suggest a five-phase analysis model. It is
not to be perceived as sequential, depending on such as the co-creation aim,
goal and outline some phases and questions are more relevant than others.
Addressing the different questions, help revealing initial values and assuring the
co-creation aspects are considered.

4.1 5-phases for analysis approach

Context for the project planning: What is the design of the co-creation project?
Which partners are involved? Who initiated the project? How was the planning of
the project conducted? From what criteria was the research method(s) selected
(including the analyses of current state, for collection of data, for the
design/innovation process, and for analyses of data/process)? How are the co-
creation stakeholders involved in the planning phase, ensuring e.g. equal
engagement, decision power, necessary resources and goal setting?

Analysis of the process: What methods have been used? How have the methods
been used? What is the impact of the choice of methods on the result(s)? Had it
been possible to use other methods? What new knowledge regarding
methodology is gained from the project? Form an academic perspective and from
the perspective of the external organization.

Follow up phase: What parts of the project is based on previous experience from
the external organization and/or previous research? What new academic and
practical knowledge is gained from the project?

401
Impact: What is the impact of the project on the organizations involved and the
society - initially, during the project, and when it was completed? Has the project
generated new policy or new working routines? What is the academic impact,
new knowledge, research and/or improved co-creation methods? How do the
students perceive their involvement regarding co-creation and
exercising/developing soft skills?

Continuous meta-analysis: To what extent has the project created impact on the
research group/students/teachers and external partners during the project? On
the individual level? On the organizational level?

5 Discussion
In this conceptual paper, the aim is to strengthen how university, society and
organisation can enhance the co-creation capacity on regional and international
level, focusing on improving quality by co-creation in higher education using
Living Lab methods.

To meet the requirement from the society i.e. graduated students that are
prepared to meet future challenges in society, who have the capability to work in
agile processes, and handle flexibility, MDH Living Lab@IPR has developed
courses and parts of courses that contain co- creation with external partners. The
aim is to let the student face real problems and to learn to collaborate with various
stakeholder. In addition, we have experienced the benefit of integrating people
with working experience in courses, by mixing young undergraduate students
with those who are on supplementary training creates value in the courses both
for teachers and for the young students. The presented model (see Figure 1)
support the development in the early phases of courses, however in order to
secure the involvement of all partners in a project or course we suggest a
combination of the model and the five phases analysis.

Based on the perception of Living Labs in a broader perspective as co-creation

or co- producing environments, as do Nesti (2017), and Fuglesang, and Vorre
Hansen (2018) the focus is on stakeholders as equal contributors and equal
beneficiaries or the results. Thus, in line with Almiral et al. (2012), the Living Lab
in focus for this paper, as most Living Labs, is motivated by involving users as
co-creators and to experiment in real-life settings. Many researchers highlight
stakeholders as research units, companies or organisations (e.g. Bergvall-
Kåreborn et al., 2009). However, embedding Living Lab methods in higher
education also mean highlighting the students as stakeholders. Therefore, these
are in this context considered as ´users ‘of the education, however at the same
time co-creators of the content, process and results. Here one need to pay
attention to possible new challenges rising on for instance equality and balance
of power between often young and novice contributors and more experienced
researchers or company stakeholders. Thus, there is a need for supporting
students with skills in how to conduct project in co-creation with several partners,
for usage both during education and afterwards in their future working life. In
addition, the teachers need to develop skills on how to teach these skills and how

402
to work with co-creation and Living lab methods in their teaching, as well as to
relate the work to learning outcomes.

Thus, as pointed out by for instance Hughes et al. (2014), there are challenges
in developing a high effective collaborative research culture, and there is room
for improvement.

Therefore, all stakeholders in such a co-creation environment need models and

guidance in order to improve quality. In this paper we contribute to this
development by proposing a cyclic approach to the co-creation process, which
by adding the five phases of analysis, assure that all stakeholders and their
perspectives are taken into consideration.

From initial attempts to implement the steps in some of the Information Design
courses at Mälardalen University we have learned that an explicit discussion
related to co-creation, and the meaning of it, create an awareness among
students according to the benefit of university-society/industry collaboration in co-
creation (Gottlieb & Eriksson, 2019). Next action is to implement the model in an
underground course together with students, teachers and external partners. An
extensive training in co-creation is needed to need the aim to strengthen the
quality in higher education by implement co-creation in courses and integrate it
as learning objectives.

6 Conclusion
We are convinced that successful co-creation between universities –
society/organizations is built on mutual contribution and knowledge sharing and
co-creation requires engagement from all involved, like living lab methods. The
challenges from a university perspective when it comes to living lab projects in
co-creation university – society/company is to keep the engagement in the project
from all involved. This needs a skill in how to conduct project in co-creation with
several partners. A skill that has to be teach to students as well as to teachers
and researchers. In addition, the external partner like employees in companies
also need some training in how to engage and contribute to knowledge in a
project. To meet the future needs for lifelong learning we suggest that living lab
methods for co-creation that have a clear process for how to develop and follow
up the different stages and steps during the project or course is necessary. By
that, it is possible to continually evaluate and improve the process.

403
References
Andersson, J., Åhgren, B., Axelsson, S., Eriksson, A., Axelsson, R., & Ahgren, B.
(2011). Organizational approaches to collaboration in vocational
rehabilitation-an international literature review. International Journal of
Integrated Care, Vol. 11.
Almirall, E., Lee, M., & Wareham, J. (2012). Mapping living labs in the landscape
of innovation methodologies, Technology Innovation Management
Review, pp. 12–18.
Axelsson, K, Höglund, L. & Mårtensson, M. (2018). Is what’s good for business
good for society? Entrepreneurship in a school setting. In U. Hytti, R.
Blackburn & S. Tegtmeier (Eds.) The Dynamics of Entrepreneurial
Contexts Frontiers in European Entrepreneurship Research, (pp 54-75).
Cheltenham: Edward Elgar Publishing Ltd.
Ballon, P., Pierson, J., & Delaere, S. (2005). Test and experimentation platforms
for broadband innovation: Examining European practice. In Studies on
Media, Information and Telecommunication (SMIT)—Interdisciplinary
Institute for BroadBand Technology (IBBT). Belgium, Brussels: Vrije
Universiteit Brussel.
Barnes, M. & Sullivan, H. (2002). Building capacity for collaboration in English
health action zones. In C. Glendinning, M. Powell & K. Rummery (Eds.)
Partnerships, New Labour and the Governance of Wealfare, pp 81-96.
Bristol, UK: Policy Press.
Berglund, A., & Bernhard, J. (2015). Co-creation beyond the expected: LAB
environments as mean to enhance learning. In 43rd SEFI Annual
Conference 2015, SEFI 2015, June 29th- July 2nd, 2015. European
Society for Engineering Education (SEFI).
Berglund, A., & Leifer, L. (2013). Why we prototype! An international comparison
of the linkage between embedded knowledge and objective learning.
Engineering Education, 8(1), 2-15.
Bergvall-Kåreborn, B., Eriksson, C., Ståhlbröst, A., & Svensson, J. (2009). A
milieu for innovation: defining living labs. Proceedings Of The 2nd Ispim
Innovation Symposium.
Borys, M., Milosz, M. & Plechawska-Wojcik, M. (2012). Using Deming cycle for
strengthening cooperation between industry and university in IT
engineering education program, 15th International Conference on
Interactive Collaborative Learning (ICL), Villach, 1-4. doi:
10.1109/ICL.2012.6402164
Brown, T. (2008). Design Thinking. Harward Business Review. 86(6), 1-10.
Bölling, M. & Eriksson, Y. (2016). Collaboration with society: The future role of
universities? Identifying challenges for evaluation, Research Evaluation,
25(2), 209–218 doi: 10.1093/reseval/rvv043
Chesbrough (2012). Open Innovation Where We’ve Been and Where We’re
Going, Research- Technology Management, 55(4), 20-27, DOI:
10.5437/08956308X5504085

404
Dell’Era, C. & Landiano, P. (2014). Living Lab: A Methodology between User-
Centred Design and Participatory Design, Creativity and Innovation
Management, 23(2), 137-154.
Fuglsang, L. & Vorre Hansen,A. (2018). A research note on living labs – exploring
the link to public value and service coproduction, presented at the RESER
Conference, 22-22nd September, 2018, Gotheburg, Sweden.
Feuerstein, K., Hesmer, A., Hribernik, K. A., Thoben, K. D., & Schumacher, J.
(2008). Living labs: A new development strategy. In J. Schumacher & V.
P. Niitamo (Eds.), European living labs (pp. 1–14). Berlin:
Wissenschaftlicher Verlag.
Hagy, S., Morrison, G., & Elfstrand, P. (2016). Co-creation in living labs. In Living
Labs: Design and Assessment of Sustainable Living (pp. 169–178).
Springer International Publishing. https://doi.org/10.1007/978-3-319-
33527-8_13
Howe, J. (2009). Crowdsourcing : why the power of the crowd is driving the future
of business . New York: Three Rivers Press.
Hughes, T. (2014). Co-creation: moving towards a framework for creating
innovation in the Triple Helix, Prometheus, 32:4, 337-350, DOI:
10.1080/08109028.2014.971613
Huxham, C. & Vangen, S. (2005). Managing to Collaborate: The theory and
practice of collaborative advantage. New York: Routledge.
Högskolelag (1992:1434). Retrieved from Sveriges Riksdag website:
https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-
forfattningssamling/hogskolelag- 19921434_sfs-1992-1434
Jones,B. & Iredale, N. (2010). Enterprise education as pedagogy.
Entrepreneurship + Education. 52(1), 7-19.
Langley, G., Moen, R., Nolan, K., Nolan, T., Norman, C., & Provost, L.. (2009).
The Improvement Guide, 2nd Ed. Jossey-Bass: San Francisco.
Leffler, E. (2014) Enterprise Learning and School Subjects – A Subject Didactic
Issue?. Journal of Education and Training, 1(2): 15-30
http://dx.doi.org/10.5296/jet.v1i2.5194
Leminen, S., Rajahonka, M. & Westerlund, M. (2017).Towards Third-Generation
Living Lab Networks in Cities. Technology Innovation Management
Review, 7(11), 21-35.
Moen, R., & Norman, C. (2006). Evolution of the PDCA cycle.
Nesti, G. (2017). Living Labs: A New Tool for Co-production? In Bisello. A,
Vettorat, D, Stephens, R., & Elsei,P (Eds.) Smart and sustainable planning
for cities and regions. Cham:Springer.
Ståhlbröst, A. (2008). Forming future IT – The living lab way of user involvement.
Doctoral Thesis Luleå: Luleå University of Technology.
Sullivan, H. & Skelcher, C. (2002). Working Across Boundaries: Collaboration in
Public Services. New York: Palgrave Macmillian.

405
Tillmar, M. & Lindkvist, L. (2007). Cooperation Against All Odds Finding Reasons
for Trust where Formal Institutions Fail. International Sociology, 22(3):
343–366. DOI: 10.1177/0268580907076575.
Vera, A. H & Simon, H.A (1993) Situated Action: A Symbolic Interpretation,
Cognitive Science 17, 7-48.
von Hippel, E. (1986). Lead Users: A Source of Novel Product Concepts,
Management Science 32(7), 791-805.

406
407
 LivingLab 65+ - Co-creation with retirement
and nursing homes
Veronika Hämmerle*1, Stephanie Lehmann1,
Cora Pauli1 and Sabina Misoch1

*Corresponding author
1 University of Applied Sciences, Interdisciplinary Competence Centre for
Ageing IKOA-FHS, Switzerland

Category: Research-in-progress

Abstract
In Switzerland many seniors live in retirement and nursing homes. At the same
time on the global level the healthcare sector is suffering from a shortage of
nursing staff. Although technology use could be a promising strategy, its potential
seems not yet fully exploited. To shed light on the specific needs and
requirements of retirement and nursing homes and their inhabitants regarding
technology use and technology implementation, we have extended our Living
Lab approach and cooperate with retirement and nursing homes throughout
Switzerland. This paper discusses challenges of this test environment and
strategies to overcome them. Furthermore, the Living Lab approach itself shall
be evaluated as a method to facilitate technology implementation in care
institutions.

Keywords: Living Lab, retirement and nursing homes, technology testing,

nursing staff, seniors

408
1 Background
In 2017, 149 000 seniors lived in retirement and nursing homes in Switzerland
(Federal Statistical Office, 2017). At the same time, the health care sector is
confronted with a global shortage of nursing staff (WHO, 2017). Although
technology could help to overcome these demographic challenges the use of
Active Assisted Living technology in retirement and nursing homes is not
widespread (Fehling, & Dassen, 2017a).

Possible reasons for the absence of technology in retirement and nursing homes
may be that technologies are not adapted to the needs of the elderly and
caregivers and that creating interoperability within the complex infrastructures of
retirement and nursing homes is difficult. The defensive attitudes of employees
towards technology in care (Fehling, & Dassen, 2017b), a lacking overview of
technological options and sparse information on systems available on the market
and their quality may also play a role. Since assistive technology is evolving
rapidly as demographic structures change, it is indispensable for institutions to
deal with the potential of these new technologies (Rüegger, Roulet Schwab, &
Eggert 2016).

2 Living Lab 65+

To close this gap and to promote dialogue between researchers, developers and
end-users, we set up Living Labs (LL) in Switzerland. In LL seniors test
innovations during their everyday life and in their "natural" living environment over
a longer period of time (up to 6 months). The aim of the participatory tests in the
LL65+ is to understand the needs of seniors regarding assistive technologies and
to test technology with target groups in their living environment. Our LL provides
a neutral hub that mediates between developers and consumers. The conducted
age research is based on the guidelines of the Swiss Ethics Committee.

Whereas previous research in the LL65+ mainly focused on tests in private

households, we extended our approach to include retirement and nursing homes
with the following considerations in mind:

2.1 Access to vulnerable target-groups

Retirement and nursing homes provide access to a hard to reach target group.
Broadening the LL65+ to include retirement and nursing homes, will allow us to
gain information on the needs and technology use of very old, fragile and
technology-averse participants, their health status being constantly monitored by
professionals. By enlisting the help of nursing staff, who can assess the health
status and motivation of participants, we can save time in recruitment and reduce
dropout rates.

2.2 Increased interoperability

Doing research in institutions means technologies can be tested while embedded
in a holistic system. A device should meet the needs and demands of older users,
caregivers and the involved institution. This implies interoperability between
different infrastructures, institute-specific routines and practices, various types of

409
devices and different professionals. Based on test results, technologies can then
be adapted to optimally fit the requirements of these environments.

2.3 Improved matching

Caregivers are an important application group for technologies. The feedback on
their needs, expectations and user-experience is indispensable for the
development of technology for their patients. The use of technology in healthcare
institutions depends on technology acceptance and openness of professionals
and decision-makers (Kowitlawakul 2011; Aloulou, Mokhtari, Tiberghien, Biswas,
Phua, Lin & Yap, 2013). The cooperation of developers, researchers, medical
and care experts and the elderly allows to shed light on the development of
technology in a multiperspectival manner and ensure that all target-users are
reached.

2.4 Caregivers taken seriously

Institutions carefully consider their investments and purchasing technology is
usually associated with high costs. By making technical innovations available free
of charge during tests, institutions can get accustomed to new products with the
support of researchers. This might lead to a higher sensitization of institutions
towards technology use and could help overcome access barriers.

The possibility of job losses is an ethical concern of caregivers and their rejection
or skepticism is often greater than that of the elderly (Evans, Hielscher, & Voss,
2018; Shire & Leimeister, 2012). Caregivers are often sensitive towards privacy
issues of seniors, especially when it comes to technology that automatically
gathers data. Testing technology in retirement and nursing homes allows nursing
staff to familiarize themselves with innovations and reduces feelings of anxiety
(Liedtke, Welfens, Rohn, & Nordmann, 2012). By testing technologies with no
strings attached, caregivers can evaluate concerns and discuss them with
researchers and developers freely.

2.5 Facilitated market entry

Through cooperation with institutions, developers can reach many potential
users. This can improve innovation development and market opportunities and
lead to a considerable market advantage, as the institutions represent a potential
field of application. If institutions are satisfied with the performance of a device
during the tests, they are more likely to invest in this technology.

3 Method
Our approach consists of 4-5 phases. First, the needs and expectations of the
end-users are assessed. Second, the end-users test the product over several
months in their everyday life. The end-users are then asked about their
experiences during installation, commissioning and use at different points in time.
The data from phase 1 and 2 provide the basis for the report that is forwarded to
the developers. Phase 4 involves the implementation of requests by the
developer. The adapted product is ideally tested again in phase 5.

410
We have performed two LL tests in retirement and nursing homes. One product
tested was a fall detector which we tested in 3 different retirement and nursing
homes. The participants aged 65 and more (N=14) were at a high risk of falling
and very frail. Only one could be interviewed personally. As the study was
focused on the interoperability of the detector with the local infrastructure and the
routines of the institutions, we refrained from involving more seniors in interviews
or observations. The nursing staff and management representatives were
interviewed in detail (N=6). The focus was on questions of needs, acceptance
and safety from the point of view of the resident and questions of user-
friendliness, safety and integration in routines from the point of view of the nursing
staff and institution. The technical functionality and compatibility of the sensor
with the locally used systems were evaluated with tests lasting 3-4 months. The
semi-structured interviews were conducted before (P1), while and after the
testing (P2). The participants’ feedback was analyzed and presented to the
developer in a report (P3). This led to a new version of the device: push messages
on the mobile phone were integrated, the fastening was improved with a click
system and the SIM card was integrated directly into the device (P4).

In our current study, a robotic seal developed for dementia care (PARO) is tested
with cognitively healthy but lonely residents. We aim to find out whether the robot
has an impact on loneliness and well-being of the elderly. Tests with 3
participants revealed that it is difficult to gain the trust of lonely people. Due to the
frailty and vulnerability of the participants, we decided neither to conduct
interviews with them nor to observe or record them.

Only in a protected and private environment, it is possible to gain access to this

group. We were able to create such a test setting by involving the participants’
activation therapists. The professionals (N=4) were interviewed regularly and
gave feedback on the state of the seniors. Additionally, activation therapists were
asked to evaluate usability and describe their own user experience in semi-
structured interviews. We prolonged the project phase to involve more
participants and refine our methods and recruiting strategy. A similar procedure
as in the fall detection study can be expected after completion.

411
More tests in retirement and nursing homes are planned in the upcoming years.
Additionally, we plan follow-up research on the impact of the LL tests on the
technology use and implementation of participating retirement and nursing
homes. Six months after completing the LL tests, we aim to investigate whether
and how the test contributed to technology acceptance in retirement and nursing
homes, whether it had an impact on the attitude towards technology and whether
the experience gained during the tests boosted the technological self-efficacy of
nursing staff. Furthermore, the number of integrated technologies since the LL
tests will be evaluated.

4 Challenges and strategies to overcome

4.1 Dropouts
Rapidly changing circumstances and high mortality rates can lead to high dropout
rates in research with elderly people. Therefore, it is indispensable to recruit
participants with care (Ogonowski, Ley, Hess, Wan & Wulf, 2013). Institutions
can be beneficial partners in the task. Attending staff can estimate which persons
are able to participate according to their health status and who would benefit from
the innovation. With the help and the motivating nature of nurses, dropout levels
are low and seniors are willing to take part in the test.

4.2 Permanent contact person

As vulnerable participants’ health status must not be endangered by tests, the
involvement of a caregiver is recommended. Caregivers perceive the concerns
of the participants and ensure that participants don’t feel negatively affected by
the testing. A specific researcher should be defined as a contact person
(Georges, Schuurman, Baccarne, & Coorevits, 2015; Hess & Ogonowski, 2010;
Ogonowski et al., 2013) and during the tests the nurse involved must receive full
backing and support of the institution and researchers.

4.3 Appropriate timing

Another challenge is the limited time and opportunities for testing because of the
high nursing workload and frailty of the elderly participants. The use of carefully
selected and developed interview instruments for seniors and caregivers are
essential. Adequate instruments should take the limited time resources of nurses
into consideration. We noticed for example that it is easier for nurses to report
their experiences in an oral interview rather than a written questionnaire. The use
of other media like phone or e-mail to conduct interviews keeps the data collection
more flexible and reduces the effort for participants.

The quality of research can be endangered by wrong timing. Institutions follow a

strict daily routine, that ensures a smooth workflow for nurses and structures the
elderly’s day (Zysberg Young, Schepp &, 2007). Research should not interrupt or
disturb these routines. Consulting nurses in advance can prevent interviews
being conducted at inappropriate times, e.g. during rest periods or immediately
after treatments.

4.4 Limited infrastructure

412
Clarifying infrastructure requirements with the developer and the institution is
crucial, as institutions may not have the necessary technical resources and
infrastructure to test certain technologies. Many retirement and nursing homes,
for example, do not provide WIFI in private rooms, a requirement for the use of
many devices. Clarifying requirements with the developer and the participating
institution before the testing is therefore important. If necessary, temporary
alternatives can be found for the testing period, e.g. portable WIFI extenders.

4.5 Adapted instruments

Vulnerable seniors might not find it easy to protect their interests and rights
(Moser-Sigmeth & Hofer, 2013). High sensitivity, ethical conduct and the
development or adaption of instruments that are operable with the special
circumstances of very old and fragile persons are needed. Our tests show that
seniors often prefer qualitative interviews over quantitative questionnaires.
Because of possible health problems like hearing deficits, visual impairments,
cognitive slowing, etc., questionnaires should be adapted accordingly (Brandt et
al., 2018; Lang, 2014; Resch & Aumayr, 2011), and should be short and easy to
understand (Mody, Miller, McGloin, Freeman, Marcantonio, Magaziner &
Studenski, 2008).

5 Conclusion
Currently, we are expanding our LL network and evaluating first results. We
gained knowledge about specific requirements and advantages of LL tests in
retirement and nursing homes and contribute to the LL movement by providing
this information. We learned that successful research must take the frailty and
impairments of seniors, as well as the limited time resources and the fears of
caregivers into account and that it is important to embed the tests into the daily
routines and given infrastructures. Moreover, it is important that decision-makers
are open to the tests. Institutions can discover new technologies, they can help
make technologies better suitable for users, and they can gain realistic insights.
Whether the LL tests contribute to an improved implementation of technology in
retirement and nursing homes shall be investigated in the follow-up evaluation.
This investigation will enrich the LL movement with information on whether the
LL-experience breaks down barriers and changes attitudes towards technology.

References
Aloulou, H., Mokhtari, M., Tiberghien, T., Biswas, J., Phua, C., Kenneth Lin, J. H.,
Yap, P. (2013). Deployment of assistive living technology in a nursing
home environment: methods and lessons learned. BMC Medical
Informatics and Decision Making 13(1), 42. doi: 10.1186/1472-6947-13-42
Brandt, M., Fietz, J., Hampel, S., Kaschowitz, J., Lazarevic, P., & Reichert, M. et
al. (2018). Methoden der empirischen Alter(n)sforschung. Weinheim:
Beltz.
Bundesamt für Statistik (2017). Sozialmedizinische Betreuung in Institutionen
und zu Hause im Jahr 2017. Mehr Pflege zu Hause, stagnierende Anzahl
Personen in Alters- und Pflegeheimen. Neuchâtel. Verfügbar unter

413
 https://www.bfs.admin.ch/bfs/de/home/statistiken/gesundheit/gesundheit
swesen/alters-pflegeheime.assetdetail.6406792.html
Claßen, K., Oswald, F., Doh, M., Kleinemas, U. & Wahl, H.-W. (2014). Umwelten
des Alterns: Wohnen, Mobilität, Technik und Medien. Stuttgart, Germany:
Kohlhammer.
Evans, M., Hielscher, V., & Voss, D. (2018). Damit Arbeit 4.0 in der Pflege
ankommt. Wie Technik die Pflege stärken kann. Policy Brief, 004, 1-11.
Düsseldorf: Hans-Böckler-Stiftung.
Fehling, P., & Dassen, T. (2017a). Motivie und Hürden bei der Etablierung
technischer Assistenzsysteme in Pflegeheimen: eine qualitative Studie.
Klinische Pflegeforschung, 3, 61-71.
Fehling, P., & Dassen, T. (2017b). Retrospektive und prospektive Deutung
technischer Innovationen in Pflegeheimen: eine qualitative Studie.
Pflegewissenschaft 9/10.
Georges, A., Schuurman, D., & Vervoort, K. (2016). Factors affecting the attrition
of test users during Living Lab field trials. Technology Innovation
Management Review, 6(1), 35-44.
Hess, J., & Ogonowski, C. (2010). Steps toward a Living Lab for socialmedia
concept evaluation and continuous user-involvement. EuroITV’10, 8th
International Interactive TV&Video Conference, June 9-11 (S. 171-174).
Tampere, Finland.
Kowitlawakul, Y. (2011): The technology acceptance model: predicting nurses’
intention to use telemedicine technology (eICU). Computers, informatics,
nursing: CIN 29(7), 411-418. doi: 10.1097/ncn.0b013e3181f9dd4a
Lang, G. (2014). Zur Befragung und Befragbarkeit von kognitiv eingeschränkten
und demenziell veränderten Menschen in Altern- und Pflegeheimen. In A.
Amann, & F. Kolland (Hrsg.), Das erzwungene Paradies des Alters?
Weitere Fragen an eine Kritische Gerontologie (S. 207-215). Wiesbaden:
Springer VS.
Liedtke, C., Welfens, M. J., Rohn, H., & Nordmann, J. (2012). Living Lab: user-
driven innovation for sustainability. International Journal of sustainability in
higher education, 13, 106-118.
Mody L., Miller D. K., McGloin J. M., Freeman M., Marcantonio E. R., Magaziner,
J., & Studenski, S. (2008). Recruitment and retention of older adults in
aging research. Journal of the American Geriatrics Society, 56, 2340-
2348. doi:10.1111/j.1532-5415.2008.02015.x
Moser-Siegmeth, V., & Hofer, K. (2013). Assistive Technologien für ältere
Menschen: Nutzen für EndanwenderInnen und Herausforderungen im
Einsatz. SWS-Rundschau, 53(1), 57-72.
https://www.ssoar.info/ssoar/handle/document/42651
Ogonowski, C., Ley, B., Hess, J., Wan, L., & Wulf, V. (2013). Designing for the
living room: Long-term user involvement in a Living Lab. CHI 2013, ACM
ISBN 978-1-4503-1899. Paris, France.

414
Resch, K., & Aumayr, G. (2011). Methodische Herausforderungen bei der
Befragung von und Testungen mit vulnerablen, älteren Menschen ab 60.
In V. Moser-Siegmeth, & G. Aumayr (Hrsg.), Alter und Technik. Theorie
und Praxis (S. 129-142). Wien: Facultas.
Rüegger, H., Roulet Schwab D., & Eggert, N. (2016). Ethische Aspekte im
Umgang mit assistierender Technologie in Institutionen der
Langzeitpflege. Hrsg. CURAVIVA Schweiz.
https://www.curaviva.ch/files/9CVU59U/brosch_a4_ethische_aspekte_w
eb2.pdf
Shire, K. A., & Leimeister, J. M. (Hrsg.). (2012). Technologiegestützte
Dienstleistungsinnovation in der Gesundheitswirtschaft. Wiesbaden:
Springer Gabler.
Zisberg A., Young H. M., Schepp, K. and Zysberg, L. (2007). A concept analysis
of routine: relevance to nursing. Journal of Advanced Nursing, 57, 442-
453. doi:10.1111/j.1365-2648.2007.04103.x

415
 Living Labs for small retailers – in search of
a framework and tools
Heleen Geerts1, Gabriela Bustamante Castillo1 and Anja
Overdiek1

1 The Hague University of Applied Sciences, The Netherlands

Category: Research-in-progress

Abstract
City centers all over Europe are challenged by the effects of recent
developments in consumer behavior and online retail. Collaborations of
municipalities, knowledge institutions, retailers and consumers are blooming up
in many regions to help this transition and find solutions which fit both in the
future of retail entrepreneurs and employees and in that of consumers and their
social space. These collaborations often take the form of a Living Lab, but don’t
always fit easily in Living Lab or field lab definitions. Future-Proof Retail, a
network of eleven retail labs in the Netherlands, currently researches a
comprehensive and practical framework for this kind of labs. First findings
include definitions, maturity scores and guidelines for stakeholder engagement.
These findings will be presented as work in progress, to share, but also to
discuss the methodological approach and possible shortcomings.

Keywords: Retail Field Lab, lab evaluation, lab maturity, lab best practices

416
1 Introduction
In 2017, 149 000 seniors lived in retirement and nursing homes in Switzerland
(Federal Statistical Office, 2017). At the same time, the health care sector is
confronted with a global shortage of nursing staff (WHO, 2017). Although
technology could help to overcome these demographic challenges the use of
Active Assisted Living technology in retirement and nursing homes is not
widespread (Fehling, & Dassen, 2017a).

Future-Proof Retail (FPR) is a national action research program with eleven

regional Living Labs. Eight HEIs (Higher Education Institutions) and nine
municipalities participate in this program led by The Hague University of Applied
Sciences. The participating researchers, technology providers and students in
the different labs all work design-led. This means that the collectively found
solutions are prototyped and tested during the lab activity period. The
companies/private actors in the labs have a very special position, compared to
the role of this stakeholder group in other types of Living Labs: When it comes
to their role (Leminen, Westerlund and Nystroem 2012) they are both utilizers
and users of new technologies and business models. Hence, the eleven labs
have all been designed to address specific business, social and skills challenges
that small retailers and employees in retail are facing today.

Influenced by changing consumer behavior and new online players, small

retailers with physical stores are fighting for their economic survival. However,
as retail activity and retail real estate contribute largely to the social environment
in city centers, the retail problem reaches further than the sector.

Future-Proof Retail was set up in May 2018 to address this problem in an

innovative way. Also, it is meant to connect existing local lab initiatives to share
knowledge and learn from each other by developing a shared vision and
evaluation procedures. As such it can be seen as a network of Living Labs. A
recent study on Living Labs in the Netherlands (Maas, Broek and Deuten 2017)
distinguished, among other, ‘real’ Living Labs and so-called Field Labs for the
make industry with the first being an ‘instrument for transition geared to societal
challenges’ and the second more an instrument for industry innovation.

The Future-Proof Retail labs do not fit neatly in either of these categories. Which
is why the objective of this research is to create a comprehensive and practical
framework to support the setup and evaluation of a Retail Field Lab (RFL) as
well as to enhance its innovation performance, creativity, efficiency and the value
of its outcomes. FPR will work with an initial subsidy until December 2019 to
reach this objective.

The subsequent paper will first situate the research objective in existing literature
and narrow down on the research gap. Then, it will describe the used
methodology. After that first findings will be presented: the RFL Framework Tool,
the RFL Maturity Tool and best practices to engage retailers in Retail Field Labs.
Finally some questions will be raised for further discussion of methodology and
limitations.

417
2 Retail Living Labs in the literature
Since the beginning of 2000, Living Labs have started to emerge. An initial focus
was the testing of technology in simulated home-like environments (Markopoulos
and Rautenberg, 2000) but the practice of Living Labs quickly expanded to other
areas of application and broadened its methods. In the following years, research
developed complementary perspectives on the Living Lab phenomenon as an
environment/context, as a methodology and as a conceptual system of
collaborative innovation (Bergvall-Kǻreborn and Ståhlbröst 2009). Nowadays,
Living Labs know very different areas of application and approaches, but
definitions of Living Labs commonly address the importance of a real-life
environment and the involvement of multiple stakeholders (Leminen and
Westerlund, 2017).

Westerlund and Leminen (2011) define Living Labs as “physical regions or virtual
realities, or interaction spaces, in which stakeholders form public-private-people
partnerships (4 Ps) of companies, public agencies, universities, users and
stakeholders, all collaborating for creation, prototyping, validating and testing of
new technologies, services, products and systems in real-life contexts”.

The handbook of the European Netwerk of Living Labs (ENOLL) states that
Living Labs share certain common elements (Malmberg, K. and Vaittinen 2017):
1. Multi-method approaches: all Living Labs combine and customize
different user-centred, co- creation methodologies to best fit their purpose.
2. User engagement: the key to success is to involve the users already at
the beginning of the process.
3. Multi-stakeholder participation: even if the focus is on users, involving all
relevant stakeholders is of crucial importance. These include all the
quadruple helix actors: representatives of public and private sector,
academia and people.
4. Real-life setting: a very specific characteristic of Living Labs is that the
activities take place in real-life settings to gain a thorough overview of the
context.
5. Co-creation: the recognition that users are equal contributors and co-
creators rather than subjects of studies. The Living Lab approach strives
for mutually valued outcomes that are results of all stakeholders being
actively engaged in the process from the very beginning.

Living Lab activities take place across many different domains, typically in health
and wellbeing, smart cities and circular economy, culture and creativity, energy
and mobility. However, literature on specific lab approaches and methodologies
in the Retail sector is scarce (Leminen and Westerlund 2008). This is surprising
as transitions in this particular sector are huge and affect cities as well as work
opportunities all over Europe and the U.S.. The forecast is that this situation will
become more critical in the coming years (see i.e. McKinsey & Company 2019).
As Retail is a very practical and result oriented sector, particularly methods and
tools for Living Labs are needed to help retailers, municipalities, real estate
stakeholders, technology providers and knowledge institutions to come up with
new solutions.

418
Overall, methods and tools are under researched in Living Labs (Leminen and
Westerlund, 2017; Nesti 2018). Looking at the existing tools-focused research in
Living Labs (Äyväri and Jyrämä 2017: Leminen 2013; Leminen, Westerlund and
Nyström 2012; Leminen and Westerlund 2017; Rits, Schuurman and Ballon
2015; Ståhlbröst and Holst 2013) a number of concepts can be applied to Living
Labs in Retail. The stakeholder roles suggested by Leminen et al. (2012) help to
draw light on the particularities of Living Labs in Retail. It can be stated that in
these labs the goal of developing technologies and new business models is as
important as the goal of engaging the entrepreneurs and their employees in
learning new skills. This double goal puts the retailers in the role of the user of
technologies and new business models as well as in the role of utilizer, who
wants to create and extract value from these new solutions. Mostly, knowledge
institutions and municipalities are enablers of these labs. It can even be said that
the labs are enabler-led. Technology companies, business students and
(sometimes) real estate stakeholders act as providers. –This particular
distribution of roles present a complex field for lab coordinators wanting to
establish ‘successful’ labs. It is from the realization of the urgency of the situation
of the Retail sector combined with the gap in research about the particularities
of Living Labs for small retailers that we formulated the following research
questions:
1. What are ‘successful’ Living Labs for small retailers?
2. How can we enable the setting up of these labs with a tool?
3. What are best practices of engaging retailers to participate in these
Living Labs?

3 Methodology
Three researchers are currently working with these research questions. They are
using action research methodology. Following the set-up and activities of six
local Living Labs for small retailers in the Netherlands and participating in their
process. To answer the first two questions a design researcher is collecting data
from all labs using questionnaires. Subsequently, she is designing, prototyping
and testing a “lab set-up” tool through several iterations, using all in all eleven
labs for testing. A second researchers focusses on the third question using more
ethnographic methods. She is taking over the role of lab coordinator herself and
reflects on this with auto-ethnography. Moreover, she organizes, leads and
reflects on inter visions with a group of ten lab coordinators. Finally, a third
researcher is following the first two with more distance to the field. She is
observing their reflections and making sure existing concepts and developments
in the broader Living Lab literature are taken into consideration when appropriate
in the process.

3.1 A Retail Field Lab (RFL) framework tool- work in progress

During the first phase of setting up a Living Lab all stakeholders that have taken
the initiative or showed interest in participating, often do not know where to start.
Who leads the way in deciding what to do? What are the shared ambitions or
objectives and who’s responsible for which project activity? Labs for small
retailers are places where the different cultural contexts and languages of

419
retailers, students, researchers, local government and citizens come together
and interact.

To organize a lab like this is not easy: It needs to have an attractive offer for the
retailer in order to have him actively participating and at the same time a lab
needs to connect to the objectives of all stakeholders involved. The above
mentioned literature (Bergvall-Kǻreborn and Ståhlbröst 2009; Leminen and
Westerlund 2017; Malmberg, K. and Vaittinen 2017; Westerlund and Leminen
2011) was used as a point of departure to create a list of guidelines for setting-
up the first activities of labs participating in the Future Proof Retail project (Figure
1).

Figure. 1

From September to December 2018 the labs’ activities and meetings were
initiated and organized locally. Researchers followed up with these activities by
keeping contact with the lab coordinators about the labs’ set up process. In
parallel to this, further literary research in relation to Living Labs within different
contexts served as a guide to learn about relevant principles throughout the
setup and execution of a Living Lab. The criteria for selection were first to make
use of the principles that were part of a Living Lab independently of their context
(e.g. healthcare, transportation or urban development), secondly take into
account the improvements necessary in Living Labs in the Netherlands
suggested by Maas, Broek and Deuten (2017). Third, the feedback received by
the Lab coordinators based on the outcome of their RFL activities.

Maas, Broek & Deuten (2017) who researched more than 90 Living Labs in the
Netherlands categorize four types of initiatives called ‘Living Labs’:

3.2 Types of Living Labs: (summary)

A- The open scientific research facilities:
• To promote knowledge valorization.
• Gives companies better access to knowledge and research facilities from
public knowledge institutes.

420
 • Science and innovation policy with which public-private partnerships are
stimulated to collaborate in research and innovation.

B- The field labs of the manufacturing industry:

• Aim to improve the competitiveness in the field and strengthen it by jointly
learning through applying new (digital) technologies in production
processes.
• This type is also a true learning environment in which students and staff
can acquire relevant knowledge and skills.
• It is considered a typical instrument for industrial policy, also for the
policies of the Action Agenda in relation to Smart Industry.

C- Commercial urban testing facilities support companies in developing:

• Testing and demonstrating innovative solutions.
• Typical tool for the local or regional innovation policy and the promotion
of a city or area as attractive locations for innovative activities.
• It fits well in regional triple helix collaboration to strengthen regional
innovation ecosystems.

D- The 'real' Living Labs:

• Typical instrument for transition, research and innovation policy aimed at
societal challenges.
• They fit well with the policy around the National Science Agenda.
• They are also an answer to the increased need in the policy to tackle
social problems in an integrated manner.

The analysis of the actual activities and outcomes of the six Retail Living Labs
has been used to place the labs in the typology provided by Maas et al. (2017).
Similarities between the characteristics of Living Labs B, C, and D of the
typology show an overlap with the characteristics of the six labs for small
retailers that were operating during the first phase (September-December
2018) of the Future Proof Retail project. This new classification let to the
identification of the FPR labs as ‘Retail Field Labs’ (RFL) (Figure 2).

421
 Figure 2. Defining Retail Field Labs

The following principles could be described as ‘pillars’ to operate a Living Lab,

thus serve as a supporting framework to structure the activities of the Retail Field
Labs and were used to create a framework for RFL practices:

Realism - Involve with Real users in a ‘real context’ situation.

Empowerment and co-creation - User centred focus. Engaging and enabling
participants through co- creation. Equal power is given to participants (target
group) to influence the process of the Lab.

Data collection and measurement - The use of methods used to collect data.
As suggested in ‘The Living Lab Methodology Handbook “Pre- measurement an
intervention and a post-measurement, where the intervention is equalled to the
real-life experiment”.

Documentation and analysis - Recollection of structure/unstructured data and

documents generated during and after RFL activities. Material such as
pictures/video during activities that are relevant to the purpose of the Lab.

Infrastructure - Facilities needed for operational structure (time, budget,

installation, network, facilities, technology, etc.)

Communication and engagement strategy - Communication and user

engagement strategies used through-out the different stages of the RFL.

422
Evaluation and reflection – Reflection and evaluation throughout process and
iteration

The outcome of this first research phase is a draft of a visual tool that could
further support RFL coordinators to set up and evaluate their activities (Figure
3). Next to the principles, to place the different actors involved in a Living Lab,
an overview of the different levels in which a Living Lab operates (Malmberg and
Vaittinen 2017) has been added to this framework.

Figure.3 Visualizing Retail Field Lab

3.3 A Retail Field Lab (RFL) maturity score - work in progress

To find out how the RFL were operating their activities in accordance to some of
the Living Lab principles, further questionnaires and interviews were held. To
measure the level of maturity of each lab a comparison was made between how
each lab scored in relation to the guidelines provided to them at the start of the
project (Figure 4- preparation score) and the answers provided by lab
coordinators to the questionnaire related to the Living Lab principles (Fig.

423
5,maturity score). The results of this show the relation between preparation and
maturity scores of each participating lab (Fig.6, relationship
preparation/maturity).

Figure.4 Preparation Score

30
25
20
15
10 creation

Measurement

Documentation
Fygital Instore

Figure. 5 – Maturity Score

424
 Figure.6 Relation between preparation score and maturity score

Considering these results and the feedback on the first version of the RFL
visualization a second version of the RFL framework tool (Figures 7 and 8) was
created and used in two workshops during the Future Proof Retail ‘Kennisfeest’,
a meeting of all participants of the eleven labs. Feedback was gathered during
the workshops which will be used throughout the operations of the new RFL as
well as the follow up activities of the initial RFL. As it is stated by Ståhlbröst and
Holst (2013), “stakeholders have to be well equipped and have sound
understanding of what is going on in their Living Lab”.

By participating in different phases of the Retail Field Labs and by systematically

evaluating outcomes, a list of best practices could be formulated. In a later phase
of the FPR project, these best practices are meant to help stakeholders with
setting up Retail Field Labs. The found best practices were ordered (and
informed) by the ‘pillars’ of the structuring activities of Retail Field Labs (Fig. 8).

425
Figure.7 Different Levels in which a Retail Field Lab Operates

Figure 8. Framework to structure the activities of Retail Field Labs

426
Best Practices to engage retailers in Retail Field Labs
Empowerment and Co-creation
In the early stages of the project, it became clear that every regional lab has its
own contextual challenges and that each context contains different types of
retailers. However, one thing small retail entrepreneurs have in common is that
they are focused on their own businesses and less on their environment.
Additionally, the (SME) retailers have a strong focus on short-term results.
Therefore, the project partners had to design a format, an environment and a
theme to trigger the retailer to participate. Nomad labs (that visit the retailers and
sets up a ‘base camp’ near the shops of the retailers) were subsequently
evaluated as most fit for this purpose.

A constant adaptation of the labs (in form and approach) to the needs of the local
stakeholders, such as the ‘recruited’ retailers (retailers who were participating
and committed to the lab), the local municipality, the involved educational
institutions and the local (shopping area) associations, seemed an essential
factor. For the national FPR research project objectives, the most important
group of these stakeholders were the retailers; therefore, the interest of this
stakeholder group was always leading within the process of developing a lab.
Adaptation of the timeline of the lab to their (working) hours, events etc. Keeping
an inventory of all needs, questions and agendas of the most important
stakeholders emerged as a best practice for the coordinator of the lab.

To keep the retailer ‘on board’, the labs needed to be very concrete about the
effect and impact of their activities. The answer to the question: “What’s in it for
me?” needed to be found repeatedly. The retail labs developed like a diorama:
all sides could possibly unfold. All sides that present a concrete ‘take away’ for
the participants of the lab needed to be made visible like organized activities,
experiences or events in the shopping area, press/media publications,
generating traffic and attention.

Within the process of developing a retail lab, trust, language, time and attitude
are keywords. A project leader from a Higher Educational Institution for example
is not communicating in the same way as an entrepreneur or shop owner. The
‘jargon’ can make or break an understanding and trust. The solution was often
finding local ‘linking pin’ organizations or individuals that know the shop owners
personally: city center or shopping mall managers and deputies of retail
associations. They translate the language of education and research,
understand the routines of retail and can make the connection between the
different professional cultures.

Communication strategy
The following communication aspects were essential for the realization of the lab
objectives:
• Each lab needs one coordinator who is the central
communicator/facilitator between different stakeholders.
• Creativity and flexibility skills of the coordinator and stakeholders (Design
Thinking skills) are essential.

427
 • Cultivating a personal approach: short lines of communication make many
things possible.
• Provision of a planning of the activities to be used and completed by the
most important stakeholders, although not all activities can be filled out in
the preparation phase. There needs to be space for
adaptations/reframing.
• Sharing enthusiasm of initiator(s), project coordinator and early adopters
via social media
• Creating ‘noise’ on the outside of the lab by organizing and initiating
attractive activities like tours, marketing campaigns and adds in order to
reach a broader public and recruit more retailers for future labs

Evaluation and reflection

All stakeholders of the labs agreed: regular and structured evaluation with all
labs using the same methodology was key. Intermediate and frequent
evaluations with all stakeholders are necessary to reframe and increase effect
and to realize the objectives of the lab. It has proven to be a big challenge to
register and archive all experiences, best and worst practices (the so-called
brilliant failures). Sharing lessons learned, being transparent and showing what
you do with those lessons is essential but needs to be balanced with reducing
complexity for (at least some) stakeholders.

4 Conclusion
We are well on our way with the research on the supporting framework, maturity
score and best practices of Retail Filed Labs as a hybrid between ‘real Living
Labs’ and Field Labs. At the ENOLL Living Lab Days, we would like to discuss
these results, the guiding methodology and its limitations. Furthermore, we would
like to brainstorm about elements to finalize our work and make results scalable
to a larger field of Living Labs.

References
Almirall, E., Lee, M. & Wareham, J. (2012). Mapping Living Labs in the landscape
of innovation methodologies. Technology Innovation Management Review,
2(9), 12-18.
Äyväri, A. & Jyrämä, A. (2017). Rethinking value proposition tools for living labs.
Journal of Service Theory and Practice, 27(5), 1024-1039.
Bergvall-Kǻreborn, B. , Hoist, M. & Ståhlbröst, A. (2009). Concept design with a
living lab approach.
Proceedings of the 42nd Hawaii International Conference on System Sciences,
1-10.
Bergvall-Kåreborn, B., & Ståhlbröst, A. (2009). Living Lab: an open and citizen-
centric approach for innovation. International Journal of Innovation and
Regional Development, 1(4), 356-370.

428
Brankaert, R., den Ouden, E. (2017). The design-driven Living Lab: a new
approach to exploring solutions to complex societal challenges.
Technology Innovation Management Review, 7(1), 44- 51.
Dell'Era, C. & Landoni, P. (2014). Living Lab: A methodology between user-
centred design and participatory design. Creativity and Innovation
Management, 23(2), 137-154.
Humble, D. (2014). The practice of key principles in living labs. Tilburg: Tilburg
University.
Leminen, S. (2013). Coordination and participation in living lab networks.
Technology Innovation Management Review, 3(11).
Leminen, S. & Westerlund, M. (2008). Living labs fostering innovations in the retail
industry: A network perspective. IMP 2008 Conference. Uppsala, Sweden.
Leminen, S. & Westerlund, M. (2017). Categorization of innovation tools in living
labs. Technology Innovation Management Review, 7(1), 15-25.
Leminen, S., Westerlund, M. & Nyström, A. G. (2012). Living Labs as open-
innovation networks.
Technology Innovation Management Review, 2(9), 6-11.
Maas, T., van den Broek, J. & Deuten, J. (2017). Living Labs in Nederland. Van
open testfaciliteit tot levend lab. Den Haag: Rathenau Instituut.
Malmberg, K. , Vaittinen, I. (eds.) (2017). Living Lab Methodology Handbook.
European Commission.
https://u4iot.eu/pdf/U4IoT_LivingLabMethodology_Handbook.pdf
Markopoulos, P. & Rauterberg, G. W. M. (2000). LivingLab: A white paper. IPO
Annual Progress Report, 35, 53-65.
McKinsey & Company (2019). Perspectives on retail and consumer goods. No. 7.
Megens, C., Hummels,C., Brombacher, A.C. & Ijsselsteijn, W.A. (2013).
Experiential design landscapes: design research in the wild. NORDES
2013. Copenhagen: ResearchGate.
Nesti, G. (2018). Co-production for innovation: the urban living lab experience.
Policy and Society, 37(3), 310-325.
Rits, O., Schuurman, D. & Ballon, P. (2015). Exploring the benefits of integrating
business model research within living lab projects. Technology Innovation
Management Review, 5(12), 19-27.
Ståhlbröst, A. & Holst, M. (2013). The living lab: methodology handbook. Norden:
Vinnova. Torjman, L. (2012). Labs designing the future. Ontario: MarS
Discovery District.
Westerlund, M. & Leminen, S. (2011). Managing the challenges of becoming an
open innovation company: experiences from living labs. Technology
Innovation Management Review, 1(1), 9-25.
Wijers, J.P., Bakker, M., Collignon, R. & Smit, G. (2019). Managing Authentic
Relationships. Facing New challenges in a changing context. Amsterdam:
Amsterdam University Press.

429
430
431
432
433
 Barriers for Test and Adoption of Digital
Innovations by End-users in a Living Lab Context
Abdolrasoul Habibipour1

1 Luleå University of Technology, Sweden

Category: Doctoral Papers

Disciplines: Adoption and diffusion of Information technology; Communities to

contribute: Information Systems, Living Labs
PhD dates: 2015 to 2020
PhD Supervisors: Prof. Anna Ståhlbröst and Associate Prof. Diana Chroneer

Abstract
In recent years, Living Labs have become a well-established innovation
approach where individual users and other stakeholders are involved to co-
create, test and evaluate digital innovations in open, collaborative, multi-
contextual and real-world settings. Despite this, in order for digital innovation to
be tested and successfully adopted by end-users, different barriers should be
understood and clearly addressed particularly in a Living Lab setting that the
participation is usually voluntary and end-users are involved with testing and
using digital technology in their real-life everyday use context. These barriers
can be related not only to the process of engaging users to test the digital
innovations, but also can be related to the innovation itself. Accordingly, the aim
of this PhD thesis is to understand the barriers for digital innovations to be tested
and adopted by end-users in a Living Lab context. In so doing, a qualitative
research approach has been used and several data collection methods have
been employed to fulfil the objective of this study. Conducting extensive literature
reviews (focusing on both barriers for user engagement as well as adoption
barriers) have been used as the secondary data source. As for the primary data
collection methods, conducting a field test in a Living Lab context, semi-
structured interviews with Living experts as well as professionals in different
digital innovation application domains, interviewing end-users of digital
technologies and finally open-ended questionnaire have been employed. So far,
a taxonomy of influential factors on participants’ drop-out behaviour in Living Lab
field test has been developed and initial list of adoption barriers for digital
innovation has been identified. In addition, the consequences of drop-out in
Living Lab field tests and recommendations that would facilitate prolonged user
engagement have been developed. An initial set of recommendations on how
the adoption barriers should be tackled has been developed. This study will also
summarize the key lessons learned from the conducted field tests, workshops,
interviews with a number of end-users and experts in Living Lab and digital
innovations field and opens up several avenues for future research in this field.

434
Keywords: User engagement, Barriers, Drop-out, Living Lab, Field test,
Adoption, Innovation

435
1 Introduction
Digital technologies have made significant influence on our society and it has
recently gained a lot of attention in various research fields including information
systems (IS) (Boston College et al., 2014; Yoo, Henfridsson, & Lyytinen, 2010)
and innovation management (IM) (University of Wisconsin–Milwaukee et al.,
2017). Despite the fact that digital innovation has been acknowledged as a
fundamental and powerful concept in the IS research (Boston College et al.,
2014), but in practice IS literature has not paid enough attention to the digital
technology’s transformative impact on industrial-age products and services (Yoo
et al., 2010). Moreover, those IS studies that have highlighted the effects of
digital technologies, heavily focused on digital technologies in organizations, and
understanding of how the development process of digital innovation can be
connected to the human actors remained cursory, and this topic deserves further
research (Lyytinen Kalle, Yoo Youngjin, & Boland Jr. Richard J., 2015).

On the other hand, due to interdisciplinary nature of IS field, IS has been defined
and viewed from different perspectives such as: social view (Land, 1985),
technical view (Davis & Olson, 1984), socio technical (Lee, 2001; Mumford,
2006) and process view (Alter, 2008). By growing interest within the IS research
field, the primarily technological perspective of the 1970s was influenced by a
more sociological perspectives that aimed to explain various emerged issues in
the IS research particularly through the information systems development (ISD)
process (Hirschheim & Klein, 2012). In contrast to the technical view that mainly
focuses on software and hardware, this socio-technical view of IS includes
people as system participants and highlights their interests, skills, incentives, and
social relations (Alter, 2008). Considering IS as a socio-technical system, “the
information systems field examines more than just the technological system, or
just the social system, or even the two side by side; in addition, it investigates
the phenomena that emerge when the two interact” (Lee, 2001, p.3).
Accordingly, a combination of technical and social sub systems is necessary in
order to enable human to interact with the digital technologies throughout
development process (Luna-Reyes, Zhang, Gil-García, & Cresswell, 2005;
Lyytinen & Newman, 2008). More recent ISD approaches have emphasized the
importance of innovativeness and creativity by involving individual users in the
ISD process. This approach is consistent with Henry Chesbrough’s (2003, 2006)
proposition, so called “open innovation” in which external knowledge and ideas
must be captured throughout the innovation process.

Nowadays, one of the most promising ways of involving users within the open
innovation are Living Labs that have become a very popular research topic in
both Information Systems (IS) as well as Innovation Management (IM) literature.
Living Labs can be seen as a way of managing challenges in the process of
innovation development, where individual users and other stakeholders are
involved to co-create, test and evaluate innovations in open, collaborative, multi-
contextual and real-world settings (Bergvall-Kareborn, Holst, & Stahlbrost, 2009;
Ståhlbröst, 2008). In contrast with the traditional research and development
projects where the prototyped product, service or system is in focus (Brønnum
& Møller, 2013), Living Labs present an outstanding approach where the focus
is on user-driven and co-creative innovation (Mulder, 2012). A major principle

436
within Living Lab research consists of capturing the real-life context in which a
digital innovation is used by end-users by means of a multi-method approach
(Bergvall-Kåreborn, Eriksson, & Ståhlbröst, 2015; Schuurman, 2015).

The process of innovation development in the Living Lab setting can happen in
different phases, including exploration, design, implementation, test, and
evaluation (Ståhlbröst, 2008). Nevertheless, within Living Lab context, testing
digital innovations has received more attention than other phases of innovation
development (Claude, Ginestet, Bonhomme, Moulène, & Escadeillas, 2017;
Følstad, 2008). As outlined by Rogers (2010), a successful test of digital
innovations will lead to a higher level of adoption as testing the innovation by
(potential) end-users is one of the main stages of innovation adoption process.

Despite this, there have been barriers for digital innovations to be tested and
adopted by end-users. These barriers can be related not only to the process of
engaging users to test the digital innovations, but also can be related to the
innovation itself. Regarding to engaging users in the process of testing digital
innovations in a Living Lab context, it is recognized that keeping users motivated
is more challenging than motivating them to start participating in the test process
in the first place (Ley et al., 2015; Pedersen et al., 2013). Consequently, users
tend to drop-out of testing digital innovations before the project or activity has
ended, as the motivations and expectations of the test participants change over
time (Georges, Schuurman, & Vervoort, 2016). The reasons for dropping out
might be due to internal factors relating to a participant’s decision to stop the
activity or external environmental factors that caused them to terminate their
engagement (O’Brien & Toms, 2008). The phenomenon of drop-out in the
innovation process is challenging in contexts especially when the participation is
voluntary and there is not an organizational tie between the participants and the
organizers of a project or activity (Ståhlbröst & Bergvall-Kåreborn, 2013). This
includes many of the more recent system development and innovation
approaches, such as user-driven innovation (De Moor et al., 2010),
crowdsourcing (Howe, 2006), and Living Lab (Bergvall-Kareborn et al., 2009;
Ståhlbröst, 2008). Therefore, understanding the phenomenon of drop-out and
how to motivate participants to engage and stay engaged through the process
of testing digital innovations is of crucial importance for open innovation
approaches.

Besides the factors that are influencing participants’ drop-out behaviour in a

Living Lab field test, there are other adoption barriers that are related to the
innovation itself. As innovation adoption has been defined as a decision to make
full use of an innovation (Rogers, 2010), an adoption barriers can be considered
as any factor that discourages or prevents end-users from acceptance to actual
use of digital innovation. These barriers can be related to the various phases of
innovation adoption process and are not limited to the test of digital innovations
by end-users. Rogers (2010) has outlined innovation adoption as a process that
involves five main stages namely, knowledge, persuasion, decision,
implementation, and confirmation.

437
Accordingly, this thesis has been focused on the challenges of engaging end-
users in the process of testing digital innovations in a Living Lab context as well
as adoption barriers of the digital innovations. Thus, the overall aim of this thesis
is to address the complexity associated with the challenges for digital innovations
to be tested and adopted by end-users in a Living Lab context. More specifically,
the stated aim is divided into two research questions:

RQ1: What are the challenges of engaging (potential) end-users to test

digital innovations in a Living Lab setting?

RQ2: What are the barriers for digital innovations to be adopted by

(potential) end-users?

The logical connection between the two research questions might be explained
in the way that a more successful user engagement in Living Lab field test might
lead to a higher adoption of digital innovation by the users. one plausible
explanation for this is that testing digital innovations in a small scale is a part of
innovation adoption process as outlined by Rogers (2010). This thesis aims to
explore this relation in a greater detail.

Regarding to the first research question, the results of this thesis will provide an
empirically grounded definition for drop-out in Living Lab field tests as well as
develop an empirically derived, comprehensive taxonomy for the various
influential factors on drop-out behaviour in a Living Lab setting. It will also
provides insight into the possible consequences of drop-out in Living Lab field
tests with the objective to provide some recommendations that would facilitate
prolonged user engagement throughout Living Lab field tests process.

In respect to the second research question, this thesis will contribute to the
understanding of adoption barriers for digital innovations from end-users
perspective while at the same time offering some recommendations on how
these adoption barriers should be tackled.

2 Theoretical Framework
In this thesis, diffusion of innovation theory (Rogers, 2010) has been used to
guide the data analysis process as well as developing data collection tools.
According to Rogers, there are five steps for adoption of innovations namely,
knowledge, persuasion, decision, implementation and confirmation.

Knowledge occurs when a potential end-user learns about the existence on the
innovation and gains some understanding of how it is functions. Persuasion
occurs when a potential end-user forms a favorable or unfavorable attitude
towards the innovation and is open to being persuaded that the innovation holds
value (show interest). Decision occurs when a potential end-user undertakes
activities, which lead to the adoption or rejection of the IoT solution (mental trial).
Implementation occurs when the innovation is actually put to use or practice (is
tested by end-user in a limited basis). And finally, Confirmation occurs when an
end-user seeks support for the adoption decision in the form of confirmation that

438
the decision was a correct one (the user may also reverse this previous decision
if exposed to conflicting messages about the innovation. Figure 1 shows the
innovation adoption process as outlined by Rogers.

Figure 1. innovation adoption process (Rogers, 2010)

This adoption process will enable me to organize and extend the discussion in
relation to the different steps of adoption. More specifically, the barriers for
engagement of end-users to test digital innovations in a Living Lab context can
be related to the forth step (i.e., implementation), where the innovation is going
to be tested by end-users before their final decision to adopt the technology.
However, the innovation adoption barriers can be related to all phases of
adoption process from knowledge phase (e.g., poor dissemination of information
about the digital innovation, lack of education, etc.) to the final confirmation stage
(e.g., expensiveness of digital innovation for end-users).

3 Research methodology
This thesis uses qualitative approach as the overall research methodology. As
stated, the aim of this thesis is to understand the barriers for digital innovations
to be tested and adopted by end-users in a Living Lab context. Therefore, an
appropriate research methodology for this research is the methodology that
facilitates a higher degree of involvement between the researchers and
participants in the research. In addition, according to Maxwell (2012), qualitative
research is more appropriate when it is important to understand the phenomena
from the participants’ point of view and particularly when it comes to social and
institutional context. Within qualitative research, instead of providing a specific
setting to conduct the study, individuals are typically involved in their natural
setting (Kaplan & Maxwell, 2005); which is in line with the nature of Living Lab
activities.

In order to meet the objectives of this thesis, both secondary and primary data
sources were used. Secondary data was collected by conducting comprehensive
literature reviews focusing on both barriers for user engagement in the Living
Lab field test process as well as on adoption barriers from end-user’s
perspective. In order to be more specific, the literature review on adoption
barriers has been focused on IoT technologies (but in different IoT application

439
domains) as it will closely be related to the digital innovations which is the main
focus of this thesis. The data gathered from the literature reviews then served as
a basis to develop the data collection tools for the primary sources of data.

Regarding to the challenges of user engagement in Living Lab field test, primary
data was collected by conducting a Living Lab field test in Botnia Living Lab,
Lulea, Sweden, followed by a post-test open-ended questionnaire sent to those
who dropped out of the field test before completing the assigned tasks within the
specified deadline. Combining the findings of this field test with three other field
tests in iMinds Living Lab in Belgium enabled me to present initial list of various
factors that influence participants drop-out behaviour in a Living Lab field tests.
This understanding then further validated by conducting 14 semi-structured
open-ended interviews with Living Lab experts in two Living Labs in Sweden and
Belgium.

Regarding to the adoption barriers for end-users, primary data was collected by
conducting two workshops with IoT experts to understand the barriers for digital
innovations to be adopted by end-users. The results of this step then combined
by the findings from a literature review in five different IoT application domains
(i.e., smart cities, smart agriculture, wearable technologies, smart home care and
autonomous driving vehicles). The findings then validated by conducting 10
semi-structured interviews with experts in five IoT domains to extend my
knowledge on adoption barriers and develop recommendations on how to tackle
these barriers. As a complementary data collection, between 20 to 30 semi-
structured interviews with actual end-users of IoT technologies in different
application domains (in two phases) are ongoing and will be complemented in
June 2019. Table 1 summarizes various data collection methods that has been
used in my PhD thesis.

As in my PhD study, the data has been collected in different phases of the
process, and in order to properly combine and analyze collected data, I’ve used
(and will be using) a qualitative interpretative approach as outlined by.

440
 Table 1: Data collection methods in my PhD thesis.

Focused Data Data Description Number

RQ collection collection
type method
1 Literature review on
Secondary Literature barriers for user
RQ1 44 articles
data source review engagement (drop-
out) in Living Labs
2 Completed by users
who started
Primary data Open-ended 32 complete
RQ1 participating in the
source questionnaire responses
Living Lab field test
but dropped out
3 14
interviews
Experts in Living Lab (6 in Botnia
Semi-
Primary data field testing focusing LL,
RQ1 structured
source on engagement Sweden; 8
interviews
barriers in iMinds
LL,
Belguim)
4 Literature review on
Secondary Literature
RQ2 adoption barriers in 18 articles
data source review
IoT domains
5 2
workshops
with 20
participants
Experts in Living Lab
(workshop
Primary data field testing focusing
RQ2 Workshop in Bilbao:
source on engagement
11
barriers
participants;
workshop in
Vienna: 9
participants)
IoT Experts in
Semi- 10
Primary data different domains
6 RQ2 structured interviews
source focusing on adoption
interviews
barriers
End-users of IoT
Semi- technologies in 20-30
Primary data
7 RQ2 structured different domains interviews
source
interviews focusing on adoption are ongoing
barriers

4 Results
The results of my thesis can be presented in two sections. First, the results of
barriers for user engagement in Living Lab field tests will be presented and then
barriers for digital innovations (and more specifically IoT devices) to be adopted
by end-users are presented.

4.1 Barriers for user engagement in Living Lab field test

441
The first phase of data collection in my thesis started by conducting a
comprehensive literature review on the topic. The aim of the literature review
was to identify documented reasons for drop-out in information systems
development processes. By reviewing 44 articles, I identified some influential
factors on drop-out behaviour (in total 29 factors) and classified them into
technical aspects, social aspects, and socio-technical aspects. The main findings
of this study indicated that the performance of the innovation, user selection,
user preparation, interaction with the users, privacy concerns and scheduling are
highly influential on this issue. However, in the abovementioned study I did not
focus on a specific phase or type of activity, and extracted the drop-out reasons
for all steps of the information systems development process such as ideation,
co-design, or co-creation, and, finally, test and evaluation. This understanding
then served as a basis for the next phases of data collection on the topic.

The next phase of data collection was to conduct a field test in Botnia Living Lab
followed by a post-test open ended questionnaire. A total of 118 participants
showed interest in participating in the field test and completely filled out the
recruitment survey. Of these, only 27 participants reached the end of the field
test and 91 participants (77%) dropped out of the activity. The main drop-out
reasons found in this user study were related to the stability of the prototype,
ease of use, privacy protection, flexibility of the prototype, effects of reminders,
and timing issues.

The findings from the previous steps then complemented across 14 semi-
structured interviews with experts in Living Lab field tests. The aim of the
interviews were to present an empirically derived taxonomy for the various
factors that influence drop-out behaviour in Living Lab field test and to
understand the extent to which each of the identified items influence participant
drop-out behaviour. The findings of this step shows that identified reasons for
dropping out can be grouped into three themes: technical aspects (innovation-
related), social aspects (participant-related), and socio-technical aspects
(process-related). Each theme consists of three categories with a total of 44
items.

Innovation-related factors: The categories under this theme are directly related
to the innovation itself and reflect the technical aspects when it comes to socio-
technical systems. Technological problems may be associated with, for example,
trouble installing the innovation, a lack of flexibility or infrastructure compatibility
issues, as well as issues with the stability and maturity of the (prototype)
innovation. Perceived usefulness of innovation highlights the importance of user
needs. When the innovation does not meet the user’s needs, it might be difficult
to maintain the same level of engagement throughout the lifetime of a field test.
Also, a participant who is voluntarily contributing in a field test must be able to
see the potential benefits of testing an innovation in their everyday life. In
addition, complexity can influence the perceived ease of use of the innovation.
When the innovation is too complex to use or is not easy to understand,
participants may become confused or discouraged. Moreover, when the
innovation is not sufficiently mature, it is difficult to keep the participants
enthusiastically engaged in the field test.

442
Participant-related factors: Some of the suggested categories were directly
related to the individuals and their everyday life contexts. This theme mainly
reflects social aspects and environment when it comes to socio-technical
systems. The participants’ attitudes or personalities, their personal contexts, and
their resources can be classified under the participant-related theme.
Participants’ attitudes can affect the participants’ willingness to test digital
innovations in a Living Lab setting. Examples of factors in this category is when
the participants forget to participate, when the innovation does not meet their
expectation, when they do not want to install something new on their device,
when they do not like the concept or idea, and when they have concerns about
their privacy or the security of their information. Furthermore, in a Living Lab
approach, the users usually test innovations within their own, real-life setting,
therefore, challenges they face in their personal lives – unrelated to the testing
activity – can negatively influence their motivation and may cause them to drop
out of a field test. Limitations in participants’ resources can also influence the
likelihood that they will drop-out. They might either have not had enough time to
be involved in the field test, or the project may place too many demands on their
resources, such as requiring them to drain their own mobile batteries or consume
part of their Internet data quota.

Process-related factors: These factors relate to the process of organizing a

field test in a Living Lab setting where the socio-technical aspects are in focus.
The three categories under this theme were associated with task design,
interaction with the participants, and the timing of the field test. Several factors
are related to the design of the Living Lab field test. For instance, when the tasks
during the field test were not fun to accomplish, participants tend to drop-out
before completing the test. The interviewees also considered items such as long
gap between the field test’s steps or a lengthy field test as influential factors that
might be associated with the task design in the field test. Interaction and
communication with the participants were considered as one of the most
important categories of items that influence a participant’s decision to drop-out.
Unclear guidelines on how to do the tasks, lack of an appropriate technical
support, and insufficient triggers to involve participants are some examples of
the items in this category. Inappropriate timing of the field test (e.g., summer
holiday) and too strict or inflexible deadlines are the most influential factors on
drop-out behaviour in this category.

Figure 2 shows an overview of the taxonomy that developed as a result of this

phase. The numbers in parentheses indicate the number of items under each
category.

443
 Figure 2. Overview of the proposed taxonomy of drop-out factors in Living Lab field test

The results of the interviews also provided insight into the possible
consequences of drop-out in Living Lab field tests with the objective to provide
some recommendations that would facilitate prolonged user engagement
throughout Living Lab field tests process. Based on the findings, drop-out in
Living Lab field tests might have some strong implications and consequences on
the whole field test process both for the project as well as for the Living Lab as a
collaborative innovation network. These consequences include but are not
limited to extra time, cost and effort for the field test organizers, reliability of the
field test results from the viewpoint of both research and project, losing
participants for future Living Lab activities, and difficulties to re-establish mutual
trust with those participants.

During the interviews, I was also interested in gathering recommendations to

understand what the organizers of a Living Lab field test should do and how they
can act in order to reduce the likelihood of drop-out in Living Lab field tests.
Spending enough time to investigate the innovation’s functionality before
conducting the actual field test, a clear and on time communication and
interaction with the test participants, giving them this feeling that their
contribution is important, managing participants’ expectation, flexible and
appropriate timing of the field test, avoiding to prolong the field test, and
considering an appropriate financial reward for the participants were the main
suggestions that we received from our interviewees.

444
Another important challenges that were identified in my study was related to the
ethical issues regarding user engagement in Living Lab activities. These ethical
considerations might be related to voluntariness of their participation, their
engagement environment, informed consent, overlooking the participants’
interests, costs and benefits of their participation and finally the ethical interaction
with the research participants throughout the process of user engagement in the
Living Lab activities.

4.2 Barriers for adoption of digital innovations

Regarding to the barriers that are related to the adoption of digital innovation,
reviewing literature in the five IoT application domains revealed that many of
these barriers are related to high cost, interoperability, lack of standardization,
security and privacy, high energy consumption, stigma for end-users, data
accuracy, trust, functionality and usability, lack of knowledge about the digital
technologies and innovations, unclear value for end-users, complexity of use and
finally accessibility of the digital technologies for end-users. Although the
adoption barriers are different depending to the application domain that was
studied. For example, stigma for end-users is particularly related to the wearable
sensors, in a sense that the user might feel ashamed because others interpret
the usage of the device as a sign of weakness or illness.
Regarding the results of the two workshops with IoT experts, the main barriers
when adopting IoT technologies were gathered and clustered into four main
categories: most important, most challenging, most expensive and most
common.

In total 45 adoption barriers were identified in the first workshop (in Bilbao) and
23 adoption barriers were identified in the second workshop (in Vienna).
Together with the participants, some of the items that were similar were
eliminated and the rest of the items were combined and clustered into the main
four categories. Also, recommendations on how to tackle IoT adoption barriers
were collected.

In brief, the most challenging adoption barriers were related to the Complexity,
intrusive and loss of control, standardization, interoperability and the relevance
of technology. When it comes to the most important barriers, isolation from the
society, killing creativity and RoI issues for end-users, usability and functionality
and relevance for end-users have been highlighted in the workshops. Regarding
the most common adoption barriers, unnecessary data gathering, force changes
on users’ daily life and the danger of cheating, legal unclarities, lack of
transparency (including Terms & Conditions) and data misuse of data controllers
have been identified. And finally, communication and power consumption issues,
design constraint like devices size and data protection have been stated as the
most expensive adoption barriers by the workshop participants.

The findings from the literature review and both workshops then were combined
and formed the basis to develop interview protocols to interview both IoT experts
(25 experts in two rounds has been conducted) as well as end-users (30
interviews have been done and 20-30 interviews are in planning and progress).

445
As mentioned in the methodology, the interviews with the IoT experts have been
already completed and the end-user’s interviews are still ongoing. The interview
results will help me to further improve my knowledge and will allow me to present
realistic practical recommendations on how to tackle these adoption barriers.

5 Thesis Contribution
My thesis will contribute to the body of knowledge in the field of Living Labs and
digital innovations by understanding the barriers for digital innovations to be
tested and adopted by end-users in a Living Lab context. Moreover, investigating
the application of diffusion of innovation theory (Rogers, 2010) in a Living Lab
context and more particularly from end-users perspective is another premiere
contribution of this study as this theory has largely been used within the
organizational context.
My thesis will also investigate the consequences of drop-out in Living Lab field
tests and provides recommendations that would facilitate prolonged user
engagement in a Living Lab activity. On the other hand, by identifying adoption
barriers for digital innovations from end-users perspective, this study, presents
some practical suggestions on how these barriers for the adoption of digital
innovations should be tackled.

This study will also summarize the key lessons learned from the conducted field
tests, workshops, interviews with a number of end-users and experts in Living
Lab and digital innovations field and points to avenues for future research.

Relevant Publications
The following articles will form the basis for my PhD dissertation and will be
(partially) included in the thesis:
1. Habibipour, A., Bergvall-Kåreborn, B., & Ståhlbröst, A. (2016). How to sustain
user engagement over time: A research agenda. In 22nd Americas
Conference on Information Systems: Surfing the IT Innovation Wave, AMCIS
2016, San Diego, United States, 11-14 August 2016.
2. Habibipour, A., Padyab, A., Bergvall-Kåreborn, B., & Ståhlbröst, A. (2017,
August). Exploring Factors Influencing Participant Drop-Out Behavior in a
Living Lab Environment. In Scandinavian Conference on Information
Systems (pp. 28-40). Springer, Cham.
3. Habibipour, A., Georges, A., Ståhlbröst, A., Schuurman, D., & Bergvall-
Kåreborn, B. (2018). A Taxonomy of Factors Influencing Drop-Out Behaviour
in Living Lab Field Tests. Technology Innovation Management Review, 5-21.
4. Habibipour, A., Ståhlbröst, A., Georges, A., Bergvall-Kåreborn, B., &
Schuurman, D. (2018). Drop-out in living lab field test: analyzing
consequences and some recommendations. In 26th European Conference
on Information Systems (ECIS2018), Portsmouth, UK, 23–28 June 2018.
5. Chronéer, D., Ståhlbröst, A., & Habibipour, A. (2019). Urban Living Labs:
Towards an Integrated Understanding of their Key Components. Technology
Innovation Management Review, 9(3), 50.
6. Habibipour, A., Padyab, A., & Ståhlbröst, A. (2019, August). Societal, Ethical
and Ecological Issues in Wearable Technologies. In Twenty-fifth Americas
Conference on Information Systems, Cancun, 2019

446
7. Padyab, A., Habibipour, A., & Ståhlbröst, A. (2019). End-User Engagement
Methodologies in IoT development. Book Chapter In Springer Series,
submitted April 2019

References
Alter, S. (2008). Defining information systems as work systems: Implications for
the IS field. European Journal of Information Systems, 17(5), 448–469.
https://doi.org/10.1057/ejis.2008.37.
Bergvall-Kåreborn, B., Eriksson, C., & Ståhlbröst, A. (2015). Places and Spaces
within Living Labs. Technology Innovation Management Review, 5(12),
37–47.
Bergvall-Kareborn, B., Holst, M., & Stahlbrost, A. (2009). Concept design with a
living lab approach. 1–10. IEEE (2009).
Boston College, Fichman, R. G., Dos Santos, B. L., University of Louisville,
Zheng, Z. (Eric), & University of Texas at Dallas. (2014). Digital Innovation
as a Fundamental and Powerful Concept in the Information Systems
Curriculum. MIS Quarterly, 38(2), 329–343.
https://doi.org/10.25300/MISQ/2014/38.2.01.
Brønnum, L., & Møller, L. (2013). The Dynamics and Facilitation of a Living Lab
Construct. ISPIM Conference Proceedings; Manchester, 1–11. Retrieved
from
https://search.proquest.com/docview/1690238048/abstract/2E6196CBD6
3D4275PQ/1
Chesbrough, H. (2003). Open innovation. Boston: Harvard Business School
Press.
Chesbrough, H. (2006). Open innovation: a new paradigm for understanding
industrial innovation. In Open Innovation: Researching a New Paradigm,
Oxford: Oxford University Press, (Journal Article), 1–12.
Claude, S., Ginestet, S., Bonhomme, M., Moulène, N., & Escadeillas, G. (2017).
The Living Lab methodology for complex environments: Insights from the
thermal refurbishment of a historical district in the city of Cahors, France.
Energy Research & Social Science, 32(Supplement C), 121–130.
https://doi.org/10.1016/j.erss.2017.01.018.
Davis, G. B., & Olson, M. H. (1984). Management information systems:
Conceptual foundations, structure, and development. McGraw-Hill, Inc.
De Moor, K., Berte, K., De Marez, L., Joseph, W., Deryckere, T., & Martens, L.
(2010). User-driven innovation? Challenges of user involvement in future
technology analysis. Science and Public Policy, 37(1), 51–61.
Følstad, A. (2008). Living Labs for Innovation and Development of Information
and Communication Technology: A Literature Review. 99-131. Retrieved
from https://brage.bibsys.no/xmlui/handle/11250/2440026

447
Georges, A., Schuurman, D., & Vervoort, K. (2016). Factors affecting the attrition
of test users during living lab field trials. Technology Innovation
Management Review, (Journal Article), 35–44.
Hirschheim, R., & Klein, H. K. (2012). A Glorious and Not-So-Short History of the
Information Systems Field. Journal of the Association for Information
Systems; Atlanta, 13(4), 188–235.
Howe, J. (2006). The rise of crowdsourcing. Wired Magazine, 14(6), 1–4.
Kaplan, B., & Maxwell, J. A. (2005). Qualitative research methods for evaluating
computer information systems. In Evaluating the organizational impact of
healthcare information systems (Vols. 1–Book, Section, pp. 30–55).
Springer.
Land, F. (1985). Is an information theory enough? The Computer Journal, 28(3),
211–215.
Lee, A. S. (2001). Editorial. MIS Quarterly, 25(1), iii–vii.
Ley, B., Ogonowski, C., Mu, M., Hess, J., Race, N., Randall, D., … Wulf, V.
(2015). At home with users: a comparative view of Living Labs. Interacting
with Computers, 27(1), 21–35.
Luna-Reyes, L. F., Zhang, J., Gil-García, J. R., & Cresswell, A. M. (2005).
Information systems development as emergent socio-technical change: A
practice approach. European Journal of Information Systems, 14(1), 93–
105. https://doi.org/10.1057/palgrave.ejis.3000524
Lyytinen, K., & Newman, M. (2008). Explaining information systems change: A
punctuated socio-technical change model. European Journal of
Information Systems, 17(6), 589–613. https://doi.org/10.1057/ejis.2008.50
Lyytinen Kalle, Yoo Youngjin, & Boland Jr. Richard J. (2015). Digital product
innovation within four classes of innovation networks. Information Systems
Journal, 26(1), 47–75. https://doi.org/10.1111/isj.12093
Maxwell, J. A. (2012). Qualitative research design: An interactive approach. Sage.
Mulder, I. (2012). Living Labbing the Rotterdam Way: Co-Creation as an Enabler
for Urban Innovation. Technology Innovation Management Review;
Ottawa, 2(9), 39–43.
Mumford, E. (2006). The story of socio-technical design: Reflections on its
successes, failures and potential. Information Systems Journal, 16(4),
317–342.
O’Brien, H. L., & Toms, E. G. (2008). What is user engagement? A conceptual
framework for defining user engagement with technology. Journal of the
American Society for Information Science and Technology, 59(6), 938–
955.
Pedersen, J., Kocsis, D., Tripathi, A., Tarrell, A., Weerakoon, A., Tahmasbi, N.,
… De Vreede, G.-J. (2013). Conceptual foundations of crowdsourcing: a
review of IS research. 579–588. IEEE.
Rogers, E. M. (2010). Diffusion of Innovations, 4th Edition. Simon and Schuster.

448
Sarker, S., Sarker, S., Xiao, X., & Ahuja, M. (2012). Managing Employees’ Use of
Mobile Technologies to Minimize Work-Life Balance Impacts. Faculty
Scholarship. Retrieved from https://ir.library.louisville.edu/faculty/359
Schuurman, D. (2015). Bridging the Gap between Open and User Innovation?:
Exploring the Value of Living Labs as a Means to Structure User
Contribution and Manage Distributed Innovation, (Journal Article).
Ståhlbröst, A. (2008). Forming future IT - The living lab way of user involvement,
Doctoral dissertation. Luleå tekniska universitet, 2008.
Ståhlbröst, A., & Bergvall-Kåreborn, B. (2013). Voluntary Contributors in Open
Innovation Processes. In Managing Open Innovation Technologies (pp.
133–149). https://doi.org/10.1007/978-3-642-31650-0_9
University of Wisconsin–Milwaukee, Nambisan, S., Lyytinen, K., Case Western
Reserve University, Majchrzak, A., University of Southern California, …
Xi’an Technological University. (2017). Digital Innovation Management:
Reinventing Innovation Management Research in a Digital World. MIS
Quarterly, 41(1), 223–238. https://doi.org/10.25300/MISQ/2017/41:1.03
Yoo, Y., Henfridsson, O., & Lyytinen, K. (2010). Research Commentary—The
New Organizing Logic of Digital Innovation: An Agenda for Information
Systems Research. Information Systems Research, 21(4), 724–735.
https://doi.org/10.1287/isre.1100.0322

449
 Beyond participation: exploring citizen
stakeholder empowerment in the co-creation of
innovation
Shelly Tsui1

1 Eindhoven University of Technology, The Netherlands

Category: Doctoral Papers

Disciplines: Science and Technology Studies; Science, Technology and

Innovation; Ethics of Innovation
PhD date: May 15, 2018
PhD Supervisors: Prof. Dr. Isabelle Reymen, Dr. Anna Wieczorek, and Dr.
Gunter Bombaerts

Abstract
This paper describes initial considerations in a dissertation research on how to
co-create ecosystemic business models based on shared resources and value
in an open innovation. There are two main research areas that address the topic
of ecosystemic business models: open innovation and innovation management,
in respect of innovation ecosystems, and business model development, in
respect of business ecosystems. This research will be an attempt to bring two
areas together by using multi-stakeholder perspective as the bridge between the
two. The proposed approach is to use service design as a methodological choice
for multi-stakeholder business model development as a core of open innovation.

Keywords: Open innovation, Ecosystemic business models, Business model

development, Innovation ecosystem

450
1 Context and motivation
In the public engagement dimension of the Responsible Research and
Innovation research framework, the European Commission is interested in
identifying ways to better engage society with innovation in science and
technology (Stilgoe, Owen, & Macnaghten, 2013). The aim is not only to achieve
outcomes of innovation that match the needs of society but also to “foster a more
scientifically literate society of knowledge-driven and empowered citizens”
(European Comission, n.a.). As such, co-creation is proposed as a means to
achieve this is based upon active collaboration among relevant stakeholders in
a given (innovation) project that connects industry, society, academia, and policy
makers (to varying degrees and involvement) (Ramaswamy & Ozcan, 2014;
Vargo, Maglio, & Akaka, 2008; Voorberg et al, 2017). Several ways in which co-
creation can be practiced are in the form of public procurement of innovation
(similar to public- private partnerships), co-creation facilities (test-beds, maker
spaces, and fab labs), and living labs.

Living labs are an increasingly utilised tool in which citizens are becoming
involved in experimental technological innovations to solve societally relevant
problems (Bergvall-Kåreborn & Ståhlbröst, 2009; Brankaert & den Ouden, 2017;
Voytenko et al, 2015). In a sense, the living lab approach has become a social
policy tool through which citizens are expected to become part of a collective
way of dealing and solving societal problems. With this comes the clear
transformation of the citizen into an active citizen-subject: a citizen who
participates in the living lab while at the same time, is a subject of the lab. This
raises several interesting issues such as the new role and expectations for
citizens, and how to engage and involve them in innovation, and for what
purposes in light of the varying interests of the different stakeholders involved.

As such, I am focusing particularly on the topic of citizen stakeholder

empowerment in and though co-creation. Th concept of empowerment is an
emerging theme in co-creation research and it can be a lens to highlight several
challenges of co-creation by identifying how asymmetries of stakeholders'
resources and capacities and perceived influence affects the extent to which co-
creation can be a promising approach to solving societal issues (Füller et al,
2010; Späth & Scolobig, 2017).

Lastly, my research is part of a broader Horizon 2020-funded project titled

“Scaling Up Co- creation: Avenues and Limits for Integrating Society in Science
and Innovation” (SCALINGS). SCALINGS aim to study how co-creation operates
across different national settings in two technical domains: urban energy and
robotics. Through a comparative approach using case studies from 9 different
countries such as the Netherlands, Germany, Denmark, and Poland, the goal is
to identify the conditions relevant in influencing the mainstreaming and upscaling
of the co-creation approach in living labs, public procurement of innovation, and
co-creation facilities. The overarching goal to contribute to developing a policy
roadmap on how to better support innovation initiatives in Europe (SCALINGS,
2018).

451
2 Aims, Objectives, and research questions
The aim of my research is to explore the extent to which the concept of
empowerment can serve as a critical and evaluative lens on the co-creation
approach as a policy tool. Co-creation increasingly requires the engagement of
societal stakeholders such as citizens and with this new development comes a
number of concerns. Since the opening up of the innovation process through
user/stakeholder inclusion, participatory and human-centred thinking in the
design world, (Bjögvinsson, Ehn, & Hillgren, 2012; Chesborough, 2003; Sanders
& Stappers, 2008; von Hippel, 1986), issues raised have ranged from more
procedural aspects on who and how to engage users, to more ethical ones such
as how to account for human values such as democracy, fairness, transparency
i.e. value-sensitive design (Friedman, Kahn, & Borning 2002).

While attention has been paid to users from various disciplines such as
management science to ethics, the language used by co-creation to describe the
so-called "co-creators" is still based on seeing them as users, rather than societal
stakeholders such as citizens. This is important to highlight as co-creation is now
increasingly used in the political context as a social policy tool, and due
acknowledgement must be paid to the emerging implications. Citizens, according
to the European Commission, are expected to be transformed into empowered
citizens through co- creation. However, how this will occur is not evident or well-
articulated in the literature.

Therefore, the aim of this research is to investigate how the co-creation approach
can lead to the empowerment of stakeholders in our innovation-driven world by
studying how different stakeholders engage and are engaged in co-creation-
based innovation projects in current literature. In other words, how would
empowerment through co-creation manifest, and how would the concept of
empowerment serve to evaluate the potential of co-creation to bring about the
empowered citizen?

As the Commission has a vision of an empowered citizen that is able to

understand the role and importance of innovation, co-creation has to be more
than just participation to achieve outcomes, and has to be investigated for its
potential to “transform” stakeholders (especially citizens) into those whom are
capable of participating in collaborative innovation in general, and not just in
single projects, without being reduced to token participants or ‘free labor’ that
only serve to benefit industrial interests (Zwick, Bonsu, & Darmody, 2008).

As such, my interest lies in understanding in how and in what way can co-
creation influence stakeholder engagement in such a way that it leads to 1) active
participation i.e. constructive and critical dialogue that allows stakeholders to
reflect on their role and potential in all relevant parts of the project and to be able
to shape the co-creation process in a way that suits their needs and visions, and
2), further curiosity and drive to initiate similar or different co-creation projects
(i.e. scaling up of the co-creation instruments). To conceptualise this, I draw upon
the concept of empowerment as a lens to evaluate co-creation's dynamic with
citizens as opposed to treating them as the traditional user, and how

452
empowerment as such can be reflected in and through the practices of the living
lab.

To illustrate this, my research investigates the case of Jouw Licht Op 040 (Your
Light on 040), a living labs-based project that aims at improving the quality of life
of citizens of Eindhoven, a Dutch city in the Netherlands, through smart lighting
solutions. The project is based on co- creative collaboration quadruple-helix set-
up consisting of the municipality, citizens, business partners, and researchers.

To guide the investigation, the main research question is: how can co-creation
empower stakeholders in the innovation process? This conceptualization of my
research problem is further elaborated in the problem statement.

3 Keywords
Co-creation
Stakeholder empowerment
Living lab
Urban energy
Public engagement

4 Problem statement
Despite the increasing use of co-creation as a social policy tool rather than just
a marketing strategy, current literature on co-creation has mainly focused on
promoting its benefits from a participatory perspective for consumers and users,
and not societal stakeholders i.e. co-creation is mainly practiced, and ought to
be practiced, because it allows for interests and needs to be better met by
(Ramaswamy & Ozcan, 2014; Vargo, Maglio, & Akaka, 2008). In other words,
language use to describe co-creation is often about that activities that contribute
to the firm’s innovation process through improving organisational capacities, or
managing stakeholders like a resource (Kazadi, Lievens, & Mahr, 2016; Sharma,
Conduit, & Hill, 2014). Furthermore, it also focuses on how the firm can deal with
these challenges, often to their benefit. This leaves the other stakeholders out of
the equation with regard to ensuring that their interests is met, aside from being
delivered a product that they want, which makes co-creation appear to be more
about how users can be useful in developing making the product or service than
being an equal partner (i.e. co-creator).

While it is indeed important to know how co-creation should be practiced,

however, the connection between how co-creation can lead to empowerment
has not been addressed, namely on how empowerment is understood as in co-
creation, and how empowerment is enacted through co-creation. Understanding
why empowerment is important to explore is because it is about the possible
societal impact of co-creation as a social process rather than simply a means
limited to how businesses strategically manage and harness competences in
product and service development. Understanding empowerment is especially
pertinent when collaborating with citizens as without being aware of the issues
that come with engaging them, co-creation runs the risk of turning into mere

453
participation i.e., an “empty ritual of participation and having [no] real power
needed to affect the outcome of the process (Arnstein, 1969), for example. This
could lead to disenchantment in such initiatives, and discourage future
participation.

However, much on co-creation has centered on improving the participatory

element in innovation, but not on how the participation or procedural practices
relate to empowerment. There have been few attempts to account for
empowerment in co-creation. For example, Späth and Scolobig (2017) have
conceptualized empowerment of stakeholders as the degree (how much and
how often) and quality (of what was expected in terms of actions from the
stakeholders) of participation. In their research, they studied stakeholder
empowerment in the planning process of developing electricity transmission lines
in France and Norway. They operationalized empowerment on three levels:
information, consultation, and cooperation. On information, this refers to
stakeholders receiving information; on consultation, stakeholders are asked for
their thoughts and opinions; and lastly, on cooperation, stakeholders'
perspectives are incorporated in the decisions made i.e. there is power-sharing
in this context. They measured the degree to which, throughout the planning
process, the stakeholders and process owners interacted with each other on
those three dimensions. This is a good attempt at exploring empowerment
through how it manifests in stakeholder engagement practices, and can be useful
for transforming an abstract concept such as empowerment into one that can
inform what practices can enact/reify empowerment. In another study from a
psychological angle, Füller et al (2014), show how empowerment is about the
perceived influence. Through a mass survey, they studied how users in a virtual
co-creation project perceived their level of influence, which in turn affected their
feeling and perception of being empowered. The perceived influence has an
effect on the willingness of users to participate in future projects. From these two
studies, in addition to operationalising empowerment in practices that may be
self-evident instances of empowerment (e.g. Information sharing might be
considered empowering because it allows a stakeholder to be informed, which
is useful for making informed choices), perception of empowerment is also
important to consider as practices embody subjective notions of what could or
could be empowering. Perhaps for some stakeholders, information sharing might
be seen as a low-level way of being empowered. Rather than presume that all
practices would have the same or equally empowering effect, it would also be
wise to supplement the study of how practices enact empowerment with the
perception of empowerment. By doing so, this pays due attention to the
differences among the stakeholders, and does not presume that co-creation
practices can be summarised in a one-size-fits-all list that will be applied in all
contexts. A sensitivity to local contexts such as culture, politics, and social
infrastructure must be developed as well, despite the good intentions of
empowerment.

As shown, empowerment in co-creation is still a new yet fertile ground for further
exploration. If the hope for co-creation is to realize the ideal vision of public
engagement by the Commission, and more generally the potential for co-creation
to better align innovation and societal interests, then there is a need to

454
conceptualize a form of co-creation that empowers its stakeholders. In the
context of involving citizens, this insight is important as co-creation practices
reflect ideas of engagement. This in turn can shape participation in not just co-
creation projects but broader debates on science, technology, and innovation.

5 Research methods
In order for co-creation to be more than just an approach that is focused solely
on improving and maximizing the more procedural aspects (i.e. how
stakeholders participate and are managed in a project), this paper suggests that
empowerment of stakeholders should be a focal aim of co- creation. In order to
make a case for empowerment’s relevance for co-creation, the research first
provides an overview of how empowerment is currently treated in the co-creation
literature.

Thereafter, literature on empowerment from development studies is discussed,

with the aim of identifying key dimensions that could transform co-creation’s
current approach to connecting businesses, society, and innovation in the
creation of social value.

On how the literature was selected, a SCOPUS search was conducted with the
key words “co- creation” and “empowerment” in either the title of articles or in the
key words of the abstract. The articles were then checked for relevance, which
was determined by the following question: “Does the author(s) addressed how
co-creation can empower, or discuss how empowerment can be used to improve
co-creation?” From this, just two papers were determined to be relevant (Füller,
Mühlbacher, Matzler, & Jawecki, 2014; Späth & Scolopig, 2017) in that they
addressed empowerment from the perspective of consumers through co-
creation.

For the study of JLO040, a qualitative approach has been taken in terms the
collection of empirical material. Recorded semi-structured interviews of around
one hour and stakeholder observations were the main methods for collecting
data, with the JLO040 project partners being the subject. Currently, 6 of 8
interviews have been completed, with the remaining 2 planned in August. The
interviews are in the process of being transcribed and coding will focus on looking
for quotes pertaining to descriptions of co-creation, citizen participation, and
empowerment.

Seen as discourses, they can help serve to understand how the project is set-
up, why, and the ideas behind them that shape the practices such as organised
activities. In addition to this, documents and websites pertaining to JLO040
(Facebook and Twitter accounts included) are also collected to provide
background information on the project. They are also framed as practices of co-
creation that will be analysed through the empowerment lens (e.g. As providing
information for knowledge building, transparency, etc).

455
6 Preliminary findings
The project, while it is focused on finding and developing innovative ways to
address urban lighting problems, is not the primary focus of JLO040. It is a public
service developed by the municipality with the aim of providing a service to its
citizens. In other words, the initiative is driven by an interest in creating public
value. From the perspective of the industrial partners, there has been difficulty in
reconciling the municipality’s aim with their own, which is primarily to innovate
and create economic value for their own business. Furthermore, the TU/e
researchers put in charge of conducting research with the citizens also have
difficulty in bringing across their approach, which is aimed at investigating
broader developments to understand how interest in urban lighting projects can
be fostered, developing conflict resolution methods, and similar issues take
place. As such, the various interests or “logics” from these three main
stakeholders appear to create challenges for co-creation, especially when it
comes to seeing stakeholders as “equal” partners and their value of input in the
process.

Despite developing and implementing lighting solutions for values such as

interesting, safety, and to make the city more beautiful that contribute to the
quality of the inhabitants lives, appears to only use the citizens’ participation and
input to inform what technical solutions are to be created. Activities are limited to
open dialogue (“co-creation workshops” or information sessions whereby the
citizens’ concerns and interest are noted down and used by the municipality and
industrial partners to inform the technology that will be under development to
address the issues identified. As such, citizens are in some way reduced to mere
subjects of data collection, and have limited involvement in actively shaping the
process of the project (e.g. project planning, how the activities or which activities
should be organized, and the direction of the project). This is due to the fact that
the industrial partners want to see a return on their investment in innovation, and
have budgeted limited time and resources to work on each testing zone. This
limits the kind of activities that can take place, and thus affects how citizens are
involved in the process. Building their capacities (in the sense of empowerment)
and knowledge base (that can be used further down the line) on innovation is not
a clear aim or outcome of the project i.e., co- creation as observed in this project
is reduced to treating participation and engagement as apolitical and pragmatic
procedures that allows data to be collected for developing suitable technologies
to address the needs identified.

456
References
Arnstein, S. (1969). A Ladder of Participation. Journal of the American Planning
Association, 35(4), 216-224.
Bergvall-Kåreborn, B. & Ståhlbröst, A. (2009). Living Lab - An Open and Citizen-
Centric Approach for Innovation. International Journal of Innovation and
Regional Development, 1(4), 356-370.
Bjögvinsson, E., Ehn, P., Hillgren, P-A. (2012). Design Things and Design
Thinking: Contemporary Participatory Design Challenges. Design Issues,
28(3), 101-116.
Brankaert, R., & Ouden, Eden. 2017. The Design-Driven Living Lab: A New
Approach to Exploring Solutions to Complex Societal Challenges.
Technology Innovation Management Review, 7(1): 44-51.
Chesbrough, H. (2003). Open innovation: The new imperative for creating and
profiting from technology.
Boston, MA: Harvard Business School Press.
European Commission. (n.d.). Public Engagement in Responsible Research and
Innovation .
Retrieved from European Commission:
https://ec.europa.eu/programmes/horizon2020/en/h2020-section/public-
engagement- responsible-research-and-innovation
European Network of Living Lab. (n.d.). About Us - What is ENoLL? Retrieved
from: https://enoll.org/aboutus/
Friedman, B., Kahn, P. H. Jr., & Borning, A. (2002). Value sensitive design:
Theory and methods. University of Washington technical report. Seattle:
University of Washington.
Füller, J., Mühlbacher, H., Matzler, K., & Jawecki, G. (2010). Consumer
Empowerment Through Internet-Based Co-creation. Journal of
Management Information Systems, 26(3), 71-102.
Hippel, E. von. (1986). Lead users: a source of novel product concepts.
Management Science, 32(7), 791-805.
Jouw Licht op 040. (n.d.). Retrieved from Jouw Licht op 040:
https://www.jouwlichtop040.nl/
Kazadi, K., Lievens, A., & Mahr, D. (2016). Stakeholder co-creation during the
innovation process: Identifying capabilities for knowledge creation among
multiple stakeholders. Journal of Business Research, 69(2), 525-540.
Ramaswamy, V. & Ozcan, K. (2014). The Co-creation Paradigm. Stanford:
Stanford University Press.
Sanders, E. B.-N., & Stappers, P. J. (,2008). Co-creation and the new landscapes
of design. Co- Design, 4(1), 5-18.
SCALINGS. (2018). Retrieved from SCALINGS: https://www.scalings.eu

457
Sharma, S., Conduit, J., & Hill, S. R. (2014). Organisational capabilities for
customer participation in healthcare service innovation. Australasian
Marketing Journal, 22(3), 179-188.
Späth, L. & Scolobig, A. (2017). Stakeholder empowerment through participatory
planning practices: the case of electricity transmission lines in France and
Norway. Energy Research & Social Science, 23, 189-198.
Stilgoe, J., Owen, R., & Macnaghten, P. (2013). Developing a framework for
responsible innovation. Research Policy, 42, 1568-1580.
Vargo, S. L., Maglio, P. P., & Akaka, M. A. (2008). On value and value co-creation:
A service systems and service logic perspective. European Journal of
Management, 26, 145– 152.
Voorberg, W., Bekkers, V., Timeus, K., Tonurist, P., & Tummers, L. (2017).
Changing public service delivery: learning in co-creation. Policy and
Society, 36(2), 178-194.
Voytenko, Y., McCormick, K., Evans, J., & Schliwa, G. (2015). Urban living labs
for sustainability and low carbon cities in Europe: towards a research
agenda. Journal of Cleaner Production, 123, 45-54.
Zwick, D., Bonsu, S. K., & Darmody, A. (2008). Putting Consumers to Work: 'Co-
creation' and new marketing govern-mentality. Journal of Consumer
Culture, 8(2), 163-196.

458
 Can Open Innovation offer new perspectives for
the development of ecosystemic business
models?
Julia Nevmerzhitskaya1

1Laurea University of Applied Sciences, Finland

Category: Doctoral Papers

Disciplines: Marketing
PhD starting date: 2004
PhD Supervisor: Martti Laaksonen

Abstract
This paper describes initial considerations in a dissertation research on how to
co-create ecosystemic business models based on shared resources and value in
an open innovation. There are two main research areas that address the topic of
ecosystemic business models: open innovation and innovation management, in
respect of innovation ecosystems, and business model development, in respect
of business ecosystems. This research will be an attempt to bring two areas
together by using multi-stakeholder perspective as the bridge between the two.
The proposed approach is to use service design as a methodological choice for
multi-stakeholder business model development as a core of open innovation.

Keywords: open innovation, ecosystemic business model, business model

development, innovation ecosystem.

1 Introduction
Business model is a concept used to describe the way a company creates value
(e.g. Chesbrough, 2010). In practical terms business models refer to the way

459
companies generate revenue. In the academic literature business models have
traditionally been described as a business logic for customer value creation
process (e.g. Vargo and Lusch, 2008). This business logic sees value creation
process as a linear value chain, in which customer value is created inside an
organization through its’ own activities, adding to the suppliers’ resources which
serve as an input. While no unified definition of a business model exist, many
scholars see business model as a firm-specific concept. For example,
Osterwalder et al. (2005, 7) define a business model as “a conceptual tool that
contains a set of elements and their relationships and allows expressing the
business logic of a specific firm”. A lot of research has been dedicated to the
elements of business models, which are combined in so-called business model
canvases. Probably the most known business model canvases are the Business
Model Canvas by Osterwalder and Pigneur (2010), Service business model
canvas (Zolnowski and Böhmann, 2014) and Service logic business model
canvas (Ojasalo and Ojasalo, 2015). They are based on what Chesbrough and
Rosenbloom (2002), describe as business model functions:
● Value proposition (i.e., the value created for users by an offering)
● Market segment(s)
● Revenue generation mechanism
● Structure of the value chain
● The cost structure and profit potential (given value proposition and value
chain structure)
● The position of the firm within the value network linking suppliers and
customers
● The competitive strategy by which the innovating firm will gain and hold
advantage over rivals.

The service business model canvas (SBMC) by Zolowski and Böhmann is based
on the BMC and concentrates on the co-creation and customer value creation
process. Ojasalo and Ojasalo (2015) added a service- dominant perspective to
the business model canvas, addressing above functions not only from a company
point of view, but also from a consumer perspective.

While available business model research offers a number of tools such as

business model canvases to help companies develop and manage their business
models, they all are centered on value creation and capture within one company,
and represent business models centered around one firm’s business logic. At the
same time the challenges we face globally (such as climate change, healthcare,
immigration) provide a significant opportunity to create new shared value through
innovative partnerships beyond the individual business model. Such partnerships
can form business systems, or ecosystems, which have the potential to provide
significant economic benefits not only for the companies, but for a broader society.
In addition, digitalization and rapid development of digital technologies, as well as
their connection to a physical world, created new conditions for highly
interconnected business environments (Livari et al, 2016). The authors argue that
traditional firm-centered models are not suitable for addressing new types of
interconnected environments based on IoT, and thus “joint efforts for synergistic
value creation and capture between all stakeholders” are needed.

460
One trend in which joint or shared value creation among different stakeholders is
a must is Circular Economy (CE) which is one of the complex problems focused
on optimizing resource-use and minimising the creation of waste across different
value chains. Importantly, CE is not about one company changing one product,
it’s about all the interconnected companies that form infrastructure and economy,
coming together and re- thinking the operating system itself (Ellen MacArthur,
2015). Thus, in developing and testing CE solutions, diversity and multi-
stakeholder collaboration is built-in as a compulsory requirement, which makes it
an excellent case example for describing and developing ecosystemic business
models.

Preliminary objectives of the research are the following:

● to define the role of multi-stakeholder engagement in developing
ecosystemic business models.
● to investigate dynamics of open innovation.
● to develop processes (e.g. facilitation of co-creation) and tools (e.g.
ecosystemic business model canvas) to help companies in transition
towards ecosystemic businesses.

In this paper, the theoretical context of ecosystemic business models is shortly

presented. Definitions are given to serve as the starting point for analysis, and the
initial framework for connecting innovations and business models in an ecosystem
is presented. This initial framework serves as the basis for discussion and further
development during the dissertation process.

2 Theoretical foundations of ecosystemic thinking

Open innovation is a paradigm that assumes that firms can and should use
external ideas as well as internal ideas, and internal and external paths to market,
as firms look to advance their technology’ (Chesbrough, 2003, p. XXIV).
Chesbrough’s (2003) definition has developed over the years. In 2006 (p.1), he
states that ‘open innovation is the use of purposive inflows and out-flows of
knowledge to accelerate internal innovation, and expand the markets for external
use of innovation, respectively’. He also noted that OI is becoming wide-ranging;
‘the future of open innovation will be more extensive, more collaborative, and
more engaged with a wider variety of participants. (Chesbrough, 2010). Open
innovation (Chesbrough, 2006) approach provides a generic framework for
involving end-users (a.k.a. customers) and other relevant key stakeholder in the
collaborative innovation process. Nowadays innovation processes have become
more and more of a joint organizational effort in which the partner section plays a
critical role. Importantly, partners within an open innovation network should
provide the resources and capabilities which your own organization lacks in order
to gain the suggested positive effects of collaborating and additional capabilities.
Open Innovation is based on quadruple helix model of innovation where civil
society joins with business, academia, and government sectors to drive changes
far beyond the scope of what any one organization can do on their own.

461
Ecosystemic innovations: An innovation is considered ‘ecosystemic’ when its
purpose is to change the fundamental nature of society; for instance, to deliver on
major transitions concerning ecological sustainability. Importantly, partners within
an open innovation network, that can also described as an ‘innovation ecosystem’
should provide the resources and capabilities which their own organization lacks
in order to gain the suggested positive effects of collaborating and additional
capabilities. While Open Innovation approach allows companies to move from
traditional closed RD activities towards more open collaborative processes, it is
still focused mainly on one company’s ability to innovate. True open innovation
process, in which shared value is co-created by a large number of stakeholders
involved, is often referred as Open Innovation 2.0. Open Innovation 2.O (OI2) is
a new paradigm based on principles of integrated collaboration, co-created
shared value, cultivated innovation ecosystems, unleashed exponential
technologies, and extraordinarily rapid adoption. (Curley, Salmelin; 2013). Open
Innovation 2.0 is about shared value creation in networks, or ecosystems.

Open Innovation 2.0 allows companies to develop ecosystemic business models,

which focus both on the firm's method of creating and capturing value as well as
any part of the ecosystem's method of creating and capturing value to the
ecosystem. In Open Innovation 2.0, innovation happens in ecosystems or
networks that go far beyond traditional organizational boundaries and include
organizations in a value chain, individuals/citizens, academia, and governments
collaborating with one another. Together such ecosystems can create value in a
way no single organization could do by itself. In other words, OI2 allows for
development of collaborative business models which are ecosystem-centered as
opposed to firm-centered.

This paper is based on the following preliminary conceptual framework of multi-

stakeholder value creation in ecosystemic business models shown in Figure 1.

Business model innovation (Chesbrough, 2010) is about defining and designing

new models for capturing business value. In the context of this research proposal,
the ecosystemic business models are used to investigate collaborative business
models of companies and other organizations. The ecosystems thinking is often
connected to the concept of open innovation and business networks but has no
clear definition. Valkokari (2015) distinguishes between business ecosystems,
innovation ecosystems, and knowledge ecosystems. Business ecosystems focus
on customer value creation. Knowledge ecosystems focus on the generation of
new knowledge, and innovation ecosystems occur as an integrating mechanism
between the exploration of new knowledge and its exploitation for value co-
creation in business ecosystems. Both the relationship and dynamics of these
three ecosystems are an important research themes, and new tools are needed
to enable cross-ecosystem models (Valkokari, 2015).

462
 Figure 1. Conceptual Framework

3 Research methodology
The proposed methodology is grounded on a constructive action research
paradigm (Cassel and Johnson, 2006) in context of a case study (Yin, 1994).
Constructive action research method aims to develop a solution to a practically
relevant problem by applying theoretical knowledge and demonstrating the
functioning and innovativeness of the suggested solution in real life (Jaatinen and
Lavikka, 2008). Action research will be carried out in a number of Living Labs.
Living Lab approach is based on collaboration, social learning and management
sense-making theories, and open innovation and co-creation where
multidisciplinary and multicultural teams co-develop user-centered solutions for a
real-life challenge. The LL adopts a participatory oriented approach in which
complex socio-technical systems can be addressed based on the practices of co-
creation and co-production. Living Lab research has emerged as an answer to
complex societal challenges and has significant potential to promote both open
innovation and co-creation of products and services.

In this study, Circular Economy (CE) as a complex problem is providing a thematic

framework for a practically relevant problem noteworthy to be solved. A project
supported by the European Commission’s H2020 circular economy programme
including 17 partners across 8 EU counties and having a total of 7.2 MEUR find
is providing the challenges and testing environment for the concept development.
The project involves a diverse set of industries including 1) recycling waste of

463
electrical and electronic equipment (WEEE), 2) LED light industry, 3) micro
farming and 4) meat supply chain.

The research milestones will follow a framework originally proposed by Kasanen

et. al. (1993) and refined by Oyegoke (2011) as follows:
1. Justify the practical relevance of the proposed problem (i.e. CE as complex
problem which requires multi-stakeholder value co-creation).
2. Present the theoretical connection (i.e. open innovation and ecosystemic
business models as presented in section 2 covering theoretical
framework).
3. Develop the solution (i.e. ecosystemic business models in OI).
4. Demonstrate that the suggested solution is working (i.e. circular economy
business models).
5. Present the research contribution including applicability of the solution.

4 Conclusions
This paper presents initial considerations in understanding ecosystemic business
models in open innovation context. It is argued that today business models are
centred around value-creation of a company in a supply chain, and do now
address collaborative value co-creation in complex societal cases, such as for
example circular economy development. In these cases the value is co-created
and shared among a wide range of stakeholders, going beyond the value chain
and including actors from quadruple helix. To understand value co-creation in
these cases, new ecosystemic business models are needed.

References
Cassell, C. & Johnson, P., (2006), Action research: Explaining the diversity,
Human Relations; Vol. 59, 6; 783-814.
Chesbrough, H., and Rosenbloom, R. S., (2002). The role of the business model
in capturing value from innovation: evidence from Xerox corporation’s
technology spin-off companies. Industrial and Corporate Change 11 (3),
529
Chesbrough, H.W., (2006). Open innovation: The new imperative for creating and
profiting from technology. Harvard Business Press.
Chesbrough, H. (2010) Business Model Innovation: Opportunities and Barriers.
Long Range Planning, Vol. 43, No. 2-3, pp. 354-363.
Curley, Martin & Salmelin, Bror (2013) Open Innovation 2.O: A New Paradigm.
Iivari, M. M., Ahokangas, P., Komi, M., Tihinen, M., Valtanen, K. (2016). Toward
Ecosystemic Business Models in the Context of Industrial Internet. Journal
of Business Models Vol. 4, No. 2, pp. 42- 59
Jaatinen, M., Lavikka, R., (2008), Common Understanding as a Basis for
Coordination, Corporate Communications: An International Journal, Vol.
13 No2., pp.147-167

464
Kasanen, E., & Lukka, K. (1993). The constructive approach in management
accounting research. Journal of management accounting research, (5), pp.
243-264.
Osterwalder, A., Pigneur, Y., and Tucci, C.L. (2005) Clarifying business models:
origins, present, and future of the concept. Communications of the
Association for Information Systems, Vol. 16, No. 1, pp. 1–25.
Ojasalo, K., & Ojasalo, J. (2015). Adapting business model thinking to service
logic: an empirical study on developing a service design tool. THE NORDIC
SCHOOL, 309.
Oyegoke, A., 2011. The constructive research approach in project management
research. International Journal of Managing Projects in Business, 4(4),
pp.573-595.
Schuurman, D., De Marez, L., & Ballon, P. (2015). Living Labs: a systematic
literature review. Open Living Lab Days 2015, Proceedings. Presented at
the Open Living Lab Days 2015.
Valkokari, Katri (2015). Business, Innovation, and Knowledge Ecosystems: How
They Differ and How to Survive and Thrive within Them. Technology
Innovation Management Review, August 2015 (Volume 5, Issue 8, pp.17-
24).
Vargo, S.L. and Lusch, R.F. (2008). Service-dominant logic: continuing the
evolution. Journal of the Academy of Marketing Science, 36, 1-10.
Zolnowski, A., Weiß, C., & Bohmann, T. (2014, January). Representing Service
Business Models with the Service Business Model Canvas -The Case of a
Mobile Payment Service in the Retail Industry. In system sciences
(HICSS), 2014 47th Hawaii International Conference on (pp. 718 -727).
IEEE.
Yin, R., (1994), Case study research: Design and methods (2nd ed.), Sage,
Newbury Park, CA.

465
466
 The Roles, Functioning and Culture of Urban
Innomediaries
Jimmy Paquet-Cormier1

1 Lancaster University, United Kingdom

Category: Doctoral Papers

Disciplines: Design, innovation management, public governance

PhD dates: June 2016 to June 2020
PhD Supervisors: Nick Dunn and Serena Pollastri

Abstract
Urban innomediaries (UI) are orchestrating the collaborative urban innovation
transition. In Europe, they aim 1) to support public organisations in their
environmental and digital transitions, 2) to orchestrate the market of urban
innovation and 3) to foster collaboration between public-private-third-academia-
civil actors. In order to study their systemic functions and their governance and
management practices, seven European organisations were analysed (four main
cases and three partial cases) using a combination qualitative and quantitative
questionnaire, interviews, ethnographic and autoetnographic methods. The
cases were selected for their reputation as a leader in their network and to
maximise the heterogeneity of the cases. Preliminary results propose six
dominant management models: the activist, the agile start-up, the territorial
strategist, the representative and coach, the national model and the urban labs.
In terms of organisational culture and climate, UI are perceived by their
employees as a dynamic and playful working environment where they work hard
on stimulating projects often without receiving a fair compensation for their work.
By reducing its financial dependency on the public sector, the one has
demonstrated that scaling up is not always the most viable option in order to
diversify funding. Moreover, results show that UI have designed and
implemented different types of management practices that combines bottom-up
and top-down dynamics in order to adapt to an ever-changing environment.
Finally, the research recognises the importance for UI to improve their
capabilities regarding organisational learning, impact assessment and
knowledge and competences management.

Keywords: Collaborative Urban Innovation, Urban Innovation, Urban

Innomediaries, City Innovation, Collaborative Innovation, Urban Transitions,
Urban Systems, Innovation Management, Change Management, Flexibility,
Adaptability

467
1 Overview and objectives
In the last decade, a significant number of cities, research centres, public
agencies, start-ups and big companies have become actively engaged in solving
a wide variety of urban problems. Through their interactions, these diverse actors
are co-developing the urban innovation market, which is estimated to grow to
£200 billion in 2030 (DBIS, 2013). In addition, the smart city systems are
estimated to expand to $400 billion by 2020 (TSB, 2013). In transdisciplinary
research, cities are moving to the forefront of the agenda for their capacity to
combine previously disconnected knowledge and use them to study multifaced
phenomenon like collaborative urban innovation.

Through the literature, the concept of urban innovation has different

interpretations (Dente & Coletti, 2011). For the purpose of this thesis, the
definition of urban innovation combines six interpretations and refers to:
• the capacity of a city to produce innovation (Cattacin, 2011). 80% of global
GDP is generated in cities (BIS, 2013);
• an emergent industry of advance technologies aiming to help cities and
their users (citizens, workers, students, tourists);
• a system which can catalyse innovation in a city (Dvir & al., 2006);
• novel ways of governing economic development at the system level;
• novel collaborative approaches for the planning, production and
management of cities (Dente & al., 2005; Nam and Pardo, 2012);
• novel arrangements for urban sustainability transition.

Most recently, innovation intermediary organisations (innomediaries) have been

considered by researcher and practitioners as being strategic actors in the
deployment and structuration of the urban innovation market (Cohen, Almirall, &
Chesbrough, 2017). They are created for their capacity to support open and
distributed innovation processes and for enhancing multi- sector collaborations.
In the last five years, many urban innomediaries37 (UI) were created for systemic
purposes. UI are semi-dependent organisations with the mission to 1- support
public organisations in their socio-ecological and socio-technical transitions
(Baccarne et al., 2016; Mulder, 2012; Hodson & Marvin, 2010; Van Lente et al.,
2012; Kivimaa, 2014); 2- orchestrate the market of urban innovation and its
network of actors (Cohen et al., 2017); and 3- foster the circulation of knowledge
and enhance collaboration between public-private-third- academia-civil actors
(Baccarne, Logghe, Schuurman, Marez, & Shusterman, 2016). For example, in
the UK Future Cities Catapult was established to grow the economy in the field
of urban innovation; in France TUBA was created as an open space for
experimenting in a real world; and in Denmark Blox Hub aims to develop a
community where organisations of all sizes and researcher can co-create better
cities. The same trend is happening in the private sector and in the third sector.

As this phenomenon is happening around the world, there is a significant number

of UI in Europe. Through its urban and research agendas and its multiple

37
The term innomediary is used as an overarching concept to refer to the family of innovative intermediary
organisations. Urban Innomediaries (UI) refers to intermediaries operating in the field/market of urban
innovation.

468
programs (e.g., Horizon 2020, JPI Europe, URBACT III, Interreg Europe, Interact
III and ESPON) the European Commission finance multiple projects to foster
collaborative innovation in cities and between them.

The rise in popularity of UI represent a crucial moment in the co-evolution public-

private- third-academia-civil actors and has led some authors to consider them
as the 4th sector of the economy (Sabeti, 2009; Rice and Mulgan, 2009; Bason,
2018). These intermediaries have also been studied as city labs (Scholl & Kemp,
2016; Scholl et al., 2018), public open innovation intermediaries (Bakici, Almirall,
& Wareham, 2013), innovation platforms (Cadilhon,et al., 2013; Cullen, et al.,
2013; Tui Homann-Kee et al., 2013), innovation labs (Lewis & Moultrie, 2005;
Gryszkiewicz, Lykourentzou, & Toivonen, 2015; Tõnurist et al., 2015; Tõnurist et
al., 2017; McGann, Blomkamp, & Lewis, 2018) and urban living labs (Baccarne
et al., 2016; Kronsell & Mukhtar-Landgren, 2018). While researcher from many
fields are interested by intermediaries and labs, very little attention has been
given to the ones acting as urban innovation engine (Dvir and al., 2006). To the
best of our knowledge no research has studied UI as a family of organisations.
Therefore, the first objective of this thesis is to examine various types of UI
and to create an organisational typology to classify them.

The concept of ‘co-evolution’ (De Vries et al., 2016; Carayannis & Campbell,
2010; Hamalainen, 2015; van Lente et al., 2003) is used as a macro narrative to
analyse the intertwined evolution of public governance, private innovation
management, organisational theories, intermediary organisations, planning
theories, economic geography and social innovation. The concept of co-evolution
is similar to the ‘quintuple helix model’ as proposed by Baccarne et al. (2016).
This framework offers a relatively recent approach to study collaborative
innovation in urban context but is limited by its strong theoretical nature. To fill
this gap, this thesis aims to study organisational innovation dynamics in a specific
context and to develop a better understanding of UI functions at the system level.
In accordance with this approach, the second objective of this research is to
critically assess how different types of UI may support collaborative
innovation processes in the field of urban innovation and to examine the
variety of interdependences between diverse agents of the European
system.

This co-evolution shows the importance of improving collaboration between and

inside organisations in order to solve complex problems (Berschuere et al., 2012;
Cattacin and Zimmer, 2016; Bason, 2018). The notion of collaborative
innovation is used in this thesis as a hypernym to represent the concepts of
network governance, open and distributed innovation, flexible and adaptable
management practices and organisational culture and climate for innovation.
Therefore, the third objective is to develop a multi-level framework in order
to examine complex dynamics which impact collaborative innovation – e.g.,
the power dynamics, knowledge and competences management, risk
management, collaboration, change management, flexibility and autonomy.

In addition, numerous academics have stated the importance for organisations to

develop a culture and climate conductive of creativity and innovation in order to

469
enhance collaborative innovation (Ansell & Torfing, 2014; Ekvall, 1996; Eveleens,
2010; Torfing, 2010). However, creating such environment is a complex task
(King & Anderson, 2002; Bledow et al., 2009). To the best of our knowledge, no
research has examined the level of creativity and innovation in UI. Equally
important, no research have studied the conditions that support or hinder
organisational creativity and innovation in the field of urban innovation. To fil this
gap, the fourth objective of this thesis is to explore the organisational
culture and climate of UI and to reflect about the characteristics of a climate
and culture supportive of collaborative innovation.

2 Research methods
Collaboration and innovation are complex phenomenon which pose many
challenges to researcher. In order to develop a broad understanding of their
dynamics at the network, organisational, team and individual levels, this thesis
combines both inductive and deductive reasoning and uses a mixed method
approach. The cross verification is made through triangulation of qualitative,
quantitative and reflexive methods to facilitates a multi-level analysis and offers
novel ways of explaining the phenomenon.

In accordance with most research on intermediaries and labs, a case study

approach was chosen in order to study this complex multi-dimensional
phenomenon in real-life context (Eisenhardt, 1989; Yin, 2014). Considering the
relative intangibility and emergence of the phenomenon, studying urban
innovation in their real-life environment was essential since it allows to deepen
our understanding about UI organisational dynamics and cultures (Galliers, 1992;
Saunders, Lewis, & Thornhill, 2003). The exploratory research has identified 145
UI around the world. Sixty-two of them are based in a European country. Seven
of them were contacted and agreed to participate in the research. The cases were
selected in order to study the leading organisations and to maximise
heterogeneity. The same instruments and framework of analysis were
implemented across the four main cases and the three partial case studies
(Eisenhardt, 1989; Yin, 2003). To maximize internal validity the research design
is based on a 5 to 8 weeks immersion in UI during which the researcher has
administered two questionnaires; did multiple in-depth, semi-directed interviews
with senior managers; and wrote about his immersion experience. In hence, an
extensive exploratory research and a systematic literature review about
intermediary organisations and innovation labs was carried out.

3 Preliminary results and reflexions

Early results propose that all UI are unique entities with a distinctive culture, size,
portfolio, mission and structure. In terms of innovation culture, UI are driven both
by the need to generate economic development and the aim to foster social
innovation. These two convergent but often conflicting discourses materialise
under different organisational forms and missions. Six types of UI have emerged
from the results: the activist, the agile start-up, the territorial strategist, the
representative and coach, the national model and the urban labs.

470
Organisations form the sample were created between 1994 and 2016. The four
main cases were established outside the government and were created as non-
for-profit associations. They have 7 to 110 employees and their budgets vary from
€1 to €23 million which comes from a combination of public (local, regional, state,
European), private and research funds. They are created and supported by all
levels of government. The main differences from other concepts of innovation
labs and urban labs are that : 1- the production of urban innovation is their priority;
2- the public sector is not their main ‘client’ and source of revenue; 3- most UI
have more employees and a bigger budget; and 4- most UI have more autonomy
from their affiliate government (if they have one). Interestingly, the results show
that this autonomy does not increase their control over their mission since
organisations like city labs already have extensive autonomy from their parent
government.

There is a consensus amongst the interviewees that UI are in constant change.

During the research, two cases went through a merger with a similar organisation
and one case had four restructuration processes. As most government request
from UI to diversify their sources of funding, senior managers have acknowledged
that there is a tension between growing their revenue and helping other
organisations. In order to adapt to an ever-evolving environment senior managers
in UI have implemented diverse forms of bottom-up and top-down mechanisms
to re-designing their organisation. Senior managers acknowledge that scaling up
comes with many difficulties and have different point of view concerning the need
to grow in order to be successful. For example, the smallest organisation of the
sample had the most success in reducing its dependency on public money (from
100% in 2014 to 10% in 2018).

Results about organisational culture and climate reveals that UI are lively places
to work at but that the work is not well payed. For example, their employees
strongly agree that they ‘work hard’ with great ‘freedom’ from their superior and
that the organisational climate is ‘playful’ and ‘lively’. On the other hand, they
strongly disagree when asked if they receive ‘good financial rewards’ and ‘fair
compensation’ for their work or if they have access to ‘financial benefits’ and
‘training opportunities’.

Overall, UI are interested in improving their performances and the findings

indicate that they should focus on improving their organisational learning, their
impact assessment and their knowledge and skills management.

471
References
Ansell, C., & Torfing, J. (2014). Collaborating on design - designing collaboration.
Public Innovation Through Collaboration and Design, c, 229–239.
https://doi.org/10.4324/9780203795958
Baccarne, B., Logghe, S., Schuurman, D., Marez, L. De, & Shusterman, N.
(2016). Governing Quintuple Helix
Innovation : Urban Living Labs and. Technology Innovation Management Review,
6(3), 22–30.
Bakici, T., Almirall, E., & Wareham, J. (2013). The role of public open innovation
intermediaries in local government and the public sector. Technology
Analysis and Strategic Management, 25(3), 311–327.
https://doi.org/10.1080/09537325.2013.764983
Bledow, R., Frese, M., Anderson, N., Erez, M., & Farr, J. (2009). Extending and
Refining the Dialectic Perspective on Innovation: There Is Nothing as
Practical as a Good Theory; Nothing as Theoretical as a Good Practice.
Industrial and Organizational Psychology, 2(03), 363–373.
https://doi.org/10.1111/j.1754- 9434.2009.01161.x
Cadilhon, J., Birachi, E., Klerkx, L., & Schut, M. (2013). Innovation platforms to
shape national policy. Innovation Platforms Practice Brief 2., (November),
1–4.
Carayannis, E. G., & Campbell, D. F. J. (2010). Triple Helix, Quadruple Helix and
Quintuple Helix and How Do Knowledge, Innovation and the Environment
Relate To Each Other? International Journal of Social Ecology and
Sustainable Development, 1(1), 41–69.
https://doi.org/10.4018/jsesd.2010010105
Cohen, B., Almirall, E., & Chesbrough, H. (2017). The City as a Lab: Open
InnOvatIOn Meets the COllabOratIve eCOnOMy. California Management
Review, 59(1), 5–13. https://doi.org/10.1177/0008125616683951
Cullen, B., Tucker, J., & Homann-Kee Tui, S. (2013). Power dynamics and
representation in innovation platforms. Innovation Platforms Practice Brief,
(4), 1–4.
Department of Business, Innovation and Skills (2013) The Smart Cities Market:
Opportunities for the UK Technology Strategy Board (2013) Solutions for
Cities: An analysis of the feasibility studies for the Future Cities
Demonstrator Programme
De Vries, H., Bekkers, V., & Tummers, L. (2016). Innovation in the public sector:
A systematic review and future research agenda. Public Administration,
94(1), 146–166. https://doi.org/10.1111/padm.12209
Dvir, R., Schwartzberg, Y., Avni, H., Webb, C., & Lettice, F. (2006). The future
center as an urban innovation engine. Journal of Knowledge Management,
10(5), 110–123. https://doi.org/10.1108/13673270610691224

472
Eisenhardt, K. M. (1989). Building Theories from Case Study Research. The
Academy of Management Review, 14(4), 532.
https://doi.org/10.2307/258557
Ekvall, G. (1996). Organizational climate for creativity and innovation. European
Journal of Work and Organizational Psychology, 5(1), 105–123.
https://doi.org/10.1080/13594329608414845
Eveleens, C. (2010). Innovation management; a literature review of innovation
process models and their implications. Science, (April), 1–16.
https://doi.org/10.1016/j.euroecorev.2008.12.001
Gryszkiewicz, L., Lykourentzou, I., & Toivonen, T. (2015). Innovation Labs:
Leveraging Openness for Radical Innovation? Ssrn, 4, 68–97.
https://doi.org/10.2139/ssrn.2556692
Hamalainen, T. J. (2015). Governance Solutions for Wicked Problems:
Metropolitan Innovation Ecosystems as Frontrunners to Sustainable Well-
Being. Technology Innovation Management Review, 5(10), 31–41.
Hodson, M., & Marvin, S. (2010). Can cities shape socio-technical transitions and
how would we know if they were? Research Policy, 39(4), 477–485.
https://doi.org/10.1016/j.respol.2010.01.020
Kivimaa, P. (2014). Government-affiliated intermediary organisations as actors in
system-level transitions.
Research Policy, 43(8), 1370–1380. https://doi.org/10.1016/j.respol.2014.02.007
Kronsell, A., & Mukhtar-Landgren, D. (2018). Experimental governance: the role
of municipalities in urban living labs. European Planning Studies, 26(5),
988–1007. https://doi.org/10.1080/09654313.2018.1435631
Lewis, M., & Moultrie, J. (2005). The Organizational Innovation Laboratory.
Creativity and Innovation Management, 14(1), 73–83.
https://doi.org/10.1111/j.1467-8691.2005.00327.x
McGann, M., Blomkamp, E., & Lewis, J. M. (2018). The rise of public sector
innovation labs: experiments in design thinking for policy. Policy Sciences,
51(3), 249–267. https://doi.org/10.1007/s11077-018-9315-7
Merindo, V., Aubouin, David W. Versailles, N., & Capdevila, Alexandra
Lechaffotec, I. (n.d.). le rôle des plateformes d’innovation dans les
écosystèmes régionaux.
The Work Foundation (2014) ‘Analysis of UK Future Cities Industries’, prepared
for the Future Cities Catapult.
Sourced from: ONS International Trade in Services, 2012.
Tõnurist, P., Kattel, R., & Lember, V. (2015). Discovering Innovation Labs in the
Public Sector. Working Papers in Technology Governance and Economic
Dynamics, (61), 1–36. Retrieved from http://hum.ttu.ee/wp/paper61.pdf
Tõnurist, P., Kattel, R., & Lember, V. (2017). Innovation labs in the public sector:
what they are and what they do? Public Management Review, 19(10),
1455–1479. https://doi.org/10.1080/14719037.2017.1287939

473
Torfing, J. (2010). Collaborative innovation in the public sector. Handbook of
Innovation in Public Services, 11(1), 301–316.
https://doi.org/10.4337/9781849809757.00032
Tui Homann-Kee, S., Adekunle, A., Lundy, M., Tucker, J., Birachi, E., Schut, M.,
… Mundy, P. (2013). What are innovation platforms ? Innovation Platforms
Practice Brief, 1(November), 1–6. Retrieved from
http://r4d.dfid.gov.uk/pdf/outputs/WaterfoodCP/Brief1.pdf
Van Lente, H., Hekkert, M., Smits, R., & van Waveren, B. (2003). Roles of
Systemic Intermediaries in Transition Processes. International Journal of
Innovation Management, 07(03), 247–279.
https://doi.org/10.1142/S1363919603000817
Van Lente, H., Hekkert, M., Smits, R., & Van Waveren, B. (2012). Systemic
intermediaries and transition processes. Shaping Urban Infrastructures:
Intermediaries and the Governance of Socio-Technical Networks, 36–52.
https://doi.org/10.4324/9781849775380

474
 Urban Living Labs as a smart city approach: how
does socio-technical innovation transform urban
development?
Hui Lyu1

1 SRF, TU Wien, Austria

Category: Doctoral Papers

Disciplines : urban planning, urban development

PhD starting date: March 2014
PhD Supervisor : Prof. Rudolf Giffinger

Abstract
Under the demand of urban sustainable development, the smart city movement
has been on stage for more than a decade, with its concept changing and
evolving during the time, from a technology- centred model to a more balanced
social and technological strategy. Meanwhile, Urban Living Labs (ULLs) came
up in recent years as an approach that uses emerging technologies to cope with
urban challenges. Nowadays, ULLs often have a focus on citizen participation
and social value creation. The linkage between these two concepts are noticed
but not clearly elaborated. This paper argued that ULLs could contribute to the
smart city strategy, but there is a lack of investigation on how ULLs’ approach is
linked to the socio-technical innovation process in the smart cities. Aiming to
explore the nature linkage between these two concepts, this paper tries to raise
an analytical model based on literature review and Delphi method survey data
from ULLs experts. It is expected that key indicators could be identified to
evaluate the socio-technical innovation approach of ULLs, as well as the smart
city transition process driven by ULLs.

Keywords: smart city, urban living lab, socio-technical innovation, urban

transition

475
1 Background and motivation
Sustainable development has been a global common view for decades. The
HABITAT III conference adopted the New Urban Agenda, calls for further effects
on urban sustainable development and provides a roadmap for all stakeholders
to act and promote urban transition (United Nations, 2017). The smart city
movement has been on stage for more than a decade, with its concept changing
and evolving during the time, from a technology-centred model to a more
balanced social and technological strategy. As defined by Schaffers et al., “the
smart city is about how people are empowered, through using technology, for
contributing to urban change and realizing their ambitions. The smart city
provides the conditions and resources for change. In this sense, the smart city is
an urban laboratory, an urban innovation ecosystem, a living lab, an agent of
change” (Schaffers, Komninos, & Pallot, 2012).

Similar to this understanding of the smart city, Urban Living Labs (ULLs) first
came up as an approach that uses emerging technologies to cope with urban
challenges. Today, ULLs often have a focus on citizen participation and social
value creation (Nesti, 2017).

As the understanding of smart cities has been evolving during the last decade,
more and more cities are taking a smart city strategy for their urban development.
But there is a gap between the plan of a smart city and the practice of it. Tools
are needed in carrying out smart city practices. During the same period, the new
understanding of smart city seems to have a lot in common with the emerging
ULLS, which seem to an effective tool in the implantation of smart city strategy.
Hence, since smart city and ULL share the same idea of citizen-centred
innovation, it would be necessary to study how ULLs work in smart city
development, and how the innovation process happens in the ULLs.

2 State of the art and general research topic

There are different models and understandings for smart cities. The first
generation of Smart City has come out at the beginning of 2000s, with a rather
high-tech oriented approach (Caragliu et al., 2011). Later, a more citizen-centric
approach of smart city was discussed to address urban challenges (Dameri,
2013). Now the discussion of smart cities is coming to another new stage, taking
the open innovation process as a key approach of smart city transition (Giffinger,
2019).

To date, the understanding of smart city and ULL are both under discussion.
Compare to traditional Living Labs, ULLs are thought to have some more extra
key components such as financing, physical places, and innovation (Chroneer,
et al., 2019). These characteristics mean that ULLs are more connected to real
urban projects, and ULLs could support smart city transition from an
implementation perspective. Smart cities and ULLs share some same
characters. These two concepts have natural links in certain aspects and ULLs
could be an effective tool for certain aspects of smart city development.

476
Socio-technical innovation is regarded as influencing the process of urban
transition (Atkinson, 1998). This innovation process could be a key to understand
the inner linkage between ULLs and smart cities. Although ULLs are promoting
the open innovation which has the power to unlock the potential for smart city
development and there are a lot of ULLs considered to have smart city
characteristics, however, there is a lack of investigation on how ULLs’ approach
is linked to the socio-technical innovation process in smart cities. This research
aims to explore the link between the two concepts, with the working title of this
research which is:

ULLs as a smart city approach - how is socio-technical innovation transforming

urban development

3 Research questions and hypothesis

The smart city concept has developed through different stages during the last
years, from a rather technological-deterministic to a more citizen-centric
approach, and then to an open innovation focusing on smart citizens rather than
on the Smart City as a high-tech solution to urban challenges (Baccarne, et al.,
2014). In this context, the smart city as a socio-technical innovation process is
changing from triple helix (University-Industry-Government) to quadruple helix
(University-Industry- Government-Citizen) model, or even to a multi-helix model
(Caragliu, et al., 2011). It is argued in this study that ULLs, as an effective tool
for smart city transition, are supporting smart cities through socio-technical
innovation approach.

From the point of view of multi-level perspective theory on socio-technical

transition, urban transition is a process that comes about through interaction
processes within and between three analytical levels: niches (micro level),
regimes (meso level), and a socio-technical landscape (macro level) (Geels,
2007). The ULLs are considered in the smart city transition process as a
container for niche innovation and a driver for evolving form niches to regimes.

This research is concerned with exploring the relationship between and smart
city development. It is argued that ULLs could support smart city through the
socio-technical innovation process in it. With the empirical analysis on ULLs, the
urban transition processes of the smart city can be more visible and clearer, and
therefore reveal the socio-technical innovative interaction in it. Based on the
preliminary literature review, the core research question is summarized as:

- How do ULLs contribute to smart city transition through socio-technical

innovation?

To answer the core research question, the following related questions should
also be investigated in the chapters:

- What should socio-technical transition be understood in the urban

development of a smart city?

477
 - How do ULLs conduct socio-technical innovation and what could be the
key indicators to measure their influence in the process?

3 Research methods
This research is using qualitative data analysis as basic research methods.
Based on the theory review of transition studies and innovation studies, it is
assumed that urban open innovation, which connects traditional technological
innovation and social innovation, is an effective approach for the urban transition
of a smart city. Therefore, an analytical model is constructed based on the
quadruple helix (government, industry, academia, and citizens), as well as an
analytical framework for the analysis of the socio-technical transition process of
ULLs.

In the empirical research design, the European Network of Living Labs (ENoLLs)
is chosen as the study database. To date, there are more than 60 living lab
members in the network that have been labelled as “Smart Cities and Regions”
(by April of 2019). A structured profile analysis will be conducted on these cases
to identify the ULLs in the database and to evaluate the characteristics of the
socio- technical innovation process in these ULLs.

Then a survey based on Delphi method would be drawn to collect data to find
the key indicators in the supposed socio-technical innovation process (I would
like to take the chance to do a first round of Delphi method survey during the
Open Living Lab Days 2019 event, if it is possible). Indicators summarised from
the survey will be used to evaluate the above mentioned ULLs, based on which
a single case will be chosen as a deep empirical research object.

The chosen case will be analysed based on observation and individual interviews
with stakeholders involved in the ULLs innovation process. The purpose of the
case study is to elaborate the interaction between stakeholders and the
innovation process in ULLs, and (hopefully) the socio-technical transition of
smart city.

The research method is illustrated below in Figure 1:

478
 demand of urban
Smart emerging of
sustainable
City
development

transition
studies a analysis model on
model ULLs in smart city
construction studies

profile analysis
ENoLL case
selection Delphimethod survey

Empirical research

Case study data collection

data analysis

and conclusion transition process of ULLs

Figure 1: Flow chart of the research methods

5 Expected outcomes
In the literature review part, it is revealed that current understandings of smart
city and ULL share several characteristics such as project-based, citizen-centred
approach, socio-technical innovation, and so on. The study is expected to
contribute to the understanding of the smart city and ULL, and as well as the
socio-technical transition theory of urban development. An analysis framework
is expected to be constructed for the analysis of the characteristics of ULLs.

The expected outcomes of empirical research include:

1. The research is expected to find the evidence that ULLs could support
smart city development from the local level, through the interactive
innovation between different stakeholders in the ULLs approach.

2. Key indicators could be identified to evaluate the socio-technical

innovation approach of ULLs, as well as the smart city transition
process driven by ULLs.

479
 3. ULLs could be understood as an effective instrument to influence the
urban transition process in smart cities. So that the results of the
research could be useful for both urban planning authorities, who
would like to invoke urban innovation, and urban living lab organizers,
who want to amplify their good practice from local level labs.

It is also expected that the result of the research could support further study on
open innovation and urban transition management.

References
Atkinson, R. (1998). Technological change and cities. Cityscape, 3(3), 129–170.
Buitendag, Baccarne B., Schuurman, D., Mechant, P., & De Marez, L. (2014). The
role of Urban Living Labs in a Smart City, (June), 1–14.
Blezer, S. (2018). The living lab concept: a new tool for urban planning and -
design.
Caragliu, A., del Bo, C., & Nijkamp, P. (2011). Smart cities in Europe. Journal of
Urban Technology, 18(2), 65–82.
Chronéer, D., Ståhlbröst, A., & Habibipour, A. (2019). Urban Living Labs: Towards
an Integrated Understanding of their Key Components. Technology
Innovation Management Review, 9(3), 50–62.
Dameri, R. P. (2013). Searching for Smart City definition: a comprehensive
proposal. International Journal of Computers & Technology, 11(5), 2544–
2551.
Geels, F. and Schot, J. (2007). Typology of sociotechnical transition pathways,
Research Policy, 3, 36, 399-417
Giffinger, R. (2019). Open Innovation in Smart City energy transition: Living Labs
as potential enabler. Presentation at 8th International Conference on
Smart Cities and Green ICT Systems. Heraklion, Greece.
Nesti, G. (2017). Co-production for innovation: the urban living lab experience.
Policy and Society, 4035, 1–16.
Schaffers, H., Komninos, N., & Pallot, M. (2012), et all. Smart cities as innovation
ecosystems sustained by the future internet. FIREBALL White Paper, EU,
(April), 1–65.
United Nations. (2017). New Urban Agenda A/RES/71/256, (April), 29.

480
481
The European Network of Living Labs (ENoLL) is the international federation
of benchmarked Living Labs in Europe and worldwide. Founded in November
2006 under the auspices of the Finnish European Presidency, the network has
grown in ‘waves’ up to this day. ENoLL counts today over 150+ active Living Labs
members worldwide. Directly, as well as through its active members, ENoLL
provides co-creation, user engagement, test and experimentation facilities
targeting innovation in many different domains such as IoT, media, energy,
mobility, healthcare, agrifood, societal transformation, etc.

in collaboration with

The Thessaloniki Active & Healthy Ageing Living Lab (Thess-AHALL) operational
since 2014, and a unique setting in the city of Thessaloniki, Central Macedonia
region (Northern Greece). The lab fosters initiatives encouraging regional
development and healthcare systems sustainability by the provision of novel
technologies and innovation. The lab is actively engaged with the end-users and
relevant community stakeholders, actively pursuing co-creation and co-design of
technological solutions to improve health and social conditions and facilitate
independent living. Thess-AHALL aims to develop a business plan for self-
sustainability, in line with the ENoLL’s strategy, facilitating the people demand
with the active involvement of SMEs within pilots.

The Thess-AHALL is governed by the Laboratory of Medical Physics, School of

Medicine, Aristotle University of Thessaloniki which has essentially spinned-off
two commercial entities: the self-funded (University based) initiative LLM Care
and Neuroanadrasi which was founded by members of the Lab based on
exploiting the neuroscientific capacity of the Lab.

482