Project Deliverable

Project Number: Project Acronym: Project Title:

258348

MyFIRE

Multidisciplinary networking of research communities in FIRE

Thematic Priority

Instrument:

Coordination and support action

Future Internet Research and Experiment (FIRE)

Title

D3.3d Routes to sustainability in FIRE some MyFIRE conclusions

Contractual Delivery Date:

Actual Delivery Date:

NA

18th May 2012

Start date of project:

Duration:

June, 1st 2010

25 Months

Organization name of lead contractor for this deliverable:

Document version:

UEdin

V1.1

Dissemination level ( Project co-funded by the European Commission within the Seventh Framework Programme)

PU PP RE CO

Public Restricted to other programme participants (including the Commission Restricted to a group defined by the consortium (including the Commission) Confidential, only for members of the consortium (including the Commission)

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

Authors (organizations) :

Philip Inglesant (UEdin)

Abstract :

This report is an extra deliverable following on from D3.3 to draw further conclusions about possible paths to sustainability of FIRE, taking an holistic view of FIRE and Future Internet research as a whole. We consider a number of options including public funding, commercial, and mixed models, and the importance of a demand-driven, high-level federation for the FIRE facilities. The report includes a case study of the BonFIRE Cloud computing experimental facility and concludes with a comparison of the FIRE Open Calls process with Open Innovation.

Keywords :

Disclaimer
THIS DOCUMENT IS PROVIDED "AS IS" WITH NO WARRANTIES WHATSOEVER, INCLUDING ANY WARRANTY OF MERCHANTABILITY, NONINFRINGEMENT, FITNESS FOR ANY PARTICULAR PURPOSE, OR ANY WARRANTY OTHERWISE ARISING OUT OF ANY PROPOSAL, SPECIFICATION OR SAMPLE. Any liability, including liability for infringement of any proprietary rights, relating to use of information in this document is disclaimed. No license, express or implied, by estoppels or otherwise, to any intellectual property rights are granted herein. The members of the Multidisciplinary networking of research communities in FIRE (MyFIRE) do not accept any liability for actions or omissions of MyFIRE members or third parties and disclaims any obligation to enforce the use of this document. This document is subject to change without notice.

MyFIRE Page 2/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

Revision History
The following table describes the main changes done in the document since it was created.
Revision Date Description Author (Organisation)

V1.0 V1.1

18 June 2012 18 June 2012

th

th

Creation Formatting revisions

Philip Inglesant (UEdin) Philip Inglesant (UEdin)

MyFIRE Page 3/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

Table of Content

MyFIRE Page 4/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

1. Executive Summary
This additional deliverable adds to the findings of D3.3 Exploitation Strategies for Testbeds by looking more closely at paths to sustainability for FIRE Experimental Facilities. Sustainability is essential not only to maximise the value return for public funding, but also to ensure that facilities are available for long enough to enable researchers to plan and perform experiments, and to allow reliability and repeatability. Currently, potential users are deterred from using testbeds if they cannot be sure of this long-term availability. This does not mean that all facilities must be available indefinitely; termination, where appropriate, is a part of sustainability. However, the current FP7 funding model does not always provide for FIRE facilities to be available for long enough to attract sufficient user interest. Conversely, attracting many more users is clearly a potential route to longer-term sustainability and to increasing the research value of testbeds. The European Commission and the FIRE Architecture Board are well aware of the need for sustainability. Federation of the existing and perhaps new FIRE experimental facilities is seen as a key part of the route to sustainability; the large FIRE facilities projects are already federations of testbeds, but this wider federation is expected to span across FIRE, building on these existing federations. Provision is being made to fund moves towards this wider high-level federation in the current 2011-12 Information and Communication Technologies FP7 Work Programme. Federation is planned to include a joint FIRE portal and common tools providing features including brokering, user access management, a one-stop-shop, measurement, and performance analysis. In this way, taking a holistic view of FIRE as a whole, the sustainability of the FIRE features, and reliable long-term experimental platforms, is more important than maintaining the specific testbeds as they currently exist. Indeed, we suggest looking wider than FIRE, such as encouraging use of FIRE by Future Internet PPP and federation including NRENS and international networks, and programmes such as the US NSF-finded GENI1, as is already occurring in various ways. Sustainability could be on the basis of a public funding model; although the EU foresees a significant reduction in its funding, other forms of national, regional, or international funding can play a part. Alternatively, commercial models are possible. Public and commercial models are not exclusive, and we explore one model from Canada in which public funding was used to provide research facilities for SMEs. We support our conclusions for FIRE sustainability with a case study of the BonFIRE experimental facility in Cloud computing. BonFIRE provides easy-to-use tools for virtual machine management, modelling, lifecycle management, and analytics. For its users, BonFIRE provides the facility for experimental research in emerging issues in Internet-as-a-Service and Internet of Services paradigms. BonFIRE has produced thorough documentation and provides experience-based know-how and support, working with its users, thereby reducing their overall costs. BonFIRE provides heterogeneity and scale beyond that available on current research testbeds, and provides tools for experimental verifiability and reproducibility. BonFIRE developed its offering through three driving experiments which were designed to investigate research questions of real concern to the Cloud community, and through Open Calls, which provided funding to support a limited number of experiments (four in the first Open Call) designed to make user of various aspects of the BonFIRE facilities. This approach is leading to a sound offering, meeting the needs of users, and hence forming the basis for longer-term sustainability within the FIRE federated facilities. Finally, we show how the FIRE model of using Open Calls to grow the user community and to strengthen the testbeds is related to Open Innovation, in which firms draw on external ideas rather than developing new ideas internally. Although FIRE is not a commercial firm, it is going beyond the earlier forms of User Innovation to capture the value created by Open Call experiments and thereby meeting more closely the needs of its research user base.

http://www.geni.net/

MyFIRE Page 5/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

Table of Figures
Figure 1: Timescale of federated FIRE facilities. ............................................................................................ 10 Figure 2: Geographically distributed testbeds and testing scenarios. .......................................................... 13 Figure 3: BonFIRE Project Timeline: How to get involved. ............................................................................ 14

MyFIRE Page 6/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

2. Introduction 2
It is clear that to provide a good service for experimenters, and to provide the best value in return for public funding, the FIRE programme must be able to sustain the availability of experimental facilities throughout the lifecycle; the FP7 ICT work programme for 2011-12 foresees implementing a demanddriven high level federation framework for all FIRE prototype facilities and beyond making the facility selfsustainable towards 2015 based on credible business models assuming a significant decrease of EU funding. [12]. Routes to FIRE sustainability have been considered by a Working Paper discussed by the FIRE Architecture Board. Report II of the FIREStation FIRE Roadmap (report I is [18, 19]) is planned to discuss sustainability alongside legal and business issues.

2.1. The importance of sustainability

Sustainability is essential not only to ensure a good return on the EU's investment in FIRE, but also to ensure the repeatability of experiments, and to enable the development and maintain reproducibility of experiments over the long term. Uncertainty around the sustainability of testbeds enormously decreases the willingness of potential users/experimenters to use such facilities, especially outside the consortium around each facility [20]. This does not mean that every facility must continue indefinitely, and in particular, it does not imply an unviable indefinite reliance on more pubic funding. A more holistic approach is needed, looking at FIRE as a whole, and at the sustainability of each set of features, rather than the specific testbeds as they currently exist. Indeed, we suggest looking wider than FIRE, such as liaison with Future Internet PPP, NRENs and GANT, US NSF-funded GENI, and within the Future Internet Assembly; a useful survey of Future Internet-related initiatives is available on the FIRE web site [17].

The authors would like to acknowledge the help of Kostas Kavoussanakis and Vegard Engen of the BonFIRE project in preparing this deliverable.

MyFIRE Page 7/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

3. Challenges towards sustainability

The current funding arrangements certainly do not prevent the disappearance of key parts of the FIRE infrastructure, and it is recognised, by the FIRE Architecture Board and others, that the existing FP7 funding model does not ensure the sustainability of FIRE facilities for long enough to attract sufficient interest in users and research projects and to establish a large enough user base. Faced with these challenges, it is possible to identify two general routes towards sustainability: what might be called the public-funding and the commercial routes. Unfortunately, there is a tension between these two approaches; focussing on one approach may tend to weaken the other, threatening the overall sustainability of the projects.

3.1. Public funding

In this approach, the experimental facility has as its main objective to allow researchers to conduct advanced experimentation, where the users are mainly research labs (in academia and industry) and universities, with the funding coming mainly from public authorities. Within this we can include European Commission research funding but also national and regional funding.

3.1.1. Open Call funding for experimental research

A very significant and innovative approach, where funding for the use of experimental facilities does not flow from users in a commercial model, but rather, funding is included in the budget allocated to the service provider to support experimenters, has been pioneered by the FIRE second wave facilities projects. A proportion (20%) of the EU funding for each of these projects was set aside for the purpose of these Open Calls, and additional funding to extension of the facilities. We discuss Open Calls as used by FIRE, and also now in other initiatives, in section 5 as a form of Open Innovation.

3.2. Commercial and other sources of income

In this approach, sustainability is achieved through a commercial business model, where users pay to use the facility. Although this is a traditional business model, there are of course many varieties of ways in which is may be made viable. The facility may even become privately owned and operated, or operate as a part-private consortium. Support for SMEs (perhaps as part of a funded service/experiment) could be a key part of this strategy. Whether or not this approach is taken as a whole, FIRE encourages collaboration between research and industry [15]. BonFIRE, along with PanLab and others, has considered this as one possible approach.

3.3. Public funding of commercial use: a model from Canada

Conversely, another possible model is for public funding to enable testbeds to be made available to small and medium enterprises, which would find it challenging or unaffordable to build their own costly R&D environments. This is the approach taken by Canadas innovative DAIR/ATIR Digital Accelerator for Innovation and Research [7], a Cloud-based project of CANARIE, Canadas Advanced Research and Innovation Network3. The objective in the initial phase was to reduce significantly the time required to design, prototype, validate and demonstrate new products and ideas, particularly at large scale. It also reduced capital, space, power and people resources that each company would otherwise be required to provide over the course of the product development cycle [8]: to let technology entrepreneurs do what they do best without having to build a dedicated R&D environment [7]. A $CAN 3 million pilot project from spring 2011 was fully subscribed; during this pilot phase, there were no charges for the use of DAIR. It is expected that DAIR will
3

http://canarie.ca/

MyFIRE Page 8/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

progress to large-scale deployment during 2012 and beyond; in this stage, there may be some cost recovery from commercial enterprises, depending on demand [8]. Thus, the model is of public funding for research resources, but not necessarily for these resources to always be provided free of charge.

3.4. State aid for research to development and innovation

The EU is aware of the risk that state aid for Research, Development, and Innovation can distort competition, crowding out private investment, supporting inefficient production, or strengthening incumbent market power [11]. These negative effects likely to be stronger in activities that are closer to commercialisation of the product or the service. Recognising this, EU rules state that aid intensity should be lower for activities linked to development and innovation than for more fundamental research-related activities. At the same time, the EU is aware of the need for state aid in appropriate cases to address market failure, a traditional justification for public intervention. Moreover, there has been a move away from assessment based on the linear distinction of the different stages of research activities [14]. Thus, there is no clear separation between the public funding and commercial approaches, or between close to market and basic research; public funding can be provided for commercial research and development, while academic facilities can be used by commercial companies, where appropriate.

3.5. Opening up FIRE testbeds

A related practice, sometimes more informal and perhaps on a smaller scale, involves the use by researchers commercial, research-based, or academic of funded facilities on as as-is, best-effort basis. This practice can widen the user base of testbeds at little cost to the testbed operators. OFELIA, for example, is currently available and is being used on such a basis, in addition to the Open Call experiments. More generally, FIRE aims ultimately to become self-sustainable towards 2015, based on credible business models assuming a significant decrease of EU funding [13], by virtue of demand-led, high-level federation, as we discuss in section 4.1. , maintaining openness towards additional testbeds and facilities, building on proven existing federation models, supporting the use of open standards, and linking with existing research infrastructures such as Gant and the NRENS, and for co-operation around the EU, nationally, and internationally. In these ways, the public funding and commercial models are in some ways overlapping. Indeed, if the tension between the two approaches can be overcome then possibly a facility could establish sufficient sustainability.

MyFIRE Page 9/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

4. Long-term sustainability of infrastructure

4.1. Integrated FIRE federation
Alongside the public funded and commercial approaches, and overarching both of them, FIRE has had the aim to strengthen sustainability by combining resources in an efficient, more soundly-based way as a common set of federated testbeds since at least the 2009 report of the Working Group on modular federation of FIRE Facilities [10]. From the socio-economic viewpoint, federation is attractive, and may present legal and business advantages in terms of management and enabling potential users to enter into one single agreement. From a technical point of view, the vision of a user accessing resources as needed, or accessing through a single centralising portal [10], and easily running experiments on heterogeneous environments, is challenging and presents many complications. Amongst of the issues, this implies the need for FIRE will need to put effort into standardised interfaces and interoperability; the current EU ICT FP7 Work Programme includes development of this portal, alongside a set of common tools addressing issues such as brokering, user access management, measurement and performance analysis [12]. However, federation can take many different forms: it is in essence a generic term for a higher level of interconnectivity, interoperability and interworking between testbeds. The federation Working Group [10] identified horizontal federation giving greater scale across a narrow range of layers of the telecom stack and vertical federation enabling system-level testing across layers, or in a layer-less model. Federation could also be considered to extend to include the sharing or exchange of test-bed related tools, as a form of software federation, or even human federation through the exchange of students and researchers. A common portal, or one-stop shop, is considered to be a basic FIRE requirement by the federation Working Group. Federation should extend outside FIRE, and is extending, with more or less established links between FIRE facilities and NRENS, Gant, and internationally, for example with the US GENI facilities. Nevertheless, fundamentally a federation represents a common set of objectives; without this, a federation is meaningless [10].

4.1.1. Generic interdisciplinary platforms

In parallel, and not necessarily within the FIRE programme, there have been suggestions about providing generic platforms, providing the basic elements of a Future Internet infrastructure: network-, storage-, computing-, service-, or content-related, along with middle ware, tools, and core services. The federation Working Group [10] suggested that such generic projects might appear in EC ICT programmes as early as 2010 or 2011; to the best of our knowledge, however, this is currently manifested mostly through the FIPPP programme (see, for example, Technology foundation: Future Internet Core Platform Objective FI.ICT2011.1.7 [13]), rather than in specifically experimental, basic research.

Figure 1: Timescale of federated FIRE facilities. Source [19]:

MyFIRE Page 10/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

4.2. Multiple funding streams

In practice, the FIRE facilities - but not necessarily the experiments using them - operate as part of longstanding research centres in universities, research arms of large firms, and specialised companies. These centres do not in general rely on a single funding stream, but on a combination of European, national, university-internal and, to some extent, commercial sources. Although this does not provide complete assurance of the continued availability of FIRE infrastructure and should certainly not be relied upon to do so, it does provide an additional level of safety so that, at least, it can be hoped that the hardware and, most importantly, the human knowledge built up during FIRE and before is not dissipated. The FIRE Architecture board has considered that FP7 and national funding sources, currently disparate and sometimes contradictory, could be aligned in some way.

4.1.

Sustainability of other FIRE facilities

It can be noted in Figure 1 that the former FIRE facilities did not cease (although there may have been funding gaps in some cases), but were in various ways largely incorporated, along with other facilities and technologies such as Grid, into the growing FIRE federation. An indication of the realistic possibilities of bringing together existing testbeds to further strengthen them and to increase sustainability is provided by the OpenLab project4, which "brings together the essential ingredients for an open, general purpose and sustainable large scale shared experimental facility, ....". OpenLab builds on the first wave FP7 ICT Call 2 FIRE projects Onelab 2 and Panlab II, both in themselves continuations of earlier projects, and, with FEDERICA, extended beyond their original EU-funded contractual end date [20]. In this way, the sustainability of projects as they become parts of new projects is illustrated. Another good example is provided by the experiment ExSec - Experimenting Scalability of Continuous Security Monitoring, running as part of the first BonFIRE Open Call. This experiment is in some ways an extension of work done in the FP6 GridTrust project, which developed a framework to perform continuous security monitoring on Grid technologies, and of the FP7 project RESERVOIR5, which adapted a portion of this framework for the policy-based access control to Cloud technologies. But indicating the ways in which FIRE facilities enable increased scale and diversity - The ExSec experiment aims to perform a much more rigorous scalability test, including different types of hypervisors and different types of Cloud environment managers, for applications requiring continuous security monitoring.

4.2. Towards Horizon 2020

Although FP7 ends in 2013, it is already clear that Future Internet Experimental platforms will continue into the forthcoming Horizon 2020 programme. Indeed, the FIRE federation has been scheduled to run to at least late 2015, which continues into the Horizon 2020 duration. A workshop to be held in November 2012 will showcase results from engineering aspects of Future Internet best practices and experimental approaches to tackling the societal Grand Challenges of Horizon 2020, and the FP7 OSIRIS project6 (Towards an Open and Sustainable ICT Research Infrastructure Strategy) is organizing a high level strategic event on "The role of ICT research infrastructures in Horizon 2020".

4 5

http://www.ict-openlab.eu/ http://www.reservoir-fp7.eu/ 6 http://www.osiris-online.eu/index.htm

MyFIRE Page 11/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

5. Case study: BonFIRE

BonFIRE is a "multi-site cloud facility for applications, services and systems research and experimentation" [5, 22]. As such, it goes to the heart of Europe's vision of the Future Internet, supporting experimentation of the Internet of Services. BonFIRE testbeds are provided using Infrastructure as a Service (IaaS) delivery, with BonFIRE providing some additional features on top of them (though a complete Platform as a Service offering is not in scope for BonFIRE). BonFIRE explicitly has to make the testbed sustainable after the end of the project among its objectives [24].

5.1. Benefits of BonFIRE

For its users, BonFIRE offers easy-to-use tools and services to support cloud federation, virtual machine management, service modelling, experiment lifecycle management, quality of service monitoring and analytics. Why should an experimenter, or other user, use BonFIRE rather than existing public cloud offerings? BonFIRE offers something different, beyond what is provided by these commercial providers: Heterogeneous resources Advanced low-level control and monitoring APIs Scale beyond current research testbeds Experimental tools for verifiability and reproducibility, notably standard (Open Virtualization Framework (OVF)7) and domain-specific experiment descriptors BonFIRE is for innovative experiments and works closely with its users to support their requirements; ultimately, using BonFIRE will reduce costs by freeing researchers to focus on innovation, rather than testbed operations [2].

5.2. Heterogeneous but seamless

BonFIRE currently comprises six geographically distributed testbeds across Europe, which offer heterogeneous Cloud resources, including compute, storage and networking [4]. BonFIRE exposes a common cloud interface for the management and monitoring of the virtual resources, based on the Open Cloud Computing Interface8. This is an open, community-led RESTful protocol and API for all kinds of resource management tasks. OCCI has evolved into a flexible API with a strong focus on interoperability while still offering a high degree of extensibility [25]. In BonFIRE there are three sorts of underlying infrastructure. They are as follows: OpenNebula-based9: provided by BonFIRE sites at The University of Edinburgh, United Kingdom (EPCC), Institut National de Recherche en Informatique et en Automatique, France (INRIA), Universitt Stuttgart, Germany (USTUTT-HLRS). Virtual Wall, provided by Interdisciplinary Institute for Broadband Technology, Belgium (IBBT)10; and Cells11, provided by Hewlett-Packard Labs, United Kingdom (HP). These testbeds also expose an OCCI interface, but this is not available to the user. The user interacts with the BonFIRE OCCI-based interface, which hides minor technical differences of the site OCCI, while still exposing site-specific functionality. The BonFIRE OCCI interface and the differences between testbeds are described, amongst other information, in detail in the very helpful BonFIRE User Documentation [3].
7 8

http://www.dmtf.org/standards/ovf http://occi-wg.org/ 9 http://www.opennebula.org/start. OpenNebula is an open-source project developing the industry standard solution for building and managing virtualized enterprise data centers and cloud infrastructures 10 http://www.ibbt.be/en/develop-test/ilab-t/virtual-wall 11 http://www.hpl.hp.com/open_innovation/cloud_collaboration/projects.html

MyFIRE Page 12/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

Figure 2: Geographically distributed testbeds and testing scenarios. Source: http://www.bonfire-project.eu/infrastructure

5.2.1. Built by service providers for service providers

There is often a pre-conception that outsourcing entails loss of control, and compromises on quality and reliability. As an example of the forward thinking of BonFIRE as it strives to build an attractive environment for users, BonFIRE is built by service providers for service providers [2]. The BonFIRE consortium also incorporates many years experience in Cloud and earlier technologies. Unlike existing commodity Cloud services, BonFIRE is working with its users in consultation, to ensure that experimental hypothesis, design and execution all work in harmony. By allowing researchers to focus on innovation rather than testbed operations echoing the thinking behind support for SMEs by Canadas DAIR programme BonFIRE will reduce costs for its users, and enable them to achieve their results more quickly and efficiently. BonFIRE is also easy to use, with an API which allows users to control and monitor experiments at a much more controllable level than would otherwise be possible. This API is also exposed through a Portal, a domain-specific experiment descriptor and, from summer 2012, the OVF Open Virtualisation Format12 language. Monitoring data are exposed through the Zabbix13 monitoring framework; these data uniquely include infrastructure (i.e. physical host), virtual machine and application metrics. BonFIRE also focusses on control: this is delivered through controlled emulated networks at the Virtual Wall site; controlled placement of VMs on specific sites and specific physical hosts on specific sites; and exclusive access to physical hosts. Bandwidth on demand, based on the GANT AutoBAHN14 facility and controlled network routing as available from the FEDERICA15 experimental facility will also feature on BonFIRE later in 2012. In this way, BonFIRE should have a sound basis for long-term sustainability, not only in itself but as part of a wider FIRE federation. BonFIRE will offer Open Access to the research community at large later in 2012; access will require vetting of the experimentation targets, with each request for access evaluated on a case-by-case basis considering
12 13

http://www.dmtf.org/standards/ovf http://www.zabbix.com/ an enterprise-class open source distributed monitoring solution for networks and applications 14 http://www.geant.net/service/autobahn/ 15 http://www.fp7-federica.eu/ Federated E-infrastructure Dedicated to European Researchers Innovating in Computing network Architecture

MyFIRE Page 13/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

business, infrastructure, quality and coverage impact, and contractual agreements will be established between parties as necessary. In this way, access to BonFIRE is developing in three stages: 1) Driving Experiments, 2) Open Call Experiments and 3) from later in 2012, controlled Open Access.

Figure 3: BonFIRE Project Timeline: How to get involved. Source: http://www.bonfire-project.eu/involved

5.3. Initial use cases: the Driving Experiments

To help to ensure that the BonFIRE facility remains state-of-the-art and applicable to the challenges facing researchers, the BonFIRE project developed in the early stages three driving experiments, which posed real research challenges in Cloud computing and acted as examples to push forward the facility requirements, development and operations. In the emerging Cloud and Infrastructure as a Service models, offering services to a commercial standard raises new questions about measuring and ensuring Quality of Service and service provision; these issues are the focus of some of the driving experiments undertaken as part of the BonFIRE project. In this way, these early experiments have a double advantage; not only do they raise the quality of the testbed offering, but they also start to envisage ways in which the service can be made ready for to support a wider user community.

5.3.1. Driving Experiment 1: Effective Cloud Software Testing

Experiment 1, in which the partners were SAP and HP Labs, addressed the real business problem of software quality, which is more complex in Cloud. Although Cloud platforms are the clear candidate as a business choice to develop new applications exploiting the Internet to conduct transactions, Cloud applications are more complex than traditional ones due to the cloud paradigm novelty and as a consequence it is more difficult to test them effectively. Software testing and quality management consume 30% - 50% of development budget according to a recent survey [1]. Therefore, organizations that have high software quality requirements and expectations might struggle to perform adequate system, performance and security tests on such applications. The BonFIRE infrastructure was shown to be a viable solution to this issue. In this experiment, a blueprint for cloud-based software testing based on business and technical criteria (i.e. ease of use, controllability, observability, predictability, repeatability, reliability, availability and cost effectiveness) was defined. On the one hand BonFIRE facilitated this definition, and on the other hand the experiment validated BonFIRE and provided feedback in order for BonFIRE to be a cloud-based software testing facility.

5.3.2. Driving Experiment 2: QoS-Oriented Service Engineering for Federated Clouds

This experiment, carried out by IT Innovation at the University of Southampton, focuses on enabling efficient service engineering tools for modelling, analysing and planning Quality of Service (QoS) for service based applications deployed within federated Clouds. Today, Infrastructure-as-a-Service (Iaas) QoS offerings are expressed in low-level terms (e.g. CPU speed, disk space, etc). Their customers, typically application users, are often interested in application-level parameters because the application is the thing

MyFIRE Page 14/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

that gives the customer the value (e.g. CFD simulation or video rendering). Therefore, the gap between the terms the Infrastructure provider offers and what the users really want is large which results in a complex relationship between application performance and resource parameters. The complexity of this relationship is increased for applications deployed across federated clouds where even low-level resource descriptions may differ due to lack of standardisation. The research questions for the experiment include: Does the expression of IaaS parameters in terms of application-class benchmarks simplify the creation of application-level QoS that can be easily understood by users, whilst being usable for application modelling to predict application performance? How does specification of QoS in application-level terms provide efficiencies for users and providers in a service marketplace? In this way, the experiment not only addresses the specific research challenges detailed above, but also provides a concrete exemplar scenario where results from Cloud research project can exploit FIRE, and can go beyond what is possible within the current project and provide driving requirements for the FIRE facility, and addressing a very urgent business requirement. This experiment produced useful results which have led to several publications [6, 23, 26].

5.3.3. Driving Experiment 3: Elasticity Requirement for Cloud -Based Applications

The third driving experiment, carried out by Atos, addresses a real business problem - how to provide resources efficiently, under extremes of changing conditions, especially in Infrastructure as a Service platforms? This is an important problem because while the current Cloud approach is very appealing for users, with pay-per-use and scalability, it is a problem for providers to maintain a sound model. The target of the experiment is to determine experimentally the elasticity requirements for cloud based web applications, to help providers to comfortably keep within SLA levels, without excessive overprovisioning of resources. To achieve this goal, in a first phase the experiment stressed web applications with different load patterns and different provisioned infrastructures. The scalability policies were verified in a second phase, by studying their behaviour under changing loads.

5.4. Expanding the user community: Open Calls

As with the other second wave and later FIRE facility projects, BonFIRE made use of an Open Call process to expand its user base and strengthen its offering. As well as being able to use the infrastructure as early users, chosen experiments are funded at up to around 200,000 each, a total of 1340k over two such calls. The application process was a simple one-stage process, with experiments lasting around 12 months. The first Open Call closed on 9th March 2011. The second open call closed on 7th March 2012, and the results were not available at the time of writing this deliverable. The interest for the BonFIRE Project in the open call experiments is in the use and improvement of the BonFIRE facility, however the selected experiments nevertheless pursue sound research results. Thus, the overall aims of the Open Call are to extend the BonFIRE user community; to identify new functionality in response to researchers requirements; to identify areas for improvement in correct functioning and performance; and to get feedback on the operation and usability of the facility. Further details about the Open Call process can be found on the FIRE web site [16]. And, ultimately, on this basis, to develop a facility which is able to attract a good number of users, deliver innovation and become sustainable.

5.4.1. Results of first BonFIRE Open Call

The first Open Call exceeded the best expectation of the BonFIRE team, attracting 28 proposals. Of these, four were selected for funding:

MyFIRE Page 15/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

5.4.2. TurboCloud - two SMEs towards what could be a real product

TurboCloud is a very promising collaboration with two Irish SMEs, which combines two complementary technology platforms: the Cloudium16 Chipset enables server-based desktop virtualisation, while the Redzinc17 Virtual Path Slice controller enables dynamic virtual path slices to deliver a right of way across the Internet without interference from unwanted traffic. Both technologies are in beta development stage. The hypothesis for this experiment is that by combining virtual path slice technology with server-based desktop virtualisation, a satisfactory user experience can be achieved especially where multimedia applications are used. This experiment therefore represents an innovative collaboration between complementary technologies; a close collaboration with the SME sector; and a relatively close-to-market development project. It has received favourable high-profile press coverage [27]. If similar usage can be established for BonFIRE in larger numbers, this represents a sound route towards eventual sustainability. For the three other experiments from the first Open Call, a notable facet is the extent to which they draw on, and extend, earlier experiments. In this way, not only are these earlier experiments sustained, and are able to lead to further research, but also BonFIRE is assured of the usefulness of these experiments in their own right.

5.4.3. VCOC buiding on eIMRT

The main objective of the VCOC (Virtual Clusters on federated sites) experiment is to evaluate the feasibility of using multiple Cloud environments to deploy Services which need the allocation of a large pool of CPUs or virtual machines to a single user. This experiment is run by CESGA18, and builds on CESGAs earlier eIMRT19 project because it uses an application which calculates the dose for radiotherapy treatments based on Monte Carlo methods, as an example to trial these services. The experiment is concerned with the time to deployment and enlargement of such clusters alongside the influence of other simultaneous operations in the process; that is, the management of these virtual clusters. A second concern is the usage of the distributed capability of Cloud providers in order to protect the service against failures. On BonFIRE, this experiment benefits from controllability over parameters such as network parameters such as bandwidth, latency or packet loss; in-depth monitoring of factors in a way which is not possible in a commercial Cloud; and experiment descriptors reducing the effort by the experimenters to set up the test environment. Moreover, exploiting BonFIREs unique characteristics, it can do this on a multi-site environment, and thereby gain very valuable information about how to deploy and manage virtual clusters across several sites. In return, VCOC has user-tested most of the components of the BonFIRE infrastructure, including the detection of performance bottlenecks, and in addition it helped to improve the final user experience. VCOC has made a large number of new requests from the project, and their implementation is an excellent example of genuinely user-led innovation.

5.4.4. ExSec: Experimenting Scalability of Continuous Security Monitoring building on GridTrust & RESERVOIR
The ExSec experiment aims to determine an empirically validated elasticity function for security monitoring; this is continuous security monitoring and therefore has the potential to impact negatively on system performance. ExSec is let by CETIC20, an applied research centre which specialises in the area of ICT, particularly in technology transfer with SMEs.
http://www.cloudiumsystems.com/ http://www.redzinc.net/ 18 http://www.cesga.es/ 19 http://eimrt.cesga.es/ Advanced Systems for Radiotherapy Planning Using Distributed Computation 20 http://www.cetic.be/
16 17

MyFIRE Page 16/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

From the point of view of sustainability going into BonFIRE, this experiment is an extension of work done in the FP6 GridTrust project, which developed a framework to perform continuous security monitoring on Grid technologies, and of the FP7 project RESERVOIR21, which adapted a portion of this framework for the policy-based access control to Cloud technologies. However, only small-scale security tests with a handful of virtual machines (and grid nodes) were performed during GridTrust and RESERVOIR. The ExSec experiment aims to perform a much more rigorous scalability test, including different types of hypervisors and different types of Cloud environment managers, for applications requiring continuous security monitoring in the cloud. Testing and validating at this scale and rigour requires that experiments are run on a real large-scale heterogeneous Cloud infrastructure, together with technical solutions and a strong level of support, such as is provided by BonFIRE. It is worth noting here, as example of sustainability and influential innovation, that the RESERVOIR project paved the way for a number of current European projects, including BonFIRE among others. Among successful outcomes were OpenNebula, maintaining openness and open source in Cloud computing. Indeed, BonFIRE uses and also contributes to OpenNebula. The ExSec experiment will benefit the research partner, CETIC, not only through the experimental results, but also by enabling CETIC to build its skills in cloud computing and to extend its consulting services in distributed systems security. The results of ExSec experiment will enable CETIC to help businesses in identifying the best security architecture that will fit their Cloud architectures and performance requirements. For BonFIRE, ExSec provides a way of testing and validating much-needed tools for monitoring the security properties of heterogeneous federated Cloud deployments, and in terms of infrastructural improvements, feedback through hands-on testing, and sharing of security-related know-how.

5.4.5. TEOS: Testing Optimisation for Service Ecosystems

The Testing Optimisation for Service Ecosystems project, led by the Service Systems Design Group at the University of Manchester22, aims to determine the conditions for achieving resilient and optimal service compositions on a distributed cloud infrastructure for the Future Internet. This experiment will deploy and test two service optimization models, characterized as global optimization and local optimization. The "global optimization" model is of interest from the point of view of sustainability, because it was developed in the FP7 SOA4All project23. TEOS aims to determine ways in which both optimization models can be distributed and deployed on multi-site Cloud, and test their performance. The multi-site computational and storage resources provided by BonFIRE represent an ideal platform for testing the two optimisation models in various configurations, and for testing in real-world scenarios. For BonFIRE, the TEOS experiments are designed to evaluate the various facilities and features of BonFIRE, such as the ability of the infrastructure to cope with dynamisms of service ecosystem, elasticity, cross-site communication between components and dynamic management of Virtual Machines; and will develop and test mechanisms such as dynamic management of experiments.

21 22

http://www.reservoir-fp7.eu/ https://research.mbs.ac.uk/service-research/ 23 http://www.soa4all.eu/

MyFIRE Page 17/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

6. Open Calls and Open Innovation

Open Access and the increasingly significant topic of Open Innovation [9] are emerging as important themes for Future Internet research, not only in Living Labs (for example, Botnia24 is a partner in the TEFIS facility project) and Smart Cities such as Smart Santander25, where they have a clear and long-standing recognition, but also in experimentation at lower levels where the "users" are experimental researchers, and where formal and informal interactions between service users and service providers are being actively encouraged. In Open Innovation, expertise is not all sought within the firm, but rather, external resources, companies, and experts are drawn into the business model. Open Innovation is related to User Innovation [30] and to Innofusion [21], but is concerned more with value capture, whereas User Innovation is more interested in value creation. FIRE is not, of course, a commercial firm, but is nevertheless capturing the value from innovation by experimenters, as well as helping to create value through strengthening the testbeds, and, in Open Calls, is a very open form of organisation.

6.1. Open Calls and sustainability

All of the FIRE facility (IP) projects in the second and later waves of FIRE (IPs) have made use of an innovative Open Calls process to enlarge their user community, strengthen their set of use cases, harden the testbed offering through identification of gaps and problems, and define user-led new functionality. Whether or not explicitly stated, the ideas behind Open Innovation are influential in the thinking behind this Open Calls process. This is innovative innovation [28]: it is not simply that there is innovation itself a complex topic, and the ultimate driver of FIRE and other FP7 research in the longer term; there is also innovation in the processes of innovation. For example, the detailed announcement for the TEFIS open call in March 2011 [29] is clear about the three objectives of the call; the call information for the other projects issuing calls contains similar objectives: The TEFIS Open Call will: Incorporate a set of experiments to run tests orchestrated across multiple and heterogeneous testbeds Support the technical activities of the TEFIS project by providing feedback about the use of the platform and new functionalities that would be interesting to include in future versions Support the dissemination activities of the project by providing use case examples Thus, the overall aims of the Open Call are, in addition to providing rigorous research outputs, to extend the user community and dissemination; to identify new functionality in response to researchers requirements; to identify areas for improvement in correct functioning and performance; and to get feedback on the operation and usability of the facility [16]. And, ultimately, on this basis, to develop a facility which is able to attract a good number of users and become self-sustaining. As well as being able to use the infrastructure as early users, chosen experiments are funded at up to 200,000 each. The application process was a simple one-stage process. These experimental projects are intended to be small one or two consortium partners and quite short in duration, with experiments lasting around 12 months.

24 25

http://www.openlivinglabs.eu http://www.smartsantander.eu/

MyFIRE Page 18/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

7. Conclusions
FIRE is developing mechanisms towards sustainability not only of its own infrastructure but, equally importantly, of the knowledge, "know-how" as human knowledge, experiences with developing Cloud and other Future Internet computing, and experimental results. These experimental results, and the experiences of running them, in turn, are increasing the European skill set in Future Internet in research centres, industry, and SMEs.

MyFIRE Page 19/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

8. References
1. Ballou, M.-C. Improving Software Quality to Drive Business Agility http://www.coverity.com/library/pdf/IDC_Improving_Software_Quality_June_2008.pdf Accessed 04/06/2012 2. BonFIRE Project. BonFIRE Cloud Quarterly - Issue 01: September 2010 http://www.bonfireproject.com/sites/default/files/bonfire-cloud-quarterly-Issue1-Sept10.pdf 3. BonFIRE Project. BonFIRE User Documentation Release 2.0 http://doc.bonfireproject.eu/R2/BonFIRE.pdf 4. BonFIRE: Infrastructure http://www.bonfire-project.eu/infrastructure Accessed 5. Supporting Innovation in an Internet of Services http://www.bonfire-project.eu/innovation Accessed 6. Bouckaert, S., Vanhie-Van Gerwen, J., Moerman, I., Phillips, S. C., Wilander, J., Ur Rehman, S., Dabbous, W. and Turletti, T. Benchmarking computers and computer networks. 2011. 7. CANARIE. DAIR: Digital Accelerator for Innovation and Research/ATIR: lAcclrateur technologique pour linnovation et la recherche http://canarie.ca/templates/news/docs/DAIR.pdf 8. DAIR Pilot Program http://canarie.ca/en/dair-program/overviewhttp://canarie.ca/en/dairprogram/overview Accessed 04/06/2012 9. Chesborough, H. W. Open Innovation: The New Imperative for Creating and Profiting from Technology. Harvard Business School Press, Boston, MA, USA, 2003. 10. Crowcroft, J., Demeester, P., Magen, J., Tran-Gia, P. and Wilander, J. Towards a collaboration and high-level federation structure for the FIRE Facility ("Wise Men" report). 2009. 11. European Commission Community Framework for State Aid for Research and Development and Innovation: 2006/C 323/01. 2006. 12. European Commission. Information and Communication Technologies Updated Work Programme 2011 and Work Programme 2012. 2011. 13. European Commission. Information and Communication Technologies Work Programme 2011-12. 2011. 14. European Commission Mid-Term Review of the R&D&I Framework: Commission Staff Working Paper: Brussels 10.08.2011. 2011. 15. FIRE: Future Internet Research and Experimentation. FIRE White Paper http://www.ictfireworks.eu/fileadmin/documents/FIRE_White_Paper_2009_v3.1.pdf Accessed 12 March 2011 16. 1st FIRE "Open Calls" information day http://www.ict-fire.eu/events/other-fire-events/1st-fireopen-calls-information-day.html Accessed 17. List of initiatives / actions of interest to FIRE http://www.ict-fire.eu/home/fire-relatedinitiatives.html Accessed 17/06/2012 18. FIREStation. FIRE Roadmap Report I - Part II: Deliverable D3.5 http://www.ictfire.eu/fileadmin/publications/deliverables/D3_5_FIRE_Roadmap_II_02122011_v1_0.pdf 19. FIREStation. FIRE Roadmap: Deliverable D3.4 http://www.ictfire.eu/fileadmin/publications/deliverables/D3_4-puCommon_roadmap_of_FIRE_test_facilities_%E2%80%93_First_version_v1.0.pdf 20. FIREStation. 1st FIRE Portfolio Update Concept of Experimentally-Driven Research and its Facilities. Deliverable D2.1. 2011. 21. Fleck, J. Innofusion or diffusation? : the nature of technological development in robotics. University of Edinburgh, 1988. 22. Hume, A., Al-Hazmi, Y., Belter, B., Campowsky, K., Carril, L. M., Carrozzo, G., Engen, V., Prez, D. G., Ponsat, J. J., Kbert, R., Liang, Y., Rohr, C. and Seghbroeck, G. V. BonFIRE: A Multi-cloud Test Facility for Internet of Services Experimentation. In TridentCom 2012 (Thessaloniki, Greece, 11/06/2012, 2012).

MyFIRE Page 20/21

258348

MyFIRE

D3.3d Routes to sustainability in FIRE

23. Marquezan, C. C., Metzger, A., Pohl, K., Engen, V., Boniface, M., Phillips, S. C. and Zlatev, Z. Adaptive Future Internet Applications: Opportunities and Challenges for Adaptive Web Services Technology Adaptive Web Services for Modular and Reusable Software Development: Tactics and Solution., 2012. 24. Martrat, J. BonFIRE: Presentation to FIRE open calls info day, Brussels, February 9th, 2011 http://www.ict-fire.eu/fileadmin/events/2010-021OpenCall/FIRE_Open_Calls_Info_Day_9_Feb_2011_-_BonFIRE.pdf 25. Open Grid Forum Open Cloud Computing Interface - Core. GFD-P-R.183 version 1.1. 2011. 26. Phillips, S. C., Engen, V. and Papay, J. Snow White Clouds and the Seven Dwarfs. In IEEE 3rd International Conference on Cloud Computing Technology and Science, CloudCom 2011 (Athens, Greece, 29/11/2011-01/12/2011, 2011). 27. Richardson, D. Irish firms receive lucrative cloud technology contracts. 2011. 28. Seely Brown, J. Foreword Open Innovation: The New Imperative for Creating and Profiting from Technology H. W. Chesborough Harvard Business School Press, Boston, MA, USA, 2003. 29. TEFIS project. Announcement of a competitive call to select new experiments for the TEFIS project http://www.tefisproject.eu/media/upload/Detailed-Call-information_TEFIS-final1.pdf 30. von Hippel, E. The Sources of Innovation. Oxford University Press, Oxford, UK; New York, NY, USA, 1988.

MyFIRE Page 21/21