Social group formation with mobile cloud services

1 Introduction world for document sharing. Similarly, thousands Social networks [7] have become very popular these days, with millions of users, from several countries and with different backgrounds and interests. Facebook [14] and Twitter [49] are the most popular social networks of them, with more than 800 million and 200 million active users, respectively. This number of people present in the network is a good opportunity to create professional and business connections around the world. People with common interests in the social network can take advantage of the social network benefits like public recognition, creation of relationships, referrals, and manage the social capital. Social groups [36] also result in sharing and collaborative relationships between the members. For example, a group of people working in the same research topic can collaborate with each other sharing resources, ideas, and knowledge, etc. This helps in achieving their common goals faster. At the same time, the cloud computing [4] domain has become quite popular and has led to new business models and application opportunities such as intensive batch processing applications for business analytics, human activity recognition, collaborative tools, among others. Cloud computing is a style of computing in which, typically, resources scalable on demand are provided as a service (aaS) over the Internet to the users who need not have knowledge of, expertise in, or control over the cloud infrastructure that sup- ports them. The provisioning of cloud services occurs at the Infrastructural level (IaaS) or Platform level (PaaS) or at the Software level (SaaS). The vast number of services provided in the cloud has become a commodity in peoples day-to-day life. For instance, collaborative tools such as Google Docs and Zoho are extensively being used by millions of people around the of pictures are being uploaded daily to cloud services such as Picasa or Flickr for sharing and storing media files. Like- wise, services such as Google Maps and Bingo Maps enable the provisioning of location-based services. However, a cloud service usually does not provide a rich functionality by its own. Several services are to be combined with each other to add a significant value for the user, pushing the mash-up idea. For example, by blending Google Maps and SoundCloud, a media file sharing service; it is possible to visualize the songs and tracks uploaded by a user in the geographical location where those media files were recorded/uploaded. Furthermore, the development of more complex and richer applications like foursquare [21] is only possible by blending several rich services like social network, locationbased services, media files storage, IaaS. To sum up, a mashup application is one that blends or mixes two or more services into a single application, combining data, presentation, and functionality, with the aim of enriching its functionality, improving the user experience, and adding more value for the user. Finally, the development of a mash-up application implies the easy and fast integration of different APIs and data sources into a single application. Meanwhile, the mobile computing domain also has advanced rapidly and enabled the new generation of cloudbased and context-aware mobile applications. Consequently, Clouds are looking forward to the mobile domain, having their expectations focused in the idea of fostering the access and consumption of cloud services at the different levels from mobile devices. Nowadays, mobile devices are equipped with embedded sensors and input devices such as cameras, GPS, accelerometer, magnetic

sensor, among others. Moreover, these new capabilities can be combined also with other services and mash-up applications giving place to mobile mash-up applications. A mobile mash-up application not only blends into a single application several services but also uses the data gathered by embedded sensors and devices in order to enrich the mobile application. For example, four- square uses the GPS sensor embedded in the device to deter- mine the users location and provide information about the services around. Similarly, Zompopo [43] is a human activity recognition application, which analyses the data collected by the accelerometer, for identifying the activities. However, these new applications that rely on the cloud are only possible due to the recent enhancements in mobile devices. Improvements on hardware (embedded sensors, memory, power consumption, touch screen, better ergonomic design, etc.), in software (more numerous and more sophisticated applications due to the release of iPhone [3] and Android [2] platforms), and in transmission (higher data transmission rates achieved with 3G and 4G technologies and ubiquity of Wi-Fi networks) have contributed toward having higher mobile penetration and better services provided to the customers. This has lead to the Mobile Cloud Computing (MCC) domain. Such a mobile cloud benefits applications from several domains like context-aware, social networks, productivity tools, biometrics. Decaf [51], a cloud- resource management application, and Bakabs [37], a cloud- based cluster estimation application based on load, which uses data from Google Analytics and linear programming techniques in the process, are clear examples of mobile applications accessing cloud services. Applications from such Mobile Cloud Computing domain usually combine cloud-based services with basic context-

capabilities are included in most of the new operating systems of mobiles phones. For example, Android in its version 4 included Face Unlock and gesture detection features. Even though face detection can be done in the device, the facial recognition process requires more time and resources. For facial recognition, a training repository of identified faces is needed and it demands huge amounts of processing power for the detection and recognition of a specific person in the repository. Face detection and facial recognition are typically used in security systems for identifying people among the crowd. However, with the mass amount of pictures present in the social network and media sharing services in the cloud, and with the increased hardware and software capabilities of the mobile devices, it is possible to apply these technologies in other scenarios such as the social group formation. To bring the benefits from these different domains together, we tried to build an Android application that aids in forming social groups of common interest, from the mobile devices. The developed CroudSTag (Social tagging crowd on cloud) application takes a set of media files (pictures and videos probably taken by the mobile device itself) from the cloud, uses the SaaS of face.com [15] to recognize people from the media files, and sends, to each recognized person, an invitation to be part of a social group on facebook. The application is explained in detail with the technological and architectural choices. The rest of the paper is organized as follows. Section 2 describes the developed application in detail, with screenshots. Section 3 discusses the cloud services employed in the realization of the application. Section 4 discusses the architectural and technological choices along with our developed Mobile Cloud Middleware (MCM) [19]. Section 5 provides a detailed analysis of the application. Section 6 discusses the related work, and Sect. 7 concludes the paper with future research directions.

functionalities such as speech recognition,

awareness, and face detection among others. Out of them, face detection, gesture recognition, and facial recognition

2 CroudSTag: application description and screenshots CroudSTag is a mobile application developed for Android devices, with the aim of aiding in the social group formation by means of facial recognition technologies and MapReduce [11]video processing. CroudSTag

camera embedded in the device is used to take the media content (screenshot 2)and when the user is finished and taps the save button, he/she is asked to select the cloud provider where the files are going to be stored (screenshot 3). Multiple clouds are considered for keeping the files, as the researcher may store the pictures in his private cloud (something like our SciCloud [42], a private cloud based on Eucalyptus technology with Walrus storage) when in Europe and while travelling through United States or Japan, he/she may upload them to public clouds like S3 or GoGrid [23]. Once the image/video is saved in the cloud, the application comes back to the main menu from where the user can take a new video/picture or start the social group formation process.

recognizes the people who appear in media content such as pictures or videos and joins them together into a social group. For example, consider a researcher who attends conferences around the world and has a set of media content (pictures and video files) of the people with whom he/she had interacted at the event. The media files are probably taken from his/her mobile itself and are stored on the cloud. The researcher later wants to create and keep connections with his acquaintances on the social network. He/she also wants to group them according to specific interests and would like to follow the groups directly from his/her mobile phone. The scenario can also be envisioned with any other type of the event or community that wants to keep its members in contact, something like alumni. 2.1 CroudSTags application flow Figure 1 shows the screenshots of the CroudSTag usage scenario, which is an extension of the original CroudSTag application presented in our previous work [41]. With the help of CroudSTag application, the user can upload pictures and videos to the cloud and store them in Amazon Simple Storage Services (S3) [1], as shown in the top cycle of the figure. When the user starts the application, three buttons are shown: facebook Login, Take Picture, and Take Video, as shown in screenshot 1. The facebook login goes directly to facebook and authenticates the user getting an authorization token, which is used for accessing the facebook graph and also for accessing the users tags during the facial recognition process. This authentication token is also needed to send the invitation to people, to join the social group. The Take Picture and Take Video buttons are used to take a picture/video, respectively, and upload the file to S3. The

Fig. 1 Screenshots of the CroudSTag usage scenario

Infrastructure (Amazon Elastic MapReduce, Amazon S3, The social group formation process starts with facebook login, as explained in above paragraph (screenshot 4). Once the user is logged-in, the application enables a new button CroudSTag on the main screen (screenshot 5). Once the user taps the button, the social group formation process is initiated and he/she is asked to select the cloud provider where the media files to be processed are stored (screenshot 6). The media content stored in the cloud is used to detect and recognize users acquaintances and contact them in facebook. The device sends the Mobile Cloud Middleware (MCM) (discussed in detail in Sect. 4) a request for the social group formation service along with the authorization token from facebook and the folder and cloud where the media files are. As already mentioned, the facebook authorization token is required by the facial recognition process in order to access the users content (pictures, friends, and tags). Face recognition is performed based on SaaS from face.com. The service is explained in detail in the next
f ol d er.ad d encod ed _i mages encod eV i d eo I nt o I mages(vid eo) for all i mage encod edi mages do

Sci-Cloud Walrus storage) and Software (face.com and facebook ) levels. Moreover, CroudSTag benefits from the composition of several cloud services that are combined together to pro- vide a complex and rich functionality to the user in an entirely transparent process. To be clear, CroudSTag relies on three different cloud services that are combined with each other to create a social group out of the people who can be identified in media files. The facial recognition and identification process employed in the application is described in the Algorithm 1 Algorithm 1:Facial Recognition and Identification
Require: f ol d er.l engt h > 0 Ensure: l i st of f acebook users i dentified {Encoding Step} for all vid eo f ol d er do

section, and MCM is discussed in detail in Sect. 4. Right after the request is received by MCM, it sends an acknowledgment back to the mobile, notifying that the process has been started. Then, the device is free to start other applications or a new process at MCM. When MCM finishes with the facial recognition process, it sends the list of people recognized, asynchronously to the device. Once this list is received, the application displays it to the user (screenshot 7), who decides whom to send the invitation to join the social group in facebook. Each invitation is sent in the form of a private message containing a link to accept the invitation and join the social group in the facebook (screenshot 8). The invitees will later join the social group based on their interests. The entire scenario is illustrated in detail in Fig .1 3 Cloud services employed in the application:
end for end for

(image)

images F or I d ent i f icat i on em pt y_li st {Pre-Processing Step} for all i mage f ol d er do i mage smoot h(image) gr ay Scal e I mage tr ans f or m T oGr ay Scale(image) gr ay Scal e I mage ui l i ze I mage H i st ogr am(gr ay Scal e I mage) eq f aces d et ect O b j ect s(gr ay Scal e I mage) if f aces.l engt h > 0 then images F or I d ent i f icat i on.ad d (i mage) end if end for

CroudSTags functionality is based on cloud services at

persons em pt y_li st

a list of key value pairs, and the application core consist of

{Facial Identification Step} for all i mage images F or I d ent i f icat i on do per sons I n I mage F aci al I d enti f i cati on Saa S(image) per sons.ad d ( per sons I n I mage) end for J S O N Res ponse encod eT o J S O N ( per sons)

two methods, Map and Reduce. Map method processes each keyvalue pair in the input list separately and outputs the result as a keyvalue pair. The output of the Map method is grouped by

In the first service, each video file is encoded into several images using ffmpeg [16], an open source utility for video processing. Afterward, all the images are pre-processed with the aim of removing the noise, selecting only those images that are good candidates for the face detection and identification process. An image with at least one face detected in it is considered as a good candidate for the identification process in face.com. So, all the images without the faces are discarded from further processing. In order to detect a face in an image, the Open Source Computer Vision (OpenCV) [8] libraries for Python have been used. OpenCV is a library of programming functions for realtime computer vision created by Intel. It provides several libraries mainly in C but also for Python programming language. It has been extensively used for facial detection, gesture recognition, robotics, motion tracking among other areas. The developed opencv-python script takes as input an image and returns a list of square coordinates where each square frames a face, which is detected in the picture. Figure 2 illustrates the face detection process using opencv libraries. However, this pre-processing step can be very timeconsuming as the encoding process generates a considerable number of images from each video file (5,000 images from a 3 min high-definition video), and the opencv script needs to be run for each image. For tackling this problem, a MapReduce [11] algorithm running at Amazon Elastic MapReduce is considered. MapReduce is based on a typical divide and conquer parallel computing model. The general idea of MapReduce is simple. Input to a MapReduce application is In CroudSTag, the map function shown by the Algorithm 2 divides the entire set of pictures into subsets, emitting for each picture a keyvalue pair of <batch id, image> in such a way that each reducer will receive a subset of pictures to process.
Fig. 2 OpenCV libraries for facial detectionimage powered by face.com

the key and is divided between different reduce tasks. Each reduce task gets a key and a list of all values assigned to this key as an input, aggregates the list of values based on a user specified function, and outputs one or more key-value pairs. ma p(key, val ue) [(key, val ue)] r ed uce(key, [val ue]) [(key, val ue)]

picture. Further, face.com uses these elements to match faces Further, the framework groups together all the key value pairs with the same key value and passes the set to the reducer function. The reducer shown in Algorithm 3 goes through the entire set of images trying to detect faces using the python-opencv libraries described above. If an image contains at least one face, the reducer emits a keyvalue pair <image name, image>. Otherwise, the image is discarded and no keyvalue pair is emitted. Algorithm 2 Map function
Require: f ol d er Require: f ol d er.l engt h > 0 Require: bat ch Si ze i 0 for all i mage f ol d er do bat ch i /bat ch Si ze key pai r < bat ch, i mage > emi t (key pai r ) i i +1 end for

detected in the pictures with the faces previously tagged in facebook. Finally, all the matches are returned as a list where each item contains the face information (position for eyes, nose, mouth, etc.), the facebook id and the name corresponding to such face. The combination of face information and facebook information is considered as an identified person. Face.com is a technology company operating in the face recognition domain and provides several SaaS from the cloud. It offers three basic solutions for facial detection: (1) PhotoFinder that scans public photographs in the network and suggests tags for the photographs which are currently un-tagged. The aim of the application is to find pictures of a person in the network. (2) PhotoTagger that lets people choose albums to scan for faces grouping the people recognized in batches called screen shots. Later, PhotoTagger suggests tags for the people present in the screen shots. This batch processing is a more efficient way to tag pictures from existing albums. (3) CelebrityFinder that scans pictures in Twitter looking for celebrities that have posted publicly [13]. In May 2010, face.com released its API and officially

Algorithm 3 Reduce function

Require: bat ch Require: i mages for all i mage i mages do f aces d et ect F aces(image) if f aces.l engt h > 0 then key pai r < image.name, i mage > emi t (key pair ) end if end for

supports languages such as PHP and JavaScript. The community also has developed client libraries for Python, C#, Flash Action Script, Java, and Ruby on Rails. The services for detection, recognition, and tagging are provided through its REST API [17]; Serverhttp.api.face.com. The services are provided for free but with quota limitations. Face.com tags and recognizes users from facebook and Twitter, the two most popular social networks with billions of photographs in their repositories. Face.com complies with facebook and Twitter security policies and thus can use the credentials of the social networks within its recognition process. The recognition process is performed on the public content and the content owned by the user. It is also possible to have private sets of photographs and tags, called namespaces. However, these private namespaces need to be tagged and

The second cloud service of the CroudSTag application is at the SaaS level, which uses the third-party services from face.com and facebook, responsible for the face detection and people identification in the social network. Face.com takes as inputs a set of image files and first recognizes faces detecting position of eyes, nose, mouth, among other elements in the

trained, and thus reduce the number of people who can be recognized and also decrease the accuracy of the results. We also have developed a face recognition service that can be used for analyzing the set of pictures that are stored in our SciCloud. However, we have decided using face.com since the service is free to access and provides more advanced features that are convenient in the context of a social networking. The CroudSTag application also uses facebook services to create the groups. Facebook Applications are the channel provided by facebook, to integrate the core facebook platform technologies such as social plug-ins, the Graph API, and Dialogs. Facebook also provides several SDKs [14] to develop applications. The JavaScript and PHP SDKs are available for web content, and the iOS and Android SDKs are available for mobile applications. The CroudSTag

service of Walrus as it was originally meant from the community for accessing Amazon S3 services. 4 Generic middleware for mobile cloud services invocation While the services employed for the CroudSTag application are interesting, the direct invocation of a cloud service (such as face.com) is resource consuming for a mobile phone. Most often, the operating system of the device tends to get stuck, if the computation offloading requires long waiting time for getting a response back. Moreover, while the handset is waiting for an answer, it cannot make another application call from the user. Hence, the mobile is not able to execute concurrent tasks. Also, such waiting time is not tolerable for the user and mobile application usability perspective. Consequently, it is necessary to rely on a middleware solution for getting the results back asynchronously. Moreover, as already explained, several cloud services are considered in the CroudSTag application. The services are bound by numerous constraints like cloud providers technology choices, platform restrictions. Cloud providers offer proprietary APIs and routines to consume the services, for example, facebook (Open Graph, Java API, etc.) and jetS3t API [29] (Amazon S3 [1] and Eucalyptus [35]).

application uses face- book Dialogs, which provide a consistent interface to display dialogs to users. These dialogs provide the interface to post messages in users walls send friend requests and application requests. The Graph API is the simple view to facebook objects (people, photos, events, and pages) and connections between each other. Every object inside facebook has a unique ID and its information can be accessed through the facebook public URL. The Graph is used to discover the relationship of one object with other objects. Relationships such as friends, events, groups, notes, albums, photographs, tags, likes, etc., can be retrieved through the Graph API. In between the video processing and facial recognition steps, there is a basic service to retrieve the images, which are generated in the MapReduce job, from the cloud storage to the MCM. The images are later sent to face.com for identifying the individuals. For accessing the storage services in Amazon S3 and Walrus from SciCloud [42], the two storage services considered in CroudSTag, the native Amazon API and jets3t [29] API, are used, respectively. jets3t needs slight modification for handling the storage

Therefore, cloud interoperability is not possible and when a mobile mash-up application is to be created, it has to be developed for a specific cloud provider increasing the application complexity at the time to be created, footprint at the time to be installed, and resource consumption at the time of running in the device. Another big problem associated with such a usage is that the cloud service provider has to supply his API specific to several mobile platforms like Android or iOS. This is most often not in the cloud providers agenda and it may take some time before the open source community comes up with compatible API for the targeted mobile platform. To counter most of these problems in general, we have designed and developed a Mobile Cloud Middleware (MCM).

MCM is introduced as an intermediary between the mobile phones and the clouds in the mobile cloud service invocation cycle. The architecture is shown in Fig. 3. MCM fosters mobile platform heterogeneity and the combination of different cloud services into a mobile mash-up application. When an application tries to connect to a basic cloud service, it connects to the TP Handler component of the middleware, which receives the request. The transportation handler can receive the requests based on several protocols like the hypertext transfer protocol (HTTP) [18] or the extensible messaging and presence protocol (XMPP) [38]. The request is then pro- cessed by the Interoperability API engine, which selects the suitable cloud API and creates a unique adapter that ensures the transactional process with the cloud. When the request is forwarded to the MCM Manager, it first creates a session assigning a unique identifier for saving the system configuration of the handset (OS, clouds credentials, etc.) and the service configuration requested (list of ser- vices, cloud providers, types of transactions, etc.) in a session space, respectively. The identifier is used for handling different requests from multiple mobile devices and for sending the notification back when the process running in the cloud is finished. Later, the interoperability API engine verifies the service configuration for selecting the suitable API, depending on the cloud vendor. A

transactional space is created for exchanging data between the clouds. The aim of the transactional space is to avoid offloading the same information from the mobile, again and again. Once the interoperability API engine decides which API set it is going to use, the MCM Manager requests for the specific routines from the Adapter Servlets. The servlets contain the set of functions for the consumption of the cloud services. Finally, MCM Manager encapsulates the API and the routine in an adapter for performing the transactions and accessing the SaaS. The result of each cloud transaction is sent back to the handset in a JSON (JavaScript Object Notation) [30] format, based on the application design. In case of the CroudSTag application, the final result is sent just once, after all the cloud services are finished. The adapter keeps the connection alive between MCM and the cloud and monitors the status of each task running within the cloud. When all the cloud services are completed, MCM Manager uses the asynchronous notification feature to push the response back to the handset.

Fig. 3 Architecture of the Mobile Cloud Middleware

a synchronicity is added to the MCM by implementing the Android Cloud to Device Messaging Framework (AC2DM) [24] and the Apple Push Notification Services (APNS) protocols for Android and iOS, respectively. APNS messages are sent through binary interface that uses streaming TCP socket design. Forwarding messages to device happens through constantly open IP connection. Similarly, AC2DM is a lightweight mechanism that lets to push a message into a queue of a third-party notification service, which is later sent to the device. Once the message is received, the system wakes up the application via Intent Broadcast, passing the raw message data received straight to the CroudSTag application. Alternatively, MCM also has support for the Mobile Host [45] concept. Generally, the mobile can check for the status of the service regularly by contacting the MCM. However, this puts a lot of load on the mobile networks. Instead we can make the mobile device a service provider, which was studied in mobile web service provisioning project. The study developed a Mobile Host, which can be used in providing web services from the smart phones. When the smart phone acts as a service provider, the mobile cloud service response can be sent directly to the device. The current implementations of the Mobile Host are available for Personal Java, J2ME, and Android platforms.

characteristics and tries to identify a person by comparing his/her face characteristics to those which are already known. This method has been broadly used in applications that rely in the detection and recognition of people using computer vision facilities. However, such applications usually require to be trained with large datasets demanding huge amount of resources and time. To tackle this problem, CroudSTag relies on Amazon Elastic MapReduce (IaaS) and face.com (SaaS) for the pruning and recognition phases. Similarly, Liu et al. [32] proposed a fast face tracking using parallel particle filter algorithm that also relies in the MapReduce framework. Likewise, Chang et al. [10] utilizes the cloud capabilities for creating an access security framework by means of facial recognition. However, these approaches focus only in the detection and recognition, usually for authentication purposes, and they lack the social group formation capabilities present in CroudSTag. Despite, the

enhancements in the facial recognition techniques have improved the accuracy and the rate of detection; they are mostly applied in biometrics and security domains like in Chang et al. work. In contrast, CroudSTag aims the social group formation of people by processing media files via facial recognition. Facial recognition technologies [12] are one of the most promising features for the mobile users. There have been

5 Related work Extensive research has been conducted in the computer vision domain during the past two decades, focusing in face detection and facial recognition. These studies have achieved huge improvements in the generation of algorithms and techniques for facial detection and recognition. However, they happen to be resource and time demanding and thus are not suitable for resource constrained devices like mobile phones. For instance, Turk and Pentland [48] developed a framework for the detection and identification of human faces, which claims to work in near-real-time. The framework describes a face as a two-dimension set of

several attempts to integrate the facial recognition capabilities to mobile devices. Some manufactures such as Sony Ericsson incorporated the feature in its model X10 [40] the Recognizer application. This application enables the user to take a picture using the camera and recognize people present in the picture. It stores the pictures locally in the device and only recognizes people who are present in the contacts list and have a picture associated. However, it lacked the opportunity to search people in the social networks or take advantage of cloud technologies, and eventually overloads the storage resources of the phone with the pictures taken.

Furthermore, Hadid et al. [26] proposes an authentication mechanism for authenticating users by means of facial recognition with the use of mobile devices. It considers Haar-like [31] features with Ad-aBoost [22] for face and eye detection, and local binary pattern (LBP) approach for face authentication. In addition, it uses the facial recognition only for authentication and the process occurs locally in the mobile device. Similarly, Ijiri et al. [28] try to solve the security and usability issues that arise when the mobile devices store private information from the user such as payments, back accounts and credit cards, personal pictures. They propose an authentication mechanism that uses the camera embedded in the device for taking a picture of the user that is analyzed with the help of facial recognition techniques. Even though, Ijiris and Hadids work have proved to be effective, their purpose is distant from CroudSTags aim. Besides, the social group formation by means of composition of cloud services at IaaS and SaaS level is characteristic that truly distinguishes CroudSTag from other applications. The Table 1 summarized the related work in terms of using facial recognition. Viewdle [50] is another promising technology to include cloud technologies and facial recognition in mobile devices. It offers a set of solutions to provide facial recognition capabilities and intends to provide the solutions for both stand- alone and mobile platforms. Currently, only the desktop application in beta version has been released that enables to detect faces in photographs and tries to match them with tags in the users facebook profile. The pictures have to be stored in the computer, and during the recognition process, it is possible to create a facebook album and upload the tagged pictures. Viewdle published in its website a demo of the mobile application, which allows detecting people from one picture taken with the camera and having access to his facebook recent activity and information. Nevertheless, this application has not been released and has some problems.

For example, the recognition process runs entirely in the device that might negatively impact the performance of the device as well as battery usage. In our CroudSTag application, the facial recognition service is supported by face.com. Face.com only provides web or desktop solutions. Since most of the recognition process is performed on cloud, there is no dearth for resources. Face.com is currently looking for commercial applications, which use its face recognition API in order to make incursion in the mobile market. We think CroudSTag may be a possible candidate. Middleware approaches similar to our MCM have also been addressed in the literature. Mobile Cloud Computing Middleware (MCCM) [52] is a project, which involves the use of a middleware, standing between the mobile and the cloud, for the consumption of web services (WS) [9] in a mobile mash-up WS application. It handles creating user profiles from the context of the mobile phone, storing the system configuration (pre-defined set of WS which can be consumed from the handset) and the service configuration (information needed for constructing a request and combining services), respectively, for managing existing resources in the Internet Cloud. However, we have observed their middleware and the API support to be tightly coupled. Moreover, the communication between the client and MCCM happens to be synchronous; therefore, the client has to wait for MCCM to resolve each request which is not suitable for time consuming data intensive tasks. Cloud agency [5] is another solution that aims to integrate GRID [6], cloud computing, and mobile agents. The specific role of GRID is to offer a common and secure infrastructure for managing the virtual cluster of the cloud through the use of mobile agents. Agents introduce features that pro- vide the users a simple way for configuring virtual clusters. Agora [34] is another middleware solution that is in the development, which will enable new large-scale mobile cluster applications and will use mobile devices as nodes of

10

a large- scale cloud computing infrastructure. Agora will enable the devices to work together seamlessly. However, we could not find any concrete implementations. March et al. [33] introduces Cloud, a framework for building rich mobile applications based on composition of cloud services. However, the work does not clarify how the framework scales under heavy workloads neither how it handles the authentication across the different cloud services. In addition, Cloud proposes a facial recognition application; however, it is restricted to detect faces and compare them against a reduced repository of known faces.

MCM mainly enables interoperability across multiple cloud architectures fostering the provisioning of composed cloud services. In addition, MCM allows building lighter and more robust applications for mobile devices since it is capable to manage transactions with multiple clouds in a transparent way for the developer who only needs to deal with a unique interface (MCM) instead of several provider-specific APIs. Therefore, MCM fosters the

creation of cloud-based mobile applications, which are enriched with the benefits of the cloud.

Table 1 Related work for facial recognition and face detection in cloud and mobile computing

Year

Author

Feature

1991, updates until 2002

Turk and Pentland [48]

Introduces a detection and identification of human faces, in near real-time recognition by means of eigenvectors

1994

Samaria and Harter [39]

Proposes the parameterization of a stochastic model for human face identification by means of continuous density Hidden Markov Models

2006

Ijiri et al. [28]

Proposes a security management system for mobile devices by face recognition. Implements an authentication method using the camera embedded in the device

2007

Hadid et al. [26]

Proposes a mechanism for face and eye detection for person authentication in mobile phones. Uses Haar-like features, Ad-aBoost and local binary Patter for authentication of users using the camera embedded in the Devices

2007

Han et al. [27]

Introduces a face recognition method based on near-infrared (NIR) light using mobile phones. The recognition is done locally in the device and it uses the NIR embedded in the device

2009

Liu et al. [32]

Proposes a fast face tracking using parallel particle filter algorithm uses multi-cue (color, histogram, edge orientation and wavelet features) face tracking algorithm implemented in MapReduce

2011

Chang et al. [10]

Proposes an access security system on cloud computing implemented in Hadoop system for mobile devices

2011

March et al. [33]

Proposes a face recognition application by means of composition of basic cloud services

2011

Srirama et al. [41]

Proposes a cloud-based mobile application that aims the social group formation by means of composition of cloud services such as facial recognition, social networks, and MapReduce

11

. One feature that really separates it from other approaches is its support for asynchronous push notification, which frees the mobile resources during most of the invocation process. Moreover, our earlier research also involved working with middleware for mobile web services, where Srirama et al. have realized a mobile web service mediation framework (MWSMF) [46] that helps in offering proper quality of service (QoS) and discovery mechanisms for the services being provided from the smart phones [44]. MWSMF is shown to be reasonably scalable [47], and our future research will try to integrate the MCM with the MWSMF. 7 Conclusions and future research directions Cloud computing and mobile computing domains have advanced rapidly and are the promising technologies for the near future. Joining the technologies together, one can envision several applications, which are suitable for the people in the social networks. The applications may also take advantage of huge user bases of the social networks, increasing their feature richness. CroudSTag is one such application built for the Android devices. The application takes a set of media files (actually collected by the phone itself) from the cloud and uses the cloud services of Amazon and SaaS of face.com to recognize and identified people out of those files. The application then enables the user to send, to each recognized person, an invitation to be part of a social group and it uses facebook for the social group creation. The social groups thus formed with people

and composed services to be performed and thus utilize clouds and MCM in the process. As already mentioned, our future research will also try to include the MCM as a component to the MWSMF. Checking the scalability of the MCM is also very interesting for us. However, if MCM can be attached to the MWSMF as a component, MCM will have the similar QoS as that of MWSMF. We are also interested in adapting the CroudSTag application to specific application domains and user communities. For example, we are in the process of extending the application, to support the researchers complete publication activities, such that the application can be used by the researcher for tracking his research from beginning to the end. The extended application will have support to store the documents, keeping track of their versions, storing the presentations and talks at the conferences in the form of videos, and notifying the user about the upcoming conferences and deadlines.

References
1. Amazon, Inc. Amazon-Amazon Web Services. http://aws.amazon. com/ 2. Android. Android. http://www.android.com/ 3. Apple, Inc. IPhone. http://www.apple.com/iphone/ 4. Armbrust M et al (2009) Above the clouds, a berkeley view of cloud computing. Technical report, University of California, Berkeley 5. Aversa R, Martino BD, Rak M, Venticinque S (2010) Cloud agency: a mobile agent based cloud system. In: 2010 international conference on complex, intelligent and software intensive systems 6. Berman F, Fox G, Hey AJ (2003) Overview of the book: grid

of common interests result in sharing and collaboration relationships between the members. The paper explained the application with detailed architectural and technological choices. The performance analysis of the application shows that the social groups can be formed with significant ease and reasonable performance latencies on the devices. The application also becomes a case study for our MCM. We are in the process of developing other applications as part of our future research, which require resource intensive tasks

com- putingmaking the global infrastructure a reality. Wiley Online Library, England 7. Boyd DM, Ellison NB (2008) Social network sites: definition, his- tory, and scholarship. J Comput Mediat Commun 13:210 230. doi:10.1111/j.1083-6101.2007.00393.x

8. Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. OReilly Media, Sebastopol 9. Cerami E (2002) Web services essentials. 1st OReilly &

Associ- ates, Sebastopol 10. Chang B, Tsai H, Lin Z, Chen C (2011) Access security on

12

cloud computing implemented in hadoop system. In: Genetic and evo- lutionary computing (ICGEC), 2011 fifth international conference on, IEEE, pp 7780 11. Dean J, Ghemawat S (2008) Mapreduce: simplified data

28. Ijiri Y, Sakuragi M, Lao S (2006) Security management for mobile devices by face recognition. In: Mobile data management, 2006, MDM 2006. 7th international conference on, IEEE, pp 4949 29. jets3t (2010) jetS3t-an open source Java toolkit for Amazon S3 and CloudFront. http://jets3t.s3.amazonaws.com. URL last visited on 3th Dec 2010 30. JSON. JSON. http://www.json.org/ 31. Lienhart R, Maydt J (2002) An extended set of haar-like features for rapid object detection. In: Image processing 2002. Proceedings of 2002 international conference on, vol 1, IEEE, pp I900 32. Liu K, Li S, Tang L, Wang L, Liu W (2009) Fast face tracking using parallel particle filter algorithm. In: Multimedia and expo, 2009, ICME 2009, IEEE international conference on, IEEE, pp 13021305 33. March V, Gu Y, Leonardi E, Goh G, Kirchberg M, Lee B (2011) [mu] cloud: towards a new paradigm of rich mobile applications. Procedia Comput Sci 5:618624 34. Narasimhan P Agora: mobile cloud-computing middleware. http:// www.cylab.cmu.edu/research/projects/2010/agora.html 35. Nurmi D, Wolski R, Grzegorczyk C, Obertelli G, Soman S, Youse L, Zagorodnov D (2009) The eucalyptus open-source cloudcomputing system. In: 9th IEEE/ACM international symposium on cluster computing and the grid (CCGRID 09), pp 124131

process- ing on large clusters. Commun ACM 51(1):107113 12. Dominik I, Sankar KP, Kong BH, Gu J, Kuroda H, Kim TH (2009) Signal processing, image processing and pattern recognition 13. face, Inc. face.com. http://face.com/ 14. facebook, Inc. facebook.com. http://www.facebook.com 15. face.com. face.com. http://face.com/ 16. FFmpeg. http://ffmpeg.org/ 17. Fielding R (2000) Architectural styles and the design of networkbased software architectures, Ph.D. thesis 18. Fielding R, Gettys J, Mogul J, Frystyk H, Masinter L, Leach P, Berners-Lee T (1999) Hypertext transfer protocolhttp/1.1. Technical report, RFC 2616 19. Flores H, Srirama SN, Paniagua C (2011) A generic middleware framework for handling process intensive hybrid cloud services from mobiles. In: The 9th international conference on advances in mobile computing and multimedia (MoMM-2011), pp 8795, ACM 20. Forouzan BA (2005) TCP/IP protocol suite. Forouzan Networking, New York 21. Foursquare. Foursquare. https://foursquare.com/ 22. Freund Y, Schapire R (1996) Experiments with a new boosting algorithm. In: Machine learning-international workshop then conference, Morgan Kaufmann publishers, Massachusetts, pp 148156 23. GoGrid. GoGrid-Complex Infrastructure Made Easy. http://www. gogrid.com/ 24. Google code labs. Android Cloud to Device Messaging Framework. ES/android/c2dm/index.html 25. Gsm Arena Inc (2011) Gsm Arena-HTC Desire. http://www. gsmarena.com/htc_desire-3077.php 26. Hadid A, Heikkila J, Silven O, Pietikainen M (2007) Face and eye detection for person authentication in mobile phones. In: Distributed smart cameras, 2007, ICDSC07m, First ACM/IEEE international conference on, IEEE, pp 101108 27. Han S, Park HA, Cho DH, Park KR, Lee S (2007) Face recognition based on near-infrared light using mobile phone. In: Beliczynski B, Dzielinski A, Iwanowski M, Ribeiro B (eds) Adaptive and natural computing algorithms, Lecture notes in computer science, vol 4432. Springer, Berlin, pp 440448 http://code.google.com/intl/esMcGraw-Hill

13

14

15