Journal of Information Processing

FULL BODY GESTURE RECOGNITION METHOD FOR INDONESIAN TRADITIONAL (PAKARENA) DANCE MOVEMENTS Ery Muchyar Hasiri 1, Hamdan Gani 2, Billy Eden William Asrul 3, Mardiah 4 , Musfirah Putri Lukman 5 Abstract: Traditional Pakarena dance is one of interesting research materials to identify the object movement in the field of computer vision. Therefore, traditional dance can always be a good future material of learning and development in the field of computer vision, especially in pattern recognition of a regional dance moves. Basically, the dance moves are similar with every movement in a motion such as in the computer game application, in which movement in dance moves of the hand, foot, curves, head, etc. can be applied into a computer vision to perform recognition on dance movements. The interesting problem arises in how to know the movement patterns of the dancers and how to classify the dance pattern that it can both be used for dance development through the application of computer vision. To solving this problems, we propose a method used in this study that can be explained sequentially as follows; kinect sensor conducts the dancers gesture recognitions, then kinect sdk perform separation using Skeletonize, and SVM, HMM to classified and clustering of further creating a dataflow dancer from the dance moves. Thus this paper results a concept that combines the techniques of Skeletonizing based on the Kinect software development kit with SVM (Support Vector Machine) and HMM (Hidden Markov Model) . The method proposed in this paper can facilitate the implementation of image processing and pattern recognition algorithms such as pattern formation skeleton movement to detect any movement of the Pakarena dancer. Keywords : Computer Vision, Kinect, Recognition, Skeleton, Support Vector Machine 1. Introduction So also in a dance in which the pattern of movement that became a topic of learning to be done on computer vision. A dance on Pakarena are unique in that movement forward, backward, body movement, hands, feet and head etc. Which is a problem that is how to do an introduction on each movement of the dancers, because each movement with the body position at the time to dance is a relatively fast movement due to the rhythm of the dance accompaniment sound. And subsequent search for a correlation of image matching based object classification using SVM (Super Vector Machine) [9],[2]. This method is used as the basis for learning the movement patterns are rapidly changing. Of variation Pakarena dance movement that became the material for further research to produce a solution to the introduction of the movement will always be ongoing every generation. In this study, we put more emphasis on the use of a combination of techniques and algorithms skeletonisasi SVM (Support Vector Machine)[9],[2] to introduce each movement and the dancers then performed clustering based on pattern matching with its skeleton. 2. Tari Pakarena Pakarena dance is a dance that originated from Gowa , South Sulawesi. Pakarena word itself comes from the local language as the means play. This dance is a tradition among the community of Gowa who still maintained today. Pakarena dance comes from Gowa, Makassar, South Sulawesi .Pakarena dance often danced the Kingdom of Gowa family relatives as a form of love Sultan Hasanuddin ( King of Gowa to XVI) in this dance. Pakarena dance expression reflects gentleness, politeness, loyalty, obedience and respect women to men. Each pattern of movement in dance Pakarena has significance.

Traditional folk dances of the ethnicity and tribes is an important part of cultural property, especially in Indonesian dance is a hallmark of a nation that became a symbol and introducing the diversity to international countries. Traditional Pakarena dance[1] is the result of the art creation from ancient predecessors in terms of creating a dance with unique movement to represent the characteristics of a culture and philosophy of life of the people of a region. Pakarena expresses the elegance of the Makassar ethnic women symbolizing their politeness, loyalty, obedience and respect towards their spouses. The dance consists of 12 parts that have their own distinct meanings although they are difficult to differentiate by untrained eyes, since patterns appear to be similar. Both the seated position which commences and ends the performance, as well as the clock wise movement represent the cycle of human life. While the ascending and descending movements symbolize the wheel of life; sometimes we are at the top, while at other times we are down. In all choreographies, the dancers hold and play the distinct traditional fan. The body gesture recognition in computer vision is still under further development, but the difference of each development is the technology used for example in the design of computer games where the first is to record every movement patterns of players the ball and then make a frame with motion or movement in did the game.
1 Dayanu Ikhsanuddin University 2 AMIK Rizky Makassar 3 STMIK Handayani Makassar 4 STMIK Tidore 5 STMIK Dipanegara

2013 Information Processing Society of Japan

Journal of Information Processing

The dance begins and ends with a sitting position as a sign of respect and courtesy of the dancers. Movement patterns play a meaningful human life cycle that is always spinning. The twist pattern is played clockwise circular motion. Then the patterns of movement up and down symbolizes human life, sometimes under and sometimes above, the movement pattern is reminiscent of the importance of patience and human consciousness in facing life. Pakarena female dancers should numbering four to seven people. Why women ? Because dance is basically reflects the character of the woman in Gowa. gentle, polite , loyal , and obedient. 3. Related Works By using a Kinect camera[2],[11][10], as the recording medium every motion of the dancers movement and be made in the form of a static image skeleton pattern and to conduct training of the pattern matching can use the technique of Support Vector Machine as an important part of measuring the computing time of a feature vector and image then used to classify the learning of the previous image[2]. 3.1 Movements in Pakarena Dance Pakarena gesture of traditional dance. In the dance movements Pakarena actually there are 12 but we only sampled 3 main motion as material for the process of matching the sample image that made its match in the system. Pakarena dance movements are divided into movements: 1. Movements in a sitting position, a sign of the beginning and end of the dance Pakarena. 2. Rotary motion clockwise. Shows the human life cycle. 3. While the up and down movement, the rhythm of life is like a mirror. Table1. The Dance Gesture Classes of Pakarena Dance Dance Gesture Name No. 1 2 3 Gesture posisi duduk Gesture naik turun Gesture berputar Fig 1. Skeletal representation [3] 2.2. Support Vector Machine (SVM) Support Vector Machines[9][2] is one of the most widely used supervised learning algorithms[2], SVM is a binary classifier i.e. it separates only 2 classes, however multiple classes can be classified by using multiple SVMs. SVM is a maximal margin classifier i.e. It finds a hyper plane (decision boundary) which separate the 2 classes by maximising the distance error margin form the 2 classes. is used to separate the 2 classes. If multiple such hyperplanes are possible, it detects the one whose distance or margin from the 2 classes is maximal. Firstly it assumes that the 2 classes are linearly seperable. Support Vectors : Objects which are closer to the decision boundary on its either side are called Support Vectors. Optimal decision boundary and margin 1. Silhouettes based approach This methods extract features from each silhouette and model the dynamics of the action explicitly, often using statistical models such as Hidden Markov models(HMM), graphical models (GM) and CRF[2]. Some of the latest silhouette based approaches are as discussed below : Action Recognition Based on Bag of 3D Points[14] This paper presents a method to recognize human actions from sequences of depth maps. STOP: Space-Time Occupancy Patterns for 3D Action Recognition[15] This paper presents Space-Time Occupancy Patterns (STOP), a new visual representation for 3D action recognition from sequences of depth maps. 2. Skeleton Based Approach Skeletons can be obtained from motion capture systems using body joint markers or directly tracked from depth maps. However, tracking of body joints from depth maps is not a completely-solved problem. For example, the joint positions returned by Kinect skeleton tracker still contains noise[2]. A few techniques which uses joint angles are as given below : Effective 3D Action Recognition Using Eigen Joints[16] Designed a new action feature descriptor for action recognition based on differences of skeleton joints,first they compute joint dfferences, then apply Principal Component Analysis (PCA) to the joint differences to obtain Eigen Joints by reducing redundancy and noise. Mining actionlet ensemble for action recognition with depth cameras[17] In this paper, an actionlet ensemble model is learnt to represent each action and to capture the intraclass variance

3.1. Skeleton Technique Representation Research techniques skeleton[6],[8] is still in research phase in the world of computer vision, because this technique can represent the pattern of body movement detected by the depth sensor camera. In this paper to detect the body movement patterns with capture using the Kinect camera to detect the depth of skeleton joints using techniques and separated them into static images. At research we can use two approach method that is :
2013 Information Processing Society of Japan

Journal of Information Processing

depends only on these support vectors. Given a set A of vectors xi with its corresponding class yi (+1, -1) the objective is to identify separating hyperplane w.x + b = 0, such that for every object, yi * (w.xi + b) comes out to be >= 1. SVM tries to maximize the margin which is equivalent to minimizing. 2.3 Hidden Markov Model A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved (hidden) states[18].A markov process is defined as a stochastic process (described by a set of states) satisfying Markov property. Markov property states that the probability of each subsequent state depends only on the previous state and no other state. In simpler Markov models (like a Markov chain), the state is directly visible to the observer, and therefore the state transition probabilities are the only parameters. In a Hidden Markov model, the states are not directly visible, but output, dependent on the state, is visible. Each state has a probability distribution over the possible output tokens. Therefore the sequence of tokens generated by an HMM gives some information about the sequence of states[2]. 3. Data Collection Dance gesture[3], we get from the Kinect camera to capture depth sensor to produce RGB Image, we collected a total of 9 samples of the main motion but we represent on table.1, starting from the beginning of the movement in the seated position, the initial movement of the dance (a). Then move up and down as the second movement (b) and the spinning motion as the last movement before Dance Pakarena closed. Training action classification of dance movements, and second phase is testing and recognition of dance movements.

Fig.3. Training Phase Can be seen from the picture how the system works and can be described into stages: In this stage kinect camera will process the image captured earlier (Movement Dance dancers Pakarena) with joint angle method or silhouettes approach method and convert them into Skeleton Then we perform the skeleton With Support Vector Machine method (SVM) and Hidden Markov Model (HMM), to classification Action Dance movement. 2. Phase Testing In this phase training data from previous training phase, to be a referenced learning using SVM and HMM systems to produce output dance recognition.

Fig.4. Testing Phase 3.4 Feature Extraction Feature extraction[13] is the extraction stage or take a skeleton image which has been produced by the kinect camera and then we enter into the next stage of the classification phase to match image we test whether the pattern image is match or not (in this case to recognize the dance movements Pakarena). 3.5 Classification At this stage of our classification using Support Vector Machine and Hidden Markov Model method [9][2]. This method has two main types of data , the training data and test data . The first stage we perform classification and training of the main movements that we have chosen from the Pakarena dance then we enter into the class to be tested (Image Movement posing Gerakan Duduk) then (image skeleton posing gerakan Naik Turun) and (Image skeleton posisi Gerakan Berputar ) . Stage two is a testing phase in which we will test example, we use (Image skeleton posing gerakan duduk ) that we

Fig.2. Tranformation Video Image to Skeleton

Images obtained by the Kinect camera and then processed directly and generate skeleton. It can be seen how we took some samples of the images (a) (b) (c) and then generate skeleton image. 4. Methodology In this paper we describe a framework method that we can use to perform a recognition of Pakarena traditional dance movements with two phase, first is phase of Training and second phase is testing. 1. Phase Training

2013 Information Processing Society of Japan

Journal of Information Processing

put in our SVM system will test the newly acquired image samples . The third stage through a process of SVM training data in each training data contains one target value ,ex. class (normal pose skeleton Gerakan Duduk ) were tested with a new input image is produced or not match the image . The goal of SVM and HMM is to produce a model that can predict the target values of the test data is only given attribute values. Robert Matthew Wham, Three-Dimensional Kinematic Analysis Using the Xbox Kinect University of Tennessee Honors Thesis Projects, 2012. 9) Tristan Fletcher,Support Vector Machines Explained www.cs.ucl.ac.uk/sta_/T.Fletcher/, 2009. 10) Marquardt, Zoe, et al. Super Mirror: a kinect interface for ballet dancers.Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems ExtendedAbstracts. ACM, 2012. 11) J. MacCormick, Talk: How does the kinect work, http://users.dickinson.edu/ jmac/selected-talks/,2011. 12) Christopher J.C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition Data Mining and Knowledge Discovery 2, 121-167, 1998 13) Joost van de Weijer, Cordelia Schmid, Coloring Local Feature Extraction Lecture Notes in Computer Science Volume 3952, 2006, pp 334-348, Computer Vision ECCV 2006 14) W Li, Z Zhang, Z Liu, Action Recognition Based on A Bag of 3D Points IEEE International Workshop on CVPR for Human Communicative Behavior Analysis in conjunction with CVPR 2010. 15) A. Vieira et al. STOP: Space-Time Occupancy Patterns for 3D Action Recognition from Depth Map Sequences. LNCS 2012 16) X. Yang and Y. Tian. Eigen joints based action recognition using nave bayes nearest Neighbor, Computer Vision and Pattern Recognition, 2012. 17) J. Wang et al. Mining actionlet ensemble for action recognition with depth cameras. CVPR 2012 18) Rabiner, Lawrence, and B. Juang. An introduction to hidden Markov models.ASSP Magazine,IEEE 3.1 4-16, 1986. 8)

4. Discussion Dance gesture, Gesture Recognition on the kinect camera technology have made use of the method and the method skeletonisation which has resulted in a fairly good image to be processed, the ability kinect camera with image reading process that can produce a depth image quality , which is pretty good skeletonisation, many researchers choose to make the rest of the image generated by utilizing kinect , so they just add recognition method. Refference : 1) Ni Luh Sulistiawati, Kontribusi Seni Tari Nusantara Dalam Membangun Pendidikan Multikultural. ISSN,0854-3461, Volume 26, Nomor 2, Juli 2011. Jurusan Ilmu Seni Tari Fakultas Seni Tari Pertunjukan, Institut Seni Indonesia Denpasar. 2) Apratim Sharma, Recognising Bharatanatyam Dance Sequences using RGB-D Data Indian Institute of Technology Kanpur. 3) Raptis, M., D. Kirovski, H. Hoppe. Real-Time Classification of Dance Gestures from Skeleton Animation, Symposium on Computer Animation, pp. 147-156, 2011. 4) Yaya Heryadi, Mohamad Ivan Fanany and Aniati Murni Arymurthy : Grammar of Dance Gesture from Bali Traditional Dance. IJCSI International Journal of Computer Science Issues, Vol. 9, Issue 6, No 1, November 2012 5) Henning Pohl, Aristotelis Hadjakos, Dance Pattern Recognition Using Dynamic Time Warping Proceedings of the 7th Sound and Music Computing Conference (SMC 2010) Telecooperation Technische Universitat Darmstadt Darmstadt, Germany 6) Khalid Saeed, Marek Tabedzki, Mariusz Rybnik, Marcin Adamski K3M: A Universal Algorithm For Image Skeletonization And A Review Of Thinning Techniques Int. J. Appl. Math. Comput. Sci., 2010, Vol. 20, No. 2, 317335. 7) Cicu Ratih Damayanthi, Dr. Dewi Agushinta R. Segmentation of Shape and Skeleton Human Body With Use Gaussian And Thinning Method Undergraduate Program, Faculty of Industrial Technology, 2009.

2013 Information Processing Society of Japan