Human communication is a combination of both verbal and nonverbal interactions.

Through facial expressions, body gestures and other non-verbal cues, a human can communicate with others. It has been well researched that facial expressions do reflect cognitive behavior, and that individuals observe others facial expressions and then use these to regulate their own behavior in social interactions. This is especially true in the communications of emotions. Facial expressions and related changes in facial patterns give us information about the emotional state of the person and help to regulate conversations with the person. Moreover, these expressions help in understanding the overall mood of the person in a better way. Facial expressions play an important role in human interactions and nonverbal communication. Classification of facial expressions could be used as an effective tool in behavioral studies and in medical rehabilitation. Facial expression analysis deals with visually recognizing and analyzing different facial motions and facial feature changes. Although there are various studies on the applications of computers in areas related to face and facial expression recognition, todays computers do not make use of the nonverbal cues common in human communications. As internet chatting becomes increasingly popular, people will not be satisfied with just text or voice communication. Since facial expression plays an essential role in natural communication between people, visual chatting, an on-line communication framework incorporating facial expression, will be more effective than traditional systems. Unfortunately, direct transfer of video of the user is not practical for the limited bandwidth of networks. This motivates the use of a virtual character imitating the users behavior, or avatar, which is desirable also because it can protect the users privacy, as well as produce impressive visual effects. The objective of this project is to propose an automatic expression recognition system that can be integrated into an online text-based communication to control the facial expressions of avatars (or agents). The traditional text-based chat message is accompanied by real-time facial expression/gesture information, which is sent over to the chat party on the Internet and performed by the agent representing the sender. The use of the system can be demonstrated with the following example. Imagine that you type an ambiguous joke and to make yourself clear you have to insert a ":-)" (smiley symbol) at the end of the sentence. If the receiver can see the senders real-time facial expressions synthesized by the agent, the smiley symbol can be substituted or emphasized with a smiling gesture when finished typing, so instead of character symbols you can use your natural facial gestures to communicate additional information. The sender virtually shows the agent how to behave while delivering the message.