Course Information

Machine Learning and pattern recognition (April/2015)

Instructor:
Dr. Musab Elkheir Salih
Dept. of Biomedical Engineering
Sudan University of Science and Technology
Phone: 0129065150

Email: 2yabaabobaba13@gmail.com

Main textbook:
Chris. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
Continue
Objectives:

• Provide an introduction to the main concepts underlying

machine learning and pattern recognition.

• Present several key machine learning and pattern recognition

algorithms.

• Try to make a balance between the understanding of

essentials behind the methods (mathematical) and practice
aspects (Matlab).
Continue
Pre-request:
Knowledge of:
• Statistics & Probability
• Linear Algebra:
• Programming: Matlab

Grading:
• Classwork: 10% short quizzes testing the understanding of basic
concepts from previous lectures
• Homework: 40% (weekly)
• Final project: 20%, Written report + Oral presentation (weekly). *
As scientific format Team work groups
• Exam:30%
 Lecture 1 outline

 Why learn

 Why machine learning and pattern recognition ?

 What is Machine Learning and pattern recognition ?

 What are the main topics ?

 What are the typical application?

why ‘learn’
 Text processing
Machine Learning Document Classification
System
 The big picture
 in recent years, Pattern recognition is the most important development in
computer science and it has large impact on many disciplines

 Machine learning perform the learning tasks and are concerned with the
computer efficiency. By using training data, then the computer infer the rules
itself and make the correct predictions

 In machine learning, one of the approaches to deal with data is to learn the
representation of the most informative structures of the data matrix

 Various of application you can do with machine learning:

• Industry: quality control
• Biometric: fingerprint
• Medicine: diagnosis
• Data mining
• Astronomy
• Diagnosing tumors, playing video games, detecting credit card fraud, recognizing
faces, reading handwriting they don't seem like similar tasks, but they are all cases
where 'machine learning' is employed to enable computers to make intelligent
decisions.

• When computers fail to do something we find easy reading handwriting,

recognizing faces its tempting to think of them as stupid machines. But its often
the case that tasks we find relatively easy to perform evade explicit codification.
How, for example, would you specify rules which correctly identified cats and only
cats including three-legged cats but excluded dogs?
 The major focus of machine learning

- To identify patterns that are hidden by complexity

- To see the information even in huge data sets
- To support intelligent decisions

• Machine Learning will let you know how to perform

accurately even on new, unseen data.

• Today many hundred machine learning algorithms exist

Why

machine learning and

pattern recognition ?
 big data problem:
volume/fast/variety/veracity
 big data problem
• Millions or billions of data records: weather forecasting : to safe our life
estimate many events such as temperature of earth and atmosphere (8 layers)
, humidity, wind direction, where low and high pressures h the use of lightning
climatology. use atmospheric modeling to make predictions If this toddy then
this tomorrow
•
Do you want to make intelligent decisions, predictions and recommendations
based on your data?
•
Do you want take the advantages of the computer program to do this
automatically?

• Human genome is become data base of un thinkable scale. How to come with
the theory of how biology works and make sense of the data automatically

• Computers are cheap, human are expensive

• Your genome, every human's genome, consists of a unique DNA sequence of A's,
T's, C's and G's that tell your cells how to operate.

• Genome: all the genes that make an organism. Huge collection of genes inside
each of your cell

• So how you are in large part is because of you genome

 Human Genome sequencing

• The goal of genome sequencing is to know the sequence of letter that

make up your own personal genome all in a matter of minutes. The
individual letters are 8-10 atoms wide and they are all packed together
into a clump like ball of yarn. To get all information from that tiny space,
scientist have to break the long string into smaller pieces . Each of these
pieces is then sperated in space and sequenced individually
• DNA bind to other DNA, IF THE SEQUENCES ARE EXACT opposite of each other .
• A’s bind to T’s
• G’s bind to C’s
• Ifthe A-T-G-C sequence of two pieces of DNA are exact opposites, they stick
together
• because the genome pieces are so very small we need some way to increase the
signal we can detect from each of the individual letter
• Scientist use enzymes to make 1000,s of copies of each of genome piece. We have
read them all somehow. Make a battle of special letters, each of a distinct color
• The sequence of these millions of pieces of DNA ar stitched together using a
computer program tocreate a complete sequence of the entire genome.
• This is the only to read the letter sequences of pieces of DNA
Tasks beyond human capabilities

 Tasks beyond human capabilities (complex & large data sets):

Astronomical data, Medical archives, and Web search engines

 The ability to process huge databases allows machine learning programs to detect
patterns that are outside the scope of human perception.
 Complex task

 Tasks performed by humans: Driving , Speech, fingerprint,

Handwritten recognition.

 Human programmer cannot provide an explicit instructions of how

such tasks should be executed.

 Rigidity of the programs, however tasks such as handwritings or

speeches change over time or from one user to another.

 Machine learning tools offer a solution to rigidity of the programs

issues: speech recognition machine improves its performance after
hearing several speeches.
Relations to AI (Artificial Intelligence),

• Machine learning can be viewed as a branch of AI (Artificial

Intelligence), since detect meaningful patterns in complex
data is a corner stone of human (and animal) intelligence.

• In contrast with traditional AI, machine learning is not

trying to build automated imitation of intelligent behavior,
but rather to use the strengths and special abilities of
computers to complement human intelligence, often
performing tasks that fall way beyond human capabilities.
What is

machine learning and

pattern recognition ?
What is learning

 Zoologists and psychologists study learning in animals

and humans.

 Learning is to gain knowledge or understanding of or

skill in by study instruction or experience and
modification of a behavioral tendency by experience.

 Many techniques in machine learning derive from the

theories of animal and human learning.

 In biology rats learning to avoid poisonous food.

Definitions
• Arthur Samuel:
Machine learning: field of study that gives computers the
ability to learn without being explicitly programmed
• checkers program, learn to recognize patterns which
led to wins and patterns which led to losses.

• Tom Mitchell:
A computer program is set to learn from an experience E
with respect to some task T and some performance
measure P if its performance on T as measured by P
improves with experience E.
• What are

the main tasks we will look at?

Main type of Machine Learning
Supervised learning
• The known class labels are available

Unsupervised learning or clustering:

• The known class labels are not available

 Reinforcement learning
• Make a sequence of decisions over time, using
reward function
The main topics are listed below
 Supervised learning: The data had the right answers
• Regression
• Classification

 Unsupervised learning
• Clustering (k-means)

Learning Representation
• Dimension reduction (SVD)
• Higher dimensional representation (Kernel methods)
Regression

 Regression sort of refers to the fact that the variable you're

trying to predict is a continuous value.
 Goal is, given a training set, to learn a function h : X → Y so
that h(x) is a “good” predictor for the corresponding value of y
Continue
Example:
• To get a car to drive itself
• Steering directions The vehicle is trying to predict
a continuous value steering directions, that keeps
the car on the road

• Suppose we have a dataset giving the living areas

and prices of many houses Given data like this,
how can we learn to predict the prices of other
houses
Classification
• This is just like the regression problem, except
that the variable you're trying to predict is
discrete rather than continuous.

• If you have a new data and you wanted the

algorithm to just produce more of the same.
 a straight line classifier

• In decision line separates the two classes.

• The New pattern (∗) is more likely to belong to class A than
class B.
• If it is not correct, a misclassification has occurred.
Ex(1) Medical image classification

(a) (b)
Examples of image regions corresponding to (a) class A and (b) class B

Based on the prediction, maybe the patient lives or dies.

Ex(2): Industry
 SVM
• support vector machines — actually takes
data and maps data to an infinite dimensional
space and then does classification using not
two features like what you 've shown, but an
infinite number of features.
Clustering
• The goal is to unravel the underlying similarities and
cluster “similar” objects together.

• Clustering algorithms are actually routinely used for

many different problems things like:

• Genes clustering
• Medical imaging
• Remote sensing
• Social network analysis
• Market segmentation
Clustering: Remote sensing
• Points that correspond to the same ground
cover type, such as water.
 Learning Representation
 In machine learning, one of the approaches
to deal with data is to learn the representation
of the most informative structures of the data
matrix

• Lower dimensional representation

• Higher-dimensional representation
Dimension reduction

 Assuming every data points has many attributes which make it

very hard to understand what going in this data.

 This representation is very useful in many application:

• face recognition algorithm

• Image compression
• Visualization
Face recognition

• size 200×320
• number of bytes we need: 200×320= 640000 bytes
• Using we need: SVD 20×200+20+20×320=1024 bytes
 Visualization

• By representing the data points as 2D features to using SVD,

it will be possible to plot these points in the feature 1,
feature2 coordinate
Higher dimensional representation
 The kernel method is one of the promising methods, which
have attracted significant attention in the area of the machine
learning
 Application:

• Kernel-based SVM
• Super-resolution image
kernel based SVM classifier

Applying mapping into higher dimensional feature space using the kernel
trick is a way to construct a separating hyperplane there
Reinforcement learning

Train a dog. So every time your dog does

something good/ bad you say, "Good dog,“/ "Bad
dog,“. Hopefully, over time, your dog will learn to
do the right things.

Reinforcement learning is applied in robotics:

• How do you write a program to:

 Make helicopter
 Teach a car how to drive
• Application
Application
Pattern recognition is in the center of a number of application areas, including:

• Computer-aided diagnosis: Assisting doctors in making diagnostic

decisions
• Speech recognition: Entering information by talking. This can enhance our
ability to communicate with deaf people, and help handicapped people to
control machines by talking to them.
• Character recognition (machine reading of bank checks, pen-computers)
• Data mining (retrieving information and turning it into knowledge:
Specific gene-sequence patterns are related to particular diseases and
play an important role in medicine.
• Biometrics: fingerprint identification
• Face recognition: Digital cameras learn to detect faces
• Industry using a lot of cameras to see if the wok is done or not