Академический Документы
Профессиональный Документы
Культура Документы
Introduction
How do you make a computer learn? The set of training examples is called the training set
Possibilities: Questions:
I Give it rules (some AI approaches, expert systems, etc.) I How large / how many?
I Give it lots of examples (pattern recognition, machine learning, I Source?
neural networks) I Generality?
I How good?
Giving it lots of examples is called training
Some systems may do on-the-fly (online) training if there is some Unsupervised training: Give the system example patterns
way to get feedback and let if figure out natural groupings
I After classification, let it know whether it got it right or
wronglearn for the next time
I Example: OCR/proofreading
Key ideas:
I Patterns from the same class should cluster together in feature
space
I Supervised training: learn the properties of the cluster
(distribution) for each class
I Unsupervised training: find the clusters from scratch
Motivation:
Goal: find natural groupings of patterns
I Minimum-distance classifiers assign patterns to the nearest
prototype
I Useful when you dont have a pre-labeled training set
I Each classs prototype should be at the mean of the classs
I More closely models neural organization training patterns
I Cant So...
I classify with labels either (since they werent learned) I Assign patterns to nearest prototype
I handle complicated distributions
I Update prototype to be the mean of the patterns assigned to it
I Clustering
I Repeat until convergence
k -means k -means
Requires:
I number of classes k Things to consider:
I minimum-distance classification I How do you know the number of classes?
I How do you seed the initial prototypes?
Algorithm:
I Zero-element clusters
Start with initial guesses at class prototypes (means) (jump to arbitrary new prototype and restart?)
Repeat I How good is the final clustering?
Assign each pattern to the nearest prototype mi (juggle, restart, and see if better)
Update each clusters prototype mi to be
the mean of the patterns assigned to it I Retry with more/fewer clusters?
until convergence or maximum number of iterations
Introduction to Pattern Recognition Introduction to Pattern Recognition
Unsupervised Training Unsupervised Training
I Hierarchical (merging) Idea: for each possible set of parameters for the distributions, how
I Bottom-up merging of clusters until good enough well does the weighted sum (i.e., mixture) of their distributions match
the histogram of the training set?
I Cluster Swapping
I Moving of patterns from one cluster to another if nearer I Strength: handles all parameters and distributions
(like k-means), usually integrated into splitting or merging
I Weakness: complicated and not always solvable
approaches
I Mixture Modelling