Cluster analysis Dr James Abdey

Overview

Applied Marketing (Market Research Methods) Topic 11: Cluster analysis

Dr James Abdey

Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Overview
Like factor analysis, cluster analysis also examines an entire set of interdependent relationships The primary objective of cluster analysis is to classify objects into relatively homogeneous groups based on the variables In essence, cluster analysis is the obverse of factor analysis reducing the number of objects rather than variables Clustering is very useful for dening target markets

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Market segmentation
Recognising customers differences is the key to successful marketing It can lead to a closer matching between products and customer needs Consumers may be clustered on the basis of benets sought from the purchase of a product (benet segmentation) Understanding buying behaviour, for example what kind of strategies do car buyers use for buying a car? Identifying new product opportunities clustering brands and products Selecting test markets

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem

Cluster analysis is a class of techniques used to classify objects or cases into relatively homogeneous groups called clusters Objects in each cluster tend to be similar to each other and dissimilar to objects in the other clusters

Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis
Both cluster analysis and discriminant analysis are concerned with classication However, discriminant analysis requires prior knowledge of the group membership for each object or case included, to develop the classication rule In contrast, in cluster analysis there is no a priori information about the group or cluster membership for any of the objects Groups or clusters are suggested by the data, not dened a priori

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

An ideal clustering solution

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Statistics associated with cluster analysis

Agglomeration schedule An agglomeration schedule gives information on the objects or cases being combined at each stage of a hierarchical clustering process Cluster centroid The cluster centroid is the mean values of the variables for all the cases or objects in a particular cluster Cluster centres The cluster centres are the initial starting points in non-hierarchical clustering. Clusters are built around these centres, or seeds Cluster membership Cluster membership indicates the cluster to which each object or case belongs

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Statistics associated with cluster analysis

Dendrogram
A dendrogram, or tree graph, is a graphical device for displaying clustering results Vertical lines represent clusters that are joined together The position of the line on the scale indicates the distances at which clusters were joined The dendrogram is read from left to right

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Distances between cluster centres

These distances indicate how separated the individual pairs of clusters are Clusters that are widely separated are distinct and therefore desirable

Statistics associated with cluster analysis

Icicle diagram
An icicle diagram is a graphical display of clustering results The columns correspond to the objects being clustered, and the rows correspond to the number of clusters An icicle diagram is read vertically

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Similarity/distance coefcient matrix

A similarity/distance coefcient matrix is a lower triangular matrix containing pairwise distances between objects or cases

Statistics associated with cluster analysis

For four objects, we have a 4 4 distance matrix 12 13 14 21 23 24 31 32 34 41 42 43 where ij is the distance between object i and object j Clearly 12 = 21 , 13 = 31 , etc. The diagonal may be left blank

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Cluster analysis Dr James Abdey

Overview Cluster analysis

Consumers were asked to express their degree of agreement with the following statements Use 1 = disagree, 7 = agree
1. 2. 3. 4. 5. 6. Shopping is fun Shopping is bad for your budget I combine shopping with eating out I try to get the best buys while shopping I dont care about shopping You can save a lot of money by comparing prices

Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Steps to be taken

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis

Construction of distances between pairs of objects (subjectively or based on a formal measure) The development of a routine, or algorithm, for forming clusters on the basis of these distances The unit of analysis is either the distance or dissimilarity between objects or the closeness, proximity or similarity

Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Formulate the problem

Perhaps the most important part of formulating the clustering problem is selecting the variables on which the clustering is based Inclusion of even one or two irrelevant variables may distort an otherwise useful clustering solution Basically, the set of variables selected should describe the similarity between objects in terms that are relevant to the market research problem The variables should be selected based on past research, theory or a consideration of the hypotheses being tested In exploratory research, the researcher should exercise judgement and intuition

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Select a distance or similarity measure

The most commonly used measure of similarity is the Euclidean distance, or its square The Euclidean distance is the square root of the sum of the squared differences in values for each variable Other distance measures are also available The city-block or Manhattan distance between two objects is the sum of the absolute differences in values for each variable The Chebychev distance between two objects is the maximum absolute difference in values for any variable

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Select a distance or similarity measure

The Euclidean distance between objects i and j is
p

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem

ij =
k =1

(xik xjk )2

Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters

For p = 2, the Euclidean distance corresponds to the straight line distance between the two points (xi 1 , xi 2 ) and (xj 1 , xj 2 ) A weight wk could be assigned to variable k if it was believed that more importance should be attached to some variables over others giving
p

Interpreting and proling the clusters Assess reliability and validity Example

ij =
k =1

wk (xik xjk )2

Select a distance or similarity measure

The city-block measure is
p

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure

ij =
k =1

|xik xjk |

Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Compared to Euclidean distance, this gives less relative weight to large differences The Chebychev distance is ij = max |xik xjk |
k

Select a distance or similarity measure

If the variables are measured in vastly different units, the clustering solution will be inuenced by the units of measurement In these cases, before clustering respondents, we must standardise the data by rescaling each variable to have a mean of zero and a standard deviation of one It is also desirable to eliminate outliers (cases with atypical values) Use of different distance measures may lead to different clustering results Hence, it is advisable to use different measures and compare the results

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Select a clustering procedure

Hierarchical clustering subsets of clusters at one level are aggregated to form the clusters at the next, higher, level
Agglomerative clustering we start by treating each object as a one-member cluster, and then proceed in a series of steps to amalgamate clusters Divisive clustering we start at the other end, treating the whole set of individuals as a single cluster and then proceed by splitting up existing clusters

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Agglomerative methods are commonly used in market research

Select a clustering procedure

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis

Non-hierarchical clustering clusters are formed by adjusting the membership of those clusters existing at any stage in the process by moving individuals in or out
Some use multivariate analysis of variance ideas in the sense that they divide the objects into groups such that the between groups variation is large and the within groups variation is small

Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Some agglomerative hierarchical methods: Linkage methods

Nearest neighbour or single linkage based on the minimum distance rule. The distance between two clusters is the distance between their closest points Farthest neighbour or complete linkage based on maximum distance rule. The distance between two clusters is the distance between their fathest points Average linkage distance between two clusters is dened as the average of the distances between all pairs of objects (where one member of the pair is from each of the clusters)

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Some agglomerative hierarchical methods: Variance methods

Aim: generate clusters that minimise the within-cluster variance Wards method at each stage in the clustering process, Wards method considers all pairs of clusters and asks how muich information would be lost if that pair were to be amalgamated For each cluster the means of all the variables are computed, then for each object the squared Euclidean distance to the cluster means is computed and these distances are summed for all the objects

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Some agglomerative hierarchical methods: Variance methods

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem

At each stage the two clusters with the smallest increase in the overall sum of squares within cluster distances are combined The method is designed for the case where the elements of the data matrix are metrical variables since only then is it sensible to compute the sums of squares involved

Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Some agglomerative hierarchical methods: centroid method

In the centroid method, the distance between two clusters is the distance between their centroids (means for all the variables) Every time objects are grouped, a new centroid is computed If the distances are Euclidean (that is, straight line distances between the points), we can measure the distances between two clusters by the distance between their averages (centroids). In so doing, we are treating the distances as metrical Of the hierarchical methods, average linkage and Wards methods have been shown to perform better than the other procedures

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Decide on the number of clusters

Theoretical, conceptual or practical considerations may suggest a certain number of clusters In hierarchical clustering, the distances at which clusters are combined can be used as criteria This information can be obtained from the agglomeration schedule or from the dendrogram The relative sizes of the clusters should be meaningful

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Interpreting and proling the clusters

Cluster analysis Dr James Abdey

Overview Cluster analysis

Interpreting and proling clusters involves examining the cluster centroids The centroids enable us to describe each cluster by assigning it a name or label It is often helpful to prole the clusters in terms of variables that were not used for clustering These may include demographic, psychographic, product usage, media usage or other variables

Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Assess reliability and validity

1. Perform cluster analysis on the same data using different distance measures. Compare the results across measures to determine the stability of the solutions 2. Use different methods of clustering and compare the results 3. Split the data randomly into halves. Perform clustering separately on each half. Compare cluster centroids across the two subsamples 4. Delete variables randomly. Perform clustering based on the reduced set of variables. Compare the results with those obtained by clustering based on the entire set of variables

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Suppose we examine ve customers of a supermarket: Adam, Brian, Carmen, Donna and Eve The entries below represent how far apart individuals are in regard to their potential buying patterns A 3 8 11 10 B 7 9 9 C D E

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

A B C D E

6 7

Cluster analysis: Example

Using the nearest neighbour (single linkage) method, rst we look for the closest pair of individuals who are Adam and Brian We construct a new distance table appropriate for the four clusters existing at the end of the rst stage the distance between two clusters is dened as the distance between their nearest members (A, B) C D E (A, B) 7 9 9 C 6 7 D E

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Cluster analysis Dr James Abdey

Overview Cluster analysis

We repeat the procedure until all objects are in one cluster More specically, the next most similar pair of objects is Donna and Eve (A, B) C (D, E) (A, B) 7 9 C 6 (D, E)

Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Cluster analysis Dr James Abdey

Overview Cluster analysis

The agglomeration schedule is: Stage Initial 1 2 3 4 Number of clusters 5 4 3 2 1 Clusters (A) (B) (C) (D) (E) (A, B) (C) (D) (E) (A, B) (C) (D, E) (A, B) (C, D, E) (A, B, C, D, E) Distance level 0 3 5 6 7

Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Using the farthest neighbour (complete linkage) method, rst we look for the most remote pair of individuals The agglomeration schedule is: Stage Initial 1 2 3 4 Number of clusters 5 4 3 2 1 Clusters (A) (B) (C) (D) (E) (A, B) (C) (D) (E) (A, B) (C) (D, E) (A, B) (C, D, E) (A, B, C, D, E) Distance level 0 3 5 7 11

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Cluster analysis Dr James Abdey

Overview

The sets of clusters produced by the farthest neighbour method coincide with those from the nearest neighbour method, although the distance levels differ at which the clusters merge Generally, the nearest and farthest neighbour methods give different results, sometimes very different, especially when there are many objects or individuals to be clustered Both methods only depend on the ordinal properties of the distances

Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Dendrogram for nearest neighbour (single linkage) clustering

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example

Cluster analysis: Example

Dendrogram for farthest neighbour (complete linkage) clustering

Cluster analysis Dr James Abdey

Overview Cluster analysis Statistics associated with cluster analysis Example Formulate the problem Select a distance or similarity measure Select a clustering procedure Decide on the number of clusters Interpreting and proling the clusters Assess reliability and validity Example