Assignment 2 - Multi-biometrics

Introduction to Biometrics Course Berk Gokberk (b.gokberk@utwente.nl) Signals and Systems Group University of Twente

In this assignment, you're going to design a multi-biometrics system. There will be two approaches: Multi-modal Multi-presentation

In the multi-modal system, face and fingerprint modalities will be fused. In the multi-presentation system, multiple samples from a single modality will be combined such as two facial images will be compared to enrolled images. For both of these systems, score-level fusion techniques will be applied. The scores of face and fingerprint recognition systems will be provided to you in the form of similarity matrices (SM). A similarity matrix contains computed scores of all possible combinations of biometric comparisons. SM is a square matrix and each matrix entry holds the comparison score of two biometric templates. A sample SM for four persons, each having three biometric templates, is shown in Figure 1. SM is symmetric along the diagonal and the diagonal entries do not contain any value, since its not meaningful to compare the same biometric templates. A similarity matrix might contain similarity or dissimilarity values. The sample SM in Figure 1 contains similarity values as genuine comparisons (red entries) have higher values. The SM matrix should be accompanied by a class label information (person ID information) which states the ID of the person for each row and column. Row and column orders are the same, such as [1 1 1 2 2 2 3 3 3 4 4 4] in the sample SM. For each similarity matrix entry, you can estimate whether the specified matrix entry holds a genuine or an impostor score by looking at the corresponding information in the class labels. Given an SM matrix, you can compute the performance of a verification system (FAR, FRR, and equal error rates-EER) and plot the ROC and DET curves.

Figure 1. A sample similarity matrix.

1. Multi-modal: Given two similarity matrices for face (SM1) and fingerprint recognition (SM2) systems, you're going to fuse the similarity scores using SUM and PRODUCT fusion rules. Sum rule simply combines two scores by S_combined = S1 + S2. Similarly, product rule obtains the combined score by multiplication, S_combined = S1 x S2. For both of these methods, three score normalization techniques will be implemented: 1. Using original scores (no score normalization) 2. Min-max score normalization 3. Z-score normalization. Refer to lecture slides for fusion rules and score normalization techniques. Given SM1 and SM2, you should compute fused similarity scores and will output equal error rates (EER) for all different fusion rules and score normalization methods. See the multi-modal section of Table 1 for a list of methods. Additionally, you'll plot twodimensional scatter plot of similarity scores for face and fingerprint recognition systems. There will be two types of points in scatter plot: genuine and impostor cases. See Figure 4 for a sample scatter plot. For the multi-modal system, each subject is enrolled by one face and one fingerprint sample. During verification, we assume that the test subject will provide one face and one fingerprint image. This situation is illustrated in Figure 2.

Enrollment Sample Face Score

Verification Sample

Fusion (Sum/Product)

Fingerprint Score

Figure 2. Multi-modal fusion architecture 2. Multi-presentation: In multi-presentation scheme, we assume that each person is represented by two biometric samples during enrollment. For instance, if face modality is used, each person will have two face images as enrollment templates. During verification, test subject will provide two biometric samples, i.e., face images. This situation is illustrated in Figure 3. Given a pair of enrollment samples and a pair of verification samples, you're going to compute four similarity scores, as shown in Figure 3. You will combine these four scores in two ways: 1. Mean: by taking their mean 2. Maximum: by taking the maximum similarity score In your report, you're going to report EERs obtained from face and fingerprint modalities using these two score combination methods. See multi-presentation part of Table 1 for an example. Enrollment Samples S1 S4 S3 S2 Figure 3. Multi-presentation fusion architecture Verification Samples

In multi-presentation approach, you will use the two similarity matrices for face and fingerprint systems as in the multi-modal approach. Each of these matrices contains comparison scores of 40 subjects, having eight samples. Thus, the size of SM is 320 x 320. In multi-presentation method, first two samples of every person will be used as enrollment set. Remaining six images per person will be used as test image pairs. You will select every possible pairing of size two out of six test samples to form your test image pairs. For example, let (F1, F2, F3, ... , F8) represent eight face images of a person. Enrollment pair will always be (F1, F2) and the test image pairs will be:

{(F3, F4), (F3, F5), (F3, F6), (F3, F7), (F3, F8), (F4, F5), (F4, F6), (F4, F7), (F4, F8), (F5, F6), (F5, F7), (F5, F8), (F6, F7), (F6, F8), (F7, F8)}. In total, there are 15 possible combinations for test pairs. Per person, the number of genuine comparisons is therefore 15: compare every test pair to the persons enrolment pair. Total number of genuine comparisons is 40 (persons) x 15 = 600. The same pairing system will be used to obtain impostor scores: each test pair of a person will be compared to the enrolment pair of all other persons. The total number of impostor comparisons will be 40 x 15 x 39 = 23400. Note that these numbers are different for the multi-modal system since a single sample will be used during enrolment. In your report, you will provide a comparison of four methods using their ROC curves. These four methods will be: 1. 2. 3. 4. individual face modality performance individual fingerprint modality performance best multi-modal method performance best multi-presentation method performance

Figure 5 shows sample ROC curves for such a comparison. A sample report is provided at the end of this document. In addition to scatter plot, performance table (Table 1 results) and ROC curves, you should provide your comments on your findings, i.e., why some of the methods performed better? why a particular method performs poorly etc. You'll deliver your report and Matlab code which outputs: 1. Scatter plot 2. EERs of 12 methods (Table 1) 3. ROC plots (Figure 5)

Note that your code will be tested with different similarity matrices. Test similarity matrices will have the same structure: 40 subjects and eight samples per subject.

Sample results to be provided in report

1. Scatter plot of face and fingerprint scores for genuine and impostor comparisons.

Figure 4. Scatter plot of face and fingerprint scores. Scores are not normalized.

2. Provide EERs for each of the methods as presented in Table 1. Table 1. Comparison of multi-biometric methods.
Method Face Modality Fingerprint Modality Face+Fingerprint (Sum rule, No normalization) Face+Fingerprint (Product rule, No normalization) Face+Fingerprint (Sum rule, Minmax normalization) Face+Fingerprint (Product rule, Minmax normalization) Face+Fingerprint (Sum rule, Z-score normalization) Face+Fingerprint (Product rule, Z-score normalization) Face+Face (Mean) Fingerprint+Fingerprint (Mean) Face+Face (Maximum) Fingerprint+Fingerprint (Maximum) Equal Error Rate (%) 12.05 21.70 8.30 14.24 8.66 8.30 7.92 19.29 6.15 6.50 5.50 6.17

Individual Modalities

Multi-Modal

Multi-presentation

3. Plot ROC curves for 1) face, 2) fingerprint, 3) best multi-modal approach, and 4) best multi-presentation approach in one figure. There should be only four ROC curves. Use legend Matlab command to write method names in the figure (See Figure 5).

Figure 5. ROC curves for face, fingerprint, best multi-modal and best multi-presentation approach.