Example as animated in Binary Classification Video 1

Bombers in red, Seagulls in black

Part 1
Score False Positive Rate
0.00
97 0.06
93 0.06
90 0.12
86 0.18
83 0.18
80 0.24
77 0.29
75 0.29
74 0.35
70 0.41
66 0.47
62 0.53
60 0.59
55 0.65
50 0.71
44 0.76
39 0.82
33 0.88
20 0.94
11 1.00

Part 3
Binary Outcomes
Score "+" = 1, "-" = 0

97 0
 93 1
90 0
86 0
83 1
80 0
77 0
75 1
74 0
70 0
66 0
62 0
60 0
55 0
50 0
44 0
39 0
33 0
20 0
11 0
total "+" - Bombers
3

Copyright Daniel Egger/ Attribution 4.0 International (CC BY 4.0)

 Part 2
True Positive Rate Calculating the Area Under the C
0.00 x-axis distance
0.00 0.06
0.33 0.00
0.33 0.06
0.33 0.06
0.67 0.00
0.67 0.06
0.67 0.06
1.00 0.00
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06
1.00 0.06

binary outcomes - 0,1 Total at threshold

reversed for ease of calculation False Positives True Positives
0 0
1 1 0
 0 1 1
1 2 1
1 3 1
0 3 2
1 4 2
1 5 2
0 5 3
1 6 3
1 7 3
1 8 3
1 9 3
1 10 3
1 11 3
1 12 3
1 13 3
1 14 3
1 15 3
1 16 3
1 17 3
total "-" Seagulls total events
17 20

Attribution 4.0 International (CC BY 4.0)

ng the Area Under the Curve as a Series of Rectangles
average of y values rectangle
0 0.0000 1.00

0.17 0.0000 0.90

0.33 0.0196
0.80
0.33 0.0196
0.70
0.50 0.0000
0.67 0.0392 0.60
0.67 0.0392
0.50
0.83 0.0000
0.40
1 0.059
1 0.059 0.30

1 0.059 0.20
1 0.059
0.10
1 0.059
1 0.059 0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70
1 0.059
1 0.059
1 0.059
1 0.059
1 0.059
1 0.059

0.824
Part 4 - Cost Function
Cost per FN £5,000,000
FP rate = x TP rate = y number FNs product
0 0 3 £15,000,000
0.06 0.00 3 £15,000,000
0.06 0.33 2 £10,000,000
0.12 0.33 2 £10,000,000
0.18 0.33 2 £10,000,000
0.18 0.67 1 £5,000,000
0.24 0.67 1 £5,000,000
0.29 0.67 1 £5,000,000
0.29 1.00 0 £0
0.35 1.00 0 £0
0.41 1.00 0 £0
0.47 1.00 0 £0
0.53 1.00 0 £0
0.59 1.00 0 £0
0.65 1.00 0 £0
0.71 1.00 0 £0
0.76 1.00 0 £0
0.82 1.00 0 £0
0.88 1.00 0 £0
0.94 1.00 0 £0
1.00 1.00 0 £0

Note: minimum cost threshold is dete

(cost per FN)/(cost per FP)
 ROC Curve

Column E

0.40 0.50 0.60 0.70 0.80 0.90 1.00

Cost per FP £4,000,000

number FPs product Total Cost Cost per event
0 £0 £15,000,000 £750,000
1 £4,000,000 £19,000,000 £950,000
 1 £4,000,000 £14,000,000 £700,000
2 £8,000,000 £18,000,000 £900,000
3 £12,000,000 £22,000,000 £1,100,000
3 £12,000,000 £17,000,000 £850,000
4 £16,000,000 £21,000,000 £1,050,000
5 £20,000,000 £25,000,000 £1,250,000
5 £20,000,000 £20,000,000 £1,000,000
6 £24,000,000 £24,000,000 £1,200,000
7 £28,000,000 £28,000,000 £1,400,000
8 £32,000,000 £32,000,000 £1,600,000
9 £36,000,000 £36,000,000 £1,800,000
10 £40,000,000 £40,000,000 £2,000,000
11 £44,000,000 £44,000,000 £2,200,000
12 £48,000,000 £48,000,000 £2,400,000
13 £52,000,000 £52,000,000 £2,600,000
14 £56,000,000 £56,000,000 £2,800,000
15 £60,000,000 £60,000,000 £3,000,000
16 £64,000,000 £64,000,000 £3,200,000
17 £68,000,000 £68,000,000 £3,400,000
Minimum per event
£14,000,000 £700,000
inimum cost threshold is determined by "cost function ratio)
r FN)/(cost per FP) here: 1.25