Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (20% of outliers version#05)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (53.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 12 0.44262 0.30328 0.51092 0.38866 0.44898 0.31122 0.72373
KNN 30 0.44262 0.30328 0.51748 0.39685 0.47170 0.33962 0.72595
KNN 49 0.44262 0.30328 0.51761 0.39701 0.48077 0.35096 0.72407
KNN 81 0.42623 0.28279 0.51893 0.39866 0.48077 0.35096 0.72272
KNNW 1 0.47541 0.34426 0.50731 0.38414 0.48000 0.35000 0.73542
KNNW 97 0.42623 0.28279 0.51728 0.39660 0.47170 0.33962 0.72413
LOF 4 0.40984 0.26230 0.48070 0.35088 0.48235 0.35294 0.74496
LOF 12 0.39344 0.24180 0.48822 0.36027 0.50000 0.37500 0.72830
LOF 38 0.44262 0.30328 0.48928 0.36160 0.46988 0.33735 0.71809
LOF 89 0.42623 0.28279 0.51317 0.39146 0.45872 0.32339 0.71829
SimplifiedLOF 26 0.39344 0.24180 0.48275 0.35344 0.50909 0.38636 0.72675
SimplifiedLOF 66 0.44262 0.30328 0.50376 0.37971 0.48521 0.35651 0.73065
SimplifiedLOF 74 0.42623 0.28279 0.50966 0.38708 0.48837 0.36047 0.73233
SimplifiedLOF 85 0.44262 0.30328 0.50995 0.38743 0.48837 0.36047 0.72971
LoOP 26 0.39344 0.24180 0.47644 0.34555 0.50617 0.38272 0.72460
LoOP 68 0.44262 0.30328 0.50242 0.37803 0.49398 0.36747 0.73085
LoOP 73 0.42623 0.28279 0.50359 0.37949 0.49398 0.36747 0.73189
LoOP 85 0.42623 0.28279 0.50937 0.38672 0.48837 0.36047 0.72958
LDOF 74 0.40984 0.26230 0.50234 0.37792 0.48780 0.35976 0.73435
LDOF 86 0.42623 0.28279 0.50683 0.38354 0.49032 0.36290 0.73193
LDOF 96 0.42623 0.28279 0.50958 0.38698 0.48045 0.35056 0.73159
LDOF 98 0.44262 0.30328 0.50704 0.38380 0.48148 0.35185 0.73152
ODIN 17 0.40212 0.25265 0.36129 0.20161 0.49162 0.36453 0.72339
ODIN 22 0.38525 0.23156 0.35668 0.19585 0.49704 0.37130 0.71731
ODIN 96 0.44262 0.30328 0.41241 0.26552 0.46739 0.33424 0.71385
ODIN 100 0.43443 0.29303 0.41350 0.26687 0.45860 0.32325 0.71359
FastABOD 17 0.42623 0.28279 0.48293 0.35366 0.46914 0.33642 0.72051
FastABOD 29 0.40984 0.26230 0.48251 0.35314 0.48485 0.35606 0.72998
FastABOD 34 0.39344 0.24180 0.47583 0.34478 0.49080 0.36350 0.72548
FastABOD 90 0.42623 0.28279 0.49070 0.36338 0.47126 0.33908 0.72279
KDEOS 14 0.34426 0.18033 0.29688 0.12111 0.40860 0.26075 0.66514
KDEOS 18 0.34426 0.18033 0.32608 0.15760 0.44578 0.30723 0.69020
KDEOS 24 0.34426 0.18033 0.33058 0.16323 0.44311 0.30389 0.68255
KDEOS 93 0.29508 0.11885 0.30618 0.13273 0.46243 0.32803 0.68496
LDF 39 0.45902 0.32377 0.46839 0.33549 0.46667 0.33333 0.72225
LDF 42 0.42623 0.28279 0.49721 0.37152 0.47619 0.34524 0.75531
LDF 43 0.44262 0.30328 0.51430 0.39287 0.49091 0.36364 0.74819
LDF 54 0.42623 0.28279 0.50562 0.38202 0.49485 0.36856 0.71285
INFLO 17 0.40984 0.26230 0.49182 0.36477 0.50909 0.38636 0.74772
INFLO 18 0.40984 0.26230 0.48891 0.36114 0.51220 0.39024 0.73965
INFLO 45 0.44262 0.30328 0.49821 0.37276 0.49682 0.37102 0.73468
INFLO 81 0.44262 0.30328 0.51661 0.39576 0.48148 0.35185 0.73999
COF 2 0.44262 0.30328 0.45322 0.31652 0.44800 0.31000 0.72155
COF 3 0.42623 0.28279 0.48661 0.35827 0.46316 0.32895 0.74106
COF 10 0.40984 0.26230 0.47410 0.34262 0.48611 0.35764 0.72662

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (53.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 8 0.44262 0.30328 0.52602 0.40752 0.51282 0.39103 0.75252
KNN 9 0.44262 0.30328 0.52762 0.40952 0.50633 0.38291 0.75228
KNN 75 0.45902 0.32377 0.51662 0.39578 0.48276 0.35345 0.74429
KNNW 1 0.47541 0.34426 0.53439 0.41799 0.50394 0.37992 0.74157
KNNW 6 0.44262 0.30328 0.52332 0.40415 0.50685 0.38356 0.74899
KNNW 25 0.44262 0.30328 0.52801 0.41002 0.49673 0.37092 0.75134
LOF 3 0.45902 0.32377 0.52302 0.40377 0.49682 0.37102 0.77734
LOF 5 0.50820 0.38525 0.53847 0.42308 0.52991 0.41239 0.76841
LOF 7 0.49180 0.36475 0.53081 0.41351 0.53125 0.41406 0.76471
SimplifiedLOF 6 0.50820 0.38525 0.55233 0.44042 0.54545 0.43182 0.77755
LoOP 7 0.52459 0.40574 0.54199 0.42748 0.53226 0.41532 0.77577
LoOP 13 0.50820 0.38525 0.54671 0.43338 0.51667 0.39583 0.77472
LoOP 17 0.52459 0.40574 0.53431 0.41789 0.52893 0.41116 0.77657
LDOF 9 0.52459 0.40574 0.53255 0.41569 0.53782 0.42227 0.77869
LDOF 12 0.49180 0.36475 0.54598 0.43247 0.54000 0.42500 0.78594
LDOF 15 0.50820 0.38525 0.55207 0.44008 0.53465 0.41832 0.79078
LDOF 20 0.52459 0.40574 0.55209 0.44011 0.53333 0.41667 0.78749
ODIN 30 0.51803 0.39754 0.50434 0.38043 0.52101 0.40126 0.77190
ODIN 35 0.49649 0.37061 0.51604 0.39506 0.54054 0.42568 0.77029
ODIN 37 0.49180 0.36475 0.51756 0.39695 0.52941 0.41176 0.76653
FastABOD 4 0.44262 0.30328 0.50870 0.38587 0.47482 0.34353 0.72682
FastABOD 12 0.45902 0.32377 0.49837 0.37297 0.48438 0.35547 0.74429
FastABOD 21 0.50820 0.38525 0.49642 0.37052 0.50820 0.38525 0.74368
KDEOS 13 0.37705 0.22131 0.47188 0.33985 0.47179 0.33974 0.74382
KDEOS 24 0.42623 0.28279 0.44741 0.30926 0.51534 0.39417 0.76297
KDEOS 52 0.49180 0.36475 0.38440 0.23050 0.51007 0.38758 0.74812
KDEOS 55 0.44262 0.30328 0.37379 0.21724 0.52482 0.40603 0.74550
LDF 56 0.36066 0.20082 0.41697 0.27121 0.44571 0.30714 0.69887
LDF 65 0.31148 0.13934 0.42720 0.28400 0.45283 0.31604 0.69759
LDF 68 0.36066 0.20082 0.44140 0.30176 0.41748 0.27184 0.67670
LDF 80 0.39344 0.24180 0.43207 0.29009 0.43972 0.29965 0.68073
INFLO 7 0.52459 0.40574 0.52262 0.40328 0.52459 0.40574 0.77002
INFLO 8 0.52459 0.40574 0.52692 0.40864 0.53333 0.41667 0.77083
INFLO 16 0.50820 0.38525 0.52940 0.41175 0.52101 0.40126 0.78111
INFLO 23 0.49180 0.36475 0.53708 0.42135 0.51563 0.39453 0.77748
COF 4 0.47541 0.34426 0.53432 0.41790 0.50000 0.37500 0.75396
COF 5 0.44262 0.30328 0.54883 0.43603 0.46809 0.33511 0.74479

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO