Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (20% of outliers version#08)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (51.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 27 0.52459 0.40574 0.59235 0.49044 0.56000 0.45000 0.74127
KNN 50 0.50820 0.38525 0.59687 0.49609 0.56000 0.45000 0.74449
KNN 51 0.50820 0.38525 0.59840 0.49800 0.56000 0.45000 0.74395
KNN 71 0.50820 0.38525 0.59608 0.49509 0.56604 0.45755 0.74227
KNNW 1 0.49180 0.36475 0.58736 0.48420 0.52381 0.40476 0.75239
KNNW 3 0.50820 0.38525 0.57938 0.47422 0.52727 0.40909 0.74221
KNNW 49 0.50820 0.38525 0.59192 0.48989 0.56000 0.45000 0.74133
KNNW 75 0.49180 0.36475 0.58975 0.48718 0.56566 0.45707 0.74080
LOF 2 0.49180 0.36475 0.51202 0.39002 0.51786 0.39732 0.75322
LOF 4 0.50820 0.38525 0.51235 0.39044 0.51128 0.38910 0.74133
LOF 81 0.50820 0.38525 0.57986 0.47482 0.55446 0.44307 0.74039
LOF 98 0.49180 0.36475 0.58370 0.47962 0.55446 0.44307 0.74160
SimplifiedLOF 6 0.49180 0.36475 0.52919 0.41148 0.51200 0.39000 0.74980
SimplifiedLOF 56 0.52459 0.40574 0.56527 0.45659 0.52459 0.40574 0.73999
SimplifiedLOF 88 0.50820 0.38525 0.57802 0.47253 0.53333 0.41667 0.74456
SimplifiedLOF 96 0.50820 0.38525 0.57877 0.47346 0.53333 0.41667 0.74321
LoOP 6 0.49180 0.36475 0.50986 0.38732 0.50420 0.38025 0.74845
LoOP 58 0.52459 0.40574 0.56113 0.45141 0.52459 0.40574 0.74033
LoOP 71 0.52459 0.40574 0.56512 0.45641 0.53448 0.41810 0.73922
LoOP 85 0.50820 0.38525 0.57460 0.46825 0.52381 0.40476 0.74291
LDOF 8 0.49180 0.36475 0.48454 0.35568 0.50667 0.38333 0.75417
LDOF 83 0.52459 0.40574 0.56192 0.45240 0.52459 0.40574 0.74281
LDOF 84 0.52459 0.40574 0.56177 0.45221 0.52893 0.41116 0.74301
LDOF 98 0.50820 0.38525 0.56850 0.46063 0.52542 0.40678 0.74308
ODIN 58 0.52927 0.41159 0.43474 0.29342 0.53659 0.42073 0.72729
ODIN 97 0.50820 0.38525 0.46402 0.33002 0.52033 0.40041 0.73854
ODIN 100 0.50820 0.38525 0.46821 0.33526 0.52033 0.40041 0.73804
FastABOD 6 0.45902 0.32377 0.53339 0.41673 0.49718 0.37147 0.74738
FastABOD 56 0.52459 0.40574 0.55524 0.44405 0.52459 0.40574 0.74187
FastABOD 68 0.52459 0.40574 0.56260 0.45325 0.55172 0.43966 0.74335
FastABOD 78 0.50820 0.38525 0.56374 0.45467 0.52991 0.41239 0.74227
KDEOS 8 0.36066 0.20082 0.28571 0.10714 0.41129 0.26411 0.66199
KDEOS 13 0.36066 0.20082 0.31420 0.14275 0.44211 0.30263 0.69215
KDEOS 15 0.32787 0.15984 0.31012 0.13765 0.46875 0.33594 0.69786
LDF 1 0.42623 0.28279 0.37439 0.21799 0.47742 0.34677 0.74076
LDF 84 0.54098 0.42623 0.52702 0.40878 0.54400 0.43000 0.71479
LDF 89 0.50820 0.38525 0.57282 0.46603 0.55102 0.43878 0.72863
LDF 98 0.50820 0.38525 0.58421 0.48027 0.53608 0.42010 0.71661
INFLO 26 0.52459 0.40574 0.55860 0.44825 0.52459 0.40574 0.75390
INFLO 35 0.50820 0.38525 0.56367 0.45459 0.52713 0.40891 0.75632
INFLO 67 0.52459 0.40574 0.57378 0.46723 0.53913 0.42391 0.74980
INFLO 94 0.50820 0.38525 0.58076 0.47595 0.52252 0.40315 0.73663
COF 6 0.45902 0.32377 0.53245 0.41557 0.52336 0.40421 0.74466
COF 28 0.49180 0.36475 0.51180 0.38974 0.49485 0.36856 0.71936
COF 46 0.44262 0.30328 0.53884 0.42355 0.48421 0.35526 0.71231

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (52.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.50820 0.38525 0.55392 0.44240 0.55357 0.44196 0.77560
KNN 3 0.52459 0.40574 0.55898 0.44873 0.55046 0.43807 0.78312
KNN 7 0.54098 0.42623 0.55349 0.44187 0.54386 0.42982 0.77728
KNNW 2 0.52459 0.40574 0.55038 0.43797 0.55856 0.44820 0.76854
KNNW 14 0.52459 0.40574 0.55723 0.44654 0.54545 0.43182 0.77862
KNNW 15 0.52459 0.40574 0.55671 0.44589 0.54545 0.43182 0.77909
KNNW 17 0.54098 0.42623 0.55580 0.44475 0.54386 0.42982 0.77882
LOF 12 0.54098 0.42623 0.53691 0.42114 0.56000 0.45000 0.78299
LOF 14 0.52459 0.40574 0.54432 0.43040 0.54962 0.43702 0.78393
LOF 16 0.52459 0.40574 0.54283 0.42853 0.54015 0.42518 0.78561
SimplifiedLOF 13 0.50820 0.38525 0.53927 0.42409 0.56716 0.45896 0.78621
SimplifiedLOF 23 0.52459 0.40574 0.55839 0.44799 0.54839 0.43548 0.79125
SimplifiedLOF 31 0.54098 0.42623 0.55250 0.44062 0.54839 0.43548 0.78628
LoOP 13 0.50820 0.38525 0.52571 0.40714 0.56923 0.46154 0.78514
LoOP 16 0.52459 0.40574 0.52507 0.40634 0.54962 0.43702 0.79011
LoOP 31 0.52459 0.40574 0.55196 0.43994 0.55172 0.43966 0.78420
LoOP 42 0.54098 0.42623 0.54976 0.43720 0.54701 0.43376 0.78064
LDOF 20 0.55738 0.44672 0.51966 0.39958 0.56757 0.45946 0.78662
LDOF 22 0.54098 0.42623 0.53205 0.41507 0.56954 0.46192 0.79300
LDOF 23 0.52459 0.40574 0.53800 0.42249 0.56209 0.45261 0.79663
LDOF 53 0.52459 0.40574 0.55339 0.44174 0.53448 0.41810 0.78440
ODIN 14 0.51148 0.38934 0.43148 0.28935 0.53165 0.41456 0.78437
ODIN 16 0.50820 0.38525 0.43385 0.29232 0.55782 0.44728 0.78396
ODIN 21 0.53279 0.41598 0.46142 0.32678 0.53333 0.41667 0.78228
ODIN 69 0.50820 0.38525 0.52743 0.40929 0.53571 0.41964 0.77684
FastABOD 4 0.52459 0.40574 0.52209 0.40261 0.52459 0.40574 0.78762
FastABOD 5 0.50820 0.38525 0.53096 0.41370 0.53435 0.41794 0.80234
FastABOD 9 0.50820 0.38525 0.50474 0.38092 0.55556 0.44444 0.78225
KDEOS 52 0.45902 0.32377 0.37470 0.21838 0.52500 0.40625 0.75672
KDEOS 72 0.44262 0.30328 0.36574 0.20718 0.54545 0.43182 0.75168
KDEOS 91 0.44262 0.30328 0.41521 0.26901 0.54054 0.42568 0.76545
KDEOS 95 0.44262 0.30328 0.42059 0.27574 0.51316 0.39145 0.76250
LDF 3 0.44262 0.30328 0.43645 0.29556 0.46377 0.32971 0.74671
LDF 67 0.27869 0.09836 0.42818 0.28522 0.46486 0.33108 0.68140
LDF 80 0.36066 0.20082 0.44804 0.31005 0.45714 0.32143 0.69041
INFLO 7 0.52459 0.40574 0.48217 0.35271 0.53846 0.42308 0.76915
INFLO 8 0.52459 0.40574 0.48643 0.35804 0.54264 0.42829 0.77936
INFLO 17 0.52459 0.40574 0.52479 0.40599 0.53901 0.42376 0.80207
INFLO 58 0.47541 0.34426 0.55062 0.43827 0.51429 0.39286 0.78413
COF 5 0.42623 0.28279 0.53103 0.41379 0.50909 0.38636 0.75796
COF 9 0.49180 0.36475 0.53348 0.41685 0.51485 0.39356 0.74664
COF 10 0.47541 0.34426 0.54343 0.42929 0.53061 0.41327 0.75060
COF 11 0.45902 0.32377 0.54405 0.43006 0.50000 0.37500 0.74745

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO