Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (20% of outliers version#09)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (53.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 3 0.45902 0.32377 0.52473 0.40591 0.48855 0.36069 0.72887
KNN 5 0.44262 0.30328 0.52425 0.40531 0.47761 0.34701 0.73243
KNN 89 0.44262 0.30328 0.53423 0.41779 0.46809 0.33511 0.72467
KNNW 1 0.44262 0.30328 0.51245 0.39056 0.47887 0.34859 0.74080
KNNW 2 0.45902 0.32377 0.51755 0.39693 0.46970 0.33712 0.73623
KNNW 95 0.44262 0.30328 0.53022 0.41277 0.48780 0.35976 0.72931
KNNW 100 0.44262 0.30328 0.53081 0.41352 0.48780 0.35976 0.72917
LOF 29 0.42623 0.28279 0.51670 0.39588 0.50350 0.37937 0.73468
LOF 40 0.42623 0.28279 0.52380 0.40475 0.49367 0.36709 0.73831
LOF 49 0.47541 0.34426 0.52464 0.40580 0.49367 0.36709 0.73676
LOF 60 0.47541 0.34426 0.52815 0.41019 0.50000 0.37500 0.73367
SimplifiedLOF 39 0.42623 0.28279 0.52281 0.40352 0.50685 0.38356 0.74590
SimplifiedLOF 94 0.47541 0.34426 0.52833 0.41041 0.50794 0.38492 0.73891
SimplifiedLOF 97 0.45902 0.32377 0.53142 0.41427 0.50794 0.38492 0.73811
LoOP 39 0.42623 0.28279 0.52472 0.40590 0.50350 0.37937 0.74570
LoOP 40 0.42623 0.28279 0.52202 0.40253 0.50980 0.38725 0.74530
LoOP 94 0.47541 0.34426 0.52598 0.40748 0.50000 0.37500 0.73680
LoOP 100 0.47541 0.34426 0.52910 0.41137 0.49612 0.37016 0.73566
LDOF 39 0.45902 0.32377 0.51123 0.38904 0.50000 0.37500 0.74610
LDOF 83 0.42623 0.28279 0.52090 0.40113 0.50725 0.38406 0.74597
LDOF 84 0.42623 0.28279 0.52209 0.40262 0.50360 0.37950 0.74671
LDOF 99 0.44262 0.30328 0.52369 0.40461 0.50000 0.37500 0.74026
ODIN 50 0.45902 0.32377 0.39389 0.24237 0.49162 0.36453 0.72497
ODIN 79 0.44262 0.30328 0.43302 0.29128 0.50360 0.37950 0.72749
ODIN 99 0.45902 0.32377 0.45997 0.32496 0.47826 0.34783 0.73109
FastABOD 6 0.44262 0.30328 0.49676 0.37095 0.47399 0.34249 0.73831
FastABOD 27 0.40984 0.26230 0.50285 0.37856 0.50292 0.37865 0.73240
FastABOD 58 0.39344 0.24180 0.50863 0.38579 0.48555 0.35694 0.73300
KDEOS 15 0.37705 0.22131 0.32260 0.15325 0.44186 0.30233 0.69397
KDEOS 16 0.34426 0.18033 0.33286 0.16608 0.44330 0.30412 0.70169
KDEOS 99 0.31148 0.13934 0.31761 0.14701 0.49724 0.37155 0.70378
KDEOS 100 0.31148 0.13934 0.31726 0.14657 0.49730 0.37162 0.70270
LDF 44 0.44262 0.30328 0.41438 0.26798 0.45802 0.32252 0.68873
LDF 71 0.40984 0.26230 0.48132 0.35165 0.46000 0.32500 0.70418
LDF 80 0.44262 0.30328 0.50189 0.37737 0.46316 0.32895 0.68671
LDF 82 0.42623 0.28279 0.49277 0.36596 0.46465 0.33081 0.67697
INFLO 16 0.49180 0.36475 0.51360 0.39200 0.50000 0.37500 0.73596
INFLO 29 0.47541 0.34426 0.52681 0.40852 0.50340 0.37925 0.75168
INFLO 49 0.47541 0.34426 0.52977 0.41222 0.51034 0.38793 0.74106
INFLO 53 0.47541 0.34426 0.53383 0.41728 0.50685 0.38356 0.74530
COF 3 0.47541 0.34426 0.50628 0.38285 0.49558 0.36947 0.73787
COF 5 0.47541 0.34426 0.54006 0.42508 0.49020 0.36275 0.74758
COF 10 0.42623 0.28279 0.51980 0.39975 0.51250 0.39062 0.75752

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (52.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 4 0.54098 0.42623 0.55749 0.44686 0.55462 0.44328 0.75937
KNN 35 0.52459 0.40574 0.55171 0.43963 0.53913 0.42391 0.76297
KNNW 2 0.49180 0.36475 0.54758 0.43447 0.52857 0.41071 0.76411
KNNW 5 0.54098 0.42623 0.55138 0.43922 0.54545 0.43182 0.76135
KNNW 7 0.54098 0.42623 0.55419 0.44273 0.55462 0.44328 0.76082
KNNW 18 0.54098 0.42623 0.55449 0.44311 0.54839 0.43548 0.76021
LOF 7 0.50820 0.38525 0.53116 0.41395 0.56250 0.45312 0.74913
LOF 37 0.55738 0.44672 0.55029 0.43786 0.55738 0.44672 0.76774
LOF 50 0.55738 0.44672 0.55267 0.44083 0.56198 0.45248 0.76579
SimplifiedLOF 9 0.50820 0.38525 0.52511 0.40639 0.56923 0.46154 0.76041
SimplifiedLOF 23 0.52459 0.40574 0.55963 0.44954 0.55385 0.44231 0.77237
SimplifiedLOF 43 0.55738 0.44672 0.55002 0.43753 0.55738 0.44672 0.77022
LoOP 9 0.50820 0.38525 0.51138 0.38923 0.56250 0.45312 0.75988
LoOP 20 0.55738 0.44672 0.55390 0.44238 0.55738 0.44672 0.76989
LoOP 21 0.52459 0.40574 0.55643 0.44553 0.56000 0.45000 0.76948
LoOP 47 0.54098 0.42623 0.55064 0.43830 0.55285 0.44106 0.77143
LDOF 17 0.54098 0.42623 0.52438 0.40548 0.54545 0.43182 0.76639
LDOF 20 0.52459 0.40574 0.54304 0.42880 0.56944 0.46181 0.77614
LDOF 26 0.54098 0.42623 0.55711 0.44639 0.55000 0.43750 0.77654
LDOF 40 0.50820 0.38525 0.56094 0.45118 0.54867 0.43584 0.77432
ODIN 20 0.46721 0.33402 0.46235 0.32793 0.55405 0.44257 0.75763
ODIN 32 0.53279 0.41598 0.49762 0.37203 0.54400 0.43000 0.76293
ODIN 73 0.50820 0.38525 0.52055 0.40069 0.52857 0.41071 0.76834
ODIN 75 0.50820 0.38525 0.51914 0.39893 0.52857 0.41071 0.76932
FastABOD 4 0.52459 0.40574 0.53228 0.41535 0.53846 0.42308 0.77755
FastABOD 6 0.54098 0.42623 0.52297 0.40371 0.54867 0.43584 0.77056
FastABOD 14 0.49180 0.36475 0.52187 0.40234 0.55172 0.43966 0.77708
KDEOS 24 0.36066 0.20082 0.38997 0.23746 0.46626 0.33282 0.70317
KDEOS 56 0.42623 0.28279 0.35554 0.19443 0.51136 0.38920 0.72279
KDEOS 91 0.37705 0.22131 0.35643 0.19553 0.55034 0.43792 0.73347
KDEOS 100 0.40984 0.26230 0.38272 0.22840 0.52980 0.41225 0.73985
LDF 3 0.45902 0.32377 0.41415 0.26768 0.47273 0.34091 0.71009
LDF 10 0.45902 0.32377 0.42707 0.28383 0.47619 0.34524 0.68268
LDF 13 0.40984 0.26230 0.43606 0.29507 0.42017 0.27521 0.66897
INFLO 9 0.52459 0.40574 0.51289 0.39111 0.52893 0.41116 0.76505
INFLO 13 0.52459 0.40574 0.53418 0.41772 0.54839 0.43548 0.76209
INFLO 37 0.52459 0.40574 0.56155 0.45194 0.53125 0.41406 0.77714
INFLO 60 0.49180 0.36475 0.55780 0.44725 0.52713 0.40891 0.79031
COF 5 0.49180 0.36475 0.51051 0.38813 0.53750 0.42188 0.74919
COF 14 0.45902 0.32377 0.54038 0.42547 0.50000 0.37500 0.75134
COF 30 0.52459 0.40574 0.52098 0.40122 0.52893 0.41116 0.72763

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO