Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (10% of outliers version#07)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 271 objects, 27 outliers (9.96%)

Download raw algorithm results (2.4 MB) Download raw algorithm evaluation table (49.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.37037 0.30070 0.36855 0.29868 0.44156 0.37976 0.73042
KNN 4 0.33333 0.25956 0.36399 0.29361 0.47500 0.41691 0.73695
KNN 24 0.33333 0.25956 0.40590 0.34016 0.41304 0.34809 0.74044
KNN 100 0.33333 0.25956 0.38878 0.32115 0.39394 0.32688 0.74294
KNNW 1 0.29630 0.21843 0.35839 0.28739 0.41935 0.35510 0.74613
KNNW 3 0.37037 0.30070 0.37671 0.30774 0.42697 0.36356 0.73634
KNNW 9 0.33333 0.25956 0.36736 0.29735 0.47500 0.41691 0.73679
KNNW 30 0.33333 0.25956 0.39886 0.33234 0.42697 0.36356 0.73922
LOF 2 0.37037 0.30070 0.34407 0.27149 0.37931 0.31063 0.76427
LOF 5 0.40741 0.34183 0.35327 0.28171 0.42857 0.36534 0.75455
LOF 45 0.37037 0.30070 0.35873 0.28777 0.46914 0.41039 0.74879
LOF 81 0.33333 0.25956 0.38414 0.31599 0.41791 0.35350 0.74408
SimplifiedLOF 1 0.40741 0.34183 0.36279 0.29228 0.41558 0.35092 0.74522
SimplifiedLOF 7 0.40741 0.34183 0.35124 0.27945 0.43137 0.36845 0.77171
SimplifiedLOF 25 0.33333 0.25956 0.37315 0.30379 0.42105 0.35699 0.75501
SimplifiedLOF 45 0.37037 0.30070 0.35212 0.28043 0.47500 0.41691 0.75562
LoOP 1 0.40741 0.34183 0.36279 0.29228 0.41558 0.35092 0.74522
LoOP 11 0.40741 0.34183 0.35780 0.28673 0.42308 0.35924 0.77565
LoOP 43 0.33333 0.25956 0.35899 0.28806 0.47222 0.41382 0.76002
LoOP 81 0.37037 0.30070 0.36880 0.29896 0.45570 0.39547 0.75304
LDOF 3 0.44444 0.38297 0.29094 0.21248 0.47273 0.41438 0.75668
LDOF 11 0.40741 0.34183 0.38080 0.31228 0.45333 0.39284 0.79660
ODIN 9 0.39024 0.32277 0.30586 0.22905 0.47059 0.41201 0.78271
ODIN 62 0.34815 0.27602 0.31044 0.23414 0.49275 0.43662 0.76245
ODIN 96 0.40741 0.34183 0.35586 0.28458 0.48276 0.42552 0.76457
ODIN 99 0.40000 0.33361 0.35865 0.28768 0.46154 0.40195 0.76579
FastABOD 11 0.29630 0.21843 0.35156 0.27981 0.41791 0.35350 0.75213
FastABOD 34 0.37037 0.30070 0.33078 0.25672 0.47059 0.41201 0.74408
FastABOD 55 0.40741 0.34183 0.32770 0.25331 0.42697 0.36356 0.74211
FastABOD 94 0.37037 0.30070 0.37005 0.30035 0.44706 0.38587 0.74089
KDEOS 16 0.29630 0.21843 0.23142 0.14637 0.32432 0.24956 0.74302
KDEOS 20 0.25926 0.17729 0.23618 0.15166 0.44156 0.37976 0.76169
LDF 77 0.37037 0.30070 0.40713 0.34153 0.41667 0.35212 0.75091
LDF 80 0.40741 0.34183 0.36864 0.29878 0.45455 0.39419 0.74044
INFLO 7 0.44444 0.38297 0.35717 0.28603 0.46429 0.40501 0.77702
INFLO 38 0.37037 0.30070 0.37113 0.30154 0.48649 0.42966 0.78992
INFLO 43 0.40741 0.34183 0.36744 0.29744 0.49275 0.43662 0.76609
INFLO 100 0.37037 0.30070 0.38461 0.31651 0.44737 0.38622 0.75622
COF 1 0.40741 0.34183 0.36279 0.29228 0.41558 0.35092 0.74522
COF 2 0.37037 0.30070 0.35879 0.28784 0.38462 0.31652 0.76237

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 271 objects, 27 outliers (9.96%)

Download raw algorithm results (2.4 MB) Download raw algorithm evaluation table (48.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.29630 0.21843 0.36867 0.29881 0.40000 0.33361 0.75698
KNN 14 0.33333 0.25956 0.34786 0.27569 0.43077 0.36778 0.77413
KNN 25 0.37037 0.30070 0.34271 0.26997 0.40000 0.33361 0.77201
KNNW 1 0.37037 0.30070 0.34629 0.27395 0.38961 0.32207 0.76222
KNNW 2 0.33333 0.25956 0.36248 0.29193 0.39024 0.32277 0.76063
KNNW 10 0.33333 0.25956 0.36020 0.28940 0.42623 0.36274 0.76563
KNNW 35 0.33333 0.25956 0.35350 0.28196 0.41791 0.35350 0.77125
LOF 1 0.37037 0.30070 0.34120 0.26830 0.38961 0.32207 0.69444
LOF 4 0.37037 0.30070 0.38454 0.31644 0.39130 0.32395 0.75683
LOF 17 0.29630 0.21843 0.35489 0.28351 0.42105 0.35699 0.77687
LOF 52 0.29630 0.21843 0.35366 0.28214 0.43750 0.37526 0.77049
SimplifiedLOF 4 0.37037 0.30070 0.39108 0.32370 0.42553 0.36196 0.76093
SimplifiedLOF 19 0.37037 0.30070 0.38452 0.31641 0.43243 0.36963 0.79797
SimplifiedLOF 45 0.33333 0.25956 0.36257 0.29203 0.45070 0.38992 0.77808
LoOP 5 0.37037 0.30070 0.38501 0.31696 0.40000 0.33361 0.76503
LoOP 14 0.33333 0.25956 0.37704 0.30811 0.43038 0.36735 0.79857
LoOP 41 0.37037 0.30070 0.36729 0.29728 0.45070 0.38992 0.77535
LoOP 59 0.40741 0.34183 0.37132 0.30175 0.43836 0.37621 0.77474
LDOF 6 0.40741 0.34183 0.36886 0.29902 0.43137 0.36845 0.77990
LDOF 14 0.40741 0.34183 0.37597 0.30692 0.43478 0.37224 0.80844
LDOF 22 0.37037 0.30070 0.37720 0.30828 0.45570 0.39547 0.79053
LDOF 23 0.37037 0.30070 0.38580 0.31784 0.45455 0.39419 0.79402
ODIN 19 0.39886 0.33234 0.36631 0.29619 0.40964 0.34431 0.79493
ODIN 39 0.41358 0.34869 0.37427 0.30503 0.44737 0.38622 0.78772
ODIN 42 0.43210 0.36926 0.36935 0.29957 0.43636 0.37399 0.78673
ODIN 50 0.38519 0.31715 0.37998 0.31137 0.41860 0.35427 0.78696
FastABOD 4 0.44444 0.38297 0.43555 0.37309 0.46512 0.40593 0.80449
FastABOD 6 0.37037 0.30070 0.43566 0.37321 0.46154 0.40195 0.80206
KDEOS 13 0.33333 0.25956 0.33954 0.26646 0.36000 0.28918 0.75713
KDEOS 81 0.29630 0.21843 0.27107 0.19041 0.40909 0.34370 0.77459
KDEOS 83 0.29630 0.21843 0.26704 0.18594 0.42857 0.36534 0.77080
LDF 3 0.22222 0.13616 0.19702 0.10817 0.32258 0.24762 0.70940
LDF 5 0.14815 0.05389 0.17194 0.08031 0.32432 0.24956 0.68534
LDF 26 0.14815 0.05389 0.26028 0.17842 0.25806 0.17597 0.60200
LDF 34 0.25926 0.17729 0.20628 0.11845 0.26891 0.18801 0.62417
INFLO 14 0.33333 0.25956 0.38605 0.31812 0.43077 0.36778 0.82362
INFLO 22 0.33333 0.25956 0.36689 0.29683 0.44156 0.37976 0.80692
INFLO 26 0.40741 0.34183 0.37198 0.30249 0.41509 0.35037 0.80358
COF 2 0.33333 0.25956 0.40458 0.33870 0.40625 0.34055 0.74340
COF 4 0.37037 0.30070 0.37511 0.30596 0.40541 0.33961 0.75395
COF 6 0.29630 0.21843 0.33536 0.26181 0.35484 0.28345 0.76230
COF 8 0.37037 0.30070 0.32747 0.25305 0.40909 0.34370 0.75152

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO