Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (5% of outliers version#09)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 256 objects, 12 outliers (4.69%)

Download raw algorithm results (2.3 MB) Download raw algorithm evaluation table (44.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.16667 0.12568 0.29545 0.26080 0.40000 0.37049 0.88559
KNN 19 0.33333 0.30055 0.31615 0.28252 0.33333 0.30055 0.87739
KNN 20 0.33333 0.30055 0.31992 0.28647 0.33333 0.30055 0.87807
KNN 41 0.25000 0.21311 0.31474 0.28104 0.33333 0.30055 0.88593
KNNW 3 0.16667 0.12568 0.29362 0.25887 0.40000 0.37049 0.88490
KNNW 9 0.25000 0.21311 0.28215 0.24684 0.40000 0.37049 0.87773
KNNW 15 0.33333 0.30055 0.29185 0.25703 0.39024 0.36026 0.88012
KNNW 91 0.25000 0.21311 0.30859 0.27459 0.33333 0.30055 0.88320
LOF 54 0.25000 0.21311 0.26784 0.23183 0.34783 0.31575 0.87363
LOF 56 0.25000 0.21311 0.26803 0.23203 0.36364 0.33234 0.87568
LOF 96 0.25000 0.21311 0.28358 0.24834 0.33333 0.30055 0.88149
SimplifiedLOF 60 0.08333 0.03825 0.26153 0.22522 0.37500 0.34426 0.86851
SimplifiedLOF 70 0.25000 0.21311 0.26909 0.23315 0.36735 0.33623 0.86919
SimplifiedLOF 97 0.25000 0.21311 0.27848 0.24300 0.36000 0.32852 0.87637
SimplifiedLOF 98 0.25000 0.21311 0.27923 0.24378 0.36000 0.32852 0.87637
LoOP 70 0.08333 0.03825 0.25920 0.22276 0.38298 0.35263 0.86680
LoOP 85 0.25000 0.21311 0.26826 0.23227 0.36735 0.33623 0.86817
LoOP 100 0.25000 0.21311 0.27454 0.23886 0.36000 0.32852 0.87602
LDOF 2 0.16667 0.12568 0.10608 0.06211 0.20833 0.16940 0.57411
LDOF 98 0.16667 0.12568 0.25315 0.21642 0.33333 0.30055 0.86407
LDOF 99 0.16667 0.12568 0.24993 0.21304 0.33962 0.30715 0.86373
LDOF 100 0.16667 0.12568 0.25008 0.21320 0.33962 0.30715 0.86510
ODIN 22 0.20000 0.16066 0.16071 0.11943 0.26923 0.23329 0.83214
ODIN 44 0.09524 0.05074 0.17257 0.13187 0.34146 0.30908 0.85605
ODIN 98 0.08333 0.03825 0.19638 0.15685 0.32836 0.29533 0.87551
ODIN 99 0.08333 0.03825 0.19992 0.16057 0.31884 0.28534 0.87415
FastABOD 19 0.08333 0.03825 0.25084 0.21400 0.34043 0.30799 0.86988
FastABOD 20 0.08333 0.03825 0.25372 0.21702 0.36364 0.33234 0.86783
FastABOD 66 0.25000 0.21311 0.25325 0.21652 0.35294 0.32112 0.85861
FastABOD 72 0.25000 0.21311 0.26237 0.22610 0.34615 0.31400 0.86373
KDEOS 3 0.08333 0.03825 0.08730 0.04241 0.19178 0.15203 0.68989
KDEOS 13 0.00000 -0.04918 0.10691 0.06299 0.24242 0.20517 0.74249
KDEOS 25 0.08333 0.03825 0.11637 0.07291 0.23729 0.19978 0.76981
KDEOS 100 0.00000 -0.04918 0.10058 0.05635 0.21978 0.18141 0.77459
LDF 38 0.41667 0.38798 0.38375 0.35344 0.50000 0.47541 0.91564
LDF 100 0.50000 0.47541 0.41722 0.38856 0.54545 0.52310 0.88046
INFLO 56 0.08333 0.03825 0.25963 0.22322 0.34043 0.30799 0.87500
INFLO 71 0.16667 0.12568 0.26102 0.22467 0.36735 0.33623 0.86851
INFLO 73 0.25000 0.21311 0.26245 0.22617 0.35294 0.32112 0.86749
INFLO 81 0.25000 0.21311 0.26563 0.22951 0.36000 0.32852 0.86817
COF 32 0.41667 0.38798 0.35820 0.32664 0.43478 0.40699 0.84734
COF 40 0.41667 0.38798 0.36500 0.33377 0.47619 0.45043 0.84290
COF 63 0.33333 0.30055 0.35792 0.32635 0.38095 0.35051 0.87466
COF 64 0.33333 0.30055 0.41877 0.39019 0.40000 0.37049 0.87329

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 256 objects, 12 outliers (4.69%)

Download raw algorithm results (2.3 MB) Download raw algorithm evaluation table (44.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.16667 0.12568 0.27192 0.23612 0.37209 0.34121 0.85895
KNN 5 0.16667 0.12568 0.28855 0.25356 0.41026 0.38125 0.85656
KNNW 1 0.16667 0.12568 0.26986 0.23395 0.32000 0.28656 0.85690
KNNW 2 0.16667 0.12568 0.27640 0.24081 0.36364 0.33234 0.86339
KNNW 9 0.16667 0.12568 0.28154 0.24621 0.41026 0.38125 0.85895
LOF 1 0.16667 0.12568 0.21002 0.17117 0.21918 0.18078 0.79030
LOF 8 0.16667 0.12568 0.24386 0.20667 0.30189 0.26755 0.82923
LOF 21 0.16667 0.12568 0.25883 0.22238 0.34783 0.31575 0.81694
LOF 24 0.16667 0.12568 0.21746 0.17897 0.37209 0.34121 0.81933
SimplifiedLOF 7 0.25000 0.21311 0.23045 0.19260 0.26667 0.23060 0.82001
SimplifiedLOF 10 0.16667 0.12568 0.23783 0.20035 0.33333 0.30055 0.83607
SimplifiedLOF 16 0.16667 0.12568 0.25328 0.21655 0.35556 0.32386 0.82719
SimplifiedLOF 20 0.16667 0.12568 0.25828 0.22180 0.34043 0.30799 0.82616
LoOP 7 0.25000 0.21311 0.22957 0.19168 0.27273 0.23696 0.82036
LoOP 10 0.16667 0.12568 0.24011 0.20274 0.33333 0.30055 0.84119
LoOP 16 0.16667 0.12568 0.24842 0.21145 0.35556 0.32386 0.82787
LoOP 20 0.16667 0.12568 0.25883 0.22238 0.34043 0.30799 0.82923
LDOF 16 0.16667 0.12568 0.20415 0.16501 0.30769 0.27364 0.83982
LDOF 17 0.25000 0.21311 0.19967 0.16031 0.32000 0.28656 0.83982
LDOF 37 0.25000 0.21311 0.24217 0.20490 0.35294 0.32112 0.82889
ODIN 22 0.22619 0.18813 0.16736 0.12641 0.25397 0.21728 0.80721
ODIN 37 0.16667 0.12568 0.20811 0.16917 0.34043 0.30799 0.82258
ODIN 46 0.16667 0.12568 0.22252 0.18428 0.34043 0.30799 0.81438
ODIN 62 0.16667 0.12568 0.21090 0.17209 0.36364 0.33234 0.81677
FastABOD 3 0.25000 0.21311 0.20069 0.16138 0.27273 0.23696 0.76639
FastABOD 5 0.16667 0.12568 0.27598 0.24037 0.29630 0.26169 0.86714
FastABOD 6 0.25000 0.21311 0.26909 0.23314 0.30769 0.27364 0.88456
FastABOD 16 0.16667 0.12568 0.26773 0.23172 0.32258 0.28926 0.86783
KDEOS 7 0.25000 0.21311 0.14961 0.10779 0.25000 0.21311 0.76400
KDEOS 12 0.16667 0.12568 0.23470 0.19707 0.28571 0.25059 0.84187
LDF 1 0.33333 0.30055 0.15708 0.11562 0.34783 0.31575 0.68374
LDF 10 0.25000 0.21311 0.29105 0.25618 0.33333 0.30055 0.83572
LDF 14 0.25000 0.21311 0.35544 0.32374 0.40000 0.37049 0.75717
INFLO 4 0.16667 0.12568 0.23473 0.19710 0.29167 0.25683 0.83743
INFLO 5 0.16667 0.12568 0.23899 0.20156 0.26471 0.22854 0.83333
INFLO 9 0.25000 0.21311 0.23557 0.19798 0.28571 0.25059 0.81762
INFLO 13 0.16667 0.12568 0.20656 0.16754 0.34783 0.31575 0.82514
COF 5 0.25000 0.21311 0.28413 0.24893 0.36364 0.33234 0.84597
COF 8 0.41667 0.38798 0.31067 0.27677 0.41667 0.38798 0.84426

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO