Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (2% of outliers version#02)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 248 objects, 4 outliers (1.61%)

Download raw algorithm results (2.2 MB) Download raw algorithm evaluation table (35.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.23770 0.26779 0.25579 0.40000 0.39016 0.58709
KNN 11 0.25000 0.23770 0.27051 0.25855 0.40000 0.39016 0.65984
KNNW 1 0.25000 0.23770 0.26954 0.25756 0.40000 0.39016 0.61680
KNNW 25 0.25000 0.23770 0.26895 0.25696 0.40000 0.39016 0.62398
LOF 1 0.25000 0.23770 0.13922 0.12511 0.33333 0.32240 0.52510
LOF 3 0.25000 0.23770 0.26145 0.24934 0.40000 0.39016 0.41496
LOF 57 0.25000 0.23770 0.26967 0.25769 0.40000 0.39016 0.64242
SimplifiedLOF 1 0.25000 0.23770 0.26101 0.24889 0.40000 0.39016 0.38730
SimplifiedLOF 99 0.25000 0.23770 0.26725 0.25524 0.40000 0.39016 0.59734
LoOP 1 0.25000 0.23770 0.26101 0.24889 0.40000 0.39016 0.38730
LoOP 57 0.25000 0.23770 0.26566 0.25362 0.40000 0.39016 0.57070
LoOP 95 0.25000 0.23770 0.26577 0.25374 0.40000 0.39016 0.57070
LDOF 2 0.25000 0.23770 0.26233 0.25024 0.40000 0.39016 0.45799
LDOF 97 0.25000 0.23770 0.26538 0.25334 0.40000 0.39016 0.55943
LDOF 98 0.25000 0.23770 0.26538 0.25334 0.40000 0.39016 0.56148
ODIN 66 0.25000 0.23770 0.07873 0.06363 0.25000 0.23770 0.58658
ODIN 72 0.25000 0.23770 0.09984 0.08508 0.28571 0.27400 0.59324
ODIN 91 0.25000 0.23770 0.10142 0.08669 0.28571 0.27400 0.62039
FastABOD 4 0.25000 0.23770 0.13686 0.12271 0.33333 0.32240 0.43340
FastABOD 5 0.25000 0.23770 0.26284 0.25076 0.40000 0.39016 0.46414
FastABOD 11 0.25000 0.23770 0.26735 0.25534 0.40000 0.39016 0.57582
FastABOD 26 0.25000 0.23770 0.26734 0.25533 0.40000 0.39016 0.58504
KDEOS 8 0.25000 0.23770 0.09628 0.08147 0.28571 0.27400 0.48258
KDEOS 9 0.25000 0.23770 0.13858 0.12446 0.33333 0.32240 0.50615
KDEOS 10 0.25000 0.23770 0.14040 0.12631 0.33333 0.32240 0.55328
KDEOS 11 0.25000 0.23770 0.14075 0.12666 0.33333 0.32240 0.53689
LDF 6 0.37500 0.36475 0.36760 0.35724 0.44444 0.43534 0.71568
LDF 7 0.50000 0.49180 0.27715 0.26530 0.50000 0.49180 0.73924
LDF 45 0.25000 0.23770 0.17964 0.16619 0.33333 0.32240 0.82480
INFLO 1 0.25000 0.23770 0.13490 0.12072 0.33333 0.32240 0.32172
INFLO 4 0.25000 0.23770 0.26194 0.24984 0.40000 0.39016 0.43955
INFLO 35 0.25000 0.23770 0.26516 0.25311 0.40000 0.39016 0.56045
INFLO 97 0.25000 0.23770 0.26578 0.25374 0.40000 0.39016 0.53791
COF 1 0.25000 0.23770 0.26101 0.24889 0.40000 0.39016 0.38730
COF 47 0.25000 0.23770 0.30761 0.29626 0.40000 0.39016 0.73668
COF 86 0.25000 0.23770 0.29411 0.28254 0.40000 0.39016 0.75205

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 248 objects, 4 outliers (1.61%)

Download raw algorithm results (2.2 MB) Download raw algorithm evaluation table (36.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.23770 0.13998 0.12588 0.33333 0.32240 0.53381
KNN 3 0.25000 0.23770 0.10211 0.08739 0.28571 0.27400 0.64242
KNNW 1 0.25000 0.23770 0.13856 0.12444 0.33333 0.32240 0.50666
KNNW 2 0.25000 0.23770 0.13881 0.12469 0.33333 0.32240 0.51332
KNNW 12 0.25000 0.23770 0.09963 0.08487 0.28571 0.27400 0.58094
LOF 1 0.25000 0.23770 0.13807 0.12394 0.33333 0.32240 0.48002
LOF 13 0.25000 0.23770 0.10264 0.08793 0.28571 0.27400 0.63832
SimplifiedLOF 1 0.25000 0.23770 0.07519 0.06003 0.25000 0.23770 0.49949
SimplifiedLOF 5 0.25000 0.23770 0.09638 0.08157 0.28571 0.27400 0.48463
SimplifiedLOF 16 0.25000 0.23770 0.10035 0.08560 0.28571 0.27400 0.59734
LoOP 1 0.25000 0.23770 0.07519 0.06003 0.25000 0.23770 0.49949
LoOP 5 0.25000 0.23770 0.09631 0.08150 0.28571 0.27400 0.48156
LoOP 16 0.25000 0.23770 0.09991 0.08515 0.28571 0.27400 0.58811
LDOF 12 0.25000 0.23770 0.07506 0.05990 0.25000 0.23770 0.46107
LDOF 20 0.25000 0.23770 0.09643 0.08162 0.28571 0.27400 0.48873
LDOF 96 0.25000 0.23770 0.09895 0.08418 0.28571 0.27400 0.56250
LDOF 97 0.25000 0.23770 0.09896 0.08419 0.28571 0.27400 0.56148
ODIN 24 0.25000 0.23770 0.07796 0.06285 0.25000 0.23770 0.56814
ODIN 46 0.25000 0.23770 0.09877 0.08400 0.28571 0.27400 0.57121
ODIN 58 0.25000 0.23770 0.09991 0.08516 0.28571 0.27400 0.60400
ODIN 72 0.25000 0.23770 0.10103 0.08629 0.28571 0.27400 0.58248
FastABOD 3 0.25000 0.23770 0.07903 0.06393 0.25000 0.23770 0.56045
FastABOD 7 0.25000 0.23770 0.09879 0.08402 0.28571 0.27400 0.56455
FastABOD 86 0.25000 0.23770 0.10054 0.08580 0.28571 0.27400 0.59529
FastABOD 100 0.25000 0.23770 0.10043 0.08568 0.28571 0.27400 0.59631
KDEOS 2 0.00000 -0.01639 0.02786 0.01193 0.10526 0.09060 0.46619
KDEOS 12 0.00000 -0.01639 0.03807 0.02230 0.14286 0.12881 0.47848
KDEOS 97 0.00000 -0.01639 0.03516 0.01934 0.11765 0.10318 0.56762
KDEOS 100 0.00000 -0.01639 0.04056 0.02483 0.14286 0.12881 0.56148
LDF 3 0.00000 -0.01639 0.09525 0.08042 0.20000 0.18689 0.85656
LDF 55 0.25000 0.23770 0.07427 0.05910 0.25000 0.23770 0.42623
LDF 58 0.25000 0.23770 0.13612 0.12196 0.33333 0.32240 0.39652
INFLO 1 0.25000 0.23770 0.13755 0.12341 0.33333 0.32240 0.46107
INFLO 100 0.25000 0.23770 0.10378 0.08909 0.28571 0.27400 0.67418
COF 1 0.25000 0.23770 0.07519 0.06003 0.25000 0.23770 0.49949
COF 5 0.25000 0.23770 0.14030 0.12621 0.33333 0.32240 0.56352
COF 31 0.25000 0.23770 0.14741 0.13343 0.33333 0.32240 0.61783
COF 34 0.25000 0.23770 0.15182 0.13792 0.33333 0.32240 0.57992

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO