Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (2% of outliers version#08)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 248 objects, 4 outliers (1.61%)

Download raw algorithm results (2.2 MB) Download raw algorithm evaluation table (34.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.23770 0.29253 0.28094 0.40000 0.39016 0.83299
KNN 2 0.25000 0.23770 0.29329 0.28170 0.40000 0.39016 0.84016
KNN 5 0.25000 0.23770 0.29436 0.28279 0.40000 0.39016 0.82787
KNNW 1 0.25000 0.23770 0.29691 0.28538 0.40000 0.39016 0.85041
LOF 1 0.25000 0.23770 0.27339 0.26148 0.40000 0.39016 0.67418
LOF 2 0.25000 0.23770 0.29548 0.28393 0.40000 0.39016 0.85758
SimplifiedLOF 1 0.25000 0.23770 0.14501 0.13100 0.33333 0.32240 0.64549
SimplifiedLOF 2 0.25000 0.23770 0.29018 0.27855 0.40000 0.39016 0.84016
LoOP 1 0.25000 0.23770 0.14501 0.13100 0.33333 0.32240 0.64549
LoOP 2 0.25000 0.23770 0.28380 0.27205 0.40000 0.39016 0.80738
LoOP 78 0.25000 0.23770 0.28728 0.27559 0.40000 0.39016 0.80020
LDOF 4 0.25000 0.23770 0.10596 0.09130 0.28571 0.27400 0.70594
LDOF 5 0.25000 0.23770 0.27103 0.25908 0.40000 0.39016 0.68648
LDOF 10 0.25000 0.23770 0.28427 0.27253 0.40000 0.39016 0.79918
LDOF 92 0.25000 0.23770 0.28558 0.27387 0.40000 0.39016 0.79098
ODIN 72 0.25000 0.23770 0.11585 0.10136 0.28571 0.27400 0.78227
ODIN 82 0.25000 0.23770 0.15889 0.14511 0.33333 0.32240 0.78535
ODIN 90 0.25000 0.23770 0.16096 0.14721 0.33333 0.32240 0.78689
ODIN 93 0.25000 0.23770 0.16063 0.14687 0.33333 0.32240 0.79150
FastABOD 5 0.25000 0.23770 0.11285 0.09830 0.28571 0.27400 0.77254
FastABOD 7 0.25000 0.23770 0.28695 0.27526 0.40000 0.39016 0.82275
FastABOD 9 0.25000 0.23770 0.29247 0.28087 0.40000 0.39016 0.84734
FastABOD 77 0.25000 0.23770 0.29386 0.28228 0.40000 0.39016 0.81660
KDEOS 2 0.00000 -0.01639 0.01590 -0.00023 0.03653 0.02074 0.41957
KDEOS 10 0.00000 -0.01639 0.03188 0.01601 0.07143 0.05621 0.69877
KDEOS 26 0.00000 -0.01639 0.03194 0.01607 0.06742 0.05213 0.68340
KDEOS 28 0.00000 -0.01639 0.03127 0.01539 0.08163 0.06658 0.66291
LDF 24 0.00000 -0.01639 0.17388 0.16033 0.40000 0.39016 0.81762
LDF 45 0.25000 0.23770 0.12079 0.10637 0.25000 0.23770 0.87295
LDF 47 0.25000 0.23770 0.13678 0.12262 0.25000 0.23770 0.90369
LDF 73 0.25000 0.23770 0.32706 0.31603 0.40000 0.39016 0.83197
INFLO 1 0.25000 0.23770 0.26686 0.25484 0.40000 0.39016 0.59836
INFLO 2 0.25000 0.23770 0.29086 0.27924 0.40000 0.39016 0.83504
COF 1 0.25000 0.23770 0.14501 0.13100 0.33333 0.32240 0.64549
COF 3 0.25000 0.23770 0.30360 0.29219 0.40000 0.39016 0.88115
COF 7 0.25000 0.23770 0.32470 0.31363 0.40000 0.39016 0.91393
COF 94 0.25000 0.23770 0.32667 0.31563 0.40000 0.39016 0.85861

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 248 objects, 4 outliers (1.61%)

Download raw algorithm results (2.2 MB) Download raw algorithm evaluation table (34.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.23770 0.27756 0.26572 0.40000 0.39016 0.68135
KNN 6 0.25000 0.23770 0.28383 0.27209 0.40000 0.39016 0.70902
KNN 57 0.25000 0.23770 0.16229 0.14856 0.33333 0.32240 0.72951
KNNW 1 0.25000 0.23770 0.27725 0.26541 0.40000 0.39016 0.71209
KNNW 81 0.25000 0.23770 0.28600 0.27430 0.40000 0.39016 0.72029
LOF 1 0.25000 0.23770 0.29398 0.28241 0.40000 0.39016 0.80430
SimplifiedLOF 1 0.25000 0.23770 0.28522 0.27350 0.40000 0.39016 0.77664
SimplifiedLOF 5 0.25000 0.23770 0.28248 0.27072 0.40000 0.39016 0.78996
LoOP 1 0.25000 0.23770 0.28522 0.27350 0.40000 0.39016 0.77664
LoOP 5 0.25000 0.23770 0.28138 0.26960 0.40000 0.39016 0.78484
LDOF 2 0.25000 0.23770 0.11992 0.10549 0.28571 0.27400 0.67418
LDOF 4 0.25000 0.23770 0.27799 0.26616 0.40000 0.39016 0.76230
LDOF 5 0.25000 0.23770 0.29192 0.28031 0.40000 0.39016 0.83914
ODIN 9 0.05000 0.03443 0.04023 0.02449 0.09524 0.08041 0.73770
ODIN 24 0.25000 0.23770 0.08885 0.07392 0.25000 0.23770 0.70953
ODIN 47 0.25000 0.23770 0.11202 0.09746 0.28571 0.27400 0.72439
ODIN 100 0.25000 0.23770 0.11471 0.10020 0.28571 0.27400 0.71721
FastABOD 3 0.25000 0.23770 0.27009 0.25812 0.40000 0.39016 0.57889
FastABOD 4 0.25000 0.23770 0.29932 0.28784 0.40000 0.39016 0.70594
FastABOD 38 0.25000 0.23770 0.28664 0.27495 0.40000 0.39016 0.76639
KDEOS 2 0.00000 -0.01639 0.03571 0.01991 0.11111 0.09654 0.69518
KDEOS 3 0.00000 -0.01639 0.07065 0.05541 0.22222 0.20947 0.67008
KDEOS 20 0.00000 -0.01639 0.06225 0.04688 0.14286 0.12881 0.76127
LDF 2 0.00000 -0.01639 0.06434 0.04901 0.15385 0.13997 0.82070
LDF 94 0.50000 0.49180 0.22690 0.21423 0.50000 0.49180 0.78484
LDF 99 0.50000 0.49180 0.43127 0.42195 0.57143 0.56440 0.74385
INFLO 1 0.25000 0.23770 0.28953 0.27788 0.40000 0.39016 0.75717
INFLO 20 0.25000 0.23770 0.28774 0.27607 0.40000 0.39016 0.79098
COF 1 0.25000 0.23770 0.28522 0.27350 0.40000 0.39016 0.77664
COF 100 0.25000 0.23770 0.30547 0.29409 0.40000 0.39016 0.79508

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO