Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

HeartDisease (2% of outliers version#07)

A data set containing medical data on heart problems. Affected patients are considered outliers and healthy people are considered inliers.

Download all data set variants used (92.9 kB). You can also access the original data. (heart.dat)

Normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (31.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.04401 0.02489 0.09836 0.08033 0.68000
KNN 71 0.00000 -0.02000 0.07515 0.05665 0.14286 0.12571 0.81111
KNN 73 0.00000 -0.02000 0.07754 0.05909 0.15385 0.13692 0.80889
KNN 99 0.00000 -0.02000 0.06568 0.04700 0.16000 0.14320 0.76222
KNNW 1 0.00000 -0.02000 0.04153 0.02236 0.09091 0.07273 0.66556
KNNW 83 0.00000 -0.02000 0.05409 0.03517 0.12903 0.11161 0.76444
KNNW 100 0.00000 -0.02000 0.05443 0.03552 0.12500 0.10750 0.76667
LOF 3 0.33333 0.32000 0.12482 0.10732 0.33333 0.32000 0.47111
LOF 4 0.33333 0.32000 0.12505 0.10755 0.33333 0.32000 0.47333
LOF 64 0.00000 -0.02000 0.05879 0.03997 0.13333 0.11600 0.76222
SimplifiedLOF 4 0.33333 0.32000 0.34845 0.33542 0.50000 0.49000 0.52444
SimplifiedLOF 82 0.00000 -0.02000 0.05294 0.03400 0.11765 0.10000 0.77111
LoOP 3 0.33333 0.32000 0.17974 0.16333 0.40000 0.38800 0.49000
LoOP 4 0.33333 0.32000 0.34647 0.33340 0.50000 0.49000 0.50111
LoOP 95 0.00000 -0.02000 0.05493 0.03603 0.11429 0.09657 0.77778
LDOF 4 0.33333 0.32000 0.13367 0.11634 0.33333 0.32000 0.68000
LDOF 11 0.33333 0.32000 0.34700 0.33394 0.50000 0.49000 0.47333
LDOF 23 0.33333 0.32000 0.34888 0.33586 0.50000 0.49000 0.53556
LDOF 50 0.00000 -0.02000 0.07262 0.05407 0.16216 0.14541 0.83778
ODIN 12 0.33333 0.32000 0.13949 0.12228 0.33333 0.32000 0.76778
ODIN 18 0.22222 0.20667 0.12500 0.10750 0.28571 0.27143 0.79778
FastABOD 3 0.00000 -0.02000 0.05189 0.03293 0.11111 0.09333 0.73111
FastABOD 9 0.00000 -0.02000 0.04987 0.03086 0.12500 0.10750 0.75556
FastABOD 33 0.00000 -0.02000 0.05727 0.03842 0.12000 0.10240 0.78889
FastABOD 94 0.00000 -0.02000 0.05963 0.04082 0.11765 0.10000 0.78667
KDEOS 2 0.00000 -0.02000 0.03000 0.01060 0.05825 0.03942 0.61333
KDEOS 50 0.00000 -0.02000 0.10541 0.08752 0.28571 0.27143 0.67111
KDEOS 53 0.00000 -0.02000 0.10611 0.08823 0.28571 0.27143 0.68000
KDEOS 100 0.00000 -0.02000 0.05159 0.03263 0.10526 0.08737 0.73778
LDF 1 0.33333 0.32000 0.18083 0.16445 0.40000 0.38800 0.49333
LDF 7 0.00000 -0.02000 0.09850 0.08047 0.25000 0.23500 0.88000
LDF 16 0.00000 -0.02000 0.19270 0.17656 0.44444 0.43333 0.84000
INFLO 2 0.33333 0.32000 0.12607 0.10859 0.33333 0.32000 0.55778
INFLO 4 0.33333 0.32000 0.35089 0.33791 0.50000 0.49000 0.55778
INFLO 52 0.00000 -0.02000 0.07311 0.05458 0.14634 0.12927 0.83333
COF 4 0.33333 0.32000 0.18096 0.16458 0.40000 0.38800 0.49111
COF 63 0.00000 -0.02000 0.06788 0.04923 0.14286 0.12571 0.82444

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (24.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.12753 0.11008 0.28571 0.27143 0.79667
KNN 18 0.00000 -0.02000 0.15156 0.13460 0.28571 0.27143 0.84444
KNN 84 0.00000 -0.02000 0.10322 0.08528 0.21053 0.19474 0.88222
KNNW 1 0.33333 0.32000 0.14371 0.12658 0.33333 0.32000 0.76000
KNNW 2 0.33333 0.32000 0.21048 0.19469 0.40000 0.38800 0.79556
KNNW 93 0.00000 -0.02000 0.12112 0.10355 0.25000 0.23500 0.86889
LOF 1 0.33333 0.32000 0.15092 0.13394 0.33333 0.32000 0.82000
LOF 82 0.00000 -0.02000 0.12063 0.10305 0.22222 0.20667 0.88667
SimplifiedLOF 4 0.33333 0.32000 0.13596 0.11868 0.33333 0.32000 0.70889
SimplifiedLOF 98 0.00000 -0.02000 0.10889 0.09107 0.21053 0.19474 0.84889
LoOP 4 0.33333 0.32000 0.13755 0.12030 0.33333 0.32000 0.71556
LoOP 96 0.00000 -0.02000 0.11645 0.09878 0.22222 0.20667 0.84889
LDOF 2 0.00000 -0.02000 0.04838 0.02935 0.13333 0.11600 0.67333
LDOF 8 0.00000 -0.02000 0.12454 0.10703 0.25000 0.23500 0.88222
LDOF 23 0.00000 -0.02000 0.17410 0.15758 0.40000 0.38800 0.80444
ODIN 16 0.33333 0.32000 0.14444 0.12733 0.33333 0.32000 0.72333
ODIN 78 0.33333 0.32000 0.16364 0.14691 0.33333 0.32000 0.85889
ODIN 100 0.00000 -0.02000 0.13810 0.12086 0.25000 0.23500 0.87889
FastABOD 3 0.00000 -0.02000 0.08050 0.06211 0.19048 0.17429 0.78000
FastABOD 4 0.00000 -0.02000 0.13949 0.12228 0.28571 0.27143 0.81556
FastABOD 7 0.00000 -0.02000 0.15074 0.13376 0.28571 0.27143 0.83778
FastABOD 74 0.00000 -0.02000 0.13384 0.11652 0.26667 0.25200 0.87556
KDEOS 11 0.33333 0.32000 0.22348 0.20795 0.40000 0.38800 0.82667
KDEOS 19 0.66667 0.66000 0.56536 0.55667 0.66667 0.66000 0.77778
LDF 1 0.33333 0.32000 0.22473 0.20922 0.40000 0.38800 0.87778
LDF 53 0.00000 -0.02000 0.12747 0.11002 0.25000 0.23500 0.89111
INFLO 3 0.33333 0.32000 0.14750 0.13045 0.33333 0.32000 0.65556
INFLO 93 0.00000 -0.02000 0.10067 0.08269 0.19048 0.17429 0.87778
COF 4 0.33333 0.32000 0.13053 0.11314 0.33333 0.32000 0.62889
COF 25 0.33333 0.32000 0.19159 0.17542 0.40000 0.38800 0.71111
COF 61 0.33333 0.32000 0.21374 0.19802 0.40000 0.38800 0.84444

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO