Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

HeartDisease (2% of outliers version#04)

A data set containing medical data on heart problems. Affected patients are considered outliers and healthy people are considered inliers.

Download all data set variants used (92.9 kB). You can also access the original data. (heart.dat)

Normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (31.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.05575 0.03686 0.12500 0.10750 0.77778
KNN 2 0.00000 -0.02000 0.06090 0.04212 0.12766 0.11021 0.80222
KNN 17 0.00000 -0.02000 0.06553 0.04684 0.13333 0.11600 0.79111
KNNW 1 0.00000 -0.02000 0.03929 0.02008 0.09524 0.07714 0.62111
KNNW 11 0.00000 -0.02000 0.05420 0.03529 0.12500 0.10750 0.77111
KNNW 36 0.00000 -0.02000 0.05690 0.03803 0.11765 0.10000 0.78444
LOF 2 0.33333 0.32000 0.14482 0.12772 0.33333 0.32000 0.63556
LOF 3 0.33333 0.32000 0.36644 0.35377 0.50000 0.49000 0.65556
LOF 5 0.33333 0.32000 0.37246 0.35991 0.50000 0.49000 0.82222
SimplifiedLOF 3 0.33333 0.32000 0.35144 0.33847 0.50000 0.49000 0.60000
SimplifiedLOF 5 0.33333 0.32000 0.36083 0.34804 0.50000 0.49000 0.74222
SimplifiedLOF 38 0.00000 -0.02000 0.09037 0.07218 0.22222 0.20667 0.81333
LoOP 3 0.33333 0.32000 0.18651 0.17024 0.40000 0.38800 0.62889
LoOP 4 0.33333 0.32000 0.35957 0.34676 0.50000 0.49000 0.72222
LoOP 5 0.33333 0.32000 0.36314 0.35041 0.50000 0.49000 0.76444
LoOP 39 0.00000 -0.02000 0.08148 0.06311 0.19048 0.17429 0.81333
LDOF 4 0.33333 0.32000 0.35252 0.33957 0.50000 0.49000 0.61556
LDOF 10 0.33333 0.32000 0.36543 0.35274 0.50000 0.49000 0.65111
LDOF 55 0.00000 -0.02000 0.07593 0.05745 0.20000 0.18400 0.78889
ODIN 4 0.14286 0.12571 0.07250 0.05395 0.20000 0.18400 0.74111
ODIN 14 0.00000 -0.02000 0.07660 0.05813 0.18182 0.16545 0.81889
ODIN 15 0.00000 -0.02000 0.07738 0.05893 0.19048 0.17429 0.80444
FastABOD 3 0.00000 -0.02000 0.04496 0.02585 0.11321 0.09547 0.72889
FastABOD 5 0.00000 -0.02000 0.06106 0.04228 0.13953 0.12233 0.80000
FastABOD 32 0.00000 -0.02000 0.05565 0.03676 0.15000 0.13300 0.77778
KDEOS 2 0.00000 -0.02000 0.01867 -0.00095 0.04959 0.03058 0.41556
KDEOS 65 0.00000 -0.02000 0.06062 0.04183 0.15385 0.13692 0.72222
KDEOS 97 0.00000 -0.02000 0.07284 0.05430 0.14286 0.12571 0.79333
KDEOS 100 0.00000 -0.02000 0.07264 0.05409 0.15385 0.13692 0.79556
LDF 1 0.33333 0.32000 0.18422 0.16790 0.40000 0.38800 0.59111
LDF 2 0.33333 0.32000 0.36444 0.35173 0.50000 0.49000 0.64444
LDF 14 0.00000 -0.02000 0.09894 0.08092 0.19048 0.17429 0.87333
INFLO 1 0.33333 0.32000 0.18717 0.17091 0.40000 0.38800 0.65111
INFLO 3 0.33333 0.32000 0.36137 0.34859 0.50000 0.49000 0.74889
INFLO 34 0.00000 -0.02000 0.08575 0.06747 0.20000 0.18400 0.84667
COF 3 0.33333 0.32000 0.13032 0.11293 0.33333 0.32000 0.57111
COF 6 0.33333 0.32000 0.19649 0.18042 0.40000 0.38800 0.75111
COF 52 0.00000 -0.02000 0.07224 0.05368 0.16000 0.14320 0.83111

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (37.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.01832 -0.00131 0.04380 0.02467 0.31222
KNN 4 0.00000 -0.02000 0.02494 0.00544 0.06154 0.04277 0.46333
KNN 17 0.00000 -0.02000 0.02452 0.00501 0.06250 0.04375 0.46667
KNN 40 0.00000 -0.02000 0.02388 0.00435 0.06186 0.04309 0.47778
KNNW 1 0.00000 -0.02000 0.02216 0.00260 0.04878 0.02976 0.39667
KNNW 22 0.00000 -0.02000 0.02357 0.00404 0.05797 0.03913 0.45333
KNNW 90 0.00000 -0.02000 0.02287 0.00333 0.05825 0.03942 0.45556
KNNW 98 0.00000 -0.02000 0.02295 0.00340 0.05941 0.04059 0.45556
LOF 1 0.00000 -0.02000 0.02669 0.00722 0.06452 0.04581 0.50111
SimplifiedLOF 1 0.00000 -0.02000 0.05328 0.03434 0.16667 0.15000 0.59333
LoOP 1 0.00000 -0.02000 0.05323 0.03430 0.16667 0.15000 0.59111
LDOF 2 0.00000 -0.02000 0.02559 0.00610 0.06061 0.04182 0.44000
LDOF 3 0.00000 -0.02000 0.04692 0.02786 0.14286 0.12571 0.50667
ODIN 1 0.01961 0.00000 0.01938 -0.00024 0.04110 0.02192 0.48444
ODIN 14 0.00000 -0.02000 0.02691 0.00745 0.06250 0.04375 0.48778
ODIN 32 0.00000 -0.02000 0.02720 0.00774 0.05714 0.03829 0.42111
FastABOD 3 0.00000 -0.02000 0.02140 0.00183 0.04839 0.02935 0.41556
FastABOD 4 0.00000 -0.02000 0.01917 -0.00044 0.05042 0.03143 0.34444
KDEOS 2 0.00000 -0.02000 0.01523 -0.00447 0.04054 0.02135 0.26444
KDEOS 100 0.00000 -0.02000 0.02334 0.00381 0.05714 0.03829 0.45111
LDF 1 0.00000 -0.02000 0.03488 0.01557 0.07692 0.05846 0.57000
INFLO 1 0.00000 -0.02000 0.01775 -0.00189 0.04054 0.02135 0.26111
INFLO 4 0.00000 -0.02000 0.03169 0.01233 0.08000 0.06160 0.50222
INFLO 5 0.00000 -0.02000 0.03215 0.01279 0.08000 0.06160 0.48444
COF 1 0.00000 -0.02000 0.04957 0.03056 0.15385 0.13692 0.59000
COF 49 0.00000 -0.02000 0.03294 0.01359 0.07547 0.05698 0.62000

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO