Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

HeartDisease (5% of outliers version#10)

A data set containing medical data on heart problems. Affected patients are considered outliers and healthy people are considered inliers.

Download all data set variants used (92.9 kB). You can also access the original data. (heart.dat)

Normalized, without duplicates

This version contains 13 attributes, 157 objects, 7 outliers (4.46%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (41.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 45 0.42857 0.40190 0.29311 0.26013 0.47059 0.44588 0.90381
KNN 50 0.57143 0.55143 0.33310 0.30198 0.57143 0.55143 0.90286
KNN 54 0.57143 0.55143 0.34156 0.31083 0.57143 0.55143 0.89810
KNNW 75 0.42857 0.40190 0.26047 0.22595 0.44444 0.41852 0.88000
KNNW 83 0.42857 0.40190 0.26566 0.23139 0.47059 0.44588 0.88095
LOF 71 0.42857 0.40190 0.29271 0.25971 0.44444 0.41852 0.89524
LOF 90 0.42857 0.40190 0.29130 0.25823 0.44444 0.41852 0.90286
LOF 96 0.42857 0.40190 0.29541 0.26252 0.47059 0.44588 0.90095
LOF 100 0.42857 0.40190 0.30027 0.26762 0.47059 0.44588 0.90190
SimplifiedLOF 95 0.14286 0.10286 0.19366 0.15603 0.36364 0.33394 0.83810
SimplifiedLOF 96 0.28571 0.25238 0.20379 0.16664 0.34783 0.31739 0.84095
SimplifiedLOF 98 0.28571 0.25238 0.20634 0.16930 0.34783 0.31739 0.84095
LoOP 92 0.28571 0.25238 0.20387 0.16671 0.36364 0.33394 0.83905
LoOP 94 0.28571 0.25238 0.21053 0.17369 0.38095 0.35206 0.84095
LoOP 98 0.28571 0.25238 0.21792 0.18142 0.38095 0.35206 0.84286
LoOP 100 0.28571 0.25238 0.21816 0.18167 0.38095 0.35206 0.84190
LDOF 2 0.14286 0.10286 0.24208 0.20671 0.30000 0.26733 0.67619
LDOF 98 0.14286 0.10286 0.15965 0.12044 0.30769 0.27538 0.80952
ODIN 70 0.28571 0.25238 0.19501 0.15744 0.36364 0.33394 0.84667
ODIN 94 0.28571 0.25238 0.27868 0.24502 0.44444 0.41852 0.88286
ODIN 99 0.28571 0.25238 0.26152 0.22706 0.41667 0.38944 0.89048
FastABOD 30 0.28571 0.25238 0.17998 0.14171 0.30000 0.26733 0.84857
FastABOD 80 0.28571 0.25238 0.23051 0.19460 0.37500 0.34583 0.87810
FastABOD 86 0.28571 0.25238 0.25624 0.22153 0.37500 0.34583 0.88095
FastABOD 92 0.28571 0.25238 0.23996 0.20450 0.37500 0.34583 0.88667
KDEOS 3 0.14286 0.10286 0.20736 0.17037 0.25000 0.21500 0.61810
KDEOS 99 0.00000 -0.04667 0.07677 0.03369 0.16393 0.12492 0.69619
LDF 28 0.42857 0.40190 0.29029 0.25717 0.50000 0.47667 0.90095
LDF 36 0.42857 0.40190 0.38264 0.35383 0.55556 0.53481 0.92476
LDF 66 0.42857 0.40190 0.43976 0.41361 0.48276 0.45862 0.94667
INFLO 64 0.28571 0.25238 0.18302 0.14490 0.34783 0.31739 0.82762
INFLO 68 0.28571 0.25238 0.20409 0.16694 0.34783 0.31739 0.84571
INFLO 70 0.28571 0.25238 0.19572 0.15818 0.40000 0.37200 0.76000
INFLO 91 0.28571 0.25238 0.22979 0.19385 0.40000 0.37200 0.77143
COF 55 0.28571 0.25238 0.28157 0.24804 0.41176 0.38431 0.92667
COF 81 0.57143 0.55143 0.47460 0.45008 0.57143 0.55143 0.92000
COF 87 0.57143 0.55143 0.54362 0.52232 0.61538 0.59744 0.91333
COF 100 0.57143 0.55143 0.56998 0.54992 0.61538 0.59744 0.91619

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 13 attributes, 157 objects, 7 outliers (4.46%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (40.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04667 0.08210 0.03926 0.15584 0.11645 0.68381
KNN 10 0.00000 -0.04667 0.09967 0.05765 0.23077 0.19487 0.70667
KNNW 1 0.00000 -0.04667 0.07401 0.03080 0.16216 0.12306 0.68286
KNNW 17 0.00000 -0.04667 0.08645 0.04382 0.17143 0.13276 0.69619
KNNW 19 0.00000 -0.04667 0.08682 0.04420 0.17143 0.13276 0.69333
KNNW 90 0.00000 -0.04667 0.08528 0.04259 0.20513 0.16803 0.64667
LOF 1 0.00000 -0.04667 0.05504 0.01094 0.11429 0.07295 0.53190
LOF 20 0.00000 -0.04667 0.08769 0.04512 0.18519 0.14716 0.69048
LOF 22 0.00000 -0.04667 0.08992 0.04745 0.19048 0.15270 0.68000
LOF 27 0.00000 -0.04667 0.08920 0.04670 0.20000 0.16267 0.65524
SimplifiedLOF 1 0.00000 -0.04667 0.06182 0.01804 0.15000 0.11033 0.62381
SimplifiedLOF 3 0.00000 -0.04667 0.07528 0.03213 0.16000 0.12080 0.63714
SimplifiedLOF 18 0.00000 -0.04667 0.07589 0.03276 0.19355 0.15591 0.60667
SimplifiedLOF 82 0.00000 -0.04667 0.07869 0.03570 0.18182 0.14364 0.63333
LoOP 1 0.00000 -0.04667 0.06169 0.01791 0.15000 0.11033 0.62190
LoOP 3 0.00000 -0.04667 0.08194 0.03910 0.18182 0.14364 0.63571
LoOP 17 0.00000 -0.04667 0.07678 0.03370 0.20000 0.16267 0.56667
LoOP 85 0.00000 -0.04667 0.07826 0.03524 0.18182 0.14364 0.65286
LDOF 2 0.00000 -0.04667 0.09457 0.05232 0.22857 0.19257 0.71429
LDOF 3 0.14286 0.10286 0.13634 0.09603 0.21053 0.17368 0.73333
ODIN 7 0.14286 0.10286 0.06179 0.01800 0.14286 0.10286 0.50333
ODIN 34 0.00000 -0.04667 0.07363 0.03040 0.15385 0.11436 0.61857
ODIN 43 0.00000 -0.04667 0.08405 0.04130 0.18182 0.14364 0.60190
ODIN 82 0.00000 -0.04667 0.07721 0.03415 0.19512 0.15756 0.61333
FastABOD 3 0.14286 0.10286 0.10917 0.06759 0.20000 0.16267 0.71810
KDEOS 4 0.14286 0.10286 0.10433 0.06254 0.20000 0.16267 0.64381
KDEOS 7 0.14286 0.10286 0.08727 0.04467 0.21053 0.17368 0.57714
LDF 5 0.14286 0.10286 0.07749 0.03444 0.15385 0.11436 0.55333
LDF 14 0.00000 -0.04667 0.10427 0.06247 0.22222 0.18593 0.69238
LDF 20 0.00000 -0.04667 0.10242 0.06053 0.20408 0.16694 0.73429
INFLO 1 0.00000 -0.04667 0.08286 0.04006 0.18519 0.14716 0.70857
INFLO 11 0.00000 -0.04667 0.08313 0.04034 0.17647 0.13804 0.55619
INFLO 98 0.00000 -0.04667 0.07705 0.03398 0.20000 0.16267 0.62238
COF 1 0.00000 -0.04667 0.06079 0.01696 0.14815 0.10840 0.61857
COF 14 0.00000 -0.04667 0.10404 0.06223 0.26667 0.23244 0.71810
COF 74 0.00000 -0.04667 0.10459 0.06280 0.22222 0.18593 0.76476
COF 85 0.00000 -0.04667 0.11287 0.07147 0.22642 0.19031 0.74476

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO