Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

HeartDisease (5% of outliers version#02)

A data set containing medical data on heart problems. Affected patients are considered outliers and healthy people are considered inliers.

Download all data set variants used (92.9 kB). You can also access the original data. (heart.dat)

Normalized, without duplicates

This version contains 13 attributes, 157 objects, 7 outliers (4.46%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (38.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.28571 0.25238 0.36113 0.33132 0.44444 0.41852 0.77095
KNN 2 0.28571 0.25238 0.37793 0.34890 0.44444 0.41852 0.79810
KNN 15 0.28571 0.25238 0.35261 0.32240 0.40000 0.37200 0.84667
KNN 48 0.42857 0.40190 0.33497 0.30394 0.42857 0.40190 0.81429
KNNW 1 0.28571 0.25238 0.34335 0.31271 0.44444 0.41852 0.65095
KNNW 9 0.28571 0.25238 0.35877 0.32884 0.44444 0.41852 0.78000
KNNW 30 0.28571 0.25238 0.34283 0.31217 0.40000 0.37200 0.82857
LOF 2 0.28571 0.25238 0.15312 0.11360 0.33333 0.30222 0.69333
LOF 27 0.28571 0.25238 0.32457 0.29305 0.40000 0.37200 0.78286
LOF 29 0.28571 0.25238 0.33574 0.30474 0.40000 0.37200 0.79714
LOF 57 0.28571 0.25238 0.32069 0.28899 0.34483 0.31425 0.81143
SimplifiedLOF 3 0.28571 0.25238 0.16860 0.12980 0.33333 0.30222 0.68952
SimplifiedLOF 72 0.28571 0.25238 0.30083 0.26820 0.33333 0.30222 0.82286
SimplifiedLOF 78 0.28571 0.25238 0.31121 0.27907 0.36364 0.33394 0.82095
SimplifiedLOF 86 0.28571 0.25238 0.31306 0.28100 0.36364 0.33394 0.82000
LoOP 3 0.28571 0.25238 0.17692 0.13851 0.30769 0.27538 0.71524
LoOP 78 0.28571 0.25238 0.31435 0.28236 0.36364 0.33394 0.82286
LoOP 79 0.28571 0.25238 0.31536 0.28341 0.36364 0.33394 0.82667
LDOF 4 0.28571 0.25238 0.20735 0.17036 0.33333 0.30222 0.67238
LDOF 5 0.14286 0.10286 0.14570 0.10583 0.35294 0.32275 0.67048
LDOF 99 0.28571 0.25238 0.30442 0.27196 0.33333 0.30222 0.81810
ODIN 7 0.42857 0.40190 0.25053 0.21555 0.42857 0.40190 0.80143
ODIN 12 0.42857 0.40190 0.38060 0.35170 0.54545 0.52424 0.78143
ODIN 64 0.28571 0.25238 0.36009 0.33023 0.40000 0.37200 0.82667
FastABOD 3 0.28571 0.25238 0.19279 0.15512 0.28571 0.25238 0.79143
FastABOD 5 0.28571 0.25238 0.37530 0.34615 0.44444 0.41852 0.82000
FastABOD 99 0.28571 0.25238 0.37722 0.34815 0.44444 0.41852 0.82381
KDEOS 4 0.14286 0.10286 0.10965 0.06810 0.22222 0.18593 0.53619
KDEOS 100 0.00000 -0.04667 0.12094 0.07992 0.26667 0.23244 0.77048
LDF 44 0.14286 0.10286 0.32124 0.28957 0.40000 0.37200 0.82286
LDF 49 0.42857 0.40190 0.34202 0.31131 0.42857 0.40190 0.82095
LDF 64 0.42857 0.40190 0.34448 0.31389 0.46154 0.43641 0.77810
LDF 75 0.42857 0.40190 0.34688 0.31640 0.46154 0.43641 0.77333
INFLO 10 0.28571 0.25238 0.19779 0.16035 0.30000 0.26733 0.71048
INFLO 77 0.14286 0.10286 0.30523 0.27280 0.35714 0.32714 0.86762
INFLO 78 0.28571 0.25238 0.30986 0.27765 0.35714 0.32714 0.86762
COF 7 0.42857 0.40190 0.21828 0.18180 0.42857 0.40190 0.69429
COF 51 0.42857 0.40190 0.34061 0.30984 0.46154 0.43641 0.76286
COF 70 0.14286 0.10286 0.27944 0.24582 0.30303 0.27051 0.82190

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 13 attributes, 157 objects, 7 outliers (4.46%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (37.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.28571 0.25238 0.18463 0.14658 0.33333 0.30222 0.72333
KNN 94 0.00000 -0.04667 0.11904 0.07793 0.22222 0.18593 0.80381
KNNW 1 0.28571 0.25238 0.22502 0.18885 0.36364 0.33394 0.78000
KNNW 2 0.28571 0.25238 0.24825 0.21317 0.40000 0.37200 0.75143
KNNW 90 0.14286 0.10286 0.13380 0.09338 0.26667 0.23244 0.78667
LOF 6 0.00000 -0.04667 0.13920 0.09903 0.33333 0.30222 0.76095
LOF 18 0.28571 0.25238 0.13169 0.09117 0.28571 0.25238 0.68571
LOF 42 0.28571 0.25238 0.14944 0.10975 0.30769 0.27538 0.78095
LOF 96 0.00000 -0.04667 0.12373 0.08284 0.23077 0.19487 0.80857
SimplifiedLOF 3 0.14286 0.10286 0.16548 0.12654 0.24242 0.20707 0.73048
SimplifiedLOF 6 0.14286 0.10286 0.14560 0.10573 0.31579 0.28386 0.73143
SimplifiedLOF 8 0.28571 0.25238 0.13752 0.09727 0.28571 0.25238 0.71048
SimplifiedLOF 96 0.14286 0.10286 0.12753 0.08681 0.23529 0.19961 0.78190
LoOP 3 0.14286 0.10286 0.16383 0.12481 0.22222 0.18593 0.72048
LoOP 6 0.14286 0.10286 0.14149 0.10143 0.32000 0.28827 0.71714
LoOP 9 0.28571 0.25238 0.13424 0.09384 0.28571 0.25238 0.70476
LoOP 97 0.14286 0.10286 0.12608 0.08529 0.23529 0.19961 0.77905
LDOF 7 0.00000 -0.04667 0.12744 0.08672 0.30000 0.26733 0.69714
LDOF 20 0.28571 0.25238 0.11932 0.07822 0.28571 0.25238 0.67810
LDOF 27 0.28571 0.25238 0.13089 0.09033 0.28571 0.25238 0.72000
LDOF 97 0.00000 -0.04667 0.11628 0.07504 0.23529 0.19961 0.77143
ODIN 13 0.28571 0.25238 0.12274 0.08180 0.28571 0.25238 0.67048
ODIN 27 0.28571 0.25238 0.15558 0.11617 0.33333 0.30222 0.67333
ODIN 29 0.28571 0.25238 0.16489 0.12592 0.33333 0.30222 0.67429
ODIN 100 0.14286 0.10286 0.14044 0.10033 0.22951 0.19355 0.80952
FastABOD 4 0.28571 0.25238 0.18611 0.14813 0.33333 0.30222 0.75429
FastABOD 5 0.28571 0.25238 0.20410 0.16695 0.36364 0.33394 0.76000
FastABOD 6 0.28571 0.25238 0.18999 0.15219 0.33333 0.30222 0.77048
FastABOD 15 0.28571 0.25238 0.20816 0.17121 0.31579 0.28386 0.76190
KDEOS 5 0.28571 0.25238 0.17270 0.13409 0.36364 0.33394 0.69429
KDEOS 6 0.28571 0.25238 0.32369 0.29213 0.40000 0.37200 0.77048
KDEOS 16 0.28571 0.25238 0.34386 0.31324 0.44444 0.41852 0.70571
LDF 4 0.28571 0.25238 0.23915 0.20364 0.40000 0.37200 0.72571
LDF 90 0.00000 -0.04667 0.12966 0.08905 0.23529 0.19961 0.81524
INFLO 2 0.14286 0.10286 0.14357 0.10360 0.35294 0.32275 0.66286
INFLO 3 0.14286 0.10286 0.16809 0.12926 0.22222 0.18593 0.71619
INFLO 9 0.28571 0.25238 0.14211 0.10207 0.28571 0.25238 0.71333
INFLO 94 0.00000 -0.04667 0.12788 0.08718 0.26923 0.23513 0.82286
COF 32 0.28571 0.25238 0.24986 0.21485 0.40000 0.37200 0.75048
COF 73 0.42857 0.40190 0.20365 0.16649 0.42857 0.40190 0.82095
COF 83 0.28571 0.25238 0.20322 0.16603 0.40000 0.37200 0.84286

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO