Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (10% of outliers version#08)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 74 objects, 7 outliers (9.46%)

Download raw algorithm results (469.4 kB) Download raw algorithm evaluation table (31.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 14 0.28571 0.21109 0.24529 0.16644 0.40000 0.33731 0.80384
KNN 27 0.14286 0.05330 0.25976 0.18243 0.48276 0.42872 0.84222
KNN 43 0.28571 0.21109 0.26874 0.19234 0.41176 0.35031 0.83582
KNNW 18 0.14286 0.05330 0.20311 0.11985 0.38095 0.31628 0.75480
KNNW 19 0.14286 0.05330 0.20625 0.12332 0.40000 0.33731 0.75693
KNNW 37 0.14286 0.05330 0.22329 0.14214 0.40000 0.33731 0.79318
LOF 18 0.28571 0.21109 0.25409 0.17616 0.42105 0.36057 0.79531
LOF 27 0.28571 0.21109 0.30393 0.23120 0.47059 0.41528 0.84009
SimplifiedLOF 1 0.14286 0.05330 0.11507 0.02262 0.20000 0.11642 0.47761
SimplifiedLOF 37 0.14286 0.05330 0.29502 0.22136 0.33333 0.26368 0.74200
SimplifiedLOF 49 0.14286 0.05330 0.24260 0.16346 0.38095 0.31628 0.80171
SimplifiedLOF 50 0.14286 0.05330 0.24074 0.16141 0.40000 0.33731 0.79957
LoOP 25 0.28571 0.21109 0.18163 0.09612 0.28571 0.21109 0.68870
LoOP 36 0.14286 0.05330 0.31473 0.24313 0.36842 0.30244 0.77612
LoOP 50 0.14286 0.05330 0.24690 0.16821 0.41176 0.35031 0.81450
LDOF 2 0.14286 0.05330 0.13546 0.04513 0.25000 0.17164 0.48188
LDOF 42 0.14286 0.05330 0.26885 0.19246 0.30000 0.22687 0.67804
LDOF 55 0.14286 0.05330 0.21428 0.13219 0.38889 0.32504 0.78038
LDOF 64 0.14286 0.05330 0.23166 0.15139 0.38889 0.32504 0.80384
ODIN 32 0.28571 0.21109 0.24569 0.16688 0.36842 0.30244 0.79638
ODIN 35 0.28571 0.21109 0.28333 0.20846 0.37838 0.31343 0.81876
ODIN 39 0.23810 0.15849 0.25707 0.17945 0.42105 0.36057 0.82196
ODIN 42 0.28571 0.21109 0.27880 0.20345 0.42105 0.36057 0.82836
FastABOD 27 0.14286 0.05330 0.16082 0.07315 0.29630 0.22278 0.69723
FastABOD 55 0.14286 0.05330 0.18751 0.10262 0.34783 0.27969 0.74200
FastABOD 68 0.14286 0.05330 0.18493 0.09977 0.35714 0.28998 0.73987
KDEOS 18 0.28571 0.21109 0.19066 0.10610 0.33333 0.26368 0.58209
KDEOS 20 0.28571 0.21109 0.20272 0.11942 0.36364 0.29715 0.55437
KDEOS 73 0.14286 0.05330 0.18871 0.10394 0.33333 0.26368 0.75053
LDF 13 0.42857 0.36887 0.31105 0.23907 0.42857 0.36887 0.71002
LDF 14 0.42857 0.36887 0.33643 0.26711 0.46154 0.40528 0.75053
LDF 15 0.28571 0.21109 0.30921 0.23704 0.50000 0.44776 0.79318
LDF 43 0.14286 0.05330 0.25094 0.17268 0.42105 0.36057 0.82729
INFLO 18 0.14286 0.05330 0.29975 0.22659 0.38462 0.32032 0.71748
INFLO 26 0.28571 0.21109 0.22789 0.14722 0.36364 0.29715 0.73881
INFLO 46 0.42857 0.36887 0.22375 0.14265 0.42857 0.36887 0.67910
COF 55 0.28571 0.21109 0.25151 0.17331 0.41667 0.35572 0.82090
COF 63 0.42857 0.36887 0.28228 0.20729 0.42857 0.36887 0.77399
COF 66 0.42857 0.36887 0.32620 0.25581 0.50000 0.44776 0.78252
COF 71 0.42857 0.36887 0.45108 0.39373 0.50000 0.44776 0.81023

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 74 objects, 7 outliers (9.46%)

Download raw algorithm results (471.2 kB) Download raw algorithm evaluation table (29.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 7 0.00000 -0.10448 0.09829 0.00408 0.23077 0.15040 0.46908
KNN 70 0.14286 0.05330 0.12048 0.02859 0.20000 0.11642 0.50320
KNN 72 0.00000 -0.10448 0.11494 0.02247 0.22951 0.14901 0.55437
KNNW 1 0.00000 -0.10448 0.07963 -0.01653 0.18750 0.10261 0.34222
KNNW 15 0.00000 -0.10448 0.09347 -0.00125 0.22642 0.14559 0.46055
KNNW 29 0.00000 -0.10448 0.09690 0.00254 0.22222 0.14096 0.47974
LOF 13 0.14286 0.05330 0.10737 0.01411 0.18462 0.09943 0.43923
LOF 16 0.14286 0.05330 0.10770 0.01447 0.18519 0.10006 0.45629
LOF 65 0.00000 -0.10448 0.10481 0.01128 0.25532 0.17752 0.52452
SimplifiedLOF 1 0.00000 -0.10448 0.08199 -0.01392 0.17722 0.09125 0.33689
SimplifiedLOF 35 0.00000 -0.10448 0.10403 0.01042 0.20833 0.12562 0.49254
SimplifiedLOF 72 0.00000 -0.10448 0.08929 -0.00586 0.21875 0.13713 0.43284
LoOP 1 0.00000 -0.10448 0.08011 -0.01600 0.17284 0.08642 0.32623
LoOP 23 0.00000 -0.10448 0.09278 -0.00201 0.22222 0.14096 0.44883
LoOP 37 0.00000 -0.10448 0.10386 0.01023 0.20000 0.11642 0.50533
LDOF 2 0.00000 -0.10448 0.07963 -0.01653 0.20588 0.12291 0.36247
LDOF 10 0.00000 -0.10448 0.10556 0.01211 0.21053 0.12804 0.51599
LDOF 37 0.00000 -0.10448 0.10143 0.00755 0.23529 0.15540 0.49254
ODIN 10 0.00000 -0.10448 0.11807 0.02592 0.26087 0.18365 0.55757
ODIN 14 0.04762 -0.05188 0.12843 0.03737 0.23529 0.15540 0.47015
ODIN 71 0.14286 0.05330 0.10149 0.00761 0.17284 0.08642 0.52026
FastABOD 3 0.00000 -0.10448 0.08949 -0.00564 0.18750 0.10261 0.42644
FastABOD 39 0.00000 -0.10448 0.08426 -0.01141 0.20290 0.11962 0.38806
KDEOS 2 0.28571 0.21109 0.23293 0.15279 0.40000 0.33731 0.54158
LDF 4 0.14286 0.05330 0.11544 0.02302 0.24138 0.16212 0.54158
LDF 5 0.14286 0.05330 0.13619 0.04594 0.25000 0.17164 0.51599
LDF 7 0.14286 0.05330 0.16218 0.07464 0.22222 0.14096 0.47122
INFLO 5 0.14286 0.05330 0.13608 0.04582 0.25000 0.17164 0.55224
COF 1 0.00000 -0.10448 0.08199 -0.01392 0.17722 0.09125 0.33689
COF 16 0.00000 -0.10448 0.10405 0.01044 0.18667 0.10169 0.44350
COF 47 0.00000 -0.10448 0.10071 0.00675 0.23729 0.15760 0.50853

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO