Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#02)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.4 kB) Download raw algorithm evaluation table (23.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.09337 0.05277 0.17647 0.13960 0.70647
KNN 11 0.00000 -0.04478 0.24708 0.21336 0.44444 0.41957 0.88557
KNN 16 0.00000 -0.04478 0.22692 0.19231 0.37500 0.34701 0.90050
KNNW 1 0.00000 -0.04478 0.06754 0.02579 0.13953 0.10101 0.54229
KNNW 26 0.00000 -0.04478 0.16061 0.12302 0.33333 0.30348 0.85075
KNNW 27 0.00000 -0.04478 0.16431 0.12689 0.33333 0.30348 0.85572
LOF 20 0.33333 0.30348 0.28472 0.25269 0.40000 0.37313 0.88060
LOF 22 0.33333 0.30348 0.28105 0.24885 0.44444 0.41957 0.90050
LOF 24 0.33333 0.30348 0.28889 0.25705 0.44444 0.41957 0.91045
SimplifiedLOF 1 0.00000 -0.04478 0.07619 0.03483 0.20000 0.16418 0.46269
SimplifiedLOF 45 0.00000 -0.04478 0.18056 0.14386 0.36364 0.33514 0.84080
SimplifiedLOF 49 0.00000 -0.04478 0.18237 0.14576 0.36364 0.33514 0.84577
LoOP 1 0.00000 -0.04478 0.07619 0.03483 0.20000 0.16418 0.46269
LoOP 45 0.00000 -0.04478 0.18434 0.14782 0.36364 0.33514 0.85075
LDOF 2 0.00000 -0.04478 0.11130 0.07151 0.28571 0.25373 0.41791
LDOF 54 0.00000 -0.04478 0.16941 0.13222 0.36364 0.33514 0.82587
LDOF 60 0.00000 -0.04478 0.17222 0.13516 0.30769 0.27669 0.85075
ODIN 12 0.33333 0.30348 0.17512 0.13818 0.33333 0.30348 0.78607
ODIN 32 0.00000 -0.04478 0.23593 0.20172 0.40000 0.37313 0.86567
ODIN 40 0.00000 -0.04478 0.30370 0.27253 0.50000 0.47761 0.85572
FastABOD 3 0.00000 -0.04478 0.05297 0.01056 0.11765 0.07814 0.48259
FastABOD 37 0.00000 -0.04478 0.12290 0.08362 0.26667 0.23383 0.78109
KDEOS 19 0.33333 0.30348 0.19724 0.16130 0.40000 0.37313 0.48259
KDEOS 67 0.00000 -0.04478 0.13651 0.09784 0.26667 0.23383 0.81592
LDF 7 0.33333 0.30348 0.22727 0.19267 0.33333 0.30348 0.87065
LDF 10 0.33333 0.30348 0.34444 0.31509 0.57143 0.55224 0.92040
LDF 12 0.33333 0.30348 0.57692 0.55798 0.57143 0.55224 0.94030
INFLO 19 0.33333 0.30348 0.16748 0.13021 0.33333 0.30348 0.65920
INFLO 23 0.00000 -0.04478 0.15301 0.11509 0.35294 0.32397 0.84080
INFLO 31 0.00000 -0.04478 0.16984 0.13267 0.33333 0.30348 0.86070
COF 21 0.00000 -0.04478 0.22407 0.18933 0.40000 0.37313 0.90050
COF 27 0.33333 0.30348 0.22549 0.19081 0.33333 0.30348 0.87065
COF 63 0.33333 0.30348 0.30480 0.27368 0.57143 0.55224 0.81095
COF 65 0.33333 0.30348 0.36458 0.33613 0.57143 0.55224 0.84080

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.4 kB) Download raw algorithm evaluation table (23.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.04987 0.00733 0.11765 0.07814 0.46020
KNN 66 0.00000 -0.04478 0.12175 0.08243 0.25000 0.21642 0.66169
KNNW 1 0.00000 -0.04478 0.04010 -0.00288 0.08889 0.04809 0.31592
KNNW 8 0.00000 -0.04478 0.06309 0.02114 0.15000 0.11194 0.57214
KNNW 11 0.00000 -0.04478 0.06327 0.02132 0.14286 0.10448 0.56716
LOF 1 0.00000 -0.04478 0.04016 -0.00282 0.11111 0.07131 0.29602
LOF 9 0.00000 -0.04478 0.09283 0.05221 0.20000 0.16418 0.62687
LOF 12 0.00000 -0.04478 0.12071 0.08134 0.28571 0.25373 0.56716
LOF 14 0.00000 -0.04478 0.12087 0.08151 0.28571 0.25373 0.57214
SimplifiedLOF 1 0.00000 -0.04478 0.04348 0.00065 0.08333 0.04229 0.24627
SimplifiedLOF 14 0.00000 -0.04478 0.08454 0.04355 0.18182 0.14518 0.69154
LoOP 1 0.00000 -0.04478 0.04286 -0.00000 0.08219 0.04110 0.23881
LoOP 14 0.00000 -0.04478 0.10434 0.06424 0.21053 0.17518 0.75124
LDOF 2 0.00000 -0.04478 0.04376 0.00094 0.10169 0.06147 0.37811
LDOF 14 0.00000 -0.04478 0.10179 0.06157 0.21053 0.17518 0.74129
LDOF 15 0.00000 -0.04478 0.09767 0.05726 0.22222 0.18740 0.71642
ODIN 6 0.00000 -0.04478 0.09841 0.05804 0.22222 0.18740 0.66667
ODIN 9 0.00000 -0.04478 0.08820 0.04738 0.17647 0.13960 0.75124
ODIN 12 0.00000 -0.04478 0.11214 0.07238 0.20000 0.16418 0.71642
ODIN 69 0.04286 -0.00000 0.04286 -0.00000 0.08219 0.04110 0.50000
FastABOD 3 0.00000 -0.04478 0.04656 0.00387 0.10345 0.06330 0.40796
FastABOD 7 0.00000 -0.04478 0.05346 0.01107 0.11538 0.07577 0.47761
KDEOS 35 0.00000 -0.04478 0.10278 0.06260 0.18182 0.14518 0.71144
KDEOS 49 0.33333 0.30348 0.15397 0.11609 0.33333 0.30348 0.63184
LDF 4 0.33333 0.30348 0.15972 0.12210 0.33333 0.30348 0.65672
LDF 5 0.33333 0.30348 0.22619 0.19154 0.40000 0.37313 0.73632
INFLO 11 0.00000 -0.04478 0.08223 0.04114 0.21053 0.17518 0.62687
INFLO 12 0.00000 -0.04478 0.08530 0.04434 0.19048 0.15423 0.62438
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 17 0.33333 0.30348 0.19695 0.16099 0.40000 0.37313 0.47761
COF 20 0.33333 0.30348 0.20294 0.16725 0.40000 0.37313 0.56716
COF 23 0.00000 -0.04478 0.08241 0.04132 0.18182 0.14518 0.57214

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO