Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#08)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.7 kB) Download raw algorithm evaluation table (22.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.09310 0.05250 0.17647 0.13960 0.69154
KNN 16 0.00000 -0.04478 0.12337 0.08412 0.26667 0.23383 0.77612
KNN 41 0.00000 -0.04478 0.12326 0.08400 0.26087 0.22777 0.79602
KNNW 1 0.00000 -0.04478 0.09286 0.05224 0.16667 0.12935 0.70149
KNNW 21 0.00000 -0.04478 0.10719 0.06721 0.22222 0.18740 0.73632
KNNW 27 0.00000 -0.04478 0.11004 0.07019 0.22222 0.18740 0.75124
KNNW 51 0.00000 -0.04478 0.10734 0.06737 0.21429 0.17910 0.76119
LOF 12 0.66667 0.65174 0.41919 0.39319 0.66667 0.65174 0.84080
LOF 13 0.33333 0.30348 0.54545 0.52510 0.57143 0.55224 0.89552
LOF 19 0.33333 0.30348 0.31217 0.28137 0.40000 0.37313 0.90547
SimplifiedLOF 11 0.33333 0.30348 0.21871 0.18373 0.40000 0.37313 0.68159
SimplifiedLOF 12 0.33333 0.30348 0.39383 0.36668 0.50000 0.47761 0.70647
SimplifiedLOF 22 0.33333 0.30348 0.42620 0.40050 0.50000 0.47761 0.81592
SimplifiedLOF 29 0.00000 -0.04478 0.17963 0.14290 0.33333 0.30348 0.85572
LoOP 12 0.33333 0.30348 0.22696 0.19235 0.40000 0.37313 0.71642
LoOP 13 0.33333 0.30348 0.40409 0.37741 0.50000 0.47761 0.76119
LoOP 23 0.33333 0.30348 0.43240 0.40699 0.50000 0.47761 0.84080
LoOP 29 0.00000 -0.04478 0.17456 0.13760 0.33333 0.30348 0.86070
LDOF 13 0.33333 0.30348 0.20901 0.17359 0.40000 0.37313 0.62687
LDOF 14 0.33333 0.30348 0.38025 0.35250 0.50000 0.47761 0.66169
LDOF 28 0.33333 0.30348 0.41204 0.38571 0.50000 0.47761 0.81592
ODIN 20 0.00000 -0.04478 0.16082 0.12324 0.27273 0.24016 0.84080
ODIN 21 0.11111 0.07131 0.13713 0.09850 0.25000 0.21642 0.78607
ODIN 24 0.00000 -0.04478 0.15455 0.11669 0.28571 0.25373 0.79602
FastABOD 3 0.00000 -0.04478 0.08555 0.04460 0.20000 0.16418 0.56219
FastABOD 9 0.00000 -0.04478 0.11163 0.07186 0.22222 0.18740 0.70647
FastABOD 61 0.00000 -0.04478 0.10290 0.06274 0.18750 0.15112 0.74129
KDEOS 2 0.33333 0.30348 0.19732 0.16138 0.40000 0.37313 0.58955
KDEOS 18 0.00000 -0.04478 0.20480 0.16920 0.44444 0.41957 0.79104
KDEOS 19 0.00000 -0.04478 0.22386 0.18910 0.44444 0.41957 0.81095
KDEOS 33 0.33333 0.30348 0.19823 0.16233 0.33333 0.30348 0.82587
LDF 5 0.33333 0.30348 0.22498 0.19028 0.40000 0.37313 0.69652
LDF 6 0.33333 0.30348 0.39477 0.36767 0.50000 0.47761 0.71642
LDF 52 0.00000 -0.04478 0.13079 0.09187 0.30000 0.26866 0.81095
INFLO 12 0.33333 0.30348 0.14478 0.10649 0.33333 0.30348 0.54478
INFLO 13 0.33333 0.30348 0.36724 0.33890 0.50000 0.47761 0.56219
INFLO 18 0.33333 0.30348 0.48120 0.45797 0.50000 0.47761 0.89552
COF 8 0.33333 0.30348 0.21057 0.17523 0.36364 0.33514 0.66667
COF 10 0.33333 0.30348 0.41317 0.38689 0.50000 0.47761 0.71144
COF 11 0.33333 0.30348 0.41568 0.38951 0.50000 0.47761 0.74129
COF 13 0.00000 -0.04478 0.16255 0.12505 0.28571 0.25373 0.80100

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.9 kB) Download raw algorithm evaluation table (22.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.04554 0.00280 0.10909 0.06920 0.40299
KNN 68 0.00000 -0.04478 0.09826 0.05788 0.23529 0.20105 0.69652
KNNW 1 0.00000 -0.04478 0.04695 0.00428 0.12000 0.08060 0.42289
KNNW 2 0.00000 -0.04478 0.04766 0.00502 0.12245 0.08316 0.42786
LOF 1 0.00000 -0.04478 0.04584 0.00312 0.11111 0.07131 0.40299
LOF 4 0.00000 -0.04478 0.10628 0.06626 0.22222 0.18740 0.65672
SimplifiedLOF 1 0.00000 -0.04478 0.04132 -0.00161 0.08451 0.04351 0.36816
SimplifiedLOF 6 0.00000 -0.04478 0.07341 0.03192 0.16667 0.12935 0.52736
SimplifiedLOF 8 0.00000 -0.04478 0.07889 0.03765 0.16667 0.12935 0.57711
LoOP 1 0.00000 -0.04478 0.04048 -0.00249 0.08219 0.04110 0.35821
LoOP 6 0.00000 -0.04478 0.09067 0.04996 0.22222 0.18740 0.54726
LoOP 8 0.00000 -0.04478 0.07910 0.03787 0.16667 0.12935 0.58458
LDOF 2 0.00000 -0.04478 0.07407 0.03261 0.16667 0.12935 0.53731
LDOF 5 0.00000 -0.04478 0.11887 0.07941 0.25000 0.21642 0.64179
LDOF 6 0.00000 -0.04478 0.12806 0.08902 0.28571 0.25373 0.59701
ODIN 1 0.04545 0.00271 0.03985 -0.00314 0.08219 0.04110 0.34080
ODIN 8 0.00000 -0.04478 0.06289 0.02093 0.12500 0.08582 0.56965
ODIN 9 0.00000 -0.04478 0.06644 0.02464 0.12121 0.08186 0.59701
FastABOD 3 0.00000 -0.04478 0.04371 0.00089 0.09836 0.05799 0.35821
FastABOD 8 0.00000 -0.04478 0.04127 -0.00166 0.10169 0.06147 0.33831
KDEOS 10 0.00000 -0.04478 0.08324 0.04220 0.17391 0.13692 0.62189
KDEOS 27 0.33333 0.30348 0.14684 0.10864 0.33333 0.30348 0.55224
LDF 1 0.00000 -0.04478 0.05265 0.01023 0.11765 0.07814 0.47264
LDF 3 0.00000 -0.04478 0.06914 0.02746 0.13793 0.09933 0.55721
LDF 5 0.00000 -0.04478 0.06254 0.02056 0.14815 0.11001 0.54726
INFLO 2 0.00000 -0.04478 0.09192 0.05126 0.22222 0.18740 0.53234
INFLO 5 0.00000 -0.04478 0.09892 0.05858 0.25000 0.21642 0.52736
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 1 0.00000 -0.04478 0.04132 -0.00161 0.08451 0.04351 0.36816
COF 6 0.00000 -0.04478 0.07468 0.03325 0.17391 0.13692 0.61692

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO