Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#04)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.6 kB) Download raw algorithm evaluation table (21.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.15000 0.11194 0.30769 0.27669 0.82090
KNN 2 0.00000 -0.04478 0.23889 0.20481 0.46154 0.43743 0.91045
KNN 15 0.00000 -0.04478 0.26111 0.22803 0.44444 0.41957 0.91542
KNNW 1 0.00000 -0.04478 0.12466 0.08546 0.23529 0.20105 0.79602
KNNW 11 0.00000 -0.04478 0.19167 0.15547 0.40000 0.37313 0.88060
KNNW 63 0.00000 -0.04478 0.20420 0.16856 0.37500 0.34701 0.88557
LOF 13 0.33333 0.30348 0.17088 0.13375 0.33333 0.30348 0.70647
LOF 22 0.33333 0.30348 0.37778 0.34992 0.57143 0.55224 0.94527
SimplifiedLOF 1 0.00000 -0.04478 0.03867 -0.00437 0.08219 0.04110 0.33333
SimplifiedLOF 43 0.00000 -0.04478 0.26869 0.23594 0.44444 0.41957 0.92040
LoOP 1 0.00000 -0.04478 0.03867 -0.00437 0.08219 0.04110 0.33333
LoOP 43 0.00000 -0.04478 0.22857 0.19403 0.40000 0.37313 0.89552
LDOF 2 0.00000 -0.04478 0.05865 0.01650 0.13793 0.09933 0.51741
LDOF 53 0.00000 -0.04478 0.24028 0.20626 0.44444 0.41957 0.89552
ODIN 29 0.22222 0.18740 0.29444 0.26285 0.46154 0.43743 0.94527
ODIN 32 0.33333 0.30348 0.25470 0.22133 0.37500 0.34701 0.91294
ODIN 37 0.16667 0.12935 0.28968 0.25788 0.50000 0.47761 0.93781
ODIN 39 0.33333 0.30348 0.31313 0.28238 0.44444 0.41957 0.93781
FastABOD 3 0.00000 -0.04478 0.17415 0.13717 0.37500 0.34701 0.86567
FastABOD 31 0.00000 -0.04478 0.19907 0.16321 0.40000 0.37313 0.88557
FastABOD 35 0.00000 -0.04478 0.19924 0.16339 0.42857 0.40299 0.88557
KDEOS 4 0.33333 0.30348 0.31349 0.28275 0.57143 0.55224 0.85572
KDEOS 69 0.00000 -0.04478 0.18515 0.14866 0.37500 0.34701 0.87562
LDF 12 0.33333 0.30348 0.46250 0.43843 0.50000 0.47761 0.89552
INFLO 39 0.00000 -0.04478 0.22540 0.19071 0.44444 0.41957 0.87065
INFLO 41 0.00000 -0.04478 0.24444 0.21061 0.44444 0.41957 0.88060
INFLO 58 0.33333 0.30348 0.13968 0.10116 0.33333 0.30348 0.60697
COF 65 0.66667 0.65174 0.48889 0.46600 0.66667 0.65174 0.95522
COF 66 0.33333 0.30348 0.45833 0.43408 0.57143 0.55224 0.96020

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.6 kB) Download raw algorithm evaluation table (23.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.05778 0.01559 0.12500 0.08582 0.53483
KNN 66 0.00000 -0.04478 0.13194 0.09308 0.25000 0.21642 0.69154
KNNW 1 0.00000 -0.04478 0.04598 0.00326 0.10169 0.06147 0.41045
KNNW 7 0.00000 -0.04478 0.06520 0.02334 0.15000 0.11194 0.59204
KNNW 11 0.00000 -0.04478 0.06743 0.02567 0.14634 0.10812 0.60199
LOF 4 0.33333 0.30348 0.14789 0.10973 0.33333 0.30348 0.56716
LOF 5 0.33333 0.30348 0.17475 0.13780 0.33333 0.30348 0.75622
LOF 6 0.00000 -0.04478 0.16603 0.12869 0.28571 0.25373 0.81095
SimplifiedLOF 1 0.00000 -0.04478 0.04348 0.00065 0.08333 0.04229 0.24627
SimplifiedLOF 15 0.00000 -0.04478 0.08287 0.04180 0.19048 0.15423 0.66169
SimplifiedLOF 19 0.00000 -0.04478 0.08613 0.04521 0.15385 0.11596 0.65672
SimplifiedLOF 23 0.00000 -0.04478 0.08357 0.04253 0.15789 0.12019 0.66667
LoOP 1 0.00000 -0.04478 0.04286 -0.00000 0.08219 0.04110 0.23881
LoOP 5 0.00000 -0.04478 0.10493 0.06485 0.22222 0.18740 0.60945
LoOP 23 0.00000 -0.04478 0.09337 0.05277 0.17647 0.13960 0.70647
LDOF 2 0.00000 -0.04478 0.04403 0.00123 0.09677 0.05633 0.37811
LDOF 5 0.00000 -0.04478 0.12401 0.08479 0.28571 0.25373 0.70647
LDOF 32 0.00000 -0.04478 0.11071 0.07090 0.19355 0.15744 0.75124
ODIN 8 0.00000 -0.04478 0.11111 0.07131 0.20000 0.16418 0.76617
ODIN 21 0.00000 -0.04478 0.12556 0.08641 0.25000 0.21642 0.72139
ODIN 69 0.04286 -0.00000 0.04286 -0.00000 0.08219 0.04110 0.50000
FastABOD 3 0.00000 -0.04478 0.05108 0.00860 0.10526 0.06520 0.45771
FastABOD 9 0.00000 -0.04478 0.05845 0.01629 0.12245 0.08316 0.53731
FastABOD 10 0.00000 -0.04478 0.05821 0.01604 0.12500 0.08582 0.53731
FastABOD 42 0.00000 -0.04478 0.05902 0.01689 0.12000 0.08060 0.53731
KDEOS 26 0.33333 0.30348 0.16521 0.12784 0.33333 0.30348 0.71144
KDEOS 32 0.33333 0.30348 0.19573 0.15971 0.33333 0.30348 0.80100
KDEOS 35 0.33333 0.30348 0.21825 0.18325 0.36364 0.33514 0.76617
LDF 4 0.00000 -0.04478 0.15595 0.11816 0.28571 0.25373 0.74129
LDF 5 0.33333 0.30348 0.16444 0.12703 0.33333 0.30348 0.66667
INFLO 4 0.00000 -0.04478 0.19444 0.15837 0.44444 0.41957 0.73632
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 13 0.00000 -0.04478 0.15993 0.12232 0.33333 0.30348 0.79104
COF 15 0.33333 0.30348 0.22261 0.18780 0.40000 0.37313 0.71642

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO