Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#03)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.3 kB) Download raw algorithm evaluation table (22.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.10051 0.06023 0.18750 0.15112 0.72637
KNN 14 0.00000 -0.04478 0.23660 0.20242 0.44444 0.41957 0.89055
KNN 16 0.00000 -0.04478 0.22143 0.18657 0.36364 0.33514 0.89552
KNNW 1 0.00000 -0.04478 0.09739 0.05697 0.21429 0.17910 0.74129
KNNW 12 0.00000 -0.04478 0.13931 0.10077 0.28571 0.25373 0.81095
KNNW 28 0.00000 -0.04478 0.16086 0.12328 0.28571 0.25373 0.84577
KNNW 63 0.00000 -0.04478 0.16727 0.12999 0.28571 0.25373 0.82587
LOF 22 0.33333 0.30348 0.26190 0.22886 0.40000 0.37313 0.89055
LOF 23 0.33333 0.30348 0.27485 0.24238 0.44444 0.41957 0.89055
LOF 27 0.33333 0.30348 0.25694 0.22367 0.36364 0.33514 0.89552
SimplifiedLOF 1 0.00000 -0.04478 0.07024 0.02861 0.18182 0.14518 0.45771
SimplifiedLOF 42 0.00000 -0.04478 0.21111 0.17579 0.44444 0.41957 0.82587
SimplifiedLOF 45 0.00000 -0.04478 0.21481 0.17966 0.44444 0.41957 0.84080
SimplifiedLOF 53 0.00000 -0.04478 0.18237 0.14576 0.36364 0.33514 0.84577
LoOP 1 0.00000 -0.04478 0.07024 0.02861 0.18182 0.14518 0.45771
LoOP 43 0.00000 -0.04478 0.19639 0.16040 0.40000 0.37313 0.82587
LoOP 53 0.00000 -0.04478 0.18620 0.14976 0.33333 0.30348 0.85075
LDOF 2 0.00000 -0.04478 0.11083 0.07101 0.25000 0.21642 0.60697
LDOF 53 0.00000 -0.04478 0.19894 0.16307 0.40000 0.37313 0.83582
LDOF 61 0.00000 -0.04478 0.18333 0.14677 0.30769 0.27669 0.85572
ODIN 32 0.33333 0.30348 0.24481 0.21100 0.40000 0.37313 0.86070
ODIN 33 0.33333 0.30348 0.24802 0.21434 0.40000 0.37313 0.86816
ODIN 39 0.33333 0.30348 0.26222 0.22919 0.44444 0.41957 0.86070
FastABOD 3 0.00000 -0.04478 0.15972 0.12210 0.36364 0.33514 0.72139
FastABOD 31 0.00000 -0.04478 0.14537 0.10710 0.30769 0.27669 0.80597
KDEOS 2 0.00000 -0.04478 0.03674 -0.00639 0.08824 0.04741 0.33085
KDEOS 68 0.00000 -0.04478 0.14227 0.10387 0.28571 0.25373 0.81095
KDEOS 69 0.00000 -0.04478 0.14394 0.10561 0.28571 0.25373 0.81592
LDF 11 0.33333 0.30348 0.45000 0.42537 0.50000 0.47761 0.87562
LDF 12 0.33333 0.30348 0.24864 0.21500 0.37500 0.34701 0.89552
INFLO 30 0.00000 -0.04478 0.16984 0.13267 0.30769 0.27669 0.85572
INFLO 44 0.00000 -0.04478 0.23116 0.19673 0.50000 0.47761 0.75622
INFLO 59 0.33333 0.30348 0.13968 0.10116 0.33333 0.30348 0.60697
COF 21 0.33333 0.30348 0.19722 0.16128 0.33333 0.30348 0.82090
COF 24 0.00000 -0.04478 0.21026 0.17489 0.37500 0.34701 0.89055
COF 38 0.00000 -0.04478 0.19167 0.15547 0.40000 0.37313 0.88060
COF 67 0.33333 0.30348 0.26638 0.23353 0.40000 0.37313 0.78109

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.5 kB) Download raw algorithm evaluation table (23.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.04783 0.00520 0.11765 0.07814 0.43781
KNN 19 0.00000 -0.04478 0.06575 0.02392 0.15000 0.11194 0.59204
KNN 66 0.00000 -0.04478 0.10085 0.06059 0.25000 0.21642 0.52239
KNNW 1 0.00000 -0.04478 0.04358 0.00075 0.10345 0.06330 0.37811
KNNW 13 0.00000 -0.04478 0.06279 0.02083 0.14286 0.10448 0.56219
KNNW 44 0.00000 -0.04478 0.06321 0.02126 0.14286 0.10448 0.57214
LOF 1 0.00000 -0.04478 0.04016 -0.00282 0.11111 0.07131 0.29602
LOF 11 0.00000 -0.04478 0.12024 0.08085 0.28571 0.25373 0.55721
LOF 61 0.00000 -0.04478 0.07096 0.02936 0.14634 0.10812 0.62189
SimplifiedLOF 1 0.00000 -0.04478 0.03851 -0.00454 0.08333 0.04229 0.33333
SimplifiedLOF 24 0.00000 -0.04478 0.08192 0.04082 0.18182 0.14518 0.58209
SimplifiedLOF 29 0.00000 -0.04478 0.08583 0.04489 0.18182 0.14518 0.61692
SimplifiedLOF 30 0.00000 -0.04478 0.08258 0.04150 0.16667 0.12935 0.62189
LoOP 1 0.00000 -0.04478 0.03810 -0.00498 0.08219 0.04110 0.32836
LoOP 24 0.00000 -0.04478 0.09171 0.05104 0.20000 0.16418 0.62189
LoOP 27 0.00000 -0.04478 0.09345 0.05286 0.20000 0.16418 0.63682
LoOP 32 0.00000 -0.04478 0.08795 0.04712 0.16667 0.12935 0.66169
LDOF 2 0.00000 -0.04478 0.04005 -0.00294 0.09677 0.05633 0.31343
LDOF 6 0.00000 -0.04478 0.08526 0.04431 0.20000 0.16418 0.63682
LDOF 33 0.00000 -0.04478 0.09786 0.05747 0.20000 0.16418 0.66667
LDOF 45 0.00000 -0.04478 0.09123 0.05054 0.17647 0.13960 0.69652
ODIN 12 0.00000 -0.04478 0.09925 0.05892 0.20000 0.16418 0.69900
ODIN 20 0.00000 -0.04478 0.10961 0.06974 0.22222 0.18740 0.63930
ODIN 69 0.04286 -0.00000 0.04286 -0.00000 0.08219 0.04110 0.50000
FastABOD 3 0.00000 -0.04478 0.04502 0.00226 0.10345 0.06330 0.37313
FastABOD 8 0.00000 -0.04478 0.05164 0.00918 0.11111 0.07131 0.45274
FastABOD 9 0.00000 -0.04478 0.05126 0.00878 0.11538 0.07577 0.45274
KDEOS 2 0.33333 0.30348 0.14644 0.10822 0.33333 0.30348 0.63184
KDEOS 4 0.33333 0.30348 0.37341 0.34536 0.50000 0.47761 0.59701
KDEOS 46 0.00000 -0.04478 0.12749 0.08843 0.28571 0.25373 0.63682
LDF 4 0.33333 0.30348 0.14781 0.10966 0.33333 0.30348 0.56716
LDF 5 0.33333 0.30348 0.21825 0.18325 0.40000 0.37313 0.70149
INFLO 5 0.00000 -0.04478 0.06930 0.02762 0.16667 0.12935 0.50249
INFLO 17 0.00000 -0.04478 0.06198 0.01998 0.13333 0.09453 0.53483
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 13 0.33333 0.30348 0.14630 0.10807 0.33333 0.30348 0.53731
COF 14 0.33333 0.30348 0.20304 0.16736 0.40000 0.37313 0.56219
COF 20 0.33333 0.30348 0.20747 0.17198 0.40000 0.37313 0.61692
COF 22 0.00000 -0.04478 0.11189 0.07212 0.25000 0.21642 0.64179

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO