Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#01)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.6 kB) Download raw algorithm evaluation table (22.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.10726 0.06728 0.21053 0.17518 0.74627
KNN 9 0.00000 -0.04478 0.26930 0.23658 0.50000 0.47761 0.89303
KNN 14 0.00000 -0.04478 0.22143 0.18657 0.36364 0.33514 0.89552
KNNW 1 0.00000 -0.04478 0.13651 0.09784 0.25000 0.21642 0.78109
KNNW 14 0.00000 -0.04478 0.17641 0.13953 0.36364 0.33514 0.84577
KNNW 21 0.00000 -0.04478 0.18845 0.15212 0.33333 0.30348 0.87065
LOF 19 0.33333 0.30348 0.51948 0.49796 0.50000 0.47761 0.93532
LOF 20 0.66667 0.65174 0.47222 0.44859 0.66667 0.65174 0.94527
SimplifiedLOF 31 0.33333 0.30348 0.44344 0.41852 0.50000 0.47761 0.87562
SimplifiedLOF 32 0.33333 0.30348 0.44949 0.42485 0.50000 0.47761 0.88060
LoOP 29 0.33333 0.30348 0.29956 0.26820 0.40000 0.37313 0.89055
LoOP 32 0.33333 0.30348 0.45276 0.42826 0.50000 0.47761 0.88557
LDOF 31 0.33333 0.30348 0.20033 0.16452 0.33333 0.30348 0.83085
LDOF 36 0.33333 0.30348 0.40370 0.37700 0.50000 0.47761 0.79104
LDOF 38 0.33333 0.30348 0.40741 0.38087 0.50000 0.47761 0.80100
LDOF 59 0.00000 -0.04478 0.20556 0.16998 0.30769 0.27669 0.87065
ODIN 13 0.33333 0.30348 0.25926 0.22609 0.44444 0.41957 0.87811
ODIN 14 0.22222 0.18740 0.31212 0.28132 0.50000 0.47761 0.89801
ODIN 20 0.33333 0.30348 0.27302 0.24046 0.40000 0.37313 0.92289
ODIN 25 0.33333 0.30348 0.33571 0.30597 0.50000 0.47761 0.86070
FastABOD 7 0.33333 0.30348 0.18056 0.14386 0.33333 0.30348 0.75622
FastABOD 33 0.33333 0.30348 0.27037 0.23770 0.40000 0.37313 0.83582
FastABOD 41 0.33333 0.30348 0.27179 0.23919 0.40000 0.37313 0.84080
KDEOS 18 0.33333 0.30348 0.14971 0.11164 0.33333 0.30348 0.58706
KDEOS 19 0.33333 0.30348 0.20849 0.17305 0.40000 0.37313 0.62687
KDEOS 61 0.00000 -0.04478 0.20263 0.16693 0.36364 0.33514 0.87065
LDF 7 0.66667 0.65174 0.44444 0.41957 0.66667 0.65174 0.91542
LDF 12 0.33333 0.30348 0.57692 0.55798 0.57143 0.55224 0.94030
INFLO 22 0.33333 0.30348 0.29870 0.26730 0.40000 0.37313 0.89552
INFLO 25 0.33333 0.30348 0.44344 0.41852 0.50000 0.47761 0.87562
INFLO 29 0.33333 0.30348 0.31250 0.28172 0.40000 0.37313 0.90050
COF 21 0.00000 -0.04478 0.21766 0.18263 0.37500 0.34701 0.89552
COF 62 0.66667 0.65174 0.41830 0.39225 0.66667 0.65174 0.83582
COF 66 0.66667 0.65174 0.58497 0.56638 0.66667 0.65174 0.84080

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.2 kB) Download raw algorithm evaluation table (23.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.05120 0.00871 0.11765 0.07814 0.47512
KNN 3 0.00000 -0.04478 0.05850 0.01634 0.15385 0.11596 0.53234
KNN 10 0.00000 -0.04478 0.05924 0.01712 0.13043 0.09150 0.54726
KNN 68 0.00000 -0.04478 0.06274 0.02077 0.12500 0.08582 0.52239
KNNW 1 0.00000 -0.04478 0.04645 0.00376 0.08889 0.04809 0.38308
KNNW 8 0.00000 -0.04478 0.05789 0.01571 0.15000 0.11194 0.53234
LOF 1 0.00000 -0.04478 0.03617 -0.00698 0.08955 0.04879 0.23134
LOF 3 0.00000 -0.04478 0.06851 0.02680 0.15385 0.11596 0.49751
LOF 8 0.00000 -0.04478 0.06021 0.01813 0.12766 0.08860 0.55224
SimplifiedLOF 1 0.00000 -0.04478 0.04016 -0.00281 0.08451 0.04351 0.35323
SimplifiedLOF 16 0.00000 -0.04478 0.06751 0.02575 0.16216 0.12465 0.60697
LoOP 1 0.00000 -0.04478 0.03932 -0.00369 0.08219 0.04110 0.34328
LoOP 16 0.00000 -0.04478 0.07075 0.02915 0.15000 0.11194 0.61692
LoOP 18 0.00000 -0.04478 0.06090 0.01885 0.15789 0.12019 0.55224
LDOF 2 0.00000 -0.04478 0.03773 -0.00536 0.10169 0.06147 0.26368
LDOF 15 0.00000 -0.04478 0.09287 0.05225 0.23529 0.20105 0.64179
LDOF 16 0.00000 -0.04478 0.09092 0.05021 0.22222 0.18740 0.65174
ODIN 1 0.04545 0.00271 0.04290 0.00004 0.08955 0.04879 0.48010
ODIN 9 0.00000 -0.04478 0.06589 0.02406 0.15000 0.11194 0.65174
ODIN 11 0.00000 -0.04478 0.09677 0.05633 0.17647 0.13960 0.60448
FastABOD 3 0.00000 -0.04478 0.04550 0.00276 0.10345 0.06330 0.39801
FastABOD 7 0.00000 -0.04478 0.04925 0.00667 0.11538 0.07577 0.44776
FastABOD 53 0.00000 -0.04478 0.05056 0.00804 0.11321 0.07350 0.44279
KDEOS 2 0.33333 0.30348 0.13592 0.09723 0.33333 0.30348 0.37313
KDEOS 17 0.00000 -0.04478 0.07798 0.03669 0.17391 0.13692 0.62189
LDF 1 0.00000 -0.04478 0.03926 -0.00376 0.08955 0.04879 0.29602
LDF 6 0.00000 -0.04478 0.09414 0.05358 0.20000 0.16418 0.64179
INFLO 5 0.00000 -0.04478 0.06930 0.02762 0.16667 0.12935 0.51244
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 1 0.00000 -0.04478 0.04016 -0.00281 0.08451 0.04351 0.35323
COF 24 0.00000 -0.04478 0.06097 0.01892 0.13333 0.09453 0.51244
COF 25 0.00000 -0.04478 0.06699 0.02522 0.12903 0.09003 0.54229
COF 26 0.00000 -0.04478 0.06942 0.02775 0.13333 0.09453 0.54229

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO