Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#06)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.6 kB) Download raw algorithm evaluation table (21.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.10370 0.06356 0.20000 0.16418 0.70398
KNN 2 0.00000 -0.04478 0.14141 0.10297 0.28571 0.25373 0.72637
KNN 10 0.00000 -0.04478 0.14538 0.10712 0.28571 0.25373 0.74129
KNNW 1 0.00000 -0.04478 0.11540 0.07579 0.21429 0.17910 0.75124
KNNW 6 0.00000 -0.04478 0.13745 0.09882 0.28571 0.25373 0.70647
KNNW 10 0.00000 -0.04478 0.13839 0.09981 0.28571 0.25373 0.71144
LOF 18 0.33333 0.30348 0.15849 0.12081 0.33333 0.30348 0.66667
LOF 19 0.33333 0.30348 0.21328 0.17805 0.40000 0.37313 0.66667
LOF 46 0.00000 -0.04478 0.09448 0.05394 0.18750 0.15112 0.71144
SimplifiedLOF 1 0.00000 -0.04478 0.10278 0.06260 0.18182 0.14518 0.72139
SimplifiedLOF 2 0.00000 -0.04478 0.11086 0.07105 0.26667 0.23383 0.71642
SimplifiedLOF 3 0.00000 -0.04478 0.10934 0.06946 0.24000 0.20597 0.76617
SimplifiedLOF 40 0.00000 -0.04478 0.11382 0.07414 0.25000 0.21642 0.65672
LoOP 1 0.00000 -0.04478 0.10278 0.06260 0.18182 0.14518 0.72139
LoOP 2 0.00000 -0.04478 0.13754 0.09892 0.30769 0.27669 0.73134
LoOP 3 0.00000 -0.04478 0.11268 0.07295 0.26087 0.22777 0.77612
LDOF 2 0.00000 -0.04478 0.11503 0.07541 0.25000 0.21642 0.65672
LDOF 52 0.00000 -0.04478 0.11520 0.07559 0.25000 0.21642 0.66667
ODIN 2 0.22222 0.18740 0.18661 0.15019 0.33333 0.30348 0.89055
ODIN 11 0.33333 0.30348 0.17563 0.13872 0.33333 0.30348 0.74627
ODIN 12 0.33333 0.30348 0.22638 0.19174 0.40000 0.37313 0.78358
FastABOD 3 0.00000 -0.04478 0.08330 0.04226 0.20000 0.16418 0.65174
FastABOD 9 0.00000 -0.04478 0.11919 0.07975 0.22222 0.18740 0.74129
FastABOD 20 0.00000 -0.04478 0.11717 0.07764 0.20690 0.17138 0.77114
KDEOS 9 0.00000 -0.04478 0.16570 0.12834 0.30769 0.27669 0.83582
KDEOS 19 0.33333 0.30348 0.15992 0.12231 0.33333 0.30348 0.67662
KDEOS 23 0.33333 0.30348 0.21946 0.18451 0.40000 0.37313 0.69652
LDF 9 0.33333 0.30348 0.16580 0.12845 0.33333 0.30348 0.70149
LDF 11 0.33333 0.30348 0.39608 0.36904 0.50000 0.47761 0.75622
LDF 20 0.00000 -0.04478 0.13056 0.09163 0.26667 0.23383 0.78109
INFLO 9 0.00000 -0.04478 0.11952 0.08009 0.26087 0.22777 0.79104
INFLO 68 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 5 0.33333 0.30348 0.18362 0.14706 0.33333 0.30348 0.66169
COF 23 0.00000 -0.04478 0.14811 0.10996 0.30000 0.26866 0.83582
COF 32 0.00000 -0.04478 0.18651 0.15008 0.40000 0.37313 0.82587
COF 67 0.33333 0.30348 0.21670 0.18162 0.40000 0.37313 0.68657

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.7 kB) Download raw algorithm evaluation table (17.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.33333 0.30348 0.32564 0.29545 0.50000 0.47761 0.80100
KNN 62 0.33333 0.30348 0.24074 0.20674 0.33333 0.30348 0.88060
KNNW 1 0.33333 0.30348 0.25915 0.22598 0.50000 0.47761 0.65672
KNNW 2 0.33333 0.30348 0.29739 0.26593 0.57143 0.55224 0.74129
KNNW 5 0.33333 0.30348 0.27076 0.23811 0.50000 0.47761 0.80100
LOF 7 0.00000 -0.04478 0.21778 0.18275 0.44444 0.41957 0.85075
LOF 14 0.33333 0.30348 0.25374 0.22033 0.40000 0.37313 0.72637
LOF 15 0.33333 0.30348 0.35294 0.32397 0.57143 0.55224 0.74627
LOF 18 0.33333 0.30348 0.35374 0.32480 0.57143 0.55224 0.75622
SimplifiedLOF 10 0.00000 -0.04478 0.22440 0.18968 0.40000 0.37313 0.89055
SimplifiedLOF 29 0.66667 0.65174 0.41063 0.38424 0.66667 0.65174 0.77612
LoOP 10 0.33333 0.30348 0.28968 0.25788 0.40000 0.37313 0.92040
LoOP 18 0.33333 0.30348 0.31111 0.28027 0.57143 0.55224 0.84577
LoOP 33 0.33333 0.30348 0.35333 0.32438 0.57143 0.55224 0.75124
LDOF 7 0.33333 0.30348 0.27485 0.24238 0.44444 0.41957 0.89055
LDOF 13 0.33333 0.30348 0.32137 0.29098 0.50000 0.47761 0.92537
LDOF 14 0.33333 0.30348 0.34028 0.31074 0.57143 0.55224 0.91542
LDOF 43 0.33333 0.30348 0.35606 0.32723 0.57143 0.55224 0.78109
ODIN 9 0.11111 0.07131 0.18095 0.14428 0.30769 0.27669 0.88557
ODIN 14 0.33333 0.30348 0.23665 0.20247 0.40000 0.37313 0.83085
ODIN 68 0.33333 0.30348 0.34762 0.31841 0.57143 0.55224 0.80846
FastABOD 4 0.33333 0.30348 0.26230 0.22927 0.50000 0.47761 0.71144
FastABOD 8 0.33333 0.30348 0.26717 0.23436 0.50000 0.47761 0.77114
KDEOS 14 0.00000 -0.04478 0.19870 0.16282 0.35294 0.32397 0.88060
KDEOS 61 0.66667 0.65174 0.57881 0.55995 0.66667 0.65174 0.79602
LDF 3 0.00000 -0.04478 0.17513 0.13820 0.35294 0.32397 0.86567
LDF 9 0.66667 0.65174 0.41667 0.39055 0.66667 0.65174 0.82587
INFLO 7 0.33333 0.30348 0.27137 0.23874 0.37500 0.34701 0.91045
INFLO 25 0.66667 0.65174 0.40359 0.37689 0.66667 0.65174 0.74129
COF 8 0.00000 -0.04478 0.16389 0.12645 0.33333 0.30348 0.85821
COF 18 0.66667 0.65174 0.40427 0.37760 0.66667 0.65174 0.68408

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO