Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#10)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.7 kB) Download raw algorithm evaluation table (20.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.11162 0.07185 0.20000 0.16418 0.73881
KNN 12 0.00000 -0.04478 0.20556 0.16998 0.44444 0.41957 0.79602
KNN 50 0.00000 -0.04478 0.17920 0.14244 0.37500 0.34701 0.87065
KNNW 1 0.00000 -0.04478 0.10160 0.06138 0.20000 0.16418 0.68657
KNNW 9 0.00000 -0.04478 0.17335 0.13634 0.28571 0.25373 0.78607
KNNW 30 0.00000 -0.04478 0.13819 0.09960 0.30769 0.27669 0.79104
KNNW 69 0.00000 -0.04478 0.13939 0.10086 0.28571 0.25373 0.81592
LOF 8 0.33333 0.30348 0.17475 0.13780 0.33333 0.30348 0.75124
LOF 15 0.00000 -0.04478 0.27579 0.24337 0.54545 0.52510 0.92537
LOF 21 0.33333 0.30348 0.34028 0.31074 0.57143 0.55224 0.91542
SimplifiedLOF 11 0.33333 0.30348 0.21528 0.18014 0.40000 0.37313 0.68159
SimplifiedLOF 12 0.33333 0.30348 0.38757 0.36015 0.50000 0.47761 0.72139
SimplifiedLOF 23 0.33333 0.30348 0.44348 0.41856 0.50000 0.47761 0.86070
SimplifiedLOF 29 0.00000 -0.04478 0.21667 0.18159 0.40000 0.37313 0.89552
LoOP 12 0.33333 0.30348 0.40079 0.37396 0.50000 0.47761 0.78109
LoOP 22 0.33333 0.30348 0.45644 0.43210 0.50000 0.47761 0.89055
LoOP 25 0.33333 0.30348 0.28327 0.25118 0.40000 0.37313 0.91542
LDOF 13 0.33333 0.30348 0.24405 0.21020 0.40000 0.37313 0.80100
LDOF 14 0.33333 0.30348 0.40546 0.37884 0.50000 0.47761 0.79602
LDOF 31 0.33333 0.30348 0.45128 0.42671 0.50000 0.47761 0.88557
ODIN 13 0.33333 0.30348 0.18519 0.14870 0.33333 0.30348 0.82338
ODIN 24 0.26667 0.23383 0.28889 0.25705 0.44444 0.41957 0.92537
ODIN 25 0.22222 0.18740 0.26667 0.23383 0.46154 0.43743 0.93035
FastABOD 3 0.00000 -0.04478 0.09669 0.05624 0.19048 0.15423 0.70149
FastABOD 8 0.00000 -0.04478 0.16169 0.12416 0.33333 0.30348 0.82587
FastABOD 53 0.00000 -0.04478 0.16574 0.12839 0.33333 0.30348 0.84577
KDEOS 20 0.33333 0.30348 0.16535 0.12798 0.33333 0.30348 0.71144
KDEOS 23 0.33333 0.30348 0.23814 0.20402 0.40000 0.37313 0.76119
KDEOS 32 0.00000 -0.04478 0.21061 0.17526 0.40000 0.37313 0.89055
LDF 9 0.00000 -0.04478 0.20217 0.16644 0.44444 0.41957 0.77114
LDF 11 0.33333 0.30348 0.20051 0.16471 0.33333 0.30348 0.74627
LDF 43 0.00000 -0.04478 0.14259 0.10420 0.26667 0.23383 0.82587
LDF 64 0.33333 0.30348 0.20333 0.16766 0.40000 0.37313 0.57214
INFLO 12 0.33333 0.30348 0.24523 0.21144 0.40000 0.37313 0.81095
INFLO 13 0.33333 0.30348 0.46991 0.44617 0.50000 0.47761 0.90050
INFLO 16 0.33333 0.30348 0.49091 0.46811 0.50000 0.47761 0.92040
COF 9 0.33333 0.30348 0.16364 0.12619 0.33333 0.30348 0.68159
COF 14 0.00000 -0.04478 0.23889 0.20481 0.50000 0.47761 0.76119
COF 67 0.33333 0.30348 0.44074 0.41570 0.50000 0.47761 0.83085

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.3 kB) Download raw algorithm evaluation table (22.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.04155 -0.00137 0.10909 0.06920 0.33582
KNN 68 0.00000 -0.04478 0.10183 0.06162 0.23529 0.20105 0.72139
KNNW 1 0.00000 -0.04478 0.03480 -0.00841 0.08451 0.04351 0.21642
KNNW 3 0.00000 -0.04478 0.04087 -0.00207 0.10909 0.06920 0.31841
KNNW 7 0.00000 -0.04478 0.04107 -0.00187 0.09375 0.05317 0.33333
KNNW 69 0.00000 -0.04478 0.04139 -0.00154 0.10714 0.06716 0.33333
LOF 1 0.00000 -0.04478 0.07585 0.03447 0.18182 0.14518 0.50746
LOF 5 0.00000 -0.04478 0.09352 0.05293 0.22222 0.18740 0.72139
SimplifiedLOF 1 0.00000 -0.04478 0.04348 0.00065 0.08333 0.04229 0.26119
SimplifiedLOF 5 0.00000 -0.04478 0.05401 0.01166 0.12500 0.08582 0.45274
LoOP 1 0.00000 -0.04478 0.04286 -0.00000 0.08219 0.04110 0.25373
LoOP 5 0.00000 -0.04478 0.06213 0.02014 0.16000 0.12239 0.52985
LDOF 2 0.00000 -0.04478 0.04452 0.00174 0.10000 0.05970 0.37811
LDOF 5 0.00000 -0.04478 0.08995 0.04920 0.19048 0.15423 0.58209
ODIN 7 0.00000 -0.04478 0.07516 0.03375 0.14815 0.11001 0.59204
ODIN 69 0.04286 -0.00000 0.04286 -0.00000 0.08219 0.04110 0.50000
FastABOD 3 0.00000 -0.04478 0.04563 0.00290 0.10169 0.06147 0.40299
KDEOS 2 0.33333 0.30348 0.36186 0.33329 0.50000 0.47761 0.47264
KDEOS 19 0.00000 -0.04478 0.05574 0.01346 0.11111 0.07131 0.50249
LDF 1 0.00000 -0.04478 0.09280 0.05218 0.22222 0.18740 0.55721
INFLO 4 0.00000 -0.04478 0.13283 0.09400 0.28571 0.25373 0.70647
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 1 0.00000 -0.04478 0.04348 0.00065 0.08333 0.04229 0.26119
COF 8 0.00000 -0.04478 0.05166 0.00920 0.12245 0.08316 0.47761
COF 11 0.00000 -0.04478 0.05282 0.01041 0.10000 0.05970 0.44776

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO