Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#07)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.0 kB) Download raw algorithm evaluation table (23.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.09051 0.04979 0.17143 0.13433 0.67662
KNN 33 0.00000 -0.04478 0.13259 0.09375 0.31579 0.28515 0.81095
KNN 35 0.00000 -0.04478 0.13725 0.09862 0.31579 0.28515 0.82090
KNN 37 0.00000 -0.04478 0.13675 0.09809 0.30000 0.26866 0.82338
KNNW 1 0.00000 -0.04478 0.07255 0.03102 0.15385 0.11596 0.56468
KNNW 51 0.00000 -0.04478 0.10096 0.06070 0.22222 0.18740 0.74129
LOF 8 0.33333 0.30348 0.26275 0.22973 0.40000 0.37313 0.80100
LOF 11 0.33333 0.30348 0.30217 0.27092 0.57143 0.55224 0.79104
LOF 19 0.33333 0.30348 0.30882 0.27788 0.40000 0.37313 0.89552
SimplifiedLOF 11 0.33333 0.30348 0.21289 0.17765 0.40000 0.37313 0.61692
SimplifiedLOF 12 0.33333 0.30348 0.38203 0.35436 0.50000 0.47761 0.62687
SimplifiedLOF 22 0.33333 0.30348 0.42460 0.39884 0.50000 0.47761 0.82587
SimplifiedLOF 30 0.00000 -0.04478 0.19630 0.16031 0.33333 0.30348 0.87065
LoOP 12 0.33333 0.30348 0.21088 0.17554 0.40000 0.37313 0.61194
LoOP 13 0.33333 0.30348 0.39041 0.36311 0.50000 0.47761 0.66169
LoOP 21 0.33333 0.30348 0.43237 0.40695 0.50000 0.47761 0.85075
LoOP 29 0.00000 -0.04478 0.17456 0.13760 0.33333 0.30348 0.86070
LDOF 13 0.33333 0.30348 0.20374 0.16809 0.40000 0.37313 0.57711
LDOF 14 0.33333 0.30348 0.37567 0.34772 0.50000 0.47761 0.63184
LDOF 29 0.33333 0.30348 0.41800 0.39194 0.50000 0.47761 0.83085
LDOF 39 0.00000 -0.04478 0.16477 0.12737 0.31579 0.28515 0.85572
ODIN 21 0.00000 -0.04478 0.17320 0.13618 0.30000 0.26866 0.85323
ODIN 22 0.08333 0.04229 0.15263 0.11469 0.27273 0.24016 0.85572
ODIN 26 0.00000 -0.04478 0.15581 0.11801 0.27273 0.24016 0.86318
FastABOD 3 0.00000 -0.04478 0.08739 0.04652 0.20000 0.16418 0.58209
FastABOD 69 0.00000 -0.04478 0.10067 0.06041 0.19048 0.15423 0.72637
KDEOS 2 0.33333 0.30348 0.13817 0.09958 0.33333 0.30348 0.42786
KDEOS 23 0.33333 0.30348 0.20295 0.16726 0.40000 0.37313 0.56716
KDEOS 41 0.00000 -0.04478 0.17857 0.14179 0.36364 0.33514 0.85075
LDF 6 0.33333 0.30348 0.22398 0.18923 0.40000 0.37313 0.70149
LDF 68 0.00000 -0.04478 0.14268 0.10429 0.26667 0.23383 0.82090
INFLO 12 0.33333 0.30348 0.14141 0.10297 0.33333 0.30348 0.43781
INFLO 13 0.33333 0.30348 0.36801 0.33971 0.50000 0.47761 0.57463
INFLO 16 0.33333 0.30348 0.42045 0.39450 0.50000 0.47761 0.83582
INFLO 19 0.00000 -0.04478 0.17350 0.13650 0.28571 0.25373 0.85075
COF 9 0.33333 0.30348 0.35333 0.32438 0.57143 0.55224 0.75124
COF 10 0.33333 0.30348 0.48627 0.46327 0.50000 0.47761 0.74627
COF 14 0.00000 -0.04478 0.13571 0.09701 0.25000 0.21642 0.76617

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.6 kB) Download raw algorithm evaluation table (22.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.04874 0.00615 0.10909 0.06920 0.44279
KNN 68 0.00000 -0.04478 0.09548 0.05498 0.23529 0.20105 0.67164
KNNW 1 0.00000 -0.04478 0.04940 0.00683 0.12000 0.08060 0.45274
KNNW 2 0.00000 -0.04478 0.04959 0.00703 0.12245 0.08316 0.45274
KNNW 3 0.00000 -0.04478 0.05002 0.00749 0.11321 0.07350 0.45274
LOF 1 0.00000 -0.04478 0.05853 0.01638 0.12903 0.09003 0.53234
LOF 3 0.00000 -0.04478 0.09942 0.05910 0.24000 0.20597 0.74129
SimplifiedLOF 1 0.00000 -0.04478 0.04133 -0.00159 0.08333 0.04229 0.37313
SimplifiedLOF 8 0.00000 -0.04478 0.09064 0.04993 0.18182 0.14518 0.62189
LoOP 1 0.00000 -0.04478 0.04092 -0.00203 0.08219 0.04110 0.36816
LoOP 6 0.00000 -0.04478 0.10159 0.06136 0.22222 0.18740 0.60199
LoOP 8 0.00000 -0.04478 0.08380 0.04278 0.20000 0.16418 0.61194
LDOF 2 0.00000 -0.04478 0.07623 0.03486 0.14634 0.10812 0.61692
LDOF 6 0.00000 -0.04478 0.13362 0.09482 0.28571 0.25373 0.61692
LDOF 8 0.00000 -0.04478 0.09197 0.05131 0.21053 0.17518 0.64677
ODIN 4 0.04762 0.00498 0.08518 0.04421 0.16667 0.12935 0.63184
ODIN 8 0.00000 -0.04478 0.07996 0.03876 0.18182 0.14518 0.65174
FastABOD 3 0.00000 -0.04478 0.04886 0.00627 0.10000 0.05970 0.43284
FastABOD 14 0.00000 -0.04478 0.04816 0.00554 0.11111 0.07131 0.42289
KDEOS 2 0.33333 0.30348 0.36537 0.33695 0.50000 0.47761 0.60448
KDEOS 10 0.00000 -0.04478 0.10547 0.06541 0.26667 0.23383 0.66169
LDF 1 0.00000 -0.04478 0.07155 0.02998 0.16000 0.12239 0.61194
LDF 3 0.00000 -0.04478 0.07795 0.03666 0.14286 0.10448 0.64677
INFLO 2 0.00000 -0.04478 0.11045 0.07062 0.22222 0.18740 0.66915
INFLO 5 0.00000 -0.04478 0.09892 0.05858 0.25000 0.21642 0.52239
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 1 0.00000 -0.04478 0.04133 -0.00159 0.08333 0.04229 0.37313
COF 5 0.00000 -0.04478 0.09714 0.05672 0.21429 0.17910 0.73632

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO