Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#09)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.1 kB) Download raw algorithm evaluation table (21.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.33333 0.30348 0.21674 0.18167 0.40000 0.37313 0.69154
KNN 47 0.00000 -0.04478 0.13175 0.09287 0.23529 0.20105 0.75622
KNNW 1 0.00000 -0.04478 0.10292 0.06275 0.20000 0.16418 0.69652
KNNW 2 0.00000 -0.04478 0.10785 0.06791 0.20000 0.16418 0.72139
KNNW 6 0.00000 -0.04478 0.13447 0.09571 0.28571 0.25373 0.68657
KNNW 67 0.00000 -0.04478 0.13995 0.10144 0.28571 0.25373 0.71144
LOF 7 0.33333 0.30348 0.31746 0.28690 0.40000 0.37313 0.89552
LOF 10 0.33333 0.30348 0.38095 0.35323 0.57143 0.55224 0.89552
LOF 13 0.33333 0.30348 0.28105 0.24885 0.44444 0.41957 0.90050
LOF 17 0.33333 0.30348 0.44074 0.41570 0.50000 0.47761 0.83085
SimplifiedLOF 10 0.33333 0.30348 0.17115 0.13403 0.33333 0.30348 0.74129
SimplifiedLOF 12 0.33333 0.30348 0.41212 0.38580 0.50000 0.47761 0.81592
SimplifiedLOF 18 0.33333 0.30348 0.29365 0.26202 0.40000 0.37313 0.89055
SimplifiedLOF 21 0.33333 0.30348 0.47403 0.45047 0.50000 0.47761 0.88060
LoOP 18 0.33333 0.30348 0.34762 0.31841 0.50000 0.47761 0.89055
LoOP 22 0.33333 0.30348 0.53030 0.50927 0.57143 0.55224 0.84080
LoOP 23 0.66667 0.65174 0.42115 0.39523 0.66667 0.65174 0.85075
LDOF 31 0.66667 0.65174 0.59903 0.58108 0.66667 0.65174 0.89552
ODIN 9 0.33333 0.30348 0.17143 0.13433 0.33333 0.30348 0.76866
ODIN 12 0.33333 0.30348 0.20040 0.16459 0.33333 0.30348 0.85572
ODIN 17 0.33333 0.30348 0.41093 0.38455 0.50000 0.47761 0.77363
FastABOD 4 0.33333 0.30348 0.30635 0.27529 0.57143 0.55224 0.82090
FastABOD 6 0.33333 0.30348 0.41071 0.38433 0.50000 0.47761 0.80597
FastABOD 53 0.33333 0.30348 0.25873 0.22554 0.40000 0.37313 0.84080
KDEOS 18 0.33333 0.30348 0.15683 0.11907 0.33333 0.30348 0.65174
KDEOS 20 0.33333 0.30348 0.21700 0.18194 0.40000 0.37313 0.69154
KDEOS 23 0.33333 0.30348 0.22329 0.18851 0.40000 0.37313 0.72139
KDEOS 44 0.00000 -0.04478 0.14394 0.10561 0.28571 0.25373 0.82587
LDF 3 0.00000 -0.04478 0.24028 0.20626 0.44444 0.41957 0.89552
LDF 7 0.33333 0.30348 0.20515 0.16956 0.40000 0.37313 0.59204
LDF 11 0.33333 0.30348 0.37576 0.34781 0.50000 0.47761 0.63682
INFLO 10 0.00000 -0.04478 0.18472 0.14822 0.31579 0.28515 0.86567
INFLO 23 0.66667 0.65174 0.40359 0.37689 0.66667 0.65174 0.73383
INFLO 27 0.66667 0.65174 0.57026 0.55102 0.66667 0.65174 0.74627
COF 9 0.33333 0.30348 0.36508 0.33665 0.60000 0.58209 0.95025
COF 11 0.33333 0.30348 0.60000 0.58209 0.57143 0.55224 0.95522

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.7 kB) Download raw algorithm evaluation table (20.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.33333 0.30348 0.15028 0.11224 0.33333 0.30348 0.57214
KNN 66 0.33333 0.30348 0.16141 0.12387 0.33333 0.30348 0.65672
KNNW 1 0.00000 -0.04478 0.13360 0.09480 0.28571 0.25373 0.66418
KNNW 3 0.33333 0.30348 0.15580 0.11800 0.33333 0.30348 0.63184
LOF 2 0.00000 -0.04478 0.16706 0.12976 0.33333 0.30348 0.77114
LOF 14 0.33333 0.30348 0.16285 0.12537 0.33333 0.30348 0.58706
LOF 15 0.33333 0.30348 0.22372 0.18896 0.40000 0.37313 0.61692
LOF 17 0.33333 0.30348 0.22674 0.19211 0.40000 0.37313 0.62687
SimplifiedLOF 7 0.33333 0.30348 0.13715 0.09851 0.33333 0.30348 0.38308
SimplifiedLOF 17 0.33333 0.30348 0.20784 0.17237 0.40000 0.37313 0.60199
SimplifiedLOF 18 0.33333 0.30348 0.20954 0.17415 0.40000 0.37313 0.61194
SimplifiedLOF 32 0.33333 0.30348 0.21178 0.17649 0.40000 0.37313 0.59701
LoOP 7 0.33333 0.30348 0.13968 0.10116 0.33333 0.30348 0.44279
LoOP 19 0.33333 0.30348 0.20394 0.16830 0.40000 0.37313 0.56716
LoOP 35 0.33333 0.30348 0.21126 0.17594 0.40000 0.37313 0.60945
LoOP 53 0.00000 -0.04478 0.12540 0.08624 0.28571 0.25373 0.62935
LDOF 6 0.33333 0.30348 0.13773 0.09912 0.33333 0.30348 0.39801
LDOF 27 0.33333 0.30348 0.19979 0.16396 0.40000 0.37313 0.52239
LDOF 37 0.33333 0.30348 0.20451 0.16890 0.40000 0.37313 0.55224
LDOF 63 0.33333 0.30348 0.15764 0.11992 0.33333 0.30348 0.61692
ODIN 4 0.33333 0.30348 0.13884 0.10028 0.33333 0.30348 0.47015
ODIN 40 0.33333 0.30348 0.15889 0.12123 0.33333 0.30348 0.65174
ODIN 45 0.00000 -0.04478 0.12078 0.08141 0.25000 0.21642 0.69403
FastABOD 4 0.33333 0.30348 0.15105 0.11304 0.33333 0.30348 0.57711
FastABOD 53 0.33333 0.30348 0.16071 0.12313 0.33333 0.30348 0.63184
KDEOS 48 0.33333 0.30348 0.15649 0.11872 0.33333 0.30348 0.59701
KDEOS 55 0.33333 0.30348 0.21454 0.17937 0.40000 0.37313 0.60697
KDEOS 58 0.33333 0.30348 0.37698 0.34909 0.50000 0.47761 0.59204
KDEOS 60 0.33333 0.30348 0.37951 0.35173 0.50000 0.47761 0.60199
LDF 1 0.33333 0.30348 0.14300 0.10462 0.33333 0.30348 0.49502
LDF 3 0.00000 -0.04478 0.12628 0.08716 0.25000 0.21642 0.79602
LDF 9 0.33333 0.30348 0.25119 0.21766 0.40000 0.37313 0.69154
LDF 10 0.33333 0.30348 0.26786 0.23507 0.40000 0.37313 0.70149
INFLO 7 0.33333 0.30348 0.14188 0.10346 0.33333 0.30348 0.49254
INFLO 18 0.33333 0.30348 0.19652 0.16054 0.40000 0.37313 0.49254
INFLO 21 0.33333 0.30348 0.19697 0.16101 0.40000 0.37313 0.48756
INFLO 43 0.00000 -0.04478 0.12834 0.08931 0.28571 0.25373 0.67164
COF 2 0.33333 0.30348 0.20519 0.16960 0.40000 0.37313 0.59204
COF 14 0.33333 0.30348 0.26176 0.22870 0.40000 0.37313 0.82090

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO