Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Hepatitis (5% of outliers version#05)

A data set for prediction whether a patient suffering from hepatitis will die (outliers) or survive (inliers).

Download all data set variants used (21.2 kB). You can also access the original data. (hepatitis.data)

Normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (420.1 kB) Download raw algorithm evaluation table (23.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 19 0.33333 0.30348 0.20833 0.17289 0.33333 0.30348 0.83582
KNN 20 0.33333 0.30348 0.21014 0.17478 0.33333 0.30348 0.84080
KNN 44 0.00000 -0.04478 0.18978 0.15350 0.30000 0.26866 0.85572
KNNW 1 0.00000 -0.04478 0.07890 0.03765 0.15385 0.11596 0.61443
KNNW 26 0.00000 -0.04478 0.13594 0.09725 0.25000 0.21642 0.79602
KNNW 49 0.00000 -0.04478 0.14484 0.10655 0.23529 0.20105 0.81095
KNNW 63 0.00000 -0.04478 0.15278 0.11484 0.25000 0.21642 0.81095
LOF 1 0.00000 -0.04478 0.04155 -0.00137 0.09524 0.05473 0.33831
LOF 35 0.00000 -0.04478 0.17350 0.13650 0.28571 0.25373 0.85075
LOF 37 0.00000 -0.04478 0.15772 0.12001 0.30000 0.26866 0.84577
LOF 43 0.00000 -0.04478 0.18434 0.14782 0.28571 0.25373 0.84080
SimplifiedLOF 1 0.00000 -0.04478 0.04286 -0.00000 0.08219 0.04110 0.23134
SimplifiedLOF 66 0.00000 -0.04478 0.17126 0.13415 0.28571 0.25373 0.82090
LoOP 1 0.00000 -0.04478 0.04286 -0.00000 0.08219 0.04110 0.23134
LoOP 64 0.00000 -0.04478 0.16800 0.13075 0.28571 0.25373 0.81592
LoOP 69 0.00000 -0.04478 0.17443 0.13746 0.28571 0.25373 0.82587
LDOF 2 0.00000 -0.04478 0.03691 -0.00621 0.09375 0.05317 0.25373
LDOF 65 0.00000 -0.04478 0.15350 0.11560 0.25000 0.21642 0.81592
LDOF 67 0.00000 -0.04478 0.16848 0.13125 0.28571 0.25373 0.81592
ODIN 41 0.00000 -0.04478 0.16993 0.13277 0.30000 0.26866 0.85323
ODIN 58 0.33333 0.30348 0.20000 0.16418 0.33333 0.30348 0.80597
FastABOD 3 0.33333 0.30348 0.22857 0.19403 0.40000 0.37313 0.75622
KDEOS 2 0.00000 -0.04478 0.03039 -0.01303 0.08571 0.04478 0.06965
KDEOS 61 0.00000 -0.04478 0.11473 0.07509 0.22222 0.18740 0.72139
KDEOS 63 0.00000 -0.04478 0.11926 0.07982 0.22222 0.18740 0.74129
KDEOS 68 0.00000 -0.04478 0.11507 0.07544 0.20690 0.17138 0.75124
LDF 7 0.00000 -0.04478 0.18526 0.14878 0.37500 0.34701 0.87562
LDF 12 0.33333 0.30348 0.40833 0.38184 0.50000 0.47761 0.80597
INFLO 41 0.00000 -0.04478 0.13704 0.09840 0.28571 0.25373 0.69154
INFLO 60 0.33333 0.30348 0.13968 0.10116 0.33333 0.30348 0.60697
COF 37 0.33333 0.30348 0.24603 0.21227 0.44444 0.41957 0.77612
COF 49 0.33333 0.30348 0.30719 0.27617 0.44444 0.41957 0.82090
COF 57 0.33333 0.30348 0.28333 0.25124 0.40000 0.37313 0.87065

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 19 attributes, 70 objects, 3 outliers (4.29%)

Download raw algorithm results (421.0 kB) Download raw algorithm evaluation table (22.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04478 0.06317 0.02122 0.13953 0.10101 0.57960
KNN 11 0.00000 -0.04478 0.09786 0.05747 0.22222 0.18740 0.70398
KNN 19 0.00000 -0.04478 0.09461 0.05407 0.19355 0.15744 0.72637
KNN 66 0.00000 -0.04478 0.09287 0.05225 0.25000 0.21642 0.37811
KNNW 1 0.00000 -0.04478 0.06866 0.02696 0.13953 0.10101 0.58706
KNNW 14 0.00000 -0.04478 0.08783 0.04699 0.19048 0.15423 0.69154
KNNW 41 0.00000 -0.04478 0.08761 0.04676 0.18182 0.14518 0.69652
KNNW 65 0.00000 -0.04478 0.08386 0.04284 0.20000 0.16418 0.65672
LOF 1 0.00000 -0.04478 0.05968 0.01758 0.12903 0.09003 0.54229
LOF 11 0.00000 -0.04478 0.13183 0.09296 0.28571 0.25373 0.66667
LOF 12 0.00000 -0.04478 0.13214 0.09328 0.28571 0.25373 0.67164
LOF 67 0.00000 -0.04478 0.12753 0.08846 0.26667 0.23383 0.76617
SimplifiedLOF 1 0.00000 -0.04478 0.04859 0.00599 0.10000 0.05970 0.41294
SimplifiedLOF 25 0.00000 -0.04478 0.10167 0.06144 0.18182 0.14518 0.71642
SimplifiedLOF 29 0.00000 -0.04478 0.10172 0.06150 0.20000 0.16418 0.69154
SimplifiedLOF 31 0.00000 -0.04478 0.10552 0.06547 0.20000 0.16418 0.71144
LoOP 1 0.00000 -0.04478 0.04818 0.00556 0.10000 0.05970 0.40796
LoOP 23 0.00000 -0.04478 0.10060 0.06033 0.20000 0.16418 0.68159
LoOP 26 0.00000 -0.04478 0.11111 0.07131 0.20000 0.16418 0.73632
LDOF 2 0.00000 -0.04478 0.05858 0.01643 0.14286 0.10448 0.54229
LDOF 33 0.00000 -0.04478 0.09883 0.05848 0.20000 0.16418 0.65672
LDOF 52 0.00000 -0.04478 0.09236 0.05172 0.17143 0.13433 0.70149
ODIN 1 0.04545 0.00271 0.05152 0.00905 0.10345 0.06330 0.58458
ODIN 21 0.00000 -0.04478 0.09683 0.05638 0.22222 0.18740 0.62189
ODIN 31 0.00000 -0.04478 0.10588 0.06585 0.20000 0.16418 0.71891
ODIN 38 0.00000 -0.04478 0.11056 0.07074 0.21053 0.17518 0.68905
FastABOD 3 0.00000 -0.04478 0.06746 0.02570 0.14286 0.10448 0.60199
FastABOD 14 0.00000 -0.04478 0.08194 0.04083 0.16216 0.12465 0.67662
FastABOD 37 0.00000 -0.04478 0.08417 0.04316 0.17391 0.13692 0.67662
KDEOS 2 0.33333 0.30348 0.36752 0.33920 0.50000 0.47761 0.62935
KDEOS 55 0.00000 -0.04478 0.13755 0.09893 0.28571 0.25373 0.70647
LDF 4 0.33333 0.30348 0.14359 0.10524 0.33333 0.30348 0.49751
LDF 5 0.33333 0.30348 0.20698 0.17147 0.40000 0.37313 0.56716
LDF 6 0.33333 0.30348 0.22698 0.19237 0.40000 0.37313 0.74129
LDF 67 0.33333 0.30348 0.17475 0.13780 0.33333 0.30348 0.75622
INFLO 18 0.00000 -0.04478 0.07839 0.03712 0.20000 0.16418 0.49751
INFLO 37 0.00000 -0.04478 0.07988 0.03868 0.18182 0.14518 0.63433
INFLO 45 0.00000 -0.04478 0.07956 0.03835 0.20000 0.16418 0.65174
INFLO 69 0.02899 -0.01449 0.04286 -0.00000 0.08219 0.04110 0.49254
COF 13 0.33333 0.30348 0.17116 0.13405 0.33333 0.30348 0.69154
COF 16 0.33333 0.30348 0.23007 0.19560 0.40000 0.37313 0.71393
COF 17 0.33333 0.30348 0.24234 0.20841 0.40000 0.37313 0.75124
COF 20 0.33333 0.30348 0.23495 0.20070 0.40000 0.37313 0.77114

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO