Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Glass

A forensic dataset describing types of glass. This version is based on a publication by Keller et al. (HiCS, [1]). The authors use the class 6 (minority) as outlier and all other classes as inliers. Unfortunately, they do not detail which attributes have been used. The original dataset consists of 9 attributes, here are only 7 attributes present. The dataset contains 214 instances, 9 outliers (21.4%) and 205 inliers (95.79%). This dataset contains only one duplicate, so we did not create a version without duplicates.

References:

[1] F. Keller, E. Mueller, and K. Boehm. HiCS: high contrast subspaces for density-based outlier ranking. In Proc. ICDE, 2012.

Download all data set variants used (6.4 kB). You can also access the original data. (real world datasets)

Normalized, without duplicates

This version contains 7 attributes, 214 objects, 9 outliers (4.21%)

Download raw algorithm results (1.9 MB) Download raw algorithm evaluation table (43.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.11111 0.07209 0.17023 0.13380 0.26316 0.23081 0.84878
KNN 2 0.11111 0.07209 0.18013 0.14414 0.28070 0.24912 0.86938
KNN 8 0.11111 0.07209 0.16719 0.13063 0.33333 0.30407 0.87480
KNNW 1 0.11111 0.07209 0.26950 0.23743 0.31250 0.28232 0.88320
KNNW 18 0.11111 0.07209 0.16163 0.12483 0.32143 0.29164 0.86775
LOF 1 0.22222 0.18808 0.25253 0.21972 0.30769 0.27730 0.67642
LOF 2 0.33333 0.30407 0.24963 0.21669 0.37500 0.34756 0.58808
LOF 11 0.22222 0.18808 0.17597 0.13979 0.29091 0.25978 0.86667
SimplifiedLOF 2 0.33333 0.30407 0.29350 0.26248 0.42105 0.39564 0.64173
SimplifiedLOF 16 0.22222 0.18808 0.17872 0.14266 0.29167 0.26057 0.86504
LoOP 3 0.33333 0.30407 0.20530 0.17041 0.38095 0.35377 0.67317
LoOP 7 0.22222 0.18808 0.21357 0.17905 0.28571 0.25436 0.61762
LoOP 18 0.22222 0.18808 0.16504 0.12839 0.25352 0.22075 0.83957
LDOF 3 0.22222 0.18808 0.09103 0.05112 0.22222 0.18808 0.58699
LDOF 18 0.11111 0.07209 0.18615 0.15042 0.20000 0.16488 0.73333
LDOF 27 0.11111 0.07209 0.12045 0.08183 0.20253 0.16752 0.77886
ODIN 4 0.22222 0.18808 0.09277 0.05294 0.23529 0.20172 0.53930
ODIN 6 0.22222 0.18808 0.12655 0.08820 0.28571 0.25436 0.55501
ODIN 18 0.15556 0.11848 0.19218 0.15671 0.20000 0.16488 0.72927
ODIN 21 0.22222 0.18808 0.20724 0.17244 0.23529 0.20172 0.71220
FastABOD 3 0.11111 0.07209 0.11137 0.07236 0.21053 0.17587 0.78862
FastABOD 30 0.11111 0.07209 0.17422 0.13797 0.25714 0.22453 0.84715
FastABOD 98 0.11111 0.07209 0.15484 0.11774 0.28125 0.24970 0.85799
KDEOS 3 0.11111 0.07209 0.07930 0.03888 0.18182 0.14590 0.50678
KDEOS 19 0.00000 -0.04390 0.10165 0.06221 0.25806 0.22549 0.71762
KDEOS 28 0.00000 -0.04390 0.08669 0.04660 0.20000 0.16488 0.74201
LDF 1 0.33333 0.30407 0.19610 0.16081 0.35294 0.32453 0.65257
LDF 2 0.33333 0.30407 0.22024 0.18600 0.37500 0.34756 0.60488
LDF 3 0.22222 0.18808 0.24423 0.21105 0.30769 0.27730 0.62385
LDF 9 0.22222 0.18808 0.23156 0.19783 0.34043 0.31147 0.90352
INFLO 2 0.33333 0.30407 0.24181 0.20852 0.33333 0.30407 0.69621
INFLO 3 0.33333 0.30407 0.29042 0.25927 0.40000 0.37366 0.74959
INFLO 18 0.22222 0.18808 0.12832 0.09005 0.22951 0.19568 0.80379
COF 3 0.33333 0.30407 0.28992 0.25874 0.40000 0.37366 0.61680
COF 57 0.00000 -0.04390 0.19099 0.15547 0.41379 0.38806 0.87859
COF 76 0.11111 0.07209 0.18813 0.15249 0.37838 0.35109 0.89539

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO