Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#07)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (207.6 kB) Download raw algorithm evaluation table (8.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.50000 0.47917 0.54167 0.52257 0.66667 0.65278 0.77083
KNN 31 0.50000 0.47917 0.60000 0.58333 0.66667 0.65278 0.91667
KNNW 1 0.50000 0.47917 0.56250 0.54427 0.66667 0.65278 0.85417
LOF 2 0.50000 0.47917 0.39286 0.36756 0.50000 0.47917 0.93750
LOF 3 0.00000 -0.04167 0.32500 0.29688 0.57143 0.55357 0.93750
LOF 47 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
SimplifiedLOF 3 0.50000 0.47917 0.33333 0.30556 0.50000 0.47917 0.88542
SimplifiedLOF 4 0.50000 0.47917 0.37500 0.34896 0.50000 0.47917 0.92708
LoOP 3 0.50000 0.47917 0.33333 0.30556 0.50000 0.47917 0.88542
LoOP 4 0.00000 -0.04167 0.30952 0.28075 0.44444 0.42130 0.92708
LDOF 8 0.00000 -0.04167 0.23611 0.20428 0.36364 0.33712 0.89583
LDOF 18 0.50000 0.47917 0.28030 0.25032 0.50000 0.47917 0.66667
LDOF 45 0.50000 0.47917 0.29545 0.26610 0.50000 0.47917 0.78125
ODIN 2 0.20000 0.16667 0.20000 0.16667 0.33333 0.30556 0.91667
ODIN 10 0.50000 0.47917 0.27500 0.24479 0.50000 0.47917 0.59896
ODIN 12 0.50000 0.47917 0.52703 0.50732 0.66667 0.65278 0.65625
ODIN 19 0.50000 0.47917 0.53846 0.51923 0.66667 0.65278 0.76562
FastABOD 3 0.50000 0.47917 0.31250 0.28385 0.50000 0.47917 0.84375
FastABOD 4 0.50000 0.47917 0.57143 0.55357 0.66667 0.65278 0.87500
KDEOS 9 0.00000 -0.04167 0.33333 0.30556 0.50000 0.47917 0.93750
KDEOS 41 0.50000 0.47917 0.27778 0.24769 0.50000 0.47917 0.63542
KDEOS 45 0.50000 0.47917 0.52857 0.50893 0.66667 0.65278 0.65625
LDF 4 0.50000 0.47917 0.32692 0.29888 0.50000 0.47917 0.87500
LDF 22 0.50000 0.47917 0.55882 0.54044 0.66667 0.65278 0.84375
INFLO 3 0.50000 0.47917 0.34091 0.31345 0.50000 0.47917 0.89583
INFLO 31 0.50000 0.47917 0.64286 0.62798 0.66667 0.65278 0.94792
INFLO 32 0.50000 0.47917 0.66667 0.65278 0.66667 0.65278 0.95833
COF 4 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.5 kB) Download raw algorithm evaluation table (12.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.07879 0.04040 0.15385 0.11859 0.60417
KNN 47 0.00000 -0.04167 0.08681 0.04876 0.20000 0.16667 0.67708
KNNW 1 0.00000 -0.04167 0.07846 0.04006 0.14815 0.11265 0.63542
LOF 1 0.00000 -0.04167 0.08893 0.05097 0.16000 0.12500 0.67708
LOF 2 0.00000 -0.04167 0.17143 0.13690 0.28571 0.25595 0.83333
SimplifiedLOF 1 0.00000 -0.04167 0.08893 0.05097 0.16000 0.12500 0.67708
SimplifiedLOF 2 0.00000 -0.04167 0.13333 0.09722 0.28571 0.25595 0.80208
LoOP 1 0.00000 -0.04167 0.08893 0.05097 0.16000 0.12500 0.67708
LoOP 2 0.00000 -0.04167 0.13942 0.10357 0.26667 0.23611 0.81250
LDOF 2 0.00000 -0.04167 0.12222 0.08565 0.23529 0.20343 0.78125
ODIN 11 0.00000 -0.04167 0.14722 0.11169 0.33333 0.30556 0.53125
ODIN 30 0.00000 -0.04167 0.07879 0.04040 0.16667 0.13194 0.66667
ODIN 49 0.04000 0.00000 0.04000 0.00000 0.07692 0.03846 0.50000
FastABOD 3 0.00000 -0.04167 0.06645 0.02755 0.13793 0.10201 0.57292
KDEOS 3 0.50000 0.47917 0.30000 0.27083 0.50000 0.47917 0.80208
KDEOS 7 0.00000 -0.04167 0.16667 0.13194 0.28571 0.25595 0.82292
LDF 1 0.00000 -0.04167 0.07994 0.04160 0.15385 0.11859 0.61458
LDF 22 0.00000 -0.04167 0.16026 0.12527 0.26667 0.23611 0.83333
INFLO 2 0.00000 -0.04167 0.21591 0.18324 0.33333 0.30556 0.87500
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 1 0.00000 -0.04167 0.08893 0.05097 0.16000 0.12500 0.67708
COF 6 0.00000 -0.04167 0.12698 0.09061 0.25000 0.21875 0.79167

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO