Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#08)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (207.8 kB) Download raw algorithm evaluation table (12.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.10101 0.06355 0.20000 0.16667 0.72917
KNN 47 0.00000 -0.04167 0.10882 0.07169 0.21053 0.17763 0.75000
KNN 48 0.00000 -0.04167 0.11111 0.07407 0.20000 0.16667 0.75000
KNNW 1 0.00000 -0.04167 0.07632 0.03783 0.18182 0.14773 0.62500
KNNW 2 0.00000 -0.04167 0.09454 0.05681 0.21053 0.17763 0.70833
LOF 2 0.50000 0.47917 0.55263 0.53399 0.66667 0.65278 0.82292
LOF 3 0.50000 0.47917 0.60000 0.58333 0.66667 0.65278 0.91667
SimplifiedLOF 1 0.00000 -0.04167 0.04174 0.00181 0.08000 0.04167 0.27083
SimplifiedLOF 4 0.00000 -0.04167 0.19643 0.16295 0.33333 0.30556 0.84375
LoOP 1 0.00000 -0.04167 0.04174 0.00181 0.08000 0.04167 0.40625
LoOP 4 0.00000 -0.04167 0.14583 0.11024 0.28571 0.25595 0.82292
LDOF 2 0.00000 -0.04167 0.03469 -0.00553 0.07843 0.04003 0.15625
LDOF 5 0.00000 -0.04167 0.10507 0.06778 0.25000 0.21875 0.48958
LDOF 6 0.00000 -0.04167 0.10965 0.07255 0.25000 0.21875 0.57292
ODIN 4 0.00000 -0.04167 0.10598 0.06873 0.20000 0.16667 0.76042
ODIN 49 0.04000 0.00000 0.04000 0.00000 0.07692 0.03846 0.50000
FastABOD 3 0.00000 -0.04167 0.04147 0.00153 0.10000 0.06250 0.29167
FastABOD 5 0.00000 -0.04167 0.05299 0.01353 0.11765 0.08088 0.45833
FastABOD 8 0.00000 -0.04167 0.05662 0.01731 0.11429 0.07738 0.48958
KDEOS 2 0.00000 -0.04167 0.03598 -0.00418 0.08000 0.04167 0.32812
KDEOS 11 0.00000 -0.04167 0.26786 0.23735 0.44444 0.42130 0.91667
LDF 1 0.00000 -0.04167 0.06548 0.02654 0.14286 0.10714 0.46875
LDF 2 0.00000 -0.04167 0.29167 0.26215 0.40000 0.37500 0.91667
LDF 3 0.00000 -0.04167 0.26786 0.23735 0.44444 0.42130 0.91667
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 1 0.00000 -0.04167 0.04174 0.00181 0.08000 0.04167 0.27083
COF 4 0.00000 -0.04167 0.23333 0.20139 0.40000 0.37500 0.84375
COF 6 0.00000 -0.04167 0.23810 0.20635 0.40000 0.37500 0.85417
COF 7 0.00000 -0.04167 0.20833 0.17535 0.33333 0.30556 0.86458

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.6 kB) Download raw algorithm evaluation table (10.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.19444 0.16088 0.36364 0.33712 0.87500
KNN 3 0.00000 -0.04167 0.20000 0.16667 0.33333 0.30556 0.87500
KNNW 1 0.00000 -0.04167 0.06621 0.02730 0.16000 0.12500 0.56250
KNNW 3 0.00000 -0.04167 0.16234 0.12744 0.30769 0.27885 0.84375
KNNW 7 0.00000 -0.04167 0.17424 0.13984 0.30769 0.27885 0.85417
KNNW 9 0.00000 -0.04167 0.18333 0.14931 0.28571 0.25595 0.85417
LOF 1 0.00000 -0.04167 0.06167 0.02257 0.14286 0.10714 0.38542
LOF 2 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
SimplifiedLOF 1 0.00000 -0.04167 0.04000 0.00000 0.07692 0.03846 0.25000
SimplifiedLOF 2 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LoOP 6 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
LDOF 6 0.50000 0.47917 0.36111 0.33449 0.50000 0.47917 0.91667
LDOF 7 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
LDOF 13 0.00000 -0.04167 0.36667 0.34028 0.57143 0.55357 0.94792
ODIN 5 0.08333 0.04514 0.13810 0.10218 0.23529 0.20343 0.86458
ODIN 10 0.00000 -0.04167 0.29167 0.26215 0.50000 0.47917 0.93750
FastABOD 3 0.00000 -0.04167 0.09762 0.06002 0.17391 0.13949 0.70833
KDEOS 17 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
LDF 2 0.50000 0.47917 0.66667 0.65278 0.66667 0.65278 0.95833
INFLO 9 0.00000 -0.04167 0.36667 0.34028 0.57143 0.55357 0.94792
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 1 0.00000 -0.04167 0.04000 0.00000 0.07692 0.03846 0.25000
COF 4 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO