Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#01)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (207.9 kB) Download raw algorithm evaluation table (4.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
KNNW 1 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
KNNW 2 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
LOF 1 0.50000 0.47917 0.36111 0.33449 0.50000 0.47917 0.91667
LOF 2 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
LOF 12 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
SimplifiedLOF 4 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
SimplifiedLOF 14 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
LoOP 3 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
LoOP 4 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
LDOF 8 0.50000 0.47917 0.45000 0.42708 0.57143 0.55357 0.95833
LDOF 14 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
ODIN 8 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.97917
ODIN 21 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
FastABOD 4 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
FastABOD 16 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
KDEOS 12 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.96875
KDEOS 13 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
KDEOS 47 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
LDF 7 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
INFLO 12 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 4 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
COF 5 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.7 kB) Download raw algorithm evaluation table (10.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.20833 0.17535 0.40000 0.37500 0.88542
KNN 5 0.00000 -0.04167 0.26667 0.23611 0.50000 0.47917 0.91667
KNNW 1 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
LOF 12 0.00000 -0.04167 0.23611 0.20428 0.36364 0.33712 0.89583
LOF 48 0.50000 0.47917 0.27941 0.24939 0.50000 0.47917 0.65625
SimplifiedLOF 1 0.00000 -0.04167 0.09007 0.05216 0.21053 0.17763 0.68750
SimplifiedLOF 3 0.00000 -0.04167 0.21667 0.18403 0.40000 0.37500 0.79167
SimplifiedLOF 19 0.00000 -0.04167 0.19643 0.16295 0.40000 0.37500 0.87500
LoOP 1 0.00000 -0.04167 0.09007 0.05216 0.21053 0.17763 0.68750
LoOP 3 0.00000 -0.04167 0.21429 0.18155 0.40000 0.37500 0.78125
LoOP 5 0.00000 -0.04167 0.23333 0.20139 0.40000 0.37500 0.84375
LoOP 15 0.00000 -0.04167 0.21591 0.18324 0.33333 0.30556 0.87500
LDOF 2 0.00000 -0.04167 0.06782 0.02898 0.12903 0.09274 0.57292
LDOF 5 0.00000 -0.04167 0.23333 0.20139 0.40000 0.37500 0.84375
LDOF 6 0.00000 -0.04167 0.26667 0.23611 0.40000 0.37500 0.89583
ODIN 2 0.25000 0.21875 0.15278 0.11748 0.33333 0.30556 0.71875
ODIN 9 0.00000 -0.04167 0.11905 0.08234 0.22222 0.18981 0.75521
FastABOD 3 0.00000 -0.04167 0.22500 0.19271 0.40000 0.37500 0.89583
FastABOD 6 0.00000 -0.04167 0.26667 0.23611 0.50000 0.47917 0.91667
KDEOS 14 0.50000 0.47917 0.32143 0.29315 0.50000 0.47917 0.86458
KDEOS 22 0.50000 0.47917 0.37500 0.34896 0.50000 0.47917 0.92708
LDF 2 0.50000 0.47917 0.56250 0.54427 0.66667 0.65278 0.85417
INFLO 13 0.00000 -0.04167 0.29167 0.26215 0.50000 0.47917 0.92708
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 5 0.50000 0.47917 0.31250 0.28385 0.50000 0.47917 0.84375
COF 6 0.50000 0.47917 0.56667 0.54861 0.66667 0.65278 0.86458
COF 7 0.50000 0.47917 0.59091 0.57386 0.66667 0.65278 0.90625
COF 14 0.00000 -0.04167 0.29167 0.26215 0.40000 0.37500 0.91667

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO