Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#10)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.0 kB) Download raw algorithm evaluation table (5.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 26 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.96875
KNN 28 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
KNNW 1 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
KNNW 44 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.96875
LOF 7 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
SimplifiedLOF 6 0.50000 0.47917 0.60000 0.58333 0.66667 0.65278 0.91667
SimplifiedLOF 13 0.50000 0.47917 0.70000 0.68750 0.66667 0.65278 0.96875
LoOP 6 0.50000 0.47917 0.34091 0.31345 0.50000 0.47917 0.89583
LoOP 15 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LoOP 40 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.96875
LDOF 13 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
LDOF 18 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
ODIN 10 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.97396
ODIN 11 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.97917
FastABOD 5 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
FastABOD 20 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
KDEOS 2 0.50000 0.47917 0.52632 0.50658 0.66667 0.65278 0.62500
KDEOS 46 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LDF 2 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
LDF 30 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
INFLO 30 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
INFLO 47 0.50000 0.47917 0.70000 0.68750 0.66667 0.65278 0.96875
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 4 0.50000 0.47917 0.32692 0.29888 0.50000 0.47917 0.87500
COF 9 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.2 kB) Download raw algorithm evaluation table (10.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.08712 0.04908 0.15385 0.11859 0.66667
KNN 20 0.00000 -0.04167 0.25000 0.21875 0.40000 0.37500 0.90625
KNN 48 0.00000 -0.04167 0.27778 0.24769 0.40000 0.37500 0.90625
KNNW 1 0.00000 -0.04167 0.12500 0.08854 0.25000 0.21875 0.71875
KNNW 44 0.00000 -0.04167 0.18750 0.15365 0.33333 0.30556 0.82292
KNNW 49 0.00000 -0.04167 0.21591 0.18324 0.33333 0.30556 0.87500
LOF 1 0.00000 -0.04167 0.08333 0.04514 0.20000 0.16667 0.44792
LOF 24 0.00000 -0.04167 0.25000 0.21875 0.40000 0.37500 0.90625
SimplifiedLOF 1 0.00000 -0.04167 0.14000 0.10417 0.28571 0.25595 0.71875
SimplifiedLOF 2 0.00000 -0.04167 0.21667 0.18403 0.40000 0.37500 0.79167
SimplifiedLOF 49 0.00000 -0.04167 0.21591 0.18324 0.33333 0.30556 0.87500
LoOP 1 0.00000 -0.04167 0.14000 0.10417 0.28571 0.25595 0.71875
LoOP 49 0.00000 -0.04167 0.22500 0.19271 0.33333 0.30556 0.88542
LDOF 2 0.00000 -0.04167 0.10819 0.07103 0.19048 0.15675 0.73958
LDOF 3 0.00000 -0.04167 0.21429 0.18155 0.40000 0.37500 0.78125
LDOF 15 0.00000 -0.04167 0.23810 0.20635 0.40000 0.37500 0.85417
LDOF 44 0.00000 -0.04167 0.21591 0.18324 0.33333 0.30556 0.87500
ODIN 3 0.33333 0.30556 0.24359 0.21207 0.40000 0.37500 0.92188
ODIN 5 0.50000 0.47917 0.28448 0.25467 0.50000 0.47917 0.75000
FastABOD 3 0.00000 -0.04167 0.12879 0.09249 0.25000 0.21875 0.73958
KDEOS 15 0.50000 0.47917 0.34091 0.31345 0.50000 0.47917 0.89583
KDEOS 16 0.50000 0.47917 0.35000 0.32292 0.50000 0.47917 0.90625
LDF 1 0.00000 -0.04167 0.03365 -0.00661 0.08000 0.04167 0.12500
LDF 21 0.00000 -0.04167 0.21111 0.17824 0.36364 0.33712 0.88542
LDF 24 0.00000 -0.04167 0.20833 0.17535 0.40000 0.37500 0.88542
LDF 33 0.00000 -0.04167 0.21591 0.18324 0.33333 0.30556 0.87500
INFLO 48 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 1 0.00000 -0.04167 0.14000 0.10417 0.28571 0.25595 0.71875
COF 2 0.00000 -0.04167 0.21667 0.18403 0.40000 0.37500 0.79167
COF 10 0.00000 -0.04167 0.18254 0.14848 0.36364 0.33712 0.86458

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO