Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#04)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (207.7 kB) Download raw algorithm evaluation table (12.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.10096 0.06350 0.22222 0.18981 0.72917
KNN 2 0.00000 -0.04167 0.12143 0.08482 0.25000 0.21875 0.78125
KNNW 1 0.00000 -0.04167 0.11688 0.08009 0.25000 0.21875 0.77083
LOF 2 0.50000 0.47917 0.61111 0.59491 0.66667 0.65278 0.92708
LOF 3 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
SimplifiedLOF 4 0.50000 0.47917 0.36111 0.33449 0.50000 0.47917 0.91667
SimplifiedLOF 5 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
LoOP 5 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
LDOF 6 0.50000 0.47917 0.39286 0.36756 0.50000 0.47917 0.93750
LDOF 9 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
ODIN 2 0.18182 0.14773 0.18182 0.14773 0.30769 0.27885 0.90625
ODIN 3 0.20000 0.16667 0.17143 0.13690 0.28571 0.25595 0.89583
FastABOD 3 0.00000 -0.04167 0.14167 0.10590 0.28571 0.25595 0.72917
FastABOD 4 0.00000 -0.04167 0.18382 0.14982 0.33333 0.30556 0.81250
KDEOS 5 0.50000 0.47917 0.34091 0.31345 0.50000 0.47917 0.89583
KDEOS 10 0.50000 0.47917 0.45000 0.42708 0.57143 0.55357 0.95833
LDF 1 0.50000 0.47917 0.59091 0.57386 0.66667 0.65278 0.90625
INFLO 3 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 5 0.50000 0.47917 0.45000 0.42708 0.57143 0.55357 0.95833
COF 6 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.4 kB) Download raw algorithm evaluation table (10.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.19444 0.16088 0.36364 0.33712 0.87500
KNNW 1 0.00000 -0.04167 0.07632 0.03783 0.18182 0.14773 0.62500
KNNW 3 0.00000 -0.04167 0.16234 0.12744 0.30769 0.27885 0.84375
KNNW 8 0.00000 -0.04167 0.17424 0.13984 0.30769 0.27885 0.85417
LOF 1 0.00000 -0.04167 0.05347 0.01403 0.11111 0.07407 0.39583
LOF 2 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
SimplifiedLOF 1 0.00000 -0.04167 0.04000 0.00000 0.07692 0.03846 0.25000
SimplifiedLOF 2 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LoOP 1 0.00000 -0.04167 0.04000 0.00000 0.07692 0.03846 0.36979
LoOP 6 0.00000 -0.04167 0.29167 0.26215 0.50000 0.47917 0.92708
LoOP 13 0.00000 -0.04167 0.33333 0.30556 0.50000 0.47917 0.93750
LDOF 6 0.50000 0.47917 0.39286 0.36756 0.50000 0.47917 0.93750
LDOF 7 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
LDOF 14 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
ODIN 5 0.08333 0.04514 0.13810 0.10218 0.23529 0.20343 0.86458
ODIN 10 0.00000 -0.04167 0.33333 0.30556 0.50000 0.47917 0.93750
FastABOD 3 0.00000 -0.04167 0.10263 0.06524 0.19048 0.15675 0.72917
KDEOS 17 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
LDF 2 0.50000 0.47917 0.66667 0.65278 0.66667 0.65278 0.95833
INFLO 9 0.00000 -0.04167 0.36667 0.34028 0.57143 0.55357 0.94792
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 1 0.00000 -0.04167 0.04000 0.00000 0.07692 0.03846 0.25000
COF 4 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO