Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#03)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (207.5 kB) Download raw algorithm evaluation table (3.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
KNNW 1 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LOF 3 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
SimplifiedLOF 6 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LoOP 4 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LDOF 8 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
ODIN 15 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
FastABOD 3 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
KDEOS 13 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LDF 2 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
INFLO 3 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
COF 3 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (209.1 kB) Download raw algorithm evaluation table (6.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.21875 0.33333 0.30556 0.50000 0.47917 0.94271
KNN 33 0.50000 0.47917 0.28571 0.25595 0.50000 0.47917 0.71875
KNNW 1 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
KNNW 3 0.50000 0.47917 0.70000 0.68750 0.66667 0.65278 0.96875
LOF 4 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
SimplifiedLOF 1 0.50000 0.47917 0.32692 0.29888 0.50000 0.47917 0.87500
SimplifiedLOF 4 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.96875
SimplifiedLOF 9 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
LoOP 1 0.50000 0.47917 0.32692 0.29888 0.50000 0.47917 0.87500
LoOP 12 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
LDOF 2 0.50000 0.47917 0.34091 0.31345 0.50000 0.47917 0.89583
LDOF 12 0.50000 0.47917 0.64286 0.62798 0.66667 0.65278 0.94792
LDOF 14 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
ODIN 3 0.33333 0.30556 0.20833 0.17535 0.40000 0.37500 0.83333
ODIN 7 0.33333 0.30556 0.30952 0.28075 0.44444 0.42130 0.95312
ODIN 9 0.20000 0.16667 0.33333 0.30556 0.50000 0.47917 0.94792
FastABOD 3 0.50000 0.47917 0.39286 0.36756 0.50000 0.47917 0.93750
FastABOD 6 0.50000 0.47917 0.45000 0.42708 0.57143 0.55357 0.95833
FastABOD 16 0.50000 0.47917 0.66667 0.65278 0.66667 0.65278 0.95833
KDEOS 3 0.50000 0.47917 0.53846 0.51923 0.66667 0.65278 0.75000
KDEOS 20 0.50000 0.47917 0.45000 0.42708 0.57143 0.55357 0.95833
KDEOS 27 0.50000 0.47917 0.66667 0.65278 0.66667 0.65278 0.95833
LDF 2 0.50000 0.47917 0.53571 0.51637 0.66667 0.65278 0.72917
LDF 4 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
INFLO 4 0.50000 0.47917 0.50000 0.47917 0.66667 0.65278 0.96875
INFLO 9 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
INFLO 49 0.51020 0.48980 0.52000 0.50000 0.66667 0.65278 0.75000
COF 7 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO