Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#06)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.2 kB) Download raw algorithm evaluation table (5.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.50000 0.47917 0.70000 0.68750 0.66667 0.65278 0.96875
KNNW 1 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
KNNW 2 0.50000 0.47917 0.75000 0.73958 0.66667 0.65278 0.97917
LOF 5 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
SimplifiedLOF 7 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LoOP 5 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LDOF 11 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
ODIN 12 0.75000 0.73958 0.83333 0.82639 0.80000 0.79167 0.99479
FastABOD 3 0.50000 0.47917 0.64286 0.62798 0.66667 0.65278 0.94792
FastABOD 5 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
FastABOD 6 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
KDEOS 10 0.50000 0.47917 0.37500 0.34896 0.50000 0.47917 0.92708
KDEOS 31 0.50000 0.47917 0.58333 0.56597 0.80000 0.79167 0.97917
KDEOS 44 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958
LDF 3 0.50000 0.47917 0.57143 0.55357 0.66667 0.65278 0.87500
LDF 18 0.50000 0.47917 0.36111 0.33449 0.50000 0.47917 0.91667
LDF 22 0.50000 0.47917 0.60000 0.58333 0.66667 0.65278 0.91667
INFLO 3 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
COF 3 0.50000 0.47917 0.35000 0.32292 0.50000 0.47917 0.90625
COF 5 0.50000 0.47917 0.83333 0.82639 0.80000 0.79167 0.98958

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (207.7 kB) Download raw algorithm evaluation table (11.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.16234 0.12744 0.30769 0.27885 0.84896
KNN 5 0.00000 -0.04167 0.26667 0.23611 0.50000 0.47917 0.91667
KNNW 1 0.00000 -0.04167 0.22500 0.19271 0.40000 0.37500 0.90104
KNNW 11 0.00000 -0.04167 0.24286 0.21131 0.44444 0.42130 0.90625
LOF 1 0.00000 -0.04167 0.05214 0.01264 0.11111 0.07407 0.45312
LOF 12 0.00000 -0.04167 0.19091 0.15720 0.30769 0.27885 0.86458
SimplifiedLOF 1 0.00000 -0.04167 0.04051 0.00053 0.08163 0.04337 0.36979
SimplifiedLOF 14 0.00000 -0.04167 0.13704 0.10108 0.28571 0.25595 0.69792
SimplifiedLOF 19 0.00000 -0.04167 0.17143 0.13690 0.28571 0.25595 0.83333
LoOP 1 0.00000 -0.04167 0.03923 -0.00080 0.07692 0.03846 0.35417
LoOP 14 0.00000 -0.04167 0.12000 0.08333 0.28571 0.25595 0.57292
LoOP 17 0.00000 -0.04167 0.14167 0.10590 0.28571 0.25595 0.72917
LDOF 2 0.00000 -0.04167 0.04372 0.00388 0.10811 0.07095 0.32292
LDOF 22 0.00000 -0.04167 0.09954 0.06202 0.20000 0.16667 0.66667
ODIN 8 0.00000 -0.04167 0.12500 0.08854 0.22222 0.18981 0.75000
ODIN 14 0.00000 -0.04167 0.09348 0.05571 0.16667 0.13194 0.75521
ODIN 49 0.04000 0.00000 0.04000 0.00000 0.07692 0.03846 0.50000
FastABOD 3 0.00000 -0.04167 0.12879 0.09249 0.28571 0.25595 0.79167
FastABOD 30 0.00000 -0.04167 0.17424 0.13984 0.30769 0.27885 0.85417
KDEOS 2 0.00000 -0.04167 0.04113 0.00118 0.09524 0.05754 0.41146
KDEOS 23 0.00000 -0.04167 0.29167 0.26215 0.40000 0.37500 0.91667
LDF 1 0.00000 -0.04167 0.05441 0.01501 0.11111 0.07407 0.47396
LDF 10 0.00000 -0.04167 0.32500 0.29688 0.57143 0.55357 0.93750
INFLO 8 0.00000 -0.04167 0.13596 0.09996 0.25000 0.21875 0.77083
INFLO 49 0.02041 -0.02041 0.03020 -0.01020 0.07692 0.03846 0.25000
COF 1 0.00000 -0.04167 0.04051 0.00053 0.08163 0.04337 0.36979
COF 11 0.00000 -0.04167 0.16667 0.13194 0.28571 0.25595 0.82292

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO