Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (5% of outliers version#09)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (207.7 kB) Download raw algorithm evaluation table (6.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.04167 0.33333 0.30556 0.50000 0.47917 0.93750
KNN 20 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
KNNW 1 0.50000 0.47917 0.45000 0.42708 0.57143 0.55357 0.95833
KNNW 2 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LOF 1 0.00000 -0.04167 0.16234 0.12744 0.30769 0.27885 0.84375
LOF 36 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
SimplifiedLOF 1 0.00000 -0.04167 0.15341 0.11813 0.30769 0.27885 0.83333
SimplifiedLOF 38 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LoOP 1 0.00000 -0.04167 0.15341 0.11813 0.30769 0.27885 0.83333
LoOP 35 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LDOF 2 0.00000 -0.04167 0.13095 0.09474 0.25000 0.21875 0.75000
LDOF 24 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
ODIN 6 0.50000 0.47917 0.27778 0.24769 0.50000 0.47917 0.67188
ODIN 22 0.00000 -0.04167 0.32500 0.29688 0.57143 0.55357 0.93750
ODIN 44 0.00000 -0.04167 0.36667 0.34028 0.57143 0.55357 0.95312
FastABOD 3 0.50000 0.47917 0.39286 0.36756 0.50000 0.47917 0.93750
FastABOD 4 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
FastABOD 39 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
KDEOS 41 0.50000 0.47917 0.39286 0.36756 0.50000 0.47917 0.93750
KDEOS 42 0.50000 0.47917 0.64286 0.62798 0.66667 0.65278 0.94792
LDF 6 0.50000 0.47917 0.41667 0.39236 0.50000 0.47917 0.94792
LDF 21 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
LDF 27 0.50000 0.47917 0.45000 0.42708 0.57143 0.55357 0.95833
INFLO 23 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
INFLO 49 0.02041 -0.02041 0.03020 -0.01020 0.07692 0.03846 0.25000
COF 10 0.50000 0.47917 0.28333 0.25347 0.50000 0.47917 0.69792
COF 13 0.50000 0.47917 0.57143 0.55357 0.66667 0.65278 0.87500
COF 42 0.50000 0.47917 0.60000 0.58333 0.66667 0.65278 0.91667

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 50 objects, 2 outliers (4.00%)

Download raw algorithm results (208.7 kB) Download raw algorithm evaluation table (8.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.21875 0.24359 0.21207 0.40000 0.37500 0.86979
KNNW 1 0.50000 0.47917 0.28846 0.25881 0.50000 0.47917 0.74479
KNNW 2 0.50000 0.47917 0.54348 0.52446 0.66667 0.65278 0.78125
KNNW 49 0.00000 -0.04167 0.21930 0.18677 0.40000 0.37500 0.80208
LOF 1 0.00000 -0.04167 0.03613 -0.00403 0.07692 0.03846 0.18750
LOF 3 0.00000 -0.04167 0.21111 0.17824 0.36364 0.33712 0.88542
LOF 5 0.00000 -0.04167 0.25000 0.21875 0.40000 0.37500 0.87500
SimplifiedLOF 1 0.00000 -0.04167 0.03042 -0.00998 0.07692 0.03846 0.13021
SimplifiedLOF 9 0.00000 -0.04167 0.25000 0.21875 0.40000 0.37500 0.90625
LoOP 1 0.00000 -0.04167 0.04000 0.00000 0.07692 0.03846 0.25000
LoOP 9 0.00000 -0.04167 0.19643 0.16295 0.40000 0.37500 0.87500
LoOP 48 0.00000 -0.04167 0.21667 0.18403 0.40000 0.37500 0.79167
LDOF 2 0.00000 -0.04167 0.04273 0.00284 0.08333 0.04514 0.28125
LDOF 13 0.00000 -0.04167 0.17361 0.13918 0.36364 0.33712 0.85417
LDOF 18 0.00000 -0.04167 0.19231 0.15865 0.40000 0.37500 0.59375
LDOF 49 0.00000 -0.04167 0.21930 0.18677 0.40000 0.37500 0.80208
ODIN 20 0.33333 0.30556 0.20238 0.16915 0.40000 0.37500 0.72917
ODIN 35 0.33333 0.30556 0.23333 0.20139 0.40000 0.37500 0.85417
FastABOD 5 0.50000 0.47917 0.29167 0.26215 0.50000 0.47917 0.76042
FastABOD 6 0.50000 0.47917 0.54348 0.52446 0.66667 0.65278 0.78125
KDEOS 11 0.50000 0.47917 0.33333 0.30556 0.50000 0.47917 0.88542
KDEOS 12 0.00000 -0.04167 0.23611 0.20428 0.36364 0.33712 0.89583
LDF 1 0.00000 -0.04167 0.03613 -0.00403 0.07692 0.03846 0.18750
LDF 2 0.00000 -0.04167 0.30952 0.28075 0.44444 0.42130 0.92708
INFLO 46 0.00000 -0.04167 0.41667 0.39236 0.66667 0.65278 0.95833
INFLO 49 0.02041 -0.02041 0.03020 -0.01020 0.07692 0.03846 0.25000
COF 1 0.00000 -0.04167 0.03042 -0.00998 0.07692 0.03846 0.13021
COF 4 0.00000 -0.04167 0.22500 0.19271 0.40000 0.37500 0.89583

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO