Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#04)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (300.9 kB) Download raw algorithm evaluation table (24.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.58333 0.68298 0.60373 0.76923 0.71154 0.90451
KNN 37 0.66667 0.58333 0.68832 0.61041 0.72727 0.65909 0.77778
KNNW 2 0.58333 0.47917 0.68549 0.60686 0.66667 0.58333 0.90104
KNNW 3 0.58333 0.47917 0.64438 0.55547 0.68966 0.61207 0.88194
KNNW 6 0.66667 0.58333 0.64202 0.55253 0.66667 0.58333 0.86458
LOF 10 0.50000 0.37500 0.64391 0.55488 0.58824 0.48529 0.83681
LOF 35 0.66667 0.58333 0.64705 0.55881 0.66667 0.58333 0.81076
LOF 41 0.66667 0.58333 0.67384 0.59230 0.66667 0.58333 0.82118
LOF 44 0.66667 0.58333 0.64738 0.55923 0.69565 0.61957 0.76215
SimplifiedLOF 13 0.58333 0.47917 0.57427 0.46784 0.60870 0.51087 0.82812
SimplifiedLOF 15 0.50000 0.37500 0.63142 0.53927 0.59459 0.49324 0.83681
SimplifiedLOF 48 0.58333 0.47917 0.62096 0.52620 0.66667 0.58333 0.77083
LoOP 15 0.50000 0.37500 0.56156 0.45195 0.57895 0.47368 0.79688
LoOP 48 0.58333 0.47917 0.57720 0.47150 0.60870 0.51087 0.75260
LoOP 50 0.58333 0.47917 0.58092 0.47615 0.63636 0.54545 0.75260
LoOP 55 0.58333 0.47917 0.61036 0.51295 0.63636 0.54545 0.76302
LDOF 28 0.58333 0.47917 0.58042 0.47552 0.58333 0.47917 0.77778
LDOF 56 0.58333 0.47917 0.60470 0.50588 0.63636 0.54545 0.75174
ODIN 27 0.58333 0.47917 0.52649 0.40812 0.60870 0.51087 0.69010
ODIN 30 0.58333 0.47917 0.57981 0.47477 0.66667 0.58333 0.71701
ODIN 43 0.58333 0.47917 0.59649 0.49561 0.63636 0.54545 0.75347
ODIN 48 0.58333 0.47917 0.60781 0.50977 0.63636 0.54545 0.74132
FastABOD 3 0.58333 0.47917 0.63657 0.54572 0.74074 0.67593 0.91319
FastABOD 6 0.75000 0.68750 0.71392 0.64240 0.75000 0.68750 0.90451
FastABOD 11 0.75000 0.68750 0.71411 0.64264 0.75000 0.68750 0.90625
KDEOS 11 0.58333 0.47917 0.52116 0.40145 0.60000 0.50000 0.82986
KDEOS 12 0.58333 0.47917 0.49877 0.37346 0.62069 0.52586 0.84549
KDEOS 21 0.41667 0.27083 0.44448 0.30559 0.64516 0.55645 0.81771
KDEOS 59 0.50000 0.37500 0.62976 0.53721 0.61538 0.51923 0.81250
LDF 7 0.66667 0.58333 0.70997 0.63746 0.76190 0.70238 0.85938
LDF 27 0.66667 0.58333 0.71653 0.64566 0.70000 0.62500 0.87153
LDF 31 0.58333 0.47917 0.72392 0.65490 0.70000 0.62500 0.86632
INFLO 32 0.66667 0.58333 0.68338 0.60422 0.74074 0.67593 0.87153
INFLO 40 0.66667 0.58333 0.69793 0.62242 0.66667 0.58333 0.85938
COF 8 0.58333 0.47917 0.62109 0.52636 0.68966 0.61207 0.87500
COF 10 0.58333 0.47917 0.56557 0.45697 0.75862 0.69828 0.90278

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (297.7 kB) Download raw algorithm evaluation table (24.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.33333 0.16667 0.34128 0.17660 0.53333 0.41667 0.69531
KNN 55 0.41667 0.27083 0.37803 0.22254 0.51282 0.39103 0.71441
KNNW 1 0.41667 0.27083 0.34038 0.17548 0.46154 0.32692 0.67361
KNNW 2 0.33333 0.16667 0.36055 0.20069 0.44000 0.30000 0.68576
LOF 2 0.33333 0.16667 0.36438 0.20547 0.50000 0.37500 0.71615
LOF 13 0.50000 0.37500 0.36474 0.20593 0.51852 0.39815 0.70312
LOF 15 0.50000 0.37500 0.35466 0.19332 0.52174 0.40217 0.67708
LOF 52 0.41667 0.27083 0.37667 0.22083 0.50000 0.37500 0.69444
SimplifiedLOF 8 0.50000 0.37500 0.36819 0.21024 0.51613 0.39516 0.75347
SimplifiedLOF 18 0.58333 0.47917 0.36658 0.20822 0.58333 0.47917 0.68403
SimplifiedLOF 19 0.50000 0.37500 0.37527 0.21908 0.56000 0.45000 0.67708
LoOP 7 0.33333 0.16667 0.34854 0.18568 0.51613 0.39516 0.73090
LoOP 9 0.41667 0.27083 0.33867 0.17334 0.55172 0.43966 0.70833
LoOP 10 0.50000 0.37500 0.33404 0.16755 0.55172 0.43966 0.64931
LoOP 19 0.50000 0.37500 0.38091 0.22614 0.52174 0.40217 0.67361
LDOF 6 0.50000 0.37500 0.41486 0.26857 0.54545 0.43182 0.64931
LDOF 7 0.50000 0.37500 0.41912 0.27390 0.57143 0.46429 0.72569
LDOF 8 0.50000 0.37500 0.42461 0.28076 0.57143 0.46429 0.74479
ODIN 2 0.34524 0.18155 0.31078 0.13847 0.43137 0.28922 0.69184
ODIN 10 0.50000 0.37500 0.35218 0.19023 0.50000 0.37500 0.66667
ODIN 12 0.44444 0.30556 0.42585 0.28231 0.47619 0.34524 0.66059
ODIN 18 0.50000 0.37500 0.39067 0.23834 0.52174 0.40217 0.65712
FastABOD 3 0.33333 0.16667 0.30975 0.13719 0.40000 0.25000 0.58854
FastABOD 14 0.25000 0.06250 0.38232 0.22791 0.39130 0.23913 0.59896
FastABOD 18 0.25000 0.06250 0.31836 0.14795 0.38298 0.22872 0.60069
KDEOS 8 0.33333 0.16667 0.55579 0.44473 0.53333 0.41667 0.78472
KDEOS 14 0.58333 0.47917 0.41797 0.27246 0.64000 0.55000 0.74826
LDF 10 0.58333 0.47917 0.42375 0.27969 0.64000 0.55000 0.74826
INFLO 11 0.58333 0.47917 0.37818 0.22272 0.58333 0.47917 0.66493
INFLO 17 0.50000 0.37500 0.44713 0.30891 0.66667 0.58333 0.81771
COF 6 0.41667 0.27083 0.37927 0.22409 0.54545 0.43182 0.77604
COF 10 0.41667 0.27083 0.39665 0.24581 0.54054 0.42568 0.79340
COF 11 0.41667 0.27083 0.41240 0.26550 0.55172 0.43966 0.77257
COF 25 0.25000 0.06250 0.35588 0.19485 0.56410 0.45513 0.73785

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO