Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#06)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (299.0 kB) Download raw algorithm evaluation table (22.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.58333 0.47917 0.71649 0.64562 0.66667 0.58333 0.84288
KNNW 1 0.58333 0.47917 0.73827 0.67283 0.70000 0.62500 0.84462
KNNW 2 0.58333 0.47917 0.74752 0.68440 0.73684 0.67105 0.85590
LOF 3 0.75000 0.68750 0.71510 0.64388 0.75000 0.68750 0.83854
LOF 4 0.66667 0.58333 0.72479 0.65598 0.69565 0.61957 0.84201
SimplifiedLOF 6 0.75000 0.68750 0.80719 0.75899 0.78261 0.72826 0.92014
LoOP 4 0.66667 0.58333 0.63396 0.54244 0.66667 0.58333 0.87674
LoOP 6 0.66667 0.58333 0.72995 0.66244 0.71429 0.64286 0.87934
LoOP 7 0.66667 0.58333 0.71717 0.64647 0.72000 0.65000 0.87847
LoOP 8 0.66667 0.58333 0.74541 0.68176 0.72000 0.65000 0.85938
LDOF 11 0.66667 0.58333 0.67728 0.59659 0.66667 0.58333 0.81944
LDOF 12 0.66667 0.58333 0.71668 0.64585 0.72727 0.65909 0.84549
LDOF 13 0.66667 0.58333 0.72149 0.65186 0.72000 0.65000 0.84896
ODIN 8 0.51852 0.39815 0.38834 0.23542 0.54545 0.43182 0.71181
ODIN 9 0.50000 0.37500 0.40868 0.26085 0.51852 0.39815 0.74740
ODIN 39 0.41667 0.27083 0.56102 0.45127 0.58824 0.48529 0.61198
ODIN 49 0.50000 0.37500 0.59841 0.49802 0.58824 0.48529 0.67795
FastABOD 5 0.66667 0.58333 0.77243 0.71554 0.73684 0.67105 0.84201
FastABOD 6 0.66667 0.58333 0.77562 0.71952 0.73684 0.67105 0.84549
KDEOS 13 0.66667 0.58333 0.54534 0.43167 0.76923 0.71154 0.84375
KDEOS 34 0.58333 0.47917 0.64528 0.55660 0.63636 0.54545 0.76910
LDF 2 0.50000 0.37500 0.67270 0.59087 0.66667 0.58333 0.79167
LDF 3 0.58333 0.47917 0.63900 0.54875 0.70000 0.62500 0.79340
LDF 17 0.58333 0.47917 0.65605 0.57007 0.60870 0.51087 0.80903
INFLO 4 0.66667 0.58333 0.77374 0.71718 0.76190 0.70238 0.90451
INFLO 49 0.68000 0.60000 0.67731 0.59664 0.72727 0.65909 0.81597
COF 6 0.75000 0.68750 0.73480 0.66849 0.75000 0.68750 0.88889

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (298.9 kB) Download raw algorithm evaluation table (25.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.41667 0.27083 0.33101 0.16377 0.51613 0.39516 0.66233
KNNW 1 0.50000 0.37500 0.38651 0.23314 0.53333 0.41667 0.69184
LOF 2 0.33333 0.16667 0.39289 0.24111 0.44444 0.30556 0.60069
LOF 4 0.50000 0.37500 0.37317 0.21646 0.56000 0.45000 0.70833
SimplifiedLOF 4 0.50000 0.37500 0.36323 0.20404 0.52174 0.40217 0.69097
SimplifiedLOF 5 0.50000 0.37500 0.37118 0.21397 0.51613 0.39516 0.69271
LoOP 5 0.41667 0.27083 0.32723 0.15904 0.50000 0.37500 0.66580
LoOP 6 0.33333 0.16667 0.31825 0.14782 0.52941 0.41176 0.66667
LDOF 9 0.33333 0.16667 0.31141 0.13926 0.51613 0.39516 0.67535
LDOF 13 0.41667 0.27083 0.30430 0.13038 0.43478 0.29348 0.62847
LDOF 16 0.41667 0.27083 0.32192 0.15240 0.43478 0.29348 0.56076
ODIN 5 0.33333 0.16667 0.31667 0.14583 0.44444 0.30556 0.63889
ODIN 6 0.36458 0.20573 0.32204 0.15255 0.45161 0.31452 0.62066
ODIN 7 0.31667 0.14583 0.29442 0.11802 0.46667 0.33333 0.57292
ODIN 12 0.41667 0.27083 0.30086 0.12608 0.41667 0.27083 0.54514
FastABOD 3 0.33333 0.16667 0.27858 0.09823 0.43243 0.29054 0.59375
KDEOS 20 0.33333 0.16667 0.42626 0.28282 0.48485 0.35606 0.65799
KDEOS 22 0.41667 0.27083 0.40717 0.25896 0.53333 0.41667 0.65104
KDEOS 24 0.41667 0.27083 0.39722 0.24653 0.51852 0.39815 0.66146
KDEOS 27 0.50000 0.37500 0.32218 0.15273 0.50000 0.37500 0.64583
LDF 2 0.41667 0.27083 0.46938 0.33672 0.51852 0.39815 0.67188
LDF 4 0.50000 0.37500 0.42870 0.28587 0.54545 0.43182 0.71181
LDF 9 0.33333 0.16667 0.35839 0.19799 0.48485 0.35606 0.72743
INFLO 12 0.41667 0.27083 0.35482 0.19352 0.55172 0.43966 0.69271
INFLO 17 0.50000 0.37500 0.33649 0.17062 0.53846 0.42308 0.65104
COF 4 0.33333 0.16667 0.35410 0.19263 0.57143 0.46429 0.73090
COF 5 0.50000 0.37500 0.38001 0.22501 0.56000 0.45000 0.74132
COF 11 0.50000 0.37500 0.38832 0.23540 0.55172 0.43966 0.75087
COF 12 0.50000 0.37500 0.38547 0.23183 0.56250 0.45313 0.75174

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO