Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#10)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (301.5 kB) Download raw algorithm evaluation table (23.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.58333 0.47917 0.65867 0.57333 0.64286 0.55357 0.87153
KNN 33 0.66667 0.58333 0.67195 0.58994 0.69565 0.61957 0.74306
KNN 37 0.66667 0.58333 0.70033 0.62541 0.72727 0.65909 0.75608
KNN 42 0.66667 0.58333 0.70956 0.63695 0.69565 0.61957 0.75694
KNNW 1 0.66667 0.58333 0.74902 0.68627 0.72727 0.65909 0.89757
LOF 32 0.66667 0.58333 0.65934 0.57418 0.66667 0.58333 0.74653
LOF 38 0.66667 0.58333 0.70465 0.63082 0.72727 0.65909 0.75174
LOF 54 0.66667 0.58333 0.79653 0.74566 0.72727 0.65909 0.91319
SimplifiedLOF 11 0.50000 0.37500 0.66223 0.57779 0.63158 0.53947 0.87674
SimplifiedLOF 13 0.66667 0.58333 0.67518 0.59398 0.66667 0.58333 0.85590
LoOP 10 0.50000 0.37500 0.53826 0.42282 0.63158 0.53947 0.84375
LoOP 13 0.58333 0.47917 0.60794 0.50992 0.60870 0.51087 0.82465
LoOP 51 0.50000 0.37500 0.62062 0.52578 0.60000 0.50000 0.74219
LDOF 15 0.50000 0.37500 0.58337 0.47922 0.57143 0.46429 0.78993
LDOF 30 0.50000 0.37500 0.60868 0.51084 0.60000 0.50000 0.75000
LDOF 32 0.58333 0.47917 0.61702 0.52127 0.58333 0.47917 0.74653
LDOF 57 0.50000 0.37500 0.62996 0.53746 0.60000 0.50000 0.71875
ODIN 47 0.58333 0.47917 0.63698 0.54623 0.63636 0.54545 0.74913
FastABOD 4 0.66667 0.58333 0.64525 0.55657 0.66667 0.58333 0.83681
FastABOD 7 0.58333 0.47917 0.71154 0.63943 0.66667 0.58333 0.87153
FastABOD 8 0.58333 0.47917 0.71095 0.63869 0.69231 0.61538 0.87500
FastABOD 10 0.66667 0.58333 0.70827 0.63533 0.72000 0.65000 0.85590
KDEOS 13 0.58333 0.47917 0.63589 0.54486 0.66667 0.58333 0.87674
KDEOS 15 0.50000 0.37500 0.56612 0.45765 0.70968 0.63710 0.87674
KDEOS 16 0.50000 0.37500 0.54766 0.43457 0.68571 0.60714 0.88368
LDF 4 0.75000 0.68750 0.80895 0.76118 0.75000 0.68750 0.92882
INFLO 7 0.50000 0.37500 0.42008 0.27510 0.58537 0.48171 0.80729
INFLO 34 0.66667 0.58333 0.68398 0.60498 0.66667 0.58333 0.78819
INFLO 36 0.66667 0.58333 0.69509 0.61887 0.72727 0.65909 0.79167
INFLO 45 0.66667 0.58333 0.71520 0.64400 0.69231 0.61538 0.80382
COF 7 0.66667 0.58333 0.69429 0.61787 0.66667 0.58333 0.87326
COF 8 0.66667 0.58333 0.70735 0.63419 0.66667 0.58333 0.88194
COF 14 0.66667 0.58333 0.67589 0.59486 0.72727 0.65909 0.81771
COF 48 0.66667 0.58333 0.73964 0.67455 0.72727 0.65909 0.78993

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (299.2 kB) Download raw algorithm evaluation table (25.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.41667 0.27083 0.31570 0.14462 0.43478 0.29348 0.65191
KNN 3 0.41667 0.27083 0.32466 0.15583 0.43478 0.29348 0.66233
KNN 57 0.25000 0.06250 0.30537 0.13171 0.46154 0.32692 0.66667
KNNW 1 0.41667 0.27083 0.34997 0.18746 0.46809 0.33511 0.68750
LOF 13 0.50000 0.37500 0.36415 0.20518 0.52174 0.40217 0.71528
SimplifiedLOF 13 0.50000 0.37500 0.35129 0.18911 0.51852 0.39815 0.71354
SimplifiedLOF 15 0.41667 0.27083 0.37631 0.22039 0.59259 0.49074 0.72917
SimplifiedLOF 16 0.41667 0.27083 0.38834 0.23543 0.59259 0.49074 0.73090
LoOP 10 0.41667 0.27083 0.30375 0.12968 0.46154 0.32692 0.65451
LoOP 11 0.50000 0.37500 0.31750 0.14687 0.50000 0.37500 0.63542
LoOP 16 0.41667 0.27083 0.33457 0.16821 0.53846 0.42308 0.63889
LDOF 3 0.16667 -0.04167 0.25381 0.06726 0.46154 0.32692 0.56944
LDOF 20 0.33333 0.16667 0.31239 0.14048 0.43750 0.29687 0.66667
LDOF 21 0.33333 0.16667 0.31719 0.14649 0.40000 0.25000 0.64410
ODIN 1 0.31579 0.14474 0.26990 0.08737 0.38710 0.23387 0.63542
ODIN 2 0.36111 0.20139 0.29913 0.12392 0.41176 0.26471 0.61632
FastABOD 3 0.16667 -0.04167 0.23088 0.03859 0.40000 0.25000 0.55729
FastABOD 4 0.25000 0.06250 0.23963 0.04954 0.40000 0.25000 0.56597
FastABOD 16 0.25000 0.06250 0.24603 0.05754 0.38095 0.22619 0.57292
KDEOS 4 0.50000 0.37500 0.51221 0.39027 0.56000 0.45000 0.76215
KDEOS 5 0.41667 0.27083 0.56775 0.45969 0.55556 0.44444 0.72569
KDEOS 24 0.41667 0.27083 0.34072 0.17590 0.59259 0.49074 0.69097
LDF 4 0.50000 0.37500 0.35734 0.19668 0.50000 0.37500 0.70833
LDF 7 0.41667 0.27083 0.39855 0.24819 0.50000 0.37500 0.73611
LDF 8 0.41667 0.27083 0.40235 0.25293 0.50000 0.37500 0.71007
INFLO 14 0.50000 0.37500 0.32679 0.15849 0.50000 0.37500 0.63715
INFLO 17 0.50000 0.37500 0.33561 0.16951 0.53333 0.41667 0.68490
INFLO 18 0.50000 0.37500 0.33846 0.17307 0.56000 0.45000 0.67708
COF 8 0.41667 0.27083 0.35410 0.19262 0.53659 0.42073 0.75868
COF 10 0.33333 0.16667 0.38026 0.22533 0.58065 0.47581 0.78993
COF 13 0.41667 0.27083 0.41563 0.26954 0.57895 0.47368 0.81250
COF 14 0.41667 0.27083 0.42011 0.27514 0.57143 0.46429 0.80035

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO