Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#08)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (301.0 kB) Download raw algorithm evaluation table (24.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.58333 0.47917 0.68860 0.61075 0.74074 0.67593 0.90799
KNN 17 0.75000 0.68750 0.69540 0.61925 0.75000 0.68750 0.80382
KNN 32 0.75000 0.68750 0.70896 0.63620 0.81818 0.77273 0.76042
KNN 35 0.75000 0.68750 0.73008 0.66259 0.81818 0.77273 0.76389
KNNW 2 0.58333 0.47917 0.71312 0.64140 0.72727 0.65909 0.92014
KNNW 42 0.75000 0.68750 0.69692 0.62115 0.75000 0.68750 0.79167
KNNW 46 0.75000 0.68750 0.72149 0.65186 0.81818 0.77273 0.79514
LOF 26 0.75000 0.68750 0.70224 0.62780 0.75000 0.68750 0.82465
LOF 36 0.75000 0.68750 0.72586 0.65733 0.81818 0.77273 0.82812
LOF 47 0.75000 0.68750 0.74320 0.67900 0.81818 0.77273 0.87153
SimplifiedLOF 21 0.58333 0.47917 0.60444 0.50555 0.62069 0.52586 0.83507
SimplifiedLOF 49 0.75000 0.68750 0.71439 0.64299 0.78261 0.72826 0.79167
SimplifiedLOF 52 0.75000 0.68750 0.71642 0.64553 0.81818 0.77273 0.78299
LoOP 50 0.75000 0.68750 0.70188 0.62735 0.78261 0.72826 0.81163
LoOP 59 0.75000 0.68750 0.68879 0.61099 0.75000 0.68750 0.82639
LDOF 31 0.58333 0.47917 0.62628 0.53285 0.66667 0.58333 0.78819
LDOF 57 0.75000 0.68750 0.69263 0.61579 0.75000 0.68750 0.76736
LDOF 59 0.75000 0.68750 0.71254 0.64067 0.81818 0.77273 0.77257
ODIN 46 0.75000 0.68750 0.70962 0.63703 0.81818 0.77273 0.78993
ODIN 49 0.75000 0.68750 0.70337 0.62921 0.78261 0.72826 0.80208
FastABOD 3 0.58333 0.47917 0.64342 0.55427 0.78571 0.73214 0.92361
FastABOD 4 0.66667 0.58333 0.68601 0.60751 0.77419 0.71774 0.93576
FastABOD 5 0.75000 0.68750 0.73944 0.67430 0.78571 0.73214 0.92708
FastABOD 6 0.75000 0.68750 0.75874 0.69843 0.78571 0.73214 0.93403
KDEOS 13 0.50000 0.37500 0.40108 0.25135 0.50000 0.37500 0.70312
KDEOS 52 0.50000 0.37500 0.51882 0.39852 0.71429 0.64286 0.83333
KDEOS 59 0.50000 0.37500 0.64968 0.56210 0.66667 0.58333 0.83681
LDF 45 0.83333 0.79167 0.81300 0.76625 0.88000 0.85000 0.93750
INFLO 32 0.75000 0.68750 0.69277 0.61596 0.76190 0.70238 0.77170
INFLO 37 0.75000 0.68750 0.70188 0.62735 0.78261 0.72826 0.84549
INFLO 38 0.75000 0.68750 0.70870 0.63587 0.81818 0.77273 0.84462
COF 33 0.58333 0.47917 0.53949 0.42436 0.60606 0.50758 0.77083
COF 41 0.58333 0.47917 0.74877 0.68597 0.73684 0.67105 0.85417
COF 47 0.58333 0.47917 0.78077 0.72596 0.73684 0.67105 0.89757

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (298.3 kB) Download raw algorithm evaluation table (25.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.50000 0.37500 0.38227 0.22784 0.53659 0.42073 0.76997
KNN 7 0.50000 0.37500 0.35370 0.19213 0.53846 0.42308 0.66493
KNNW 1 0.41667 0.27083 0.42416 0.28020 0.53333 0.41667 0.78472
KNNW 4 0.50000 0.37500 0.36324 0.20405 0.51163 0.38953 0.74653
LOF 8 0.41667 0.27083 0.41288 0.26611 0.62500 0.53125 0.79688
LOF 11 0.50000 0.37500 0.37586 0.21983 0.50000 0.37500 0.73785
SimplifiedLOF 1 0.50000 0.37500 0.41791 0.27239 0.52174 0.40217 0.66667
SimplifiedLOF 7 0.41667 0.27083 0.40187 0.25233 0.62500 0.53125 0.81944
SimplifiedLOF 8 0.33333 0.16667 0.39543 0.24429 0.64706 0.55882 0.81250
LoOP 1 0.50000 0.37500 0.41963 0.27454 0.52174 0.40217 0.68663
LoOP 6 0.41667 0.27083 0.40088 0.25110 0.62857 0.53571 0.81944
LDOF 2 0.33333 0.16667 0.35642 0.19552 0.42105 0.27632 0.48785
LDOF 5 0.33333 0.16667 0.42155 0.27693 0.48485 0.35606 0.65451
LDOF 7 0.33333 0.16667 0.40952 0.26190 0.54054 0.42568 0.74306
LDOF 12 0.33333 0.16667 0.32724 0.15905 0.55556 0.44444 0.72222
ODIN 2 0.40625 0.25781 0.36378 0.20473 0.47059 0.33824 0.74479
ODIN 3 0.43750 0.29687 0.36216 0.20270 0.46154 0.32692 0.75955
ODIN 4 0.37500 0.21875 0.34206 0.17758 0.50000 0.37500 0.73785
FastABOD 3 0.41667 0.27083 0.33243 0.16554 0.46154 0.32692 0.69618
FastABOD 4 0.41667 0.27083 0.33001 0.16251 0.46154 0.32692 0.70312
FastABOD 5 0.50000 0.37500 0.32837 0.16046 0.50000 0.37500 0.68924
KDEOS 6 0.58333 0.47917 0.62441 0.53052 0.63636 0.54545 0.82465
KDEOS 7 0.58333 0.47917 0.61422 0.51777 0.60870 0.51087 0.84896
KDEOS 14 0.41667 0.27083 0.45292 0.31615 0.64865 0.56081 0.83333
LDF 6 0.50000 0.37500 0.44739 0.30924 0.60606 0.50758 0.82465
LDF 7 0.50000 0.37500 0.43243 0.29054 0.62069 0.52586 0.77604
INFLO 7 0.41667 0.27083 0.43074 0.28843 0.66667 0.58333 0.83681
INFLO 14 0.50000 0.37500 0.35000 0.18751 0.50000 0.37500 0.71007
COF 1 0.50000 0.37500 0.41791 0.27239 0.52174 0.40217 0.66667
COF 6 0.41667 0.27083 0.40852 0.26064 0.64516 0.55645 0.82118
COF 12 0.50000 0.37500 0.46100 0.32625 0.61111 0.51389 0.84028

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO