Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (75% of outliers)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 195 objects, 147 outliers (75.38%)

Download raw algorithm results (1.7 MB) Download raw algorithm evaluation table (48.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.78231 0.11565 0.84405 0.36645 0.88554 0.53502 0.64753
KNN 4 0.80272 0.19855 0.84770 0.38126 0.87087 0.47541 0.65235
KNN 5 0.81633 0.25383 0.83854 0.34406 0.86982 0.47115 0.63350
KNNW 5 0.80272 0.19855 0.84630 0.37560 0.88554 0.53502 0.64116
KNNW 8 0.79592 0.17092 0.84491 0.36993 0.87952 0.51054 0.64399
LOF 5 0.80272 0.19855 0.80510 0.20822 0.86136 0.43676 0.59609
LOF 6 0.79592 0.17092 0.81725 0.25757 0.86136 0.43676 0.61196
LOF 9 0.78231 0.11565 0.82770 0.30005 0.85965 0.42982 0.60190
LOF 35 0.75510 0.00510 0.76297 0.03706 0.88288 0.52421 0.46358
SimplifiedLOF 11 0.78912 0.14328 0.81646 0.25437 0.87425 0.48915 0.60346
SimplifiedLOF 13 0.79592 0.17092 0.81627 0.25360 0.87425 0.48915 0.60643
SimplifiedLOF 14 0.78231 0.11565 0.81527 0.24954 0.87500 0.49219 0.60728
LoOP 1 0.74011 -0.05581 0.75856 0.01917 0.85965 0.42982 0.46995
LoOP 8 0.78836 0.14021 0.78601 0.13068 0.85965 0.42982 0.56739
LoOP 13 0.78776 0.13776 0.80012 0.18798 0.85965 0.42982 0.58312
LoOP 16 0.78033 0.10761 0.80049 0.18950 0.85965 0.42982 0.57866
LDOF 12 0.77551 0.08801 0.76867 0.06023 0.86905 0.46801 0.51559
LDOF 14 0.78231 0.11565 0.77895 0.10200 0.86905 0.46801 0.54053
LDOF 16 0.76871 0.06037 0.78536 0.12802 0.86647 0.45753 0.55315
ODIN 2 0.75737 0.01431 0.76558 0.04766 0.86905 0.46801 0.52388
ODIN 3 0.77041 0.06728 0.76367 0.03990 0.85965 0.42982 0.52608
ODIN 5 0.74694 -0.02806 0.76712 0.05391 0.86391 0.44712 0.51651
FastABOD 6 0.78912 0.14328 0.85654 0.41720 0.88822 0.54588 0.65505
FastABOD 12 0.78912 0.14328 0.87138 0.47748 0.88822 0.54588 0.66908
FastABOD 15 0.78912 0.14328 0.87031 0.47312 0.88822 0.54588 0.66993
FastABOD 94 0.79592 0.17092 0.85798 0.42303 0.88288 0.52421 0.64881
KDEOS 5 0.76871 0.06037 0.77293 0.07753 0.87240 0.48164 0.52622
KDEOS 10 0.76190 0.03274 0.82518 0.28979 0.86826 0.46482 0.57809
KDEOS 28 0.78231 0.11565 0.79111 0.15140 0.86826 0.46482 0.58673
KDEOS 29 0.78912 0.14328 0.78859 0.14116 0.86826 0.46482 0.58574
LDF 6 0.78231 0.11565 0.84285 0.36156 0.86217 0.44007 0.60218
LDF 24 0.75510 0.00510 0.70562 -0.19593 0.87952 0.51054 0.40179
LDF 94 0.79592 0.17092 0.75458 0.00299 0.86217 0.44007 0.49008
INFLO 5 0.75850 0.01892 0.77137 0.07119 0.86647 0.45753 0.56590
INFLO 6 0.77017 0.06630 0.77972 0.10511 0.86136 0.43676 0.59198
INFLO 8 0.77467 0.08460 0.78094 0.11007 0.85965 0.42982 0.58248
INFLO 10 0.76871 0.06037 0.79462 0.16563 0.85965 0.42982 0.58390
COF 7 0.80952 0.22619 0.81052 0.23024 0.87164 0.47854 0.60920
COF 11 0.77551 0.08801 0.81908 0.26500 0.88146 0.51843 0.58688
COF 98 0.79592 0.17092 0.86046 0.43312 0.85965 0.42982 0.64966
COF 100 0.80272 0.19855 0.86193 0.43910 0.85965 0.42982 0.64753

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 195 objects, 147 outliers (75.38%)

Download raw algorithm results (1.7 MB) Download raw algorithm evaluation table (46.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.75510 0.00510 0.74400 -0.04002 0.86726 0.46073 0.47725
KNN 3 0.76190 0.03274 0.73216 -0.08808 0.85965 0.42982 0.46606
KNNW 2 0.76190 0.03274 0.74016 -0.05559 0.86471 0.45037 0.47704
LOF 2 0.77551 0.08801 0.76119 0.02984 0.86217 0.44007 0.51878
LOF 45 0.73469 -0.07781 0.76342 0.03890 0.85965 0.42982 0.46825
LOF 55 0.74150 -0.05017 0.74062 -0.05375 0.86726 0.46073 0.46542
SimplifiedLOF 1 0.75988 0.02451 0.75860 0.01932 0.85965 0.42982 0.49568
SimplifiedLOF 9 0.76190 0.03274 0.73960 -0.05789 0.85965 0.42982 0.49674
SimplifiedLOF 11 0.77551 0.08801 0.74348 -0.04210 0.86310 0.44382 0.49192
SimplifiedLOF 14 0.78231 0.11565 0.74015 -0.05565 0.85965 0.42982 0.48356
LoOP 1 0.76099 0.02903 0.75876 0.01998 0.85965 0.42982 0.49787
LoOP 10 0.76138 0.03061 0.73052 -0.09476 0.85965 0.42982 0.47329
LDOF 12 0.76871 0.06037 0.76668 0.05215 0.88024 0.51347 0.54110
LDOF 13 0.77551 0.08801 0.76295 0.03700 0.86982 0.47115 0.52310
ODIN 1 0.75964 0.02353 0.76530 0.04653 0.85965 0.42982 0.52402
ODIN 8 0.73824 -0.06339 0.72728 -0.10794 0.86217 0.44007 0.43275
FastABOD 3 0.74150 -0.05017 0.71746 -0.14780 0.85965 0.42982 0.43056
KDEOS 2 0.76463 0.04379 0.73848 -0.06244 0.85965 0.42982 0.48172
KDEOS 3 0.75510 0.00510 0.72942 -0.09922 0.86217 0.44007 0.46854
KDEOS 6 0.74830 -0.02253 0.74776 -0.02472 0.85965 0.42982 0.46046
KDEOS 15 0.76871 0.06037 0.74356 -0.04178 0.85965 0.42982 0.48143
LDF 30 0.78231 0.11565 0.82722 0.29809 0.85965 0.42982 0.62061
LDF 34 0.75510 0.00510 0.83714 0.33839 0.85965 0.42982 0.64328
LDF 50 0.75510 0.00510 0.70809 -0.18589 0.86471 0.45037 0.43495
INFLO 9 0.76947 0.06347 0.77205 0.07396 0.86217 0.44007 0.55116
INFLO 12 0.75948 0.02287 0.78352 0.12054 0.86217 0.44007 0.53323
INFLO 46 0.75566 0.00736 0.73346 -0.08281 0.86982 0.47115 0.46443
COF 36 0.84354 0.36437 0.82530 0.29027 0.89720 0.58236 0.68870
COF 39 0.83673 0.33673 0.83593 0.33345 0.90282 0.60521 0.70132
COF 45 0.82993 0.30910 0.85569 0.41373 0.89441 0.57104 0.73377
COF 46 0.82993 0.30910 0.85467 0.40961 0.89164 0.55979 0.73647

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO