Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#03)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (300.3 kB) Download raw algorithm evaluation table (23.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.58333 0.47917 0.70327 0.62909 0.66667 0.58333 0.89062
KNNW 1 0.66667 0.58333 0.69465 0.61831 0.66667 0.58333 0.87847
KNNW 2 0.66667 0.58333 0.69151 0.61438 0.69565 0.61957 0.88194
KNNW 4 0.50000 0.37500 0.68129 0.60161 0.64516 0.55645 0.88368
LOF 3 0.83333 0.79167 0.76045 0.70057 0.83333 0.79167 0.86111
LOF 4 0.75000 0.68750 0.82692 0.78365 0.80000 0.75000 0.95312
SimplifiedLOF 5 0.75000 0.68750 0.65860 0.57325 0.75000 0.68750 0.83507
SimplifiedLOF 6 0.75000 0.68750 0.72195 0.65243 0.78261 0.72826 0.86111
SimplifiedLOF 7 0.75000 0.68750 0.72161 0.65201 0.78261 0.72826 0.89236
LoOP 5 0.75000 0.68750 0.64173 0.55216 0.78261 0.72826 0.81250
LoOP 6 0.75000 0.68750 0.68138 0.60173 0.78261 0.72826 0.84462
LoOP 8 0.66667 0.58333 0.65906 0.57383 0.66667 0.58333 0.86111
LDOF 11 0.58333 0.47917 0.55379 0.44223 0.58333 0.47917 0.78125
LDOF 13 0.50000 0.37500 0.61023 0.51279 0.54545 0.43182 0.79514
LDOF 14 0.41667 0.27083 0.58834 0.48542 0.60606 0.50758 0.80556
ODIN 5 0.53030 0.41288 0.51198 0.38997 0.62500 0.53125 0.82899
FastABOD 5 0.58333 0.47917 0.62189 0.52736 0.60870 0.51087 0.79167
FastABOD 7 0.58333 0.47917 0.65861 0.57327 0.66667 0.58333 0.81597
FastABOD 12 0.58333 0.47917 0.66797 0.58496 0.66667 0.58333 0.83333
FastABOD 59 0.50000 0.37500 0.65351 0.56689 0.60000 0.50000 0.83854
KDEOS 15 0.66667 0.58333 0.67047 0.58809 0.72000 0.65000 0.89583
KDEOS 16 0.66667 0.58333 0.66485 0.58106 0.69231 0.61538 0.90799
LDF 2 0.66667 0.58333 0.72988 0.66235 0.69565 0.61957 0.84462
LDF 4 0.50000 0.37500 0.66855 0.58569 0.70968 0.63710 0.89062
LDF 49 0.66667 0.58333 0.64682 0.55852 0.76923 0.71154 0.80035
INFLO 3 0.66667 0.58333 0.60622 0.50778 0.66667 0.58333 0.76736
INFLO 4 0.50000 0.37500 0.61185 0.51481 0.57143 0.46429 0.79167
INFLO 5 0.58333 0.47917 0.53511 0.41889 0.61538 0.51923 0.80729
COF 6 0.75000 0.68750 0.88286 0.85357 0.81818 0.77273 0.95312
COF 8 0.75000 0.68750 0.87352 0.84190 0.85714 0.82143 0.96701

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (299.6 kB) Download raw algorithm evaluation table (24.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.50000 0.37500 0.43119 0.28899 0.60606 0.50758 0.82292
KNN 3 0.50000 0.37500 0.44033 0.30041 0.59259 0.49074 0.80729
KNN 5 0.58333 0.47917 0.42608 0.28260 0.58333 0.47917 0.77170
KNNW 1 0.41667 0.27083 0.42547 0.28184 0.62500 0.53125 0.81597
KNNW 2 0.50000 0.37500 0.40145 0.25182 0.60606 0.50758 0.81076
KNNW 8 0.50000 0.37500 0.43743 0.29679 0.61538 0.51923 0.80208
LOF 4 0.50000 0.37500 0.52972 0.41215 0.62500 0.53125 0.81944
LOF 5 0.58333 0.47917 0.39397 0.24246 0.58333 0.47917 0.74566
SimplifiedLOF 5 0.50000 0.37500 0.54750 0.43438 0.63158 0.53947 0.84549
SimplifiedLOF 6 0.50000 0.37500 0.51856 0.39820 0.64706 0.55882 0.85243
SimplifiedLOF 7 0.58333 0.47917 0.43722 0.29652 0.62857 0.53571 0.83681
LoOP 2 0.41667 0.27083 0.37776 0.22220 0.44444 0.30556 0.62674
LoOP 6 0.41667 0.27083 0.44541 0.30676 0.58065 0.47581 0.79688
LoOP 11 0.41667 0.27083 0.37714 0.22142 0.62500 0.53125 0.78125
LDOF 6 0.50000 0.37500 0.42615 0.28269 0.50000 0.37500 0.67188
LDOF 11 0.33333 0.16667 0.35033 0.18791 0.58824 0.48529 0.76215
ODIN 7 0.41667 0.27083 0.37485 0.21856 0.56250 0.45313 0.75174
ODIN 9 0.43333 0.29167 0.38236 0.22795 0.51852 0.39815 0.76476
FastABOD 3 0.41667 0.27083 0.45661 0.32076 0.54545 0.43182 0.79861
FastABOD 5 0.41667 0.27083 0.45021 0.31276 0.56250 0.45313 0.78299
FastABOD 39 0.41667 0.27083 0.49548 0.36935 0.51852 0.39815 0.76736
KDEOS 7 0.33333 0.16667 0.49731 0.37164 0.52174 0.40217 0.78472
KDEOS 8 0.50000 0.37500 0.46925 0.33656 0.55556 0.44444 0.80035
KDEOS 12 0.41667 0.27083 0.43103 0.28879 0.64706 0.55882 0.83160
KDEOS 23 0.41667 0.27083 0.43874 0.29843 0.66667 0.58333 0.81076
LDF 2 0.58333 0.47917 0.58568 0.48209 0.60870 0.51087 0.75868
LDF 3 0.58333 0.47917 0.67773 0.59716 0.63636 0.54545 0.83854
LDF 10 0.58333 0.47917 0.49357 0.36696 0.69231 0.61538 0.84028
INFLO 4 0.58333 0.47917 0.52899 0.41123 0.60870 0.51087 0.77083
INFLO 5 0.33333 0.16667 0.41063 0.26328 0.58537 0.48171 0.80295
INFLO 7 0.41667 0.27083 0.38408 0.23010 0.64706 0.55882 0.77083
COF 2 0.58333 0.47917 0.43459 0.29324 0.63636 0.54545 0.60243
COF 5 0.50000 0.37500 0.54470 0.43088 0.60000 0.50000 0.84549
COF 6 0.50000 0.37500 0.50912 0.38641 0.60606 0.50758 0.85590
COF 15 0.50000 0.37500 0.41777 0.27222 0.66667 0.58333 0.78646

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO