Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#07)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (300.6 kB) Download raw algorithm evaluation table (24.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.75000 0.68750 0.74690 0.68363 0.78571 0.73214 0.92448
KNNW 2 0.66667 0.58333 0.67445 0.59306 0.69565 0.61957 0.89062
KNNW 3 0.66667 0.58333 0.69702 0.62128 0.75862 0.69828 0.90451
LOF 12 0.66667 0.58333 0.58915 0.48644 0.69565 0.61957 0.86111
LOF 14 0.58333 0.47917 0.62082 0.52603 0.66667 0.58333 0.88021
LOF 34 0.66667 0.58333 0.66207 0.57759 0.76923 0.71154 0.82465
LOF 41 0.66667 0.58333 0.69411 0.61764 0.74074 0.67593 0.83854
SimplifiedLOF 40 0.66667 0.58333 0.64492 0.55615 0.66667 0.58333 0.84896
SimplifiedLOF 45 0.66667 0.58333 0.66958 0.58697 0.69565 0.61957 0.85590
SimplifiedLOF 48 0.66667 0.58333 0.67912 0.59890 0.72727 0.65909 0.85417
SimplifiedLOF 51 0.66667 0.58333 0.68870 0.61088 0.72727 0.65909 0.85069
LoOP 42 0.66667 0.58333 0.64600 0.55750 0.66667 0.58333 0.84809
LoOP 50 0.66667 0.58333 0.67812 0.59765 0.76190 0.70238 0.85330
LoOP 51 0.66667 0.58333 0.67807 0.59759 0.76190 0.70238 0.85417
LDOF 37 0.58333 0.47917 0.62643 0.53304 0.66667 0.58333 0.84375
LDOF 50 0.66667 0.58333 0.63880 0.54849 0.66667 0.58333 0.82465
LDOF 58 0.66667 0.58333 0.67716 0.59645 0.72727 0.65909 0.84201
ODIN 45 0.62500 0.53125 0.64116 0.55145 0.64000 0.55000 0.81510
ODIN 47 0.66667 0.58333 0.61563 0.51954 0.66667 0.58333 0.80382
ODIN 50 0.58333 0.47917 0.62351 0.52939 0.70000 0.62500 0.74826
FastABOD 6 0.66667 0.58333 0.59858 0.49822 0.75862 0.69828 0.86979
FastABOD 7 0.66667 0.58333 0.60814 0.51017 0.78571 0.73214 0.87674
FastABOD 8 0.58333 0.47917 0.67906 0.59883 0.78571 0.73214 0.88194
FastABOD 55 0.66667 0.58333 0.68677 0.60847 0.69231 0.61538 0.87847
KDEOS 56 0.50000 0.37500 0.63451 0.54313 0.73333 0.66667 0.85764
KDEOS 59 0.66667 0.58333 0.69507 0.61884 0.73333 0.66667 0.87847
LDF 18 0.50000 0.37500 0.65902 0.57378 0.71429 0.64286 0.90104
LDF 21 0.75000 0.68750 0.70937 0.63671 0.78261 0.72826 0.88715
LDF 25 0.75000 0.68750 0.73502 0.66877 0.81818 0.77273 0.88194
LDF 31 0.75000 0.68750 0.74595 0.68243 0.81818 0.77273 0.89583
INFLO 26 0.66667 0.58333 0.63567 0.54458 0.66667 0.58333 0.85417
INFLO 33 0.66667 0.58333 0.68515 0.60643 0.76190 0.70238 0.86806
COF 59 0.83333 0.79167 0.83463 0.79329 0.83333 0.79167 0.90451

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (298.5 kB) Download raw algorithm evaluation table (25.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 3 0.50000 0.37500 0.44483 0.30604 0.52174 0.40217 0.71788
KNN 4 0.58333 0.47917 0.40852 0.26065 0.58333 0.47917 0.70486
KNN 48 0.33333 0.16667 0.38343 0.22929 0.50000 0.37500 0.72483
KNNW 5 0.41667 0.27083 0.41316 0.26645 0.50000 0.37500 0.70312
KNNW 7 0.41667 0.27083 0.42548 0.28186 0.51852 0.39815 0.68056
KNNW 9 0.50000 0.37500 0.37718 0.22148 0.53846 0.42308 0.65451
LOF 9 0.58333 0.47917 0.43216 0.29020 0.59259 0.49074 0.66146
LOF 11 0.58333 0.47917 0.54111 0.42639 0.69231 0.61538 0.72569
LOF 14 0.58333 0.47917 0.54880 0.43601 0.64000 0.55000 0.79861
SimplifiedLOF 10 0.58333 0.47917 0.42212 0.27765 0.61538 0.51923 0.76736
SimplifiedLOF 14 0.58333 0.47917 0.53111 0.41389 0.66667 0.58333 0.78819
SimplifiedLOF 15 0.58333 0.47917 0.54070 0.42587 0.69231 0.61538 0.77431
LoOP 15 0.66667 0.58333 0.51067 0.38834 0.66667 0.58333 0.74132
LoOP 19 0.66667 0.58333 0.47924 0.34905 0.72000 0.65000 0.75521
LDOF 18 0.58333 0.47917 0.44575 0.30718 0.60870 0.51087 0.69097
LDOF 19 0.58333 0.47917 0.47892 0.34865 0.66667 0.58333 0.70139
LDOF 24 0.58333 0.47917 0.52332 0.40415 0.61538 0.51923 0.70312
LDOF 27 0.58333 0.47917 0.47170 0.33963 0.60870 0.51087 0.71528
ODIN 16 0.53333 0.41667 0.43431 0.29289 0.59259 0.49074 0.72830
ODIN 17 0.58333 0.47917 0.43398 0.29247 0.58333 0.47917 0.73698
ODIN 18 0.54167 0.42708 0.47615 0.34519 0.57143 0.46429 0.73698
FastABOD 3 0.25000 0.06250 0.31536 0.14420 0.37037 0.21296 0.55556
FastABOD 6 0.25000 0.06250 0.30599 0.13249 0.40000 0.25000 0.56597
FastABOD 25 0.25000 0.06250 0.31384 0.14230 0.37931 0.22414 0.57292
KDEOS 22 0.58333 0.47917 0.48736 0.35920 0.60000 0.50000 0.74653
KDEOS 23 0.58333 0.47917 0.49114 0.36393 0.64286 0.55357 0.75521
KDEOS 24 0.50000 0.37500 0.46763 0.33454 0.66667 0.58333 0.75000
LDF 4 0.50000 0.37500 0.49455 0.36819 0.61538 0.51923 0.81250
LDF 5 0.58333 0.47917 0.53917 0.42396 0.66667 0.58333 0.75174
LDF 8 0.58333 0.47917 0.56369 0.45461 0.64000 0.55000 0.77951
INFLO 11 0.58333 0.47917 0.49097 0.36372 0.64000 0.55000 0.75087
INFLO 17 0.58333 0.47917 0.44936 0.31170 0.66667 0.58333 0.79514
COF 12 0.58333 0.47917 0.51013 0.38767 0.66667 0.58333 0.84983
COF 13 0.58333 0.47917 0.49715 0.37144 0.69231 0.61538 0.84028
COF 15 0.58333 0.47917 0.52962 0.41202 0.69231 0.61538 0.80382

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO