Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#04)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (233.9 kB) Download raw algorithm evaluation table (16.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40000 0.33750 0.40773 0.34603 0.50000 0.44792 0.85417
KNNW 1 0.40000 0.33750 0.43571 0.37693 0.61538 0.57532 0.89375
LOF 3 0.40000 0.33750 0.53496 0.48652 0.57143 0.52679 0.82500
LOF 4 0.40000 0.33750 0.56642 0.52126 0.57143 0.52679 0.81250
SimplifiedLOF 3 0.60000 0.55833 0.52714 0.47789 0.60000 0.55833 0.87917
SimplifiedLOF 5 0.60000 0.55833 0.70275 0.67178 0.66667 0.63194 0.89583
LoOP 5 0.60000 0.55833 0.67000 0.63563 0.66667 0.63194 0.88333
LDOF 6 0.60000 0.55833 0.48268 0.42879 0.60000 0.55833 0.77083
LDOF 7 0.60000 0.55833 0.46612 0.41050 0.66667 0.63194 0.80417
ODIN 2 0.30000 0.22708 0.24275 0.16386 0.40000 0.33750 0.79792
ODIN 3 0.40000 0.33750 0.25779 0.18048 0.40000 0.33750 0.76042
ODIN 14 0.40000 0.33750 0.47500 0.42031 0.57143 0.52679 0.66458
ODIN 15 0.40000 0.33750 0.48493 0.43128 0.57143 0.52679 0.71458
FastABOD 3 0.40000 0.33750 0.48745 0.43405 0.50000 0.44792 0.79167
FastABOD 6 0.60000 0.55833 0.42714 0.36747 0.60000 0.55833 0.87500
KDEOS 9 0.40000 0.33750 0.37286 0.30753 0.66667 0.63194 0.82500
KDEOS 12 0.40000 0.33750 0.42511 0.36522 0.50000 0.44792 0.84167
KDEOS 23 0.60000 0.55833 0.50275 0.45095 0.60000 0.55833 0.81667
KDEOS 26 0.60000 0.55833 0.57428 0.52994 0.66667 0.63194 0.83333
LDF 4 0.40000 0.33750 0.53523 0.48681 0.50000 0.44792 0.89583
INFLO 2 0.40000 0.33750 0.39333 0.33014 0.44444 0.38657 0.67083
INFLO 3 0.40000 0.33750 0.55513 0.50879 0.57143 0.52679 0.83750
COF 5 0.60000 0.55833 0.84444 0.82824 0.75000 0.72396 0.97500

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.2 kB) Download raw algorithm evaluation table (18.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 3 0.20000 0.11667 0.20095 0.11772 0.33333 0.26389 0.72500
KNN 4 0.20000 0.11667 0.19869 0.11523 0.31250 0.24089 0.73333
KNN 5 0.20000 0.11667 0.19789 0.11434 0.36364 0.29735 0.68333
KNNW 1 0.40000 0.33750 0.28463 0.21012 0.44444 0.38657 0.68542
KNNW 6 0.00000 -0.10417 0.18987 0.10548 0.33333 0.26389 0.70833
LOF 2 0.40000 0.33750 0.20895 0.12655 0.40000 0.33750 0.54167
LOF 47 0.40000 0.33750 0.27725 0.20197 0.44444 0.38657 0.80417
LOF 48 0.40000 0.33750 0.30927 0.23732 0.50000 0.44792 0.75000
SimplifiedLOF 3 0.20000 0.11667 0.22876 0.14842 0.36364 0.29735 0.53750
SimplifiedLOF 5 0.40000 0.33750 0.22057 0.13938 0.40000 0.33750 0.59583
SimplifiedLOF 25 0.00000 -0.10417 0.15228 0.06398 0.30303 0.23043 0.67083
LoOP 3 0.40000 0.33750 0.19274 0.10865 0.40000 0.33750 0.55417
LoOP 5 0.20000 0.11667 0.20440 0.12153 0.36364 0.29735 0.58333
LoOP 23 0.00000 -0.10417 0.14378 0.05459 0.32000 0.24917 0.63750
LDOF 4 0.20000 0.11667 0.11727 0.02532 0.20000 0.11667 0.43333
LDOF 7 0.20000 0.11667 0.20181 0.11867 0.28571 0.21131 0.56667
LDOF 8 0.20000 0.11667 0.16190 0.07460 0.25000 0.17188 0.59167
ODIN 1 0.15385 0.06571 0.13471 0.04457 0.22222 0.14120 0.66042
ODIN 6 0.26667 0.19028 0.18429 0.09932 0.33333 0.26389 0.55833
FastABOD 4 0.20000 0.11667 0.15922 0.07164 0.26667 0.19028 0.60833
FastABOD 17 0.20000 0.11667 0.18094 0.09562 0.33333 0.26389 0.63333
FastABOD 30 0.20000 0.11667 0.18224 0.09705 0.33333 0.26389 0.63750
KDEOS 21 0.40000 0.33750 0.27292 0.19718 0.44444 0.38657 0.62917
KDEOS 23 0.40000 0.33750 0.27665 0.20130 0.44444 0.38657 0.65000
KDEOS 34 0.20000 0.11667 0.19783 0.11427 0.33333 0.26389 0.74167
LDF 1 0.40000 0.33750 0.28492 0.21044 0.40000 0.33750 0.66667
LDF 46 0.40000 0.33750 0.36701 0.30108 0.54545 0.49811 0.87083
LDF 48 0.40000 0.33750 0.37004 0.30442 0.57143 0.52679 0.84583
INFLO 16 0.20000 0.11667 0.25268 0.17483 0.47059 0.41544 0.78750
INFLO 17 0.20000 0.11667 0.24678 0.16832 0.50000 0.44792 0.78958
INFLO 48 0.21250 0.13047 0.11502 0.02284 0.22222 0.14120 0.46875
COF 24 0.20000 0.11667 0.28881 0.21473 0.47059 0.41544 0.82917
COF 28 0.40000 0.33750 0.27160 0.19573 0.40000 0.33750 0.80417

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO