Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#10)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (231.8 kB) Download raw algorithm evaluation table (16.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40000 0.33750 0.48835 0.43505 0.50000 0.44792 0.86667
KNN 4 0.60000 0.55833 0.49436 0.44169 0.60000 0.55833 0.81250
KNN 5 0.40000 0.33750 0.49892 0.44673 0.54545 0.49811 0.83750
KNNW 1 0.60000 0.55833 0.60939 0.56871 0.60000 0.55833 0.92917
LOF 1 0.40000 0.33750 0.61621 0.57623 0.57143 0.52679 0.88333
LOF 2 0.60000 0.55833 0.53273 0.48405 0.60000 0.55833 0.87292
SimplifiedLOF 1 0.40000 0.33750 0.60794 0.56710 0.66667 0.63194 0.93750
LoOP 1 0.40000 0.33750 0.60794 0.56710 0.66667 0.63194 0.93750
LDOF 4 0.40000 0.33750 0.46404 0.40821 0.47619 0.42163 0.87500
LDOF 11 0.40000 0.33750 0.54882 0.50182 0.57143 0.52679 0.85000
LDOF 15 0.20000 0.11667 0.46810 0.41269 0.53333 0.48472 0.88750
LDOF 32 0.40000 0.33750 0.51352 0.46284 0.66667 0.63194 0.76667
ODIN 1 0.29412 0.22059 0.29412 0.22059 0.45455 0.39773 0.87500
ODIN 3 0.60000 0.55833 0.43897 0.38053 0.60000 0.55833 0.83542
FastABOD 4 0.60000 0.55833 0.51251 0.46173 0.60000 0.55833 0.85833
FastABOD 9 0.40000 0.33750 0.51930 0.46923 0.54545 0.49811 0.88750
FastABOD 16 0.60000 0.55833 0.52303 0.47335 0.60000 0.55833 0.83750
KDEOS 9 0.60000 0.55833 0.58389 0.54054 0.60000 0.55833 0.88750
KDEOS 12 0.40000 0.33750 0.65714 0.62143 0.61538 0.57532 0.92917
KDEOS 13 0.60000 0.55833 0.67139 0.63716 0.60000 0.55833 0.92500
KDEOS 15 0.20000 0.11667 0.55238 0.50575 0.66667 0.63194 0.92917
LDF 2 0.60000 0.55833 0.68552 0.65276 0.75000 0.72396 0.86042
LDF 7 0.60000 0.55833 0.75507 0.72956 0.75000 0.72396 0.82083
LDF 25 0.60000 0.55833 0.68000 0.64667 0.66667 0.63194 0.90833
INFLO 1 0.60000 0.55833 0.77311 0.74947 0.75000 0.72396 0.93750
INFLO 49 0.60816 0.56735 0.52107 0.47118 0.66667 0.63194 0.78750
COF 1 0.40000 0.33750 0.60794 0.56710 0.66667 0.63194 0.93750
COF 2 0.60000 0.55833 0.56667 0.52153 0.60000 0.55833 0.91667

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (233.4 kB) Download raw algorithm evaluation table (19.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 3 0.00000 -0.10417 0.16383 0.07673 0.30769 0.23558 0.63542
KNN 4 0.20000 0.11667 0.15457 0.06651 0.25000 0.17188 0.61458
KNN 6 0.20000 0.11667 0.15938 0.07182 0.33333 0.26389 0.53333
KNNW 1 0.20000 0.11667 0.18058 0.09522 0.29412 0.22059 0.66667
KNNW 12 0.20000 0.11667 0.16423 0.07718 0.36364 0.29735 0.50000
LOF 1 0.20000 0.11667 0.21632 0.13469 0.28571 0.21131 0.58958
LOF 2 0.20000 0.11667 0.30581 0.23350 0.33333 0.26389 0.58333
LOF 16 0.00000 -0.10417 0.14965 0.06107 0.27586 0.20043 0.64167
SimplifiedLOF 1 0.20000 0.11667 0.23310 0.15322 0.33333 0.26389 0.62083
SimplifiedLOF 4 0.20000 0.11667 0.32638 0.25621 0.33333 0.26389 0.69583
SimplifiedLOF 6 0.20000 0.11667 0.20570 0.12296 0.41667 0.35590 0.76667
LoOP 1 0.20000 0.11667 0.23238 0.15242 0.33333 0.26389 0.61667
LoOP 5 0.20000 0.11667 0.36137 0.29485 0.36364 0.29735 0.77500
LoOP 6 0.20000 0.11667 0.21327 0.13132 0.41667 0.35590 0.77917
LDOF 2 0.20000 0.11667 0.16217 0.07489 0.25000 0.17188 0.55417
LDOF 5 0.20000 0.11667 0.38192 0.31754 0.43478 0.37591 0.80833
LDOF 6 0.20000 0.11667 0.39150 0.32812 0.37500 0.30990 0.81250
ODIN 2 0.33333 0.26389 0.20793 0.12542 0.36364 0.29735 0.68958
FastABOD 3 0.20000 0.11667 0.14774 0.05897 0.23529 0.15564 0.59167
FastABOD 6 0.20000 0.11667 0.15545 0.06748 0.25000 0.17188 0.56667
FastABOD 8 0.00000 -0.10417 0.13413 0.04393 0.26667 0.19028 0.55417
KDEOS 7 0.40000 0.33750 0.42049 0.36012 0.50000 0.44792 0.87500
LDF 1 0.20000 0.11667 0.18286 0.09774 0.28571 0.21131 0.49375
LDF 2 0.20000 0.11667 0.32104 0.25032 0.33333 0.26389 0.65833
LDF 20 0.00000 -0.10417 0.24157 0.16256 0.47059 0.41544 0.80833
INFLO 3 0.20000 0.11667 0.35011 0.28241 0.34783 0.27989 0.75000
INFLO 6 0.00000 -0.10417 0.20185 0.11871 0.41667 0.35590 0.77083
INFLO 52 0.26154 0.18462 0.27547 0.20000 0.33333 0.26389 0.60000
COF 1 0.20000 0.11667 0.23310 0.15322 0.33333 0.26389 0.62083
COF 5 0.20000 0.11667 0.35515 0.28798 0.33333 0.26389 0.74375
COF 7 0.00000 -0.10417 0.17484 0.08889 0.35714 0.29018 0.71875

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO