Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#03)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (233.6 kB) Download raw algorithm evaluation table (14.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40000 0.33750 0.67980 0.64644 0.62500 0.58594 0.94167
KNNW 1 0.60000 0.55833 0.73026 0.70216 0.72727 0.69886 0.95208
LOF 3 0.60000 0.55833 0.56759 0.52254 0.60000 0.55833 0.70000
LOF 4 0.60000 0.55833 0.64933 0.61280 0.75000 0.72396 0.72500
LOF 49 0.60000 0.55833 0.73571 0.70818 0.66667 0.63194 0.94583
SimplifiedLOF 5 0.60000 0.55833 0.63030 0.59179 0.60000 0.55833 0.85000
SimplifiedLOF 8 0.60000 0.55833 0.71030 0.68013 0.75000 0.72396 0.85833
SimplifiedLOF 19 0.40000 0.33750 0.57709 0.53304 0.57143 0.52679 0.87917
LoOP 7 0.60000 0.55833 0.61048 0.56990 0.60000 0.55833 0.84583
LoOP 8 0.60000 0.55833 0.69697 0.66540 0.75000 0.72396 0.85000
LoOP 9 0.60000 0.55833 0.70857 0.67821 0.75000 0.72396 0.85000
LoOP 11 0.40000 0.33750 0.62115 0.58168 0.57143 0.52679 0.85833
LDOF 8 0.60000 0.55833 0.65140 0.61508 0.75000 0.72396 0.74167
LDOF 15 0.40000 0.33750 0.54564 0.49831 0.57143 0.52679 0.78750
ODIN 3 0.32000 0.24917 0.27407 0.19846 0.35294 0.28554 0.82708
ODIN 5 0.60000 0.55833 0.43461 0.37572 0.60000 0.55833 0.77917
ODIN 9 0.60000 0.55833 0.56703 0.52193 0.66667 0.63194 0.81250
FastABOD 4 0.60000 0.55833 0.64667 0.60986 0.66667 0.63194 0.93750
KDEOS 14 0.40000 0.33750 0.32799 0.25798 0.50000 0.44792 0.83333
KDEOS 25 0.40000 0.33750 0.50606 0.45461 0.50000 0.44792 0.87917
KDEOS 45 0.40000 0.33750 0.58389 0.54054 0.57143 0.52679 0.87917
LDF 2 0.60000 0.55833 0.66032 0.62493 0.75000 0.72396 0.76250
LDF 49 0.60000 0.55833 0.71250 0.68255 0.66667 0.63194 0.93333
INFLO 11 0.40000 0.33750 0.55062 0.50381 0.57143 0.52679 0.84583
INFLO 50 0.61600 0.57600 0.63774 0.60000 0.75000 0.72396 0.80000
COF 6 0.60000 0.55833 0.76667 0.74236 0.75000 0.72396 0.94167
COF 8 0.60000 0.55833 0.67758 0.64399 0.72727 0.69886 0.95417
COF 22 0.80000 0.77917 0.73564 0.70810 0.80000 0.77917 0.85000

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.0 kB) Download raw algorithm evaluation table (17.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.20000 0.11667 0.29405 0.22051 0.46154 0.40545 0.78958
KNN 2 0.20000 0.11667 0.28849 0.21438 0.50000 0.44792 0.75625
KNNW 1 0.60000 0.55833 0.44273 0.38469 0.60000 0.55833 0.83750
LOF 2 0.40000 0.33750 0.34222 0.27370 0.40000 0.33750 0.55417
LOF 4 0.40000 0.33750 0.40234 0.34008 0.50000 0.44792 0.60000
LOF 12 0.40000 0.33750 0.31333 0.24181 0.47059 0.41544 0.84167
LOF 14 0.20000 0.11667 0.32119 0.25048 0.50000 0.44792 0.80833
SimplifiedLOF 3 0.40000 0.33750 0.38593 0.32196 0.44444 0.38657 0.67500
SimplifiedLOF 4 0.40000 0.33750 0.43270 0.37361 0.44444 0.38657 0.77500
SimplifiedLOF 5 0.20000 0.11667 0.40238 0.34013 0.50000 0.44792 0.70417
LoOP 3 0.40000 0.33750 0.37702 0.31213 0.44444 0.38657 0.63958
LoOP 4 0.40000 0.33750 0.41223 0.35101 0.50000 0.44792 0.66667
LoOP 20 0.20000 0.11667 0.24839 0.17010 0.40000 0.33750 0.73542
LDOF 3 0.20000 0.11667 0.32222 0.25162 0.33333 0.26389 0.63333
LDOF 5 0.40000 0.33750 0.35780 0.29091 0.44444 0.38657 0.52500
LDOF 7 0.40000 0.33750 0.40409 0.34201 0.50000 0.44792 0.61667
ODIN 1 0.23077 0.15064 0.18724 0.10258 0.33333 0.26389 0.74583
ODIN 15 0.40000 0.33750 0.21491 0.13313 0.40000 0.33750 0.67292
FastABOD 3 0.40000 0.33750 0.43462 0.37573 0.46154 0.40545 0.73750
FastABOD 4 0.40000 0.33750 0.54555 0.49821 0.57143 0.52679 0.74167
FastABOD 5 0.40000 0.33750 0.48127 0.42724 0.50000 0.44792 0.74583
KDEOS 4 0.20000 0.11667 0.37941 0.31476 0.36364 0.29735 0.74583
KDEOS 26 0.60000 0.55833 0.35051 0.28286 0.60000 0.55833 0.74167
KDEOS 29 0.40000 0.33750 0.44334 0.38535 0.46154 0.40545 0.74167
LDF 2 0.40000 0.33750 0.47923 0.42498 0.54545 0.49811 0.67917
LDF 4 0.60000 0.55833 0.34106 0.27242 0.60000 0.55833 0.70417
LDF 7 0.40000 0.33750 0.29206 0.21831 0.46154 0.40545 0.81667
INFLO 4 0.40000 0.33750 0.38168 0.31728 0.44444 0.38657 0.67500
INFLO 15 0.40000 0.33750 0.33833 0.26941 0.50000 0.44792 0.87917
INFLO 17 0.20000 0.11667 0.29842 0.22534 0.53333 0.48472 0.81458
INFLO 49 0.41224 0.35102 0.22327 0.14236 0.44444 0.38657 0.67083
COF 3 0.40000 0.33750 0.40712 0.34537 0.40000 0.33750 0.78333
COF 4 0.40000 0.33750 0.50353 0.45181 0.50000 0.44792 0.82917
COF 5 0.40000 0.33750 0.55156 0.50484 0.57143 0.52679 0.82083

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO