Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#05)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.8 kB) Download raw algorithm evaluation table (13.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.60000 0.55833 0.64365 0.60653 0.66667 0.63194 0.93333
KNN 24 0.60000 0.55833 0.58458 0.54131 0.72727 0.69886 0.86250
KNNW 1 0.60000 0.55833 0.65810 0.62248 0.72727 0.69886 0.94375
LOF 3 0.80000 0.77917 0.88500 0.87302 0.80000 0.77917 0.98333
LOF 4 0.60000 0.55833 0.87619 0.86329 0.83333 0.81597 0.98333
SimplifiedLOF 7 0.80000 0.77917 0.96667 0.96319 0.90909 0.89962 0.99583
LoOP 7 0.80000 0.77917 0.78619 0.76392 0.83333 0.81597 0.97917
LoOP 10 0.60000 0.55833 0.79444 0.77303 0.72727 0.69886 0.97083
LDOF 13 0.60000 0.55833 0.77833 0.75524 0.76923 0.74519 0.97083
LDOF 17 0.80000 0.77917 0.80091 0.78017 0.80000 0.77917 0.96667
ODIN 6 0.50000 0.44792 0.45275 0.39574 0.66667 0.63194 0.89792
ODIN 7 0.50000 0.44792 0.45776 0.40128 0.66667 0.63194 0.91458
ODIN 9 0.60000 0.55833 0.42756 0.36793 0.60000 0.55833 0.87083
ODIN 46 0.60000 0.55833 0.55315 0.50661 0.66667 0.63194 0.83958
FastABOD 4 0.60000 0.55833 0.74167 0.71476 0.76923 0.74519 0.97083
FastABOD 5 0.60000 0.55833 0.75952 0.73447 0.83333 0.81597 0.97500
KDEOS 11 0.60000 0.55833 0.53476 0.48630 0.66667 0.63194 0.92500
KDEOS 14 0.40000 0.33750 0.51313 0.46242 0.62500 0.58594 0.93333
KDEOS 16 0.40000 0.33750 0.43056 0.37124 0.66667 0.63194 0.92083
KDEOS 47 0.60000 0.55833 0.58889 0.54606 0.60000 0.55833 0.90833
LDF 3 0.60000 0.55833 0.47556 0.42093 0.66667 0.63194 0.93333
LDF 4 0.60000 0.55833 0.73333 0.70556 0.75000 0.72396 0.92500
INFLO 3 0.60000 0.55833 0.58032 0.53660 0.60000 0.55833 0.92500
INFLO 8 0.60000 0.55833 0.60758 0.56670 0.72727 0.69886 0.95417
INFLO 28 0.60000 0.55833 0.64424 0.60718 0.72727 0.69886 0.95000
COF 4 0.60000 0.55833 0.75952 0.73447 0.83333 0.81597 0.97500
COF 8 0.80000 0.77917 0.61000 0.56938 0.80000 0.77917 0.94167

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.5 kB) Download raw algorithm evaluation table (17.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.20000 0.11667 0.32143 0.25074 0.52632 0.47697 0.87292
KNN 2 0.20000 0.11667 0.45203 0.39494 0.50000 0.44792 0.86667
KNN 23 0.40000 0.33750 0.23425 0.15449 0.40000 0.33750 0.47083
KNNW 1 0.20000 0.11667 0.40794 0.34626 0.57143 0.52679 0.90000
KNNW 3 0.40000 0.33750 0.39333 0.33014 0.53333 0.48472 0.89167
KNNW 4 0.20000 0.11667 0.47583 0.42123 0.53333 0.48472 0.88750
LOF 1 0.60000 0.55833 0.38553 0.32153 0.60000 0.55833 0.75208
LOF 3 0.40000 0.33750 0.43248 0.37336 0.57143 0.52679 0.91667
LOF 11 0.20000 0.11667 0.48353 0.42973 0.57143 0.52679 0.89167
SimplifiedLOF 1 0.60000 0.55833 0.47429 0.41952 0.66667 0.63194 0.88333
SimplifiedLOF 4 0.40000 0.33750 0.41571 0.35485 0.58824 0.54534 0.91250
SimplifiedLOF 13 0.20000 0.11667 0.51571 0.46527 0.58824 0.54534 0.91250
LoOP 1 0.60000 0.55833 0.47429 0.41952 0.66667 0.63194 0.88333
LoOP 12 0.40000 0.33750 0.44333 0.38535 0.58824 0.54534 0.91667
LoOP 13 0.20000 0.11667 0.49488 0.44226 0.53333 0.48472 0.89583
LDOF 11 0.40000 0.33750 0.41117 0.34983 0.61538 0.57532 0.87917
LDOF 17 0.40000 0.33750 0.41830 0.35771 0.57143 0.52679 0.83333
ODIN 6 0.47500 0.42031 0.45000 0.39271 0.58824 0.54534 0.93125
FastABOD 3 0.40000 0.33750 0.33985 0.27108 0.50000 0.44792 0.85833
KDEOS 16 0.60000 0.55833 0.51091 0.45996 0.72727 0.69886 0.94167
KDEOS 17 0.60000 0.55833 0.55476 0.50838 0.66667 0.63194 0.93333
LDF 1 0.40000 0.33750 0.27125 0.19534 0.44444 0.38657 0.60208
LDF 2 0.40000 0.33750 0.52286 0.47315 0.66667 0.63194 0.83750
LDF 3 0.40000 0.33750 0.58056 0.53686 0.58824 0.54534 0.92500
LDF 4 0.40000 0.33750 0.56551 0.52025 0.62500 0.58594 0.92917
INFLO 4 0.40000 0.33750 0.43939 0.38100 0.54545 0.49811 0.90417
INFLO 10 0.40000 0.33750 0.51117 0.46025 0.53333 0.48472 0.87917
INFLO 13 0.40000 0.33750 0.38532 0.32129 0.61538 0.57532 0.82708
INFLO 50 0.42400 0.36400 0.28994 0.21597 0.50000 0.44792 0.68542
COF 1 0.60000 0.55833 0.47429 0.41952 0.66667 0.63194 0.88333
COF 6 0.40000 0.33750 0.57262 0.52810 0.76923 0.74519 0.95417
COF 10 0.40000 0.33750 0.66551 0.63067 0.62500 0.58594 0.93750

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO