Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#01)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.4 kB) Download raw algorithm evaluation table (15.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40000 0.33750 0.41389 0.35284 0.57143 0.52679 0.88333
KNN 26 0.60000 0.55833 0.40553 0.34361 0.61538 0.57532 0.75833
KNN 30 0.60000 0.55833 0.47220 0.41722 0.72727 0.69886 0.77083
KNN 36 0.60000 0.55833 0.47294 0.41804 0.72727 0.69886 0.77917
KNNW 1 0.40000 0.33750 0.50644 0.45503 0.66667 0.63194 0.91667
LOF 3 0.40000 0.33750 0.66190 0.62669 0.66667 0.63194 0.90833
LOF 38 0.60000 0.55833 0.41444 0.35345 0.61538 0.57532 0.82917
SimplifiedLOF 3 0.40000 0.33750 0.39264 0.32937 0.50000 0.44792 0.85000
SimplifiedLOF 6 0.40000 0.33750 0.58549 0.54231 0.57143 0.52679 0.88750
SimplifiedLOF 7 0.40000 0.33750 0.55238 0.50575 0.61538 0.57532 0.91667
LoOP 4 0.40000 0.33750 0.43662 0.37793 0.50000 0.44792 0.88750
LoOP 6 0.40000 0.33750 0.63106 0.59263 0.58824 0.54534 0.92083
LoOP 8 0.20000 0.11667 0.41258 0.35139 0.62500 0.58594 0.90833
LDOF 8 0.20000 0.11667 0.43810 0.37956 0.47059 0.41544 0.85417
LDOF 9 0.40000 0.33750 0.32059 0.24982 0.47059 0.41544 0.81667
LDOF 14 0.40000 0.33750 0.61111 0.57060 0.57143 0.52679 0.80000
LDOF 16 0.40000 0.33750 0.51556 0.46510 0.66667 0.63194 0.78750
ODIN 3 0.40000 0.33750 0.28941 0.21539 0.40000 0.33750 0.82917
ODIN 12 0.40000 0.33750 0.39629 0.33340 0.40000 0.33750 0.72292
ODIN 41 0.40000 0.33750 0.34942 0.28166 0.57143 0.52679 0.77500
ODIN 46 0.50000 0.44792 0.36553 0.29944 0.54545 0.49811 0.80833
FastABOD 6 0.60000 0.55833 0.47000 0.41479 0.60000 0.55833 0.83333
FastABOD 9 0.60000 0.55833 0.49540 0.44283 0.66667 0.63194 0.84167
FastABOD 14 0.60000 0.55833 0.59393 0.55164 0.66667 0.63194 0.83750
KDEOS 7 0.40000 0.33750 0.55556 0.50926 0.57143 0.52679 0.91667
KDEOS 9 0.60000 0.55833 0.48250 0.42859 0.61538 0.57532 0.91667
KDEOS 11 0.60000 0.55833 0.51121 0.46029 0.66667 0.63194 0.93333
LDF 1 0.60000 0.55833 0.44889 0.39148 0.60000 0.55833 0.87500
LDF 4 0.40000 0.33750 0.49488 0.44226 0.50000 0.44792 0.89167
LDF 39 0.60000 0.55833 0.48333 0.42951 0.60000 0.55833 0.92500
INFLO 4 0.20000 0.11667 0.26667 0.19028 0.40000 0.33750 0.82083
INFLO 23 0.60000 0.55833 0.37826 0.31350 0.60000 0.55833 0.74583
INFLO 26 0.60000 0.55833 0.43887 0.38042 0.72727 0.69886 0.76667
INFLO 33 0.60000 0.55833 0.47220 0.41722 0.72727 0.69886 0.77083
COF 4 0.60000 0.55833 0.67833 0.64483 0.76923 0.74519 0.96250
COF 5 0.40000 0.33750 0.73929 0.71213 0.76923 0.74519 0.96250

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.0 kB) Download raw algorithm evaluation table (18.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 47 0.40000 0.33750 0.28978 0.21580 0.44444 0.38657 0.77500
KNN 48 0.20000 0.11667 0.26955 0.19346 0.47059 0.41544 0.81250
KNN 49 0.10000 0.00625 0.24254 0.16364 0.42105 0.36075 0.81458
KNNW 1 0.20000 0.11667 0.28660 0.21229 0.40000 0.33750 0.74167
LOF 2 0.20000 0.11667 0.39473 0.33168 0.40000 0.33750 0.77083
LOF 3 0.40000 0.33750 0.32121 0.25051 0.40000 0.33750 0.80000
LOF 4 0.20000 0.11667 0.26095 0.18397 0.42105 0.36075 0.71250
LOF 37 0.20000 0.11667 0.26435 0.18772 0.36364 0.29735 0.80417
SimplifiedLOF 1 0.40000 0.33750 0.49846 0.44622 0.57143 0.52679 0.67917
SimplifiedLOF 6 0.20000 0.11667 0.25445 0.17678 0.38095 0.31647 0.77917
LoOP 1 0.40000 0.33750 0.49774 0.44542 0.57143 0.52679 0.67500
LoOP 7 0.20000 0.11667 0.36501 0.29886 0.36364 0.29735 0.77500
LDOF 2 0.40000 0.33750 0.51678 0.46644 0.57143 0.52679 0.79167
LDOF 15 0.60000 0.55833 0.49087 0.43783 0.60000 0.55833 0.77500
ODIN 3 0.40000 0.33750 0.26978 0.19371 0.44444 0.38657 0.66458
ODIN 9 0.20000 0.11667 0.31905 0.24812 0.42857 0.36905 0.80208
ODIN 10 0.30000 0.22708 0.40893 0.34736 0.46154 0.40545 0.72500
ODIN 41 0.26667 0.19028 0.27826 0.20308 0.50000 0.44792 0.78542
FastABOD 3 0.20000 0.11667 0.26088 0.18389 0.35294 0.28554 0.67083
FastABOD 4 0.20000 0.11667 0.34222 0.27370 0.33333 0.26389 0.62917
FastABOD 10 0.20000 0.11667 0.33875 0.26986 0.36364 0.29735 0.60417
KDEOS 3 0.40000 0.33750 0.42626 0.36650 0.50000 0.44792 0.70000
KDEOS 7 0.40000 0.33750 0.41621 0.35540 0.40000 0.33750 0.80833
KDEOS 15 0.40000 0.33750 0.45146 0.39432 0.44444 0.38657 0.77083
KDEOS 16 0.40000 0.33750 0.38992 0.32637 0.54545 0.49811 0.78333
LDF 3 0.00000 -0.10417 0.25262 0.17476 0.50000 0.44792 0.77917
LDF 4 0.20000 0.11667 0.27842 0.20325 0.42857 0.36905 0.73333
LDF 20 0.40000 0.33750 0.20338 0.12040 0.40000 0.33750 0.50417
INFLO 6 0.20000 0.11667 0.28039 0.20543 0.37500 0.30990 0.81667
INFLO 28 0.40000 0.33750 0.37982 0.31522 0.66667 0.63194 0.75417
INFLO 49 0.41224 0.35102 0.22327 0.14236 0.44444 0.38657 0.67083
COF 1 0.40000 0.33750 0.49846 0.44622 0.57143 0.52679 0.67917
COF 5 0.20000 0.11667 0.45011 0.39283 0.50000 0.44792 0.83750

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO