Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#06)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (233.5 kB) Download raw algorithm evaluation table (16.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.20000 0.11667 0.44229 0.38419 0.47059 0.41544 0.85000
KNN 27 0.40000 0.33750 0.32959 0.25975 0.40000 0.33750 0.43958
KNN 37 0.40000 0.33750 0.38909 0.32545 0.50000 0.44792 0.51250
KNNW 1 0.60000 0.55833 0.63889 0.60127 0.66667 0.63194 0.92083
LOF 1 0.60000 0.55833 0.55424 0.50781 0.66667 0.63194 0.93750
LOF 3 0.60000 0.55833 0.73111 0.70310 0.71429 0.68452 0.95833
SimplifiedLOF 1 0.60000 0.55833 0.53026 0.48132 0.72727 0.69886 0.93750
SimplifiedLOF 5 0.60000 0.55833 0.76429 0.73973 0.66667 0.63194 0.96250
SimplifiedLOF 6 0.60000 0.55833 0.69262 0.66060 0.76923 0.74519 0.96250
LoOP 1 0.60000 0.55833 0.53026 0.48132 0.72727 0.69886 0.93750
LoOP 4 0.60000 0.55833 0.70873 0.67839 0.71429 0.68452 0.96250
LoOP 6 0.40000 0.33750 0.63929 0.60171 0.76923 0.74519 0.95417
LDOF 2 0.40000 0.33750 0.36179 0.29531 0.53333 0.48472 0.82083
LDOF 5 0.40000 0.33750 0.37720 0.31232 0.40000 0.33750 0.72500
ODIN 2 0.36364 0.29735 0.33853 0.26962 0.50000 0.44792 0.89375
ODIN 3 0.42857 0.36905 0.36414 0.29790 0.50000 0.44792 0.89375
ODIN 4 0.40000 0.33750 0.43095 0.37168 0.50000 0.44792 0.87708
FastABOD 3 0.40000 0.33750 0.55611 0.50987 0.57143 0.52679 0.80417
KDEOS 6 0.40000 0.33750 0.34508 0.27686 0.50000 0.44792 0.86667
KDEOS 13 0.40000 0.33750 0.61429 0.57411 0.66667 0.63194 0.94583
KDEOS 14 0.40000 0.33750 0.52540 0.47596 0.71429 0.68452 0.94583
LDF 3 0.80000 0.77917 0.71476 0.68505 0.80000 0.77917 0.95000
INFLO 1 0.60000 0.55833 0.55513 0.50879 0.72727 0.69886 0.89167
INFLO 2 0.60000 0.55833 0.73095 0.70293 0.66667 0.63194 0.94167
INFLO 3 0.60000 0.55833 0.74416 0.71751 0.75000 0.72396 0.93333
COF 3 0.60000 0.55833 0.68667 0.65403 0.72727 0.69886 0.95833
COF 4 0.80000 0.77917 0.71000 0.67979 0.80000 0.77917 0.94583
COF 5 0.80000 0.77917 0.76882 0.74474 0.80000 0.77917 0.94167

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (233.9 kB) Download raw algorithm evaluation table (18.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.20000 0.11667 0.19583 0.11207 0.40000 0.33750 0.63333
KNN 3 0.20000 0.11667 0.22272 0.14176 0.42857 0.36905 0.67708
KNNW 1 0.40000 0.33750 0.31993 0.24909 0.50000 0.44792 0.72500
LOF 2 0.40000 0.33750 0.30282 0.23020 0.50000 0.44792 0.61458
LOF 3 0.40000 0.33750 0.38986 0.32630 0.50000 0.44792 0.51667
LOF 4 0.40000 0.33750 0.23616 0.15659 0.40000 0.33750 0.69583
SimplifiedLOF 1 0.20000 0.11667 0.25641 0.17895 0.40000 0.33750 0.76250
SimplifiedLOF 2 0.40000 0.33750 0.29637 0.22307 0.44444 0.38657 0.72083
SimplifiedLOF 3 0.40000 0.33750 0.31727 0.24615 0.50000 0.44792 0.68333
SimplifiedLOF 5 0.40000 0.33750 0.38108 0.31661 0.40000 0.33750 0.72083
LoOP 1 0.20000 0.11667 0.25641 0.17895 0.40000 0.33750 0.76250
LoOP 2 0.40000 0.33750 0.30256 0.22991 0.44444 0.38657 0.72917
LoOP 5 0.40000 0.33750 0.33415 0.26479 0.50000 0.44792 0.71458
LDOF 2 0.20000 0.11667 0.19070 0.10640 0.28571 0.21131 0.53333
LDOF 4 0.20000 0.11667 0.24817 0.16985 0.33333 0.26389 0.74583
LDOF 5 0.20000 0.11667 0.32949 0.25964 0.33333 0.26389 0.65417
LDOF 6 0.00000 -0.10417 0.18500 0.10011 0.37500 0.30990 0.70417
ODIN 2 0.40000 0.33750 0.26128 0.18433 0.40000 0.33750 0.75625
FastABOD 3 0.20000 0.11667 0.21599 0.13432 0.46154 0.40545 0.63750
KDEOS 3 0.40000 0.33750 0.32577 0.25554 0.50000 0.44792 0.70833
KDEOS 5 0.40000 0.33750 0.35651 0.28948 0.50000 0.44792 0.76667
KDEOS 8 0.40000 0.33750 0.35432 0.28706 0.46154 0.40545 0.80417
LDF 2 0.40000 0.33750 0.41750 0.35683 0.50000 0.44792 0.68958
LDF 11 0.40000 0.33750 0.23832 0.15897 0.40000 0.33750 0.69167
INFLO 2 0.40000 0.33750 0.31525 0.24392 0.50000 0.44792 0.68125
INFLO 3 0.40000 0.33750 0.48500 0.43135 0.50000 0.44792 0.83750
COF 2 0.40000 0.33750 0.31092 0.23914 0.44444 0.38657 0.74167
COF 5 0.40000 0.33750 0.42113 0.36083 0.50000 0.44792 0.70000
COF 11 0.20000 0.11667 0.25007 0.17195 0.40000 0.33750 0.77083

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO