Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#02)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.5 kB) Download raw algorithm evaluation table (15.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40000 0.33750 0.42698 0.36729 0.57143 0.52679 0.91250
KNN 24 0.60000 0.55833 0.38667 0.32278 0.60000 0.55833 0.76250
KNNW 1 0.60000 0.55833 0.50095 0.44897 0.66667 0.63194 0.92500
LOF 2 0.80000 0.77917 0.86000 0.84542 0.80000 0.77917 0.97500
SimplifiedLOF 4 0.60000 0.55833 0.78095 0.75813 0.75000 0.72396 0.94583
SimplifiedLOF 6 0.80000 0.77917 0.81263 0.79311 0.80000 0.77917 0.93750
LoOP 2 0.60000 0.55833 0.42000 0.35958 0.60000 0.55833 0.90000
LoOP 5 0.60000 0.55833 0.69679 0.66520 0.66667 0.63194 0.93333
LoOP 6 0.60000 0.55833 0.66429 0.62932 0.66667 0.63194 0.93750
LDOF 7 0.60000 0.55833 0.56359 0.51813 0.60000 0.55833 0.91667
LDOF 10 0.40000 0.33750 0.48996 0.43683 0.62500 0.58594 0.92500
LDOF 14 0.40000 0.33750 0.59460 0.55237 0.57143 0.52679 0.88333
ODIN 3 0.28571 0.21131 0.31429 0.24286 0.50000 0.44792 0.88542
ODIN 10 0.40000 0.33750 0.38315 0.31890 0.66667 0.63194 0.75208
ODIN 11 0.53333 0.48472 0.44744 0.38988 0.66667 0.63194 0.76250
FastABOD 5 0.60000 0.55833 0.49778 0.44546 0.71429 0.68452 0.94167
FastABOD 6 0.60000 0.55833 0.52595 0.47657 0.76923 0.74519 0.95000
KDEOS 8 0.60000 0.55833 0.55333 0.50681 0.60000 0.55833 0.90833
KDEOS 10 0.60000 0.55833 0.54915 0.50218 0.66667 0.63194 0.93333
KDEOS 12 0.40000 0.33750 0.50794 0.45668 0.66667 0.63194 0.93333
LDF 1 0.60000 0.55833 0.57926 0.53543 0.60000 0.55833 0.87500
LDF 2 0.60000 0.55833 0.53929 0.49129 0.66667 0.63194 0.89583
LDF 4 0.60000 0.55833 0.42000 0.35958 0.60000 0.55833 0.90000
INFLO 2 0.60000 0.55833 0.84444 0.82824 0.75000 0.72396 0.97500
COF 2 0.60000 0.55833 0.41048 0.34907 0.60000 0.55833 0.89167
COF 5 0.60000 0.55833 0.85833 0.84358 0.76923 0.74519 0.97917
COF 6 0.60000 0.55833 0.59619 0.55413 0.83333 0.81597 0.96250

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (233.9 kB) Download raw algorithm evaluation table (17.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.20000 0.11667 0.25452 0.17687 0.44444 0.38657 0.75625
KNN 5 0.20000 0.11667 0.27502 0.19950 0.50000 0.44792 0.69167
KNN 19 0.40000 0.33750 0.22429 0.14348 0.40000 0.33750 0.58750
KNNW 1 0.20000 0.11667 0.33517 0.26592 0.50000 0.44792 0.81042
KNNW 50 0.40000 0.33750 0.22844 0.14807 0.40000 0.33750 0.60833
LOF 12 0.20000 0.11667 0.29114 0.21731 0.50000 0.44792 0.72917
LOF 13 0.20000 0.11667 0.29595 0.22261 0.46154 0.40545 0.82500
LOF 21 0.40000 0.33750 0.23556 0.15593 0.40000 0.33750 0.62500
SimplifiedLOF 1 0.40000 0.33750 0.37256 0.30721 0.40000 0.33750 0.70208
SimplifiedLOF 20 0.20000 0.11667 0.31465 0.24326 0.53333 0.48472 0.79167
LoOP 1 0.40000 0.33750 0.37220 0.30681 0.40000 0.33750 0.70000
LoOP 21 0.20000 0.11667 0.21553 0.13382 0.38095 0.31647 0.71458
LDOF 2 0.20000 0.11667 0.36871 0.30295 0.40000 0.33750 0.70833
LDOF 15 0.40000 0.33750 0.21660 0.13499 0.40000 0.33750 0.60417
LDOF 16 0.40000 0.33750 0.23265 0.15272 0.44444 0.38657 0.57917
ODIN 13 0.40000 0.33750 0.23303 0.15314 0.40000 0.33750 0.72708
ODIN 15 0.40000 0.33750 0.25638 0.17891 0.40000 0.33750 0.77917
ODIN 28 0.40000 0.33750 0.25872 0.18150 0.46154 0.40545 0.60833
FastABOD 3 0.20000 0.11667 0.25434 0.17667 0.33333 0.26389 0.67083
FastABOD 4 0.20000 0.11667 0.33574 0.26654 0.33333 0.26389 0.66667
FastABOD 34 0.20000 0.11667 0.35507 0.28789 0.33333 0.26389 0.67917
KDEOS 3 0.40000 0.33750 0.42857 0.36905 0.44444 0.38657 0.74167
KDEOS 21 0.40000 0.33750 0.28298 0.20829 0.47059 0.41544 0.77500
KDEOS 23 0.00000 -0.10417 0.25155 0.17359 0.53333 0.48472 0.76250
LDF 7 0.40000 0.33750 0.24593 0.16739 0.40000 0.33750 0.70000
LDF 22 0.40000 0.33750 0.30273 0.23010 0.42857 0.36905 0.80417
LDF 23 0.40000 0.33750 0.28849 0.21437 0.50000 0.44792 0.73333
LDF 31 0.20000 0.11667 0.28433 0.20978 0.46154 0.40545 0.80833
INFLO 8 0.20000 0.11667 0.26736 0.19104 0.40000 0.33750 0.80000
INFLO 17 0.40000 0.33750 0.32472 0.25437 0.50000 0.44792 0.74167
INFLO 48 0.40833 0.34670 0.21630 0.13467 0.44444 0.38657 0.57500
COF 1 0.40000 0.33750 0.37256 0.30721 0.40000 0.33750 0.70208
COF 13 0.20000 0.11667 0.32333 0.25285 0.42857 0.36905 0.82292
COF 48 0.20000 0.11667 0.31270 0.24110 0.50000 0.44792 0.72500
COF 51 0.20000 0.11667 0.39192 0.32858 0.42857 0.36905 0.71875

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO