Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#01)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (300.3 kB) Download raw algorithm evaluation table (23.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.58333 0.74733 0.68416 0.71429 0.64286 0.92014
KNNW 1 0.75000 0.68750 0.85321 0.81652 0.84615 0.80769 0.95833
LOF 3 0.75000 0.68750 0.79538 0.74422 0.78571 0.73214 0.91840
LOF 4 0.75000 0.68750 0.83454 0.79317 0.81818 0.77273 0.90799
SimplifiedLOF 5 0.66667 0.58333 0.73694 0.67118 0.76190 0.70238 0.89931
SimplifiedLOF 7 0.66667 0.58333 0.77639 0.72049 0.74074 0.67593 0.91493
SimplifiedLOF 8 0.66667 0.58333 0.77272 0.71590 0.72727 0.65909 0.91840
LoOP 5 0.58333 0.47917 0.69895 0.62369 0.70000 0.62500 0.88368
LoOP 7 0.58333 0.47917 0.76096 0.70120 0.70000 0.62500 0.90104
LoOP 8 0.66667 0.58333 0.71840 0.64800 0.69231 0.61538 0.89410
LDOF 4 0.58333 0.47917 0.46038 0.32547 0.64000 0.55000 0.75521
LDOF 9 0.50000 0.37500 0.54954 0.43692 0.56000 0.45000 0.75868
LDOF 10 0.50000 0.37500 0.59012 0.48765 0.55556 0.44444 0.75694
ODIN 5 0.58333 0.47917 0.50228 0.37785 0.58333 0.47917 0.80469
ODIN 7 0.58333 0.47917 0.46454 0.33067 0.60870 0.51087 0.76128
ODIN 45 0.50000 0.37500 0.56389 0.45486 0.57143 0.46429 0.59115
FastABOD 3 0.75000 0.68750 0.84950 0.81188 0.78261 0.72826 0.93924
FastABOD 4 0.75000 0.68750 0.83848 0.79810 0.81481 0.76852 0.95486
FastABOD 5 0.75000 0.68750 0.84928 0.81160 0.81481 0.76852 0.95833
KDEOS 12 0.58333 0.47917 0.67035 0.58794 0.66667 0.58333 0.87847
KDEOS 16 0.75000 0.68750 0.58652 0.48316 0.75000 0.68750 0.90104
KDEOS 17 0.66667 0.58333 0.60227 0.50284 0.72000 0.65000 0.90799
LDF 2 0.50000 0.37500 0.72520 0.65650 0.64286 0.55357 0.88542
LDF 50 0.75000 0.68750 0.75098 0.68873 0.75000 0.68750 0.86632
LDF 51 0.75000 0.68750 0.78307 0.72884 0.75000 0.68750 0.85590
INFLO 4 0.66667 0.58333 0.78748 0.73435 0.76190 0.70238 0.89062
INFLO 5 0.58333 0.47917 0.72663 0.65829 0.75862 0.69828 0.91493
COF 6 0.83333 0.79167 0.91504 0.89380 0.85714 0.82143 0.96007
COF 7 0.83333 0.79167 0.90698 0.88372 0.83333 0.79167 0.96528

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (299.2 kB) Download raw algorithm evaluation table (25.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.50000 0.37500 0.35964 0.19955 0.51852 0.39815 0.72222
KNNW 1 0.33333 0.16667 0.36830 0.21038 0.47826 0.34783 0.70399
KNNW 3 0.33333 0.16667 0.32750 0.15937 0.46809 0.33511 0.71181
KNNW 4 0.41667 0.27083 0.33195 0.16494 0.46667 0.33333 0.71181
LOF 3 0.58333 0.47917 0.47090 0.33862 0.58333 0.47917 0.76736
LOF 4 0.58333 0.47917 0.48183 0.35229 0.61538 0.51923 0.79688
SimplifiedLOF 4 0.50000 0.37500 0.48784 0.35980 0.58824 0.48529 0.81076
SimplifiedLOF 6 0.50000 0.37500 0.43991 0.29989 0.61111 0.51389 0.80556
LoOP 5 0.41667 0.27083 0.48562 0.35703 0.62857 0.53571 0.79861
LoOP 8 0.58333 0.47917 0.39543 0.24429 0.58333 0.47917 0.75260
LDOF 5 0.33333 0.16667 0.43040 0.28800 0.45161 0.31452 0.69965
LDOF 7 0.33333 0.16667 0.42037 0.27547 0.56250 0.45313 0.74826
LDOF 8 0.41667 0.27083 0.41029 0.26286 0.53333 0.41667 0.74306
LDOF 12 0.41667 0.27083 0.37422 0.21778 0.52632 0.40789 0.75694
ODIN 1 0.42105 0.27632 0.36075 0.20093 0.51613 0.39516 0.74132
FastABOD 3 0.33333 0.16667 0.29830 0.12288 0.44898 0.31122 0.64236
FastABOD 4 0.41667 0.27083 0.30332 0.12915 0.43478 0.29348 0.64236
KDEOS 8 0.41667 0.27083 0.38512 0.23140 0.60000 0.50000 0.79167
KDEOS 18 0.41667 0.27083 0.49900 0.37375 0.53846 0.42308 0.78299
KDEOS 20 0.58333 0.47917 0.49245 0.36556 0.58333 0.47917 0.78299
LDF 4 0.41667 0.27083 0.43651 0.29564 0.57143 0.46429 0.73958
LDF 5 0.50000 0.37500 0.36884 0.21105 0.50000 0.37500 0.64410
LDF 10 0.41667 0.27083 0.41144 0.26429 0.51852 0.39815 0.77083
INFLO 4 0.50000 0.37500 0.46609 0.33261 0.55000 0.43750 0.76910
COF 4 0.50000 0.37500 0.52948 0.41185 0.66667 0.58333 0.81597
COF 5 0.50000 0.37500 0.51885 0.39856 0.66667 0.58333 0.81684
COF 9 0.58333 0.47917 0.43083 0.28854 0.58824 0.48529 0.81424
COF 10 0.58333 0.47917 0.44393 0.30492 0.62500 0.53125 0.83160

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO