Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#09)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (301.2 kB) Download raw algorithm evaluation table (23.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.58333 0.73746 0.67182 0.74074 0.67593 0.89062
KNN 34 0.75000 0.68750 0.75248 0.69059 0.75000 0.68750 0.80035
KNN 42 0.75000 0.68750 0.77686 0.72107 0.78261 0.72826 0.81597
KNNW 1 0.66667 0.58333 0.71999 0.64999 0.72727 0.65909 0.89236
KNNW 2 0.66667 0.58333 0.73974 0.67468 0.74074 0.67593 0.90451
KNNW 4 0.66667 0.58333 0.77730 0.72162 0.72000 0.65000 0.89583
LOF 32 0.75000 0.68750 0.73498 0.66872 0.75000 0.68750 0.80208
LOF 47 0.66667 0.58333 0.79623 0.74529 0.76190 0.70238 0.90625
LOF 53 0.75000 0.68750 0.84348 0.80435 0.80000 0.75000 0.90451
SimplifiedLOF 18 0.66667 0.58333 0.68767 0.60958 0.69231 0.61538 0.83854
SimplifiedLOF 43 0.66667 0.58333 0.71854 0.64817 0.72727 0.65909 0.79340
SimplifiedLOF 54 0.66667 0.58333 0.73948 0.67436 0.72727 0.65909 0.79688
LoOP 45 0.66667 0.58333 0.69120 0.61400 0.66667 0.58333 0.78819
LoOP 49 0.66667 0.58333 0.70231 0.62789 0.72727 0.65909 0.78906
LoOP 52 0.66667 0.58333 0.72180 0.65225 0.72727 0.65909 0.79948
LoOP 56 0.66667 0.58333 0.71339 0.64174 0.72727 0.65909 0.81076
LDOF 26 0.66667 0.58333 0.66084 0.57606 0.66667 0.58333 0.80035
LDOF 28 0.58333 0.47917 0.68842 0.61052 0.64000 0.55000 0.80208
LDOF 56 0.66667 0.58333 0.72634 0.65792 0.72727 0.65909 0.78819
ODIN 24 0.66667 0.58333 0.61625 0.52032 0.66667 0.58333 0.73611
ODIN 44 0.66667 0.58333 0.72503 0.65629 0.72727 0.65909 0.80729
ODIN 48 0.66667 0.58333 0.71774 0.64717 0.72727 0.65909 0.81944
ODIN 50 0.66667 0.58333 0.70175 0.62719 0.76190 0.70238 0.76910
FastABOD 6 0.66667 0.58333 0.77406 0.71758 0.76190 0.70238 0.91319
FastABOD 8 0.75000 0.68750 0.78939 0.73674 0.76190 0.70238 0.89236
FastABOD 10 0.75000 0.68750 0.81451 0.76814 0.78261 0.72826 0.89583
KDEOS 15 0.41667 0.27083 0.44841 0.31051 0.60606 0.50758 0.82292
KDEOS 44 0.41667 0.27083 0.42989 0.28736 0.66667 0.58333 0.77778
KDEOS 58 0.66667 0.58333 0.69293 0.61617 0.66667 0.58333 0.80556
KDEOS 59 0.66667 0.58333 0.71832 0.64790 0.66667 0.58333 0.80903
LDF 22 0.75000 0.68750 0.76901 0.71126 0.76190 0.70238 0.83681
LDF 25 0.75000 0.68750 0.79230 0.74038 0.81818 0.77273 0.86458
LDF 42 0.66667 0.58333 0.83614 0.79518 0.81481 0.76852 0.91840
INFLO 42 0.75000 0.68750 0.79103 0.73878 0.78261 0.72826 0.85330
INFLO 43 0.75000 0.68750 0.79139 0.73924 0.78261 0.72826 0.85417
COF 6 0.58333 0.47917 0.60517 0.50646 0.63636 0.54545 0.81597
COF 10 0.58333 0.47917 0.56057 0.45071 0.66667 0.58333 0.82639
COF 12 0.50000 0.37500 0.56294 0.45368 0.68966 0.61207 0.82292
COF 46 0.58333 0.47917 0.68507 0.60634 0.64286 0.55357 0.77257

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (299.0 kB) Download raw algorithm evaluation table (25.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.06250 0.26281 0.07851 0.48000 0.35000 0.62760
KNN 57 0.25000 0.06250 0.30562 0.13203 0.52632 0.40789 0.71441
KNN 58 0.25000 0.06250 0.31059 0.13824 0.52632 0.40789 0.71181
KNNW 1 0.25000 0.06250 0.28288 0.10360 0.46809 0.33511 0.63715
KNNW 2 0.25000 0.06250 0.26043 0.07554 0.48980 0.36224 0.63368
LOF 6 0.25000 0.06250 0.32817 0.16021 0.54545 0.43182 0.71007
LOF 7 0.41667 0.27083 0.35732 0.19665 0.54054 0.42568 0.71875
SimplifiedLOF 1 0.25000 0.06250 0.38504 0.23130 0.44444 0.30556 0.65104
SimplifiedLOF 7 0.25000 0.06250 0.33840 0.17299 0.54545 0.43182 0.74653
SimplifiedLOF 11 0.41667 0.27083 0.33077 0.16346 0.48780 0.35976 0.71701
LoOP 1 0.25000 0.06250 0.38389 0.22987 0.44444 0.30556 0.64757
LoOP 5 0.33333 0.16667 0.34410 0.18012 0.51613 0.39516 0.66146
LoOP 7 0.25000 0.06250 0.33255 0.16569 0.48649 0.35811 0.70226
LoOP 11 0.41667 0.27083 0.29958 0.12448 0.44444 0.30556 0.64583
LDOF 7 0.41667 0.27083 0.33652 0.17066 0.50000 0.37500 0.67708
LDOF 13 0.41667 0.27083 0.40615 0.25768 0.52174 0.40217 0.73785
ODIN 3 0.38333 0.22917 0.31313 0.14141 0.42105 0.27632 0.67882
ODIN 4 0.29167 0.11458 0.31848 0.14810 0.50000 0.37500 0.68924
ODIN 5 0.30952 0.13690 0.34511 0.18138 0.50000 0.37500 0.70920
FastABOD 3 0.16667 -0.04167 0.24205 0.05256 0.40741 0.25926 0.57986
FastABOD 4 0.25000 0.06250 0.23133 0.03916 0.39286 0.24107 0.55556
KDEOS 6 0.50000 0.37500 0.45709 0.32137 0.57143 0.46429 0.73090
KDEOS 14 0.33333 0.16667 0.39976 0.24970 0.66667 0.58333 0.81250
KDEOS 16 0.50000 0.37500 0.45765 0.32207 0.57143 0.46429 0.80729
LDF 6 0.41667 0.27083 0.36026 0.20033 0.52632 0.40789 0.72917
INFLO 1 0.33333 0.16667 0.37069 0.21336 0.50000 0.37500 0.69010
INFLO 17 0.50000 0.37500 0.35085 0.18856 0.51613 0.39516 0.71701
COF 5 0.41667 0.27083 0.37152 0.21440 0.51064 0.38830 0.75521
COF 6 0.41667 0.27083 0.38756 0.23445 0.58537 0.48171 0.78646
COF 11 0.33333 0.16667 0.38149 0.22686 0.57895 0.47368 0.79340
COF 12 0.33333 0.16667 0.38005 0.22506 0.60000 0.50000 0.78819

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO