Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#09)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (54.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.44000 0.30000 0.36815 0.21019 0.45912 0.32390 0.70229
KNN 2 0.44000 0.30000 0.36971 0.21213 0.47022 0.33777 0.70331
KNN 3 0.40800 0.26000 0.36291 0.20364 0.47324 0.34155 0.70520
KNN 79 0.33600 0.17000 0.36372 0.20465 0.46489 0.33111 0.72046
KNNW 3 0.42400 0.28000 0.36841 0.21051 0.46991 0.33739 0.70194
KNNW 4 0.44000 0.30000 0.36945 0.21181 0.46377 0.32971 0.70403
KNNW 100 0.34400 0.18000 0.36348 0.20435 0.45209 0.31511 0.71674
LOF 93 0.37600 0.22000 0.33687 0.17108 0.44068 0.30085 0.70168
LOF 100 0.36000 0.20000 0.33965 0.17456 0.44514 0.30643 0.70598
SimplifiedLOF 35 0.34400 0.18000 0.28552 0.10690 0.38026 0.22532 0.62064
SimplifiedLOF 100 0.32800 0.16000 0.30720 0.13400 0.40217 0.25272 0.64549
LoOP 45 0.34400 0.18000 0.28205 0.10256 0.36863 0.21078 0.61245
LoOP 85 0.33600 0.17000 0.29582 0.11977 0.39153 0.23942 0.63476
LoOP 99 0.33600 0.17000 0.30021 0.12526 0.38876 0.23595 0.63570
LoOP 100 0.32800 0.16000 0.30015 0.12519 0.38967 0.23709 0.63606
LDOF 35 0.28800 0.11000 0.26768 0.08460 0.37798 0.22248 0.59165
LDOF 86 0.34400 0.18000 0.28433 0.10542 0.37288 0.21610 0.60926
LDOF 98 0.35200 0.19000 0.28620 0.10775 0.37002 0.21253 0.60805
ODIN 95 0.35400 0.19250 0.29937 0.12421 0.40107 0.25134 0.65168
ODIN 99 0.34133 0.17667 0.30378 0.12972 0.40860 0.26075 0.65443
ODIN 100 0.34286 0.17857 0.30368 0.12960 0.40449 0.25562 0.65555
FastABOD 32 0.47200 0.34000 0.41163 0.26454 0.50196 0.37745 0.73650
FastABOD 36 0.48000 0.35000 0.41260 0.26575 0.49618 0.37023 0.73734
FastABOD 97 0.47200 0.34000 0.41986 0.27482 0.49110 0.36388 0.74448
KDEOS 2 0.24462 0.05578 0.23972 0.04965 0.34277 0.17846 0.54544
KDEOS 17 0.28000 0.10000 0.23085 0.03857 0.36174 0.20217 0.55685
KDEOS 33 0.20000 0.00000 0.21590 0.01987 0.37750 0.22187 0.55254
KDEOS 100 0.21600 0.02000 0.21824 0.02279 0.36992 0.21240 0.56253
LDF 42 0.36800 0.21000 0.32939 0.16174 0.46006 0.32508 0.68405
LDF 51 0.38400 0.23000 0.33308 0.16635 0.44138 0.30172 0.69109
LDF 92 0.36800 0.21000 0.35904 0.19880 0.44888 0.31110 0.72190
LDF 100 0.36000 0.20000 0.35622 0.19528 0.45570 0.31962 0.72570
INFLO 58 0.34400 0.18000 0.29544 0.11930 0.44306 0.30383 0.64569
INFLO 94 0.32800 0.16000 0.31209 0.14011 0.45532 0.31915 0.66427
INFLO 99 0.32800 0.16000 0.31326 0.14158 0.45923 0.32403 0.66080
INFLO 100 0.32800 0.16000 0.31435 0.14294 0.45923 0.32403 0.66210
COF 35 0.40000 0.25000 0.35582 0.19477 0.43360 0.29201 0.67229
COF 44 0.39200 0.24000 0.36523 0.20654 0.46991 0.33739 0.68486
COF 100 0.39200 0.24000 0.41002 0.26252 0.45509 0.31886 0.72243

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (54.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 9 0.46400 0.33000 0.37100 0.21374 0.47287 0.34109 0.67016
KNN 15 0.45600 0.32000 0.37311 0.21638 0.47191 0.33989 0.67189
KNN 70 0.40000 0.25000 0.36837 0.21046 0.47525 0.34406 0.68110
KNN 92 0.40000 0.25000 0.36396 0.20495 0.48466 0.35583 0.67586
KNNW 13 0.44800 0.31000 0.35130 0.18913 0.46465 0.33081 0.66803
KNNW 17 0.44000 0.30000 0.36188 0.20236 0.47350 0.34187 0.67035
KNNW 59 0.43200 0.29000 0.36891 0.21113 0.46429 0.33036 0.67438
KNNW 83 0.41600 0.27000 0.36871 0.21089 0.46316 0.32895 0.67640
LOF 74 0.41600 0.27000 0.34831 0.18539 0.44444 0.30556 0.67246
LOF 85 0.41600 0.27000 0.35379 0.19224 0.45675 0.32093 0.67970
LOF 100 0.39200 0.24000 0.35791 0.19739 0.45517 0.31897 0.68410
SimplifiedLOF 89 0.41600 0.27000 0.33280 0.16600 0.41600 0.27000 0.63685
SimplifiedLOF 98 0.41600 0.27000 0.33618 0.17023 0.42623 0.28279 0.63907
SimplifiedLOF 100 0.41600 0.27000 0.33604 0.17005 0.41935 0.27419 0.63963
LoOP 88 0.41600 0.27000 0.32679 0.15849 0.42188 0.27734 0.63190
LoOP 90 0.41600 0.27000 0.32750 0.15938 0.42688 0.28360 0.63313
LoOP 91 0.41600 0.27000 0.32635 0.15794 0.42857 0.28571 0.63290
LoOP 97 0.40800 0.26000 0.32687 0.15858 0.42353 0.27941 0.63643
LDOF 86 0.40800 0.26000 0.32421 0.15526 0.41296 0.26619 0.62696
LDOF 99 0.40000 0.25000 0.33118 0.16398 0.41667 0.27083 0.63450
LDOF 100 0.40000 0.25000 0.33126 0.16407 0.41176 0.26471 0.63542
ODIN 89 0.38133 0.22667 0.31047 0.13808 0.42857 0.28571 0.61708
ODIN 96 0.38400 0.23000 0.31248 0.14060 0.41569 0.26961 0.61938
ODIN 98 0.39467 0.24333 0.31060 0.13825 0.41107 0.26383 0.62025
ODIN 100 0.40000 0.25000 0.31191 0.13989 0.41304 0.26630 0.61910
FastABOD 86 0.45600 0.32000 0.38401 0.23002 0.48665 0.35831 0.70592
FastABOD 87 0.46400 0.33000 0.38405 0.23006 0.48521 0.35651 0.70603
FastABOD 98 0.46400 0.33000 0.38480 0.23100 0.48665 0.35831 0.70682
FastABOD 99 0.46400 0.33000 0.38477 0.23097 0.48630 0.35788 0.70685
KDEOS 8 0.22400 0.03000 0.20600 0.00750 0.33564 0.16956 0.50794
KDEOS 99 0.18400 -0.02000 0.22217 0.02771 0.37037 0.21296 0.56530
KDEOS 100 0.18400 -0.02000 0.22206 0.02757 0.37209 0.21512 0.56581
LDF 73 0.41600 0.27000 0.36426 0.20533 0.46099 0.32624 0.68965
LDF 93 0.38400 0.23000 0.37025 0.21282 0.46686 0.33357 0.69755
LDF 96 0.39200 0.24000 0.37146 0.21433 0.46328 0.32910 0.69789
LDF 97 0.39200 0.24000 0.37206 0.21507 0.46328 0.32910 0.69760
INFLO 71 0.41600 0.27000 0.34421 0.18026 0.48379 0.35474 0.68005
INFLO 74 0.40000 0.25000 0.34892 0.18615 0.49377 0.36721 0.69838
INFLO 96 0.38400 0.23000 0.35302 0.19127 0.49357 0.36697 0.68650
INFLO 98 0.37600 0.22000 0.35147 0.18934 0.49479 0.36849 0.68506
COF 89 0.37600 0.22000 0.34882 0.18602 0.43658 0.29572 0.67767
COF 99 0.40000 0.25000 0.35521 0.19402 0.42604 0.28254 0.69132
COF 100 0.40000 0.25000 0.35649 0.19561 0.42407 0.28009 0.69225

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO