Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#01)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (55.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.41600 0.27000 0.36672 0.20840 0.48408 0.35510 0.72774
KNN 21 0.36800 0.21000 0.36114 0.20142 0.49162 0.36453 0.73648
KNN 57 0.35200 0.19000 0.37020 0.21275 0.47721 0.34651 0.74266
KNN 58 0.34400 0.18000 0.36990 0.21237 0.47701 0.34626 0.74282
KNNW 6 0.40800 0.26000 0.36399 0.20499 0.48000 0.35000 0.72923
KNNW 41 0.38400 0.23000 0.36570 0.20712 0.48795 0.35994 0.73858
KNNW 76 0.38400 0.23000 0.36917 0.21146 0.48333 0.35417 0.74128
KNNW 99 0.36000 0.20000 0.36872 0.21090 0.47802 0.34753 0.74170
LOF 88 0.36000 0.20000 0.33192 0.16490 0.47788 0.34735 0.71966
LOF 97 0.36800 0.21000 0.33474 0.16843 0.47592 0.34490 0.72451
LOF 100 0.36800 0.21000 0.33680 0.17100 0.47429 0.34286 0.72568
SimplifiedLOF 90 0.33600 0.17000 0.29772 0.12215 0.41604 0.27005 0.66198
SimplifiedLOF 99 0.32800 0.16000 0.30028 0.12535 0.42588 0.28235 0.66733
SimplifiedLOF 100 0.32000 0.15000 0.30125 0.12657 0.42424 0.28030 0.66834
LoOP 49 0.33600 0.17000 0.27751 0.09688 0.38393 0.22991 0.62797
LoOP 100 0.33600 0.17000 0.29412 0.11765 0.41711 0.27139 0.65826
LDOF 75 0.34400 0.18000 0.27943 0.09928 0.40217 0.25272 0.63058
LDOF 76 0.34400 0.18000 0.27945 0.09932 0.40331 0.25414 0.63101
LDOF 100 0.32800 0.16000 0.28149 0.10186 0.38916 0.23645 0.63184
ODIN 82 0.35600 0.19500 0.30124 0.12654 0.43939 0.29924 0.67059
ODIN 83 0.35400 0.19250 0.30293 0.12867 0.44221 0.30276 0.67130
ODIN 99 0.34100 0.17625 0.31039 0.13798 0.43902 0.29878 0.67974
ODIN 100 0.33600 0.17000 0.31018 0.13772 0.44142 0.30177 0.68052
FastABOD 24 0.49600 0.37000 0.39899 0.24874 0.50196 0.37745 0.74672
FastABOD 62 0.48000 0.35000 0.40937 0.26171 0.50923 0.38653 0.75693
FastABOD 84 0.48000 0.35000 0.41269 0.26586 0.50735 0.38419 0.75947
FastABOD 98 0.47200 0.34000 0.41171 0.26464 0.50725 0.38406 0.76011
KDEOS 14 0.26400 0.08000 0.23714 0.04642 0.34982 0.18728 0.55758
KDEOS 16 0.25600 0.07000 0.24855 0.06068 0.35226 0.19032 0.56386
KDEOS 41 0.19200 -0.01000 0.22971 0.03713 0.37782 0.22227 0.55875
KDEOS 100 0.21600 0.02000 0.22318 0.02897 0.37500 0.21875 0.58192
LDF 61 0.38400 0.23000 0.34504 0.18131 0.47778 0.34722 0.72632
LDF 99 0.37600 0.22000 0.35968 0.19960 0.48466 0.35583 0.74630
LDF 100 0.36800 0.21000 0.36006 0.20008 0.48916 0.36146 0.74621
INFLO 52 0.36000 0.20000 0.29008 0.11260 0.45511 0.31889 0.66322
INFLO 99 0.34400 0.18000 0.30984 0.13730 0.47458 0.34322 0.68782
INFLO 100 0.34400 0.18000 0.31008 0.13760 0.47558 0.34448 0.68694
COF 93 0.42400 0.28000 0.39970 0.24963 0.48538 0.35673 0.74221
COF 99 0.42400 0.28000 0.40818 0.26022 0.47500 0.34375 0.74749
COF 100 0.44000 0.30000 0.40762 0.25953 0.47647 0.34559 0.74722

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (54.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 12 0.44000 0.30000 0.35651 0.19563 0.47143 0.33929 0.69228
KNN 18 0.43200 0.29000 0.36176 0.20220 0.46622 0.33277 0.69512
KNN 72 0.40000 0.25000 0.36171 0.20214 0.47701 0.34626 0.70246
KNN 89 0.41600 0.27000 0.35881 0.19852 0.47929 0.34911 0.69861
KNNW 26 0.44000 0.30000 0.35520 0.19400 0.46377 0.32971 0.69226
KNNW 58 0.42400 0.28000 0.36006 0.20007 0.46459 0.33074 0.69813
KNNW 90 0.42400 0.28000 0.35877 0.19846 0.47253 0.34066 0.69925
KNNW 92 0.42400 0.28000 0.35902 0.19878 0.47154 0.33943 0.69933
LOF 93 0.40000 0.25000 0.34060 0.17576 0.46309 0.32886 0.69226
LOF 94 0.39200 0.24000 0.34090 0.17613 0.46309 0.32886 0.69250
LOF 100 0.39200 0.24000 0.34056 0.17570 0.46483 0.33104 0.69211
SimplifiedLOF 81 0.37600 0.22000 0.30944 0.13680 0.40751 0.25938 0.63848
SimplifiedLOF 100 0.36800 0.21000 0.31568 0.14460 0.41926 0.27408 0.64675
LoOP 76 0.37600 0.22000 0.29734 0.12167 0.39572 0.24465 0.63330
LoOP 96 0.36000 0.20000 0.30567 0.13208 0.40712 0.25891 0.64626
LoOP 98 0.36000 0.20000 0.30449 0.13061 0.41131 0.26414 0.64710
LoOP 99 0.36000 0.20000 0.30379 0.12974 0.41344 0.26680 0.64436
LDOF 99 0.36000 0.20000 0.31278 0.14098 0.41192 0.26491 0.63712
LDOF 100 0.36000 0.20000 0.31377 0.14221 0.41192 0.26491 0.63806
ODIN 70 0.32200 0.15250 0.28811 0.11014 0.40415 0.25518 0.62071
ODIN 93 0.37600 0.22000 0.29232 0.11541 0.39012 0.23765 0.61986
ODIN 98 0.36000 0.20000 0.29644 0.12055 0.39568 0.24460 0.62141
ODIN 100 0.36000 0.20000 0.29631 0.12039 0.39394 0.24242 0.62254
FastABOD 83 0.43200 0.29000 0.36936 0.21170 0.47799 0.34748 0.70374
FastABOD 98 0.43200 0.29000 0.36971 0.21214 0.47975 0.34969 0.70499
FastABOD 100 0.43200 0.29000 0.36997 0.21247 0.47975 0.34969 0.70531
KDEOS 7 0.23200 0.04000 0.21936 0.02420 0.33829 0.17286 0.52307
KDEOS 91 0.21600 0.02000 0.22652 0.03315 0.37629 0.22036 0.58002
KDEOS 100 0.22400 0.03000 0.23190 0.03988 0.37374 0.21717 0.58549
LDF 73 0.40000 0.25000 0.35131 0.18913 0.46622 0.33277 0.70470
LDF 91 0.40000 0.25000 0.35543 0.19429 0.47619 0.34524 0.71059
LDF 93 0.39200 0.24000 0.35505 0.19381 0.47977 0.34971 0.70995
LDF 100 0.39200 0.24000 0.35654 0.19567 0.47712 0.34641 0.70896
INFLO 42 0.36800 0.21000 0.30593 0.13241 0.47396 0.34245 0.66409
INFLO 83 0.35200 0.19000 0.32769 0.15961 0.50129 0.37661 0.69344
INFLO 95 0.36800 0.21000 0.32782 0.15978 0.50402 0.38003 0.69186
COF 91 0.32800 0.16000 0.32831 0.16038 0.44193 0.30241 0.68303
COF 98 0.36000 0.20000 0.33545 0.16931 0.43597 0.29496 0.68927

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO