Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#06)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (55.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.35200 0.19000 0.32864 0.16080 0.45026 0.31283 0.71408
KNN 12 0.35200 0.19000 0.32209 0.15261 0.45742 0.32178 0.71929
KNN 14 0.36000 0.20000 0.31951 0.14939 0.46269 0.32836 0.71661
KNN 75 0.32000 0.15000 0.32405 0.15507 0.47596 0.34495 0.71736
KNNW 1 0.36000 0.20000 0.31983 0.14979 0.42735 0.28419 0.66279
KNNW 68 0.32800 0.16000 0.32225 0.15281 0.46883 0.33603 0.71709
KNNW 98 0.31200 0.14000 0.32474 0.15592 0.46455 0.33068 0.71832
KNNW 100 0.31200 0.14000 0.32501 0.15627 0.46535 0.33168 0.71824
LOF 77 0.33600 0.17000 0.29380 0.11725 0.43006 0.28758 0.68661
LOF 98 0.33600 0.17000 0.30228 0.12785 0.43836 0.29795 0.69872
LOF 100 0.32800 0.16000 0.30394 0.12993 0.43678 0.29598 0.70016
SimplifiedLOF 25 0.28800 0.11000 0.24183 0.05229 0.38489 0.23112 0.59672
SimplifiedLOF 45 0.33600 0.17000 0.25490 0.06863 0.36491 0.20614 0.60802
SimplifiedLOF 100 0.32000 0.15000 0.26923 0.08653 0.38369 0.22962 0.63616
LoOP 25 0.28000 0.10000 0.23677 0.04597 0.37818 0.22273 0.58963
LoOP 55 0.33600 0.17000 0.25344 0.06680 0.36364 0.20455 0.60505
LoOP 100 0.32000 0.15000 0.26360 0.07950 0.37736 0.22170 0.62226
LDOF 2 0.26400 0.08000 0.25587 0.06984 0.36960 0.21200 0.59378
LDOF 28 0.27200 0.09000 0.23967 0.04959 0.37168 0.21460 0.57982
LDOF 59 0.31200 0.14000 0.25476 0.06845 0.36713 0.20892 0.60570
LDOF 62 0.34400 0.18000 0.25407 0.06758 0.36850 0.21063 0.60493
ODIN 96 0.33600 0.17000 0.27736 0.09670 0.41057 0.26321 0.65976
ODIN 100 0.33600 0.17000 0.27934 0.09917 0.41379 0.26724 0.66367
FastABOD 49 0.44000 0.30000 0.37079 0.21349 0.47215 0.34019 0.73962
FastABOD 74 0.43200 0.29000 0.37113 0.21391 0.47541 0.34426 0.74366
FastABOD 97 0.43200 0.29000 0.37415 0.21768 0.47059 0.33824 0.74530
FastABOD 100 0.43200 0.29000 0.37386 0.21732 0.46933 0.33667 0.74570
KDEOS 9 0.23200 0.04000 0.23339 0.04173 0.35216 0.19020 0.53803
KDEOS 11 0.25600 0.07000 0.22734 0.03418 0.34171 0.17714 0.53571
KDEOS 38 0.20000 0.00000 0.21785 0.02232 0.37063 0.21329 0.54384
KDEOS 99 0.21600 0.02000 0.22108 0.02635 0.36229 0.20287 0.57166
LDF 65 0.36000 0.20000 0.31421 0.14276 0.44571 0.30714 0.70542
LDF 97 0.33600 0.17000 0.32577 0.15721 0.46683 0.33354 0.72264
LDF 99 0.33600 0.17000 0.32584 0.15730 0.47202 0.34002 0.72235
LDF 100 0.33600 0.17000 0.32625 0.15781 0.47174 0.33968 0.72242
INFLO 55 0.35200 0.19000 0.26615 0.08269 0.43975 0.29968 0.62638
INFLO 99 0.32000 0.15000 0.28307 0.10384 0.48366 0.35458 0.66942
COF 91 0.40800 0.26000 0.36416 0.20520 0.46688 0.33360 0.72552
COF 92 0.42400 0.28000 0.36044 0.20055 0.45714 0.32143 0.72414
COF 100 0.40800 0.26000 0.37748 0.22185 0.45665 0.32081 0.73384

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (55.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 14 0.40800 0.26000 0.32581 0.15726 0.43243 0.29054 0.67545
KNN 72 0.36800 0.21000 0.32741 0.15926 0.44385 0.30481 0.68804
KNN 81 0.36800 0.21000 0.32833 0.16041 0.45697 0.32122 0.68788
KNN 85 0.36000 0.20000 0.32835 0.16044 0.45481 0.31851 0.68786
KNNW 29 0.40800 0.26000 0.31836 0.14795 0.42466 0.28082 0.67240
KNNW 97 0.37600 0.22000 0.32536 0.15669 0.43769 0.29711 0.68438
KNNW 98 0.37600 0.22000 0.32536 0.15670 0.43769 0.29711 0.68450
KNNW 100 0.37600 0.22000 0.32548 0.15685 0.43769 0.29711 0.68448
LOF 74 0.36800 0.21000 0.31173 0.13966 0.46302 0.32878 0.69416
LOF 85 0.37600 0.22000 0.31548 0.14436 0.45317 0.31647 0.70010
LOF 96 0.36800 0.21000 0.32108 0.15135 0.45806 0.32258 0.70490
LOF 99 0.37600 0.22000 0.32172 0.15215 0.45513 0.31891 0.70373
SimplifiedLOF 79 0.33600 0.17000 0.28069 0.10086 0.39140 0.23925 0.63093
SimplifiedLOF 97 0.33600 0.17000 0.28860 0.11075 0.40767 0.25959 0.64275
SimplifiedLOF 100 0.33600 0.17000 0.28826 0.11033 0.40876 0.26095 0.64326
LoOP 88 0.33600 0.17000 0.27281 0.09101 0.38617 0.23271 0.62171
LoOP 99 0.33600 0.17000 0.27778 0.09723 0.39306 0.24133 0.63150
LoOP 100 0.33600 0.17000 0.27768 0.09710 0.39420 0.24275 0.63102
LDOF 73 0.32800 0.16000 0.26670 0.08338 0.38484 0.23105 0.61162
LDOF 99 0.32800 0.16000 0.27979 0.09974 0.39326 0.24157 0.62562
ODIN 87 0.34400 0.18000 0.27292 0.09115 0.39713 0.24641 0.62779
ODIN 99 0.33600 0.17000 0.28026 0.10033 0.40220 0.25275 0.63405
ODIN 100 0.33120 0.16400 0.27896 0.09870 0.40346 0.25432 0.63368
FastABOD 20 0.39200 0.24000 0.32884 0.16105 0.42651 0.28314 0.67934
FastABOD 79 0.38400 0.23000 0.33649 0.17061 0.44373 0.30466 0.68901
FastABOD 99 0.38400 0.23000 0.33879 0.17349 0.44373 0.30466 0.69171
KDEOS 3 0.18400 -0.02000 0.21879 0.02349 0.33438 0.16798 0.50125
KDEOS 66 0.15200 -0.06000 0.20587 0.00734 0.37736 0.22170 0.55090
KDEOS 94 0.20800 0.01000 0.21674 0.02093 0.36735 0.20918 0.57174
KDEOS 100 0.20800 0.01000 0.21838 0.02298 0.37088 0.21360 0.57622
LDF 89 0.38400 0.23000 0.33517 0.16896 0.46281 0.32851 0.71837
LDF 90 0.39200 0.24000 0.33523 0.16904 0.46537 0.33172 0.71734
LDF 91 0.39200 0.24000 0.33558 0.16948 0.46797 0.33496 0.71782
LDF 94 0.39200 0.24000 0.33589 0.16986 0.46591 0.33239 0.71747
INFLO 47 0.35200 0.19000 0.28549 0.10686 0.48101 0.35127 0.64893
INFLO 83 0.32800 0.16000 0.29997 0.12496 0.50262 0.37827 0.66627
INFLO 96 0.32800 0.16000 0.30102 0.12628 0.50820 0.38525 0.65550
INFLO 100 0.33600 0.17000 0.30181 0.12726 0.50538 0.38172 0.65989
COF 64 0.34400 0.18000 0.28301 0.10377 0.40000 0.25000 0.64976
COF 94 0.31200 0.14000 0.30704 0.13380 0.45000 0.31250 0.69221
COF 98 0.33600 0.17000 0.31375 0.14219 0.44886 0.31108 0.69721
COF 99 0.33600 0.17000 0.31407 0.14259 0.44321 0.30402 0.69675

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO