Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#08)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (55.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.39200 0.24000 0.35406 0.19258 0.46729 0.33411 0.73130
KNN 2 0.39200 0.24000 0.36078 0.20098 0.47619 0.34524 0.73594
KNN 26 0.36800 0.21000 0.35032 0.18791 0.49148 0.36436 0.74542
KNN 73 0.36800 0.21000 0.35399 0.19249 0.48804 0.36005 0.74776
KNNW 4 0.40800 0.26000 0.35935 0.19919 0.47146 0.33933 0.73418
KNNW 63 0.39200 0.24000 0.35159 0.18948 0.49357 0.36697 0.74526
KNNW 100 0.36800 0.21000 0.35345 0.19181 0.48426 0.35533 0.74725
LOF 99 0.33600 0.17000 0.31477 0.14347 0.46838 0.33548 0.72470
SimplifiedLOF 88 0.32000 0.15000 0.27828 0.09786 0.40602 0.25752 0.65718
SimplifiedLOF 92 0.31200 0.14000 0.27915 0.09894 0.41327 0.26658 0.65840
SimplifiedLOF 100 0.32000 0.15000 0.28198 0.10247 0.41221 0.26527 0.66382
LoOP 93 0.32000 0.15000 0.27450 0.09313 0.40609 0.25761 0.64704
LoOP 95 0.32800 0.16000 0.27459 0.09323 0.40295 0.25369 0.64880
LoOP 100 0.32000 0.15000 0.27756 0.09695 0.40409 0.25512 0.65342
LDOF 77 0.32800 0.16000 0.26679 0.08349 0.38532 0.23165 0.62498
LDOF 89 0.32000 0.15000 0.26935 0.08669 0.38605 0.23256 0.62936
LDOF 92 0.31200 0.14000 0.26797 0.08496 0.39524 0.24405 0.62696
ODIN 83 0.34560 0.18200 0.29474 0.11843 0.42922 0.28653 0.67623
ODIN 91 0.32571 0.15714 0.29630 0.12037 0.43556 0.29444 0.68059
ODIN 100 0.32800 0.16000 0.29806 0.12258 0.43146 0.28933 0.68501
FastABOD 41 0.46400 0.33000 0.40990 0.26237 0.51163 0.38953 0.76718
FastABOD 86 0.48000 0.35000 0.41396 0.26745 0.50720 0.38401 0.77267
FastABOD 100 0.46400 0.33000 0.41517 0.26896 0.50289 0.37861 0.77363
KDEOS 15 0.26400 0.08000 0.24086 0.05107 0.36794 0.20993 0.57789
KDEOS 24 0.24000 0.05000 0.24161 0.05201 0.37427 0.21784 0.57843
KDEOS 97 0.20000 0.00000 0.23076 0.03845 0.38367 0.22959 0.58712
KDEOS 100 0.20000 0.00000 0.23229 0.04036 0.37938 0.22423 0.58920
LDF 90 0.36000 0.20000 0.33527 0.16909 0.48792 0.35990 0.74258
LDF 97 0.38400 0.23000 0.33777 0.17222 0.48227 0.35284 0.74400
LDF 100 0.37600 0.22000 0.33922 0.17403 0.48157 0.35197 0.74467
INFLO 73 0.33600 0.17000 0.28626 0.10783 0.46934 0.33668 0.66826
INFLO 99 0.31200 0.14000 0.29949 0.12436 0.49676 0.37095 0.69398
INFLO 100 0.31200 0.14000 0.29959 0.12448 0.49565 0.36957 0.69402
COF 93 0.40800 0.26000 0.36396 0.20495 0.47467 0.34333 0.73384
COF 97 0.40800 0.26000 0.37131 0.21414 0.47826 0.34783 0.73827
COF 99 0.39200 0.24000 0.37340 0.21675 0.48753 0.35942 0.73659
COF 100 0.39200 0.24000 0.37347 0.21684 0.48333 0.35417 0.73734

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (55.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 11 0.40000 0.25000 0.33152 0.16440 0.43767 0.29709 0.66918
KNN 71 0.36800 0.21000 0.32691 0.15864 0.43784 0.29730 0.66606
KNNW 15 0.39200 0.24000 0.32221 0.15276 0.42173 0.27716 0.66328
KNNW 71 0.39200 0.24000 0.32487 0.15609 0.43127 0.28908 0.66574
KNNW 74 0.39200 0.24000 0.32518 0.15647 0.43127 0.28908 0.66613
KNNW 83 0.37600 0.22000 0.32545 0.15681 0.42895 0.28619 0.66587
LOF 73 0.36000 0.20000 0.31325 0.14156 0.42809 0.28512 0.67093
LOF 93 0.36000 0.20000 0.32038 0.15047 0.43854 0.29817 0.67906
LOF 99 0.35200 0.19000 0.32055 0.15069 0.44304 0.30380 0.67864
SimplifiedLOF 97 0.32800 0.16000 0.29368 0.11710 0.39286 0.24107 0.63291
SimplifiedLOF 100 0.32800 0.16000 0.29445 0.11806 0.39130 0.23913 0.63379
LoOP 77 0.32800 0.16000 0.27925 0.09906 0.37449 0.21811 0.61824
LoOP 94 0.32000 0.15000 0.28416 0.10520 0.37838 0.22297 0.62779
LoOP 96 0.32000 0.15000 0.28454 0.10567 0.38235 0.22794 0.62653
LoOP 99 0.31200 0.14000 0.28636 0.10796 0.38196 0.22745 0.62779
LDOF 96 0.32800 0.16000 0.29442 0.11802 0.38692 0.23365 0.62512
LDOF 99 0.32800 0.16000 0.29532 0.11915 0.39164 0.23956 0.62589
LDOF 100 0.32000 0.15000 0.29554 0.11942 0.39062 0.23828 0.62579
ODIN 62 0.28000 0.10000 0.27397 0.09246 0.37940 0.22425 0.60230
ODIN 93 0.34880 0.18600 0.28050 0.10062 0.37073 0.21341 0.60716
ODIN 96 0.34000 0.17500 0.28276 0.10345 0.37193 0.21491 0.60899
ODIN 97 0.33600 0.17000 0.28375 0.10469 0.37299 0.21624 0.60820
FastABOD 72 0.42400 0.28000 0.34908 0.18635 0.45349 0.31686 0.69755
FastABOD 92 0.42400 0.28000 0.35080 0.18850 0.46043 0.32554 0.69934
FastABOD 97 0.42400 0.28000 0.35051 0.18814 0.46209 0.32762 0.69938
FastABOD 99 0.42400 0.28000 0.35056 0.18820 0.46209 0.32762 0.69955
KDEOS 5 0.22400 0.03000 0.20542 0.00677 0.34022 0.17528 0.51722
KDEOS 82 0.21600 0.02000 0.22229 0.02786 0.37113 0.21392 0.56518
KDEOS 98 0.20000 0.00000 0.22907 0.03633 0.36975 0.21218 0.57285
KDEOS 100 0.20000 0.00000 0.22887 0.03609 0.36975 0.21218 0.57304
LDF 85 0.37600 0.22000 0.33098 0.16372 0.44898 0.31122 0.69342
LDF 95 0.38400 0.23000 0.33132 0.16415 0.46110 0.32637 0.69067
LDF 96 0.39200 0.24000 0.33144 0.16430 0.45714 0.32143 0.69075
LDF 99 0.39200 0.24000 0.33150 0.16437 0.45455 0.31818 0.68994
INFLO 80 0.32000 0.15000 0.31578 0.14472 0.50928 0.38660 0.68072
INFLO 87 0.33600 0.17000 0.31463 0.14329 0.50526 0.38158 0.68162
INFLO 93 0.32800 0.16000 0.31365 0.14206 0.50938 0.38673 0.67538
INFLO 99 0.34400 0.18000 0.31180 0.13975 0.50135 0.37668 0.66760
COF 58 0.32000 0.15000 0.29655 0.12069 0.39516 0.24395 0.65298
COF 91 0.29600 0.12000 0.31078 0.13848 0.43454 0.29318 0.67431
COF 100 0.32000 0.15000 0.32205 0.15257 0.42826 0.28532 0.68014

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO