Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#05)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (54.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40800 0.26000 0.37089 0.21362 0.47368 0.34211 0.73850
KNN 17 0.36000 0.20000 0.35004 0.18755 0.50532 0.38165 0.73235
KNNW 1 0.43200 0.29000 0.37846 0.22308 0.46286 0.32857 0.72474
KNNW 2 0.42400 0.28000 0.37990 0.22488 0.47230 0.34038 0.73584
KNNW 6 0.40000 0.25000 0.37138 0.21422 0.47701 0.34626 0.73928
KNNW 33 0.35200 0.19000 0.35343 0.19178 0.50000 0.37500 0.73419
LOF 2 0.35200 0.19000 0.28544 0.10680 0.36879 0.21099 0.58625
LOF 98 0.31200 0.14000 0.31675 0.14593 0.45953 0.32441 0.70539
LOF 100 0.32000 0.15000 0.31804 0.14755 0.45745 0.32181 0.70744
SimplifiedLOF 6 0.29600 0.12000 0.29445 0.11807 0.40443 0.25554 0.64102
SimplifiedLOF 15 0.32800 0.16000 0.27710 0.09637 0.38554 0.23193 0.62197
SimplifiedLOF 100 0.32000 0.15000 0.28553 0.10691 0.39623 0.24528 0.64414
LoOP 6 0.31200 0.14000 0.29344 0.11680 0.39153 0.23942 0.64281
LoOP 7 0.30400 0.13000 0.28568 0.10710 0.40181 0.25226 0.63714
LoOP 89 0.33600 0.17000 0.27996 0.09995 0.39344 0.24180 0.63296
LDOF 9 0.34400 0.18000 0.29057 0.11322 0.38202 0.22753 0.63562
LDOF 10 0.35200 0.19000 0.28712 0.10890 0.38760 0.23450 0.63645
LDOF 89 0.32800 0.16000 0.27509 0.09386 0.39353 0.24191 0.62136
ODIN 37 0.32960 0.16200 0.27806 0.09757 0.37164 0.21455 0.61470
ODIN 100 0.31800 0.14750 0.29474 0.11842 0.42222 0.27778 0.66930
FastABOD 10 0.51200 0.39000 0.42368 0.27960 0.52778 0.40972 0.76502
FastABOD 56 0.47200 0.34000 0.42724 0.28405 0.54019 0.42524 0.77072
FastABOD 72 0.46400 0.33000 0.43048 0.28811 0.53074 0.41343 0.77270
FastABOD 96 0.46400 0.33000 0.42554 0.28192 0.52769 0.40961 0.77344
KDEOS 6 0.26400 0.08000 0.27718 0.09647 0.36559 0.20699 0.59738
KDEOS 7 0.25600 0.07000 0.27515 0.09393 0.38693 0.23367 0.61656
KDEOS 13 0.29600 0.12000 0.25496 0.06871 0.37610 0.22012 0.61242
LDF 72 0.36800 0.21000 0.32766 0.15958 0.46866 0.33583 0.71518
LDF 94 0.32000 0.15000 0.34154 0.17693 0.49620 0.37025 0.73283
LDF 98 0.31200 0.14000 0.34230 0.17787 0.49351 0.36688 0.73349
LDF 100 0.32000 0.15000 0.34220 0.17775 0.49357 0.36697 0.73392
INFLO 3 0.29600 0.12000 0.30430 0.13038 0.39836 0.24795 0.61471
INFLO 61 0.34400 0.18000 0.28209 0.10261 0.43590 0.29487 0.62714
INFLO 96 0.31200 0.14000 0.30117 0.12646 0.47912 0.34890 0.66787
COF 82 0.39200 0.24000 0.35362 0.19202 0.47230 0.34038 0.71978
COF 96 0.38400 0.23000 0.37533 0.21916 0.48619 0.35773 0.74125
COF 99 0.38400 0.23000 0.37417 0.21772 0.49419 0.36773 0.74298
COF 100 0.38400 0.23000 0.37429 0.21786 0.49292 0.36615 0.74562

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (54.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 11 0.42400 0.28000 0.34423 0.18029 0.42570 0.28213 0.65440
KNN 12 0.40800 0.26000 0.34398 0.17998 0.42804 0.28506 0.65706
KNN 83 0.36800 0.21000 0.33440 0.16800 0.44384 0.30479 0.65090
KNNW 20 0.37600 0.22000 0.33739 0.17173 0.41958 0.27448 0.65190
KNNW 68 0.39200 0.24000 0.33635 0.17044 0.42473 0.28091 0.65309
KNNW 80 0.39200 0.24000 0.33586 0.16983 0.42975 0.28719 0.65374
KNNW 92 0.38400 0.23000 0.33511 0.16889 0.43213 0.29017 0.65274
LOF 90 0.36800 0.21000 0.32439 0.15549 0.42612 0.28265 0.66450
LOF 93 0.37600 0.22000 0.32467 0.15583 0.42902 0.28628 0.66350
LOF 100 0.37600 0.22000 0.32591 0.15739 0.42657 0.28322 0.66384
SimplifiedLOF 64 0.33600 0.17000 0.29008 0.11259 0.38389 0.22986 0.60250
SimplifiedLOF 98 0.33600 0.17000 0.29960 0.12450 0.39599 0.24499 0.61696
SimplifiedLOF 99 0.33600 0.17000 0.30018 0.12522 0.39500 0.24375 0.61728
LoOP 70 0.33600 0.17000 0.28544 0.10679 0.37813 0.22267 0.60626
LoOP 75 0.33600 0.17000 0.28857 0.11071 0.38876 0.23595 0.60991
LoOP 93 0.33600 0.17000 0.29080 0.11350 0.38384 0.22980 0.61464
LoOP 100 0.32800 0.16000 0.29032 0.11290 0.38265 0.22832 0.61563
LDOF 83 0.33600 0.17000 0.29310 0.11638 0.38095 0.22619 0.59894
LDOF 89 0.33600 0.17000 0.29516 0.11895 0.38947 0.23684 0.60181
LDOF 98 0.32800 0.16000 0.29749 0.12187 0.38442 0.23052 0.60694
LDOF 99 0.32800 0.16000 0.29794 0.12242 0.38542 0.23177 0.60680
ODIN 85 0.34000 0.17500 0.26534 0.08167 0.35802 0.19753 0.58790
ODIN 99 0.31200 0.14000 0.27184 0.08980 0.36656 0.20820 0.58859
ODIN 100 0.31400 0.14250 0.27145 0.08931 0.36760 0.20950 0.58795
FastABOD 77 0.40000 0.25000 0.36139 0.20174 0.45533 0.31916 0.68709
FastABOD 97 0.40000 0.25000 0.36307 0.20384 0.45748 0.32185 0.68933
FastABOD 100 0.40000 0.25000 0.36332 0.20415 0.45748 0.32185 0.68960
KDEOS 12 0.25600 0.07000 0.19897 -0.00128 0.34254 0.17818 0.50234
KDEOS 93 0.20800 0.01000 0.21912 0.02390 0.37736 0.22170 0.56592
KDEOS 99 0.21600 0.02000 0.22004 0.02505 0.37327 0.21659 0.56744
LDF 82 0.40000 0.25000 0.33921 0.17401 0.45428 0.31785 0.68120
LDF 88 0.38400 0.23000 0.34119 0.17649 0.45732 0.32165 0.68256
LDF 100 0.39200 0.24000 0.34193 0.17741 0.46386 0.32982 0.67757
INFLO 41 0.33600 0.17000 0.29489 0.11862 0.45000 0.31250 0.63861
INFLO 96 0.32000 0.15000 0.31878 0.14847 0.50407 0.38008 0.67080
COF 96 0.32800 0.16000 0.31882 0.14853 0.42328 0.27910 0.66510
COF 99 0.33600 0.17000 0.31889 0.14861 0.42136 0.27671 0.66644
COF 100 0.34400 0.18000 0.31756 0.14695 0.41499 0.26873 0.66451

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO