Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (20% of outliers version#10)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (52.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.50820 0.38525 0.57941 0.47426 0.52713 0.40891 0.76374
KNN 2 0.52459 0.40574 0.57350 0.46688 0.53333 0.41667 0.75702
KNN 5 0.52459 0.40574 0.57635 0.47044 0.54400 0.43000 0.75578
KNN 42 0.47541 0.34426 0.59008 0.48759 0.51899 0.39873 0.75773
KNNW 1 0.50820 0.38525 0.58822 0.48527 0.52941 0.41176 0.77402
KNNW 2 0.54098 0.42623 0.58618 0.48273 0.54400 0.43000 0.76955
LOF 39 0.52459 0.40574 0.57067 0.46334 0.53333 0.41667 0.75712
LOF 42 0.52459 0.40574 0.57040 0.46300 0.54412 0.43015 0.75914
LOF 100 0.49180 0.36475 0.57799 0.47249 0.51908 0.39885 0.75363
SimplifiedLOF 44 0.54098 0.42623 0.56366 0.45458 0.54098 0.42623 0.76095
SimplifiedLOF 69 0.50820 0.38525 0.56946 0.46182 0.53901 0.42376 0.76350
SimplifiedLOF 100 0.52459 0.40574 0.57806 0.47258 0.52893 0.41116 0.75994
LoOP 52 0.54098 0.42623 0.56264 0.45331 0.54098 0.42623 0.75870
LoOP 57 0.52459 0.40574 0.56553 0.45691 0.53623 0.42029 0.76310
LoOP 100 0.50820 0.38525 0.57304 0.46630 0.52713 0.40891 0.75679
LDOF 70 0.50820 0.38525 0.55103 0.43879 0.52414 0.40517 0.75873
LDOF 73 0.49180 0.36475 0.55464 0.44330 0.52857 0.41071 0.75759
LDOF 100 0.50820 0.38525 0.56218 0.45272 0.52632 0.40789 0.75356
ODIN 83 0.50820 0.38525 0.48869 0.36087 0.51282 0.39103 0.74866
ODIN 94 0.50000 0.37500 0.51447 0.39309 0.51908 0.39885 0.75024
ODIN 98 0.50820 0.38525 0.51853 0.39816 0.51908 0.39885 0.75296
ODIN 99 0.50820 0.38525 0.51827 0.39784 0.51908 0.39885 0.75302
FastABOD 18 0.49180 0.36475 0.52481 0.40602 0.49180 0.36475 0.74348
FastABOD 74 0.49180 0.36475 0.55422 0.44277 0.51724 0.39655 0.75524
FastABOD 83 0.49180 0.36475 0.55400 0.44250 0.52174 0.40217 0.75396
FastABOD 96 0.49180 0.36475 0.56201 0.45251 0.51282 0.39103 0.75316
KDEOS 18 0.32787 0.15984 0.30599 0.13249 0.44172 0.30215 0.66649
KDEOS 25 0.39344 0.24180 0.29916 0.12395 0.44920 0.31150 0.68174
KDEOS 26 0.37705 0.22131 0.30121 0.12652 0.46154 0.32692 0.68463
LDF 47 0.49180 0.36475 0.43416 0.29270 0.50420 0.38025 0.75605
LDF 77 0.52459 0.40574 0.56789 0.45987 0.54400 0.43000 0.75094
LDF 79 0.52459 0.40574 0.54371 0.42963 0.54701 0.43376 0.75531
INFLO 53 0.54098 0.42623 0.57488 0.46859 0.54545 0.43182 0.76001
INFLO 92 0.50820 0.38525 0.58101 0.47626 0.53448 0.41810 0.77036
INFLO 100 0.50820 0.38525 0.58109 0.47637 0.53913 0.42391 0.76035
COF 2 0.55738 0.44672 0.52708 0.40886 0.57391 0.46739 0.74153
COF 6 0.47541 0.34426 0.50018 0.37523 0.48529 0.35662 0.75383

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (52.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 5 0.52459 0.40574 0.54795 0.43493 0.56637 0.45796 0.75783
KNN 6 0.55738 0.44672 0.54601 0.43251 0.57143 0.46429 0.75632
KNNW 7 0.52459 0.40574 0.54229 0.42786 0.56637 0.45796 0.74563
KNNW 13 0.52459 0.40574 0.54610 0.43262 0.56140 0.45175 0.75074
KNNW 24 0.55738 0.44672 0.53901 0.42376 0.56140 0.45175 0.75034
KNNW 93 0.54098 0.42623 0.53565 0.41956 0.55000 0.43750 0.75195
LOF 36 0.52459 0.40574 0.53238 0.41548 0.55172 0.43966 0.74718
LOF 66 0.54098 0.42623 0.53648 0.42061 0.54545 0.43182 0.75202
LOF 94 0.52459 0.40574 0.54084 0.42605 0.53571 0.41964 0.75625
LOF 95 0.52459 0.40574 0.53893 0.42366 0.53659 0.42073 0.75685
SimplifiedLOF 17 0.49180 0.36475 0.52755 0.40944 0.56296 0.45370 0.74295
SimplifiedLOF 20 0.50820 0.38525 0.53996 0.42495 0.56296 0.45370 0.74194
SimplifiedLOF 95 0.54098 0.42623 0.53513 0.41891 0.54098 0.42623 0.74879
SimplifiedLOF 97 0.54098 0.42623 0.53587 0.41984 0.54098 0.42623 0.74960
LoOP 17 0.47541 0.34426 0.51438 0.39297 0.56296 0.45370 0.74100
LoOP 50 0.52459 0.40574 0.52879 0.41098 0.54701 0.43376 0.74160
LoOP 55 0.52459 0.40574 0.53341 0.41676 0.54701 0.43376 0.73985
LoOP 92 0.54098 0.42623 0.53295 0.41618 0.54098 0.42623 0.73949
LDOF 38 0.49180 0.36475 0.53384 0.41730 0.56522 0.45652 0.74321
LDOF 56 0.54098 0.42623 0.53772 0.42215 0.55000 0.43750 0.74557
LDOF 63 0.52459 0.40574 0.54047 0.42559 0.55385 0.44231 0.74745
LDOF 65 0.52459 0.40574 0.54076 0.42595 0.54701 0.43376 0.74610
ODIN 47 0.52787 0.40984 0.48282 0.35353 0.53782 0.42227 0.74197
ODIN 97 0.52459 0.40574 0.49525 0.36907 0.55118 0.43898 0.74899
ODIN 98 0.52459 0.40574 0.50213 0.37766 0.55118 0.43898 0.74950
ODIN 100 0.51913 0.39891 0.50824 0.38530 0.55118 0.43898 0.74926
FastABOD 7 0.52459 0.40574 0.46601 0.33251 0.52459 0.40574 0.71674
FastABOD 44 0.50820 0.38525 0.49481 0.36851 0.56693 0.45866 0.74597
FastABOD 72 0.50820 0.38525 0.50508 0.38135 0.54545 0.43182 0.74516
FastABOD 96 0.52459 0.40574 0.50225 0.37782 0.53659 0.42073 0.74792
KDEOS 92 0.40984 0.26230 0.36464 0.20580 0.51220 0.39024 0.70908
KDEOS 93 0.44262 0.30328 0.36514 0.20643 0.51220 0.39024 0.71090
KDEOS 94 0.44262 0.30328 0.36976 0.21220 0.51220 0.39024 0.71264
LDF 3 0.40984 0.26230 0.34063 0.17579 0.45872 0.32339 0.63521
LDF 43 0.32787 0.15984 0.38051 0.22563 0.45614 0.32018 0.70619
LDF 52 0.31148 0.13934 0.40044 0.25055 0.46328 0.32910 0.70156
LDF 77 0.36066 0.20082 0.42004 0.27505 0.44776 0.30970 0.68140
INFLO 22 0.50820 0.38525 0.51876 0.39845 0.54795 0.43493 0.73992
INFLO 28 0.52459 0.40574 0.52738 0.40922 0.53691 0.42114 0.75457
INFLO 61 0.50820 0.38525 0.53838 0.42297 0.53226 0.41532 0.76767
COF 12 0.44262 0.30328 0.52411 0.40514 0.50000 0.37500 0.73186
COF 29 0.49180 0.36475 0.49870 0.37338 0.49655 0.37069 0.73529
COF 52 0.47541 0.34426 0.49199 0.36499 0.52414 0.40517 0.74442
COF 55 0.45902 0.32377 0.48905 0.36131 0.53061 0.41327 0.74006

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO