Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (20% of outliers version#07)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (54.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.47541 0.34426 0.53224 0.41530 0.50382 0.37977 0.74463
KNN 17 0.47541 0.34426 0.54163 0.42703 0.52174 0.40217 0.75064
KNN 29 0.47541 0.34426 0.54391 0.42989 0.51534 0.39417 0.75292
KNN 30 0.47541 0.34426 0.54294 0.42867 0.51220 0.39024 0.75400
KNNW 3 0.47541 0.34426 0.53104 0.41380 0.49624 0.37030 0.74577
KNNW 36 0.45902 0.32377 0.54098 0.42623 0.50617 0.38272 0.74946
KNNW 46 0.45902 0.32377 0.53975 0.42469 0.50617 0.38272 0.75013
KNNW 56 0.45902 0.32377 0.53930 0.42412 0.51220 0.39024 0.74980
LOF 36 0.45902 0.32377 0.51957 0.39946 0.49689 0.37112 0.74127
LOF 79 0.45902 0.32377 0.53339 0.41673 0.50331 0.37914 0.74798
LOF 89 0.45902 0.32377 0.53310 0.41637 0.51948 0.39935 0.74664
LOF 95 0.45902 0.32377 0.53556 0.41945 0.51613 0.39516 0.74745
SimplifiedLOF 62 0.49180 0.36475 0.52512 0.40641 0.51190 0.38988 0.74335
SimplifiedLOF 88 0.47541 0.34426 0.52946 0.41183 0.52023 0.40029 0.74711
LoOP 55 0.47541 0.34426 0.52204 0.40255 0.52023 0.40029 0.74546
LoOP 60 0.47541 0.34426 0.52175 0.40219 0.50588 0.38235 0.74607
LoOP 66 0.49180 0.36475 0.52304 0.40380 0.50299 0.37874 0.74274
LoOP 75 0.49180 0.36475 0.52727 0.40909 0.50888 0.38609 0.74503
LDOF 54 0.40984 0.26230 0.50732 0.38415 0.52288 0.40359 0.73327
LDOF 82 0.44262 0.30328 0.51553 0.39441 0.50549 0.38187 0.73952
LDOF 97 0.47541 0.34426 0.51420 0.39276 0.49612 0.37016 0.73744
ODIN 48 0.47541 0.34426 0.38498 0.23122 0.50000 0.37500 0.72460
ODIN 97 0.43716 0.29645 0.46008 0.32510 0.50323 0.37903 0.74066
ODIN 98 0.43716 0.29645 0.46027 0.32533 0.50000 0.37500 0.74133
FastABOD 7 0.45902 0.32377 0.50720 0.38400 0.50340 0.37925 0.74241
FastABOD 39 0.49180 0.36475 0.50998 0.38748 0.49180 0.36475 0.73858
FastABOD 86 0.45902 0.32377 0.52615 0.40769 0.49080 0.36350 0.74288
KDEOS 14 0.32787 0.15984 0.31792 0.14740 0.43564 0.29455 0.67072
KDEOS 21 0.37705 0.22131 0.29584 0.11980 0.41414 0.26768 0.66615
KDEOS 99 0.36066 0.20082 0.30917 0.13646 0.46243 0.32803 0.68127
LDF 55 0.44262 0.30328 0.47372 0.34215 0.53846 0.42308 0.76001
LDF 72 0.49180 0.36475 0.54946 0.43683 0.52632 0.40789 0.75322
LDF 73 0.52459 0.40574 0.54328 0.42910 0.55118 0.43898 0.75517
INFLO 28 0.44262 0.30328 0.50656 0.38320 0.51656 0.39570 0.73683
INFLO 57 0.49180 0.36475 0.52175 0.40219 0.49689 0.37112 0.73522
INFLO 99 0.47541 0.34426 0.53130 0.41413 0.51613 0.39516 0.75148
COF 5 0.45902 0.32377 0.50138 0.37673 0.49635 0.37044 0.72978
COF 27 0.49180 0.36475 0.47853 0.34817 0.50450 0.38063 0.71775
COF 30 0.49180 0.36475 0.51218 0.39023 0.50877 0.38596 0.72071
COF 32 0.49180 0.36475 0.50316 0.37895 0.53571 0.41964 0.72306

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (52.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 4 0.45902 0.32377 0.50094 0.37617 0.49296 0.36620 0.71335
KNN 7 0.49180 0.36475 0.49746 0.37182 0.50340 0.37925 0.72696
KNN 15 0.47541 0.34426 0.49056 0.36320 0.51007 0.38758 0.72847
KNN 27 0.47541 0.34426 0.49326 0.36658 0.50685 0.38356 0.73166
KNNW 2 0.45902 0.32377 0.49365 0.36707 0.50746 0.38433 0.70519
KNNW 8 0.47541 0.34426 0.49867 0.37333 0.49645 0.37057 0.71365
KNNW 12 0.47541 0.34426 0.50096 0.37620 0.50000 0.37500 0.71815
KNNW 71 0.47541 0.34426 0.49228 0.36535 0.50340 0.37925 0.72796
LOF 13 0.49180 0.36475 0.48982 0.36228 0.49612 0.37016 0.72158
LOF 46 0.47541 0.34426 0.48131 0.35164 0.50704 0.38380 0.72091
LOF 93 0.45902 0.32377 0.49353 0.36692 0.50000 0.37500 0.72749
SimplifiedLOF 23 0.50820 0.38525 0.49330 0.36662 0.50820 0.38525 0.71977
SimplifiedLOF 24 0.49180 0.36475 0.49566 0.36958 0.50407 0.38008 0.72239
SimplifiedLOF 99 0.47541 0.34426 0.49293 0.36616 0.49573 0.36966 0.72783
LoOP 31 0.50820 0.38525 0.47715 0.34644 0.50820 0.38525 0.72003
LoOP 80 0.47541 0.34426 0.48507 0.35634 0.49682 0.37102 0.72766
LDOF 46 0.50820 0.38525 0.47853 0.34816 0.51613 0.39516 0.71889
LDOF 71 0.47541 0.34426 0.48876 0.36096 0.50350 0.37937 0.72407
LDOF 74 0.47541 0.34426 0.48833 0.36041 0.50370 0.37963 0.72581
ODIN 14 0.44521 0.30651 0.36977 0.21221 0.50725 0.38406 0.72410
ODIN 54 0.43607 0.29508 0.44641 0.30802 0.52778 0.40972 0.71802
ODIN 73 0.47541 0.34426 0.43650 0.29563 0.51429 0.39286 0.72084
ODIN 96 0.45902 0.32377 0.45335 0.31669 0.50350 0.37937 0.72259
FastABOD 5 0.49180 0.36475 0.47998 0.34998 0.49587 0.36983 0.72333
FastABOD 67 0.45902 0.32377 0.48034 0.35042 0.52632 0.40789 0.72944
FastABOD 70 0.45902 0.32377 0.47707 0.34633 0.53435 0.41794 0.72857
FastABOD 96 0.45902 0.32377 0.47691 0.34614 0.51163 0.38953 0.72984
KDEOS 11 0.31148 0.13934 0.33247 0.16559 0.39837 0.24797 0.63424
KDEOS 89 0.36066 0.20082 0.32398 0.15498 0.48718 0.35897 0.68832
KDEOS 90 0.37705 0.22131 0.32452 0.15564 0.48408 0.35510 0.68745
KDEOS 97 0.37705 0.22131 0.32892 0.16115 0.48718 0.35897 0.69202
LDF 3 0.37705 0.22131 0.34340 0.17925 0.40000 0.25000 0.63350
LDF 12 0.36066 0.20082 0.40814 0.26017 0.40278 0.25347 0.66178
LDF 13 0.37705 0.22131 0.41799 0.27249 0.40789 0.25987 0.65191
LDF 85 0.26230 0.07787 0.32408 0.15510 0.42553 0.28191 0.61583
INFLO 20 0.49180 0.36475 0.46546 0.33182 0.51200 0.39000 0.72279
INFLO 33 0.45902 0.32377 0.48570 0.35712 0.51316 0.39145 0.73162
INFLO 63 0.45902 0.32377 0.49374 0.36718 0.50000 0.37500 0.75672
COF 13 0.44262 0.30328 0.47681 0.34602 0.48611 0.35764 0.70351
COF 19 0.44262 0.30328 0.48359 0.35449 0.48366 0.35458 0.69726
COF 71 0.45902 0.32377 0.45054 0.31318 0.46957 0.33696 0.70176

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO