Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (5% of outliers version#06)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 256 objects, 12 outliers (4.69%)

Download raw algorithm results (2.3 MB) Download raw algorithm evaluation table (42.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 3 0.33333 0.30055 0.39212 0.36223 0.46154 0.43506 0.70236
KNN 10 0.50000 0.47541 0.42389 0.39555 0.52174 0.49822 0.68887
KNN 48 0.50000 0.47541 0.42854 0.40044 0.52174 0.49822 0.68921
KNNW 7 0.33333 0.30055 0.38872 0.35865 0.44444 0.41712 0.69399
KNNW 33 0.41667 0.38798 0.40455 0.37527 0.44444 0.41712 0.68613
KNNW 45 0.41667 0.38798 0.40949 0.38045 0.48000 0.45443 0.68545
KNNW 89 0.41667 0.38798 0.41036 0.38136 0.46154 0.43506 0.68579
LOF 6 0.25000 0.21311 0.33865 0.30612 0.37500 0.34426 0.75342
LOF 8 0.41667 0.38798 0.35119 0.31928 0.42857 0.40047 0.74522
LOF 12 0.41667 0.38798 0.39793 0.36832 0.48000 0.45443 0.72609
LOF 100 0.41667 0.38798 0.40998 0.38097 0.48000 0.45443 0.68204
SimplifiedLOF 11 0.41667 0.38798 0.39264 0.36277 0.44444 0.41712 0.74078
SimplifiedLOF 14 0.41667 0.38798 0.38330 0.35297 0.45455 0.42772 0.74556
SimplifiedLOF 23 0.33333 0.30055 0.40845 0.37935 0.47059 0.44455 0.73122
SimplifiedLOF 25 0.33333 0.30055 0.41029 0.38129 0.47059 0.44455 0.72951
LoOP 10 0.50000 0.47541 0.40091 0.37145 0.50000 0.47541 0.74010
LoOP 14 0.41667 0.38798 0.40373 0.37441 0.45455 0.42772 0.74351
LoOP 21 0.33333 0.30055 0.41101 0.38204 0.47059 0.44455 0.73668
LDOF 4 0.33333 0.30055 0.29748 0.26293 0.34783 0.31575 0.71824
LDOF 19 0.33333 0.30055 0.37128 0.34036 0.40000 0.37049 0.76469
LDOF 68 0.33333 0.30055 0.40667 0.37749 0.47059 0.44455 0.73156
ODIN 6 0.14286 0.10070 0.11280 0.06917 0.22951 0.19162 0.72729
ODIN 55 0.26667 0.23060 0.17714 0.13667 0.38710 0.35695 0.69826
ODIN 67 0.35714 0.32553 0.21597 0.17741 0.36364 0.33234 0.69911
ODIN 100 0.33333 0.30055 0.27037 0.23449 0.38095 0.35051 0.69416
FastABOD 13 0.50000 0.47541 0.40830 0.37920 0.50000 0.47541 0.67008
FastABOD 68 0.33333 0.30055 0.36916 0.33814 0.40000 0.37049 0.70253
KDEOS 13 0.00000 -0.04918 0.11403 0.07046 0.27027 0.23438 0.69911
KDEOS 20 0.16667 0.12568 0.11094 0.06722 0.20896 0.17005 0.72917
KDEOS 91 0.00000 -0.04918 0.11761 0.07421 0.25806 0.22158 0.69672
LDF 51 0.33333 0.30055 0.23892 0.20149 0.42105 0.39258 0.70116
LDF 88 0.41667 0.38798 0.32684 0.29374 0.41667 0.38798 0.65027
LDF 90 0.41667 0.38798 0.38642 0.35624 0.47059 0.44455 0.65301
LDF 91 0.41667 0.38798 0.37327 0.34245 0.47619 0.45043 0.65847
INFLO 10 0.50000 0.47541 0.38636 0.35619 0.50000 0.47541 0.72473
INFLO 12 0.41667 0.38798 0.42457 0.39627 0.47619 0.45043 0.72883
INFLO 19 0.33333 0.30055 0.40432 0.37502 0.47059 0.44455 0.74727
COF 6 0.33333 0.30055 0.28643 0.25134 0.38095 0.35051 0.70765
COF 8 0.33333 0.30055 0.32919 0.29619 0.42105 0.39258 0.70765
COF 9 0.33333 0.30055 0.32586 0.29271 0.40000 0.37049 0.72063
COF 10 0.33333 0.30055 0.33186 0.29900 0.42105 0.39258 0.71585

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 256 objects, 12 outliers (4.69%)

Download raw algorithm results (2.3 MB) Download raw algorithm evaluation table (40.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.41667 0.38798 0.35433 0.32257 0.47619 0.45043 0.64122
KNN 2 0.41667 0.38798 0.35564 0.32395 0.45455 0.42772 0.65164
KNN 57 0.33333 0.30055 0.34031 0.30787 0.44444 0.41712 0.68545
KNNW 1 0.41667 0.38798 0.39408 0.36428 0.43478 0.40699 0.63132
KNNW 2 0.41667 0.38798 0.35224 0.32039 0.47619 0.45043 0.63559
KNNW 97 0.33333 0.30055 0.33817 0.30563 0.44444 0.41712 0.67657
LOF 2 0.41667 0.38798 0.41141 0.38246 0.47619 0.45043 0.66257
LOF 5 0.41667 0.38798 0.42674 0.39855 0.52632 0.50302 0.69296
LOF 6 0.41667 0.38798 0.42642 0.39821 0.52632 0.50302 0.69365
SimplifiedLOF 1 0.41667 0.38798 0.36505 0.33382 0.45455 0.42772 0.66308
SimplifiedLOF 2 0.41667 0.38798 0.41521 0.38645 0.52632 0.50302 0.67316
SimplifiedLOF 6 0.41667 0.38798 0.43158 0.40363 0.52632 0.50302 0.67896
SimplifiedLOF 17 0.41667 0.38798 0.36609 0.33492 0.50000 0.47541 0.70663
LoOP 1 0.41667 0.38798 0.36505 0.33382 0.45455 0.42772 0.66308
LoOP 2 0.41667 0.38798 0.41521 0.38645 0.52632 0.50302 0.67794
LoOP 6 0.41667 0.38798 0.42966 0.40161 0.52632 0.50302 0.67606
LoOP 19 0.41667 0.38798 0.37348 0.34267 0.52632 0.50302 0.70594
LDOF 3 0.33333 0.30055 0.31807 0.28453 0.40000 0.37049 0.73805
LDOF 4 0.41667 0.38798 0.38485 0.35460 0.44444 0.41712 0.71107
LDOF 6 0.41667 0.38798 0.41395 0.38513 0.52632 0.50302 0.71858
ODIN 17 0.33333 0.30055 0.17907 0.13870 0.33333 0.30055 0.69570
ODIN 41 0.50000 0.47541 0.34582 0.31365 0.50000 0.47541 0.66752
ODIN 45 0.50000 0.47541 0.33399 0.30123 0.54545 0.52310 0.66769
ODIN 59 0.41667 0.38798 0.36796 0.33688 0.52632 0.50302 0.67333
FastABOD 4 0.33333 0.30055 0.32618 0.29304 0.37500 0.34426 0.66803
FastABOD 12 0.25000 0.21311 0.33654 0.30391 0.37500 0.34426 0.68238
FastABOD 25 0.33333 0.30055 0.35465 0.32291 0.40000 0.37049 0.66974
FastABOD 57 0.33333 0.30055 0.32387 0.29062 0.42105 0.39258 0.65505
KDEOS 10 0.25000 0.21311 0.14851 0.10663 0.32258 0.28926 0.66940
KDEOS 18 0.16667 0.12568 0.13954 0.09722 0.30769 0.27364 0.70731
KDEOS 39 0.25000 0.21311 0.25439 0.21772 0.28571 0.25059 0.65745
LDF 64 0.25000 0.21311 0.29525 0.26059 0.37500 0.34426 0.68306
LDF 67 0.25000 0.21311 0.31417 0.28044 0.40000 0.37049 0.63490
LDF 72 0.25000 0.21311 0.32523 0.29204 0.40000 0.37049 0.65061
LDF 99 0.33333 0.30055 0.32038 0.28696 0.40000 0.37049 0.63832
INFLO 2 0.41667 0.38798 0.42893 0.40084 0.52632 0.50302 0.72883
INFLO 7 0.41667 0.38798 0.37802 0.34743 0.50000 0.47541 0.75854
COF 3 0.50000 0.47541 0.43500 0.40721 0.57143 0.55035 0.66359
COF 4 0.41667 0.38798 0.39202 0.36212 0.47619 0.45043 0.68204

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO