Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (5% of outliers version#08)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 256 objects, 12 outliers (4.69%)

Download raw algorithm results (2.3 MB) Download raw algorithm evaluation table (42.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.21311 0.26640 0.23032 0.37500 0.34426 0.78313
KNN 3 0.33333 0.30055 0.27232 0.23653 0.37500 0.34426 0.79628
KNN 11 0.33333 0.30055 0.31568 0.28203 0.37500 0.34426 0.80669
KNN 44 0.25000 0.21311 0.34839 0.31634 0.37500 0.34426 0.80089
KNNW 1 0.25000 0.21311 0.37545 0.34473 0.40000 0.37049 0.80413
LOF 2 0.33333 0.30055 0.19474 0.15513 0.33333 0.30055 0.74317
LOF 6 0.33333 0.30055 0.24622 0.20914 0.40000 0.37049 0.81284
LOF 14 0.33333 0.30055 0.26100 0.22465 0.40000 0.37049 0.83607
LOF 70 0.33333 0.30055 0.33138 0.29850 0.35294 0.32112 0.79781
SimplifiedLOF 6 0.33333 0.30055 0.23345 0.19575 0.37037 0.33940 0.79064
SimplifiedLOF 15 0.33333 0.30055 0.26626 0.23017 0.40000 0.37049 0.81899
SimplifiedLOF 32 0.25000 0.21311 0.25532 0.21870 0.30000 0.26557 0.83367
SimplifiedLOF 73 0.25000 0.21311 0.33209 0.29924 0.33333 0.30055 0.82445
LoOP 6 0.33333 0.30055 0.22761 0.18962 0.37037 0.33940 0.78518
LoOP 16 0.33333 0.30055 0.25647 0.21990 0.40000 0.37049 0.81250
LoOP 20 0.33333 0.30055 0.25418 0.21750 0.36364 0.33234 0.83231
LoOP 72 0.33333 0.30055 0.33620 0.30356 0.34783 0.31575 0.82514
LDOF 13 0.33333 0.30055 0.24276 0.20552 0.38462 0.35435 0.78552
LDOF 14 0.33333 0.30055 0.25031 0.21344 0.40000 0.37049 0.79986
LDOF 54 0.25000 0.21311 0.24035 0.20299 0.30435 0.27014 0.83265
LDOF 81 0.25000 0.21311 0.31123 0.27735 0.30000 0.26557 0.82172
ODIN 28 0.25000 0.21311 0.18023 0.13992 0.31579 0.28214 0.81814
ODIN 72 0.33333 0.30055 0.22642 0.18838 0.34783 0.31575 0.80277
ODIN 76 0.33333 0.30055 0.24078 0.20344 0.38095 0.35051 0.80294
ODIN 99 0.33333 0.30055 0.26753 0.23151 0.38095 0.35051 0.79713
FastABOD 5 0.33333 0.30055 0.19166 0.15190 0.34783 0.31575 0.74010
FastABOD 10 0.33333 0.30055 0.20208 0.16283 0.38095 0.35051 0.80669
FastABOD 67 0.33333 0.30055 0.30442 0.27021 0.42105 0.39258 0.79372
FastABOD 84 0.33333 0.30055 0.32522 0.29203 0.40000 0.37049 0.79064
KDEOS 15 0.16667 0.12568 0.08899 0.04419 0.23077 0.19294 0.64652
KDEOS 77 0.08333 0.03825 0.11722 0.07381 0.21739 0.17890 0.77049
KDEOS 80 0.08333 0.03825 0.11305 0.06943 0.24096 0.20363 0.77015
LDF 2 0.25000 0.21311 0.26251 0.22624 0.33333 0.30055 0.83060
LDF 64 0.25000 0.21311 0.24977 0.21287 0.35294 0.32112 0.79098
LDF 96 0.25000 0.21311 0.29667 0.26208 0.33333 0.30055 0.74351
INFLO 5 0.33333 0.30055 0.29337 0.25861 0.37037 0.33940 0.79577
INFLO 14 0.33333 0.30055 0.25625 0.21967 0.40000 0.37049 0.81831
INFLO 20 0.33333 0.30055 0.26158 0.22527 0.36364 0.33234 0.83675
INFLO 72 0.33333 0.30055 0.33120 0.29831 0.34783 0.31575 0.82070
COF 2 0.41667 0.38798 0.25327 0.21655 0.41667 0.38798 0.70697
COF 8 0.33333 0.30055 0.33628 0.30364 0.44444 0.41712 0.81216
COF 11 0.33333 0.30055 0.37221 0.34134 0.42105 0.39258 0.82992

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 256 objects, 12 outliers (4.69%)

Download raw algorithm results (2.3 MB) Download raw algorithm evaluation table (44.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.21311 0.22183 0.18356 0.33333 0.30055 0.80635
KNN 2 0.25000 0.21311 0.23094 0.19312 0.34043 0.30799 0.81557
KNN 4 0.25000 0.21311 0.22722 0.18922 0.35897 0.32745 0.80430
KNNW 1 0.16667 0.12568 0.22370 0.18552 0.34286 0.31054 0.83231
KNNW 2 0.25000 0.21311 0.21819 0.17974 0.32558 0.29241 0.81967
KNNW 3 0.25000 0.21311 0.22334 0.18515 0.35000 0.31803 0.81899
KNNW 4 0.25000 0.21311 0.22496 0.18684 0.35000 0.31803 0.81592
LOF 3 0.25000 0.21311 0.18700 0.14702 0.32000 0.28656 0.81728
LOF 6 0.25000 0.21311 0.21587 0.17731 0.37838 0.34781 0.84187
LOF 7 0.25000 0.21311 0.22964 0.19175 0.38095 0.35051 0.83743
LOF 10 0.25000 0.21311 0.24520 0.20807 0.36364 0.33234 0.82684
SimplifiedLOF 6 0.25000 0.21311 0.21258 0.17385 0.37838 0.34781 0.85417
SimplifiedLOF 10 0.25000 0.21311 0.25206 0.21527 0.36364 0.33234 0.83197
SimplifiedLOF 11 0.33333 0.30055 0.24273 0.20548 0.36364 0.33234 0.83675
LoOP 1 0.25000 0.21311 0.13506 0.09252 0.28571 0.25059 0.65813
LoOP 6 0.16667 0.12568 0.21070 0.17188 0.38889 0.35883 0.84495
LoOP 12 0.25000 0.21311 0.23613 0.19856 0.34783 0.31575 0.83641
LDOF 12 0.25000 0.21311 0.23594 0.19836 0.40000 0.37049 0.85007
LDOF 17 0.33333 0.30055 0.25688 0.22033 0.42857 0.40047 0.83982
LDOF 20 0.41667 0.38798 0.25435 0.21768 0.41667 0.38798 0.82343
ODIN 27 0.33333 0.30055 0.21332 0.17463 0.36364 0.33234 0.81233
ODIN 40 0.27778 0.24226 0.20393 0.16478 0.34483 0.31261 0.82548
ODIN 55 0.20833 0.16940 0.22336 0.18517 0.32432 0.29109 0.80789
FastABOD 4 0.25000 0.21311 0.15859 0.11721 0.29630 0.26169 0.74146
FastABOD 5 0.16667 0.12568 0.19250 0.15278 0.37838 0.34781 0.76776
FastABOD 11 0.25000 0.21311 0.23835 0.20090 0.37500 0.34426 0.79303
FastABOD 37 0.25000 0.21311 0.21114 0.17235 0.29508 0.26041 0.80020
KDEOS 7 0.16667 0.12568 0.09754 0.05316 0.18605 0.14602 0.68784
KDEOS 19 0.16667 0.12568 0.19430 0.15467 0.23077 0.19294 0.74419
KDEOS 84 0.00000 -0.04918 0.11684 0.07340 0.27273 0.23696 0.78620
KDEOS 90 0.16667 0.12568 0.13621 0.09373 0.27273 0.23696 0.79918
LDF 2 0.25000 0.21311 0.14811 0.10622 0.27027 0.23438 0.74863
LDF 3 0.16667 0.12568 0.12783 0.08493 0.28571 0.25059 0.75956
LDF 96 0.08333 0.03825 0.15496 0.11340 0.17978 0.13944 0.66974
INFLO 9 0.25000 0.21311 0.19906 0.15967 0.28571 0.25059 0.80362
INFLO 12 0.25000 0.21311 0.21525 0.17665 0.31579 0.28214 0.83231
INFLO 48 0.25000 0.21311 0.23460 0.19696 0.35294 0.32112 0.81523
INFLO 52 0.25000 0.21311 0.23673 0.19919 0.35294 0.32112 0.82070
COF 4 0.25000 0.21311 0.23174 0.19396 0.36364 0.33234 0.86236
COF 8 0.50000 0.47541 0.29458 0.25989 0.50000 0.47541 0.83436

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO