Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (10% of outliers version#08)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 271 objects, 27 outliers (9.96%)

Download raw algorithm results (2.4 MB) Download raw algorithm evaluation table (48.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40741 0.34183 0.47586 0.41786 0.47619 0.41823 0.80191
KNN 3 0.40741 0.34183 0.47189 0.41345 0.48780 0.43113 0.80358
KNN 8 0.40741 0.34183 0.47838 0.42066 0.51163 0.45759 0.79645
KNN 11 0.40741 0.34183 0.48323 0.42605 0.48889 0.43233 0.79387
KNNW 2 0.44444 0.38297 0.47032 0.41171 0.45455 0.39419 0.80009
KNNW 3 0.40741 0.34183 0.47730 0.41946 0.47619 0.41823 0.80298
KNNW 49 0.40741 0.34183 0.48322 0.42603 0.51163 0.45759 0.78992
KNNW 59 0.40741 0.34183 0.48328 0.42610 0.51163 0.45759 0.78962
LOF 6 0.40741 0.34183 0.42549 0.36192 0.45614 0.39596 0.83713
LOF 11 0.40741 0.34183 0.47206 0.41365 0.45361 0.39315 0.84350
LOF 27 0.40741 0.34183 0.48223 0.42494 0.46667 0.40765 0.82195
LOF 82 0.40741 0.34183 0.47929 0.42167 0.52381 0.47112 0.78370
SimplifiedLOF 9 0.44444 0.38297 0.45617 0.39600 0.46753 0.40861 0.84699
SimplifiedLOF 36 0.40741 0.34183 0.49599 0.44022 0.46575 0.40664 0.83212
SimplifiedLOF 90 0.40741 0.34183 0.48449 0.42744 0.50000 0.44467 0.80237
LoOP 9 0.44444 0.38297 0.44432 0.38283 0.45283 0.39228 0.84366
LoOP 39 0.40741 0.34183 0.49572 0.43992 0.46154 0.40195 0.83030
LoOP 78 0.40741 0.34183 0.48998 0.43355 0.50000 0.44467 0.80973
LDOF 35 0.40741 0.34183 0.48284 0.42561 0.43836 0.37621 0.84183
LDOF 41 0.44444 0.38297 0.48378 0.42665 0.45283 0.39228 0.84001
LDOF 52 0.44444 0.38297 0.47599 0.41800 0.48485 0.42784 0.83075
LDOF 74 0.40741 0.34183 0.49180 0.43557 0.45833 0.39839 0.81906
ODIN 37 0.34259 0.26985 0.28893 0.21024 0.43333 0.37063 0.81368
ODIN 71 0.43210 0.36926 0.34342 0.27077 0.45070 0.38992 0.80889
ODIN 99 0.40741 0.34183 0.35687 0.28571 0.44444 0.38297 0.79645
FastABOD 9 0.40741 0.34183 0.40189 0.33571 0.41509 0.35037 0.77520
FastABOD 16 0.40741 0.34183 0.45040 0.38959 0.47826 0.42053 0.80161
FastABOD 17 0.40741 0.34183 0.46763 0.40872 0.48889 0.43233 0.79569
FastABOD 63 0.40741 0.34183 0.49376 0.43774 0.48889 0.43233 0.80131
KDEOS 11 0.33333 0.25956 0.24302 0.15926 0.36735 0.29734 0.73027
KDEOS 16 0.33333 0.25956 0.26417 0.18275 0.35294 0.28134 0.74970
KDEOS 90 0.25926 0.17729 0.24884 0.16572 0.37705 0.30812 0.78886
KDEOS 96 0.25926 0.17729 0.25295 0.17029 0.39216 0.32490 0.78537
LDF 54 0.37037 0.30070 0.42407 0.36034 0.42105 0.35699 0.74833
LDF 68 0.40741 0.34183 0.40620 0.34049 0.40741 0.34183 0.77019
LDF 69 0.40741 0.34183 0.40687 0.34124 0.43137 0.36845 0.77216
LDF 71 0.40741 0.34183 0.40047 0.33413 0.43478 0.37224 0.76472
INFLO 6 0.48148 0.42410 0.38942 0.32185 0.49057 0.43419 0.82165
INFLO 8 0.44444 0.38297 0.41434 0.34954 0.47368 0.41544 0.84199
INFLO 74 0.40741 0.34183 0.49822 0.44269 0.50000 0.44467 0.81436
INFLO 96 0.40741 0.34183 0.49199 0.43577 0.51163 0.45759 0.81239
COF 4 0.44444 0.38297 0.44557 0.38422 0.50000 0.44467 0.81709
COF 6 0.37037 0.30070 0.44851 0.38748 0.45161 0.39093 0.82574
COF 12 0.33333 0.25956 0.46031 0.40059 0.45070 0.38992 0.79979

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 271 objects, 27 outliers (9.96%)

Download raw algorithm results (2.4 MB) Download raw algorithm evaluation table (48.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.29630 0.21843 0.39340 0.32628 0.41026 0.34500 0.79880
KNN 3 0.33333 0.25956 0.40766 0.34211 0.43077 0.36778 0.79174
KNN 6 0.29630 0.21843 0.41488 0.35013 0.40625 0.34055 0.78172
KNN 81 0.37037 0.30070 0.37805 0.30922 0.38095 0.31245 0.74362
KNNW 1 0.37037 0.30070 0.42927 0.36612 0.43038 0.36735 0.79470
KNNW 4 0.29630 0.21843 0.42244 0.35853 0.41026 0.34500 0.80146
LOF 3 0.33333 0.25956 0.35303 0.28144 0.42105 0.35699 0.81633
LOF 5 0.33333 0.25956 0.42062 0.35651 0.40000 0.33361 0.80783
LOF 8 0.33333 0.25956 0.41383 0.34896 0.42105 0.35699 0.82498
LOF 11 0.33333 0.25956 0.39994 0.33355 0.44444 0.38297 0.82362
SimplifiedLOF 4 0.37037 0.30070 0.38170 0.31329 0.40000 0.33361 0.82256
SimplifiedLOF 10 0.33333 0.25956 0.42880 0.36559 0.43243 0.36963 0.83030
SimplifiedLOF 17 0.37037 0.30070 0.40941 0.34406 0.45946 0.39965 0.82529
LoOP 6 0.40741 0.34183 0.36830 0.29840 0.40741 0.34183 0.81724
LoOP 12 0.37037 0.30070 0.42199 0.35803 0.43243 0.36963 0.82574
LoOP 14 0.33333 0.25956 0.40392 0.33796 0.43243 0.36963 0.82726
LoOP 17 0.33333 0.25956 0.40729 0.34171 0.45070 0.38992 0.82590
LDOF 6 0.40741 0.34183 0.35321 0.28164 0.43636 0.37399 0.82256
LDOF 22 0.40741 0.34183 0.41773 0.35329 0.47727 0.41943 0.84107
LDOF 23 0.40741 0.34183 0.42567 0.36212 0.48276 0.42552 0.84001
ODIN 13 0.28704 0.20814 0.27710 0.19711 0.43243 0.36963 0.81117
ODIN 21 0.45679 0.39668 0.33063 0.25656 0.48276 0.42552 0.80442
ODIN 24 0.48148 0.42410 0.32815 0.25381 0.48148 0.42410 0.79766
ODIN 87 0.33333 0.25956 0.39271 0.32551 0.40000 0.33361 0.78051
FastABOD 6 0.37037 0.30070 0.40413 0.33820 0.46512 0.40593 0.80829
FastABOD 7 0.37037 0.30070 0.42206 0.35811 0.47619 0.41823 0.80707
FastABOD 8 0.40741 0.34183 0.42290 0.35904 0.46914 0.41039 0.80237
KDEOS 15 0.37037 0.30070 0.26461 0.18323 0.38462 0.31652 0.78643
KDEOS 91 0.33333 0.25956 0.34595 0.27358 0.43182 0.36895 0.80039
KDEOS 92 0.33333 0.25956 0.33534 0.26180 0.44186 0.38010 0.79994
KDEOS 94 0.33333 0.25956 0.33715 0.26380 0.44186 0.38010 0.80404
LDF 5 0.29630 0.21843 0.18898 0.09924 0.32787 0.25349 0.61658
LDF 100 0.22222 0.13616 0.29173 0.21335 0.29167 0.21329 0.68443
INFLO 10 0.40741 0.34183 0.39976 0.33333 0.42254 0.35864 0.81603
INFLO 16 0.33333 0.25956 0.40604 0.34032 0.44737 0.38622 0.81011
INFLO 17 0.33333 0.25956 0.40696 0.34134 0.43590 0.37348 0.81026
INFLO 21 0.33333 0.25956 0.40040 0.33405 0.43011 0.36705 0.82210
COF 1 0.33333 0.25956 0.33739 0.26407 0.39437 0.32735 0.73991
COF 3 0.25926 0.17729 0.29397 0.21585 0.38462 0.31652 0.77201
COF 5 0.37037 0.30070 0.36874 0.29889 0.37037 0.30070 0.75349
COF 10 0.33333 0.25956 0.37969 0.31104 0.37838 0.30959 0.76321

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO