Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (20% of outliers version#03)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (54.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 4 0.49180 0.36475 0.53046 0.41307 0.49587 0.36983 0.77358
KNN 8 0.49180 0.36475 0.53196 0.41496 0.50000 0.37500 0.77385
KNN 32 0.45902 0.32377 0.53776 0.42220 0.48718 0.35897 0.76455
KNN 74 0.45902 0.32377 0.53138 0.41423 0.51534 0.39417 0.75974
KNNW 1 0.47541 0.34426 0.53723 0.42154 0.52273 0.40341 0.77798
KNNW 12 0.50820 0.38525 0.53157 0.41447 0.50820 0.38525 0.77412
LOF 3 0.42623 0.28279 0.47864 0.34830 0.48780 0.35976 0.76451
LOF 9 0.37705 0.22131 0.48379 0.35474 0.51136 0.38920 0.76035
LOF 44 0.47541 0.34426 0.51105 0.38881 0.48649 0.35811 0.75732
LOF 97 0.45902 0.32377 0.52316 0.40395 0.50000 0.37500 0.75847
SimplifiedLOF 4 0.40984 0.26230 0.46797 0.33496 0.50888 0.38609 0.75410
SimplifiedLOF 66 0.49180 0.36475 0.50884 0.38604 0.49600 0.37000 0.76236
SimplifiedLOF 70 0.49180 0.36475 0.51348 0.39185 0.50407 0.38008 0.76539
SimplifiedLOF 88 0.49180 0.36475 0.52023 0.40029 0.50000 0.37500 0.76337
LoOP 3 0.45902 0.32377 0.45512 0.31891 0.50867 0.38584 0.75094
LoOP 70 0.49180 0.36475 0.51091 0.38864 0.50000 0.37500 0.76256
LoOP 73 0.49180 0.36475 0.51513 0.39392 0.50794 0.38492 0.76361
LoOP 91 0.49180 0.36475 0.51619 0.39524 0.50000 0.37500 0.75726
LDOF 88 0.45902 0.32377 0.50122 0.37652 0.49231 0.36538 0.75679
LDOF 94 0.45902 0.32377 0.50433 0.38041 0.49612 0.37016 0.75860
LDOF 97 0.45902 0.32377 0.50542 0.38177 0.50394 0.37992 0.75853
LDOF 100 0.45902 0.32377 0.50722 0.38403 0.50394 0.37992 0.75813
ODIN 90 0.45902 0.32377 0.49046 0.36308 0.51136 0.38920 0.75588
ODIN 98 0.46448 0.33060 0.49926 0.37407 0.50286 0.37857 0.75779
FastABOD 5 0.42623 0.28279 0.48828 0.36035 0.49315 0.36644 0.75114
FastABOD 58 0.47541 0.34426 0.50984 0.38731 0.47761 0.34701 0.75363
FastABOD 94 0.45902 0.32377 0.51666 0.39583 0.48855 0.36069 0.75961
FastABOD 100 0.44262 0.30328 0.51803 0.39753 0.48855 0.36069 0.75907
KDEOS 11 0.32787 0.15984 0.31150 0.13938 0.44025 0.30031 0.69296
KDEOS 20 0.31148 0.13934 0.32990 0.16238 0.42927 0.28659 0.69168
KDEOS 99 0.26230 0.07787 0.30563 0.13204 0.47000 0.33750 0.69854
LDF 27 0.39344 0.24180 0.43189 0.28986 0.50323 0.37903 0.75222
LDF 44 0.49180 0.36475 0.45709 0.32136 0.49180 0.36475 0.75349
LDF 45 0.44262 0.30328 0.47046 0.33808 0.49351 0.36688 0.76364
LDF 79 0.44262 0.30328 0.53776 0.42221 0.48980 0.36224 0.74409
INFLO 7 0.36066 0.20082 0.48494 0.35617 0.51220 0.39024 0.76552
INFLO 8 0.37705 0.22131 0.48844 0.36055 0.50617 0.38272 0.76713
INFLO 71 0.49180 0.36475 0.51215 0.39019 0.49206 0.36508 0.75611
INFLO 92 0.45902 0.32377 0.51731 0.39664 0.48855 0.36069 0.75873
COF 3 0.47541 0.34426 0.47746 0.34682 0.51111 0.38889 0.76969
COF 43 0.42623 0.28279 0.50770 0.38463 0.49032 0.36290 0.73475

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 305 objects, 61 outliers (20.00%)

Download raw algorithm results (2.7 MB) Download raw algorithm evaluation table (54.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.45902 0.32377 0.48899 0.36123 0.47154 0.33943 0.72302
KNN 4 0.44262 0.30328 0.48354 0.35443 0.49524 0.36905 0.72642
KNN 7 0.44262 0.30328 0.47886 0.34858 0.47706 0.34633 0.73206
KNNW 1 0.44262 0.30328 0.48441 0.35551 0.45833 0.32292 0.72084
KNNW 2 0.44262 0.30328 0.48668 0.35835 0.46400 0.33000 0.72413
KNNW 6 0.44262 0.30328 0.48382 0.35478 0.48649 0.35811 0.72393
KNNW 17 0.44262 0.30328 0.48086 0.35107 0.48148 0.35185 0.72776
LOF 4 0.42623 0.28279 0.46793 0.33492 0.47407 0.34259 0.70210
LOF 5 0.42623 0.28279 0.46788 0.33485 0.47742 0.34677 0.70546
LOF 7 0.45902 0.32377 0.46211 0.32764 0.47154 0.33943 0.71661
LOF 21 0.45902 0.32377 0.45944 0.32431 0.46429 0.33036 0.72071
SimplifiedLOF 5 0.44262 0.30328 0.48313 0.35391 0.47059 0.33824 0.73240
SimplifiedLOF 14 0.45902 0.32377 0.47287 0.34109 0.46667 0.33333 0.73569
SimplifiedLOF 21 0.45902 0.32377 0.46774 0.33467 0.47205 0.34006 0.73824
SimplifiedLOF 22 0.45902 0.32377 0.46664 0.33330 0.47799 0.34748 0.73791
LoOP 5 0.42623 0.28279 0.47953 0.34941 0.46988 0.33735 0.73092
LoOP 14 0.45902 0.32377 0.46995 0.33744 0.47059 0.33824 0.73555
LoOP 20 0.45902 0.32377 0.46401 0.33002 0.47134 0.33917 0.73851
LoOP 23 0.45902 0.32377 0.46716 0.33395 0.47799 0.34748 0.73747
LDOF 5 0.47541 0.34426 0.46680 0.33350 0.48387 0.35484 0.73361
LDOF 13 0.37705 0.22131 0.48289 0.35361 0.47423 0.34278 0.74086
LDOF 14 0.39344 0.24180 0.47381 0.34226 0.46857 0.33571 0.74093
ODIN 18 0.45537 0.31922 0.44133 0.30166 0.48571 0.35714 0.73018
ODIN 47 0.41530 0.26913 0.46471 0.33089 0.45714 0.32143 0.71832
FastABOD 5 0.45902 0.32377 0.49412 0.36766 0.49091 0.36364 0.72030
FastABOD 6 0.47541 0.34426 0.49403 0.36754 0.47826 0.34783 0.72091
FastABOD 10 0.44262 0.30328 0.46923 0.33654 0.46667 0.33333 0.72823
KDEOS 14 0.39344 0.24180 0.42431 0.28039 0.45662 0.32078 0.72003
KDEOS 15 0.40984 0.26230 0.41783 0.27229 0.45192 0.31490 0.72413
KDEOS 86 0.32787 0.15984 0.32323 0.15403 0.47619 0.34524 0.69692
LDF 9 0.40984 0.26230 0.36402 0.20503 0.41600 0.27000 0.66367
LDF 11 0.39344 0.24180 0.41543 0.26929 0.44872 0.31090 0.67872
INFLO 2 0.45902 0.32377 0.44984 0.31229 0.47059 0.33824 0.71506
INFLO 6 0.44262 0.30328 0.47917 0.34896 0.48921 0.36151 0.74876
INFLO 7 0.39344 0.24180 0.47522 0.34403 0.46154 0.32692 0.75155
COF 5 0.39344 0.24180 0.46406 0.33007 0.45517 0.31897 0.70499
COF 9 0.37705 0.22131 0.45056 0.31320 0.47368 0.34211 0.68510
COF 30 0.42623 0.28279 0.44149 0.30186 0.46729 0.33411 0.68550

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO