Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Arrhythmia (2% of outliers version#09)

Data set contains patient records classified as normal or as exhibiting some type of cardiac arrhythmia. In total, there are 14 types of arrhythmia and 1 type that brings together all the other different types. However, 3 types of arrhythmia have no data. Again, we treat healthy people as inliers and patients suffering from arrhythmia as outliers.

Download all data set variants used (9.2 MB). You can also access the original data. (arrhythmia.data)

Normalized, without duplicates

This version contains 259 attributes, 248 objects, 4 outliers (1.61%)

Download raw algorithm results (2.2 MB) Download raw algorithm evaluation table (33.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.23770 0.29008 0.27844 0.40000 0.39016 0.75051
KNN 97 0.25000 0.23770 0.28831 0.27664 0.40000 0.39016 0.76537
KNNW 1 0.25000 0.23770 0.28469 0.27296 0.40000 0.39016 0.74693
KNNW 3 0.25000 0.23770 0.28791 0.27624 0.40000 0.39016 0.74488
KNNW 10 0.25000 0.23770 0.28713 0.27545 0.40000 0.39016 0.75717
LOF 1 0.25000 0.23770 0.29704 0.28551 0.40000 0.39016 0.69416
LOF 5 0.25000 0.23770 0.29670 0.28517 0.40000 0.39016 0.78586
SimplifiedLOF 1 0.25000 0.23770 0.32627 0.31522 0.40000 0.39016 0.78381
LoOP 1 0.25000 0.23770 0.32627 0.31522 0.40000 0.39016 0.78381
LDOF 2 0.25000 0.23770 0.27726 0.26542 0.40000 0.39016 0.65881
LDOF 4 0.25000 0.23770 0.31147 0.30019 0.40000 0.39016 0.75000
LDOF 89 0.25000 0.23770 0.29653 0.28500 0.40000 0.39016 0.77459
ODIN 18 0.12000 0.10557 0.09667 0.08186 0.20690 0.19389 0.81814
ODIN 75 0.25000 0.23770 0.10596 0.09130 0.25000 0.23770 0.78381
ODIN 79 0.25000 0.23770 0.12480 0.11045 0.28571 0.27400 0.78330
ODIN 91 0.25000 0.23770 0.12871 0.11443 0.28571 0.27400 0.79047
FastABOD 3 0.25000 0.23770 0.26665 0.25463 0.40000 0.39016 0.58094
FastABOD 12 0.25000 0.23770 0.30200 0.29055 0.40000 0.39016 0.76127
FastABOD 62 0.25000 0.23770 0.28575 0.27404 0.40000 0.39016 0.76947
KDEOS 2 0.00000 -0.01639 0.01882 0.00274 0.04545 0.02981 0.52305
KDEOS 7 0.00000 -0.01639 0.07776 0.06264 0.22222 0.20947 0.74795
KDEOS 11 0.00000 -0.01639 0.09735 0.08255 0.20000 0.18689 0.76127
KDEOS 98 0.00000 -0.01639 0.08801 0.07306 0.21053 0.19758 0.77152
LDF 59 0.25000 0.23770 0.11371 0.09918 0.25000 0.23770 0.74283
LDF 76 0.25000 0.23770 0.29819 0.28668 0.40000 0.39016 0.84119
LDF 98 0.25000 0.23770 0.37224 0.36195 0.40000 0.39016 0.91803
INFLO 1 0.25000 0.23770 0.32196 0.31084 0.40000 0.39016 0.77254
INFLO 98 0.25000 0.23770 0.29319 0.28160 0.40000 0.39016 0.79406
COF 1 0.25000 0.23770 0.32627 0.31522 0.40000 0.39016 0.78381

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 259 attributes, 248 objects, 4 outliers (1.61%)

Download raw algorithm results (2.2 MB) Download raw algorithm evaluation table (33.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.25000 0.23770 0.33410 0.32318 0.40000 0.39016 0.76025
KNN 86 0.25000 0.23770 0.34486 0.33412 0.40000 0.39016 0.84939
KNNW 1 0.25000 0.23770 0.33294 0.32200 0.40000 0.39016 0.79867
KNNW 4 0.25000 0.23770 0.33398 0.32306 0.40000 0.39016 0.78484
KNNW 89 0.25000 0.23770 0.32192 0.31081 0.40000 0.39016 0.83299
LOF 1 0.50000 0.49180 0.43267 0.42337 0.57143 0.56440 0.78279
LOF 82 0.25000 0.23770 0.30925 0.29792 0.40000 0.39016 0.84119
SimplifiedLOF 1 0.50000 0.49180 0.38518 0.37510 0.50000 0.49180 0.66855
SimplifiedLOF 2 0.50000 0.49180 0.39897 0.38912 0.50000 0.49180 0.72439
SimplifiedLOF 86 0.25000 0.23770 0.31138 0.30010 0.40000 0.39016 0.82480
LoOP 1 0.50000 0.49180 0.38518 0.37510 0.50000 0.49180 0.66855
LoOP 4 0.50000 0.49180 0.40323 0.39344 0.50000 0.49180 0.74027
LoOP 85 0.25000 0.23770 0.31131 0.30002 0.40000 0.39016 0.82377
LDOF 4 0.50000 0.49180 0.44178 0.43263 0.57143 0.56440 0.77664
LDOF 84 0.25000 0.23770 0.30982 0.29850 0.40000 0.39016 0.81250
ODIN 51 0.30000 0.28852 0.16537 0.15169 0.33333 0.32240 0.80943
ODIN 99 0.25000 0.23770 0.14176 0.12769 0.28571 0.27400 0.83607
FastABOD 3 0.50000 0.49180 0.38896 0.37894 0.50000 0.49180 0.74898
FastABOD 6 0.50000 0.49180 0.39701 0.38712 0.50000 0.49180 0.83094
FastABOD 96 0.25000 0.23770 0.30036 0.28889 0.40000 0.39016 0.84119
KDEOS 6 0.25000 0.23770 0.19625 0.18307 0.33333 0.32240 0.72643
KDEOS 85 0.00000 -0.01639 0.07673 0.06160 0.15385 0.13997 0.79201
LDF 28 0.00000 -0.01639 0.07011 0.05487 0.16000 0.14623 0.84939
LDF 81 0.50000 0.49180 0.38450 0.37441 0.50000 0.49180 0.62295
LDF 90 0.50000 0.49180 0.51206 0.50406 0.66667 0.66120 0.71107
INFLO 1 0.25000 0.23770 0.36402 0.35359 0.44444 0.43534 0.67930
INFLO 5 0.25000 0.23770 0.36825 0.35789 0.44444 0.43534 0.76434
INFLO 86 0.25000 0.23770 0.31142 0.30013 0.40000 0.39016 0.83914
COF 1 0.50000 0.49180 0.38518 0.37510 0.50000 0.49180 0.66855
COF 3 0.25000 0.23770 0.41301 0.40339 0.44444 0.43534 0.79816
COF 45 0.25000 0.23770 0.29179 0.28018 0.40000 0.39016 0.84016

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO