Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

HeartDisease (2% of outliers version#02)

A data set containing medical data on heart problems. Affected patients are considered outliers and healthy people are considered inliers.

Download all data set variants used (92.9 kB). You can also access the original data. (heart.dat)

Normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (20.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.66000 0.80952 0.80571 0.80000 0.79600 0.99111
KNNW 5 0.33333 0.32000 0.64286 0.63571 0.60000 0.59200 0.98667
KNNW 7 0.66667 0.66000 0.66667 0.66000 0.66667 0.66000 0.98444
LOF 21 0.66667 0.66000 0.60819 0.60035 0.66667 0.66000 0.96222
LOF 26 0.66667 0.66000 0.65556 0.64867 0.66667 0.66000 0.98222
SimplifiedLOF 3 0.33333 0.32000 0.15333 0.13640 0.33333 0.32000 0.82889
SimplifiedLOF 39 0.33333 0.32000 0.53333 0.52400 0.50000 0.49000 0.96667
SimplifiedLOF 49 0.33333 0.32000 0.51667 0.50700 0.50000 0.49000 0.97111
SimplifiedLOF 77 0.33333 0.32000 0.34444 0.33133 0.57143 0.56286 0.96444
LoOP 5 0.33333 0.32000 0.16999 0.15339 0.33333 0.32000 0.88000
LoOP 39 0.33333 0.32000 0.55758 0.54873 0.50000 0.49000 0.97556
LoOP 47 0.33333 0.32000 0.55556 0.54667 0.50000 0.49000 0.97778
LoOP 76 0.33333 0.32000 0.34028 0.32708 0.57143 0.56286 0.96222
LDOF 24 0.33333 0.32000 0.45079 0.43981 0.50000 0.49000 0.89556
LDOF 41 0.66667 0.66000 0.43889 0.42767 0.66667 0.66000 0.95778
LDOF 98 0.33333 0.32000 0.33535 0.32206 0.50000 0.49000 0.97111
ODIN 18 0.40000 0.38800 0.34524 0.33214 0.40000 0.38800 0.97333
ODIN 73 0.33333 0.32000 0.52222 0.51267 0.50000 0.49000 0.96111
ODIN 79 0.33333 0.32000 0.36667 0.35400 0.57143 0.56286 0.93556
FastABOD 5 0.66667 0.66000 0.65556 0.64867 0.66667 0.66000 0.98222
FastABOD 6 0.66667 0.66000 0.83333 0.83000 0.80000 0.79600 0.99333
KDEOS 46 0.33333 0.32000 0.19206 0.17590 0.33333 0.32000 0.90889
KDEOS 76 0.00000 -0.02000 0.24444 0.22933 0.44444 0.43333 0.95556
KDEOS 92 0.33333 0.32000 0.47857 0.46814 0.50000 0.49000 0.95111
LDF 2 0.33333 0.32000 0.51266 0.50291 0.57143 0.56286 0.82667
LDF 20 0.33333 0.32000 0.55556 0.54667 0.50000 0.49000 0.97778
INFLO 5 0.33333 0.32000 0.17670 0.16024 0.33333 0.32000 0.89333
INFLO 43 0.33333 0.32000 0.54444 0.53533 0.50000 0.49000 0.97556
INFLO 50 0.33333 0.32000 0.40000 0.38800 0.57143 0.56286 0.96667
COF 7 0.66667 0.66000 0.46032 0.44952 0.66667 0.66000 0.97111
COF 19 0.33333 0.32000 0.60952 0.60171 0.60000 0.59200 0.98444

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (23.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.00000 -0.02000 0.16176 0.14499 0.28571 0.27143 0.91333
KNN 4 0.33333 0.32000 0.18254 0.16619 0.33333 0.32000 0.88222
KNN 9 0.33333 0.32000 0.18889 0.17267 0.33333 0.32000 0.88000
KNNW 5 0.00000 -0.02000 0.14271 0.12556 0.28571 0.27143 0.90889
KNNW 8 0.33333 0.32000 0.19444 0.17833 0.33333 0.32000 0.90000
LOF 9 0.33333 0.32000 0.16358 0.14685 0.33333 0.32000 0.86667
LOF 15 0.33333 0.32000 0.19298 0.17684 0.33333 0.32000 0.89556
LOF 20 0.33333 0.32000 0.19299 0.17685 0.33333 0.32000 0.87778
SimplifiedLOF 14 0.33333 0.32000 0.21136 0.19559 0.33333 0.32000 0.93333
LoOP 16 0.33333 0.32000 0.21806 0.20242 0.33333 0.32000 0.93778
LoOP 17 0.33333 0.32000 0.22540 0.20990 0.33333 0.32000 0.94222
LoOP 35 0.33333 0.32000 0.24426 0.22915 0.40000 0.38800 0.89556
LoOP 43 0.33333 0.32000 0.24949 0.23448 0.40000 0.38800 0.88444
LDOF 17 0.33333 0.32000 0.19556 0.17947 0.33333 0.32000 0.91778
LDOF 24 0.33333 0.32000 0.21160 0.19583 0.33333 0.32000 0.93333
LDOF 35 0.33333 0.32000 0.23958 0.22438 0.40000 0.38800 0.90222
LDOF 37 0.33333 0.32000 0.24554 0.23045 0.40000 0.38800 0.90667
ODIN 7 0.00000 -0.02000 0.19369 0.17757 0.36364 0.35091 0.91667
ODIN 9 0.00000 -0.02000 0.20833 0.19250 0.36364 0.35091 0.94444
ODIN 11 0.00000 -0.02000 0.17965 0.16325 0.35294 0.34000 0.95000
ODIN 74 0.33333 0.32000 0.19834 0.18231 0.33333 0.32000 0.81889
FastABOD 3 0.00000 -0.02000 0.07299 0.05445 0.16667 0.15000 0.82889
FastABOD 11 0.00000 -0.02000 0.17311 0.15657 0.30769 0.29385 0.93778
FastABOD 44 0.00000 -0.02000 0.16296 0.14622 0.33333 0.32000 0.91333
KDEOS 11 0.33333 0.32000 0.26587 0.25119 0.40000 0.38800 0.85111
KDEOS 17 0.33333 0.32000 0.36881 0.35619 0.50000 0.49000 0.79556
KDEOS 30 0.33333 0.32000 0.42810 0.41667 0.50000 0.49000 0.93333
KDEOS 49 0.33333 0.32000 0.24864 0.23361 0.37500 0.36250 0.95333
LDF 5 0.33333 0.32000 0.19795 0.18190 0.33333 0.32000 0.92222
LDF 13 0.33333 0.32000 0.20337 0.18743 0.33333 0.32000 0.86444
INFLO 10 0.00000 -0.02000 0.19762 0.18157 0.33333 0.32000 0.94000
INFLO 11 0.33333 0.32000 0.21755 0.20190 0.33333 0.32000 0.93778
INFLO 35 0.33333 0.32000 0.24603 0.23095 0.40000 0.38800 0.88889
INFLO 37 0.33333 0.32000 0.24854 0.23351 0.40000 0.38800 0.89778
COF 12 0.00000 -0.02000 0.14282 0.12567 0.30769 0.29385 0.91000
COF 40 0.33333 0.32000 0.20213 0.18618 0.33333 0.32000 0.85556
COF 47 0.33333 0.32000 0.29532 0.28123 0.44444 0.43333 0.86889

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO