Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

HeartDisease (2% of outliers version#08)

A data set containing medical data on heart problems. Affected patients are considered outliers and healthy people are considered inliers.

Download all data set variants used (92.9 kB). You can also access the original data. (heart.dat)

Normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (34.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.03303 0.01369 0.07547 0.05698 0.61889
KNN 68 0.00000 -0.02000 0.06928 0.05066 0.14286 0.12571 0.76444
KNN 69 0.00000 -0.02000 0.07101 0.05243 0.15385 0.13692 0.76000
KNN 95 0.00000 -0.02000 0.06763 0.04899 0.19048 0.17429 0.73333
KNNW 1 0.00000 -0.02000 0.03499 0.01569 0.07792 0.05948 0.62778
KNNW 59 0.00000 -0.02000 0.05351 0.03458 0.14286 0.12571 0.72667
KNNW 87 0.00000 -0.02000 0.05371 0.03478 0.13793 0.12069 0.72667
LOF 1 0.00000 -0.02000 0.03073 0.01135 0.08696 0.06870 0.44667
LOF 88 0.00000 -0.02000 0.05616 0.03729 0.13333 0.11600 0.70889
LOF 100 0.00000 -0.02000 0.05605 0.03717 0.14286 0.12571 0.66222
SimplifiedLOF 1 0.00000 -0.02000 0.07476 0.05625 0.22222 0.20667 0.63000
SimplifiedLOF 99 0.00000 -0.02000 0.04940 0.03039 0.11429 0.09657 0.71111
LoOP 1 0.00000 -0.02000 0.07467 0.05616 0.22222 0.20667 0.62778
LoOP 85 0.00000 -0.02000 0.04744 0.02839 0.11765 0.10000 0.70667
LDOF 2 0.00000 -0.02000 0.09669 0.07863 0.25000 0.23500 0.71333
LDOF 4 0.00000 -0.02000 0.06007 0.04127 0.14286 0.12571 0.74222
ODIN 3 0.07692 0.05846 0.04085 0.02167 0.12500 0.10750 0.57333
ODIN 32 0.00000 -0.02000 0.04529 0.02619 0.10811 0.09027 0.71667
ODIN 93 0.00000 -0.02000 0.05236 0.03341 0.13793 0.12069 0.67222
ODIN 97 0.00000 -0.02000 0.05600 0.03712 0.13793 0.12069 0.68889
FastABOD 3 0.00000 -0.02000 0.03971 0.02050 0.09524 0.07714 0.67333
FastABOD 86 0.00000 -0.02000 0.06840 0.04977 0.16000 0.14320 0.79111
FastABOD 88 0.00000 -0.02000 0.06959 0.05099 0.16667 0.15000 0.79111
KDEOS 2 0.00000 -0.02000 0.03297 0.01363 0.06383 0.04511 0.65333
KDEOS 3 0.00000 -0.02000 0.06909 0.05047 0.15385 0.13692 0.71111
KDEOS 7 0.00000 -0.02000 0.09837 0.08034 0.28571 0.27143 0.51333
LDF 1 0.00000 -0.02000 0.02959 0.01018 0.08000 0.06160 0.45333
LDF 14 0.00000 -0.02000 0.09307 0.07493 0.25000 0.23500 0.72222
LDF 17 0.00000 -0.02000 0.09269 0.07454 0.22222 0.20667 0.78889
INFLO 1 0.00000 -0.02000 0.03630 0.01703 0.09091 0.07273 0.64222
INFLO 83 0.00000 -0.02000 0.06514 0.04645 0.14286 0.12571 0.80222
INFLO 97 0.00000 -0.02000 0.06743 0.04877 0.19048 0.17429 0.60444
COF 1 0.00000 -0.02000 0.07476 0.05625 0.22222 0.20667 0.63000
COF 92 0.00000 -0.02000 0.06092 0.04214 0.13333 0.11600 0.80000

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 13 attributes, 153 objects, 3 outliers (1.96%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (34.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.02527 0.00578 0.05405 0.03514 0.49556
KNN 85 0.00000 -0.02000 0.05383 0.03491 0.11765 0.10000 0.77333
KNN 92 0.00000 -0.02000 0.05335 0.03441 0.12766 0.11021 0.77111
KNNW 1 0.00000 -0.02000 0.02975 0.01034 0.06383 0.04511 0.56778
KNNW 83 0.00000 -0.02000 0.04327 0.02414 0.10169 0.08373 0.71556
KNNW 95 0.00000 -0.02000 0.04385 0.02473 0.10000 0.08200 0.72222
LOF 1 0.00000 -0.02000 0.02938 0.00997 0.05714 0.03829 0.55333
LOF 88 0.00000 -0.02000 0.05244 0.03348 0.12766 0.11021 0.76667
LOF 91 0.00000 -0.02000 0.05335 0.03441 0.12766 0.11021 0.77111
SimplifiedLOF 1 0.00000 -0.02000 0.02995 0.01055 0.07273 0.05418 0.58000
SimplifiedLOF 91 0.00000 -0.02000 0.03905 0.01983 0.09677 0.07871 0.68667
SimplifiedLOF 99 0.00000 -0.02000 0.04044 0.02125 0.09677 0.07871 0.69778
LoOP 1 0.00000 -0.02000 0.02977 0.01037 0.07273 0.05418 0.57556
LoOP 94 0.00000 -0.02000 0.03746 0.01821 0.09677 0.07871 0.67111
LoOP 99 0.00000 -0.02000 0.03857 0.01934 0.09677 0.07871 0.68222
LDOF 2 0.00000 -0.02000 0.03091 0.01153 0.06452 0.04581 0.56222
LDOF 8 0.00000 -0.02000 0.05630 0.03742 0.13636 0.11909 0.78667
ODIN 1 0.01961 0.00000 0.02408 0.00456 0.05128 0.03231 0.58667
ODIN 83 0.00000 -0.02000 0.05271 0.03376 0.10526 0.08737 0.72667
ODIN 95 0.00000 -0.02000 0.04586 0.02677 0.10909 0.09127 0.74000
ODIN 100 0.00000 -0.02000 0.04609 0.02701 0.11538 0.09769 0.73778
FastABOD 3 0.00000 -0.02000 0.02733 0.00788 0.05505 0.03615 0.52889
FastABOD 13 0.00000 -0.02000 0.05827 0.03944 0.14286 0.12571 0.72667
FastABOD 100 0.00000 -0.02000 0.05000 0.03100 0.09677 0.07871 0.74444
KDEOS 2 0.00000 -0.02000 0.02373 0.00420 0.04918 0.03016 0.53222
KDEOS 8 0.00000 -0.02000 0.09913 0.08111 0.25000 0.23500 0.74000
KDEOS 9 0.00000 -0.02000 0.05431 0.03540 0.11765 0.10000 0.74889
LDF 1 0.00000 -0.02000 0.03396 0.01464 0.09091 0.07273 0.52444
LDF 75 0.00000 -0.02000 0.05415 0.03523 0.13953 0.12233 0.77333
LDF 86 0.00000 -0.02000 0.05501 0.03611 0.13636 0.11909 0.77778
INFLO 1 0.00000 -0.02000 0.02221 0.00265 0.05634 0.03746 0.40444
INFLO 2 0.00000 -0.02000 0.05720 0.03835 0.18182 0.16545 0.51333
INFLO 94 0.00000 -0.02000 0.05258 0.03363 0.13333 0.11600 0.76889
COF 1 0.00000 -0.02000 0.02948 0.01007 0.07143 0.05286 0.57667
COF 84 0.00000 -0.02000 0.04183 0.02266 0.10909 0.09127 0.70667

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO