Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

HeartDisease (20% of outliers version#03)

A data set containing medical data on heart problems. Affected patients are considered outliers and healthy people are considered inliers.

Download all data set variants used (92.9 kB). You can also access the original data. (heart.dat)

Normalized, without duplicates

This version contains 13 attributes, 187 objects, 37 outliers (19.79%)

Download raw algorithm results (1.6 MB) Download raw algorithm evaluation table (51.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 52 0.51351 0.39351 0.49057 0.36491 0.54369 0.43113 0.79514
KNN 79 0.43243 0.29243 0.50438 0.38212 0.56098 0.45268 0.79261
KNN 94 0.48649 0.35982 0.52620 0.40933 0.54118 0.42800 0.78874
KNNW 85 0.40541 0.25874 0.39457 0.24523 0.52083 0.40264 0.76523
KNNW 98 0.40541 0.25874 0.40022 0.25227 0.53846 0.42462 0.76973
KNNW 100 0.40541 0.25874 0.40046 0.25257 0.53846 0.42462 0.76973
LOF 76 0.51351 0.39351 0.44606 0.30942 0.54206 0.42910 0.76919
LOF 94 0.48649 0.35982 0.48732 0.36085 0.57143 0.46571 0.77928
LOF 99 0.51351 0.39351 0.50083 0.37770 0.56566 0.45852 0.79009
LOF 100 0.51351 0.39351 0.50254 0.37984 0.56566 0.45852 0.78901
SimplifiedLOF 90 0.24324 0.05658 0.29074 0.11579 0.48855 0.36239 0.69171
SimplifiedLOF 95 0.27027 0.09027 0.29988 0.12718 0.48438 0.35719 0.70108
SimplifiedLOF 100 0.27027 0.09027 0.31254 0.14297 0.48819 0.36194 0.71315
LoOP 89 0.24324 0.05658 0.29206 0.11743 0.48855 0.36239 0.69514
LoOP 100 0.27027 0.09027 0.32457 0.15796 0.48387 0.35656 0.71712
LDOF 3 0.27027 0.09027 0.23259 0.04330 0.34906 0.18849 0.51243
LDOF 4 0.24324 0.05658 0.28573 0.10954 0.34043 0.17773 0.51784
LDOF 99 0.21622 0.02288 0.27929 0.10151 0.46715 0.33572 0.67784
LDOF 100 0.21622 0.02288 0.28099 0.10364 0.46715 0.33572 0.67946
ODIN 98 0.49189 0.36656 0.41693 0.27310 0.54762 0.43603 0.76423
ODIN 100 0.48649 0.35982 0.41398 0.26943 0.55422 0.44426 0.76414
FastABOD 69 0.43243 0.29243 0.37675 0.22302 0.50000 0.37667 0.74667
FastABOD 90 0.40541 0.25874 0.38578 0.23427 0.50382 0.38142 0.75351
FastABOD 99 0.40541 0.25874 0.38910 0.23841 0.50382 0.38142 0.75568
FastABOD 100 0.40541 0.25874 0.38932 0.23869 0.50382 0.38142 0.75568
KDEOS 5 0.27027 0.09027 0.20884 0.01369 0.33880 0.17570 0.47964
KDEOS 8 0.21622 0.02288 0.24480 0.05851 0.33945 0.17651 0.46955
KDEOS 100 0.16216 -0.04450 0.23241 0.04307 0.44444 0.30741 0.60306
LDF 68 0.62162 0.52829 0.62427 0.53159 0.64000 0.55120 0.81676
LDF 69 0.62162 0.52829 0.61735 0.52297 0.64000 0.55120 0.81712
LDF 70 0.64865 0.56198 0.61725 0.52284 0.64865 0.56198 0.81495
INFLO 99 0.40541 0.25874 0.45686 0.32288 0.66667 0.58444 0.77667
INFLO 100 0.43243 0.29243 0.45497 0.32053 0.66667 0.58444 0.77703
COF 67 0.51351 0.39351 0.40676 0.26043 0.51351 0.39351 0.75748
COF 98 0.48649 0.35982 0.49249 0.36731 0.60465 0.50713 0.80018
COF 99 0.48649 0.35982 0.49811 0.37431 0.60000 0.50133 0.79874
COF 100 0.48649 0.35982 0.49764 0.37373 0.61538 0.52051 0.79892

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 13 attributes, 187 objects, 37 outliers (19.79%)

Download raw algorithm results (1.6 MB) Download raw algorithm evaluation table (49.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 4 0.48649 0.35982 0.39245 0.24259 0.54545 0.43333 0.75333
KNN 8 0.48649 0.35982 0.40673 0.26039 0.55238 0.44197 0.78640
KNN 9 0.45946 0.32613 0.40836 0.26242 0.54118 0.42800 0.78486
KNN 10 0.45946 0.32613 0.40704 0.26077 0.55556 0.44593 0.78117
KNNW 3 0.48649 0.35982 0.38926 0.23862 0.51111 0.39052 0.72973
KNNW 10 0.48649 0.35982 0.40283 0.25552 0.54737 0.43572 0.76486
KNNW 12 0.48649 0.35982 0.40214 0.25466 0.55319 0.44298 0.76739
KNNW 18 0.43243 0.29243 0.40117 0.25346 0.54167 0.42861 0.76937
LOF 17 0.45946 0.32613 0.36095 0.20332 0.50505 0.38296 0.75027
LOF 19 0.48649 0.35982 0.35906 0.20096 0.53191 0.41645 0.74486
LOF 28 0.43243 0.29243 0.37084 0.21565 0.51282 0.39265 0.74577
SimplifiedLOF 26 0.48649 0.35982 0.35627 0.19749 0.50000 0.37667 0.72955
SimplifiedLOF 43 0.45946 0.32613 0.36109 0.20349 0.52273 0.40500 0.73477
SimplifiedLOF 46 0.40541 0.25874 0.36219 0.20487 0.51613 0.39677 0.73676
LoOP 26 0.45946 0.32613 0.35635 0.19759 0.48193 0.35414 0.72964
LoOP 27 0.45946 0.32613 0.35950 0.20151 0.48193 0.35414 0.72649
LoOP 33 0.48649 0.35982 0.35485 0.19572 0.49315 0.36813 0.72369
LoOP 45 0.40541 0.25874 0.35027 0.19000 0.51685 0.39768 0.72640
LDOF 28 0.40541 0.25874 0.31285 0.14335 0.45669 0.32268 0.68198
LDOF 37 0.35135 0.19135 0.31601 0.14730 0.49057 0.36491 0.69027
LDOF 47 0.35135 0.19135 0.33229 0.16759 0.47458 0.34497 0.70342
LDOF 58 0.35135 0.19135 0.33462 0.17049 0.47312 0.34315 0.69694
ODIN 14 0.44595 0.30928 0.36435 0.20756 0.45614 0.32199 0.70901
ODIN 16 0.42568 0.28401 0.36157 0.20409 0.46154 0.32872 0.71784
ODIN 17 0.41892 0.27559 0.35729 0.19875 0.49412 0.36933 0.71468
ODIN 23 0.45946 0.32613 0.34609 0.18479 0.46154 0.32872 0.70595
FastABOD 6 0.48649 0.35982 0.38925 0.23859 0.50000 0.37667 0.70757
FastABOD 11 0.40541 0.25874 0.38063 0.22785 0.53191 0.41645 0.72216
FastABOD 20 0.45946 0.32613 0.40166 0.25407 0.53191 0.41645 0.72829
FastABOD 65 0.43243 0.29243 0.38759 0.23653 0.52336 0.40579 0.73261
KDEOS 23 0.27027 0.09027 0.36141 0.20389 0.38272 0.23045 0.61153
KDEOS 85 0.43243 0.29243 0.34482 0.18321 0.51064 0.38993 0.72432
KDEOS 98 0.40541 0.25874 0.34908 0.18852 0.54945 0.43832 0.73387
KDEOS 99 0.40541 0.25874 0.34940 0.18892 0.54945 0.43832 0.73459
LDF 14 0.45946 0.32613 0.41140 0.26622 0.54945 0.43832 0.77604
LDF 15 0.43243 0.29243 0.40897 0.26318 0.56180 0.45371 0.77135
LDF 16 0.48649 0.35982 0.41306 0.26828 0.55556 0.44593 0.77441
LDF 19 0.48649 0.35982 0.41499 0.27068 0.53846 0.42462 0.77477
INFLO 16 0.43243 0.29243 0.37350 0.21896 0.53435 0.41949 0.76486
INFLO 18 0.45946 0.32613 0.36446 0.20770 0.50725 0.38570 0.75009
INFLO 34 0.40541 0.25874 0.34455 0.18287 0.55046 0.43957 0.69333
COF 51 0.45946 0.32613 0.43502 0.29565 0.58586 0.48370 0.78252
COF 64 0.54054 0.42721 0.44505 0.30816 0.54167 0.42861 0.79081
COF 69 0.51351 0.39351 0.45249 0.31744 0.54545 0.43333 0.79658

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO