Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#02)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (55.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.40000 0.25000 0.37274 0.21592 0.48171 0.35213 0.72130
KNN 5 0.41600 0.27000 0.36162 0.20202 0.46409 0.33011 0.72609
KNN 88 0.35200 0.19000 0.36451 0.20563 0.49036 0.36295 0.73962
KNN 100 0.35200 0.19000 0.36507 0.20634 0.48634 0.35792 0.74096
KNNW 3 0.40800 0.26000 0.36851 0.21064 0.48876 0.36096 0.71838
KNNW 4 0.41600 0.27000 0.37231 0.21539 0.48555 0.35694 0.72139
KNNW 98 0.36000 0.20000 0.36560 0.20700 0.47493 0.34367 0.73803
LOF 84 0.33600 0.17000 0.31795 0.14744 0.45989 0.32487 0.69808
LOF 99 0.36000 0.20000 0.32809 0.16012 0.45014 0.31268 0.71250
LOF 100 0.36000 0.20000 0.32782 0.15978 0.45014 0.31268 0.71277
SimplifiedLOF 75 0.31200 0.14000 0.28452 0.10565 0.39169 0.23961 0.63138
SimplifiedLOF 100 0.29600 0.12000 0.28989 0.11237 0.40341 0.25426 0.64240
LoOP 99 0.32000 0.15000 0.28266 0.10333 0.39180 0.23975 0.63118
LoOP 100 0.29600 0.12000 0.28343 0.10429 0.39394 0.24242 0.63168
LDOF 67 0.32000 0.15000 0.26734 0.08417 0.36923 0.21154 0.60931
LDOF 71 0.29600 0.12000 0.26783 0.08479 0.37349 0.21687 0.61024
LDOF 76 0.29600 0.12000 0.26660 0.08325 0.37581 0.21976 0.60818
ODIN 83 0.34933 0.18667 0.29566 0.11958 0.41631 0.27039 0.66337
ODIN 91 0.33867 0.17333 0.30121 0.12651 0.43069 0.28837 0.67059
ODIN 100 0.34000 0.17500 0.30796 0.13495 0.43069 0.28837 0.67566
FastABOD 42 0.49600 0.37000 0.41422 0.26777 0.49640 0.37050 0.75619
FastABOD 69 0.47200 0.34000 0.41740 0.27175 0.50923 0.38653 0.76021
FastABOD 99 0.46400 0.33000 0.42349 0.27936 0.50370 0.37963 0.76371
FastABOD 100 0.47200 0.34000 0.42342 0.27927 0.50370 0.37963 0.76382
KDEOS 4 0.20800 0.01000 0.23666 0.04583 0.35413 0.19266 0.54229
KDEOS 39 0.20800 0.01000 0.21762 0.02203 0.37201 0.21502 0.55419
KDEOS 55 0.24800 0.06000 0.21333 0.01666 0.36720 0.20900 0.55218
KDEOS 100 0.23200 0.04000 0.22374 0.02968 0.36581 0.20726 0.57198
LDF 60 0.40000 0.25000 0.32852 0.16065 0.44571 0.30714 0.70179
LDF 100 0.38400 0.23000 0.35719 0.19649 0.47837 0.34796 0.74240
INFLO 100 0.32800 0.16000 0.29846 0.12308 0.46696 0.33370 0.66194
COF 97 0.41600 0.27000 0.37929 0.22411 0.47091 0.33864 0.70291
COF 99 0.43200 0.29000 0.39348 0.24186 0.46234 0.32792 0.70744
COF 100 0.41600 0.27000 0.39729 0.24661 0.46866 0.33583 0.70934

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (55.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 11 0.40800 0.26000 0.33702 0.17128 0.43526 0.29408 0.66972
KNN 14 0.40000 0.25000 0.34043 0.17554 0.44000 0.30000 0.67262
KNN 62 0.37600 0.22000 0.33799 0.17249 0.44444 0.30556 0.67860
KNN 77 0.36800 0.21000 0.33531 0.16914 0.45055 0.31319 0.67523
KNNW 2 0.40000 0.25000 0.31779 0.14724 0.42244 0.27805 0.64845
KNNW 22 0.40000 0.25000 0.33532 0.16916 0.44505 0.30632 0.66923
KNNW 63 0.37600 0.22000 0.33720 0.17150 0.43275 0.29094 0.67291
KNNW 88 0.36800 0.21000 0.33658 0.17072 0.43820 0.29775 0.67406
LOF 72 0.38400 0.23000 0.33064 0.16330 0.45181 0.31476 0.68738
LOF 88 0.37600 0.22000 0.33869 0.17336 0.48930 0.36162 0.69779
LOF 89 0.37600 0.22000 0.33893 0.17366 0.48632 0.35790 0.69830
LOF 100 0.37600 0.22000 0.34049 0.17561 0.48916 0.36146 0.69691
SimplifiedLOF 88 0.36800 0.21000 0.30168 0.12711 0.39634 0.24543 0.63613
SimplifiedLOF 100 0.36800 0.21000 0.30594 0.13243 0.40752 0.25940 0.64171
LoOP 86 0.36800 0.21000 0.28650 0.10812 0.37805 0.22256 0.62230
LoOP 100 0.36800 0.21000 0.29226 0.11533 0.38987 0.23734 0.63375
LDOF 87 0.34400 0.18000 0.29183 0.11479 0.38255 0.22819 0.61597
LDOF 96 0.33600 0.17000 0.29753 0.12191 0.39490 0.24363 0.62418
LDOF 100 0.34400 0.18000 0.29953 0.12441 0.38978 0.23722 0.62754
ODIN 96 0.36000 0.20000 0.28731 0.10914 0.39335 0.24169 0.62872
ODIN 97 0.36160 0.20200 0.28773 0.10966 0.39039 0.23799 0.62817
ODIN 100 0.35733 0.19667 0.28958 0.11198 0.39752 0.24689 0.62711
FastABOD 11 0.40800 0.26000 0.33996 0.17495 0.44068 0.30085 0.67642
FastABOD 89 0.39200 0.24000 0.35333 0.19166 0.45070 0.31338 0.69133
FastABOD 100 0.39200 0.24000 0.35397 0.19246 0.44966 0.31208 0.69238
KDEOS 93 0.22400 0.03000 0.22344 0.02930 0.36998 0.21248 0.57730
KDEOS 95 0.23200 0.04000 0.22380 0.02975 0.36897 0.21121 0.57776
KDEOS 100 0.23200 0.04000 0.22532 0.03164 0.36762 0.20953 0.58046
LDF 86 0.39200 0.24000 0.35151 0.18939 0.49080 0.36350 0.71195
LDF 87 0.40000 0.25000 0.35199 0.18999 0.48930 0.36162 0.71291
LDF 96 0.40800 0.26000 0.35347 0.19183 0.48447 0.35559 0.71080
INFLO 44 0.36000 0.20000 0.29732 0.12165 0.47619 0.34524 0.65194
INFLO 100 0.32800 0.16000 0.32429 0.15536 0.51832 0.39791 0.68650
COF 42 0.34400 0.18000 0.29142 0.11427 0.37795 0.22244 0.63320
COF 96 0.32800 0.16000 0.33065 0.16332 0.46552 0.33190 0.70540
COF 100 0.32800 0.16000 0.32850 0.16062 0.47027 0.33784 0.70250

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO