Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#10)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (54.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40800 0.26000 0.37633 0.22041 0.46032 0.32540 0.73473
KNN 2 0.42400 0.28000 0.37621 0.22026 0.48021 0.35026 0.74270
KNNW 1 0.44000 0.30000 0.36603 0.20754 0.46715 0.33394 0.71183
KNNW 4 0.43200 0.29000 0.37795 0.22244 0.47293 0.34117 0.73890
KNNW 5 0.44000 0.30000 0.37754 0.22193 0.47727 0.34659 0.73864
LOF 93 0.34400 0.18000 0.31189 0.13986 0.44986 0.31233 0.70304
LOF 99 0.36000 0.20000 0.31348 0.14185 0.44205 0.30256 0.70526
LOF 100 0.35200 0.19000 0.31520 0.14400 0.44560 0.30699 0.70587
SimplifiedLOF 12 0.33600 0.17000 0.26752 0.08440 0.37945 0.22431 0.60986
SimplifiedLOF 99 0.28800 0.11000 0.28566 0.10708 0.40729 0.25912 0.65301
SimplifiedLOF 100 0.28800 0.11000 0.28570 0.10713 0.40719 0.25898 0.65344
LoOP 9 0.32800 0.16000 0.26592 0.08240 0.36229 0.20287 0.60030
LoOP 95 0.28800 0.11000 0.27898 0.09873 0.39744 0.24679 0.63750
LoOP 100 0.29600 0.12000 0.28111 0.10139 0.39645 0.24556 0.63981
LDOF 8 0.29600 0.12000 0.25333 0.06666 0.35714 0.19643 0.58446
LDOF 26 0.26400 0.08000 0.26446 0.08058 0.38950 0.23687 0.60942
LDOF 27 0.26400 0.08000 0.26509 0.08136 0.38174 0.22718 0.60861
ODIN 49 0.32463 0.15579 0.29182 0.11477 0.38235 0.22794 0.62434
ODIN 60 0.32000 0.15000 0.29484 0.11855 0.38127 0.22659 0.62948
ODIN 97 0.30400 0.13000 0.28987 0.11234 0.40506 0.25633 0.64716
ODIN 100 0.31520 0.14400 0.29058 0.11322 0.40299 0.25373 0.64961
FastABOD 6 0.45600 0.32000 0.39716 0.24645 0.50157 0.37696 0.74536
FastABOD 8 0.47200 0.34000 0.40744 0.25931 0.49342 0.36678 0.74890
FastABOD 99 0.46400 0.33000 0.41691 0.27114 0.49375 0.36719 0.76243
FastABOD 100 0.46400 0.33000 0.41698 0.27122 0.49477 0.36847 0.76237
KDEOS 7 0.28000 0.10000 0.23479 0.04349 0.35229 0.19037 0.56544
KDEOS 9 0.27200 0.09000 0.25838 0.07297 0.35873 0.19841 0.57342
KDEOS 36 0.25600 0.07000 0.22355 0.02944 0.37931 0.22414 0.57149
KDEOS 99 0.20800 0.01000 0.22806 0.03507 0.36851 0.21064 0.58498
LDF 89 0.41600 0.27000 0.33740 0.17175 0.46469 0.33087 0.72944
LDF 99 0.38400 0.23000 0.33942 0.17427 0.47059 0.33824 0.73142
LDF 100 0.37600 0.22000 0.33965 0.17456 0.47059 0.33824 0.73128
INFLO 54 0.28000 0.10000 0.28181 0.10226 0.44536 0.30670 0.63693
INFLO 74 0.29600 0.12000 0.28341 0.10427 0.44219 0.30274 0.64501
INFLO 84 0.32000 0.15000 0.28265 0.10331 0.44075 0.30094 0.63330
INFLO 100 0.31200 0.14000 0.28595 0.10744 0.43892 0.29865 0.63667
COF 88 0.45600 0.32000 0.40562 0.25703 0.49419 0.36773 0.75578
COF 99 0.43200 0.29000 0.41312 0.26640 0.50323 0.37903 0.76434
COF 100 0.43200 0.29000 0.41155 0.26443 0.50485 0.38107 0.76496

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (54.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 12 0.40000 0.25000 0.32400 0.15499 0.42804 0.28506 0.66209
KNN 13 0.40800 0.26000 0.32397 0.15496 0.43321 0.29152 0.66331
KNN 63 0.36000 0.20000 0.31700 0.14625 0.43732 0.29665 0.66524
KNN 78 0.36800 0.21000 0.31577 0.14471 0.44382 0.30478 0.66154
KNNW 19 0.38400 0.23000 0.31857 0.14821 0.43357 0.29196 0.65946
KNNW 30 0.40800 0.26000 0.31883 0.14854 0.42321 0.27901 0.66048
KNNW 44 0.40000 0.25000 0.31973 0.14966 0.42133 0.27667 0.66170
KNNW 62 0.40000 0.25000 0.31804 0.14755 0.42388 0.27985 0.66245
LOF 82 0.38400 0.23000 0.31998 0.14998 0.44079 0.30099 0.68926
LOF 87 0.36800 0.21000 0.32093 0.15116 0.44660 0.30825 0.69040
LOF 90 0.37600 0.22000 0.32070 0.15088 0.44884 0.31106 0.69072
LOF 91 0.36800 0.21000 0.32046 0.15057 0.45033 0.31291 0.68998
SimplifiedLOF 85 0.33600 0.17000 0.28693 0.10866 0.39059 0.23824 0.62946
SimplifiedLOF 99 0.33600 0.17000 0.29123 0.11404 0.40000 0.25000 0.63573
SimplifiedLOF 100 0.33600 0.17000 0.29046 0.11307 0.40367 0.25459 0.63494
LoOP 88 0.32800 0.16000 0.27695 0.09618 0.39091 0.23864 0.61595
LoOP 94 0.34400 0.18000 0.27793 0.09742 0.38652 0.23315 0.61768
LoOP 99 0.34400 0.18000 0.27983 0.09979 0.38350 0.22937 0.62059
LoOP 100 0.34400 0.18000 0.27999 0.09998 0.38350 0.22937 0.61992
LDOF 92 0.32800 0.16000 0.28908 0.11135 0.37975 0.22468 0.61658
LDOF 99 0.32800 0.16000 0.29154 0.11442 0.38955 0.23694 0.62146
LDOF 100 0.32800 0.16000 0.29096 0.11370 0.39320 0.24150 0.62026
ODIN 84 0.36229 0.20286 0.28766 0.10958 0.37193 0.21491 0.62018
ODIN 88 0.34057 0.17571 0.29001 0.11251 0.37594 0.21992 0.62142
ODIN 97 0.33600 0.17000 0.28797 0.10996 0.38596 0.23246 0.62214
FastABOD 11 0.40000 0.25000 0.32932 0.16165 0.42759 0.28448 0.66787
FastABOD 96 0.39200 0.24000 0.33859 0.17324 0.44526 0.30657 0.67970
FastABOD 100 0.39200 0.24000 0.33878 0.17348 0.44526 0.30657 0.67990
KDEOS 6 0.24000 0.05000 0.23646 0.04557 0.34703 0.18379 0.53886
KDEOS 82 0.20000 0.00000 0.22509 0.03136 0.37148 0.21434 0.57552
KDEOS 100 0.21600 0.02000 0.23110 0.03888 0.36992 0.21240 0.58243
LDF 74 0.38400 0.23000 0.33019 0.16273 0.45552 0.31940 0.70222
LDF 80 0.38400 0.23000 0.32964 0.16205 0.45614 0.32018 0.70318
LDF 85 0.38400 0.23000 0.32995 0.16243 0.46246 0.32808 0.70258
LDF 88 0.39200 0.24000 0.32938 0.16173 0.46246 0.32808 0.70085
INFLO 41 0.33600 0.17000 0.28554 0.10692 0.45771 0.32214 0.63106
INFLO 96 0.32800 0.16000 0.30805 0.13506 0.51213 0.39016 0.66437
INFLO 98 0.32800 0.16000 0.30829 0.13537 0.51087 0.38859 0.66327
COF 28 0.35200 0.19000 0.26740 0.08425 0.38828 0.23535 0.62813
COF 91 0.32800 0.16000 0.32082 0.15102 0.44986 0.31233 0.69424
COF 100 0.32800 0.16000 0.33070 0.16337 0.43767 0.29709 0.69750

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO