Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#07)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (55.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.40000 0.25000 0.35678 0.19598 0.46819 0.33524 0.72966
KNN 30 0.36800 0.21000 0.36063 0.20079 0.50132 0.37665 0.74638
KNN 75 0.36000 0.20000 0.36222 0.20277 0.48864 0.36080 0.75019
KNN 80 0.36000 0.20000 0.36190 0.20237 0.48662 0.35827 0.75060
KNNW 5 0.42400 0.28000 0.35746 0.19683 0.47130 0.33912 0.73454
KNNW 52 0.37600 0.22000 0.35783 0.19729 0.49140 0.36425 0.74499
KNNW 100 0.36000 0.20000 0.36047 0.20059 0.48936 0.36170 0.74882
LOF 89 0.37600 0.22000 0.33575 0.16969 0.46829 0.33537 0.72854
LOF 92 0.37600 0.22000 0.33786 0.17232 0.47596 0.34495 0.73098
LOF 98 0.36800 0.21000 0.34218 0.17773 0.47086 0.33858 0.73429
SimplifiedLOF 39 0.36800 0.21000 0.28194 0.10242 0.38298 0.22872 0.63882
SimplifiedLOF 96 0.33600 0.17000 0.30477 0.13096 0.42506 0.28132 0.67358
SimplifiedLOF 100 0.32800 0.16000 0.30659 0.13324 0.42466 0.28082 0.67726
LoOP 50 0.35200 0.19000 0.28100 0.10126 0.38979 0.23724 0.63942
LoOP 100 0.33600 0.17000 0.30126 0.12657 0.42227 0.27784 0.66962
LDOF 61 0.35200 0.19000 0.28987 0.11234 0.40000 0.25000 0.64758
LDOF 68 0.36000 0.20000 0.29049 0.11312 0.39467 0.24333 0.64701
LDOF 82 0.37600 0.22000 0.28929 0.11162 0.41040 0.26301 0.64413
LDOF 83 0.38400 0.23000 0.28897 0.11121 0.40922 0.26153 0.64421
ODIN 68 0.34533 0.18167 0.30349 0.12936 0.42950 0.28688 0.67962
ODIN 93 0.33800 0.17250 0.31533 0.14416 0.45040 0.31300 0.69962
ODIN 100 0.32600 0.15750 0.31776 0.14719 0.44346 0.30432 0.70330
FastABOD 34 0.48800 0.36000 0.40675 0.25844 0.50151 0.37689 0.76539
FastABOD 50 0.46400 0.33000 0.40794 0.25992 0.51133 0.38916 0.76931
FastABOD 93 0.46400 0.33000 0.40938 0.26172 0.50658 0.38322 0.77264
FastABOD 100 0.46400 0.33000 0.40928 0.26159 0.50566 0.38208 0.77298
KDEOS 3 0.24000 0.05000 0.24513 0.05641 0.34380 0.17975 0.54682
KDEOS 21 0.27200 0.09000 0.22737 0.03421 0.35143 0.18929 0.54525
KDEOS 97 0.21600 0.02000 0.23642 0.04553 0.38416 0.23020 0.59688
KDEOS 100 0.23200 0.04000 0.23711 0.04638 0.38171 0.22714 0.59883
LDF 93 0.40000 0.25000 0.36639 0.20799 0.48469 0.35587 0.75098
LDF 95 0.40800 0.26000 0.36733 0.20917 0.48363 0.35453 0.75186
LDF 98 0.40800 0.26000 0.36931 0.21164 0.48411 0.35513 0.75269
INFLO 74 0.36000 0.20000 0.30580 0.13225 0.46025 0.32531 0.67598
INFLO 99 0.36000 0.20000 0.32181 0.15226 0.49565 0.36957 0.71090
COF 95 0.47200 0.34000 0.39811 0.24763 0.49135 0.36419 0.74038
COF 100 0.45600 0.32000 0.40827 0.26034 0.50187 0.37734 0.74659

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (55.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 12 0.44800 0.31000 0.37210 0.21513 0.46792 0.33491 0.69207
KNN 57 0.41600 0.27000 0.38214 0.22767 0.47696 0.34621 0.70549
KNN 60 0.40800 0.26000 0.38308 0.22885 0.48087 0.35109 0.70459
KNN 86 0.39200 0.24000 0.37871 0.22339 0.48913 0.36141 0.70183
KNNW 30 0.43200 0.29000 0.37263 0.21579 0.46565 0.33206 0.68966
KNNW 61 0.41600 0.27000 0.37972 0.22465 0.46418 0.33023 0.69805
KNNW 95 0.40800 0.26000 0.37900 0.22375 0.47027 0.33784 0.69925
KNNW 98 0.40000 0.25000 0.37890 0.22362 0.47154 0.33943 0.69912
LOF 92 0.41600 0.27000 0.37168 0.21460 0.47751 0.34689 0.70632
LOF 100 0.40800 0.26000 0.37550 0.21938 0.47887 0.34859 0.70987
SimplifiedLOF 76 0.40000 0.25000 0.33510 0.16888 0.40637 0.25797 0.65960
SimplifiedLOF 97 0.39200 0.24000 0.34616 0.18269 0.42718 0.28398 0.66883
SimplifiedLOF 98 0.39200 0.24000 0.34627 0.18284 0.42675 0.28344 0.66822
SimplifiedLOF 100 0.39200 0.24000 0.34626 0.18282 0.43087 0.28859 0.66840
LoOP 91 0.41600 0.27000 0.33140 0.16424 0.41767 0.27209 0.65134
LoOP 92 0.41600 0.27000 0.33204 0.16505 0.41767 0.27209 0.65317
LDOF 77 0.39200 0.24000 0.32676 0.15845 0.40719 0.25898 0.65182
LDOF 97 0.37600 0.22000 0.34155 0.17694 0.41549 0.26937 0.66762
LDOF 99 0.37600 0.22000 0.34252 0.17815 0.41281 0.26601 0.66766
ODIN 86 0.36533 0.20667 0.30695 0.13368 0.41237 0.26546 0.63787
ODIN 91 0.39040 0.23800 0.30703 0.13379 0.42403 0.28004 0.63367
ODIN 100 0.39467 0.24333 0.31610 0.14512 0.42105 0.27632 0.63550
FastABOD 53 0.44000 0.30000 0.37105 0.21381 0.45588 0.31985 0.67653
FastABOD 92 0.43200 0.29000 0.37435 0.21794 0.46324 0.32904 0.68181
FastABOD 100 0.43200 0.29000 0.37477 0.21847 0.46324 0.32904 0.68261
KDEOS 3 0.24000 0.05000 0.21236 0.01545 0.33378 0.16722 0.50290
KDEOS 91 0.22400 0.03000 0.22777 0.03471 0.37643 0.22054 0.59358
KDEOS 98 0.23200 0.04000 0.22982 0.03727 0.37562 0.21952 0.59664
LDF 81 0.42400 0.28000 0.38659 0.23324 0.48936 0.36170 0.72298
LDF 96 0.42400 0.28000 0.39289 0.24111 0.49470 0.36837 0.72880
LDF 100 0.41600 0.27000 0.39345 0.24182 0.49351 0.36688 0.72931
INFLO 66 0.40000 0.25000 0.34508 0.18135 0.48889 0.36111 0.67688
INFLO 95 0.37600 0.22000 0.36297 0.20371 0.51295 0.39119 0.70286
INFLO 99 0.36800 0.21000 0.36381 0.20477 0.51031 0.38789 0.70499
COF 87 0.37600 0.22000 0.35402 0.19253 0.44828 0.31034 0.69767
COF 99 0.36800 0.21000 0.36495 0.20619 0.46154 0.32692 0.70782
COF 100 0.36800 0.21000 0.36772 0.20965 0.46005 0.32506 0.70859

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO