Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (20% of outliers version#03)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.5 MB) Download raw algorithm evaluation table (54.6 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.43200 0.29000 0.38263 0.22829 0.50398 0.37997 0.75969
KNN 16 0.38400 0.23000 0.36860 0.21075 0.51500 0.39375 0.76766
KNN 44 0.34400 0.18000 0.36207 0.20258 0.52130 0.40163 0.76221
KNNW 2 0.43200 0.29000 0.37667 0.22084 0.49852 0.37315 0.74779
KNNW 16 0.39200 0.24000 0.37381 0.21726 0.50556 0.38194 0.76773
KNNW 85 0.36000 0.20000 0.36515 0.20643 0.51889 0.39861 0.76472
LOF 25 0.35200 0.19000 0.29425 0.11781 0.42534 0.28167 0.67835
LOF 95 0.35200 0.19000 0.33983 0.17479 0.48951 0.36189 0.74310
LOF 100 0.35200 0.19000 0.34218 0.17773 0.48276 0.35345 0.74528
SimplifiedLOF 28 0.35200 0.19000 0.27631 0.09538 0.40512 0.25640 0.64389
SimplifiedLOF 100 0.32000 0.15000 0.30169 0.12711 0.42925 0.28656 0.68310
LoOP 44 0.35200 0.19000 0.27868 0.09836 0.40100 0.25125 0.64251
LoOP 100 0.34400 0.18000 0.29516 0.11896 0.42081 0.27602 0.67106
LDOF 62 0.35200 0.19000 0.28109 0.10137 0.39574 0.24468 0.64210
LDOF 67 0.33600 0.17000 0.28705 0.10881 0.39911 0.24889 0.64773
LDOF 96 0.33600 0.17000 0.28459 0.10574 0.40991 0.26239 0.64894
LDOF 100 0.34400 0.18000 0.28655 0.10818 0.40879 0.26099 0.65211
ODIN 78 0.34800 0.18500 0.30782 0.13477 0.45794 0.32243 0.70114
ODIN 94 0.32800 0.16000 0.31301 0.14126 0.46336 0.32920 0.71037
ODIN 99 0.33600 0.17000 0.31629 0.14536 0.46118 0.32647 0.71393
ODIN 100 0.33867 0.17333 0.31577 0.14471 0.46226 0.32783 0.71473
FastABOD 6 0.48800 0.36000 0.40893 0.26116 0.52632 0.40789 0.76926
FastABOD 30 0.48000 0.35000 0.43134 0.28918 0.55844 0.44805 0.79342
FastABOD 92 0.47200 0.34000 0.43958 0.29947 0.54140 0.42675 0.80218
FastABOD 100 0.47200 0.34000 0.43893 0.29867 0.54313 0.42891 0.80248
KDEOS 14 0.27200 0.09000 0.23395 0.04244 0.36364 0.20455 0.55491
KDEOS 24 0.24000 0.05000 0.24831 0.06038 0.37302 0.21627 0.57533
KDEOS 100 0.22400 0.03000 0.24083 0.05104 0.39919 0.24898 0.60938
LDF 64 0.38400 0.23000 0.35060 0.18826 0.49519 0.36899 0.74790
LDF 94 0.36800 0.21000 0.36935 0.21169 0.53439 0.41799 0.77147
INFLO 66 0.35200 0.19000 0.29420 0.11775 0.45217 0.31522 0.66232
INFLO 99 0.32000 0.15000 0.31834 0.14792 0.50549 0.38187 0.70861
INFLO 100 0.32000 0.15000 0.31838 0.14798 0.50455 0.38068 0.70552
COF 98 0.44800 0.31000 0.41261 0.26576 0.52267 0.40333 0.76117
COF 100 0.44000 0.30000 0.41806 0.27257 0.51667 0.39583 0.76272

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 625 objects, 125 outliers (20.00%)

Download raw algorithm results (5.4 MB) Download raw algorithm evaluation table (55.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 13 0.44800 0.31000 0.36332 0.20415 0.46377 0.32971 0.69886
KNN 14 0.42400 0.28000 0.36275 0.20343 0.46763 0.33453 0.69826
KNN 15 0.43200 0.29000 0.36668 0.20834 0.46099 0.32624 0.69967
KNN 16 0.43200 0.29000 0.36624 0.20779 0.46643 0.33304 0.70080
KNNW 23 0.43200 0.29000 0.35693 0.19617 0.45993 0.32491 0.69405
KNNW 29 0.42400 0.28000 0.35843 0.19804 0.46528 0.33160 0.69515
KNNW 43 0.42400 0.28000 0.35872 0.19839 0.45205 0.31507 0.69592
KNNW 54 0.43200 0.29000 0.35878 0.19848 0.45361 0.31701 0.69590
LOF 82 0.39200 0.24000 0.33518 0.16898 0.46057 0.32571 0.69832
LOF 88 0.39200 0.24000 0.33684 0.17106 0.46349 0.32937 0.69952
LOF 93 0.38400 0.23000 0.33892 0.17366 0.45936 0.32420 0.70002
LOF 97 0.38400 0.23000 0.33972 0.17465 0.45833 0.32292 0.69912
SimplifiedLOF 74 0.36800 0.21000 0.30226 0.12782 0.39623 0.24528 0.65254
SimplifiedLOF 94 0.36800 0.21000 0.31267 0.14084 0.40189 0.25236 0.66056
SimplifiedLOF 100 0.36000 0.20000 0.31334 0.14167 0.40828 0.26036 0.66037
LoOP 92 0.36800 0.21000 0.29731 0.12163 0.39059 0.23824 0.64323
LoOP 100 0.36000 0.20000 0.29943 0.12429 0.39623 0.24528 0.64433
LDOF 54 0.29600 0.12000 0.27580 0.09475 0.39542 0.24427 0.62216
LDOF 95 0.36800 0.21000 0.30176 0.12720 0.38641 0.23301 0.64640
LDOF 100 0.36800 0.21000 0.30439 0.13049 0.39387 0.24234 0.64800
ODIN 41 0.29600 0.12000 0.26318 0.07898 0.39248 0.24061 0.61689
ODIN 91 0.34400 0.18000 0.28264 0.10330 0.37318 0.21647 0.62142
ODIN 99 0.34400 0.18000 0.29028 0.11285 0.38415 0.23018 0.62590
ODIN 100 0.34400 0.18000 0.28990 0.11237 0.38415 0.23018 0.62654
FastABOD 70 0.44000 0.30000 0.37316 0.21645 0.46914 0.33642 0.70597
FastABOD 95 0.44000 0.30000 0.37487 0.21859 0.47352 0.34190 0.70810
FastABOD 100 0.44000 0.30000 0.37548 0.21936 0.47352 0.34190 0.70894
KDEOS 86 0.21600 0.02000 0.22961 0.03701 0.38537 0.23171 0.59763
KDEOS 91 0.23200 0.04000 0.23036 0.03795 0.38000 0.22500 0.59867
KDEOS 97 0.23200 0.04000 0.23170 0.03963 0.38026 0.22532 0.60181
LDF 60 0.40800 0.26000 0.34599 0.18249 0.46405 0.33007 0.71034
LDF 74 0.40000 0.25000 0.35125 0.18906 0.47312 0.34140 0.71605
LDF 86 0.38400 0.23000 0.35338 0.19172 0.47445 0.34307 0.71450
LDF 87 0.38400 0.23000 0.35298 0.19122 0.47839 0.34798 0.71426
INFLO 77 0.36800 0.21000 0.31945 0.14931 0.50390 0.37987 0.67504
INFLO 81 0.35200 0.19000 0.32621 0.15776 0.52219 0.40274 0.68882
INFLO 83 0.35200 0.19000 0.32789 0.15986 0.52219 0.40274 0.68679
COF 98 0.36800 0.21000 0.33741 0.17176 0.46761 0.33451 0.72069
COF 99 0.36800 0.21000 0.33659 0.17073 0.47123 0.33904 0.71794

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO