Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (2% of outliers version#09)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.5 MB) Download raw algorithm evaluation table (41.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.04634 0.02727 0.09091 0.07273 0.74940
KNN 9 0.00000 -0.02000 0.03934 0.02012 0.09890 0.08088 0.72440
KNNW 1 0.00000 -0.02000 0.04660 0.02754 0.11594 0.09826 0.66230
KNNW 6 0.00000 -0.02000 0.04436 0.02525 0.09231 0.07415 0.73380
LOF 3 0.10000 0.08200 0.03864 0.01941 0.10526 0.08737 0.59960
LOF 6 0.00000 -0.02000 0.03698 0.01772 0.11111 0.09333 0.65600
LOF 11 0.00000 -0.02000 0.04369 0.02456 0.10526 0.08737 0.70080
SimplifiedLOF 3 0.10000 0.08200 0.04266 0.02352 0.13333 0.11600 0.58060
SimplifiedLOF 19 0.00000 -0.02000 0.03883 0.01961 0.08333 0.06500 0.67420
LoOP 1 0.00000 -0.02000 0.02640 0.00693 0.08696 0.06870 0.51500
LoOP 19 0.00000 -0.02000 0.03871 0.01949 0.08696 0.06870 0.66580
LoOP 27 0.00000 -0.02000 0.03759 0.01834 0.09091 0.07273 0.63980
LDOF 2 0.00000 -0.02000 0.01840 -0.00124 0.04687 0.02781 0.45180
LDOF 21 0.00000 -0.02000 0.03590 0.01662 0.10000 0.08200 0.60160
LDOF 42 0.00000 -0.02000 0.03727 0.01801 0.08333 0.06500 0.64100
LDOF 60 0.00000 -0.02000 0.03307 0.01373 0.08163 0.06327 0.64260
ODIN 10 0.05000 0.03100 0.03479 0.01548 0.08889 0.07067 0.61760
ODIN 16 0.00000 -0.02000 0.03836 0.01913 0.10000 0.08200 0.66140
ODIN 32 0.00000 -0.02000 0.03908 0.01986 0.11765 0.10000 0.63080
ODIN 39 0.00000 -0.02000 0.03694 0.01767 0.12903 0.11161 0.62930
FastABOD 3 0.00000 -0.02000 0.04006 0.02086 0.10687 0.08901 0.69340
FastABOD 52 0.00000 -0.02000 0.05464 0.03573 0.15789 0.14105 0.75780
FastABOD 69 0.00000 -0.02000 0.05529 0.03640 0.15789 0.14105 0.75860
FastABOD 78 0.00000 -0.02000 0.05493 0.03603 0.15385 0.13692 0.76020
KDEOS 5 0.10000 0.08200 0.07592 0.05744 0.20000 0.18400 0.64820
KDEOS 6 0.20000 0.18400 0.07933 0.06092 0.21053 0.19474 0.60560
KDEOS 9 0.20000 0.18400 0.22543 0.20994 0.33333 0.32000 0.60880
LDF 1 0.00000 -0.02000 0.01874 -0.00088 0.05000 0.03100 0.39410
LDF 5 0.00000 -0.02000 0.04919 0.03017 0.14815 0.13111 0.63660
LDF 11 0.00000 -0.02000 0.04213 0.02297 0.10000 0.08200 0.71320
INFLO 1 0.00000 -0.02000 0.02274 0.00319 0.05590 0.03702 0.53600
INFLO 25 0.00000 -0.02000 0.03526 0.01597 0.09091 0.07273 0.61060
INFLO 30 0.00000 -0.02000 0.03586 0.01658 0.08333 0.06500 0.62220
INFLO 86 0.00000 -0.02000 0.03205 0.01269 0.07519 0.05669 0.67800
COF 14 0.00000 -0.02000 0.05765 0.03881 0.15385 0.13692 0.72780
COF 19 0.10000 0.08200 0.05288 0.03394 0.15385 0.13692 0.70980
COF 24 0.10000 0.08200 0.06188 0.04312 0.18182 0.16545 0.69660

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.4 MB) Download raw algorithm evaluation table (42.1 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.04371 0.02458 0.13333 0.11600 0.66930
KNN 3 0.00000 -0.02000 0.04299 0.02384 0.12308 0.10554 0.72140
KNNW 1 0.00000 -0.02000 0.04366 0.02453 0.15000 0.13300 0.55990
KNNW 72 0.00000 -0.02000 0.03682 0.01756 0.07895 0.06053 0.70960
LOF 1 0.00000 -0.02000 0.01677 -0.00290 0.04329 0.02416 0.36570
LOF 20 0.00000 -0.02000 0.04666 0.02760 0.11215 0.09439 0.74340
LOF 62 0.00000 -0.02000 0.04451 0.02540 0.12389 0.10637 0.72380
SimplifiedLOF 1 0.00000 -0.02000 0.02312 0.00358 0.06061 0.04182 0.50090
SimplifiedLOF 33 0.00000 -0.02000 0.03994 0.02073 0.12766 0.11021 0.69360
SimplifiedLOF 95 0.00000 -0.02000 0.04090 0.02172 0.10000 0.08200 0.72180
LoOP 1 0.00000 -0.02000 0.02293 0.00339 0.06061 0.04182 0.49450
LoOP 33 0.00000 -0.02000 0.03903 0.01981 0.12245 0.10490 0.68060
LoOP 97 0.00000 -0.02000 0.04248 0.02333 0.10000 0.08200 0.72080
LDOF 2 0.00000 -0.02000 0.02738 0.00792 0.09091 0.07273 0.49520
LDOF 52 0.00000 -0.02000 0.04416 0.02504 0.13953 0.12233 0.68340
LDOF 93 0.00000 -0.02000 0.04668 0.02761 0.10959 0.09178 0.72780
LDOF 100 0.00000 -0.02000 0.04609 0.02702 0.10667 0.08880 0.72900
ODIN 1 0.00613 -0.01374 0.02163 0.00206 0.04615 0.02708 0.47290
ODIN 53 0.00000 -0.02000 0.05454 0.03563 0.14634 0.12927 0.75680
ODIN 55 0.00000 -0.02000 0.05480 0.03589 0.15385 0.13692 0.75540
ODIN 56 0.00000 -0.02000 0.05708 0.03823 0.15385 0.13692 0.75600
FastABOD 3 0.00000 -0.02000 0.03468 0.01538 0.08955 0.07134 0.61800
FastABOD 7 0.00000 -0.02000 0.03649 0.01722 0.11765 0.10000 0.66180
FastABOD 81 0.00000 -0.02000 0.04085 0.02167 0.11321 0.09547 0.70000
FastABOD 97 0.00000 -0.02000 0.04053 0.02134 0.11111 0.09333 0.70120
KDEOS 2 0.00000 -0.02000 0.03127 0.01190 0.09524 0.07714 0.58060
KDEOS 72 0.00000 -0.02000 0.03882 0.01960 0.10526 0.08737 0.67100
KDEOS 73 0.00000 -0.02000 0.03880 0.01957 0.10909 0.09127 0.67140
KDEOS 89 0.00000 -0.02000 0.03703 0.01777 0.09901 0.08099 0.67880
LDF 1 0.00000 -0.02000 0.02202 0.00246 0.08333 0.06500 0.37350
LDF 15 0.00000 -0.02000 0.05572 0.03684 0.13953 0.12233 0.77940
INFLO 1 0.00000 -0.02000 0.02207 0.00251 0.05882 0.04000 0.55090
INFLO 33 0.00000 -0.02000 0.04366 0.02454 0.11765 0.10000 0.74200
INFLO 63 0.00000 -0.02000 0.04755 0.02850 0.09722 0.07917 0.77820
COF 25 0.20000 0.18400 0.06671 0.04805 0.20000 0.18400 0.72820
COF 70 0.20000 0.18400 0.09453 0.07642 0.25000 0.23500 0.76680
COF 71 0.20000 0.18400 0.09938 0.08137 0.25000 0.23500 0.76940
COF 76 0.20000 0.18400 0.08450 0.06619 0.22222 0.20667 0.77740

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO