Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (2% of outliers version#08)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.5 MB) Download raw algorithm evaluation table (39.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.10000 0.08200 0.05847 0.03964 0.14286 0.12571 0.67300
KNN 6 0.10000 0.08200 0.09210 0.07394 0.16667 0.15000 0.74180
KNN 48 0.10000 0.08200 0.06582 0.04714 0.14286 0.12571 0.76520
KNNW 2 0.10000 0.08200 0.04497 0.02587 0.11765 0.10000 0.62160
KNNW 8 0.10000 0.08200 0.06950 0.05089 0.15385 0.13692 0.70960
KNNW 26 0.10000 0.08200 0.07422 0.05570 0.15385 0.13692 0.74960
KNNW 99 0.10000 0.08200 0.07400 0.05548 0.15385 0.13692 0.76300
LOF 2 0.10000 0.08200 0.05225 0.03330 0.14286 0.12571 0.56130
LOF 4 0.10000 0.08200 0.05705 0.03819 0.15385 0.13692 0.54800
LOF 98 0.00000 -0.02000 0.04673 0.02766 0.09524 0.07714 0.74880
SimplifiedLOF 2 0.10000 0.08200 0.06922 0.05060 0.16667 0.15000 0.45680
SimplifiedLOF 97 0.00000 -0.02000 0.04393 0.02481 0.09615 0.07808 0.68140
LoOP 2 0.10000 0.08200 0.03891 0.01969 0.13333 0.11600 0.46110
LoOP 6 0.10000 0.08200 0.05820 0.03936 0.15385 0.13692 0.55250
LoOP 11 0.10000 0.08200 0.05853 0.03970 0.15385 0.13692 0.56800
LoOP 59 0.10000 0.08200 0.04833 0.02930 0.10526 0.08737 0.66460
LDOF 3 0.10000 0.08200 0.06628 0.04760 0.15385 0.13692 0.61340
LDOF 98 0.00000 -0.02000 0.04040 0.02121 0.10811 0.09027 0.62280
ODIN 9 0.03636 0.01709 0.04153 0.02236 0.13793 0.12069 0.58260
ODIN 39 0.10000 0.08200 0.04501 0.02591 0.10169 0.08373 0.66390
ODIN 66 0.10000 0.08200 0.04903 0.03001 0.11111 0.09333 0.70630
ODIN 100 0.07500 0.05650 0.04799 0.02895 0.09524 0.07714 0.72520
FastABOD 3 0.10000 0.08200 0.04300 0.02386 0.11111 0.09333 0.61380
FastABOD 4 0.10000 0.08200 0.05530 0.03640 0.13793 0.12069 0.59320
FastABOD 99 0.10000 0.08200 0.06112 0.04235 0.13333 0.11600 0.70520
KDEOS 21 0.10000 0.08200 0.02920 0.00979 0.10526 0.08737 0.45820
KDEOS 98 0.00000 -0.02000 0.03860 0.01937 0.13793 0.12069 0.60220
KDEOS 100 0.00000 -0.02000 0.03836 0.01913 0.13333 0.11600 0.60500
LDF 2 0.10000 0.08200 0.06161 0.04284 0.15385 0.13692 0.52310
LDF 94 0.00000 -0.02000 0.04963 0.03062 0.11429 0.09657 0.76400
INFLO 2 0.10000 0.08200 0.07384 0.05532 0.16667 0.15000 0.45920
INFLO 97 0.00000 -0.02000 0.04284 0.02370 0.10101 0.08303 0.72920
COF 24 0.10000 0.08200 0.07441 0.05590 0.18750 0.17125 0.75680
COF 49 0.20000 0.18400 0.09280 0.07466 0.21053 0.19474 0.71380
COF 61 0.20000 0.18400 0.09593 0.07785 0.23529 0.22000 0.70460

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.4 MB) Download raw algorithm evaluation table (41.8 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.04331 0.02418 0.15873 0.14190 0.55620
KNN 15 0.00000 -0.02000 0.04949 0.03048 0.13333 0.11600 0.64460
KNN 48 0.00000 -0.02000 0.04237 0.02322 0.10959 0.09178 0.66040
KNNW 1 0.00000 -0.02000 0.03464 0.01534 0.09375 0.07563 0.54080
KNNW 7 0.00000 -0.02000 0.04433 0.02522 0.15873 0.14190 0.56760
KNNW 31 0.00000 -0.02000 0.04808 0.02904 0.13559 0.11831 0.63540
KNNW 62 0.00000 -0.02000 0.04455 0.02544 0.11765 0.10000 0.64540
LOF 1 0.00000 -0.02000 0.04060 0.02141 0.10000 0.08200 0.65290
LOF 22 0.00000 -0.02000 0.05494 0.03604 0.15686 0.14000 0.65300
LOF 28 0.00000 -0.02000 0.05505 0.03615 0.13793 0.12069 0.67640
LOF 75 0.00000 -0.02000 0.04829 0.02925 0.12245 0.10490 0.72580
SimplifiedLOF 1 0.10000 0.08200 0.06315 0.04442 0.15385 0.13692 0.60150
SimplifiedLOF 18 0.00000 -0.02000 0.04804 0.02900 0.18750 0.17125 0.55480
SimplifiedLOF 86 0.00000 -0.02000 0.04759 0.02854 0.11765 0.10000 0.66900
LoOP 1 0.10000 0.08200 0.06350 0.04477 0.15385 0.13692 0.62510
LoOP 18 0.00000 -0.02000 0.05297 0.03403 0.20690 0.19103 0.56360
LoOP 91 0.00000 -0.02000 0.04792 0.02887 0.12658 0.10911 0.68340
LDOF 3 0.00000 -0.02000 0.06821 0.04957 0.20513 0.18923 0.69680
LDOF 4 0.10000 0.08200 0.08759 0.06934 0.27586 0.26138 0.62000
LDOF 5 0.20000 0.18400 0.10057 0.08258 0.27273 0.25818 0.58440
ODIN 2 0.05333 0.03440 0.03249 0.01314 0.09412 0.07600 0.49650
ODIN 35 0.00000 -0.02000 0.05262 0.03368 0.15152 0.13455 0.66470
ODIN 62 0.00000 -0.02000 0.05320 0.03427 0.14634 0.12927 0.62200
ODIN 100 0.00000 -0.02000 0.04790 0.02886 0.12903 0.11161 0.69150
FastABOD 4 0.10000 0.08200 0.04572 0.02663 0.11594 0.09826 0.59940
FastABOD 5 0.00000 -0.02000 0.04241 0.02326 0.12903 0.11161 0.60260
KDEOS 5 0.10000 0.08200 0.09564 0.07755 0.27273 0.25818 0.54560
KDEOS 6 0.20000 0.18400 0.11357 0.09584 0.25000 0.23500 0.54120
KDEOS 97 0.00000 -0.02000 0.04013 0.02093 0.11364 0.09591 0.62640
LDF 15 0.00000 -0.02000 0.05522 0.03632 0.16327 0.14653 0.62920
LDF 19 0.10000 0.08200 0.05772 0.03888 0.14815 0.13111 0.65160
LDF 22 0.10000 0.08200 0.06100 0.04222 0.12121 0.10364 0.68160
LDF 55 0.00000 -0.02000 0.05339 0.03446 0.13861 0.12139 0.73800
INFLO 5 0.10000 0.08200 0.05286 0.03391 0.14286 0.12571 0.53060
INFLO 9 0.00000 -0.02000 0.05048 0.03149 0.18750 0.17125 0.56360
INFLO 14 0.00000 -0.02000 0.05360 0.03468 0.17647 0.16000 0.57660
INFLO 46 0.00000 -0.02000 0.05088 0.03190 0.14493 0.12783 0.71500
COF 1 0.10000 0.08200 0.06317 0.04443 0.15385 0.13692 0.60210
COF 34 0.00000 -0.02000 0.05237 0.03342 0.16216 0.14541 0.66080
COF 100 0.00000 -0.02000 0.04547 0.02638 0.10526 0.08737 0.69300

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO