Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (2% of outliers version#06)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.5 MB) Download raw algorithm evaluation table (41.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.06974 0.05113 0.18462 0.16831 0.78900
KNN 4 0.00000 -0.02000 0.06192 0.04316 0.16216 0.14541 0.79760
KNNW 1 0.00000 -0.02000 0.06833 0.04970 0.16949 0.15288 0.80430
KNNW 2 0.00000 -0.02000 0.07043 0.05183 0.17544 0.15895 0.80300
KNNW 3 0.00000 -0.02000 0.07123 0.05266 0.16949 0.15288 0.80180
LOF 1 0.10000 0.08200 0.08398 0.06566 0.16667 0.15000 0.63840
LOF 2 0.10000 0.08200 0.06604 0.04736 0.19048 0.17429 0.71280
LOF 99 0.00000 -0.02000 0.04839 0.02936 0.11650 0.09883 0.76680
SimplifiedLOF 1 0.10000 0.08200 0.07311 0.05457 0.16667 0.15000 0.58510
SimplifiedLOF 99 0.00000 -0.02000 0.04538 0.02629 0.11034 0.09255 0.73960
LoOP 1 0.10000 0.08200 0.07308 0.05455 0.16667 0.15000 0.58450
LoOP 24 0.00000 -0.02000 0.06071 0.04192 0.17647 0.16000 0.74060
LoOP 50 0.00000 -0.02000 0.04867 0.02964 0.11111 0.09333 0.74270
LDOF 8 0.10000 0.08200 0.06247 0.04372 0.17391 0.15739 0.65640
LDOF 18 0.10000 0.08200 0.07449 0.05598 0.18750 0.17125 0.74080
LDOF 24 0.00000 -0.02000 0.07354 0.05501 0.23077 0.21538 0.73820
ODIN 26 0.06000 0.04120 0.06970 0.05110 0.16000 0.14320 0.77250
ODIN 28 0.06000 0.04120 0.06336 0.04463 0.13793 0.12069 0.77630
ODIN 32 0.10000 0.08200 0.06926 0.05064 0.16327 0.14653 0.76690
FastABOD 5 0.20000 0.18400 0.10391 0.08599 0.21053 0.19474 0.80200
FastABOD 6 0.20000 0.18400 0.10575 0.08787 0.22222 0.20667 0.80600
FastABOD 10 0.00000 -0.02000 0.09521 0.07711 0.23529 0.22000 0.81580
FastABOD 85 0.10000 0.08200 0.09542 0.07733 0.22222 0.20667 0.83500
KDEOS 5 0.10000 0.08200 0.05044 0.03145 0.12500 0.10750 0.59040
KDEOS 7 0.10000 0.08200 0.14101 0.12383 0.18182 0.16545 0.64100
KDEOS 9 0.10000 0.08200 0.14276 0.12561 0.18182 0.16545 0.62980
KDEOS 74 0.00000 -0.02000 0.05190 0.03294 0.13699 0.11973 0.75040
LDF 1 0.10000 0.08200 0.08027 0.06188 0.15385 0.13692 0.69000
LDF 2 0.10000 0.08200 0.08829 0.07006 0.25000 0.23500 0.71420
LDF 100 0.00000 -0.02000 0.05292 0.03397 0.12844 0.11101 0.78220
INFLO 1 0.10000 0.08200 0.04807 0.02903 0.11111 0.09333 0.62700
INFLO 2 0.00000 -0.02000 0.06140 0.04263 0.20000 0.18400 0.59220
INFLO 18 0.10000 0.08200 0.06394 0.04522 0.15789 0.14105 0.73460
INFLO 78 0.00000 -0.02000 0.04895 0.02993 0.10714 0.08929 0.76800
COF 4 0.00000 -0.02000 0.06935 0.05074 0.25000 0.23500 0.58230
COF 20 0.20000 0.18400 0.08267 0.06432 0.22222 0.20667 0.74740
COF 22 0.20000 0.18400 0.08424 0.06593 0.21053 0.19474 0.74380
COF 95 0.10000 0.08200 0.06860 0.04997 0.12500 0.10750 0.78220

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.4 MB) Download raw algorithm evaluation table (40.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.05260 0.03365 0.13953 0.12233 0.71000
KNN 2 0.00000 -0.02000 0.05164 0.03267 0.12245 0.10490 0.71400
KNNW 1 0.10000 0.08200 0.06767 0.04902 0.17143 0.15486 0.72620
LOF 1 0.00000 -0.02000 0.03154 0.01217 0.07595 0.05747 0.60240
LOF 20 0.00000 -0.02000 0.06251 0.04376 0.18519 0.16889 0.69720
LOF 21 0.00000 -0.02000 0.06237 0.04362 0.20000 0.18400 0.69620
LOF 85 0.00000 -0.02000 0.04056 0.02137 0.11628 0.09860 0.71160
SimplifiedLOF 1 0.10000 0.08200 0.03018 0.01079 0.10000 0.08200 0.54140
SimplifiedLOF 23 0.00000 -0.02000 0.05455 0.03564 0.14545 0.12836 0.68540
SimplifiedLOF 27 0.00000 -0.02000 0.05695 0.03809 0.17778 0.16133 0.67100
LoOP 1 0.10000 0.08200 0.03050 0.01111 0.10000 0.08200 0.56390
LoOP 28 0.00000 -0.02000 0.05408 0.03516 0.16327 0.14653 0.68380
LoOP 31 0.00000 -0.02000 0.05541 0.03652 0.15385 0.13692 0.69640
LDOF 5 0.10000 0.08200 0.05236 0.03340 0.20690 0.19103 0.51000
LDOF 6 0.10000 0.08200 0.06047 0.04168 0.24000 0.22480 0.53680
LDOF 31 0.00000 -0.02000 0.05979 0.04099 0.16667 0.15000 0.70100
ODIN 6 0.07692 0.05846 0.05828 0.03945 0.15094 0.13396 0.64800
ODIN 38 0.00000 -0.02000 0.06286 0.04411 0.18868 0.17245 0.68890
ODIN 39 0.00000 -0.02000 0.06361 0.04488 0.18868 0.17245 0.69430
ODIN 42 0.00000 -0.02000 0.06225 0.04350 0.17021 0.15362 0.70010
FastABOD 3 0.10000 0.08200 0.07046 0.05187 0.16667 0.15000 0.74940
FastABOD 97 0.00000 -0.02000 0.05578 0.03689 0.13158 0.11421 0.75620
KDEOS 6 0.10000 0.08200 0.03326 0.01392 0.10000 0.08200 0.55020
KDEOS 75 0.00000 -0.02000 0.05230 0.03335 0.14815 0.13111 0.67460
KDEOS 80 0.00000 -0.02000 0.05229 0.03334 0.16000 0.14320 0.67300
LDF 1 0.00000 -0.02000 0.03217 0.01281 0.08247 0.06412 0.63140
LDF 5 0.00000 -0.02000 0.04913 0.03012 0.12658 0.10911 0.74780
LDF 15 0.00000 -0.02000 0.07524 0.05675 0.23810 0.22286 0.70800
INFLO 1 0.10000 0.08200 0.04168 0.02251 0.13333 0.11600 0.53540
INFLO 31 0.00000 -0.02000 0.05204 0.03308 0.16667 0.15000 0.66400
INFLO 94 0.00000 -0.02000 0.04307 0.02393 0.11628 0.09860 0.70780
COF 1 0.10000 0.08200 0.03020 0.01080 0.10000 0.08200 0.54140
COF 14 0.00000 -0.02000 0.06549 0.04680 0.20690 0.19103 0.73480
COF 15 0.10000 0.08200 0.06906 0.05044 0.17647 0.16000 0.75880
COF 16 0.10000 0.08200 0.06704 0.04838 0.16000 0.14320 0.76340

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO