Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Pima (2% of outliers version#01)

The data set contains medical data on diabetes. Patients suffering from diabetes were considered outliers.

Download all data set variants used (694.8 kB). You can also access the original data. (pima-indians-diabetes.data)

Normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.5 MB) Download raw algorithm evaluation table (42.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.04751 0.02846 0.12121 0.10364 0.74340
KNN 3 0.00000 -0.02000 0.04432 0.02521 0.13333 0.11600 0.72020
KNNW 1 0.00000 -0.02000 0.04927 0.03025 0.12658 0.10911 0.72250
KNNW 2 0.00000 -0.02000 0.04931 0.03029 0.11594 0.09826 0.74860
KNNW 3 0.00000 -0.02000 0.04873 0.02970 0.13559 0.11831 0.74520
LOF 1 0.00000 -0.02000 0.03053 0.01114 0.08696 0.06870 0.58330
LOF 3 0.00000 -0.02000 0.04095 0.02177 0.13043 0.11304 0.60260
LOF 5 0.00000 -0.02000 0.04114 0.02197 0.11765 0.10000 0.66120
LOF 39 0.00000 -0.02000 0.03371 0.01438 0.08889 0.07067 0.67920
SimplifiedLOF 1 0.00000 -0.02000 0.02757 0.00812 0.06164 0.04288 0.62260
SimplifiedLOF 3 0.00000 -0.02000 0.03555 0.01626 0.11236 0.09461 0.63220
SimplifiedLOF 4 0.00000 -0.02000 0.03967 0.02046 0.09877 0.08074 0.70660
SimplifiedLOF 5 0.00000 -0.02000 0.03974 0.02053 0.09231 0.07415 0.70260
LoOP 1 0.00000 -0.02000 0.02755 0.00810 0.06143 0.04266 0.62200
LoOP 5 0.00000 -0.02000 0.04183 0.02267 0.11538 0.09769 0.70980
LDOF 2 0.00000 -0.02000 0.02568 0.00619 0.06667 0.04800 0.52240
LDOF 7 0.00000 -0.02000 0.03490 0.01560 0.12121 0.10364 0.61360
LDOF 8 0.00000 -0.02000 0.03771 0.01846 0.11765 0.10000 0.66100
LDOF 10 0.00000 -0.02000 0.03459 0.01529 0.08333 0.06500 0.67040
ODIN 2 0.05195 0.03299 0.04506 0.02597 0.09195 0.07379 0.78030
FastABOD 3 0.10000 0.08200 0.07644 0.05797 0.19355 0.17742 0.71040
FastABOD 5 0.00000 -0.02000 0.07666 0.05819 0.19512 0.17902 0.76160
FastABOD 6 0.10000 0.08200 0.08036 0.06197 0.19355 0.17742 0.76820
FastABOD 11 0.00000 -0.02000 0.05686 0.03800 0.12121 0.10364 0.76880
KDEOS 2 0.10000 0.08200 0.07310 0.05456 0.16667 0.15000 0.58700
KDEOS 3 0.10000 0.08200 0.12372 0.10619 0.18182 0.16545 0.57900
KDEOS 13 0.00000 -0.02000 0.03193 0.01257 0.06393 0.04521 0.66240
LDF 1 0.00000 -0.02000 0.02355 0.00402 0.05281 0.03386 0.56470
LDF 2 0.00000 -0.02000 0.04685 0.02778 0.14286 0.12571 0.68640
LDF 39 0.00000 -0.02000 0.03852 0.01929 0.10000 0.08200 0.70820
INFLO 1 0.00000 -0.02000 0.03050 0.01111 0.07547 0.05698 0.58840
INFLO 5 0.00000 -0.02000 0.03884 0.01961 0.09677 0.07871 0.67820
INFLO 8 0.00000 -0.02000 0.03684 0.01757 0.07692 0.05846 0.71460
INFLO 24 0.00000 -0.02000 0.03657 0.01730 0.09756 0.07951 0.70620
COF 60 0.00000 -0.02000 0.05526 0.03637 0.14545 0.12836 0.78040
COF 84 0.00000 -0.02000 0.06342 0.04469 0.14286 0.12571 0.78400
COF 85 0.00000 -0.02000 0.06282 0.04408 0.12903 0.11161 0.78640
COF 88 0.10000 0.08200 0.06033 0.04153 0.12766 0.11021 0.76880

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 8 attributes, 510 objects, 10 outliers (1.96%)

Download raw algorithm results (4.4 MB) Download raw algorithm evaluation table (41.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.02000 0.04813 0.02909 0.12121 0.10364 0.71920
KNN 2 0.00000 -0.02000 0.05643 0.03755 0.13953 0.12233 0.72000
KNN 5 0.00000 -0.02000 0.05486 0.03596 0.17778 0.16133 0.70080
KNN 98 0.00000 -0.02000 0.05556 0.03667 0.11628 0.09860 0.72680
KNNW 1 0.00000 -0.02000 0.05924 0.04042 0.13333 0.11600 0.77390
KNNW 12 0.00000 -0.02000 0.05359 0.03467 0.15385 0.13692 0.71480
LOF 1 0.00000 -0.02000 0.03039 0.01100 0.07692 0.05846 0.61140
LOF 88 0.00000 -0.02000 0.04288 0.02374 0.09524 0.07714 0.69080
LOF 99 0.00000 -0.02000 0.04496 0.02586 0.09524 0.07714 0.69660
LOF 100 0.00000 -0.02000 0.04503 0.02593 0.09524 0.07714 0.69600
SimplifiedLOF 1 0.00000 -0.02000 0.03334 0.01401 0.08696 0.06870 0.67780
SimplifiedLOF 44 0.00000 -0.02000 0.03291 0.01357 0.10526 0.08737 0.59940
SimplifiedLOF 100 0.00000 -0.02000 0.04033 0.02114 0.10000 0.08200 0.64440
LoOP 1 0.00000 -0.02000 0.03331 0.01398 0.08696 0.06870 0.67700
LoOP 92 0.00000 -0.02000 0.04061 0.02142 0.13793 0.12069 0.61300
LoOP 100 0.00000 -0.02000 0.04129 0.02211 0.13793 0.12069 0.62100
LDOF 2 0.00000 -0.02000 0.02378 0.00426 0.07407 0.05556 0.46540
LDOF 33 0.00000 -0.02000 0.03216 0.01281 0.06897 0.05034 0.63200
LDOF 78 0.00000 -0.02000 0.04007 0.02087 0.14286 0.12571 0.60880
LDOF 100 0.00000 -0.02000 0.04248 0.02333 0.14286 0.12571 0.62040
ODIN 1 0.03614 0.01687 0.03006 0.01066 0.06818 0.04955 0.62920
ODIN 6 0.09091 0.07273 0.03346 0.01412 0.09524 0.07714 0.60360
ODIN 25 0.00000 -0.02000 0.03308 0.01374 0.10811 0.09027 0.55380
ODIN 26 0.00000 -0.02000 0.03349 0.01416 0.10811 0.09027 0.55700
FastABOD 3 0.00000 -0.02000 0.05183 0.03287 0.12903 0.11161 0.75840
FastABOD 7 0.00000 -0.02000 0.06284 0.04410 0.13559 0.11831 0.78620
FastABOD 66 0.00000 -0.02000 0.06573 0.04705 0.17021 0.15362 0.77820
FastABOD 82 0.00000 -0.02000 0.06650 0.04783 0.17021 0.15362 0.77980
KDEOS 6 0.00000 -0.02000 0.04134 0.02217 0.08889 0.07067 0.71260
KDEOS 10 0.00000 -0.02000 0.04671 0.02764 0.17391 0.15739 0.63660
KDEOS 11 0.10000 0.08200 0.05281 0.03386 0.13333 0.11600 0.64240
LDF 1 0.00000 -0.02000 0.03065 0.01127 0.08000 0.06160 0.60700
LDF 84 0.00000 -0.02000 0.04730 0.02825 0.09524 0.07714 0.71820
LDF 99 0.00000 -0.02000 0.04900 0.02998 0.11111 0.09333 0.71700
LDF 100 0.00000 -0.02000 0.04913 0.03011 0.11111 0.09333 0.71720
INFLO 1 0.00000 -0.02000 0.02344 0.00391 0.06557 0.04689 0.57920
INFLO 30 0.00000 -0.02000 0.03822 0.01899 0.10714 0.08929 0.70080
INFLO 74 0.00000 -0.02000 0.04354 0.02441 0.14286 0.12571 0.61760
INFLO 99 0.00000 -0.02000 0.04774 0.02869 0.12500 0.10750 0.69800
COF 1 0.00000 -0.02000 0.03334 0.01401 0.08696 0.06870 0.67770
COF 13 0.00000 -0.02000 0.03418 0.01486 0.07843 0.06000 0.64820
COF 19 0.00000 -0.02000 0.02990 0.01050 0.09524 0.07714 0.55380

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO