Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#02)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (299.8 kB) Download raw algorithm evaluation table (22.7 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 2 0.75000 0.68750 0.79640 0.74549 0.80000 0.75000 0.88715
KNN 3 0.75000 0.68750 0.80209 0.75261 0.76923 0.71154 0.88368
KNNW 4 0.66667 0.58333 0.78035 0.72544 0.76923 0.71154 0.88889
KNNW 6 0.83333 0.79167 0.79500 0.74375 0.83333 0.79167 0.88021
LOF 4 0.75000 0.68750 0.76216 0.70270 0.75000 0.68750 0.92882
LOF 33 0.66667 0.58333 0.73224 0.66530 0.80000 0.75000 0.70312
LOF 50 0.66667 0.58333 0.80944 0.76180 0.80000 0.75000 0.87847
SimplifiedLOF 6 0.75000 0.68750 0.76744 0.70930 0.78261 0.72826 0.93576
SimplifiedLOF 8 0.66667 0.58333 0.79716 0.74645 0.74074 0.67593 0.93750
LoOP 6 0.58333 0.47917 0.67255 0.59069 0.69231 0.61538 0.90799
LoOP 17 0.66667 0.58333 0.70866 0.63583 0.66667 0.58333 0.82118
LoOP 56 0.66667 0.58333 0.73333 0.66667 0.80000 0.75000 0.76736
LDOF 16 0.50000 0.37500 0.64899 0.56124 0.61538 0.51923 0.83160
LDOF 21 0.66667 0.58333 0.66643 0.58304 0.66667 0.58333 0.78299
LDOF 56 0.66667 0.58333 0.71385 0.64231 0.73684 0.67105 0.71701
LDOF 59 0.66667 0.58333 0.72077 0.65097 0.73684 0.67105 0.71875
ODIN 46 0.66667 0.58333 0.70182 0.62727 0.70000 0.62500 0.72222
ODIN 51 0.66667 0.58333 0.74928 0.68659 0.80000 0.75000 0.79601
ODIN 52 0.66667 0.58333 0.69650 0.62062 0.66667 0.58333 0.80035
FastABOD 10 0.50000 0.37500 0.65704 0.57129 0.61538 0.51923 0.80208
FastABOD 11 0.58333 0.47917 0.66522 0.58152 0.59259 0.49074 0.80903
KDEOS 16 0.58333 0.47917 0.66268 0.57835 0.69231 0.61538 0.88368
KDEOS 19 0.66667 0.58333 0.67376 0.59220 0.71429 0.64286 0.86458
KDEOS 22 0.66667 0.58333 0.64006 0.55007 0.74074 0.67593 0.84896
KDEOS 59 0.66667 0.58333 0.71577 0.64471 0.69565 0.61957 0.80035
LDF 3 0.75000 0.68750 0.77873 0.72341 0.81481 0.76852 0.93403
LDF 49 0.75000 0.68750 0.82865 0.78581 0.78261 0.72826 0.93403
INFLO 5 0.50000 0.37500 0.64947 0.56184 0.66667 0.58333 0.88368
INFLO 33 0.66667 0.58333 0.72994 0.66243 0.80000 0.75000 0.77865
INFLO 34 0.66667 0.58333 0.73075 0.66343 0.80000 0.75000 0.77865
INFLO 52 0.68167 0.60208 0.72994 0.66243 0.80000 0.75000 0.78906
COF 5 0.66667 0.58333 0.72118 0.65148 0.81481 0.76852 0.93924
COF 13 0.83333 0.79167 0.83354 0.79193 0.83333 0.79167 0.87326

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (299.0 kB) Download raw algorithm evaluation table (24.9 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.33333 0.16667 0.27917 0.09896 0.41379 0.26724 0.58247
KNN 57 0.33333 0.16667 0.35084 0.18855 0.56410 0.45513 0.74479
KNNW 1 0.41667 0.27083 0.34500 0.18125 0.44444 0.30556 0.61458
LOF 2 0.50000 0.37500 0.43991 0.29989 0.54545 0.43182 0.64410
LOF 3 0.41667 0.27083 0.38991 0.23739 0.46154 0.32692 0.67795
SimplifiedLOF 2 0.50000 0.37500 0.41799 0.27248 0.54545 0.43182 0.64236
SimplifiedLOF 8 0.41667 0.27083 0.34683 0.18353 0.46154 0.32692 0.69792
LoOP 3 0.41667 0.27083 0.42247 0.27809 0.46154 0.32692 0.63281
LoOP 4 0.50000 0.37500 0.39983 0.24979 0.52174 0.40217 0.60417
LoOP 8 0.41667 0.27083 0.33401 0.16752 0.44444 0.30556 0.68924
LDOF 3 0.41667 0.27083 0.35921 0.19901 0.47619 0.34524 0.62153
LDOF 13 0.41667 0.27083 0.36162 0.20202 0.51429 0.39286 0.71007
LDOF 16 0.41667 0.27083 0.44263 0.30329 0.50000 0.37500 0.67014
ODIN 2 0.42949 0.28686 0.38628 0.23284 0.43750 0.29687 0.73698
ODIN 3 0.47619 0.34524 0.41867 0.27334 0.48000 0.35000 0.67882
FastABOD 3 0.33333 0.16667 0.23308 0.04135 0.38462 0.23077 0.48785
KDEOS 16 0.50000 0.37500 0.38332 0.22916 0.50000 0.37500 0.68056
KDEOS 17 0.50000 0.37500 0.41561 0.26951 0.50000 0.37500 0.68229
KDEOS 20 0.50000 0.37500 0.44712 0.30890 0.54545 0.43182 0.65972
LDF 2 0.41667 0.27083 0.46117 0.32646 0.45455 0.31818 0.62326
LDF 3 0.41667 0.27083 0.38901 0.23626 0.51429 0.39286 0.74219
LDF 4 0.41667 0.27083 0.40452 0.25565 0.52941 0.41176 0.72049
INFLO 2 0.41667 0.27083 0.41929 0.27411 0.46154 0.32692 0.61719
INFLO 3 0.50000 0.37500 0.40117 0.25146 0.52174 0.40217 0.63194
INFLO 4 0.50000 0.37500 0.40571 0.25714 0.50000 0.37500 0.67361
COF 2 0.50000 0.37500 0.38282 0.22852 0.52174 0.40217 0.60764
COF 12 0.50000 0.37500 0.44291 0.30364 0.56250 0.45313 0.74479

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO