Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (20% of outliers version#05)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (298.9 kB) Download raw algorithm evaluation table (23.3 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.58333 0.72788 0.65985 0.76923 0.71154 0.90972
KNNW 1 0.66667 0.58333 0.74114 0.67643 0.70000 0.62500 0.91146
KNNW 2 0.66667 0.58333 0.79416 0.74270 0.72727 0.65909 0.92188
KNNW 3 0.66667 0.58333 0.76497 0.70621 0.74074 0.67593 0.91667
LOF 3 0.58333 0.47917 0.73497 0.66871 0.80000 0.75000 0.93576
SimplifiedLOF 6 0.75000 0.68750 0.73443 0.66804 0.81818 0.77273 0.90104
SimplifiedLOF 8 0.75000 0.68750 0.73560 0.66950 0.75000 0.68750 0.92014
LoOP 6 0.75000 0.68750 0.72074 0.65093 0.76190 0.70238 0.89583
LDOF 24 0.50000 0.37500 0.58472 0.48091 0.56000 0.45000 0.78472
LDOF 26 0.58333 0.47917 0.59232 0.49040 0.58333 0.47917 0.77778
LDOF 58 0.58333 0.47917 0.63937 0.54921 0.70000 0.62500 0.68403
ODIN 5 0.51667 0.39583 0.47555 0.34443 0.52632 0.40789 0.76215
ODIN 23 0.58333 0.47917 0.52135 0.40169 0.58333 0.47917 0.62066
ODIN 50 0.58333 0.47917 0.65067 0.56334 0.70000 0.62500 0.69878
ODIN 51 0.58333 0.47917 0.66625 0.58281 0.70000 0.62500 0.71615
FastABOD 7 0.50000 0.37500 0.73582 0.66977 0.71429 0.64286 0.87153
FastABOD 14 0.66667 0.58333 0.74841 0.68551 0.66667 0.58333 0.86632
KDEOS 9 0.66667 0.58333 0.77238 0.71547 0.70000 0.62500 0.88368
KDEOS 11 0.66667 0.58333 0.76956 0.71195 0.72727 0.65909 0.88194
KDEOS 14 0.66667 0.58333 0.67963 0.59954 0.72000 0.65000 0.91319
LDF 22 0.66667 0.58333 0.79133 0.73916 0.76190 0.70238 0.85417
LDF 24 0.66667 0.58333 0.79679 0.74599 0.80000 0.75000 0.81076
LDF 25 0.75000 0.68750 0.79612 0.74514 0.80000 0.75000 0.81076
INFLO 6 0.50000 0.37500 0.61385 0.51731 0.63158 0.53947 0.86111
INFLO 29 0.66667 0.58333 0.56658 0.45823 0.66667 0.58333 0.75087
INFLO 37 0.58333 0.47917 0.65622 0.57028 0.73684 0.67105 0.65451
INFLO 41 0.58333 0.47917 0.72869 0.66087 0.73684 0.67105 0.80208
COF 7 0.58333 0.47917 0.71205 0.64007 0.73333 0.66667 0.89757
COF 10 0.66667 0.58333 0.79871 0.74838 0.73684 0.67105 0.87847
COF 11 0.66667 0.58333 0.78909 0.73636 0.76190 0.70238 0.86806
COF 12 0.75000 0.68750 0.76239 0.70299 0.75000 0.68750 0.85590

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 60 objects, 12 outliers (20.00%)

Download raw algorithm results (298.2 kB) Download raw algorithm evaluation table (25.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.33333 0.16667 0.36126 0.20157 0.54545 0.43182 0.74219
KNN 6 0.50000 0.37500 0.36477 0.20597 0.50000 0.37500 0.70573
KNN 7 0.50000 0.37500 0.40453 0.25566 0.56000 0.45000 0.73958
KNNW 1 0.41667 0.27083 0.38488 0.23110 0.48276 0.35345 0.69792
KNNW 9 0.41667 0.27083 0.37085 0.21356 0.55172 0.43966 0.73264
KNNW 10 0.50000 0.37500 0.37397 0.21746 0.55172 0.43966 0.72743
KNNW 12 0.50000 0.37500 0.37898 0.22373 0.57143 0.46429 0.72396
LOF 11 0.50000 0.37500 0.36323 0.20404 0.52174 0.40217 0.71528
LOF 12 0.58333 0.47917 0.37091 0.21364 0.58333 0.47917 0.68576
LOF 13 0.50000 0.37500 0.37672 0.22090 0.52174 0.40217 0.69271
SimplifiedLOF 1 0.41667 0.27083 0.48715 0.35893 0.50000 0.37500 0.62153
SimplifiedLOF 15 0.41667 0.27083 0.34646 0.18308 0.51852 0.39815 0.68056
SimplifiedLOF 16 0.50000 0.37500 0.36556 0.20695 0.51613 0.39516 0.69792
SimplifiedLOF 21 0.41667 0.27083 0.36822 0.21028 0.50000 0.37500 0.71354
LoOP 1 0.41667 0.27083 0.49059 0.36324 0.50000 0.37500 0.63802
LoOP 17 0.33333 0.16667 0.31018 0.13773 0.46154 0.32692 0.64931
LDOF 2 0.50000 0.37500 0.51325 0.39156 0.54545 0.43182 0.63889
ODIN 7 0.33333 0.16667 0.33993 0.17492 0.42105 0.27632 0.61024
ODIN 10 0.25000 0.06250 0.29919 0.12399 0.46154 0.32692 0.66753
ODIN 11 0.33333 0.16667 0.28847 0.11059 0.47368 0.34211 0.66580
FastABOD 3 0.33333 0.16667 0.33985 0.17481 0.50000 0.37500 0.67361
FastABOD 6 0.33333 0.16667 0.36189 0.20236 0.43478 0.29348 0.64583
KDEOS 3 0.41667 0.27083 0.45937 0.32421 0.47619 0.34524 0.63194
KDEOS 24 0.41667 0.27083 0.40640 0.25799 0.54545 0.43182 0.73785
KDEOS 33 0.41667 0.27083 0.37622 0.22027 0.55172 0.43966 0.70312
LDF 8 0.58333 0.47917 0.47872 0.34840 0.62500 0.53125 0.80903
LDF 9 0.58333 0.47917 0.47335 0.34168 0.64000 0.55000 0.81424
INFLO 1 0.25000 0.06250 0.42085 0.27606 0.40816 0.26020 0.62240
INFLO 12 0.50000 0.37500 0.34381 0.17976 0.53333 0.41667 0.61719
INFLO 19 0.41667 0.27083 0.31974 0.14967 0.53846 0.42308 0.69965
COF 1 0.41667 0.27083 0.48715 0.35893 0.50000 0.37500 0.62153
COF 12 0.50000 0.37500 0.41094 0.26368 0.59259 0.49074 0.72483
COF 14 0.50000 0.37500 0.41958 0.27448 0.61538 0.51923 0.73438
COF 16 0.50000 0.37500 0.39698 0.24622 0.54545 0.43182 0.74045

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO