Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#07)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.0 kB) Download raw algorithm evaluation table (15.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.60000 0.55833 0.50714 0.45580 0.66667 0.63194 0.88333
KNNW 1 0.40000 0.33750 0.44127 0.38307 0.50000 0.44792 0.88958
KNNW 2 0.60000 0.55833 0.49311 0.44031 0.66667 0.63194 0.88750
LOF 15 0.60000 0.55833 0.35992 0.29325 0.60000 0.55833 0.64167
LOF 18 0.60000 0.55833 0.41444 0.35345 0.60000 0.55833 0.85833
LOF 51 0.40000 0.33750 0.45532 0.39859 0.57143 0.52679 0.51250
SimplifiedLOF 5 0.40000 0.33750 0.36987 0.30423 0.50000 0.44792 0.88333
SimplifiedLOF 6 0.40000 0.33750 0.41597 0.35513 0.54545 0.49811 0.88333
SimplifiedLOF 22 0.60000 0.55833 0.36854 0.30276 0.60000 0.55833 0.70833
LoOP 6 0.40000 0.33750 0.36889 0.30315 0.54545 0.49811 0.87917
LoOP 7 0.20000 0.11667 0.43829 0.37978 0.53333 0.48472 0.82917
LoOP 24 0.60000 0.55833 0.35774 0.29083 0.60000 0.55833 0.66250
LDOF 10 0.20000 0.11667 0.26458 0.18798 0.42857 0.36905 0.78333
LDOF 25 0.60000 0.55833 0.36551 0.29942 0.60000 0.55833 0.68750
ODIN 24 0.60000 0.55833 0.36183 0.29535 0.60000 0.55833 0.66458
ODIN 44 0.60000 0.55833 0.45774 0.40125 0.60000 0.55833 0.67500
ODIN 50 0.20000 0.11667 0.27940 0.20434 0.46154 0.40545 0.73958
FastABOD 4 0.60000 0.55833 0.42812 0.36854 0.60000 0.55833 0.82083
FastABOD 6 0.60000 0.55833 0.45989 0.40363 0.66667 0.63194 0.82917
FastABOD 50 0.60000 0.55833 0.54037 0.49249 0.60000 0.55833 0.84583
KDEOS 12 0.20000 0.11667 0.40748 0.34576 0.57143 0.52679 0.90417
KDEOS 50 0.60000 0.55833 0.37461 0.30947 0.60000 0.55833 0.74167
LDF 7 0.60000 0.55833 0.36854 0.30276 0.60000 0.55833 0.70833
LDF 19 0.60000 0.55833 0.46965 0.41441 0.60000 0.55833 0.91667
LDF 45 0.60000 0.55833 0.50889 0.45773 0.72727 0.69886 0.91667
INFLO 1 0.20000 0.11667 0.22745 0.14698 0.37500 0.30990 0.77500
INFLO 22 0.60000 0.55833 0.35590 0.28880 0.60000 0.55833 0.67083
INFLO 34 0.60000 0.55833 0.39601 0.33310 0.60000 0.55833 0.73750
COF 41 0.60000 0.55833 0.38930 0.32568 0.60000 0.55833 0.86875
COF 42 0.60000 0.55833 0.39882 0.33620 0.60000 0.55833 0.88125
COF 46 0.60000 0.55833 0.52556 0.47613 0.60000 0.55833 0.87917

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.4 kB) Download raw algorithm evaluation table (19.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.00000 -0.10417 0.16341 0.07626 0.28571 0.21131 0.65833
KNN 3 0.20000 0.11667 0.18504 0.10015 0.36364 0.29735 0.63750
KNNW 1 0.20000 0.11667 0.21305 0.13108 0.36364 0.29735 0.70833
LOF 1 0.40000 0.33750 0.26226 0.18542 0.40000 0.33750 0.73333
LOF 2 0.40000 0.33750 0.36726 0.30135 0.44444 0.38657 0.60000
SimplifiedLOF 1 0.20000 0.11667 0.33139 0.26174 0.46154 0.40545 0.82917
SimplifiedLOF 2 0.40000 0.33750 0.31955 0.24867 0.44444 0.38657 0.77917
SimplifiedLOF 4 0.20000 0.11667 0.38180 0.31740 0.37037 0.30478 0.77500
LoOP 1 0.20000 0.11667 0.33139 0.26174 0.46154 0.40545 0.82917
LoOP 2 0.40000 0.33750 0.33167 0.26205 0.44444 0.38657 0.80417
LoOP 4 0.40000 0.33750 0.41879 0.35824 0.44444 0.38657 0.79167
LDOF 2 0.40000 0.33750 0.33468 0.26537 0.44444 0.38657 0.81667
LDOF 3 0.40000 0.33750 0.37404 0.30883 0.47619 0.42163 0.86667
LDOF 4 0.40000 0.33750 0.46670 0.41115 0.45455 0.39773 0.86250
LDOF 5 0.40000 0.33750 0.44820 0.39072 0.50000 0.44792 0.77083
ODIN 1 0.29412 0.22059 0.29412 0.22059 0.45455 0.39773 0.87500
ODIN 2 0.42353 0.36348 0.30397 0.23146 0.44444 0.38657 0.81250
FastABOD 3 0.20000 0.11667 0.22364 0.14277 0.40000 0.33750 0.71250
KDEOS 5 0.40000 0.33750 0.46004 0.40379 0.47059 0.41544 0.87083
KDEOS 6 0.40000 0.33750 0.38095 0.31647 0.50000 0.44792 0.83750
KDEOS 7 0.40000 0.33750 0.46930 0.41402 0.44444 0.38657 0.85833
LDF 2 0.20000 0.11667 0.32584 0.25562 0.33333 0.26389 0.63333
LDF 9 0.40000 0.33750 0.22271 0.14174 0.40000 0.33750 0.65417
LDF 20 0.40000 0.33750 0.26292 0.18614 0.46154 0.40545 0.73333
LDF 21 0.40000 0.33750 0.27000 0.19396 0.42857 0.36905 0.78333
INFLO 1 0.40000 0.33750 0.29519 0.22177 0.40000 0.33750 0.76250
INFLO 2 0.40000 0.33750 0.43336 0.37433 0.44444 0.38657 0.78750
INFLO 4 0.20000 0.11667 0.40288 0.34068 0.37500 0.30990 0.80417
COF 1 0.20000 0.11667 0.33139 0.26174 0.46154 0.40545 0.82917
COF 2 0.40000 0.33750 0.33347 0.26404 0.44444 0.38657 0.79583
COF 4 0.40000 0.33750 0.40978 0.34830 0.44444 0.38657 0.77083

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO