Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Parkinson (10% of outliers version#09)

The data set consists of medical data distinguishing healthy people from those suffering from Parkinson's disease. The latter were labeled as outliers.

Download all data set variants used (278.6 kB). You can also access the original data. (parkinsons.data)

Normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (234.7 kB) Download raw algorithm evaluation table (15.2 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.40000 0.33750 0.61944 0.57980 0.57143 0.52679 0.90417
KNNW 1 0.60000 0.55833 0.66359 0.62855 0.60000 0.55833 0.92500
LOF 3 0.40000 0.33750 0.57762 0.53362 0.66667 0.63194 0.93333
SimplifiedLOF 5 0.40000 0.33750 0.45370 0.39679 0.54545 0.49811 0.89167
SimplifiedLOF 6 0.40000 0.33750 0.59988 0.55821 0.57143 0.52679 0.88750
SimplifiedLOF 8 0.60000 0.55833 0.44222 0.38412 0.60000 0.55833 0.89167
LoOP 5 0.60000 0.55833 0.38762 0.32383 0.60000 0.55833 0.86250
LoOP 14 0.40000 0.33750 0.58370 0.54034 0.57143 0.52679 0.85833
LDOF 7 0.40000 0.33750 0.32548 0.25522 0.54545 0.49811 0.67917
LDOF 12 0.40000 0.33750 0.52406 0.47449 0.57143 0.52679 0.70833
LDOF 13 0.40000 0.33750 0.53533 0.48693 0.57143 0.52679 0.71667
LDOF 16 0.40000 0.33750 0.51374 0.46308 0.57143 0.52679 0.75000
ODIN 5 0.25000 0.17188 0.25000 0.17188 0.40000 0.33750 0.83958
ODIN 7 0.40000 0.33750 0.25042 0.17234 0.40000 0.33750 0.77917
ODIN 20 0.40000 0.33750 0.47841 0.42408 0.57143 0.52679 0.57083
ODIN 23 0.40000 0.33750 0.48561 0.43203 0.57143 0.52679 0.57917
FastABOD 3 0.40000 0.33750 0.33703 0.26797 0.44444 0.38657 0.77500
FastABOD 6 0.40000 0.33750 0.54960 0.50269 0.57143 0.52679 0.87917
FastABOD 51 0.40000 0.33750 0.55560 0.50931 0.57143 0.52679 0.84167
KDEOS 13 0.20000 0.11667 0.41349 0.35240 0.71429 0.68452 0.92083
KDEOS 16 0.60000 0.55833 0.47606 0.42148 0.60000 0.55833 0.92083
KDEOS 20 0.40000 0.33750 0.47980 0.42561 0.62500 0.58594 0.92917
KDEOS 51 0.40000 0.33750 0.52584 0.47645 0.57143 0.52679 0.74583
LDF 4 0.60000 0.55833 0.46167 0.40559 0.60000 0.55833 0.87083
LDF 39 0.40000 0.33750 0.52222 0.47245 0.50000 0.44792 0.88333
LDF 47 0.60000 0.55833 0.55458 0.50819 0.66667 0.63194 0.81250
INFLO 5 0.40000 0.33750 0.29318 0.21955 0.42857 0.36905 0.80417
INFLO 50 0.60000 0.55833 0.55774 0.51167 0.60000 0.55833 0.78333
COF 5 0.60000 0.55833 0.68429 0.65140 0.66667 0.63194 0.91667
COF 6 0.60000 0.55833 0.68333 0.65035 0.72727 0.69886 0.94167
COF 9 0.60000 0.55833 0.60000 0.55833 0.72727 0.69886 0.95000

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates

This version contains 22 attributes, 53 objects, 5 outliers (9.43%)

Download raw algorithm results (233.9 kB) Download raw algorithm evaluation table (19.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.20000 0.11667 0.23284 0.15292 0.42105 0.36075 0.77083
KNN 9 0.20000 0.11667 0.26555 0.18904 0.53333 0.48472 0.74167
KNNW 1 0.20000 0.11667 0.22259 0.14161 0.36364 0.29735 0.69375
KNNW 3 0.20000 0.11667 0.22580 0.14515 0.38095 0.31647 0.76250
KNNW 11 0.20000 0.11667 0.25095 0.17292 0.46154 0.40545 0.72500
LOF 2 0.60000 0.55833 0.34088 0.27222 0.60000 0.55833 0.73333
LOF 13 0.20000 0.11667 0.24277 0.16390 0.40000 0.33750 0.78333
SimplifiedLOF 2 0.40000 0.33750 0.32795 0.25794 0.44444 0.38657 0.78750
SimplifiedLOF 5 0.20000 0.11667 0.26920 0.19308 0.46154 0.40545 0.80833
SimplifiedLOF 17 0.20000 0.11667 0.24712 0.16869 0.50000 0.44792 0.75000
LoOP 3 0.40000 0.33750 0.28037 0.20541 0.44444 0.38657 0.76250
LoOP 4 0.40000 0.33750 0.31892 0.24797 0.44444 0.38657 0.82083
LDOF 4 0.20000 0.11667 0.17834 0.09275 0.28571 0.21131 0.65000
LDOF 7 0.20000 0.11667 0.20068 0.11741 0.33333 0.26389 0.74583
LDOF 13 0.00000 -0.10417 0.18093 0.09561 0.38462 0.32051 0.73333
ODIN 3 0.24444 0.16574 0.17424 0.08823 0.25000 0.17188 0.66875
ODIN 4 0.20000 0.11667 0.20392 0.12100 0.28571 0.21131 0.70833
ODIN 9 0.00000 -0.10417 0.17303 0.08688 0.32258 0.25202 0.71875
FastABOD 3 0.00000 -0.10417 0.15375 0.06560 0.27586 0.20043 0.64167
FastABOD 15 0.00000 -0.10417 0.18777 0.10317 0.33333 0.26389 0.68333
FastABOD 38 0.00000 -0.10417 0.19612 0.11238 0.33333 0.26389 0.70000
KDEOS 2 0.40000 0.33750 0.24889 0.17065 0.40000 0.33750 0.66667
KDEOS 8 0.20000 0.11667 0.25138 0.17340 0.41667 0.35590 0.81667
KDEOS 24 0.20000 0.11667 0.32298 0.25246 0.53333 0.48472 0.78333
KDEOS 27 0.20000 0.11667 0.41571 0.35485 0.50000 0.44792 0.78333
LDF 2 0.60000 0.55833 0.60132 0.55979 0.66667 0.63194 0.74167
LDF 10 0.20000 0.11667 0.31800 0.24696 0.57143 0.52679 0.81250
INFLO 2 0.40000 0.33750 0.25805 0.18077 0.44444 0.38657 0.70833
INFLO 4 0.20000 0.11667 0.31168 0.23998 0.50000 0.44792 0.85833
INFLO 15 0.40000 0.33750 0.32421 0.25382 0.57143 0.52679 0.74583
COF 4 0.60000 0.55833 0.35716 0.29020 0.60000 0.55833 0.80208
COF 7 0.20000 0.11667 0.39946 0.33691 0.44444 0.38657 0.82083

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO