Supplementary Material for
On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study
by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle
Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8

Lymphography

This dataset represents patients divided into four classes according to radiological examination results. Two classes (1 and 4) are represented by only 6 instances. These classes were jointly considered as outliers. In this way, the dataset was first used by Lazarevic and Kumar [1], and then also in [2,3]. (Note: Lazarevic and Kumar name classes 2 and 4 as outliers but their experimental results suggest that they actually used classes 1 and 4, as we do here. The processed database has 3 numerical attributes, 16 categorical attributes and 148 instances, namely 6 outliers (4.05%) and 142 inliers (95.95%).

References:

[1] A. Lazarevic and V. Kumar. Feature bagging for outlier detection. In Proc. KDD, pages 157-166, 2005.
[2] H. V. Nguyen, H. H. Ang, and V. Gopalkrishnan. Mining outliers with ensemble of heterogeneous detectors on random subspaces. In Proc. DASFAA, pages 368-383, 2010.
[3] A. Zimek, M. Gaudet, R. J. G. B. Campello, and J. Sander. Subsampling for efficient and effective unsupervised outlier detection ensembles. In Proc. KDD, pages 428-436, 2013.

Download all data set variants used (13.0 kB). You can also access the original data. (lymphography-data)

Normalized, without duplicates, idf weighted categorial attributes

This version contains 18 attributes, 148 objects, 6 outliers (4.05%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (17.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 14 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
KNNW 39 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LOF 62 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
SimplifiedLOF 98 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LoOP 14 0.83333 0.82629 0.62500 0.60915 0.83333 0.82629 0.96596
LoOP 43 0.83333 0.82629 0.94444 0.94210 0.90909 0.90525 0.99648
LoOP 47 0.83333 0.82629 0.95833 0.95657 0.90909 0.90525 0.99765
LDOF 56 0.83333 0.82629 0.85556 0.84945 0.85714 0.85111 0.99413
LDOF 66 0.83333 0.82629 0.91508 0.91149 0.92308 0.91983 0.99648
LDOF 86 0.83333 0.82629 0.95833 0.95657 0.90909 0.90525 0.99765
ODIN 16 0.83333 0.82629 0.83258 0.82551 0.83333 0.82629 0.96479
ODIN 52 0.83333 0.82629 0.94841 0.94623 0.92308 0.91983 0.99765
ODIN 55 0.83333 0.82629 0.97619 0.97518 0.92308 0.91983 0.99883
FastABOD 23 0.83333 0.82629 0.93056 0.92762 0.85714 0.85111 0.99648
FastABOD 25 0.83333 0.82629 0.94841 0.94623 0.92308 0.91983 0.99765
KDEOS 73 0.33333 0.30516 0.47652 0.45440 0.66667 0.65258 0.97066
KDEOS 96 0.66667 0.65258 0.53046 0.51062 0.66667 0.65258 0.97653
KDEOS 99 0.66667 0.65258 0.65840 0.64397 0.66667 0.65258 0.98122
LDF 13 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
INFLO 15 0.83333 0.82629 0.74643 0.73571 0.83333 0.82629 0.98592
INFLO 60 0.83333 0.82629 0.94841 0.94623 0.92308 0.91983 0.99765
INFLO 62 0.83333 0.82629 0.97619 0.97518 0.92308 0.91983 0.99883
COF 40 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Normalized, without duplicates, 1-of-n encoding

This version contains 47 attributes, 148 objects, 6 outliers (4.05%)

Download raw algorithm results (1.1 MB) Download raw algorithm evaluation table (20.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 3 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
KNNW 5 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LOF 23 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
SimplifiedLOF 46 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LoOP 43 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LDOF 37 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
ODIN 17 0.83333 0.82629 0.76026 0.75013 0.83333 0.82629 0.98885
ODIN 21 0.83333 0.82629 0.91667 0.91315 0.90909 0.90525 0.99531
ODIN 40 0.83333 0.82629 0.95833 0.95657 0.90909 0.90525 0.99765
FastABOD 13 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
KDEOS 99 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
LDF 16 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
INFLO 34 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
COF 53 0.66667 0.65258 0.80492 0.79668 0.80000 0.79155 0.98122
COF 56 0.83333 0.82629 0.79603 0.78741 0.83333 0.82629 0.95540

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Normalized, without duplicates, categorial attributes removed

This version contains 3 attributes, 148 objects, 6 outliers (4.05%)

Download raw algorithm results (331.4 kB) Download raw algorithm evaluation table (26.4 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.65258 0.40240 0.37715 0.66667 0.65258 0.81045
KNN 2 0.66667 0.65258 0.60518 0.58850 0.72727 0.71575 0.80634
KNN 6 0.66667 0.65258 0.67632 0.66265 0.72727 0.71575 0.87148
KNN 7 0.66667 0.65258 0.68639 0.67314 0.72727 0.71575 0.86326
KNNW 3 0.66667 0.65258 0.37462 0.34820 0.66667 0.65258 0.79930
KNNW 5 0.66667 0.65258 0.48389 0.46209 0.72727 0.71575 0.83744
KNNW 13 0.66667 0.65258 0.67342 0.65962 0.72727 0.71575 0.84683
KNNW 20 0.66667 0.65258 0.67150 0.65762 0.72727 0.71575 0.86678
LOF 47 0.66667 0.65258 0.67610 0.66241 0.66667 0.65258 0.88204
LOF 50 0.66667 0.65258 0.68007 0.66655 0.66667 0.65258 0.88322
LOF 67 0.66667 0.65258 0.65579 0.64124 0.72727 0.71575 0.82570
SimplifiedLOF 47 0.53333 0.51362 0.59494 0.57783 0.60000 0.58310 0.93369
SimplifiedLOF 68 0.66667 0.65258 0.67263 0.65880 0.66667 0.65258 0.88087
SimplifiedLOF 86 0.66667 0.65258 0.67610 0.66241 0.72727 0.71575 0.87148
SimplifiedLOF 88 0.66667 0.65258 0.68065 0.66715 0.72727 0.71575 0.87500
LoOP 43 0.43333 0.40939 0.41686 0.39222 0.53333 0.51362 0.91725
LoOP 69 0.50000 0.47887 0.62689 0.61112 0.66667 0.65258 0.85270
LoOP 71 0.50000 0.47887 0.66152 0.64722 0.66667 0.65258 0.85739
LoOP 77 0.66667 0.65258 0.65755 0.64308 0.66667 0.65258 0.85035
LDOF 42 0.43333 0.40939 0.40540 0.38027 0.50000 0.47887 0.91256
LDOF 71 0.50000 0.47887 0.61784 0.60170 0.66667 0.65258 0.88322
LDOF 88 0.66667 0.65258 0.67610 0.66241 0.66667 0.65258 0.88204
LDOF 91 0.66667 0.65258 0.68465 0.67132 0.66667 0.65258 0.88439
ODIN 19 0.66667 0.65258 0.46062 0.43783 0.66667 0.65258 0.89495
ODIN 32 0.66667 0.65258 0.55248 0.53357 0.66667 0.65258 0.92371
ODIN 97 0.66667 0.65258 0.63771 0.62240 0.66667 0.65258 0.82570
FastABOD 39 0.05000 0.00986 0.12236 0.08528 0.32258 0.29396 0.73650
FastABOD 43 0.66667 0.65258 0.64609 0.63114 0.72727 0.71575 0.68838
KDEOS 82 0.43333 0.40939 0.44470 0.42123 0.50000 0.47887 0.80927
KDEOS 85 0.43333 0.40939 0.44681 0.42343 0.50000 0.47887 0.81279
KDEOS 100 0.33333 0.30516 0.28774 0.25765 0.40000 0.37465 0.83862
LDF 7 0.66667 0.65258 0.57298 0.55494 0.66667 0.65258 0.81338
LDF 56 0.66667 0.65258 0.67667 0.66300 0.66667 0.65258 0.88908
LDF 67 0.66667 0.65258 0.65879 0.64438 0.72727 0.71575 0.83979
INFLO 41 0.50000 0.47887 0.59971 0.58279 0.66667 0.65258 0.81866
INFLO 47 0.66667 0.65258 0.66693 0.65286 0.66667 0.65258 0.86385
INFLO 54 0.66667 0.65258 0.68216 0.66873 0.66667 0.65258 0.87324
COF 55 0.56667 0.54836 0.56987 0.55170 0.60000 0.58310 0.94190
COF 80 0.70000 0.68732 0.68833 0.67516 0.72727 0.71575 0.92430
COF 83 0.70000 0.68732 0.68949 0.67637 0.72727 0.71575 0.93134

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates, idf weighted categorial attributes

This version contains 18 attributes, 148 objects, 6 outliers (4.05%)

Download raw algorithm results (1.3 MB) Download raw algorithm evaluation table (22.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.83333 0.82629 0.88889 0.88419 0.83333 0.82629 0.99237
KNN 4 0.83333 0.82629 0.97619 0.97518 0.92308 0.91983 0.99883
KNNW 1 0.83333 0.82629 0.90476 0.90074 0.90909 0.90525 0.99061
KNNW 8 0.83333 0.82629 0.95833 0.95657 0.90909 0.90525 0.99765
KNNW 22 0.83333 0.82629 0.94841 0.94623 0.92308 0.91983 0.99765
LOF 35 0.83333 0.82629 0.80198 0.79362 0.83333 0.82629 0.98709
LOF 47 0.83333 0.82629 0.85556 0.84945 0.85714 0.85111 0.99413
LOF 55 0.83333 0.82629 0.89722 0.89288 0.85714 0.85111 0.99531
LOF 63 0.66667 0.65258 0.91071 0.90694 0.85714 0.85111 0.99531
SimplifiedLOF 26 0.83333 0.82629 0.73750 0.72641 0.83333 0.82629 0.98357
SimplifiedLOF 59 0.83333 0.82629 0.85556 0.84945 0.85714 0.85111 0.99413
SimplifiedLOF 71 0.66667 0.65258 0.91071 0.90694 0.85714 0.85111 0.99531
LoOP 48 0.83333 0.82629 0.75192 0.74144 0.83333 0.82629 0.98709
LoOP 57 0.83333 0.82629 0.84167 0.83498 0.83333 0.82629 0.99296
LoOP 69 0.66667 0.65258 0.83571 0.82877 0.85714 0.85111 0.99296
LDOF 60 0.66667 0.65258 0.66481 0.65065 0.66667 0.65258 0.98239
LDOF 74 0.66667 0.65258 0.70417 0.69167 0.80000 0.79155 0.98709
LDOF 87 0.66667 0.65258 0.77460 0.76508 0.80000 0.79155 0.98944
ODIN 93 0.83333 0.82629 0.82146 0.81392 0.83333 0.82629 0.99120
ODIN 96 0.83333 0.82629 0.84167 0.83498 0.83333 0.82629 0.99296
FastABOD 14 0.66667 0.65258 0.69713 0.68433 0.66667 0.65258 0.98005
FastABOD 51 0.66667 0.65258 0.78849 0.77956 0.85714 0.85111 0.99061
KDEOS 52 0.16667 0.13146 0.16484 0.12955 0.30000 0.27042 0.86385
KDEOS 88 0.16667 0.13146 0.19522 0.16122 0.37500 0.34859 0.90141
KDEOS 100 0.16667 0.13146 0.23771 0.20550 0.35714 0.32998 0.91315
LDF 6 0.83333 0.82629 0.82146 0.81392 0.83333 0.82629 0.99061
LDF 9 0.83333 0.82629 0.81786 0.81016 0.92308 0.91983 0.99413
LDF 25 0.83333 0.82629 0.95833 0.95657 0.90909 0.90525 0.99765
INFLO 45 0.83333 0.82629 0.80198 0.79362 0.83333 0.82629 0.98709
INFLO 49 0.83333 0.82629 0.82146 0.81392 0.83333 0.82629 0.99061
INFLO 61 0.66667 0.65258 0.81349 0.80561 0.85714 0.85111 0.99178
COF 11 0.83333 0.82629 0.84903 0.84265 0.83333 0.82629 0.97887
COF 25 0.83333 0.82629 0.93333 0.93052 0.90909 0.90525 0.99531

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates, 1-of-n encoding

This version contains 47 attributes, 148 objects, 6 outliers (4.05%)

Download raw algorithm results (1.0 MB) Download raw algorithm evaluation table (33.0 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.65258 0.74676 0.73606 0.72727 0.71575 0.98592
KNN 2 0.66667 0.65258 0.79259 0.78383 0.72727 0.71575 0.98709
KNNW 1 0.75000 0.73944 0.71253 0.70038 0.76923 0.75948 0.98063
KNNW 4 0.66667 0.65258 0.82083 0.81326 0.72727 0.71575 0.98885
LOF 12 0.83333 0.82629 0.88333 0.87840 0.83333 0.82629 0.99413
SimplifiedLOF 19 0.83333 0.82629 0.83889 0.83208 0.83333 0.82629 0.98709
SimplifiedLOF 37 0.66667 0.65258 0.80370 0.79541 0.75000 0.73944 0.98826
LoOP 8 0.50000 0.47887 0.57037 0.55222 0.75000 0.73944 0.98122
LoOP 13 0.66667 0.65258 0.78535 0.77628 0.70588 0.69345 0.98592
LoOP 24 0.66667 0.65258 0.78704 0.77804 0.66667 0.65258 0.98592
LDOF 12 0.83333 0.82629 0.66859 0.65459 0.83333 0.82629 0.98592
LDOF 13 0.83333 0.82629 0.80198 0.79362 0.83333 0.82629 0.98709
ODIN 17 0.83333 0.82629 0.75327 0.74284 0.83333 0.82629 0.98415
ODIN 20 0.83333 0.82629 0.90000 0.89577 0.90909 0.90525 0.99120
ODIN 25 0.83333 0.82629 0.91026 0.90646 0.90909 0.90525 0.99237
ODIN 27 0.83333 0.82629 0.86869 0.86314 0.83333 0.82629 0.99472
FastABOD 7 0.33333 0.30516 0.52539 0.50534 0.57143 0.55332 0.85446
FastABOD 32 0.66667 0.65258 0.52002 0.49974 0.66667 0.65258 0.91432
FastABOD 48 0.66667 0.65258 0.51908 0.49876 0.66667 0.65258 0.92488
KDEOS 58 0.83333 0.82629 0.78938 0.78048 0.83333 0.82629 0.98357
KDEOS 62 0.83333 0.82629 0.89583 0.89143 0.90909 0.90525 0.98826
KDEOS 70 0.83333 0.82629 0.91026 0.90646 0.90909 0.90525 0.99178
LDF 10 0.66667 0.65258 0.55486 0.53605 0.72727 0.71575 0.96596
LDF 15 0.50000 0.47887 0.66786 0.65382 0.66667 0.65258 0.96127
LDF 16 0.50000 0.47887 0.63581 0.62042 0.60000 0.58310 0.96714
INFLO 8 0.50000 0.47887 0.71266 0.70052 0.66667 0.65258 0.98122
INFLO 9 0.66667 0.65258 0.74937 0.73878 0.66667 0.65258 0.97887
INFLO 12 0.66667 0.65258 0.73286 0.72157 0.72727 0.71575 0.96596
COF 61 0.50000 0.47887 0.46132 0.43856 0.60000 0.58310 0.84742
COF 75 0.33333 0.30516 0.54952 0.53049 0.66667 0.65258 0.94366
COF 78 0.50000 0.47887 0.58799 0.57058 0.66667 0.65258 0.69836

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO

Not normalized, without duplicates, categorial attributes removed

This version contains 3 attributes, 148 objects, 6 outliers (4.05%)

Download raw algorithm results (313.8 kB) Download raw algorithm evaluation table (28.5 kB)

Best Parameters

The following table contains the best (overall and per-method) results for each method and evaluation measure (when the same score was achieved twice, only the smallest k is given).
The Maximum F1-Measure is complimentary in addition to the measures in the original publication.

Algorithm k P@n Adj. P@n AP Adj. AP Max-F1 Adj. MF1 ROC AUC
KNN 1 0.66667 0.65258 0.61351 0.59718 0.72727 0.71575 0.81808
KNN 2 0.66667 0.65258 0.62462 0.60876 0.66667 0.65258 0.80692
KNN 4 0.66667 0.65258 0.48945 0.46788 0.72727 0.71575 0.85563
KNNW 2 0.66667 0.65258 0.47462 0.45243 0.72727 0.71575 0.81514
KNNW 5 0.66667 0.65258 0.57278 0.55473 0.72727 0.71575 0.83862
LOF 42 0.66667 0.65258 0.56727 0.54899 0.66667 0.65258 0.82923
LOF 43 0.66667 0.65258 0.59227 0.57505 0.66667 0.65258 0.83040
LOF 50 0.50000 0.47887 0.51847 0.49812 0.54545 0.52625 0.83509
SimplifiedLOF 43 0.50000 0.47887 0.38917 0.36336 0.50000 0.47887 0.92430
SimplifiedLOF 76 0.66667 0.65258 0.45792 0.43501 0.66667 0.65258 0.83040
SimplifiedLOF 82 0.66667 0.65258 0.56990 0.55173 0.66667 0.65258 0.83627
LoOP 52 0.50000 0.47887 0.43129 0.40726 0.50000 0.47887 0.91491
LoOP 55 0.50000 0.47887 0.45249 0.42935 0.53333 0.51362 0.91021
LoOP 78 0.66667 0.65258 0.39868 0.37327 0.66667 0.65258 0.82923
LDOF 50 0.36667 0.33991 0.32263 0.29401 0.47059 0.44822 0.89261
LDOF 51 0.36667 0.33991 0.29315 0.26329 0.47059 0.44822 0.89378
LDOF 76 0.33333 0.30516 0.31942 0.29067 0.57143 0.55332 0.86678
LDOF 97 0.33333 0.30516 0.37482 0.34840 0.57143 0.55332 0.85857
ODIN 5 0.66667 0.65258 0.53178 0.51199 0.66667 0.65258 0.87265
ODIN 8 0.66667 0.65258 0.69605 0.68321 0.72727 0.71575 0.89671
ODIN 30 0.52381 0.50369 0.55037 0.53137 0.53333 0.51362 0.91373
FastABOD 15 0.50000 0.47887 0.57492 0.55696 0.61538 0.59913 0.81631
FastABOD 22 0.50000 0.47887 0.46596 0.44340 0.61538 0.59913 0.82688
FastABOD 26 0.66667 0.65258 0.48059 0.45864 0.66667 0.65258 0.81984
KDEOS 3 0.50000 0.47887 0.56796 0.54970 0.61538 0.59913 0.83627
KDEOS 20 0.00000 -0.04225 0.18781 0.15349 0.34783 0.32027 0.83862
LDF 5 0.66667 0.65258 0.60767 0.59109 0.72727 0.71575 0.85563
INFLO 19 0.07143 0.03219 0.14685 0.11080 0.33333 0.30516 0.82688
INFLO 42 0.50000 0.47887 0.49116 0.46966 0.50000 0.47887 0.78638
INFLO 43 0.50000 0.47887 0.53283 0.51309 0.60000 0.58310 0.78873
INFLO 56 0.50000 0.47887 0.39901 0.37361 0.61538 0.59913 0.76995
COF 69 0.50000 0.47887 0.36800 0.34129 0.61538 0.59913 0.89495
COF 70 0.66667 0.65258 0.42049 0.39600 0.66667 0.65258 0.87383
COF 74 0.50000 0.47887 0.48372 0.46191 0.60000 0.58310 0.84683

Plots

Precision at n
Adjusted precision at n
Average precision
Adjusted average precision
Maximum F1 score
Adjusted maximum F1 score
ROC AUC
Diversity
A: KNN, B: KNNW, C: LOF, D: SimplifiedLOF, E: LoOP, F: LDOF
G: ODIN, H: KDEOS, I: COF, J: FastABOD, K: LDF, L: INFLO