weka.classifiers.functions
Class SMO

java.lang.Object
  weka.classifiers.Classifier
      weka.classifiers.functions.SMO

All Implemented Interfaces:

java.lang.Cloneable, OptionHandler, java.io.Serializable, WeightedInstancesHandler

public class SMO

extends Classifier

implements WeightedInstancesHandler

Implements John C. Platt's sequential minimal optimization algorithm for training a support vector classifier using polynomial or RBF kernels. This implementation globally replaces all missing values and transforms nominal attributes into binary ones. It also normalizes all attributes by default. (Note that the coefficients in the output are based on the normalized/standardized data, not the original data.) Multi-class problems are solved using pairwise classification. To obtain proper probability estimates, use the option that fits logistic regression models to the outputs of the support vector machine. In the multi-class case the predicted probabilities will be coupled using Hastie and Tibshirani's pairwise coupling method. Note: for improved speed standardization should be turned off when operating on SparseInstances.

For more information on the SMO algorithm, see

J. Platt (1998). Fast Training of Support Vector Machines using Sequential Minimal Optimization. Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola, eds., MIT Press.

S.S. Keerthi, S.K. Shevade, C. Bhattacharyya, K.R.K. Murthy, Improvements to Platt's SMO Algorithm for SVM Classifier Design. Neural Computation, 13(3), pp 637-649, 2001.

Valid options are:

-C num
The complexity constant C. (default 1)

-E num
The exponent for the polynomial kernel. (default 1)

-G num
Gamma for the RBF kernel. (default 0.01)

-N <0|1|2>
Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)

-F
Feature-space normalization (only for non-linear polynomial kernels).

-O
Use lower-order terms (only for non-linear polynomial kernels).

-R
Use the RBF kernel. (default poly)

-A num
Sets the size of the kernel cache. Should be a prime number. (default 1000003)

-T num
Sets the tolerance parameter. (default 1.0e-3)

-P num
Sets the epsilon for round-off error. (default 1.0e-12)

-M
Fit logistic models to SVM outputs.

-V num
Number of folds for cross-validation used to generate data for logistic models. (default -1, use training data) -W num
Random number seed for cross-validation. (default 1)

Version:

$Revision: 1.50 $

Author:

Eibe Frank (eibe@cs.waikato.ac.nz), Shane Legg (shane@intelligenesis.net) (sparse vector code), Stuart Inglis (stuart@reeltwo.com) (sparse vector code)

See Also:

Serialized Form

Nested Class Summary

private class

SMO.BinarySMO Class for building a binary support vector machine.

Field Summary

static int	FILTER_NONE
static int	FILTER_NORMALIZE The filter to apply to the training data
static int	FILTER_STANDARDIZE
private double	m_C The complexity parameter.
private int	m_cacheSize The size of the cache (a prime number)
private boolean	m_checksTurnedOff Turn off all checks and conversions?
private Attribute	m_classAttribute The class attribute
private SMO.BinarySMO[][]	m_classifiers The binary classifier(s)
private int	m_classIndex The class index from the training data
private static double	m_Del Precision constant for updating sets
private double	m_eps Epsilon for rounding.
private double	m_exponent The exponent for the polynomial kernel.
private boolean	m_featureSpaceNormalization Feature-space normalization?
private Filter	m_Filter The filter used to standardize/normalize all values.
private int	m_filterType Whether to normalize/standardize/neither
private boolean	m_fitLogisticModels Whether logistic models are to be fit
private double	m_gamma Gamma for the RBF kernel.
private boolean	m_lowerOrder Use lower-order terms?
private ReplaceMissingValues	m_Missing The filter used to get rid of missing values.
private NominalToBinary	m_NominalToBinary The filter used to make attributes numeric.
private int	m_numFolds The number of folds for the internal cross-validation
private boolean	m_onlyNumeric Only numeric attributes in the dataset?
private int	m_randomSeed The random number seed
private double	m_tol Tolerance for accuracy of result.
private boolean	m_useRBF Use RBF kernel?
static Tag[]	TAGS_FILTER

Fields inherited from class weka.classifiers.Classifier

m_Debug

Constructor Summary

SMO()

Method Summary

java.lang.String[][][]	attributeNames() Returns the attribute names.
double[][]	bias() Returns the bias of each binary SMO.
void	buildClassifier(Instances insts) Method for building the classifier.
java.lang.String	buildLogisticModelsTipText() Returns the tip text for this property
java.lang.String	cacheSizeTipText() Returns the tip text for this property
java.lang.String[]	classAttributeNames()
java.lang.String	cTipText() Returns the tip text for this property
double[]	distributionForInstance(Instance inst) Estimates class probabilities for given instance.
java.lang.String	epsilonTipText() Returns the tip text for this property
java.lang.String	exponentTipText() Returns the tip text for this property
java.lang.String	featureSpaceNormalizationTipText() Returns the tip text for this property
java.lang.String	filterTypeTipText() Returns the tip text for this property
java.lang.String	gammaTipText() Returns the tip text for this property
boolean	getBuildLogisticModels() Get the value of buildLogisticModels.
double	getC() Get the value of C.
int	getCacheSize() Get the size of the kernel cache
double	getEpsilon() Get the value of epsilon.
double	getExponent() Get the value of exponent.
boolean	getFeatureSpaceNormalization() Check whether feature spaces is being normalized.
SelectedTag	getFilterType() Gets how the training data will be transformed.
double	getGamma() Get the value of gamma.
boolean	getLowerOrderTerms() Check whether lower-order terms are being used.
int	getNumFolds() Get the value of numFolds.
java.lang.String[]	getOptions() Gets the current settings of the classifier.
int	getRandomSeed() Get the value of randomSeed.
double	getToleranceParameter() Get the value of tolerance parameter.
boolean	getUseRBF() Check if the RBF kernel is to be used.
java.lang.String	globalInfo() Returns a string describing classifier
java.util.Enumeration	listOptions() Returns an enumeration describing the available options.
java.lang.String	lowerOrderTermsTipText() Returns the tip text for this property
static void	main(java.lang.String[] argv) Main method for testing this class.
int	numClassAttributeValues()
java.lang.String	numFoldsTipText() Returns the tip text for this property
int[]	obtainVotes(Instance inst) Returns an array of votes for the given instance.
double[]	pairwiseCoupling(double[][] n, double[][] r) Implements pairwise coupling.
java.lang.String	randomSeedTipText() Returns the tip text for this property
void	setBuildLogisticModels(boolean newbuildLogisticModels) Set the value of buildLogisticModels.
void	setC(double v) Set the value of C.
void	setCacheSize(int v) Set the value of the kernel cache.
void	setEpsilon(double v) Set the value of epsilon.
void	setExponent(double v) Set the value of exponent.
void	setFeatureSpaceNormalization(boolean v) Set whether feature space is normalized.
void	setFilterType(SelectedTag newType) Sets how the training data will be transformed.
void	setGamma(double v) Set the value of gamma.
void	setLowerOrderTerms(boolean v) Set whether lower-order terms are to be used.
void	setNumFolds(int newnumFolds) Set the value of numFolds.
void	setOptions(java.lang.String[] options) Parses a given list of options.
void	setRandomSeed(int newrandomSeed) Set the value of randomSeed.
void	setToleranceParameter(double v) Set the value of tolerance parameter.
void	setUseRBF(boolean v) Set if the RBF kernel is to be used.
int[][][]	sparseIndices() Returns the indices in sparse format.
double[][][]	sparseWeights() Returns the weights in sparse format.
java.lang.String	toleranceParameterTipText() Returns the tip text for this property
java.lang.String	toString() Prints out the classifier.
void	turnChecksOff() Turns off checks for missing values, etc.
void	turnChecksOn() Turns on checks for missing values, etc.
java.lang.String	useRBFTipText() Returns the tip text for this property

Methods inherited from class weka.classifiers.Classifier

classifyInstance, debugTipText, forName, getDebug, makeCopies, setDebug

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

FILTER_NORMALIZE

public static final int FILTER_NORMALIZE

The filter to apply to the training data

See Also:

Constant Field Values

FILTER_STANDARDIZE

public static final int FILTER_STANDARDIZE

See Also:

Constant Field Values

FILTER_NONE

public static final int FILTER_NONE

See Also:

Constant Field Values

TAGS_FILTER

public static final Tag[] TAGS_FILTER

m_classifiers

private SMO.BinarySMO[][] m_classifiers

The binary classifier(s)

m_exponent

private double m_exponent

The exponent for the polynomial kernel.

m_lowerOrder

private boolean m_lowerOrder

Use lower-order terms?

m_gamma

private double m_gamma

Gamma for the RBF kernel.

m_C

private double m_C

The complexity parameter.

m_eps

private double m_eps

Epsilon for rounding.

m_tol

private double m_tol

Tolerance for accuracy of result.

m_filterType

private int m_filterType

Whether to normalize/standardize/neither

m_featureSpaceNormalization

private boolean m_featureSpaceNormalization

Feature-space normalization?

m_useRBF

private boolean m_useRBF

Use RBF kernel? (default: poly)

m_cacheSize

private int m_cacheSize

The size of the cache (a prime number)

m_NominalToBinary

private NominalToBinary m_NominalToBinary

The filter used to make attributes numeric.

m_Filter

private Filter m_Filter

The filter used to standardize/normalize all values.

m_Missing

private ReplaceMissingValues m_Missing

The filter used to get rid of missing values.

m_onlyNumeric

private boolean m_onlyNumeric

Only numeric attributes in the dataset?

m_classIndex

private int m_classIndex

The class index from the training data

m_classAttribute

private Attribute m_classAttribute

The class attribute

m_checksTurnedOff

private boolean m_checksTurnedOff

Turn off all checks and conversions? Turning them off assumes that data is purely numeric, doesn't contain any missing values, and has a nominal class. Turning them off also means that no header information will be stored if the machine is linear. Finally, it also assumes that no instance has a weight equal to 0.

m_Del

private static double m_Del

Precision constant for updating sets

m_fitLogisticModels

private boolean m_fitLogisticModels

Whether logistic models are to be fit

m_numFolds

private int m_numFolds

The number of folds for the internal cross-validation

m_randomSeed

private int m_randomSeed

The random number seed

Constructor Detail

SMO

public SMO()

Method Detail

globalInfo

public java.lang.String globalInfo()

Returns a string describing classifier

Returns:

a description suitable for displaying in the explorer/experimenter gui

turnChecksOff

public void turnChecksOff()

Turns off checks for missing values, etc. Use with caution.

turnChecksOn

public void turnChecksOn()

Turns on checks for missing values, etc.

buildClassifier

public void buildClassifier(Instances insts)
                     throws java.lang.Exception

Method for building the classifier. Implements a one-against-one wrapper for multi-class problems.

Specified by:

buildClassifier in class Classifier

Parameters:

insts - the set of training instances

Throws:

java.lang.Exception - if the classifier can't be built successfully

distributionForInstance

public double[] distributionForInstance(Instance inst)
                                 throws java.lang.Exception

Estimates class probabilities for given instance.

Overrides:

distributionForInstance in class Classifier

Parameters:

inst - the instance to be classified

Returns:

an array containing the estimated membership probabilities of the test instance in each class or the numeric prediction

Throws:

java.lang.Exception - if distribution could not be computed successfully

pairwiseCoupling

public double[] pairwiseCoupling(double[][] n,
                                 double[][] r)

Implements pairwise coupling.

Parameters:

n - the sum of weights used to train each model

r - the probability estimate from each model

Returns:

the coupled estimates

obtainVotes

public int[] obtainVotes(Instance inst)
                  throws java.lang.Exception

Returns an array of votes for the given instance.

Parameters:

inst - the instance

Returns:

array of votex

Throws:

java.lang.Exception - if something goes wrong

sparseWeights

public double[][][] sparseWeights()

Returns the weights in sparse format.

sparseIndices

public int[][][] sparseIndices()

Returns the indices in sparse format.

bias

public double[][] bias()

Returns the bias of each binary SMO.

numClassAttributeValues

public int numClassAttributeValues()

classAttributeNames

public java.lang.String[] classAttributeNames()

attributeNames

public java.lang.String[][][] attributeNames()

Returns the attribute names.

listOptions

public java.util.Enumeration listOptions()

Returns an enumeration describing the available options.

Specified by:

listOptions in interface OptionHandler

Overrides:

listOptions in class Classifier

Returns:

an enumeration of all the available options.

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses a given list of options. Valid options are:

-C num
The complexity constant C. (default 1)

-E num
The exponent for the polynomial kernel. (default 1)

-G num
Gamma for the RBF kernel. (default 0.01)

-N <0|1|2>
Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)

-F
Feature-space normalization (only for non-linear polynomial kernels).

-O
Use lower-order terms (only for non-linear polynomial kernels).

-R
Use RBF kernel (default poly).

-A num
Sets the size of the kernel cache. Should be a prime number. (default 1000003)

-T num
Sets the tolerance parameter. (default 1.0e-3)

-P num
Sets the epsilon for round-off error. (default 1.0e-12)

-M
Fit logistic models to SVM outputs.

-V num
Number of folds for cross-validation used to generate data for logistic models. (default -1, use training data) -W num
Random number seed. (default 1)

Specified by:

setOptions in interface OptionHandler

Overrides:

setOptions in class Classifier

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of the classifier.

Specified by:

getOptions in interface OptionHandler

Overrides:

getOptions in class Classifier

Returns:

an array of strings suitable for passing to setOptions

exponentTipText

public java.lang.String exponentTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getExponent

public double getExponent()

Get the value of exponent.

Returns:

Value of exponent.

setExponent

public void setExponent(double v)

Set the value of exponent. If linear kernel is used, rescaling and lower-order terms are turned off.

Parameters:

v - Value to assign to exponent.

gammaTipText

public java.lang.String gammaTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getGamma

public double getGamma()

Get the value of gamma.

Returns:

Value of gamma.

setGamma

public void setGamma(double v)

Set the value of gamma.

Parameters:

v - Value to assign to gamma.

cTipText

public java.lang.String cTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getC

public double getC()

Get the value of C.

Returns:

Value of C.

setC

public void setC(double v)

Set the value of C.

Parameters:

v - Value to assign to C.

toleranceParameterTipText

public java.lang.String toleranceParameterTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getToleranceParameter

public double getToleranceParameter()

Get the value of tolerance parameter.

Returns:

Value of tolerance parameter.

setToleranceParameter

public void setToleranceParameter(double v)

Set the value of tolerance parameter.

Parameters:

v - Value to assign to tolerance parameter.

epsilonTipText

public java.lang.String epsilonTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getEpsilon

public double getEpsilon()

Get the value of epsilon.

Returns:

Value of epsilon.

setEpsilon

public void setEpsilon(double v)

Set the value of epsilon.

Parameters:

v - Value to assign to epsilon.

cacheSizeTipText

public java.lang.String cacheSizeTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getCacheSize

public int getCacheSize()

Get the size of the kernel cache

Returns:

Size of kernel cache.

setCacheSize

public void setCacheSize(int v)

Set the value of the kernel cache.

Parameters:

v - Size of kernel cache.

filterTypeTipText

public java.lang.String filterTypeTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getFilterType

public SelectedTag getFilterType()

Gets how the training data will be transformed. Will be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.

Returns:

the filtering mode

setFilterType

public void setFilterType(SelectedTag newType)

Sets how the training data will be transformed. Should be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.

Parameters:

newType - the new filtering mode

useRBFTipText

public java.lang.String useRBFTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getUseRBF

public boolean getUseRBF()

Check if the RBF kernel is to be used.

Returns:

true if RBF

setUseRBF

public void setUseRBF(boolean v)

Set if the RBF kernel is to be used.

Parameters:

v - true if RBF

featureSpaceNormalizationTipText

public java.lang.String featureSpaceNormalizationTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getFeatureSpaceNormalization

public boolean getFeatureSpaceNormalization()
                                     throws java.lang.Exception

Check whether feature spaces is being normalized.

Returns:

true if feature space is normalized.

Throws:

java.lang.Exception

setFeatureSpaceNormalization

public void setFeatureSpaceNormalization(boolean v)
                                  throws java.lang.Exception

Set whether feature space is normalized.

Parameters:

v - true if feature space is to be normalized.

Throws:

java.lang.Exception

lowerOrderTermsTipText

public java.lang.String lowerOrderTermsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getLowerOrderTerms

public boolean getLowerOrderTerms()

Check whether lower-order terms are being used.

Returns:

Value of lowerOrder.

setLowerOrderTerms

public void setLowerOrderTerms(boolean v)

Set whether lower-order terms are to be used. Defaults to false if a linear machine is built.

Parameters:

v - Value to assign to lowerOrder.

buildLogisticModelsTipText

public java.lang.String buildLogisticModelsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getBuildLogisticModels

public boolean getBuildLogisticModels()

Get the value of buildLogisticModels.

Returns:

Value of buildLogisticModels.

setBuildLogisticModels

public void setBuildLogisticModels(boolean newbuildLogisticModels)

Set the value of buildLogisticModels.

Parameters:

newbuildLogisticModels - Value to assign to buildLogisticModels.

numFoldsTipText

public java.lang.String numFoldsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getNumFolds

public int getNumFolds()

Get the value of numFolds.

Returns:

Value of numFolds.

setNumFolds

public void setNumFolds(int newnumFolds)

Set the value of numFolds.

Parameters:

newnumFolds - Value to assign to numFolds.

randomSeedTipText

public java.lang.String randomSeedTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getRandomSeed

public int getRandomSeed()

Get the value of randomSeed.

Returns:

Value of randomSeed.

setRandomSeed

public void setRandomSeed(int newrandomSeed)

Set the value of randomSeed.

Parameters:

newrandomSeed - Value to assign to randomSeed.

toString

public java.lang.String toString()

Prints out the classifier.

Returns:

a description of the classifier as a string

main

public static void main(java.lang.String[] argv)

Main method for testing this class.