weka.classifiers.meta
Class AdditiveRegression

java.lang.Object
  weka.classifiers.Classifier
      weka.classifiers.meta.AdditiveRegression

All Implemented Interfaces:

AdditionalMeasureProducer, java.lang.Cloneable, OptionHandler, java.io.Serializable, WeightedInstancesHandler

public class AdditiveRegression

extends Classifier

implements OptionHandler, AdditionalMeasureProducer, WeightedInstancesHandler

Meta classifier that enhances the performance of a regression base classifier. Each iteration fits a model to the residuals left by the classifier on the previous iteration. Prediction is accomplished by adding the predictions of each classifier. Smoothing is accomplished through varying the shrinkage (learning rate) parameter.

For more information see:

Friedman, J.H. (1999). Stochastic Gradient Boosting. Technical Report Stanford University. http://www-stat.stanford.edu/~jhf/ftp/stobst.ps.

Valid options from the command line are:

-W classifierstring
Classifierstring should contain the full class name of a classifier followed by options to the classifier. (required).

-S shrinkage rate
Smaller values help prevent overfitting and have a smoothing effect (but increase learning time). (default = 1.0, ie no shrinkage).

-M max models
Set the maximum number of models to generate. Values <= 0 indicate no maximum, ie keep going until the reduction in error threshold is reached. (default = 10).

-D
Debugging output.

Version:

$Revision: 1.14 $

Author:

Mark Hall (mhall@cs.waikato.ac.nz)

See Also:

Serialized Form

Field Summary

private FastVector	m_additiveModels The list of iteratively generated models.
protected Classifier	m_Classifier Base classifier.
private int	m_classIndex Class index.
private boolean	m_debug Produce debugging output.
protected int	m_maxModels Maximum number of models to produce. -1 indicates keep going until the error threshold is met.
protected double	m_shrinkage Shrinkage (Learning rate).

Fields inherited from class weka.classifiers.Classifier

m_Debug

Constructor Summary

AdditiveRegression()
Default constructor specifying DecisionStump as the classifier

AdditiveRegression(Classifier classifier)
Constructor which takes base classifier as argument.

Method Summary

void	buildClassifier(Instances data) Build the classifier on the supplied data
java.lang.String	classifierTipText() Returns the tip text for this property
double	classifyInstance(Instance inst) Classify an instance.
java.lang.String	debugTipText() Returns the tip text for this property
java.util.Enumeration	enumerateMeasures() Returns an enumeration of the additional measure names
Classifier	getClassifier() Gets the classifier used.
protected java.lang.String	getClassifierSpec() Gets the classifier specification string, which contains the class name of the classifier and any options to the classifier
boolean	getDebug() Gets whether debugging has been turned on
int	getMaxModels() Get the max number of models to generate
double	getMeasure(java.lang.String additionalMeasureName) Returns the value of the named measure
java.lang.String[]	getOptions() Gets the current settings of the Classifier.
double	getShrinkage() Get the shrinkage rate.
java.lang.String	globalInfo() Returns a string describing this attribute evaluator
java.util.Enumeration	listOptions() Returns an enumeration describing the available options.
static void	main(java.lang.String[] argv) Main method for testing this class.
java.lang.String	maxModelsTipText() Returns the tip text for this property
double	measureNumIterations() return the number of iterations (base classifiers) completed
private Instances	residualReplace(Instances data, Classifier c, boolean useShrinkage) Replace the class values of the instances from the current iteration with residuals ater predicting with the supplied classifier.
void	setClassifier(Classifier classifier) Sets the classifier
void	setDebug(boolean d) Set whether debugging output is produced.
void	setMaxModels(int maxM) Set the maximum number of models to generate
void	setOptions(java.lang.String[] options) Parses a given list of options.
void	setShrinkage(double l) Set the shrinkage parameter
java.lang.String	shrinkageTipText() Returns the tip text for this property
java.lang.String	toString() Returns textual description of the classifier.

Methods inherited from class weka.classifiers.Classifier

distributionForInstance, forName, makeCopies

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

m_Classifier

protected Classifier m_Classifier

Base classifier.

m_classIndex

private int m_classIndex

Class index.

m_shrinkage

protected double m_shrinkage

Shrinkage (Learning rate). Default = no shrinkage.

m_additiveModels

private FastVector m_additiveModels

The list of iteratively generated models.

m_debug

private boolean m_debug

Produce debugging output.

m_maxModels

protected int m_maxModels

Maximum number of models to produce. -1 indicates keep going until the error threshold is met.

Constructor Detail

AdditiveRegression

public AdditiveRegression()

Default constructor specifying DecisionStump as the classifier

AdditiveRegression

public AdditiveRegression(Classifier classifier)

Constructor which takes base classifier as argument.

Parameters:

classifier - the base classifier to use

Method Detail

globalInfo

public java.lang.String globalInfo()

Returns a string describing this attribute evaluator

Returns:

a description of the evaluator suitable for displaying in the explorer/experimenter gui

listOptions

public java.util.Enumeration listOptions()

Returns an enumeration describing the available options.

Specified by:

listOptions in interface OptionHandler

Overrides:

listOptions in class Classifier

Returns:

an enumeration of all the available options.

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses a given list of options. Valid options are:

-W classifierstring
Classifierstring should contain the full class name of a classifier followed by options to the classifier. (required).

-S shrinkage rate
Smaller values help prevent overfitting and have a smoothing effect (but increase learning time). (default = 1.0, ie. no shrinkage).

-D
Debugging output.

-M max models
Set the maximum number of models to generate. Values <= 0 indicate no maximum, ie keep going until the reduction in error threshold is reached. (default = 10).

Specified by:

setOptions in interface OptionHandler

Overrides:

setOptions in class Classifier

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of the Classifier.

Specified by:

getOptions in interface OptionHandler

Overrides:

getOptions in class Classifier

Returns:

an array of strings suitable for passing to setOptions

debugTipText

public java.lang.String debugTipText()

Returns the tip text for this property

Overrides:

debugTipText in class Classifier

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

setDebug

public void setDebug(boolean d)

Set whether debugging output is produced.

Overrides:

setDebug in class Classifier

Parameters:

d - true if debugging output is to be produced

getDebug

public boolean getDebug()

Gets whether debugging has been turned on

Overrides:

getDebug in class Classifier

Returns:

true if debugging has been turned on

classifierTipText

public java.lang.String classifierTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

setClassifier

public void setClassifier(Classifier classifier)

Sets the classifier

Parameters:

classifier - the classifier with all options set.

getClassifier

public Classifier getClassifier()

Gets the classifier used.

Returns:

the classifier

getClassifierSpec

protected java.lang.String getClassifierSpec()

Gets the classifier specification string, which contains the class name of the classifier and any options to the classifier

Returns:

the classifier string.

maxModelsTipText

public java.lang.String maxModelsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

setMaxModels

public void setMaxModels(int maxM)

Set the maximum number of models to generate

Parameters:

maxM - the maximum number of models

getMaxModels

public int getMaxModels()

Get the max number of models to generate

Returns:

the max number of models to generate

shrinkageTipText

public java.lang.String shrinkageTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

setShrinkage

public void setShrinkage(double l)

Set the shrinkage parameter

Parameters:

l - the shrinkage rate.

getShrinkage

public double getShrinkage()

Get the shrinkage rate.

Returns:

the value of the learning rate

buildClassifier

public void buildClassifier(Instances data)
                     throws java.lang.Exception

Build the classifier on the supplied data

Specified by:

buildClassifier in class Classifier

Parameters:

data - the training data

Throws:

java.lang.Exception - if the classifier could not be built successfully

classifyInstance

public double classifyInstance(Instance inst)
                        throws java.lang.Exception

Classify an instance.

Overrides:

classifyInstance in class Classifier

Parameters:

inst - the instance to predict

Returns:

a prediction for the instance

Throws:

java.lang.Exception - if an error occurs

residualReplace

private Instances residualReplace(Instances data,
                                  Classifier c,
                                  boolean useShrinkage)
                           throws java.lang.Exception

Replace the class values of the instances from the current iteration with residuals ater predicting with the supplied classifier.

Parameters:

data - the instances to predict

c - the classifier to use

useShrinkage - whether shrinkage is to be applied to the model's output

Returns:

a new set of instances with class values replaced by residuals

Throws:

java.lang.Exception

enumerateMeasures

public java.util.Enumeration enumerateMeasures()

Returns an enumeration of the additional measure names

Specified by:

enumerateMeasures in interface AdditionalMeasureProducer

Returns:

an enumeration of the measure names

getMeasure

public double getMeasure(java.lang.String additionalMeasureName)

Returns the value of the named measure

Specified by:

getMeasure in interface AdditionalMeasureProducer

Parameters:

additionalMeasureName - the name of the measure to query for its value

Returns:

the value of the named measure

Throws:

java.lang.IllegalArgumentException - if the named measure is not supported

measureNumIterations

public double measureNumIterations()

return the number of iterations (base classifiers) completed

Returns:

the number of iterations (same as number of base classifier models)

toString

public java.lang.String toString()

Returns textual description of the classifier.

Returns:

a description of the classifier as a string

main

public static void main(java.lang.String[] argv)

Main method for testing this class.

Parameters:

argv - should contain the following arguments: -t training file [-T test file] [-c class index]