weka.classifiers
Class BVDecomposeSegCVSub

java.lang.Object
  weka.classifiers.BVDecomposeSegCVSub

All Implemented Interfaces:

OptionHandler

public class BVDecomposeSegCVSub

extends java.lang.Object

implements OptionHandler

This class performs Bias-Variance decomposion on any classifier using the sub-sampled cross-validation procedure as specified in:

Geoffrey I. Webb & Paul Conilione (2002), Estimating bias and variance from data , School of Computer Science and Software Engineering, Monash University, Australia

The Kohavi and Wolpert definition of bias and variance is specified in:

R. Kohavi & D. Wolpert (1996), Bias plus variance decomposition for zero-one loss functions, in Proc. of the Thirteenth International Machine Learning Conference (ICML96) download postscript.

The Webb definition of bias and variance is specified in:

Geoffrey I. Webb (2000), MultiBoosting: A Technique for Combining Boosting and Wagging, Machine Learning, 40(2), pages 159-196

Valid options are:

-c num
Specify the index of the class attribute (default last).

-D
Turn on debugging output.

-l num
Set the number times each instance is to be classified (default 10).

-p num
Set the proportion of instances that are the same between any two training sets. Training set size/(Dataset size - 1) < num < 1.0 (Default is Training set size/(Dataset size - 1) )

-s num
Set the seed for the dataset randomisation (default 1).

-t filename
Set the arff file to use for the decomposition (required).

-T num
Set the size of the training sets. Must be greater than 0 and less size of the dataset. (default half of dataset size)

-W classname
Specify the full class name of a learner to perform the decomposition on (required).

Options after -- are passed to the designated sub-learner.

Version:

$Revision: 1.1 $

Author:

Paul Conilione (paulc4321@yahoo.com.au)

Field Summary

protected Classifier	m_Classifier An instantiated base classifier used for getting and testing options.
protected java.lang.String[]	m_ClassifierOptions The options to be passed to the base classifier.
protected int	m_ClassifyIterations The number of times an instance is classified
protected int	m_ClassIndex The index of the class attribute
protected java.lang.String	m_DataFileName The name of the data file used for the decomposition
protected boolean	m_Debug Debugging mode, gives extra output if true.
protected double	m_Error The error rate
protected double	m_KWBias The calculated Kohavi & Wolpert bias (squared)
protected double	m_KWSigma The calculated Kohavi & Wolpert sigma
protected double	m_KWVariance The calculated Kohavi & Wolpert variance
protected double	m_P Proportion of instances common between any two training sets.
protected int	m_Seed The random number seed
protected int	m_TrainSize The training set size
protected double	m_WBias The calculated Webb bias
protected double	m_WVariance The calculated Webb variance

Constructor Summary

BVDecomposeSegCVSub()

Method Summary

void	decompose() Carry out the bias-variance decomposition using the sub-sampled cross-validation method.
java.util.Vector	findCentralTendencies(double[] predProbs) Finds the central tendency, given the classifications for an instance.
Classifier	getClassifier() Gets the name of the classifier being analysed
int	getClassifyIterations() Gets the number of times an instance is classified
int	getClassIndex() Get the index (starting from 1) of the attribute used as the class.
java.lang.String	getDataFileName() Get the name of the data file used for the decomposition
boolean	getDebug() Gets whether debugging is turned on
double	getError() Get the calculated error rate
double	getKWBias() Get the calculated bias squared according to the Kohavi and Wolpert definition
double	getKWSigma() Get the calculated sigma according to the Kohavi and Wolpert definition
double	getKWVariance() Get the calculated variance according to the Kohavi and Wolpert definition
java.lang.String[]	getOptions() Gets the current settings of the CheckClassifier.
double	getP() Get the proportion of instances that are common between two training sets.
int	getSeed() Gets the random number seed
int	getTrainSize() Get the training size
double	getWBias() Get the calculated bias according to the Webb definition
double	getWVariance() Get the calculated variance according to the Webb definition
java.util.Enumeration	listOptions() Returns an enumeration describing the available options.
static void	main(java.lang.String[] args) Test method for this class
void	randomize(int[] index, java.util.Random random) Accepts an array of ints and randomises the values in the array, using the random seed.
void	setClassifier(Classifier newClassifier) Set the classifiers being analysed
void	setClassifyIterations(int classifyIterations) Sets the number of times an instance is classified
void	setClassIndex(int classIndex) Sets index of attribute to discretize on
void	setDataFileName(java.lang.String dataFileName) Sets the name of the dataset file.
void	setDebug(boolean debug) Sets debugging mode
void	setOptions(java.lang.String[] options) Sets the OptionHandler's options using the given list.
void	setP(double proportion) Set the proportion of instances that are common between two training sets used to train a classifier.
void	setSeed(int seed) Sets the random number seed
void	setTrainSize(int size) Set the training size.
java.lang.String	toString() Returns description of the bias-variance decomposition results.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

m_Debug

protected boolean m_Debug

Debugging mode, gives extra output if true.

m_Classifier

protected Classifier m_Classifier

An instantiated base classifier used for getting and testing options.

m_ClassifierOptions

protected java.lang.String[] m_ClassifierOptions

The options to be passed to the base classifier.

m_ClassifyIterations

protected int m_ClassifyIterations

The number of times an instance is classified

m_DataFileName

protected java.lang.String m_DataFileName

The name of the data file used for the decomposition

m_ClassIndex

protected int m_ClassIndex

The index of the class attribute

m_Seed

protected int m_Seed

The random number seed

m_KWBias

protected double m_KWBias

The calculated Kohavi & Wolpert bias (squared)

m_KWVariance

protected double m_KWVariance

The calculated Kohavi & Wolpert variance

m_KWSigma

protected double m_KWSigma

The calculated Kohavi & Wolpert sigma

m_WBias

protected double m_WBias

The calculated Webb bias

m_WVariance

protected double m_WVariance

The calculated Webb variance

m_Error

protected double m_Error

The error rate

m_TrainSize

protected int m_TrainSize

The training set size

m_P

protected double m_P

Proportion of instances common between any two training sets.

Constructor Detail

BVDecomposeSegCVSub

public BVDecomposeSegCVSub()

Method Detail

listOptions

public java.util.Enumeration listOptions()

Returns an enumeration describing the available options.

Specified by:

listOptions in interface OptionHandler

Returns:

an enumeration of all the available options.

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Sets the OptionHandler's options using the given list. All options will be set (or reset) during this call (i.e. incremental setting of options is not possible).

Specified by:

setOptions in interface OptionHandler

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of the CheckClassifier.

Specified by:

getOptions in interface OptionHandler

Returns:

an array of strings suitable for passing to setOptions

setClassifier

public void setClassifier(Classifier newClassifier)

Set the classifiers being analysed

Parameters:

newClassifier - the Classifier to use.

getClassifier

public Classifier getClassifier()

Gets the name of the classifier being analysed

Returns:

the classifier being analysed.

setDebug

public void setDebug(boolean debug)

Sets debugging mode

Parameters:

debug - true if debug output should be printed

getDebug

public boolean getDebug()

Gets whether debugging is turned on

Returns:

true if debugging output is on

setSeed

public void setSeed(int seed)

Sets the random number seed

getSeed

public int getSeed()

Gets the random number seed

Returns:

the random number seed

setClassifyIterations

public void setClassifyIterations(int classifyIterations)

Sets the number of times an instance is classified

Parameters:

classifyIterations - number of times an instance is classified

getClassifyIterations

public int getClassifyIterations()

Gets the number of times an instance is classified

Returns:

the maximum number of times an instance is classified

setDataFileName

public void setDataFileName(java.lang.String dataFileName)

Sets the name of the dataset file.

Parameters:

dataFileName - name of dataset file.

getDataFileName

public java.lang.String getDataFileName()

Get the name of the data file used for the decomposition

Returns:

the name of the data file

getClassIndex

public int getClassIndex()

Get the index (starting from 1) of the attribute used as the class.

Returns:

the index of the class attribute

setClassIndex

public void setClassIndex(int classIndex)

Sets index of attribute to discretize on

Parameters:

classIndex - the index (starting from 1) of the class attribute

getKWBias

public double getKWBias()

Get the calculated bias squared according to the Kohavi and Wolpert definition

Returns:

the bias squared

getWBias

public double getWBias()

Get the calculated bias according to the Webb definition

Returns:

the bias

getKWVariance

public double getKWVariance()

Get the calculated variance according to the Kohavi and Wolpert definition

Returns:

the variance

getWVariance

public double getWVariance()

Get the calculated variance according to the Webb definition

Returns:

getKWSigma

public double getKWSigma()

Get the calculated sigma according to the Kohavi and Wolpert definition

Returns:

the sigma

setTrainSize

public void setTrainSize(int size)

Set the training size.

Parameters:

size - the size of the training set

getTrainSize

public int getTrainSize()

Get the training size

Returns:

the size of the training set

setP

public void setP(double proportion)

Set the proportion of instances that are common between two training sets used to train a classifier.

Parameters:

proportion - the proportion of instances that are common between training sets.

getP

public double getP()

Get the proportion of instances that are common between two training sets.

Returns:

the proportion

getError

public double getError()

Get the calculated error rate

Returns:

the error rate

decompose

public void decompose()
               throws java.lang.Exception

Carry out the bias-variance decomposition using the sub-sampled cross-validation method.

Throws:

java.lang.Exception - if the decomposition couldn't be carried out

findCentralTendencies

public java.util.Vector findCentralTendencies(double[] predProbs)

Finds the central tendency, given the classifications for an instance. Where the central tendency is defined as the class that was most commonly selected for a given instance.

For example, instance 'x' may be classified out of 3 classes y = {1, 2, 3}, so if x is classified 10 times, and is classified as follows, '1' = 2 times, '2' = 5 times and '3' = 3 times. Then the central tendency is '2'.

However, it is important to note that this method returns a list of all classes that have the highest number of classifications. In cases where there are several classes with the largest number of classifications, then all of these classes are returned. For example if 'x' is classified '1' = 4 times, '2' = 4 times and '3' = 2 times. Then '1' and '2' are returned.

Parameters:

predProbs - the array of classifications for a single instance.

Returns:

a Vector containing Integer objects which store the class(s) which are the central tendency.

toString

public java.lang.String toString()

Returns description of the bias-variance decomposition results.

Returns:

the bias-variance decomposition results as a string

main

public static void main(java.lang.String[] args)

Test method for this class

Parameters:

args - the command line arguments

randomize

public final void randomize(int[] index,
                            java.util.Random random)

Accepts an array of ints and randomises the values in the array, using the random seed.

Parameters:

index - is the array of integers

random - is the Random seed.