weka.classifiers.functions
Class MultilayerPerceptron

java.lang.Object
  weka.classifiers.Classifier
      weka.classifiers.functions.MultilayerPerceptron

All Implemented Interfaces:

java.lang.Cloneable, OptionHandler, java.io.Serializable, WeightedInstancesHandler

public class MultilayerPerceptron

extends Classifier

implements OptionHandler, WeightedInstancesHandler

A Classifier that uses backpropagation to classify instances. This network can be built by hand, created by an algorithm or both. The network can also be monitored and modified during training time. The nodes in this network are all sigmoid (except for when the class is numeric in which case the the output nodes become unthresholded linear units).

Version:

$Revision: 1.1 $

Author:

Malcolm Ware (mfw4@cs.waikato.ac.nz)

See Also:

Serialized Form

Nested Class Summary

(package private) class	MultilayerPerceptron.ControlPanel This provides the basic controls for working with the neuralnetwork
protected class	MultilayerPerceptron.NeuralEnd This inner class is used to connect the nodes in the network up to the data that they are classifying, Note that objects of this class are only suitable to go on the attribute side or class side of the network and not both.
private class	MultilayerPerceptron.NodePanel Inner class used to draw the nodes onto.

Field Summary

private boolean	m_accepted a flag to state that the network should be accepted the way it is.
private double[]	m_attributeBases The base values for all the attributes.
private double[]	m_attributeRanges The ranges for all the attributes.
private boolean	m_autoBuild A flag to tell the build classifier to automatically build a neural net.
private MultilayerPerceptron.ControlPanel	m_controlPanel The control panel.
private Instance	m_currentInstance The current instance running through the network.
private boolean	m_decay This flag states that the user wants the learning rate to decay.
private int	m_driftThreshold The number to to use to quit on validation testing.
private int	m_epoch Shows the number of the epoch that the network just finished.
private double	m_error Shows the error of the epoch that the network just finished.
private FastVector	m_graphers A Vector list of the graphers.
private boolean	m_gui A flag to state that the gui for the network should be brought up.
private java.lang.String	m_hiddenLayers The string that defines the hidden layers
private MultilayerPerceptron.NeuralEnd[]	m_inputs The input units.
private Instances	m_instances The training instances.
private double	m_learningRate This is the learning rate for the network.
private LinearUnit	m_linearUnit This is a linear unit.
private double	m_momentum This is the momentum for the network.
private NeuralConnection[]	m_neuralNodes All the nodes that actually comprise the logical neural net.
private int	m_nextId The next id number available for default naming.
private MultilayerPerceptron.NodePanel	m_nodePanel The panel the nodes are displayed on.
private NominalToBinary	m_nominalToBinaryFilter The actual filter.
private boolean	m_normalizeAttributes This flag states that the user wants the input values normalized.
private boolean	m_normalizeClass This flag states that the user wants the class to be normalized while processing in the network is done.
private int	m_numAttributes The number of attributes.
private int	m_numClasses The number of classes.
private int	m_numEpochs The number of epochs to train through.
private boolean	m_numeric A flag to say that it's a numeric class.
private MultilayerPerceptron.NeuralEnd[]	m_outputs The output units.
private java.util.Random	m_random The actual random number generator.
private long	m_randomSeed The number used to seed the random number generator.
private boolean	m_reset This flag states that the user wants the network to restart if it is found to be generating infinity or NaN for the error value.
private FastVector	m_selected A Vector list of the units currently selected.
private SigmoidUnit	m_sigmoidUnit this is a sigmoid unit.
private boolean	m_stopIt a flag to state if the network should be running, or stopped.
private boolean	m_stopped a flag to state that the network has in fact stopped.
private boolean	m_useNomToBin A flag to state that a nominal to binary filter should be used.
private int	m_valSize An int to say how big the validation set should be.
private javax.swing.JFrame	m_win The window for the network.

Fields inherited from class weka.classifiers.Classifier

m_Debug

Constructor Summary

MultilayerPerceptron()
The constructor.

Method Summary

private void	addNode(NeuralConnection n) Call this function to place a node into the network list.
java.lang.String	autoBuildTipText()
void	blocker(boolean tf) A function used to stop the code that called buildclassifier from continuing on before the user has finished the decision tree.
void	buildClassifier(Instances i) Call this function to build and train a neural network for the training data provided.
private double	calculateErrors() This will cause the error values to be calculated for all nodes.
private void	calculateOutputs() This will cause the output values of all the nodes to be calculated.
java.lang.String	decayTipText()
double[]	distributionForInstance(Instance i) Call this function to predict the class of an instance once a classification model has been built with the buildClassifier call.
boolean	getAutoBuild()
boolean	getDecay()
boolean	getGUI()
java.lang.String	getHiddenLayers()
double	getLearningRate()
double	getMomentum()
boolean	getNominalToBinaryFilter()
boolean	getNormalizeAttributes()
boolean	getNormalizeNumericClass()
java.lang.String[]	getOptions() Gets the current settings of NeuralNet.
long	getRandomSeed()
boolean	getReset()
int	getTrainingTime()
int	getValidationSetSize()
int	getValidationThreshold()
java.lang.String	globalInfo() This will return a string describing the classifier.
java.lang.String	GUITipText()
java.lang.String	hiddenLayersTipText()
java.lang.String	learningRateTipText()
java.util.Enumeration	listOptions() Returns an enumeration describing the available options.
static void	main(java.lang.String[] argv) Main method for testing this class.
java.lang.String	momentumTipText()
java.lang.String	nominalToBinaryFilterTipText()
java.lang.String	normalizeAttributesTipText()
java.lang.String	normalizeNumericClassTipText()
java.lang.String	randomSeedTipText()
private boolean	removeNode(NeuralConnection n) Call this function to remove the passed node from the list.
private void	resetNetwork() this will reset all the nodes in the network.
java.lang.String	resetTipText()
void	setAutoBuild(boolean a) This will set whether the network is automatically built or if it is left up to the user.
private Instances	setClassType(Instances inst) This function sets what the m_numeric flag to represent the passed class it also performs the normalization of the attributes if applicable and sets up the info to normalize the class.
void	setDecay(boolean d)
private void	setEndsToLinear() This will go through all the nodes and check if they are connected to a pure output unit.
void	setGUI(boolean a) This will set whether A GUI is brought up to allow interaction by the user with the neural network during training.
void	setHiddenLayers(java.lang.String h) This will set what the hidden layers are made up of when auto build is enabled.
void	setLearningRate(double l) The learning rate can be set using this command.
void	setMomentum(double m) The momentum can be set using this command.
void	setNominalToBinaryFilter(boolean f)
void	setNormalizeAttributes(boolean a)
void	setNormalizeNumericClass(boolean c)
void	setOptions(java.lang.String[] options) Parses a given list of options.
void	setRandomSeed(long l) This seeds the random number generator, that is used when a random number is needed for the network.
void	setReset(boolean r) This sets the network up to be able to reset itself with the current settings and the learning rate at half of what it is currently.
void	setTrainingTime(int n) Set the number of training epochs to perform.
private void	setupHiddenLayer() Call this function to automatically generate the hidden units
private void	setupInputs() This creates the required input units.
private void	setupOutputs() This creates the required output units.
void	setValidationSetSize(int a) This will set the size of the validation set.
void	setValidationThreshold(int t) This sets the threshold to use for when validation testing is being done.
java.lang.String	toString()
java.lang.String	trainingTimeTipText()
private void	updateDisplay() Call this function to update the control panel for the gui.
private void	updateNetworkWeights(double l, double m) This will cause the weight values to be updated based on the learning rate, momentum and the errors that have been calculated for each node.
java.lang.String	validationSetSizeTipText()
java.lang.String	validationThresholdTipText()

Methods inherited from class weka.classifiers.Classifier

classifyInstance, debugTipText, forName, getDebug, makeCopies, setDebug

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

m_instances

private Instances m_instances

The training instances.

m_currentInstance

private Instance m_currentInstance

The current instance running through the network.

m_numeric

private boolean m_numeric

A flag to say that it's a numeric class.

m_attributeRanges

private double[] m_attributeRanges

The ranges for all the attributes.

m_attributeBases

private double[] m_attributeBases

The base values for all the attributes.

m_outputs

private MultilayerPerceptron.NeuralEnd[] m_outputs

The output units.(only feeds the errors, does no calcs)

m_inputs

private MultilayerPerceptron.NeuralEnd[] m_inputs

The input units.(only feeds the inputs does no calcs)

m_neuralNodes

private NeuralConnection[] m_neuralNodes

All the nodes that actually comprise the logical neural net.

m_numClasses

private int m_numClasses

The number of classes.

m_numAttributes

private int m_numAttributes

The number of attributes.

m_nodePanel

private MultilayerPerceptron.NodePanel m_nodePanel

The panel the nodes are displayed on.

m_controlPanel

private MultilayerPerceptron.ControlPanel m_controlPanel

The control panel.

m_nextId

private int m_nextId

The next id number available for default naming.

m_selected

private FastVector m_selected

A Vector list of the units currently selected.

m_graphers

private FastVector m_graphers

A Vector list of the graphers.

m_numEpochs

private int m_numEpochs

The number of epochs to train through.

m_stopIt

private boolean m_stopIt

a flag to state if the network should be running, or stopped.

m_stopped

private boolean m_stopped

a flag to state that the network has in fact stopped.

m_accepted

private boolean m_accepted

a flag to state that the network should be accepted the way it is.

m_win

private javax.swing.JFrame m_win

The window for the network.

m_autoBuild

private boolean m_autoBuild

A flag to tell the build classifier to automatically build a neural net.

m_gui

private boolean m_gui

A flag to state that the gui for the network should be brought up. To allow interaction while training.

m_valSize

private int m_valSize

An int to say how big the validation set should be.

m_driftThreshold

private int m_driftThreshold

The number to to use to quit on validation testing.

m_randomSeed

private long m_randomSeed

The number used to seed the random number generator.

m_random

private java.util.Random m_random

The actual random number generator.

m_useNomToBin

private boolean m_useNomToBin

A flag to state that a nominal to binary filter should be used.

m_nominalToBinaryFilter

private NominalToBinary m_nominalToBinaryFilter

The actual filter.

m_hiddenLayers

private java.lang.String m_hiddenLayers

The string that defines the hidden layers

m_normalizeAttributes

private boolean m_normalizeAttributes

This flag states that the user wants the input values normalized.

m_decay

private boolean m_decay

This flag states that the user wants the learning rate to decay.

m_learningRate

private double m_learningRate

This is the learning rate for the network.

m_momentum

private double m_momentum

This is the momentum for the network.

m_epoch

private int m_epoch

Shows the number of the epoch that the network just finished.

m_error

private double m_error

Shows the error of the epoch that the network just finished.

m_reset

private boolean m_reset

This flag states that the user wants the network to restart if it is found to be generating infinity or NaN for the error value. This would restart the network with the current options except that the learning rate would be smaller than before, (perhaps half of its current value). This option will not be available if the gui is chosen (if the gui is open the user can fix the network themselves, it is an architectural minefield for the network to be reset with the gui open).

m_normalizeClass

private boolean m_normalizeClass

This flag states that the user wants the class to be normalized while processing in the network is done. (the final answer will be in the original range regardless). This option will only be used when the class is numeric.

m_sigmoidUnit

private SigmoidUnit m_sigmoidUnit

this is a sigmoid unit.

m_linearUnit

private LinearUnit m_linearUnit

This is a linear unit.

Constructor Detail

MultilayerPerceptron

public MultilayerPerceptron()

The constructor.

Method Detail

main

public static void main(java.lang.String[] argv)

Main method for testing this class.

Parameters:

argv - should contain command line options (see setOptions)

setDecay

public void setDecay(boolean d)

Parameters:

d - True if the learning rate should decay.

getDecay

public boolean getDecay()

Returns:

the flag for having the learning rate decay.

setReset

public void setReset(boolean r)

This sets the network up to be able to reset itself with the current settings and the learning rate at half of what it is currently. This will only happen if the network creates NaN or infinite errors. Also this will continue to happen until the network is trained properly. The learning rate will also get set back to it's original value at the end of this. This can only be set to true if the GUI is not brought up.

Parameters:

r - True if the network should restart with it's current options and set the learning rate to half what it currently is.

getReset

public boolean getReset()

Returns:

The flag for reseting the network.

setNormalizeNumericClass

public void setNormalizeNumericClass(boolean c)

Parameters:

c - True if the class should be normalized (the class will only ever be normalized if it is numeric). (Normalization puts the range between -1 - 1).

getNormalizeNumericClass

public boolean getNormalizeNumericClass()

Returns:

The flag for normalizing a numeric class.

setNormalizeAttributes

public void setNormalizeAttributes(boolean a)

Parameters:

a - True if the attributes should be normalized (even nominal attributes will get normalized here) (range goes between -1 - 1).

getNormalizeAttributes

public boolean getNormalizeAttributes()

Returns:

The flag for normalizing attributes.

setNominalToBinaryFilter

public void setNominalToBinaryFilter(boolean f)

Parameters:

f - True if a nominalToBinary filter should be used on the data.

getNominalToBinaryFilter

public boolean getNominalToBinaryFilter()

Returns:

The flag for nominal to binary filter use.

setRandomSeed

public void setRandomSeed(long l)

This seeds the random number generator, that is used when a random number is needed for the network.

Parameters:

l - The seed.

getRandomSeed

public long getRandomSeed()

Returns:

The seed for the random number generator.

setValidationThreshold

public void setValidationThreshold(int t)

This sets the threshold to use for when validation testing is being done. It works by ending testing once the error on the validation set has consecutively increased a certain number of times.

Parameters:

t - The threshold to use for this.

getValidationThreshold

public int getValidationThreshold()

Returns:

The threshold used for validation testing.

setLearningRate

public void setLearningRate(double l)

The learning rate can be set using this command. NOTE That this is a static variable so it affect all networks that are running. Must be greater than 0 and no more than 1.

Parameters:

l - The New learning rate.

getLearningRate

public double getLearningRate()

Returns:

The learning rate for the nodes.

setMomentum

public void setMomentum(double m)

The momentum can be set using this command. THE same conditions apply to this as to the learning rate.

Parameters:

m - The new Momentum.

getMomentum

public double getMomentum()

Returns:

The momentum for the nodes.

setAutoBuild

public void setAutoBuild(boolean a)

This will set whether the network is automatically built or if it is left up to the user. (there is nothing to stop a user from altering an autobuilt network however).

Parameters:

a - True if the network should be auto built.

getAutoBuild

public boolean getAutoBuild()

Returns:

The auto build state.

setHiddenLayers

public void setHiddenLayers(java.lang.String h)

This will set what the hidden layers are made up of when auto build is enabled. Note to have no hidden units, just put a single 0, Any more 0's will indicate that the string is badly formed and make it unaccepted. Negative numbers, and floats will do the same. There are also some wildcards. These are 'a' = (number of attributes + number of classes) / 2, 'i' = number of attributes, 'o' = number of classes, and 't' = number of attributes + number of classes.

Parameters:

h - A string with a comma seperated list of numbers. Each number is the number of nodes to be on a hidden layer.

getHiddenLayers

public java.lang.String getHiddenLayers()

Returns:

A string representing the hidden layers, each number is the number of nodes on a hidden layer.

setGUI

public void setGUI(boolean a)

This will set whether A GUI is brought up to allow interaction by the user with the neural network during training.

Parameters:

a - True if gui should be created.

getGUI

public boolean getGUI()

Returns:

The true if should show gui.

setValidationSetSize

public void setValidationSetSize(int a)

This will set the size of the validation set.

Parameters:

a - The size of the validation set, as a percentage of the whole.

getValidationSetSize

public int getValidationSetSize()

Returns:

The percentage size of the validation set.

setTrainingTime

public void setTrainingTime(int n)

Set the number of training epochs to perform. Must be greater than 0.

Parameters:

n - The number of epochs to train through.

getTrainingTime

public int getTrainingTime()

Returns:

The number of epochs to train through.

addNode

private void addNode(NeuralConnection n)

Call this function to place a node into the network list.

Parameters:

n - The node to place in the list.

removeNode

private boolean removeNode(NeuralConnection n)

Call this function to remove the passed node from the list. This will only remove the node if it is in the neuralnodes list.

Parameters:

n - The neuralConnection to remove.

Returns:

True if removed false if not (because it wasn't there).

setClassType

private Instances setClassType(Instances inst)
                        throws java.lang.Exception

This function sets what the m_numeric flag to represent the passed class it also performs the normalization of the attributes if applicable and sets up the info to normalize the class. (note that regardless of the options it will fill an array with the range and base, set to normalize all attributes and the class to be between -1 and 1)

Parameters:

inst - the instances.

Returns:

The modified instances. This needs to be done. If the attributes are normalized then deep copies will be made of all the instances which will need to be passed back out.

Throws:

java.lang.Exception

blocker

public void blocker(boolean tf)

A function used to stop the code that called buildclassifier from continuing on before the user has finished the decision tree.

Parameters:

tf - True to stop the thread, False to release the thread that is waiting there (if one).

updateDisplay

private void updateDisplay()

Call this function to update the control panel for the gui.

resetNetwork

private void resetNetwork()

this will reset all the nodes in the network.

calculateOutputs

private void calculateOutputs()

This will cause the output values of all the nodes to be calculated. Note that the m_currentInstance is used to calculate these values.

calculateErrors

private double calculateErrors()
                        throws java.lang.Exception

This will cause the error values to be calculated for all nodes. Note that the m_currentInstance is used to calculate these values. Also the output values should have been calculated first.

Returns:

The squared error.

Throws:

java.lang.Exception

updateNetworkWeights

private void updateNetworkWeights(double l,
                                  double m)

This will cause the weight values to be updated based on the learning rate, momentum and the errors that have been calculated for each node.

Parameters:

l - The learning rate to update with.

m - The momentum to update with.

setupInputs

private void setupInputs()
                  throws java.lang.Exception

This creates the required input units.

Throws:

java.lang.Exception

setupOutputs

private void setupOutputs()
                   throws java.lang.Exception

This creates the required output units.

Throws:

java.lang.Exception

setupHiddenLayer

private void setupHiddenLayer()

Call this function to automatically generate the hidden units

setEndsToLinear

private void setEndsToLinear()

This will go through all the nodes and check if they are connected to a pure output unit. If so they will be set to be linear units. If not they will be set to be sigmoid units.

buildClassifier

public void buildClassifier(Instances i)
                     throws java.lang.Exception

Call this function to build and train a neural network for the training data provided.

Specified by:

buildClassifier in class Classifier

Parameters:

i - The training data.

Throws:

Throws - exception if can't build classification properly.

java.lang.Exception - if the classifier has not been generated successfully

distributionForInstance

public double[] distributionForInstance(Instance i)
                                 throws java.lang.Exception

Call this function to predict the class of an instance once a classification model has been built with the buildClassifier call.

Overrides:

distributionForInstance in class Classifier

Parameters:

i - The instance to classify.

Returns:

A double array filled with the probabilities of each class type.

Throws:

if - can't classify instance.

java.lang.Exception - if distribution could not be computed successfully

listOptions

public java.util.Enumeration listOptions()

Returns an enumeration describing the available options.

Specified by:

listOptions in interface OptionHandler

Overrides:

listOptions in class Classifier

Returns:

an enumeration of all the available options.

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses a given list of options. Valid options are:

-L num
Set the learning rate. (default 0.3)

-M num
Set the momentum (default 0.2)

-N num
Set the number of epochs to train through. (default 500)

-V num
Set the percentage size of the validation set from the training to use. (default 0 (no validation set is used, instead num of epochs is used)

-S num
Set the seed for the random number generator. (default 0)

-E num
Set the threshold for the number of consequetive errors allowed during validation testing. (default 20)

-G
Bring up a GUI for the neural net.

-A
Do not automatically create the connections in the net. (can only be used if -G is specified)

-B
Do Not automatically Preprocess the instances with a nominal to binary filter.

-H str
Set the number of nodes to be used on each layer. Each number represents its own layer and the num of nodes on that layer. Each number should be comma seperated. There are also the wildcards 'a', 'i', 'o', 't' (default 4)

-C
Do not automatically Normalize the class if it's numeric.

-I
Do not automatically Normalize the attributes.

-R
Do not allow the network to be automatically reset.

-D
Cause the learning rate to decay as training is done.

Specified by:

setOptions in interface OptionHandler

Overrides:

setOptions in class Classifier

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of NeuralNet.

Specified by:

getOptions in interface OptionHandler

Overrides:

getOptions in class Classifier

Returns:

an array of strings suitable for passing to setOptions()

toString

public java.lang.String toString()

Returns:

string describing the model.

globalInfo

public java.lang.String globalInfo()

This will return a string describing the classifier.

Returns:

The string.

learningRateTipText

public java.lang.String learningRateTipText()

Returns:

a string to describe the learning rate option.

momentumTipText

public java.lang.String momentumTipText()

Returns:

a string to describe the momentum option.

autoBuildTipText

public java.lang.String autoBuildTipText()

Returns:

a string to describe the AutoBuild option.

randomSeedTipText

public java.lang.String randomSeedTipText()

Returns:

a string to describe the random seed option.

validationThresholdTipText

public java.lang.String validationThresholdTipText()

Returns:

a string to describe the validation threshold option.

GUITipText

public java.lang.String GUITipText()

Returns:

a string to describe the GUI option.

validationSetSizeTipText

public java.lang.String validationSetSizeTipText()

Returns:

a string to describe the validation size option.

trainingTimeTipText

public java.lang.String trainingTimeTipText()

Returns:

a string to describe the learning rate option.

nominalToBinaryFilterTipText

public java.lang.String nominalToBinaryFilterTipText()

Returns:

a string to describe the nominal to binary option.

hiddenLayersTipText

public java.lang.String hiddenLayersTipText()

Returns:

a string to describe the hidden layers in the network.

normalizeNumericClassTipText

public java.lang.String normalizeNumericClassTipText()

Returns:

a string to describe the nominal to binary option.

normalizeAttributesTipText

public java.lang.String normalizeAttributesTipText()

Returns:

a string to describe the nominal to binary option.

resetTipText

public java.lang.String resetTipText()

Returns:

a string to describe the Reset option.

decayTipText

public java.lang.String decayTipText()

Returns:

a string to describe the Decay option.