gov.sandia.cognition.learning.algorithm.perceptron

Class BatchMultiPerceptron<CategoryType>

java.lang.Object

gov.sandia.cognition.util.AbstractCloneableSerializable

gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>

gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends InputType,OutputType>>,ResultType>

gov.sandia.cognition.learning.algorithm.AbstractAnytimeSupervisedBatchLearner<Vectorizable,CategoryType,LinearMultiCategorizer<CategoryType>>

gov.sandia.cognition.learning.algorithm.perceptron.BatchMultiPerceptron<CategoryType>

Type Parameters:

CategoryType - The type of output categories. Can be any type that has a valid equals and hashCode method.

All Implemented Interfaces:

AnytimeAlgorithm<LinearMultiCategorizer<CategoryType>>, IterativeAlgorithm, MeasurablePerformanceAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>, BatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>, SupervisedBatchLearner<Vectorizable,CategoryType,LinearMultiCategorizer<CategoryType>>, VectorFactoryContainer, CloneableSerializable, java.io.Serializable, java.lang.Cloneable

@PublicationReference(title="Ultraconservative Online Algorithms for Multiclass Problems",
                      author={"Koby Crammer","Yoram Singer"},
                      year=2003,
                      type=Journal,
                      publication="Journal of Machine Learning Research",
                      pages={951,991},
                      url="http://portal.acm.org/citation.cfm?id=944936")
public class BatchMultiPerceptron<CategoryType>
extends AbstractAnytimeSupervisedBatchLearner<Vectorizable,CategoryType,LinearMultiCategorizer<CategoryType>>
implements MeasurablePerformanceAlgorithm, VectorFactoryContainer

Implements a multi-class version of the standard batch Perceptron learning algorithm. It learns over labeled examples that have a vector input and an arbitrary output label. This version keeps a separate Perceptron as the prototype for each category. When it makes an error, it subtracts the input from the incorrect prototype and adds it to the correct prototype. This implementation also allows a margin to be enforced, which in the multi-class case means that the output value for the actual class must be at least the given margin from the next highest class.

Since:

3.2.0

Author:

Justin Basilico

See Also:

Perceptron, OnlinePerceptron, Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_MAX_ITERATIONS The default maximum number of iterations, 100.
static double	DEFAULT_MIN_MARGIN The default minimum margin is 0.0.
protected int	errorCount The number of errors on the most recent iteration.
protected double	minMargin The minimum margin to enforce.
protected LinearMultiCategorizer<CategoryType>	result The linear categorizer created by the algorithm.
protected VectorFactory<?>	vectorFactory The factory to create weight vectors.

Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

data, keepGoing

Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

maxIterations

Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

DEFAULT_ITERATION, iteration

Constructor Summary

Constructors

Constructor and Description
BatchMultiPerceptron() Creates a new BatchMultiPerceptron with default parameters.
BatchMultiPerceptron(int maxIterations) Creates a new BatchMultiPerceptron with the given maximum number of iterations and a default margin of 0.0.
BatchMultiPerceptron(int maxIterations, double minMargin) Creates a new BatchMultiPerceptron with the given maximum number of iterations and margin to enforce.
BatchMultiPerceptron(int maxIterations, double minMargin, VectorFactory<?> vectorFactory) Creates a new BatchMultiPerceptron with the given parameters.

Method Summary

Modifier and Type	Method and Description
protected void	cleanupAlgorithm() Called to clean up the learning algorithm's state after learning has finished.
int	getErrorCount() Gets the error count of the most recent iteration.
double	getMinMargin() Gets the minimum margin to enforce.
NamedValue<java.lang.Integer>	getPerformance() Gets the performance, which is the error count on the last iteration.
LinearMultiCategorizer<CategoryType>	getResult() Gets the current result of the algorithm.
VectorFactory<?>	getVectorFactory() Gets the VectorFactory used to create the weight vector.
protected boolean	initializeAlgorithm() Called to initialize the learning algorithm's state based on the data that is stored in the data field.
protected void	setErrorCount(int errorCount) Sets the error count of the most recent iteration.
void	setMinMargin(double minMargin) Gets the minimum margin to enforce.
protected void	setResult(LinearMultiCategorizer<CategoryType> result) Sets the result of the algorithm.
void	setVectorFactory(VectorFactory<?> vectorFactory) Sets the VectorFactory used to create the weight vector.
protected boolean	step() Called to take a single step of the learning algorithm.

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

clone, getData, getKeepGoing, learn, setData, setKeepGoing, stop

Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

getMaxIterations, isResultValid, setMaxIterations

Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner

learn

Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable

clone

Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm

getMaxIterations, setMaxIterations

Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm

addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener

Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm

isResultValid

Field Detail

DEFAULT_MAX_ITERATIONS

public static final int DEFAULT_MAX_ITERATIONS

The default maximum number of iterations, 100.

See Also:

Constant Field Values

DEFAULT_MIN_MARGIN

public static final double DEFAULT_MIN_MARGIN

The default minimum margin is 0.0.

See Also:

Constant Field Values

minMargin

protected double minMargin

The minimum margin to enforce. Must be non-negative.

vectorFactory

protected VectorFactory<?> vectorFactory

The factory to create weight vectors.

result

protected transient LinearMultiCategorizer<CategoryType> result

The linear categorizer created by the algorithm.

errorCount

protected transient int errorCount

The number of errors on the most recent iteration.

Constructor Detail

BatchMultiPerceptron

public BatchMultiPerceptron()

Creates a new BatchMultiPerceptron with default parameters.

BatchMultiPerceptron

public BatchMultiPerceptron(int maxIterations)

Creates a new BatchMultiPerceptron with the given maximum number of iterations and a default margin of 0.0.

Parameters:

maxIterations - The maximum number of iterations. Must be positive.

BatchMultiPerceptron

public BatchMultiPerceptron(int maxIterations,
                            double minMargin)

Creates a new BatchMultiPerceptron with the given maximum number of iterations and margin to enforce.

Parameters:

maxIterations - The maximum number of iterations. Must be positive.

minMargin - The minimum margin to enforce. Must be non-negative.

BatchMultiPerceptron

public BatchMultiPerceptron(int maxIterations,
                            double minMargin,
                            VectorFactory<?> vectorFactory)

Creates a new BatchMultiPerceptron with the given parameters.

Parameters:

maxIterations - The maximum number of iterations. Must be positive.

minMargin - The minimum margin to enforce. Must be non-negative.

vectorFactory - The factory to use to create weight vectors.

Method Detail

initializeAlgorithm

protected boolean initializeAlgorithm()

Description copied from class: AbstractAnytimeBatchLearner

Called to initialize the learning algorithm's state based on the data that is stored in the data field. The return value indicates if the algorithm can be run or not based on the initialization.

Specified by:

initializeAlgorithm in class AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>

Returns:

True if the learning algorithm can be run and false if it cannot.

step

protected boolean step()

Description copied from class: AbstractAnytimeBatchLearner

Called to take a single step of the learning algorithm.

Specified by:

step in class AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>

Returns:

True if another step can be taken and false it the algorithm should halt.

cleanupAlgorithm

protected void cleanupAlgorithm()

Description copied from class: AbstractAnytimeBatchLearner

Called to clean up the learning algorithm's state after learning has finished.

Specified by:

cleanupAlgorithm in class AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>

getResult

public LinearMultiCategorizer<CategoryType> getResult()

Description copied from interface: AnytimeAlgorithm

Gets the current result of the algorithm.

Specified by:

getResult in interface AnytimeAlgorithm<LinearMultiCategorizer<CategoryType>>

Returns:

Current result of the algorithm.

setResult

protected void setResult(LinearMultiCategorizer<CategoryType> result)

Sets the result of the algorithm.

Parameters:

result - The result of the algorithm.

getMinMargin

public double getMinMargin()

Gets the minimum margin to enforce. Any value less than or equal to this is considered an error for the algorithm.

Returns:

The minimum margin. Cannot be negative.

setMinMargin

public void setMinMargin(double minMargin)

Gets the minimum margin to enforce. Any value less than or equal to this is considered an error for the algorithm.

Parameters:

minMargin - The minimum margin. Cannot be negative.

getVectorFactory

public VectorFactory<?> getVectorFactory()

Gets the VectorFactory used to create the weight vector.

Specified by:

getVectorFactory in interface VectorFactoryContainer

Returns:

The VectorFactory used to create the weight vector.

setVectorFactory

public void setVectorFactory(VectorFactory<?> vectorFactory)

Sets the VectorFactory used to create the weight vector.

Parameters:

vectorFactory - The VectorFactory used to create the weight vector.

getErrorCount

public int getErrorCount()

Gets the error count of the most recent iteration.

Returns:

The current error count.

setErrorCount

protected void setErrorCount(int errorCount)

Sets the error count of the most recent iteration.

Parameters:

errorCount - The current error count.

getPerformance

public NamedValue<java.lang.Integer> getPerformance()

Gets the performance, which is the error count on the last iteration.

Specified by:

getPerformance in interface MeasurablePerformanceAlgorithm

Returns:

The performance of the algorithm.