gov.sandia.cognition.learning.algorithm.perceptron

Class OnlineShiftingPerceptron

java.lang.Object

gov.sandia.cognition.util.AbstractCloneableSerializable

gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner<InputOutputPair<? extends InputType,OutputType>,ResultType>

gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>

gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner

gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner

gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner

gov.sandia.cognition.learning.algorithm.perceptron.OnlineShiftingPerceptron

All Implemented Interfaces:

BatchAndIncrementalLearner<InputOutputPair<? extends Vectorizable,java.lang.Boolean>,LinearBinaryCategorizer>, BatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vectorizable,java.lang.Boolean>>,LinearBinaryCategorizer>, IncrementalLearner<InputOutputPair<? extends Vectorizable,java.lang.Boolean>,LinearBinaryCategorizer>, KernelizableBinaryCategorizerOnlineLearner, SupervisedBatchAndIncrementalLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>, SupervisedBatchLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>, SupervisedIncrementalLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>, VectorFactoryContainer, CloneableSerializable, java.io.Serializable, java.lang.Cloneable

@PublicationReference(author={"Giovanni Cavallanti","Nicolo Cesa-Bianchi","Claudio Gentile"},
                      title="Tracking the best hyperplane with a simple budget Perceptron",
                      year=2007,
                      type=Journal,
                      publication="Machine Learning",
                      pages={143,167},
                      url="http://www.springerlink.com/index/H40NV525LX161227.pdf")
public class OnlineShiftingPerceptron
extends AbstractLinearCombinationOnlineLearner

An implementation of the Shifting Perceptron algorithm. It is like the Perceptron except that it adds a decay to the existing weights, which is parameterized by lambda.

Since:

3.3.0

Author:

Justin Basilico

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	OnlineShiftingPerceptron.LinearResult This is the result learned by the shifting perceptron.

Field Summary

Fields

Modifier and Type	Field and Description
static double	DEFAULT_LAMBDA The default value of lambda is 0.001.
static boolean	DEFAULT_UPDATE_BIAS Algorithm does not update the bias by default.
protected double	lambda The lambda parameter for controlling how much shifting occurs.

Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner

updateBias

Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner

vectorFactory

Constructor Summary

Constructors

Constructor and Description
OnlineShiftingPerceptron() Creates a new OnlineShiftingPerceptron with default parameters.
OnlineShiftingPerceptron(double lambda) Creates a new OnlineShiftingPerceptron with the given parameters.
OnlineShiftingPerceptron(double lambda, VectorFactory<?> vectorFactory) Creates a new OnlineShiftingPerceptron with the given parameters.

Method Summary

Modifier and Type	Method and Description
protected <InputType> double	computeDecay(DefaultKernelBinaryCategorizer<InputType> target, InputType input, boolean actualCategory, double predicted, double update) Computes the decay scalar for the existing weights.
protected double	computeDecay(LinearBinaryCategorizer target, Vector input, boolean actualCategory, double predicted, double update) Computes the decay scalar for the existing weight vector.
<InputType> double	computeUpdate(DefaultKernelBinaryCategorizer<InputType> target, InputType input, boolean label, double predicted) Compute the update weight in the linear case.
double	computeUpdate(LinearBinaryCategorizer target, Vector input, boolean label, double predicted) Compute the update weight in the linear case.
LinearBinaryCategorizer	createInitialLearnedObject() Creates a new initial learned object, before any data is given.
double	getLambda() Gets the lambda parameter, which controls how much shifting and decay there is in the weight vector.
void	setLambda(double lambda) Sets the lambda parameter, which controls how much shifting and decay there is in the weight vector.

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner

computeRescaling, computeRescaling, createInitialLearnedObject, initialize, initialize, isUpdateBias, setUpdateBias, update, update

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner

createKernelLearner, learn, update, update, update

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner

getVectorFactory, setVectorFactory, update

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner

update

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner

clone, learn, learn, update

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface gov.sandia.cognition.learning.algorithm.SupervisedIncrementalLearner

update

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchAndIncrementalLearner

learn

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner

learn

Methods inherited from interface gov.sandia.cognition.learning.algorithm.IncrementalLearner

update, update

Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable

clone

Field Detail

DEFAULT_UPDATE_BIAS

public static final boolean DEFAULT_UPDATE_BIAS

Algorithm does not update the bias by default.

See Also:

Constant Field Values

DEFAULT_LAMBDA

public static final double DEFAULT_LAMBDA

The default value of lambda is 0.001.

See Also:

Constant Field Values

lambda

protected double lambda

The lambda parameter for controlling how much shifting occurs.

Constructor Detail

OnlineShiftingPerceptron

public OnlineShiftingPerceptron()

Creates a new OnlineShiftingPerceptron with default parameters.

OnlineShiftingPerceptron

public OnlineShiftingPerceptron(double lambda)

Creates a new OnlineShiftingPerceptron with the given parameters.

Parameters:

lambda - The lambda parameter to control the amount of shift. Must be positive.

OnlineShiftingPerceptron

public OnlineShiftingPerceptron(double lambda,
                                VectorFactory<?> vectorFactory)

Creates a new OnlineShiftingPerceptron with the given parameters.

Parameters:

lambda - The lambda parameter to control the amount of shift. Must be positive.

vectorFactory - The vector factory to use.

Method Detail

createInitialLearnedObject

public LinearBinaryCategorizer createInitialLearnedObject()

Description copied from interface: IncrementalLearner

Creates a new initial learned object, before any data is given.

Specified by:

createInitialLearnedObject in interface IncrementalLearner<InputOutputPair<? extends Vectorizable,java.lang.Boolean>,LinearBinaryCategorizer>

Overrides:

createInitialLearnedObject in class AbstractOnlineLinearBinaryCategorizerLearner

Returns:

The initial learned object.

computeUpdate

public double computeUpdate(LinearBinaryCategorizer target,
                            Vector input,
                            boolean label,
                            double predicted)

Description copied from class: AbstractLinearCombinationOnlineLearner

Compute the update weight in the linear case. Must be implemented by subclasses.

Specified by:

computeUpdate in class AbstractLinearCombinationOnlineLearner

Parameters:

target - Target to compute the update for.

input - Input to use in computing the update.

label - The actual category of the input.

predicted - The predicted category of the input.

Returns:

The update weight for how much to add the input to the target. May be zero if no update is needed.

computeUpdate

public <InputType> double computeUpdate(DefaultKernelBinaryCategorizer<InputType> target,
                                        InputType input,
                                        boolean label,
                                        double predicted)

Description copied from class: AbstractLinearCombinationOnlineLearner

Compute the update weight in the linear case. Must be implemented by subclasses.

Specified by:

computeUpdate in class AbstractLinearCombinationOnlineLearner

Type Parameters:

InputType - The input value for learning.

Parameters:

target - Target to compute the update for.

input - Input to use in computing the update.

label - The actual category of the input.

predicted - The predicted category of the input.

Returns:

The update weight for how much to add the input to the target. May be zero if no update is needed.

computeDecay

protected double computeDecay(LinearBinaryCategorizer target,
                              Vector input,
                              boolean actualCategory,
                              double predicted,
                              double update)

Description copied from class: AbstractLinearCombinationOnlineLearner

Computes the decay scalar for the existing weight vector. Can be overridden. The default implementation just returns 1.0, which means no change. Typically this will be a value between 0.0 and 1.0.

Overrides:

computeDecay in class AbstractLinearCombinationOnlineLearner

Parameters:

target - Target to compute the update for.

input - Input to use in computing the update.

actualCategory - The actual category of the input.

predicted - The predicted category of the input.

update - The value from the computeUpdate step.

Returns:

The decay to apply to the weight vector. Usually between 0.0 and 1.0.

computeDecay

protected <InputType> double computeDecay(DefaultKernelBinaryCategorizer<InputType> target,
                                          InputType input,
                                          boolean actualCategory,
                                          double predicted,
                                          double update)

Description copied from class: AbstractLinearCombinationOnlineLearner

Computes the decay scalar for the existing weights. Can be overridden. The default implementation just returns 1.0, which means no change. Typically this will be a value between 0.0 and 1.0.

Overrides:

computeDecay in class AbstractLinearCombinationOnlineLearner

Type Parameters:

InputType - The input value for learning.

Parameters:

target - Target to compute the update for.

input - Input to use in computing the update.

actualCategory - The actual category of the input.

predicted - The predicted category of the input.

update - The value from the computeUpdate step.

Returns:

The decay to apply to the weights. Usually between 0.0 and 1.0.

getLambda

public double getLambda()

Gets the lambda parameter, which controls how much shifting and decay there is in the weight vector.

Returns:

The lambda parameter. Must be positive.

setLambda

public void setLambda(double lambda)

Sets the lambda parameter, which controls how much shifting and decay there is in the weight vector.

Parameters:

lambda - The lambda parameter. Must be positive.