gov.sandia.cognition.learning.algorithm.perceptron

Class OnlinePassiveAggressivePerceptron.LinearSoftMargin

java.lang.Object

gov.sandia.cognition.util.AbstractCloneableSerializable

gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner<InputOutputPair<? extends InputType,OutputType>,ResultType>

gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>

gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner

gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner

gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner

gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron

gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.AbstractSoftMargin

gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.LinearSoftMargin

All Implemented Interfaces:

BatchAndIncrementalLearner<InputOutputPair<? extends Vectorizable,java.lang.Boolean>,LinearBinaryCategorizer>, BatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vectorizable,java.lang.Boolean>>,LinearBinaryCategorizer>, IncrementalLearner<InputOutputPair<? extends Vectorizable,java.lang.Boolean>,LinearBinaryCategorizer>, KernelizableBinaryCategorizerOnlineLearner, SupervisedBatchAndIncrementalLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>, SupervisedBatchLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>, SupervisedIncrementalLearner<Vectorizable,java.lang.Boolean,LinearBinaryCategorizer>, VectorFactoryContainer, CloneableSerializable, java.io.Serializable, java.lang.Cloneable

Enclosing class:

OnlinePassiveAggressivePerceptron

public static class OnlinePassiveAggressivePerceptron.LinearSoftMargin
extends OnlinePassiveAggressivePerceptron.AbstractSoftMargin

An implementation of the linear soft-margin variant of the Passive- Aggressive algorithm (PA-I). It trades-off quickly responding to errors with keeping around history in the case of non-linearly separable data.

Solves the optimization problem:
w_{t+1} = argmin_{w \in R^n} 0.5 ||w - w_t||^2 + C * \epsilon such that l(w; (x_t, y_t)) <= \epslion \epsilon >= 0 where l(w; (x_t, y_t)) is the hinge loss (0 if y(w * x) >= 1) and 1 - y(w * x) otherwise.

The actual update step of the PA algorithm is simple: w_{t+1} = w_t + \tau_t * y_t * x_t where \tau_t = min(C, l_t / ||x_t||^2)

Note: This algorithm does not modify the bias of the linear categorizer. Thus, it is recommended that a bias term be added to the inputs prior to use.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron

OnlinePassiveAggressivePerceptron.AbstractSoftMargin, OnlinePassiveAggressivePerceptron.LinearSoftMargin, OnlinePassiveAggressivePerceptron.QuadraticSoftMargin

Field Summary

Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.AbstractSoftMargin

aggressiveness, DEFAULT_AGGRESSIVENESS

Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron

DEFAULT_UPDATE_BIAS

Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner

updateBias

Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner

vectorFactory

Constructor Summary

Constructors

Constructor and Description
LinearSoftMargin() Creates a new LinearSoftMargin with default parameters.
LinearSoftMargin(double aggressiveness) Creates a new LinearSoftMargin with the given aggressiveness.
LinearSoftMargin(double aggressiveness, VectorFactory<?> vectorFactory) Creates a new LinearSoftMargin with the given parameters.

Method Summary

Modifier and Type	Method and Description
protected double	computeUpdate(double actual, double predicted, double loss, double inputNorm2Squared) Compute the update value (tau) for the algorithm.

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.AbstractSoftMargin

getAggressiveness, setAggressiveness

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron

computeUpdate, computeUpdate

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner

computeDecay, computeDecay, computeRescaling, computeRescaling, createInitialLearnedObject, initialize, initialize, isUpdateBias, setUpdateBias, update, update

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner

createKernelLearner, learn, update, update, update

Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner

createInitialLearnedObject, getVectorFactory, setVectorFactory, update

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner

update

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner

clone, learn, learn, update

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface gov.sandia.cognition.learning.algorithm.SupervisedIncrementalLearner

update

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchAndIncrementalLearner

learn

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner

learn

Methods inherited from interface gov.sandia.cognition.learning.algorithm.IncrementalLearner

createInitialLearnedObject, update, update

Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable

clone

Constructor Detail

LinearSoftMargin

public LinearSoftMargin()

Creates a new LinearSoftMargin with default parameters.

LinearSoftMargin

public LinearSoftMargin(double aggressiveness)

Creates a new LinearSoftMargin with the given aggressiveness.

Parameters:

aggressiveness - The aggressiveness. Must be positive.

LinearSoftMargin

public LinearSoftMargin(double aggressiveness,
                        VectorFactory<?> vectorFactory)

Creates a new LinearSoftMargin with the given parameters.

Parameters:

aggressiveness - The aggressiveness. Must be positive.

vectorFactory - The factory to use to create new weight vectors.

Method Detail

computeUpdate

protected double computeUpdate(double actual,
                               double predicted,
                               double loss,
                               double inputNorm2Squared)

Description copied from class: OnlinePassiveAggressivePerceptron

Compute the update value (tau) for the algorithm. Other variants of the algorithm should override this method.

Overrides:

computeUpdate in class OnlinePassiveAggressivePerceptron

Parameters:

actual - The actual label represented as a double (-1 or +1).

predicted - The value predicted by the current categorizer (w * x + b).

loss - The loss function (1 - predicted).

inputNorm2Squared - The squared 2-norm of the input (||x||^2).

Returns:

The update value to scale the input by. Should be > 0.0. May be 0.0 in degenerate cases such as a zero-vector input.