gov.sandia.cognition.learning.algorithm.factor.machine

Class FactorizationMachineStochasticGradient

java.lang.Object

gov.sandia.cognition.util.AbstractCloneableSerializable

gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>

gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends InputType,OutputType>>,ResultType>

gov.sandia.cognition.learning.algorithm.AbstractAnytimeSupervisedBatchLearner<Vector,java.lang.Double,FactorizationMachine>

gov.sandia.cognition.learning.algorithm.factor.machine.AbstractFactorizationMachineLearner

gov.sandia.cognition.learning.algorithm.factor.machine.FactorizationMachineStochasticGradient

All Implemented Interfaces:

AnytimeAlgorithm<FactorizationMachine>, IterativeAlgorithm, MeasurablePerformanceAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vector,java.lang.Double>>,FactorizationMachine>, BatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vector,java.lang.Double>>,FactorizationMachine>, SupervisedBatchLearner<Vector,java.lang.Double,FactorizationMachine>, CloneableSerializable, Randomized, java.io.Serializable, java.lang.Cloneable

@PublicationReference(title="Factorization Machines",author="Steffen Rendle",year=2010,type=Conference,publication="Proceedings of the 10th IEEE International Conference on Data Mining (ICDM)",url="http://www.inf.uni-konstanz.de/~rendle/pdf/Rendle2010FM.pdf") @PublicationReference(title="Factorization Machines with libFM",author="Steffen Rendle",year=2012,type=Journal,publication="ACM Transactions on Intelligent Systems Technology",url="http://www.csie.ntu.edu.tw/~b97053/paper/Factorization%20Machines%20with%20libFM.pdf",notes="Algorithm 1: Stochastic Gradient Descent (SGD)")
public class FactorizationMachineStochasticGradient
extends AbstractFactorizationMachineLearner
implements MeasurablePerformanceAlgorithm

Implements a Stochastic Gradient Descent (SGD) algorithm for learning a Factorization Machine.

Since:

3.4.0

Author:

Justin Basilico

See Also:

FactorizationMachine, Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.ArrayList<? extends InputOutputPair<? extends Vector,java.lang.Double>>	dataList The input data represented as a list for fast access.
static double	DEFAULT_LEARNING_RATE The default learning rate is 0.001.
protected double	learningRate The learning rate for the algorithm.
protected double	totalChange The total change in factorization machine parameters for the current iteration.
protected double	totalError The total error for the current iteration.

Fields inherited from class gov.sandia.cognition.learning.algorithm.factor.machine.AbstractFactorizationMachineLearner

biasEnabled, biasRegularization, DEFAULT_BIAS_ENABLED, DEFAULT_BIAS_REGULARIZATION, DEFAULT_FACTOR_COUNT, DEFAULT_FACTOR_REGULARIZATION, DEFAULT_MAX_ITERATIONS, DEFAULT_SEED_SCALE, DEFAULT_WEIGHT_REGULARIZATION, DEFAULT_WEIGHTS_ENABLED, dimensionality, factorCount, factorRegularization, random, result, seedScale, weightRegularization, weightsEnabled

Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

data, keepGoing

Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

maxIterations

Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

DEFAULT_ITERATION, iteration

Constructor Summary

Constructors

Constructor and Description
FactorizationMachineStochasticGradient() Creates a new FactorizationMachineStochasticGradient with default parameters.
FactorizationMachineStochasticGradient(int factorCount, double learningRate, double biasRegularization, double weightRegularization, double factorRegularization, double seedScale, int maxIterations, java.util.Random random) Creates a new AbstractFactorizationMachineLearner.

Method Summary

Modifier and Type	Method and Description
protected void	cleanupAlgorithm() Called to clean up the learning algorithm's state after learning has finished.
double	getLearningRate() Gets the learning rate.
double	getObjective() Gets the total objective, which is the mean squared error plus the regularization terms.
NamedValue<? extends java.lang.Number>	getPerformance() Gets the name-value pair that describes the current performance of the algorithm.
double	getRegularizationPenalty() Gets the regularization penalty term for the current result.
double	getTotalChange() Gets the total change from the current iteration.
double	getTotalError() Gets the total squared error from the current iteration.
protected boolean	initializeAlgorithm() Called to initialize the learning algorithm's state based on the data that is stored in the data field.
void	setLearningRate(double learningRate) Gets the learning rate.
protected boolean	step() Called to take a single step of the learning algorithm.
protected void	update(InputOutputPair<? extends Vector,java.lang.Double> example) Performs a single update of step of the stochastic gradient descent by updating according to the given example.

Methods inherited from class gov.sandia.cognition.learning.algorithm.factor.machine.AbstractFactorizationMachineLearner

getBiasRegularization, getFactorCount, getFactorRegularization, getRandom, getResult, getSeedScale, getWeightRegularization, isBiasEnabled, isFactorsEnabled, isWeightsEnabled, setBiasEnabled, setBiasRegularization, setFactorCount, setFactorRegularization, setRandom, setSeedScale, setWeightRegularization, setWeightsEnabled

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

clone, getData, getKeepGoing, learn, setData, setKeepGoing, stop

Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

getMaxIterations, isResultValid, setMaxIterations

Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner

learn

Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable

clone

Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm

getMaxIterations, setMaxIterations

Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm

addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener

Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm

isResultValid

Field Detail

DEFAULT_LEARNING_RATE

public static final double DEFAULT_LEARNING_RATE

The default learning rate is 0.001.

See Also:

Constant Field Values

learningRate

protected double learningRate

The learning rate for the algorithm. Must be positive.

dataList

protected transient java.util.ArrayList<? extends InputOutputPair<? extends Vector,java.lang.Double>> dataList

The input data represented as a list for fast access.

totalError

protected transient double totalError

The total error for the current iteration.

totalChange

protected transient double totalChange

The total change in factorization machine parameters for the current iteration.

Constructor Detail

FactorizationMachineStochasticGradient

public FactorizationMachineStochasticGradient()

Creates a new FactorizationMachineStochasticGradient with default parameters.

FactorizationMachineStochasticGradient

public FactorizationMachineStochasticGradient(int factorCount,
                                              double learningRate,
                                              double biasRegularization,
                                              double weightRegularization,
                                              double factorRegularization,
                                              double seedScale,
                                              int maxIterations,
                                              java.util.Random random)

Creates a new AbstractFactorizationMachineLearner.

Parameters:

factorCount - The number of factors to use. Zero means no factors. Cannot be negative.

learningRate - The learning rate. Must be positive.

biasRegularization - The regularization term for the bias. Cannot be negative.

weightRegularization - The regularization term for the linear weights. Cannot be negative.

factorRegularization - The regularization term for the factor matrix. Cannot be negative.

seedScale - The random initialization scale for the factors. Multiplied by a random Gaussian to initialize each factor value. Cannot be negative.

maxIterations - The maximum number of iterations for the algorithm to run. Cannot be negative.

random - The random number generator.

Method Detail

initializeAlgorithm

protected boolean initializeAlgorithm()

Description copied from class: AbstractAnytimeBatchLearner

Called to initialize the learning algorithm's state based on the data that is stored in the data field. The return value indicates if the algorithm can be run or not based on the initialization.

Overrides:

initializeAlgorithm in class AbstractFactorizationMachineLearner

Returns:

True if the learning algorithm can be run and false if it cannot.

step

protected boolean step()

Description copied from class: AbstractAnytimeBatchLearner

Called to take a single step of the learning algorithm.

Specified by:

step in class AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vector,java.lang.Double>>,FactorizationMachine>

Returns:

True if another step can be taken and false it the algorithm should halt.

update

protected void update(InputOutputPair<? extends Vector,java.lang.Double> example)

Performs a single update of step of the stochastic gradient descent by updating according to the given example.

Parameters:

example - The example to do a stochastic gradient step for.

cleanupAlgorithm

protected void cleanupAlgorithm()

Description copied from class: AbstractAnytimeBatchLearner

Called to clean up the learning algorithm's state after learning has finished.

Specified by:

cleanupAlgorithm in class AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends Vector,java.lang.Double>>,FactorizationMachine>

getTotalChange

public double getTotalChange()

Gets the total change from the current iteration.

Returns:

The total change in the parameters of the factorization machine.

getTotalError

public double getTotalError()

Gets the total squared error from the current iteration.

Returns:

The total squared error.

getRegularizationPenalty

public double getRegularizationPenalty()

Gets the regularization penalty term for the current result. It computes the squared 2-norm of the parameters of the factorization machine, each multiplied with their appropriate regularization weight.

Returns:

The regularization penalty term for the objective.

getObjective

public double getObjective()

Gets the total objective, which is the mean squared error plus the regularization terms.

Returns:

The value of the optimization objective.

getPerformance

public NamedValue<? extends java.lang.Number> getPerformance()

Description copied from interface: MeasurablePerformanceAlgorithm

Gets the name-value pair that describes the current performance of the algorithm. For most algorithms, this is the value that they are attempting to optimize.

Specified by:

getPerformance in interface MeasurablePerformanceAlgorithm

Returns:

The name-value pair that describes the current performance of the algorithm.

getLearningRate

public double getLearningRate()

Gets the learning rate. It governs the step size of the algorithm.

Returns:

The learning rate. Must be positive.

setLearningRate

public void setLearningRate(double learningRate)

Gets the learning rate. It governs the step size of the algorithm.

Parameters:

learningRate - The learning rate. Must be positive.