gov.sandia.cognition.learning.algorithm.pca

Class GeneralizedHebbianAlgorithm

java.lang.Object

gov.sandia.cognition.util.AbstractCloneableSerializable

gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>

gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>

gov.sandia.cognition.learning.algorithm.pca.GeneralizedHebbianAlgorithm

All Implemented Interfaces:

AnytimeAlgorithm<PrincipalComponentsAnalysisFunction>, IterativeAlgorithm, MeasurablePerformanceAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>, BatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>, PrincipalComponentsAnalysis, CloneableSerializable, java.io.Serializable, java.lang.Cloneable

@CodeReview(reviewer="Kevin R. Dixon",
            date="2008-07-23",
            changesNeeded=false,
            comments={"Added PublicationReference to Sanger\'s master\'s thesis.","Minor changes to javadoc.","Looks fine."})
 @PublicationReference(author="Terrence D. Sanger",
                      title="Optimal Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network",
                      type=Thesis,
                      year=1989,
                      url="http://ece-classweb.ucsd.edu/winter06/ece173/documents/Sanger%201989%20--%20Optimal%20Unsupervised%20Learning%20in%20a%20Single-layer%20Linear%20FeedforwardNN.pdf")
public class GeneralizedHebbianAlgorithm
extends AbstractAnytimeBatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>
implements PrincipalComponentsAnalysis, MeasurablePerformanceAlgorithm

Implementation of the Generalized Hebbian Algorithm, also known as Sanger's Rule, which is a generalization of Oja's Rule. This algorithm is an iterative version of Principal Component Analysis. GHA finds the "num" Vectors corresponding to the "num" largest singular values of the covariance matrix of the data. The result is a VectorFunction that maps the input space onto a reduced "num" dimensional space, which captures the directions of maximal variance. The ith row in the resulting matrix approximates the i-th column of the "U" matrix of the Singular Value Decomposition. Amazingly, this implementation is faster than the time taken to perform closed-form SVD on datasets, and is practical on datasets too large for an SVD.

Since:

2.0

Author:

Kevin R. Dixon

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	PERFORMANCE_NAME The performance name is "Change".

Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

data, keepGoing

Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

maxIterations

Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

DEFAULT_ITERATION, iteration

Constructor Summary

Constructors

Constructor and Description
GeneralizedHebbianAlgorithm(int numComponents, double learningRate, int maxIterations, double minChange) Creates a new instance of GeneralizedHebbianAlgorithm

Method Summary

Modifier and Type	Method and Description
protected void	cleanupAlgorithm() Called to clean up the learning algorithm's state after learning has finished.
GeneralizedHebbianAlgorithm	clone() This makes public the clone method on the Object class and removes the exception that it throws.
double	getChange() Gets the change in in the last completed step of the algorithm.
double	getLearningRate() Getter for learningRate
double	getMinChange() Getter for minChange
int	getNumComponents() Gets the number of components used in the PCA dimension reduction.
NamedValue<java.lang.Double>	getPerformance() Gets the name-value pair that describes the current performance of the algorithm.
PrincipalComponentsAnalysisFunction	getResult() Gets the VectorFunction that maps from the input space to the reduced output space of "getNumComponents" dimensions.
protected boolean	initializeAlgorithm() Called to initialize the learning algorithm's state based on the data that is stored in the data field.
void	setLearningRate(double learningRate) Setter for learningRate
void	setMinChange(double minChange) Setter for minChange
void	setNumComponents(int numComponents) Setter for numComponents
protected void	setResult(PrincipalComponentsAnalysisFunction result) Setter for result
protected boolean	step() Called to take a single step of the learning algorithm.

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

getData, getKeepGoing, learn, setData, setKeepGoing, stop

Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

getMaxIterations, isResultValid, setMaxIterations

Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner

learn

Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm

getMaxIterations, setMaxIterations

Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm

addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener

Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm

isResultValid

Field Detail

PERFORMANCE_NAME

public static final java.lang.String PERFORMANCE_NAME

The performance name is "Change".

See Also:

Constant Field Values

Constructor Detail

GeneralizedHebbianAlgorithm

public GeneralizedHebbianAlgorithm(int numComponents,
                                   double learningRate,
                                   int maxIterations,
                                   double minChange)

Creates a new instance of GeneralizedHebbianAlgorithm

Parameters:

minChange - Minimum change below which to stop iterating, greater than or equal to zero, typically 1e-10

numComponents - Number of components to extract from the data, must be greater than zero

learningRate - Learning rate, or step size, (0,1], typically ~0.1

maxIterations - Maximum number of iterations before stopping

Method Detail

clone

public GeneralizedHebbianAlgorithm clone()

Description copied from class: AbstractCloneableSerializable

This makes public the clone method on the Object class and removes the exception that it throws. Its default behavior is to automatically create a clone of the exact type of object that the clone is called on and to copy all primitives but to keep all references, which means it is a shallow copy. Extensions of this class may want to override this method (but call super.clone() to implement a "smart copy". That is, to target the most common use case for creating a copy of the object. Because of the default behavior being a shallow copy, extending classes only need to handle fields that need to have a deeper copy (or those that need to be reset). Some of the methods in ObjectUtil may be helpful in implementing a custom clone method. Note: The contract of this method is that you must use super.clone() as the basis for your implementation.

Specified by:

clone in interface CloneableSerializable

Overrides:

clone in class AbstractAnytimeBatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>

Returns:

A clone of this object.

initializeAlgorithm

protected boolean initializeAlgorithm()

Description copied from class: AbstractAnytimeBatchLearner

Called to initialize the learning algorithm's state based on the data that is stored in the data field. The return value indicates if the algorithm can be run or not based on the initialization.

Specified by:

initializeAlgorithm in class AbstractAnytimeBatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>

Returns:

True if the learning algorithm can be run and false if it cannot.

cleanupAlgorithm

protected void cleanupAlgorithm()

Description copied from class: AbstractAnytimeBatchLearner

Called to clean up the learning algorithm's state after learning has finished.

Specified by:

cleanupAlgorithm in class AbstractAnytimeBatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>

step

protected boolean step()

Description copied from class: AbstractAnytimeBatchLearner

Called to take a single step of the learning algorithm.

Specified by:

step in class AbstractAnytimeBatchLearner<java.util.Collection<Vector>,PrincipalComponentsAnalysisFunction>

Returns:

True if another step can be taken and false it the algorithm should halt.

getLearningRate

public double getLearningRate()

Getter for learningRate

Returns:

Learning rate, or step size, (0,1], typically ~0.1

setLearningRate

public void setLearningRate(double learningRate)

Setter for learningRate

Parameters:

learningRate - Learning rate, or step size, (0,1], typically ~0.1

getMinChange

public double getMinChange()

Getter for minChange

Returns:

Minimum change below which to stop iterating, greater than or equal to zero, typically 1e-10

setMinChange

public void setMinChange(double minChange)

Setter for minChange

Parameters:

minChange - Minimum change below which to stop iterating, greater than or equal to zero, typically 1e-10

getNumComponents

public int getNumComponents()

Description copied from interface: PrincipalComponentsAnalysis

Gets the number of components used in the PCA dimension reduction.

Specified by:

getNumComponents in interface PrincipalComponentsAnalysis

Returns:

Number of components used in the PCA dimension reduction

setNumComponents

public void setNumComponents(int numComponents)

Setter for numComponents

Parameters:

numComponents - Number of components to extract from the data, must be greater than zero

getResult

public PrincipalComponentsAnalysisFunction getResult()

Description copied from interface: PrincipalComponentsAnalysis

Gets the VectorFunction that maps from the input space to the reduced output space of "getNumComponents" dimensions.

Specified by:

getResult in interface AnytimeAlgorithm<PrincipalComponentsAnalysisFunction>

Specified by:

getResult in interface PrincipalComponentsAnalysis

Returns:

PCA function that reduces the dimensionality of the input space to a (hopefully) simpler and smaller output space

setResult

protected void setResult(PrincipalComponentsAnalysisFunction result)

Setter for result

Parameters:

result - Vector function that maps the input space onto a numComponents-dimension Vector representing the directions of maximal variance (information gain). The ith row in the matrix approximates the i-th column of the "U" matrix of the Singular Value Decomposition.

getChange

public double getChange()

Gets the change in in the last completed step of the algorithm.

Returns:

The change in the last completed step of the algorithm.

getPerformance

public NamedValue<java.lang.Double> getPerformance()

Description copied from interface: MeasurablePerformanceAlgorithm

Gets the name-value pair that describes the current performance of the algorithm. For most algorithms, this is the value that they are attempting to optimize.

Specified by:

getPerformance in interface MeasurablePerformanceAlgorithm

Returns:

The name-value pair that describes the current performance of the algorithm.