gov.sandia.cognition.learning.algorithm.regression

Class LocallyWeightedFunction<InputType,OutputType>

java.lang.Object

gov.sandia.cognition.learning.algorithm.regression.LocallyWeightedFunction<InputType,OutputType>

Type Parameters:

InputType - Input class to map onto the Output class

OutputType - Output of the Evaluator

All Implemented Interfaces:

Evaluator<InputType,OutputType>

@PublicationReference(author="Andrew W. Moore",
                      title="Instance-based learning (aka Case-based or Memory-based or non-parametric)",
                      type=WebPage,
                      year=2006,
                      url="http://www.autonlab.org/tutorials/mbl.html")
public class LocallyWeightedFunction<InputType,OutputType>
extends java.lang.Object
implements Evaluator<InputType,OutputType>

LocallyWeightedFunction is a generalization of the k-nearest neighbor concept, also known as "Instance-Based Learning", "Memory-Based Learning", "Nonparametric Regression", "Case-Based Regression", or "Kernel-Based Regression". This approach essentially has no up-front learning time, but creates a local function approximation in response to a evaluate() call. The local function approximation is created by weighting the original dataset by a value given by a Kernel against each input sample in the dataset.

KernelWeightedRobustRegression is different from LocallyWeightedFunction in that KWRR creates a global function approximator and holds for all inputs. Thus, up-front learning time for KWRR is relatively high, but evaluation time is relatively low. On the other hand, LWL creates a local function approximator in response to each evaluation, and LWL does not create a global function approximator. As such, LWL has (almost) no up-front learning time, but each evaluation requires relatively high computation. The cost of LWL function evaluation depends strongly on the type of learner given to the algorithm. If you use fast or closed-form learners, then you may not notice the evaluation time. But if you use some brain-dead iterative technique, like Gradient Descent, then use LWL at your own risk.

KWRR is more appropriate when you know the general structure of your data, but it is riddled with outliers. LWL is more appropriate when you don't know/understand the general trend of your data AND you can afford evaluation time to be somewhat costly.

Author:

Kevin R. Dixon

See Also:

KernelWeightedRobustRegression

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LocallyWeightedFunction.Learner<InputType,OutputType> Learning algorithm for creating LocallyWeightedFunctions.

Constructor Summary

Constructors

Constructor and Description
LocallyWeightedFunction(Kernel<? super InputType> kernel, java.util.Collection<? extends InputOutputPair<? extends InputType,OutputType>> rawData, SupervisedBatchLearner<InputType,OutputType,?> learner) Evaluator that implements the concept of LocallyWeightedLearning.

Method Summary

Modifier and Type	Method and Description
OutputType	evaluate(InputType input) This function re-weights the dataset according to the Kernel value between the input and each input in the dataset.
Kernel<? super InputType>	getKernel() Getter for kernel
SupervisedBatchLearner<InputType,OutputType,?>	getLearner() Getter for learner
Evaluator<? super InputType,? extends OutputType>	getLocalApproximator() Getter for localApproximator
void	setKernel(Kernel<? super InputType> kernel) Setter for kernel
void	setLearner(SupervisedBatchLearner<InputType,OutputType,?> learner) Setter for learner
void	setLocalApproximator(Evaluator<? super InputType,? extends OutputType> localApproximator) Setter for localApproximator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LocallyWeightedFunction

public LocallyWeightedFunction(Kernel<? super InputType> kernel,
                               java.util.Collection<? extends InputOutputPair<? extends InputType,OutputType>> rawData,
                               SupervisedBatchLearner<InputType,OutputType,?> learner)

Evaluator that implements the concept of LocallyWeightedLearning. That is, given an input point, this function re-weights the dataset according to how "close" the dataset inputs are to the given input. An inner-loop learner then uses the re-weighted to compute a local function approximator for this input. The output of this class is the output of the local function approximator, which is recomputed each time evaluate() is called. Thus, evaluate() on this method is relatively expensive (because it calls learn() on the given BatchLearner)

Parameters:

kernel - Kernel that provides the weights between an input and each sample in the input dataset

rawData - Original (weighted) dataset

learner - Learner that takes the Collection of WeightedInputOutputPairs from the Kernel reweighting and creates a local function approximation at the given input. I would strongly recommend using fast or closed-form learners for this.

Method Detail

getKernel

public Kernel<? super InputType> getKernel()

Getter for kernel

Returns:

Kernel that provides the weights between an input and each sample in the input dataset

setKernel

public void setKernel(Kernel<? super InputType> kernel)

Setter for kernel

Parameters:

kernel - Kernel that provides the weights between an input and each sample in the input dataset

getLearner

public SupervisedBatchLearner<InputType,OutputType,?> getLearner()

Getter for learner

Returns:

Learner that takes the Collection of WeightedInputOutputPairs from the Kernel reweighting and creates a local function approximation at the given input. I would strongly recommend using fast or closed-form learners for this.

setLearner

public void setLearner(SupervisedBatchLearner<InputType,OutputType,?> learner)

Setter for learner

Parameters:

learner - Learner that takes the Collection of WeightedInputOutputPairs from the Kernel reweighting and creates a local function approximation at the given input. I would strongly recommend using fast or closed-form learners for this.

evaluate

public OutputType evaluate(InputType input)

This function re-weights the dataset according to the Kernel value between the input and each input in the dataset. This re-weighted dataset is then given to the learner to create a local approximator that then evaluate the input to produce the prediction

Specified by:

evaluate in interface Evaluator<InputType,OutputType>

Parameters:

input - Input to create a local approximator for, using the Kernel to weight the original dataset

Returns:

Approximation at the given input using the Kernel weights, the original (weighted) dataset, and the BatchLearner

getLocalApproximator

public Evaluator<? super InputType,? extends OutputType> getLocalApproximator()

Getter for localApproximator

Returns:

Dataset containing the weights in response to an evaluate() call. The weights in this dataset will be a product of the original dataset weights times the weights from the Kernel response to the given input to the evaluate() method call.

setLocalApproximator

public void setLocalApproximator(Evaluator<? super InputType,? extends OutputType> localApproximator)

Setter for localApproximator

Parameters:

localApproximator - Dataset containing the weights in response to an evaluate() call. The weights in this dataset will be a product of the original dataset weights times the weights from the Kernel response to the given input to the evaluate() method call.