gov.sandia.cognition.learning.algorithm.ensemble

Class BaggingRegressionLearner<InputType>

java.lang.Object

gov.sandia.cognition.util.AbstractCloneableSerializable

gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>

gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends InputType,OutputType>>,ResultType>

gov.sandia.cognition.learning.algorithm.AbstractAnytimeSupervisedBatchLearner<InputType,OutputType,EnsembleType>

gov.sandia.cognition.learning.algorithm.ensemble.AbstractBaggingLearner<InputType,java.lang.Double,Evaluator<? super InputType,? extends java.lang.Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>

gov.sandia.cognition.learning.algorithm.ensemble.BaggingRegressionLearner<InputType>

Type Parameters:

InputType - The input type for supervised learning. Passed on to the internal learning algorithm. Also the input type for the learned ensemble.

All Implemented Interfaces:

AnytimeAlgorithm<AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>, IterativeAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<java.util.Collection<? extends InputOutputPair<? extends InputType,java.lang.Double>>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>, BatchLearner<java.util.Collection<? extends InputOutputPair<? extends InputType,java.lang.Double>>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>, BatchLearnerContainer<BatchLearner<? super java.util.Collection<? extends InputOutputPair<? extends InputType,java.lang.Double>>,? extends Evaluator<? super InputType,? extends java.lang.Number>>>, SupervisedBatchLearner<InputType,java.lang.Double,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>, CloneableSerializable, Randomized, java.io.Serializable, java.lang.Cloneable

@PublicationReference(title="Bagging Predictors",
                      author="Leo Breiman",
                      year=1996,
                      type=Journal,
                      publication="Machine Learning",
                      pages={123,140},
                      url="http://www.springerlink.com/index/L4780124W2874025.pdf")
public class BaggingRegressionLearner<InputType>
extends AbstractBaggingLearner<InputType,java.lang.Double,Evaluator<? super InputType,? extends java.lang.Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>

Learns an ensemble for regression by randomly sampling with replacement (duplicates allowed) some percentage of the size of the data (defaults to 100%) on each iteration to train a new ensemble member. The random sample is referred to as a bag. Each learned ensemble member is given equal weight. The idea here is that randomly sampling from the data and learning an ensemble member that has high variance (such as a decision tree) with respect to the input data, one can improve the performance of that algorithm. By default, the algorithm runs the maxIterations number of steps to create that number of ensemble members. However, one can also use out-of-bag (OOB) error on each iteration to determine a stopping criteria. The OOB error is determined by looking at the performance of the categorizer on the examples that it has not seen.

Since:

3.3.3

Author:

Justin Basilico

See Also:

BaggingCategorizerLearner, Serialized Form

Field Summary

Fields inherited from class gov.sandia.cognition.learning.algorithm.ensemble.AbstractBaggingLearner

bag, dataInBag, dataList, DEFAULT_MAX_ITERATIONS, DEFAULT_PERCENT_TO_SAMPLE, ensemble, learner, percentToSample, random

Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

data, keepGoing

Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

maxIterations

Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

DEFAULT_ITERATION, iteration

Constructor Summary

Constructors

Constructor and Description
BaggingRegressionLearner() Creates a new, empty BaggingRegressionLearner.
BaggingRegressionLearner(BatchLearner<? super java.util.Collection<? extends InputOutputPair<? extends InputType,java.lang.Double>>,? extends Evaluator<? super InputType,? extends java.lang.Number>> learner) Creates a new instance of BaggingRegressionLearner.
BaggingRegressionLearner(BatchLearner<? super java.util.Collection<? extends InputOutputPair<? extends InputType,java.lang.Double>>,? extends Evaluator<? super InputType,? extends java.lang.Number>> learner, int maxIterations, double percentToSample, java.util.Random random) Creates a new instance of BaggingRegressionLearner.

Method Summary

Modifier and Type	Method and Description
protected void	addEnsembleMember(Evaluator<? super InputType,? extends java.lang.Number> member) Adds a new member to the ensemble.
protected AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>	createInitialEnsemble() Create the initial, empty ensemble for the algorithm to use.

Methods inherited from class gov.sandia.cognition.learning.algorithm.ensemble.AbstractBaggingLearner

cleanupAlgorithm, fillBag, getBag, getDataInBag, getDataList, getEnsemble, getLearner, getPercentToSample, getRandom, getResult, initializeAlgorithm, setBag, setDataInBag, setDataList, setEnsemble, setLearner, setPercentToSample, setRandom, step

Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner

clone, getData, getKeepGoing, learn, setData, setKeepGoing, stop

Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm

getMaxIterations, isResultValid, setMaxIterations

Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm

addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner

learn

Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable

clone

Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm

getMaxIterations, setMaxIterations

Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm

addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener

Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm

isResultValid

Constructor Detail

BaggingRegressionLearner

public BaggingRegressionLearner()

Creates a new, empty BaggingRegressionLearner.

BaggingRegressionLearner

public BaggingRegressionLearner(BatchLearner<? super java.util.Collection<? extends InputOutputPair<? extends InputType,java.lang.Double>>,? extends Evaluator<? super InputType,? extends java.lang.Number>> learner)

Creates a new instance of BaggingRegressionLearner.

Parameters:

learner - The learner to use to create the categorizer on each iteration.

BaggingRegressionLearner

public BaggingRegressionLearner(BatchLearner<? super java.util.Collection<? extends InputOutputPair<? extends InputType,java.lang.Double>>,? extends Evaluator<? super InputType,? extends java.lang.Number>> learner,
                                int maxIterations,
                                double percentToSample,
                                java.util.Random random)

Creates a new instance of BaggingRegressionLearner.

Parameters:

learner - The learner to use to create the regression function on each iteration.

maxIterations - The maximum number of iterations to run for, which is also the number of learners to create.

percentToSample - The percentage of the total size of the data to sample on each iteration. Must be positive.

random - The random number generator to use.

Method Detail

createInitialEnsemble

protected AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>> createInitialEnsemble()

Description copied from class: AbstractBaggingLearner

Create the initial, empty ensemble for the algorithm to use.

Specified by:

createInitialEnsemble in class AbstractBaggingLearner<InputType,java.lang.Double,Evaluator<? super InputType,? extends java.lang.Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>

Returns:

A new ensemble for the algorithm to use.

addEnsembleMember

protected void addEnsembleMember(Evaluator<? super InputType,? extends java.lang.Number> member)

Description copied from class: AbstractBaggingLearner

Adds a new member to the ensemble.

Specified by:

addEnsembleMember in class AbstractBaggingLearner<InputType,java.lang.Double,Evaluator<? super InputType,? extends java.lang.Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends java.lang.Number>>>

Parameters:

member - The new member to add to the ensemble.