LearnerValidationExperiment (Cognitive Foundry)

java.lang.Object
- gov.sandia.cognition.util.AbstractCloneableSerializable
- - gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
  - - gov.sandia.cognition.learning.experiment.AbstractValidationFoldExperiment<InputDataType,FoldDataType>
    - - gov.sandia.cognition.learning.experiment.LearnerValidationExperiment<InputDataType,FoldDataType,LearnedType,StatisticType,SummaryType>

Type Parameters:

InputDataType - The type of the data to perform the experiment with. This will be passed to the fold creator to create a number of folds on which to validate the performance of the learning algorithm.

FoldDataType - The type of data created by the fold creator that will go into the learning algorithm. Typically, this is the same as the InputDataType, but it does not need to be. It just needs to match the output of the fold creator and the input of the learning algorithm.

LearnedType - The type of the output produced by the learning algorithm whose performance will be evaluated on each fold of data.

StatisticType - The type of the statistic generated by the performance evaluator on the learned object for each fold. It is created by passing the learned object plus the test data for the fold into the performance evaluator.

SummaryType - The type produced by the summarizer at the end of the experiment from a collection of the given statistics (one for each fold). This represents the performance result for the learning algorithm for the whole experiment.

All Implemented Interfaces:

LearningExperiment, PerformanceEvaluator<BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType>,java.util.Collection<? extends InputDataType>,SummaryType>, CloneableSerializable, java.io.Serializable, java.lang.Cloneable

Direct Known Subclasses:

ParallelLearnerValidationExperiment, SupervisedLearnerValidationExperiment
```
@PublicationReference(author="Wikipedia",
                      title="Decriptive statistics",
                      type=WebPage,
                      year=2008,
                      url="http://en.wikipedia.org/wiki/Descriptive_statistics")
public class LearnerValidationExperiment<InputDataType,FoldDataType,LearnedType,StatisticType,SummaryType>
extends AbstractValidationFoldExperiment<InputDataType,FoldDataType>
implements PerformanceEvaluator<BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType>,java.util.Collection<? extends InputDataType>,SummaryType>
```
The LearnerValidationExperiment class implements an experiment where a supervised machine learning algorithm is evaluated by applying it to a set of folds created from a given set of data.

Since:

2.0

Author:

Justin Basilico

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType>`	`performanceEvaluator` The evaluator to use to compute the performance of the learned object on each fold.
`protected java.util.ArrayList<StatisticType>`	`statistics` The performance evaluations made during the experiment.
`protected Summarizer<? super StatisticType,? extends SummaryType>`	`summarizer` The summarizer for summarizing the result of the performance evaluator from all the folds.
`protected SummaryType`	`summary` The summary of the performance evaluations made at the end of the experiment.

Fields inherited from class gov.sandia.cognition.learning.experiment.AbstractValidationFoldExperiment
foldCreator, numTrials

Fields inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
listeners

Constructor Summary

Constructors
Constructor and Description
`LearnerValidationExperiment()` Creates a new instance of SupervisedLearnerExperiment.
`LearnerValidationExperiment(ValidationFoldCreator<InputDataType,FoldDataType> foldCreator, PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator, Summarizer<? super StatisticType,? extends SummaryType> summarizer)` Creates a new instance of SupervisedLearnerExperiment.

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`SummaryType`	`evaluate(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner, java.util.Collection<? extends InputDataType> data)` Deprecated. Use evaluatePerformance instead. Performs the experiment.
`SummaryType`	`evaluatePerformance(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner, java.util.Collection<? extends InputDataType> data)` Evaluates the performance of an object with regards to given data.
`BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType>`	`getLearner()` Gets the learner the experiment is being run on.
`PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType>`	`getPerformanceEvaluator()` Gets the performance evaluator to apply to each fold.
`java.util.ArrayList<StatisticType>`	`getStatistics()` Gets the performance evaluations for the trials of the experiment.
`Summarizer<? super StatisticType,? extends SummaryType>`	`getSummarizer()` Gets the summarizer of the performance evaluations.
`SummaryType`	`getSummary()` Gets the summary of the experiment.
`protected void`	`runTrial(PartitionedDataset<FoldDataType> fold)` Runs a single trial of the experiment on one fold of the data.
`protected void`	`setLearner(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner)` Sets the learner the experiment is being run on.
`void`	`setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator)` Sets the performance evaluator to apply to each fold.
`protected void`	`setStatistics(java.util.ArrayList<StatisticType> statistics)` Sets the performance evaluations for the trials of the experiment.
`void`	`setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)` Sets the summarizer of the performance evaluations.
`protected void`	`setSummary(SummaryType summary)` Sets the summary of the experiment.

Methods inherited from class gov.sandia.cognition.learning.experiment.AbstractValidationFoldExperiment
getFoldCreator, getNumTrials, runExperiment, setFoldCreator, setNumTrials

Methods inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
addListener, fireExperimentEnded, fireExperimentStarted, fireTrialEnded, fireTrialStarted, getListeners, removeListener, setListeners

Methods inherited from class gov.sandia.cognition.util.AbstractCloneableSerializable
clone

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - performanceEvaluator
```
protected PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator
```
    The evaluator to use to compute the performance of the learned object on each fold.
  - summarizer
```
protected Summarizer<? super StatisticType,? extends SummaryType> summarizer
```
    The summarizer for summarizing the result of the performance evaluator from all the folds.
  - statistics
```
protected java.util.ArrayList<StatisticType> statistics
```
    The performance evaluations made during the experiment.
  - summary
```
protected SummaryType summary
```
    The summary of the performance evaluations made at the end of the experiment.
- Constructor Detail
  - LearnerValidationExperiment
```
public LearnerValidationExperiment()
```
    Creates a new instance of SupervisedLearnerExperiment.
  - LearnerValidationExperiment
```
public LearnerValidationExperiment(ValidationFoldCreator<InputDataType,FoldDataType> foldCreator,
                                   PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator,
                                   Summarizer<? super StatisticType,? extends SummaryType> summarizer)
```
    Creates a new instance of SupervisedLearnerExperiment.
    
    Parameters:
    
    foldCreator - The object to use for creating the folds.
    
    performanceEvaluator - The evaluator to use to compute the performance of the learned object on each fold.
    
    summarizer - The summarizer for summarizing the result of the performance evaluator from all the folds.
- Method Detail
  - evaluate
```
@Deprecated
public SummaryType evaluate(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner,
                                        java.util.Collection<? extends InputDataType> data)
```
    Deprecated. Use evaluatePerformance instead. Performs the experiment.
    
    Parameters:
    
    data - The data to use.
    
    learner - The learner to perform the experiment on.
    
    Returns:
    
    The summary of the experiment.
  - evaluatePerformance
```
public SummaryType evaluatePerformance(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner,
                                       java.util.Collection<? extends InputDataType> data)
```
    Description copied from interface: PerformanceEvaluator
    
    Evaluates the performance of an object with regards to given data.
    
    Specified by:
    
    evaluatePerformance in interface PerformanceEvaluator<BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType>,java.util.Collection<? extends InputDataType>,SummaryType>
    
    Parameters:
    
    learner - The object to evaluate the performance of.
    
    data - The data to evaluate the object using.
    
    Returns:
    
    The performance evaluation result.
  - runTrial
```
protected void runTrial(PartitionedDataset<FoldDataType> fold)
```
    Description copied from class: AbstractValidationFoldExperiment
    
    Runs a single trial of the experiment on one fold of the data.
    
    Specified by:
    
    runTrial in class AbstractValidationFoldExperiment<InputDataType,FoldDataType>
    
    Parameters:
    
    fold - The fold to run the trial of the experiment on.
  - getPerformanceEvaluator
```
public PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> getPerformanceEvaluator()
```
    Gets the performance evaluator to apply to each fold.
    
    Returns:
    
    The performance evaluator to apply to each fold.
  - setPerformanceEvaluator
```
public void setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator)
```
    Sets the performance evaluator to apply to each fold.
    
    Parameters:
    
    performanceEvaluator - The performance evaluator to apply to each fold.
  - getSummarizer
```
public Summarizer<? super StatisticType,? extends SummaryType> getSummarizer()
```
    Gets the summarizer of the performance evaluations.
    
    Returns:
    
    The summarizer of the performance evaluations.
  - setSummarizer
```
public void setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)
```
    Sets the summarizer of the performance evaluations.
    
    Parameters:
    
    summarizer - The summarizer of the performance evaluations.
  - getLearner
```
public BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> getLearner()
```
    Gets the learner the experiment is being run on.
    
    Returns:
    
    The learner.
  - setLearner
```
protected void setLearner(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner)
```
    Sets the learner the experiment is being run on.
    
    Parameters:
    
    learner - The learner.
  - getStatistics
```
public java.util.ArrayList<StatisticType> getStatistics()
```
    Gets the performance evaluations for the trials of the experiment.
    
    Returns:
    
    The performance evaluations for the trials of the experiment.
  - setStatistics
```
protected void setStatistics(java.util.ArrayList<StatisticType> statistics)
```
    Sets the performance evaluations for the trials of the experiment.
    
    Parameters:
    
    statistics - The performance evaluations for the trials of the experiment.
  - getSummary
```
public SummaryType getSummary()
```
    Gets the summary of the experiment.
    
    Returns:
    
    The summary of the experiment.
  - setSummary
```
protected void setSummary(SummaryType summary)
```
    Sets the summary of the experiment.
    
    Parameters:
    
    summary - The summary of the experiment.

Class LearnerValidationExperiment<InputDataType,FoldDataType,LearnedType,StatisticType,SummaryType>

Field Summary

Fields inherited from class gov.sandia.cognition.learning.experiment.AbstractValidationFoldExperiment

Fields inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment

Constructor Summary

Method Summary

Methods inherited from class gov.sandia.cognition.learning.experiment.AbstractValidationFoldExperiment

Methods inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment

Methods inherited from class gov.sandia.cognition.util.AbstractCloneableSerializable

Methods inherited from class java.lang.Object

Field Detail

performanceEvaluator

summarizer

statistics

summary

Constructor Detail

LearnerValidationExperiment

LearnerValidationExperiment

Method Detail

evaluate

evaluatePerformance

runTrial

getPerformanceEvaluator

setPerformanceEvaluator

getSummarizer

setSummarizer

getLearner

setLearner

getStatistics

setStatistics

getSummary

setSummary