OnlineLearnerValidationExperiment (Cognitive Foundry)

java.lang.Object
- gov.sandia.cognition.util.AbstractCloneableSerializable
- - gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
  - - gov.sandia.cognition.learning.experiment.OnlineLearnerValidationExperiment<DataType,LearnedType,StatisticType,SummaryType>

Type Parameters:

DataType - The type of the data to perform the experiment with. It will be used as input into the learning algorithm.

LearnedType - The type of the output produced by the learning algorithm whose performance will be evaluated on each data item.

StatisticType - The type of the statistic generated by the performance evaluator on the learned object for each data item. It is created by passing the learned object plus the data item into the performance evaluator.

SummaryType - The type produced by the summarizer at the end of the experiment from a collection of the given statistics (one for each item). This represents the performance result for the learning algorithm for the whole experiment.

All Implemented Interfaces:

LearningExperiment, PerformanceEvaluator<IncrementalLearner<? super DataType,LearnedType>,java.util.Collection<? extends DataType>,SummaryType>, CloneableSerializable, java.io.Serializable, java.lang.Cloneable
```
public class OnlineLearnerValidationExperiment<DataType,LearnedType,StatisticType,SummaryType>
extends AbstractLearningExperiment
implements PerformanceEvaluator<IncrementalLearner<? super DataType,LearnedType>,java.util.Collection<? extends DataType>,SummaryType>, java.io.Serializable
```
Implements an experiment where an incremental supervised machine learning algorithm is evaluated by applying it to a set of data by successively testing on each item and then training on it.

Since:

3.0

Author:

Justin Basilico

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected int`	`numTrials` The number of trials in the experiment, which is the number of folds in the experiment.
`protected PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType>`	`performanceEvaluator` The evaluator to use to compute the performance of the learned object on each fold.
`protected java.util.ArrayList<StatisticType>`	`statistics` The performance evaluations made during the experiment.
`protected Summarizer<? super StatisticType,? extends SummaryType>`	`summarizer` The summarizer for summarizing the result of the performance evaluator from all the folds.
`protected SummaryType`	`summary` The summary of the performance evaluations made at the end of the experiment.

Fields inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
listeners

Constructor Summary

Constructors
Constructor and Description
`OnlineLearnerValidationExperiment()` Creates a new instance of IncrementalLearnerValidationExperiment.
`OnlineLearnerValidationExperiment(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType> performanceEvaluator, Summarizer<? super StatisticType,? extends SummaryType> summarizer)` Creates a new instance of IncrementalLearnerValidationExperiment.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`SummaryType`	`evaluatePerformance(IncrementalLearner<? super DataType,LearnedType> learner, java.util.Collection<? extends DataType> data)` Performs the experiment.
`int`	`getNumTrials()` Gets the number of trials.
`PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType>`	`getPerformanceEvaluator()` Gets the performance evaluator to apply to each fold.
`java.util.ArrayList<StatisticType>`	`getStatistics()` Gets the performance evaluations for the trials of the experiment.
`Summarizer<? super StatisticType,? extends SummaryType>`	`getSummarizer()` Gets the summarizer of the performance evaluations.
`SummaryType`	`getSummary()` Gets the summary of the experiment.
`protected void`	`setNumTrials(int numTrials)` Sets the number of trials.
`void`	`setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType> performanceEvaluator)` Sets the performance evaluator to apply to each fold.
`protected void`	`setStatistics(java.util.ArrayList<StatisticType> statistics)` Sets the performance evaluations for the trials of the experiment.
`void`	`setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)` Sets the summarizer of the performance evaluations.
`protected void`	`setSummary(SummaryType summary)` Sets the summary of the experiment.

Methods inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
addListener, fireExperimentEnded, fireExperimentStarted, fireTrialEnded, fireTrialStarted, getListeners, removeListener, setListeners

Methods inherited from class gov.sandia.cognition.util.AbstractCloneableSerializable
clone

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - performanceEvaluator
```
protected PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType> performanceEvaluator
```
    The evaluator to use to compute the performance of the learned object on each fold.
  - summarizer
```
protected Summarizer<? super StatisticType,? extends SummaryType> summarizer
```
    The summarizer for summarizing the result of the performance evaluator from all the folds.
  - numTrials
```
protected int numTrials
```
    The number of trials in the experiment, which is the number of folds in the experiment.
  - statistics
```
protected java.util.ArrayList<StatisticType> statistics
```
    The performance evaluations made during the experiment.
  - summary
```
protected SummaryType summary
```
    The summary of the performance evaluations made at the end of the experiment.
- Constructor Detail
  - OnlineLearnerValidationExperiment
```
public OnlineLearnerValidationExperiment()
```
    Creates a new instance of IncrementalLearnerValidationExperiment.
  - OnlineLearnerValidationExperiment
```
public OnlineLearnerValidationExperiment(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType> performanceEvaluator,
                                         Summarizer<? super StatisticType,? extends SummaryType> summarizer)
```
    Creates a new instance of IncrementalLearnerValidationExperiment.
    
    Parameters:
    
    performanceEvaluator - The evaluator to use to compute the performance of the learned object on each fold.
    
    summarizer - The summarizer for summarizing the result of the performance evaluator from all the folds.
- Method Detail
  - evaluatePerformance
```
public SummaryType evaluatePerformance(IncrementalLearner<? super DataType,LearnedType> learner,
                                       java.util.Collection<? extends DataType> data)
```
    Performs the experiment.
    
    Specified by:
    
    evaluatePerformance in interface PerformanceEvaluator<IncrementalLearner<? super DataType,LearnedType>,java.util.Collection<? extends DataType>,SummaryType>
    
    Parameters:
    
    data - The data to use.
    
    learner - The learner to perform the experiment on.
    
    Returns:
    
    The summary of the experiment.
  - getPerformanceEvaluator
```
public PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType> getPerformanceEvaluator()
```
    Gets the performance evaluator to apply to each fold.
    
    Returns:
    
    The performance evaluator to apply to each fold.
  - setPerformanceEvaluator
```
public void setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends DataType>,? extends StatisticType> performanceEvaluator)
```
    Sets the performance evaluator to apply to each fold.
    
    Parameters:
    
    performanceEvaluator - The performance evaluator to apply to each fold.
  - getSummarizer
```
public Summarizer<? super StatisticType,? extends SummaryType> getSummarizer()
```
    Gets the summarizer of the performance evaluations.
    
    Returns:
    
    The summarizer of the performance evaluations.
  - setSummarizer
```
public void setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)
```
    Sets the summarizer of the performance evaluations.
    
    Parameters:
    
    summarizer - The summarizer of the performance evaluations.
  - getNumTrials
```
public int getNumTrials()
```
    Gets the number of trials. Will be equal to the number of data points that the experiment is being run over.
    
    Specified by:
    
    getNumTrials in interface LearningExperiment
    
    Returns:
    
    The number of trials.
  - setNumTrials
```
protected void setNumTrials(int numTrials)
```
    Sets the number of trials.
    
    Parameters:
    
    numTrials - The number of trials.
  - getStatistics
```
public java.util.ArrayList<StatisticType> getStatistics()
```
    Gets the performance evaluations for the trials of the experiment.
    
    Returns:
    
    The performance evaluations for the trials of the experiment.
  - setStatistics
```
protected void setStatistics(java.util.ArrayList<StatisticType> statistics)
```
    Sets the performance evaluations for the trials of the experiment.
    
    Parameters:
    
    statistics - The performance evaluations for the trials of the experiment.
  - getSummary
```
public SummaryType getSummary()
```
    Gets the summary of the experiment.
    
    Returns:
    
    The summary of the experiment.
  - setSummary
```
protected void setSummary(SummaryType summary)
```
    Sets the summary of the experiment.
    
    Parameters:
    
    summary - The summary of the experiment.

Class OnlineLearnerValidationExperiment<DataType,LearnedType,StatisticType,SummaryType>

Field Summary

Fields inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment

Constructor Summary

Method Summary

Methods inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment

Methods inherited from class gov.sandia.cognition.util.AbstractCloneableSerializable

Methods inherited from class java.lang.Object

Field Detail

performanceEvaluator

summarizer

numTrials

statistics

summary

Constructor Detail

OnlineLearnerValidationExperiment

OnlineLearnerValidationExperiment

Method Detail

evaluatePerformance

getPerformanceEvaluator

setPerformanceEvaluator

getSummarizer

setSummarizer

getNumTrials

setNumTrials

getStatistics

setStatistics

getSummary

setSummary