InputDataType
- The type of the data to perform the experiment with.
This will be passed to the fold creator to create a number of folds
on which to validate the performance of the learning algorithm.FoldDataType
- The type of data created by the fold creator that will go into
the learning algorithm. Typically, this is the same as the
InputDataType, but it does not need to be. It just needs to match
the output of the fold creator and the input of the learning
algorithm.LearnedType
- The type of the output produced by the learning
algorithm whose performance will be evaluated on each fold of data.StatisticType
- The type of the statistic generated by the
performance evaluator on the learned object for each fold. It is
created by passing the learned object plus the test data for the
fold into the performance evaluator.SummaryType
- The type produced by the summarizer at the end of
the experiment from a collection of the given statistics (one for
each fold). This represents the performance result for the learning
algorithm for the whole experiment.@PublicationReference(author="Wikipedia", title="Decriptive statistics", type=WebPage, year=2008, url="http://en.wikipedia.org/wiki/Descriptive_statistics") public class LearnerValidationExperiment<InputDataType,FoldDataType,LearnedType,StatisticType,SummaryType> extends AbstractValidationFoldExperiment<InputDataType,FoldDataType> implements PerformanceEvaluator<BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType>,java.util.Collection<? extends InputDataType>,SummaryType>
LearnerValidationExperiment
class implements an experiment where
a supervised machine learning algorithm is evaluated by applying it to a set
of folds created from a given set of data.Modifier and Type | Field and Description |
---|---|
protected PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> |
performanceEvaluator
The evaluator to use to compute the performance of the learned object on
each fold.
|
protected java.util.ArrayList<StatisticType> |
statistics
The performance evaluations made during the experiment.
|
protected Summarizer<? super StatisticType,? extends SummaryType> |
summarizer
The summarizer for summarizing the result of the performance evaluator
from all the folds.
|
protected SummaryType |
summary
The summary of the performance evaluations made at the end of the
experiment.
|
foldCreator, numTrials
listeners
Constructor and Description |
---|
LearnerValidationExperiment()
Creates a new instance of SupervisedLearnerExperiment.
|
LearnerValidationExperiment(ValidationFoldCreator<InputDataType,FoldDataType> foldCreator,
PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator,
Summarizer<? super StatisticType,? extends SummaryType> summarizer)
Creates a new instance of SupervisedLearnerExperiment.
|
Modifier and Type | Method and Description |
---|---|
SummaryType |
evaluate(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner,
java.util.Collection<? extends InputDataType> data)
Deprecated.
Use evaluatePerformance instead.
Performs the experiment.
|
SummaryType |
evaluatePerformance(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner,
java.util.Collection<? extends InputDataType> data)
Evaluates the performance of an object with regards to given data.
|
BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> |
getLearner()
Gets the learner the experiment is being run on.
|
PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> |
getPerformanceEvaluator()
Gets the performance evaluator to apply to each fold.
|
java.util.ArrayList<StatisticType> |
getStatistics()
Gets the performance evaluations for the trials of the experiment.
|
Summarizer<? super StatisticType,? extends SummaryType> |
getSummarizer()
Gets the summarizer of the performance evaluations.
|
SummaryType |
getSummary()
Gets the summary of the experiment.
|
protected void |
runTrial(PartitionedDataset<FoldDataType> fold)
Runs a single trial of the experiment on one fold of the data.
|
protected void |
setLearner(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner)
Sets the learner the experiment is being run on.
|
void |
setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator)
Sets the performance evaluator to apply to each fold.
|
protected void |
setStatistics(java.util.ArrayList<StatisticType> statistics)
Sets the performance evaluations for the trials of the experiment.
|
void |
setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)
Sets the summarizer of the performance evaluations.
|
protected void |
setSummary(SummaryType summary)
Sets the summary of the experiment.
|
getFoldCreator, getNumTrials, runExperiment, setFoldCreator, setNumTrials
addListener, fireExperimentEnded, fireExperimentStarted, fireTrialEnded, fireTrialStarted, getListeners, removeListener, setListeners
clone
protected PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator
protected Summarizer<? super StatisticType,? extends SummaryType> summarizer
protected java.util.ArrayList<StatisticType> statistics
protected SummaryType summary
public LearnerValidationExperiment()
public LearnerValidationExperiment(ValidationFoldCreator<InputDataType,FoldDataType> foldCreator, PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator, Summarizer<? super StatisticType,? extends SummaryType> summarizer)
foldCreator
- The object to use for creating the folds.performanceEvaluator
- The evaluator to use to compute the
performance of the learned object on each fold.summarizer
- The summarizer for summarizing the result of the
performance evaluator from all the folds.@Deprecated public SummaryType evaluate(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner, java.util.Collection<? extends InputDataType> data)
data
- The data to use.learner
- The learner to perform the experiment on.public SummaryType evaluatePerformance(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner, java.util.Collection<? extends InputDataType> data)
PerformanceEvaluator
evaluatePerformance
in interface PerformanceEvaluator<BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType>,java.util.Collection<? extends InputDataType>,SummaryType>
learner
- The object to evaluate the performance of.data
- The data to evaluate the object using.protected void runTrial(PartitionedDataset<FoldDataType> fold)
AbstractValidationFoldExperiment
runTrial
in class AbstractValidationFoldExperiment<InputDataType,FoldDataType>
fold
- The fold to run the trial of the experiment on.public PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> getPerformanceEvaluator()
public void setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super java.util.Collection<? extends FoldDataType>,? extends StatisticType> performanceEvaluator)
performanceEvaluator
- The performance evaluator to apply to each fold.public Summarizer<? super StatisticType,? extends SummaryType> getSummarizer()
public void setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)
summarizer
- The summarizer of the performance evaluations.public BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> getLearner()
protected void setLearner(BatchLearner<? super java.util.Collection<? extends FoldDataType>,? extends LearnedType> learner)
learner
- The learner.public java.util.ArrayList<StatisticType> getStatistics()
protected void setStatistics(java.util.ArrayList<StatisticType> statistics)
statistics
- The performance evaluations for the trials of the experiment.public SummaryType getSummary()
protected void setSummary(SummaryType summary)
summary
- The summary of the experiment.