gov.sandia.cognition.learning.performance

Class AnytimeBatchLearnerValidationPerformanceReporter<DataType,ObjectType>

java.lang.Object

gov.sandia.cognition.util.AbstractCloneableSerializable

gov.sandia.cognition.algorithm.event.AbstractIterativeAlgorithmListener

gov.sandia.cognition.learning.performance.AnytimeBatchLearnerValidationPerformanceReporter<DataType,ObjectType>

Type Parameters:

DataType - The type of data this reports validation performance for.

ObjectType - The type of object that is returned by the anytime learner that performance can be computed for.

All Implemented Interfaces:

IterativeAlgorithmListener, CloneableSerializable, java.io.Serializable, java.lang.Cloneable

public class AnytimeBatchLearnerValidationPerformanceReporter<DataType,ObjectType>
extends AbstractIterativeAlgorithmListener

A performance reporter for a validation set. Can be attached to an anytime algorithm to report results after each step. It reports the result of evaluating performance on both the training and validation sets.

Since:

3.4.0

Author:

Justin Basilico

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	DEFAULT_FORMAT The default format for reporting performance is "Iteration %d. Train: %s. Validation: %s".
protected java.lang.String	format The format for the performance report, passed to String.format.
protected java.io.PrintStream	out The print stream to report performance to.
protected PerformanceEvaluator<? super ObjectType,? super DataType,?>	performanceEvaluator The performance evaluator.
protected DataType	validationData The validation dataset to use.

Constructor Summary

Constructors

Constructor and Description
AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator, DataType validationData) Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to the given print stream using the default format.
AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator, DataType validationData, java.io.PrintStream out) Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to the given print stream using the default format.
AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator, DataType validationData, java.io.PrintStream out, java.lang.String format) Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to the given print stream and format.
AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator, DataType validationData, java.lang.String format) Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to System.out and the given format.

Method Summary

Modifier and Type	Method and Description
PerformanceEvaluator<? super ObjectType,? super DataType,?>	getPerformanceEvaluator() Gets the performance evaluator.
DataType	getValidationData() Gets the validation dataset.
void	setPerformanceEvaluator(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator) Sets the performance evaluator.
void	setValidationData(DataType validationData) Sets the validation dataset.
void	stepEnded(IterativeAlgorithm algorithm) This method is called when the algorithm has ended a step of its execution.

Methods inherited from class gov.sandia.cognition.algorithm.event.AbstractIterativeAlgorithmListener

algorithmEnded, algorithmStarted, stepStarted

Methods inherited from class gov.sandia.cognition.util.AbstractCloneableSerializable

clone

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_FORMAT

public static final java.lang.String DEFAULT_FORMAT

The default format for reporting performance is "Iteration %d. Train: %s. Validation: %s".

See Also:

Constant Field Values

performanceEvaluator

protected PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator

The performance evaluator. Produces the metrics.

validationData

protected DataType validationData

The validation dataset to use.

out

protected java.io.PrintStream out

The print stream to report performance to.

format

protected java.lang.String format

The format for the performance report, passed to String.format.

Constructor Detail

AnytimeBatchLearnerValidationPerformanceReporter

public AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator,
                                                        DataType validationData)

Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to the given print stream using the default format.

Parameters:

performanceEvaluator - The performance evaluator. Produces the metrics.

validationData - The validation dataset to use.

AnytimeBatchLearnerValidationPerformanceReporter

public AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator,
                                                        DataType validationData,
                                                        java.io.PrintStream out)

Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to the given print stream using the default format.

Parameters:

performanceEvaluator - The performance evaluator. Produces the metrics.

validationData - The validation dataset to use.

out - Print stream to report the status to.

AnytimeBatchLearnerValidationPerformanceReporter

public AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator,
                                                        DataType validationData,
                                                        java.lang.String format)

Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to System.out and the given format.

Parameters:

performanceEvaluator - The performance evaluator. Produces the metrics.

validationData - The validation dataset to use.

format - Format string for the status messages, which passed to String.format.

AnytimeBatchLearnerValidationPerformanceReporter

public AnytimeBatchLearnerValidationPerformanceReporter(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator,
                                                        DataType validationData,
                                                        java.io.PrintStream out,
                                                        java.lang.String format)

Creates a new AnytimeBatchLearnerValidationPerformanceReporter that reports to the given print stream and format.

Parameters:

performanceEvaluator - The performance evaluator. Produces the metrics.

validationData - The validation dataset to use.

out - Print stream to report the status to.

format - Format string for the status messages, which passed to String.format.

Method Detail

stepEnded

public void stepEnded(IterativeAlgorithm algorithm)

Description copied from interface: IterativeAlgorithmListener

This method is called when the algorithm has ended a step of its execution.

Specified by:

stepEnded in interface IterativeAlgorithmListener

Overrides:

stepEnded in class AbstractIterativeAlgorithmListener

Parameters:

algorithm - The algorithm that has ended another step of its execution.

getPerformanceEvaluator

public PerformanceEvaluator<? super ObjectType,? super DataType,?> getPerformanceEvaluator()

Gets the performance evaluator. It produces the metrics.

Returns:

The performance evaluator.

setPerformanceEvaluator

public void setPerformanceEvaluator(PerformanceEvaluator<? super ObjectType,? super DataType,?> performanceEvaluator)

Sets the performance evaluator. It produces the metrics.

Parameters:

performanceEvaluator - The performance evaluator.

getValidationData

public DataType getValidationData()

Gets the validation dataset.

Returns:

The validation dataset.

setValidationData

public void setValidationData(DataType validationData)

Sets the validation dataset.

Parameters:

validationData - The validation dataset.