Uses of Package
gov.sandia.cognition.text.token

Packages that use gov.sandia.cognition.text.token

Package	Description
gov.sandia.cognition.text.term	Provides term representing text content in documents.
gov.sandia.cognition.text.token	Provides text tokenization algorithms.

Classes in gov.sandia.cognition.text.token used by gov.sandia.cognition.text.term

Class and Description
Token Interface for a meaningful chunk of text, called a token.

Classes in gov.sandia.cognition.text.token used by gov.sandia.cognition.text.token

Class and Description
AbstractCharacterBasedTokenizer An abstract implementation of a tokenizer that considers each character individually.
AbstractTokenizer Abstract implementation of the Tokenizer interface.
Token Interface for a meaningful chunk of text, called a token.
Tokenizer Interface for a class that converts strings into tokens.