public class LetterNumberTokenizer extends AbstractCharacterBasedTokenizer
Constructor and Description |
---|
LetterNumberTokenizer()
Creates a new
LetterNumberTokenizer . |
Modifier and Type | Method and Description |
---|---|
boolean |
isTokenMember(char c)
Determines if the given character is considered to be part of a token.
|
tokenize
tokenize, tokenize
clone
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
clone
public LetterNumberTokenizer()
LetterNumberTokenizer
.public boolean isTokenMember(char c)
AbstractCharacterBasedTokenizer
isTokenMember
in class AbstractCharacterBasedTokenizer
c
- A character.