psiprobe.tokenizer

Class Tokenizer

java.lang.Object

psiprobe.tokenizer.Tokenizer

Direct Known Subclasses:

StringTokenizer

public class Tokenizer
extends Object

The Class Tokenizer.

Field Summary

Fields

Modifier and Type	Field and Description
static int	TT_BLOCK The Constant TT_BLOCK.
static int	TT_ERROR The Constant TT_ERROR.
static int	TT_SYMBOL The Constant TT_SYMBOL.
static int	TT_TOKEN The Constant TT_TOKEN.

Constructor Summary

Constructors

Constructor and Description
Tokenizer() Instantiates a new tokenizer.
Tokenizer(Reader reader) Instantiates a new tokenizer.
Tokenizer(Reader reader, int cacheBufferSize) Instantiates a new tokenizer.

Method Summary

Modifier and Type	Method and Description
void	addSymbol(String text) Adds the symbol.
void	addSymbol(String text, boolean hidden) Adds the symbol.
void	addSymbol(String startText, String endText, boolean hidden) Adds the symbol.
void	addSymbol(TokenizerSymbol symbol) Adds the symbol.
boolean	getNextBoolean(String trueValue, boolean defaultValue) Gets the next boolean.
long	getNextLong(long defaultValue) Gets the next long.
String	getNextString(String defaultValue) Gets the next string.
Token	getToken() Gets the token.
boolean	hasMore() Checks for more.
Token	nextToken() Next token.
void	pushBack() Push back.
void	setReader(Reader reader) Sets the reader.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TT_TOKEN

public static final int TT_TOKEN

The Constant TT_TOKEN.

See Also:

Constant Field Values

TT_SYMBOL

public static final int TT_SYMBOL

The Constant TT_SYMBOL.

See Also:

Constant Field Values

TT_BLOCK

public static final int TT_BLOCK

The Constant TT_BLOCK.

See Also:

Constant Field Values

TT_ERROR

public static final int TT_ERROR

The Constant TT_ERROR.

See Also:

Constant Field Values

Constructor Detail

Tokenizer

public Tokenizer()

Instantiates a new tokenizer.

Tokenizer

public Tokenizer(Reader reader)

Instantiates a new tokenizer.

Parameters:

reader - the reader

Tokenizer

public Tokenizer(Reader reader,
                 int cacheBufferSize)

Instantiates a new tokenizer.

Parameters:

reader - the reader

cacheBufferSize - the cache buffer size

Method Detail

getToken

public Token getToken()
               throws IOException

Gets the token.

Returns:

the token

Throws:

IOException - Signals that an I/O exception has occurred.

nextToken

public Token nextToken()
                throws IOException

Next token.

Returns:

the token

Throws:

IOException - Signals that an I/O exception has occurred.

pushBack

public void pushBack()

Push back.

setReader

public void setReader(Reader reader)

Sets the reader.

Parameters:

reader - the new reader

hasMore

public boolean hasMore()
                throws IOException

Checks for more.

Returns:

true, if successful

Throws:

IOException - Signals that an I/O exception has occurred.

addSymbol

public void addSymbol(String text)

Adds the symbol.

Parameters:

text - the text

addSymbol

public void addSymbol(String text,
                      boolean hidden)

Adds the symbol.

Parameters:

text - the text

hidden - the hidden

addSymbol

public void addSymbol(String startText,
                      String endText,
                      boolean hidden)

Adds the symbol.

Parameters:

startText - the start text

endText - the end text

hidden - the hidden

addSymbol

public void addSymbol(TokenizerSymbol symbol)

Adds the symbol.

Parameters:

symbol - the symbol

getNextString

public String getNextString(String defaultValue)
                     throws IOException

Gets the next string.

Parameters:

defaultValue - the default value

Returns:

the next string

Throws:

IOException - Signals that an I/O exception has occurred.

getNextBoolean

public boolean getNextBoolean(String trueValue,
                              boolean defaultValue)
                       throws IOException

Gets the next boolean.

Parameters:

trueValue - the true value

defaultValue - the default value

Returns:

the next boolean

Throws:

IOException - Signals that an I/O exception has occurred.

getNextLong

public long getNextLong(long defaultValue)
                 throws IOException

Gets the next long.

Parameters:

defaultValue - the default value

Returns:

the next long

Throws:

IOException - Signals that an I/O exception has occurred.