com.voytechs.jnetstream.npl
Class TokenList

java.lang.Object
  com.voytechs.jnetstream.npl.TokenList

Direct Known Subclasses:

ExpTokenizer

public abstract class TokenList
extends java.lang.Object

Field Summary

protected java.io.StreamTokenizer

in

Constructor Summary

TokenList(java.io.StreamTokenizer in)

Method Summary

int	consumeTokens(int n) Consumes N number of tokens from the stream.
protected Token	fetchRawTokenFromStream() Fetches the RAW token.
protected Token	fetchTokenFromStream() Fetches the next token from the raw stream.
abstract java.lang.String	getFilename() Returns the filename of the stream if known.
int	getLineNumber() Returns the line number of the curren token.
boolean	hasMoreTokens() Tells if there are more tokens in the stream.
void	insert(Token token) Inserts the given token into the stream buffer.
Token	lookAhead() Looks ahead 1 token, but does not advance the token sequence in the stream.
Token	lookAhead(int nth) Gets the Nth token in the stream from the current token position.
Token	lookAheadNoTranslation(int nth) Gets the Nth token in the stream from the current token position.
static void	main(java.lang.String[] args) Checks to see if there are any more tokens in the stream.
Token	nextToken() Returns the next token.
Token	nextTokenNoTranslation() Fetches the next token either from stream or cache with no translation.
Token	previousToken()
void	pushBack() Push back upto 2 to previous tokens.
abstract Token	translateNextToken() Translate the next token on the tokenlist into a new token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

in

protected java.io.StreamTokenizer in

Constructor Detail

TokenList

public TokenList(java.io.StreamTokenizer in)

Parameters:

-

Throws:

Method Detail

previousToken

public Token previousToken()

pushBack

public void pushBack()

Push back upto 2 to previous tokens. If 2 nextToken() calls were made 2 pushBack() calls will put all of the tokens back on the list to be returned again with the nextToken() calls.

insert

public void insert(Token token)

Inserts the given token into the stream buffer.

nextToken

public Token nextToken()

Returns the next token.

Returns:

Next token from the stream.

lookAhead

public Token lookAhead()

Looks ahead 1 token, but does not advance the token sequence in the stream.

Returns:

peeked token. 1 token ahead.

lookAhead

public Token lookAhead(int nth)

Gets the Nth token in the stream from the current token position. N number of tokens will read and remembered from the stream. The remember tokens will be returned normally in correct sequece when the nextToken() method is called. If there are not enough tokens in the stream, appropriate exception is thrown.

Returns:

Nth token from the stream. null if not enough tokens.

translateNextToken

public abstract Token translateNextToken()

Translate the next token on the tokenlist into a new token.

consumeTokens

public int consumeTokens(int n)

Consumes N number of tokens from the stream.

hasMoreTokens

public boolean hasMoreTokens()

Tells if there are more tokens in the stream.

Returns:

true if there is another token to return, false if none left.

nextTokenNoTranslation

public Token nextTokenNoTranslation()

Fetches the next token either from stream or cache with no translation. Tokens are returned as is with no translation.

lookAheadNoTranslation

public Token lookAheadNoTranslation(int nth)

Gets the Nth token in the stream from the current token position. N number of tokens will read and remembered from the stream. The remember tokens will be returned normally in correct sequece when the nextToken() method is called. If there are not enough tokens in the stream, appropriate exception is thrown. This method does no translation on the token.

Returns:

Nth token from the stream. null if not enough tokens.

fetchTokenFromStream

protected Token fetchTokenFromStream()

Fetches the next token from the raw stream. This function also does custom TokenType translation on the stream of tokens. Subclasses are not required to call super.fetchTokenFromStream().

Returns:

Returns the translated token.

fetchRawTokenFromStream

protected Token fetchRawTokenFromStream()

Fetches the RAW token. No custom token type translations are done in this method.

getFilename

public abstract java.lang.String getFilename()

Returns the filename of the stream if known.

getLineNumber

public int getLineNumber()

Returns the line number of the curren token.

main

public static void main(java.lang.String[] args)

Checks to see if there are any more tokens in the stream. /** Test function for TokenList

Parameters:

args - command line arguments