Class Tokenizer


  • public class Tokenizer
    extends java.lang.Object
    Splits a String into tokens using a specified set of String delimiters. Delimeters are searched for in the that they appear in the delimiter list. If one delimiter is part of another delimiter, make sure you add them in the proper order for correct behavoir
    Version:
    $Revision: 3633 $ $Date: 2003-03-01 08:38:44 +0100 (Sat, 01 Mar 2003) $
    Author:
    Keith Visco
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String[] DEFAULT_DELIMITERS
      The set of delimiters to use, if none are specified
    • Constructor Summary

      Constructors 
      Constructor Description
      Tokenizer​(java.lang.String pattern)
      Creates a new Tokenizer using the given String and the default set of delimiters.
      Tokenizer​(java.lang.String pattern, java.lang.String[] delimiters)
      Creates a new Tokenizer using the given String and the default set of delimiters
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void advance​(int offset)
      Advances the index of this tokenizer ahead by the given offset
      int countTokens()
      Counts the number of times nextToken can be called without returning null
      int getPosition()  
      boolean hasMoreTokens()
      Determines if there are any tokens available
      boolean hasToken​(java.lang.String token)
      Determines if the specified token is contained in the token list of this Tokenizer
      boolean isDelimiter​(java.lang.String token)
      Determines if the specified token is a delimiter for this Tokenizer
      java.lang.String lookAhead​(int offset)
      Allows looking ahead for tokens without affecting the token sequence as called by nextToken or previousToken
      java.lang.String nextToken()
      Retrieves the next available token
      java.lang.String previousToken()
      Retrieves the previous token
      void resetPosition()
      Resets the position of the token pointer to the beginning
      void setPosition​(int position)
      Sets the current position of this tokenizer.
      java.lang.String toString()  
      • Methods inherited from class java.lang.Object

        equals, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Field Detail

      • DEFAULT_DELIMITERS

        public static final java.lang.String[] DEFAULT_DELIMITERS
        The set of delimiters to use, if none are specified
    • Constructor Detail

      • Tokenizer

        public Tokenizer​(java.lang.String pattern)
        Creates a new Tokenizer using the given String and the default set of delimiters. The default delimiters are: " ", "\n", "\r", and "\t";
      • Tokenizer

        public Tokenizer​(java.lang.String pattern,
                         java.lang.String[] delimiters)
        Creates a new Tokenizer using the given String and the default set of delimiters
    • Method Detail

      • advance

        public void advance​(int offset)
        Advances the index of this tokenizer ahead by the given offset
      • countTokens

        public int countTokens()
        Counts the number of times nextToken can be called without returning null
      • getPosition

        public int getPosition()
      • hasMoreTokens

        public boolean hasMoreTokens()
        Determines if there are any tokens available
        Returns:
        true if there are tokens available, otherwise false
      • hasToken

        public boolean hasToken​(java.lang.String token)
        Determines if the specified token is contained in the token list of this Tokenizer
        Parameters:
        token - the String to look for in the token list
        Returns:
        true if the String argument is contained in this Tokenizer's token list, otherwise false
      • isDelimiter

        public boolean isDelimiter​(java.lang.String token)
        Determines if the specified token is a delimiter for this Tokenizer
        Parameters:
        token - the String to compare to the delimiters
        Returns:
        true if the String argument is a delimiter
      • lookAhead

        public java.lang.String lookAhead​(int offset)
        Allows looking ahead for tokens without affecting the token sequence as called by nextToken or previousToken
      • nextToken

        public java.lang.String nextToken()
        Retrieves the next available token
        Returns:
        the next available token or null if there are none
      • previousToken

        public java.lang.String previousToken()
        Retrieves the previous token
        Returns:
        the previous token or null if a previous token is not available
      • resetPosition

        public void resetPosition()
        Resets the position of the token pointer to the beginning
      • setPosition

        public void setPosition​(int position)
        Sets the current position of this tokenizer.
        Parameters:
        position - the index value to set the current position to. if position is greater than the number of tokens, the tokenizer is advanced to the end. If the given position is less than 0, the tokenizer is positioned at the beginning
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object