org.apache.felix.gogo.runtime

## Class Tokenizer

- java.lang.Object
- - org.apache.felix.gogo.runtime.BaseTokenizer
  - - org.apache.felix.gogo.runtime.Tokenizer

    public class Tokenizer
    extends BaseTokenizer

### Field Summary

Fields
`protected boolean` `inArray`
`protected Token` `last`
`protected Token` `pushed`
`protected int` `word`

### Fields inherited from class org.apache.felix.gogo.runtime.BaseTokenizer

`ch, column, EOT, index, line, text`

### Constructor Summary

Constructors
Constructor and Description
`Tokenizer(CharSequence text)`

### Method Summary

All Methods Instance Methods Concrete Methods
`Token` `next()`
`void` `push(Token token)`
`Token` `readHereDoc(boolean ignoreLeadingTabs)`
`Token` `text()`

### Methods inherited from class org.apache.felix.gogo.runtime.BaseTokenizer

`eot, escape, find, getch, getch, isBlank, peek, skipQuote, skipSpace, skipSpace`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### inArray

    protected boolean inArray

#### word

    protected int word

#### pushed

    protected Token pushed

#### last

    protected Token last

### Constructor Detail

#### Tokenizer

    public Tokenizer(CharSequence text)

### Method Detail

#### text

    public Token text()

Overrides:
  `text` in class `BaseTokenizer`

#### next

    public Token next()

#### push

    public void push(Token token)

#### readHereDoc

    public Token readHereDoc(boolean ignoreLeadingTabs)