Package org.apache.sysds.runtime.transform.tokenize.applier

## Class TokenizerApplierPosition

- java.lang.Object
- - org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier
  - - org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplierPosition

All Implemented Interfaces:
  `Serializable`

    public class TokenizerApplierPosition
    extends TokenizerApplier

See Also:
  Serialized Form

### Field Summary

### Fields inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

`PADDING_STRING`

### Constructor Summary

Constructors
`TokenizerApplierPosition​(int numIdCols, int maxTokens, boolean wideFormat, boolean applyPadding)`

### Method Summary

All Methods Instance Methods Concrete Methods
`int` `appendTokensLong​(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)`
`int` `appendTokensWide​(int row, List<Object> keys, List<Token> tokenList, FrameBlock out)`
`int` `applyInternalRepresentation​(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)`
`int` `getNumRows​(DocumentRepresentation[] internalRepresentation)`
`Types.ValueType[]` `getOutSchema()`

### Methods inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

`allocateInternalMeta, applyInternalRepresentation, build, getApplyTasks, getBuildTasks, getMaxNumRows, getMaxTokens, getNumCols, getOutputRow, getOutputRow, hasPadding, isWideFormat`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### TokenizerApplierPosition

    public TokenizerApplierPosition​(int numIdCols,
                                    int maxTokens,
                                    boolean wideFormat,
                                    boolean applyPadding)

### Method Detail

#### getNumRows

    public int getNumRows​(DocumentRepresentation[] internalRepresentation)

Specified by:
  `getNumRows` in class `TokenizerApplier`

#### applyInternalRepresentation

    public int applyInternalRepresentation​(DocumentRepresentation[] internalRepresentation,
                                           FrameBlock out,
                                           int inputRowStart,
                                           int blk)

#### appendTokensLong

    public int appendTokensLong​(int row,
                                List<Object> keys,
                                List<Token> tokenList,
                                FrameBlock out)

#### appendTokensWide

    public int appendTokensWide​(int row,
                                List<Object> keys,
                                List<Token> tokenList,
                                FrameBlock out)

#### getOutSchema

    public Types.ValueType[] getOutSchema()

Specified by:
  `getOutSchema` in class `TokenizerApplier`