Package org.apache.sysds.runtime.transform.tokenize.builder

## Class TokenizerBuilderWhitespaceSplit

- java.lang.Object
- - org.apache.sysds.runtime.transform.tokenize.builder.TokenizerBuilder
  - - org.apache.sysds.runtime.transform.tokenize.builder.TokenizerBuilderWhitespaceSplit

All Implemented Interfaces:
  `Serializable`

Direct Known Subclasses:
  `TokenizerBuilderNgram`

    public class TokenizerBuilderWhitespaceSplit
    extends TokenizerBuilder

See Also:
  Serialized Form

### Field Summary

Fields
`String` `regex`

### Constructor Summary

Constructors
`TokenizerBuilderWhitespaceSplit​(int[] idCols, int tokenizeCol, org.apache.wink.json4j.JSONObject params)`

### Method Summary

All Methods Instance Methods Concrete Methods
`void` `createInternalRepresentation​(FrameBlock in, DocumentRepresentation[] internalRepresentation, int rowStart, int blk)`
`List<Token>` `splitToTokens​(String text)`

### Methods inherited from class org.apache.sysds.runtime.transform.tokenize.builder.TokenizerBuilder

`createInternalRepresentation, getTasks`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### regex

    public String regex

### Constructor Detail

#### TokenizerBuilderWhitespaceSplit

    public TokenizerBuilderWhitespaceSplit​(int[] idCols,
                                           int tokenizeCol,
                                           org.apache.wink.json4j.JSONObject params)
                                    throws org.apache.wink.json4j.JSONException

Throws:
  `org.apache.wink.json4j.JSONException`

### Method Detail

#### splitToTokens

    public List<Token> splitToTokens​(String text)

#### createInternalRepresentation

    public void createInternalRepresentation​(FrameBlock in,
                                             DocumentRepresentation[] internalRepresentation,
                                             int rowStart,
                                             int blk)

Specified by:
  `createInternalRepresentation` in class `TokenizerBuilder`