org.apache.sysds.runtime.transform.tokenize

## Class TokenizerPreWhitespaceSplit

- java.lang.Object
- - org.apache.sysds.runtime.transform.tokenize.TokenizerPreWhitespaceSplit

All Implemented Interfaces:
  Serializable, TokenizerPre

    public class TokenizerPreWhitespaceSplit
    extends Object
    implements TokenizerPre

See Also:
  Serialized Form

### Constructor Summary

Constructors
Constructor and Description
`TokenizerPreWhitespaceSplit(List<Integer> idCols, int tokenizeCol, org.apache.wink.json4j.JSONObject params)`

### Method Summary

All Methods Instance Methods Concrete Methods
`List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token>` `splitToTokens(String text)`
`List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens>` `tokenizePre(FrameBlock in)`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### TokenizerPreWhitespaceSplit

    public TokenizerPreWhitespaceSplit(List<Integer> idCols,
                                       int tokenizeCol,
                                       org.apache.wink.json4j.JSONObject params)
                                throws org.apache.wink.json4j.JSONException

Throws:
  `org.apache.wink.json4j.JSONException`

### Method Detail

#### splitToTokens

    public List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> splitToTokens(String text)

#### tokenizePre

    public List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tokenizePre(FrameBlock in)

Specified by:
  `tokenizePre` in interface `TokenizerPre`