org.apache.sysds.runtime.transform.tokenize

## Interface TokenizerPre

All Superinterfaces:
  Serializable

All Known Implementing Classes:
  TokenizerPreNgram, TokenizerPreWhitespaceSplit

    public interface TokenizerPre
    extends Serializable

### Method Summary

All Methods Instance Methods Abstract Methods
`List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens>` `tokenizePre(FrameBlock in)`

### Method Detail

#### tokenizePre

    List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tokenizePre(FrameBlock in)