org.apache.sysds.api.jmlc

## Class Connection

- java.lang.Object
- - org.apache.sysds.api.jmlc.Connection

All Implemented Interfaces:
  Closeable, AutoCloseable

    public class Connection
    extends Object
    implements Closeable

Interaction with SystemDS using the JMLC (Java Machine Learning Connector) API is initiated with a `Connection` object. The JMLC API is designed after JDBC. A DML script is precompiled by calling the `prepareScript(String, String[], String[])` method or the `prepareScript(String, Map, String[], String[])` method on the `Connection` object, which returns a `PreparedScript` object. Note that this is similar to calling a `prepareStatement` method on a JDBC `Connection` object.

Following this, input variable data is passed to the script by calling the `setFrame`, `setMatrix`, and `setScalar` methods of the `PreparedScript` object. The script is executed via `PreparedScript`'s `executeScript` method, which returns a `ResultVariables` object, which is similar to a JDBC `ResultSet`. Data can be read from a `ResultVariables` object by calling its `getFrame` and `getMatrix` methods.

### Constructor Summary

Constructors
Constructor and Description
`Connection()` Connection constructor, the starting point for any other JMLC API calls.
`Connection(CompilerConfig.ConfigType... cconfigs)` Connection constructor, the starting point for any other JMLC API calls.
`Connection(DMLConfig dmlconfig)` Connection constructor, the starting point for any other JMLC API calls.
`Connection(DMLConfig dmlconfig, CompilerConfig.ConfigType... cconfigs)` Connection constructor, the starting point for any other JMLC API calls.

### Method Summary

All Methods Instance Methods Concrete Methods
`void` `close()` Close connection to SystemDS, which clears the thread-local DML and compiler configurations.
`double[][]` `convertToDoubleMatrix(InputStream input, int rows, int cols)` Converts an input stream of a string matrix in textcell format into a dense double array.
`double[][]` `convertToDoubleMatrix(InputStream input, int rows, int cols, String format)` Converts an input stream of a string matrix in csv or textcell format into a dense double array.
`double[][]` `convertToDoubleMatrix(String input, int rows, int cols)` Converts an input string representation of a matrix in textcell format into a dense double array.
`double[][]` `convertToDoubleMatrix(String input, String meta)` Converts an input string representation of a matrix in csv or textcell format into a dense double array.
`FrameBlock` `convertToFrame(InputStream input, int rows, int cols)` Converts an input stream of a string frame in textcell format into a frame block.
`FrameBlock` `convertToFrame(InputStream input, int rows, int cols, String format)` Converts an input stream of a frame in csv or textcell format into a frame block.
`FrameBlock` `convertToFrame(InputStream input, String meta)` Converts an input stream of a string frame in csv or textcell format into a frame block.
`FrameBlock` `convertToFrame(String input, int rows, int cols)` Converts an input string representation of a frame in textcell format into a frame block.
`FrameBlock` `convertToFrame(String input, String meta)` Converts an input string representation of a frame in csv or textcell format into a frame block.
`MatrixBlock` `convertToMatrix(InputStream input, int rows, int cols)` Converts an input stream of a string matrix in text format into a matrix block.
`MatrixBlock` `convertToMatrix(InputStream input, int rows, int cols, String format)` Converts an input stream of a string matrix in csv or text format into a matrix block.
`MatrixBlock` `convertToMatrix(InputStream input, String meta)` Converts an input stream of a string matrix in csv or textcell format into a matrix block.
`MatrixBlock` `convertToMatrix(String input, int rows, int cols)` Converts an input string representation of a matrix in textcell format into a matrix block.
`MatrixBlock` `convertToMatrix(String input, String meta)` Converts an input string representation of a matrix in csv or textcell format into a matrix block.
`String[][]` `convertToStringFrame(InputStream input, int rows, int cols)` Converts an input stream of a string frame in textcell format into a dense string array.
`String[][]` `convertToStringFrame(InputStream input, int rows, int cols, String format)` Converts an input stream of a string frame in csv or text format into a dense string array.
`String[][]` `convertToStringFrame(String input, int rows, int cols)` Converts an input stream of a string frame in textcell format into a dense string array.
`String[][]` `convertToStringFrame(String input, String meta)` Converts an input string representation of a frame in csv or textcell format into a dense string array.
`void` `gatherMemStats(boolean stats)` Sets a boolean flag indicating if memory profiling statistics should be gathered.
`PreparedScript` `prepareScript(String script, Map<String,String> nsscripts, Map<String,String> args, String[] inputs, String[] outputs)` Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
`PreparedScript` `prepareScript(String script, Map<String,String> args, String[] inputs, String[] outputs)` Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
`PreparedScript` `prepareScript(String script, String[] inputs, String[] outputs)` Prepares (precompiles) a script and registers input and output variables.
`double[][]` `readDoubleMatrix(String fname)` Reads an input matrix in arbitrary format from HDFS into a dense double array.
`double[][]` `readDoubleMatrix(String fname, Types.FileFormat fmt, long rows, long cols, int blen, long nnz)` Reads an input matrix in arbitrary format from HDFS into a dense double array.
`String` `readScript(String fname)` Read a DML or PyDML file as a string.
`String[][]` `readStringFrame(String fname)` Reads an input frame in arbitrary format from HDFS into a dense string array.
`String[][]` `readStringFrame(String fname, Types.FileFormat fmt, long rows, long cols)` Reads an input frame in arbitrary format from HDFS into a dense string array.
`FrameBlock` `readTransformMetaDataFromFile(String metapath)` Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
`FrameBlock` `readTransformMetaDataFromFile(String spec, String metapath)` Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
`FrameBlock` `readTransformMetaDataFromFile(String spec, String metapath, String colDelim)` Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
`FrameBlock` `readTransformMetaDataFromPath(String metapath)` Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
`FrameBlock` `readTransformMetaDataFromPath(String spec, String metapath)` Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
`FrameBlock` `readTransformMetaDataFromPath(String spec, String metapath, String colDelim)` Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
`void` `setLineage(boolean lt)` Sets a boolean flag indicating if lineage trace should be captured
`void` `setStatistics(boolean stats)` Sets a boolean flag indicating if runtime statistics should be gathered Same behavior as in "MLContext.setStatistics()"

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### Connection

    public Connection()

Connection constructor, the starting point for any other JMLC API calls.

#### Connection

    public Connection(CompilerConfig.ConfigType... cconfigs)

Connection constructor, the starting point for any other JMLC API calls. This variant allows to enable a set of boolean compiler configurations.

Parameters:
  `cconfigs` - one or many boolean compiler configurations to enable.

#### Connection

    public Connection(DMLConfig dmlconfig,
                      CompilerConfig.ConfigType... cconfigs)

Connection constructor, the starting point for any other JMLC API calls. This variant allows to pass a global dml configuration and enable a set of boolean compiler configurations.

Parameters:
  `dmlconfig` - a dml configuration.
  `cconfigs` - one or many boolean compiler configurations to enable.

#### Connection

    public Connection(DMLConfig dmlconfig)

Connection constructor, the starting point for any other JMLC API calls. This variant allows to pass a global dml configuration.

Parameters:
  `dmlconfig` - a dml configuration.

### Method Detail

#### setStatistics

    public void setStatistics(boolean stats)

Sets a boolean flag indicating if runtime statistics should be gathered Same behavior as in "MLContext.setStatistics()"

Parameters:
  `stats` - boolean value with true indicating statistics should be gathered

#### setLineage

    public void setLineage(boolean lt)

Sets a boolean flag indicating if lineage trace should be captured

Parameters:
  `lt` - boolean value with true indicating lineage should be captured

#### gatherMemStats

    public void gatherMemStats(boolean stats)

Sets a boolean flag indicating if memory profiling statistics should be gathered. The option is false by default.

Parameters:
  `stats` - boolean value with true indicating memory statistics should be gathered

#### prepareScript

    public PreparedScript prepareScript(String script,
                                        String[] inputs,
                                        String[] outputs)

Prepares (precompiles) a script and registers input and output variables.

Parameters:
  `script` - string representing the DML or PyDML script
  `inputs` - string array of input variables to register
  `outputs` - string array of output variables to register
Returns:
  PreparedScript object representing the precompiled script

#### prepareScript

    public PreparedScript prepareScript(String script,
                                        Map<String,String> args,
                                        String[] inputs,
                                        String[] outputs)

Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.

Parameters:
  `script` - string representing the DML or PyDML script
  `args` - map of input parameters ($) and their values
  `inputs` - string array of input variables to register
  `outputs` - string array of output variables to register
Returns:
  PreparedScript object representing the precompiled script

#### prepareScript

    public PreparedScript prepareScript(String script,
                                        Map<String,String> nsscripts,
                                        Map<String,String> args,
                                        String[] inputs,
                                        String[] outputs)

Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.

Parameters:
  `script` - string representing of the DML or PyDML script
  `nsscripts` - map (name, script) of the DML or PyDML namespace scripts
  `args` - map of input parameters ($) and their values
  `inputs` - string array of input variables to register
  `outputs` - string array of output variables to register
Returns:
  PreparedScript object representing the precompiled script

#### close

    public void close()

Close connection to SystemDS, which clears the thread-local DML and compiler configurations.

Specified by:
  `close` in interface `Closeable`
Specified by:
  `close` in interface `AutoCloseable`

#### readScript

    public String readScript(String fname)
                      throws IOException

Read a DML or PyDML file as a string.

Parameters:
  `fname` - the filename of the script
Returns:
  string content of the script file
Throws:
  `IOException` - if IOException occurs

#### readDoubleMatrix

    public double[][] readDoubleMatrix(String fname)
                                throws IOException

Reads an input matrix in arbitrary format from HDFS into a dense double array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input matrix
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

#### readDoubleMatrix

    public double[][] readDoubleMatrix(String fname,
                                       Types.FileFormat fmt,
                                       long rows,
                                       long cols,
                                       int blen,
                                       long nnz)
                                throws IOException

Reads an input matrix in arbitrary format from HDFS into a dense double array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input matrix
  `fmt` - file format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
  `blen` - block length
  `nnz` - number of non-zero values, -1 indicates unknown
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

#### convertToDoubleMatrix

    public double[][] convertToDoubleMatrix(String input,
                                            String meta)
                                     throws IOException

Converts an input string representation of a matrix in csv or textcell format into a dense double array. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string matrix in csv or textcell format
  `meta` - string representing SystemDS matrix metadata in JSON format
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

#### convertToDoubleMatrix

    public double[][] convertToDoubleMatrix(String input,
                                            int rows,
                                            int cols)
                                     throws IOException

Converts an input string representation of a matrix in textcell format into a dense double array.

Parameters:
  `input` - string matrix in textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

#### convertToDoubleMatrix

    public double[][] convertToDoubleMatrix(InputStream input,
                                            int rows,
                                            int cols)
                                     throws IOException

Converts an input stream of a string matrix in textcell format into a dense double array.

Parameters:
  `input` - InputStream to a string matrix in textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

#### convertToDoubleMatrix

    public double[][] convertToDoubleMatrix(InputStream input,
                                            int rows,
                                            int cols,
                                            String format)
                                     throws IOException

Converts an input stream of a string matrix in csv or textcell format into a dense double array.

Parameters:
  `input` - InputStream to a string matrix in csv or textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
  `format` - input format of the given stream
Returns:
  matrix as a two-dimensional double array
Throws:
  `IOException` - if IOException occurs

#### convertToMatrix

    public MatrixBlock convertToMatrix(String input,
                                       String meta)
                                throws IOException

Converts an input string representation of a matrix in csv or textcell format into a matrix block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string matrix in csv or textcell format
  `meta` - string representing SystemDS matrix metadata in JSON format
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

#### convertToMatrix

    public MatrixBlock convertToMatrix(InputStream input,
                                       String meta)
                                throws IOException

Converts an input stream of a string matrix in csv or textcell format into a matrix block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - InputStream to a string matrix in csv or textcell format
  `meta` - string representing SystemDS matrix metadata in JSON format
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

#### convertToMatrix

    public MatrixBlock convertToMatrix(String input,
                                       int rows,
                                       int cols)
                                throws IOException

Converts an input string representation of a matrix in textcell format into a matrix block.

Parameters:
  `input` - string matrix in textcell format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

#### convertToMatrix

    public MatrixBlock convertToMatrix(InputStream input,
                                       int rows,
                                       int cols)
                                throws IOException

Converts an input stream of a string matrix in text format into a matrix block.

Parameters:
  `input` - InputStream to a string matrix in text format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

#### convertToMatrix

    public MatrixBlock convertToMatrix(InputStream input,
                                       int rows,
                                       int cols,
                                       String format)
                                throws IOException

Converts an input stream of a string matrix in csv or text format into a matrix block.

Parameters:
  `input` - InputStream to a string matrix in csv or text format
  `rows` - number of rows in the matrix
  `cols` - number of columns in the matrix
  `format` - input format of the given stream
Returns:
  matrix as a matrix block
Throws:
  `IOException` - if IOException occurs

#### readStringFrame

    public String[][] readStringFrame(String fname)
                               throws IOException

Reads an input frame in arbitrary format from HDFS into a dense string array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

#### readStringFrame

    public String[][] readStringFrame(String fname,
                                      Types.FileFormat fmt,
                                      long rows,
                                      long cols)
                               throws IOException

Reads an input frame in arbitrary format from HDFS into a dense string array. NOTE: this call currently only supports default configurations for CSV.

Parameters:
  `fname` - the filename of the input frame
  `fmt` - file format type
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

#### convertToStringFrame

    public String[][] convertToStringFrame(String input,
                                           String meta)
                                    throws IOException

Converts an input string representation of a frame in csv or textcell format into a dense string array. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string frame in csv or textcell format
  `meta` - string representing SystemDS frame metadata in JSON format
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

#### convertToStringFrame

    public String[][] convertToStringFrame(String input,
                                           int rows,
                                           int cols)
                                    throws IOException

Converts an input stream of a string frame in textcell format into a dense string array.

Parameters:
  `input` - string frame in textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

#### convertToStringFrame

    public String[][] convertToStringFrame(InputStream input,
                                           int rows,
                                           int cols)
                                    throws IOException

Converts an input stream of a string frame in textcell format into a dense string array.

Parameters:
  `input` - InputStream to a string frame in text format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

#### convertToStringFrame

    public String[][] convertToStringFrame(InputStream input,
                                           int rows,
                                           int cols,
                                           String format)
                                    throws IOException

Converts an input stream of a string frame in csv or text format into a dense string array.

Parameters:
  `input` - InputStream to a string frame in csv or text format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
  `format` - input format of the given stream
Returns:
  frame as a two-dimensional string array
Throws:
  `IOException` - if IOException occurs

#### convertToFrame

    public FrameBlock convertToFrame(String input,
                                     String meta)
                              throws IOException

Converts an input string representation of a frame in csv or textcell format into a frame block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - string frame in csv or textcell format
  `meta` - string representing SystemDS frame metadata in JSON format
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

#### convertToFrame

    public FrameBlock convertToFrame(InputStream input,
                                     String meta)
                              throws IOException

Converts an input stream of a string frame in csv or textcell format into a frame block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.

Parameters:
  `input` - InputStream to a string frame in csv or textcell format
  `meta` - string representing SystemDS frame metadata in JSON format
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

#### convertToFrame

    public FrameBlock convertToFrame(String input,
                                     int rows,
                                     int cols)
                              throws IOException

Converts an input string representation of a frame in textcell format into a frame block.

Parameters:
  `input` - string frame in textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

#### convertToFrame

    public FrameBlock convertToFrame(InputStream input,
                                     int rows,
                                     int cols)
                              throws IOException

Converts an input stream of a string frame in textcell format into a frame block.

Parameters:
  `input` - InputStream to a string frame in textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

#### convertToFrame

    public FrameBlock convertToFrame(InputStream input,
                                     int rows,
                                     int cols,
                                     String format)
                              throws IOException

Converts an input stream of a frame in csv or textcell format into a frame block.

Parameters:
  `input` - InputStream to a string frame in csv or textcell format
  `rows` - number of rows in the frame
  `cols` - number of columns in the frame
  `format` - input format of the given stream
Returns:
  frame as a frame block
Throws:
  `IOException` - if IOException occurs

#### readTransformMetaDataFromFile

    public FrameBlock readTransformMetaDataFromFile(String metapath)
                                             throws IOException

Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.

Parameters:
  `metapath` - hdfs file path to meta data directory
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

#### readTransformMetaDataFromFile

    public FrameBlock readTransformMetaDataFromFile(String spec,
                                                    String metapath)
                                             throws IOException

Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.

Parameters:
  `spec` - transform specification as json string
  `metapath` - hdfs file path to meta data directory
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

#### readTransformMetaDataFromFile

    public FrameBlock readTransformMetaDataFromFile(String spec,
                                                    String metapath,
                                                    String colDelim)
                                             throws IOException

Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.

Parameters:
  `spec` - transform specification as json string
  `metapath` - hdfs file path to meta data directory
  `colDelim` - separator for processing column names in the meta data file 'column.names'
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

#### readTransformMetaDataFromPath

    public FrameBlock readTransformMetaDataFromPath(String metapath)
                                             throws IOException

Reads transform meta data from the class path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.

Parameters:
  `metapath` - resource path to meta data directory
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

#### readTransformMetaDataFromPath

    public FrameBlock readTransformMetaDataFromPath(String spec,
                                                    String metapath)
                                             throws IOException

Reads transform meta data from the class path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.

Parameters:
  `spec` - transform specification as json string
  `metapath` - resource path to meta data directory
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs

#### readTransformMetaDataFromPath

    public FrameBlock readTransformMetaDataFromPath(String spec,
                                                    String metapath,
                                                    String colDelim)
                                             throws IOException

Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.

Parameters:
  `spec` - transform specification as json string
  `metapath` - resource path to meta data directory
  `colDelim` - separator for processing column names in the meta data file 'column.names'
Returns:
  FrameBlock object representing transform metadata
Throws:
  `IOException` - if IOException occurs