Package org.apache.sysds.api.mlcontext

## Class MLContextUtil

- java.lang.Object
- - org.apache.sysds.api.mlcontext.MLContextUtil

    public final class MLContextUtil
    extends Object

Utility class containing methods for working with the MLContext API.

### Field Summary

Fields
`static Class[]` `ALL_SUPPORTED_DATA_TYPES` All data types supported by the MLContext API.
`static Class[]` `BASIC_DATA_TYPES` Basic data types supported by the MLContext API.
`static String` `BUILD_TIME_NOT_AVAILABLE` Build time not available message.
`static Class[]` `COMPLEX_DATA_TYPES` Complex data types supported by the MLContext API.
`static String` `VERSION_NOT_AVAILABLE` Version not available message.

### Constructor Summary

Constructors
`MLContextUtil()`

### Method Summary

All Methods Static Methods Concrete Methods
`static void` `checkInputParameterType​(String parameterName, Object parameterValue)` Verify that the type of input parameter value is supported.
`static void` `checkInputValueType​(String name, Object value)` Verify that the type of input value is supported.
`static void` `checkInputValueTypes​(Map<String,​Object> inputs)` Verify that the types of input values are supported.
`static Map<String,​String>` `convertInputParametersForParser​(Map<String,​Object> basicInputParameterMap)` Converts non-string basic input parameter values to strings to pass to the parser.
`static Data` `convertInputType​(String parameterName, Object parameterValue)` Convert input types to internal SystemDS representations
`static Data` `convertInputType​(String parameterName, Object parameterValue, Metadata metadata)` Convert input types to internal SystemDS representations
`static void` `deleteRemoveVariableInstructions​(Program progam)` Delete the 'remove variable' instructions from a runtime program.
`static String` `determineOutputTypeAsString​(LocalVariableMap symbolTable, String outputName)` Obtain a symbol table output type as a String
`static String` `displayInputs​(String name, Map<String,​Object> map, LocalVariableMap symbolTable)` Obtain a display of script inputs.
`static String` `displayMap​(String mapName, Map<String,​Object> map)` Display the keys and values in a Map
`static String` `displayOutputs​(String name, Set<String> outputNames, LocalVariableMap symbolTable)` Obtain a display of the script outputs.
`static String` `displayOutputs​(Set<String> outputNames, LocalVariableMap symbolTable)` Obtain a display of the script outputs.
`static String` `displaySet​(String setName, Set<String> set)` Display the values in a Set
`static String` `displaySymbolTable​(String name, LocalVariableMap symbolTable)` Display the keys and values in the symbol table
`static String` `displaySymbolTable​(LocalVariableMap symbolTable)` Display the keys and values in the symbol table
`static boolean` `doesDataFrameLookLikeMatrix​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)` Examine the DataFrame schema to determine whether the data appears to be a matrix.
`static boolean` `doesSymbolTableContainFrameObject​(LocalVariableMap symbolTable, String variableName)` Determine if the symbol table contains a FrameObject with the given variable name.
`static boolean` `doesSymbolTableContainMatrixObject​(LocalVariableMap symbolTable, String variableName)` Determine if the symbol table contains a MatrixObject with the given variable name.
`static String` `getBasicTypeString​(Object object)` Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)
`static org.apache.spark.api.java.JavaSparkContext` `getJavaSparkContext​(MLContext mlContext)` Obtain the Java Spark Context
`static org.apache.spark.api.java.JavaSparkContext` `getJavaSparkContextFromProxy()` Obtain the Java Spark Context from the MLContextProxy
`static org.apache.spark.SparkContext` `getSparkContext​(MLContext mlContext)` Obtain the Spark Context
`static org.apache.spark.SparkContext` `getSparkContextFromProxy()` Obtain the Spark Context from the MLContextProxy
`static org.apache.spark.sql.SparkSession` `getSparkSessionFromProxy()` Obtain the Spark Session from the MLContextProxy
`static boolean` `isBasicType​(Object object)` Is the object one of the supported basic data types? (Integer, Boolean, Double, String)
`static boolean` `isComplexType​(Object object)` Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)
`static boolean` `isCSVLineAllNumbers​(String line)` If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)
`static boolean` `isSparkVersionSupported​(String sparkVersion, String minimumRecommendedSparkVersion)` Determine whether the Spark version is supported.
`static String` `quotedString​(String str)` Return a double-quoted string with inner single and double quotes escaped.
`static void` `setCompilerConfig()` Set SystemDS compiler configuration properties for MLContext
`static void` `setConfig​(String configFilePath)` Set SystemDS configuration properties based on a configuration file.
`static void` `setDefaultConfig()` Set default SystemDS configuration properties.
`static void` `verifySparkVersionSupported​(org.apache.spark.sql.SparkSession spark)` Check that the Spark version is supported.
`static String` `welcomeMessage()` The SystemDS welcome message

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### VERSION_NOT_AVAILABLE

    public static final String VERSION_NOT_AVAILABLE

Version not available message.

See Also:
  Constant Field Values

#### BUILD_TIME_NOT_AVAILABLE

    public static final String BUILD_TIME_NOT_AVAILABLE

Build time not available message.

See Also:
  Constant Field Values

#### BASIC_DATA_TYPES

    public static final Class[] BASIC_DATA_TYPES

Basic data types supported by the MLContext API.

#### COMPLEX_DATA_TYPES

    public static final Class[] COMPLEX_DATA_TYPES

Complex data types supported by the MLContext API.

#### ALL_SUPPORTED_DATA_TYPES

    public static final Class[] ALL_SUPPORTED_DATA_TYPES

All data types supported by the MLContext API.

### Constructor Detail

#### MLContextUtil

    public MLContextUtil()

### Method Detail

#### isSparkVersionSupported

    public static boolean isSparkVersionSupported​(String sparkVersion,
                                                  String minimumRecommendedSparkVersion)

Determine whether the Spark version is supported.

Parameters:
  `sparkVersion` - Spark version string (ie, "1.5.0").
  `minimumRecommendedSparkVersion` - Minimum recommended Spark version string (ie, "2.1.0").
Returns:
  `true` if Spark version supported; otherwise `false`.

#### verifySparkVersionSupported

    public static void verifySparkVersionSupported​(org.apache.spark.sql.SparkSession spark)

Check that the Spark version is supported. If it isn't supported, throw an MLContextException.

Parameters:
  `spark` - SparkSession
Throws:
  `MLContextException` - thrown if Spark version isn't supported

#### setDefaultConfig

    public static void setDefaultConfig()

Set default SystemDS configuration properties.

#### setConfig

    public static void setConfig​(String configFilePath)

Set SystemDS configuration properties based on a configuration file.

Parameters:
  `configFilePath` - Path to configuration file.
Throws:
  `MLContextException` - if configuration file was not found or a parse exception occurred

#### setCompilerConfig

    public static void setCompilerConfig()

Set SystemDS compiler configuration properties for MLContext

#### checkInputValueTypes

    public static void checkInputValueTypes​(Map<String,​Object> inputs)

Verify that the types of input values are supported.

Parameters:
  `inputs` - Map of String/Object pairs
Throws:
  `MLContextException` - if an input value type is not supported

#### checkInputValueType

    public static void checkInputValueType​(String name,
                                           Object value)

Verify that the type of input value is supported.

Parameters:
  `name` - The name of the input
  `value` - The value of the input
Throws:
  `MLContextException` - if the input value type is not supported

#### checkInputParameterType

    public static void checkInputParameterType​(String parameterName,
                                               Object parameterValue)

Verify that the type of input parameter value is supported.

Parameters:
  `parameterName` - The name of the input parameter
  `parameterValue` - The value of the input parameter
Throws:
  `MLContextException` - if the input parameter value type is not supported

#### isBasicType

    public static boolean isBasicType​(Object object)

Is the object one of the supported basic data types? (Integer, Boolean, Double, String)

Parameters:
  `object` - the object type to be examined
Returns:
  `true` if type is a basic data type; otherwise `false`.

#### getBasicTypeString

    public static String getBasicTypeString​(Object object)

Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)

Parameters:
  `object` - the object type to be examined
Returns:
  a String representing the type as a SystemDS scalar value type

#### isComplexType

    public static boolean isComplexType​(Object object)

Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)

Parameters:
  `object` - the object type to be examined
Returns:
  `true` if type is a complex data type; otherwise `false`.

#### convertInputParametersForParser

    public static Map<String,​String> convertInputParametersForParser​(Map<String,​Object> basicInputParameterMap)

Converts non-string basic input parameter values to strings to pass to the parser.

Parameters:
  `basicInputParameterMap` - map of input parameters
Returns:
  map of String/String name/value pairs

#### convertInputType

    public static Data convertInputType​(String parameterName,
                                        Object parameterValue)

Convert input types to internal SystemDS representations

Parameters:
  `parameterName` - The name of the input parameter
  `parameterValue` - The value of the input parameter
Returns:
  input in SystemDS data representation

#### convertInputType

    public static Data convertInputType​(String parameterName,
                                        Object parameterValue,
                                        Metadata metadata)

Convert input types to internal SystemDS representations

Parameters:
  `parameterName` - The name of the input parameter
  `parameterValue` - The value of the input parameter
  `metadata` - matrix/frame metadata
Returns:
  input in SystemDS data representation

#### isCSVLineAllNumbers

    public static boolean isCSVLineAllNumbers​(String line)

If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)

Parameters:
  `line` - a line of the RDD
Returns:
  `true` if all the csv-separated values are numbers, `false` otherwise

#### doesDataFrameLookLikeMatrix

    public static boolean doesDataFrameLookLikeMatrix​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)

Examine the DataFrame schema to determine whether the data appears to be a matrix.

Parameters:
  `df` - the DataFrame
Returns:
  `true` if the DataFrame appears to be a matrix, `false` otherwise

#### quotedString

    public static String quotedString​(String str)

Return a double-quoted string with inner single and double quotes escaped.

Parameters:
  `str` - the original string
Returns:
  double-quoted string with inner single and double quotes escaped

#### displayMap

    public static String displayMap​(String mapName,
                                    Map<String,​Object> map)

Display the keys and values in a Map

Parameters:
  `mapName` - the name of the map
  `map` - Map of String keys and Object values
Returns:
  the keys and values in the Map as a String

#### displaySet

    public static String displaySet​(String setName,
                                    Set<String> set)

Display the values in a Set

Parameters:
  `setName` - the name of the Set
  `set` - Set of String values
Returns:
  the values in the Set as a String

#### displaySymbolTable

    public static String displaySymbolTable​(String name,
                                            LocalVariableMap symbolTable)

Display the keys and values in the symbol table

Parameters:
  `name` - the name of the symbol table
  `symbolTable` - the LocalVariableMap
Returns:
  the keys and values in the symbol table as a String

#### displaySymbolTable

    public static String displaySymbolTable​(LocalVariableMap symbolTable)

Display the keys and values in the symbol table

Parameters:
  `symbolTable` - the LocalVariableMap
Returns:
  the keys and values in the symbol table as a String

#### determineOutputTypeAsString

    public static String determineOutputTypeAsString​(LocalVariableMap symbolTable,
                                                     String outputName)

Obtain a symbol table output type as a String

Parameters:
  `symbolTable` - the symbol table
  `outputName` - the name of the output variable
Returns:
  the symbol table output type for a variable as a String

#### displayInputs

    public static String displayInputs​(String name,
                                       Map<String,​Object> map,
                                       LocalVariableMap symbolTable)

Obtain a display of script inputs.

Parameters:
  `name` - the title to display for the inputs
  `map` - the map of inputs
  `symbolTable` - the symbol table
Returns:
  the script inputs represented as a String

#### displayOutputs

    public static String displayOutputs​(String name,
                                        Set<String> outputNames,
                                        LocalVariableMap symbolTable)

Obtain a display of the script outputs.

Parameters:
  `name` - the title to display for the outputs
  `outputNames` - the names of the output variables
  `symbolTable` - the symbol table
Returns:
  the script outputs represented as a String

#### displayOutputs

    public static String displayOutputs​(Set<String> outputNames,
                                        LocalVariableMap symbolTable)

Obtain a display of the script outputs.

Parameters:
  `outputNames` - the names of the output variables
  `symbolTable` - the symbol table
Returns:
  the script outputs represented as a String

#### welcomeMessage

    public static String welcomeMessage()

The SystemDS welcome message

Returns:
  the SystemDS welcome message

#### getSparkContext

    public static org.apache.spark.SparkContext getSparkContext​(MLContext mlContext)

Obtain the Spark Context

Parameters:
  `mlContext` - the SystemDS MLContext
Returns:
  the Spark Context

#### getJavaSparkContext

    public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContext​(MLContext mlContext)

Obtain the Java Spark Context

Parameters:
  `mlContext` - the SystemDS MLContext
Returns:
  the Java Spark Context

#### getSparkContextFromProxy

    public static org.apache.spark.SparkContext getSparkContextFromProxy()

Obtain the Spark Context from the MLContextProxy

Returns:
  the Spark Context

#### getJavaSparkContextFromProxy

    public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContextFromProxy()

Obtain the Java Spark Context from the MLContextProxy

Returns:
  the Java Spark Context

#### getSparkSessionFromProxy

    public static org.apache.spark.sql.SparkSession getSparkSessionFromProxy()

Obtain the Spark Session from the MLContextProxy

Returns:
  the Spark Session

#### doesSymbolTableContainFrameObject

    public static boolean doesSymbolTableContainFrameObject​(LocalVariableMap symbolTable,
                                                            String variableName)

Determine if the symbol table contains a FrameObject with the given variable name.

Parameters:
  `symbolTable` - the LocalVariableMap
  `variableName` - the variable name
Returns:
  `true` if the variable in the symbol table is a FrameObject, `false` otherwise.

#### doesSymbolTableContainMatrixObject

    public static boolean doesSymbolTableContainMatrixObject​(LocalVariableMap symbolTable,
                                                             String variableName)

Determine if the symbol table contains a MatrixObject with the given variable name.

Parameters:
  `symbolTable` - the LocalVariableMap
  `variableName` - the variable name
Returns:
  `true` if the variable in the symbol table is a MatrixObject, `false` otherwise.

#### deleteRemoveVariableInstructions

    public static void deleteRemoveVariableInstructions​(Program progam)

Delete the 'remove variable' instructions from a runtime program.

Parameters:
  `progam` - runtime program