Package org.apache.sysds.resource

## Class ResourceCompiler

- java.lang.Object
- - org.apache.sysds.resource.ResourceCompiler

    public class ResourceCompiler
    extends Object

This class does full or partial program recompilation based on given runtime program. It uses the methods provided by `hops.recompile.Recompiler`). It keeps a state of the current recompilation phase in order to decide when to do full recompilation and when not.

### Field Summary

Fields
`static long` `DEFAULT_DRIVER_MEMORY`
`static int` `DEFAULT_DRIVER_THREADS`
`static long` `DEFAULT_EXECUTOR_MEMORY`
`static int` `DEFAULT_EXECUTOR_THREADS`
`static int` `DEFAULT_NUMBER_EXECUTORS`

### Constructor Summary

Constructors
`ResourceCompiler()`

### Method Summary

All Methods Static Methods Concrete Methods
`static Program` `compile​(String filePath, Map<String,​String> args)`
`static Program` `compile​(String filePath, Map<String,​String> args, HashMap<String,​String> replaceVars)`
`static Program` `doFullRecompilation​(Program program)` Recompiling a given program for resource optimization.
`static void` `replaceFilename​(DMLProgram dmlp, HashMap<String,​String> replaceVars)`
`static void` `setSingleNodeResourceConfigs​(long nodeMemory, int nodeCores)` Sets resource configurations for executions in single-node mode including the hardware configurations for the node running the CP.
`static void` `setSparkClusterResourceConfigs​(long driverMemory, int driverCores, int numExecutors, long executorMemory, int executorCores)` Sets resource configurations for executions in hybrid mode including the hardware configurations for the node running the CP and the worker nodes running Spark executors

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### DEFAULT_DRIVER_MEMORY

    public static final long DEFAULT_DRIVER_MEMORY

See Also:
  Constant Field Values

#### DEFAULT_DRIVER_THREADS

    public static final int DEFAULT_DRIVER_THREADS

See Also:
  Constant Field Values

#### DEFAULT_EXECUTOR_MEMORY

    public static final long DEFAULT_EXECUTOR_MEMORY

See Also:
  Constant Field Values

#### DEFAULT_EXECUTOR_THREADS

    public static final int DEFAULT_EXECUTOR_THREADS

See Also:
  Constant Field Values

#### DEFAULT_NUMBER_EXECUTORS

    public static final int DEFAULT_NUMBER_EXECUTORS

See Also:
  Constant Field Values

### Constructor Detail

#### ResourceCompiler

    public ResourceCompiler()

### Method Detail

#### compile

    public static Program compile​(String filePath,
                                  Map<String,​String> args)
                           throws IOException

Throws:
  `IOException`

#### compile

    public static Program compile​(String filePath,
                                  Map<String,​String> args,
                                  HashMap<String,​String> replaceVars)
                           throws IOException

Throws:
  `IOException`

#### replaceFilename

    public static void replaceFilename​(DMLProgram dmlp,
                                       HashMap<String,​String> replaceVars)

#### doFullRecompilation

    public static Program doFullRecompilation​(Program program)

Recompiling a given program for resource optimization. This method should always be called after setting the target resources to `InfrastructureAnalyzer` and `SparkExecutionContext`

Parameters:
  `program` - program to be recompiled
Returns:
  the recompiled program as a new `Program` instance

#### setSingleNodeResourceConfigs

    public static void setSingleNodeResourceConfigs​(long nodeMemory,
                                                    int nodeCores)

Sets resource configurations for executions in single-node mode including the hardware configurations for the node running the CP.

Parameters:
  `nodeMemory` - memory budget for the node running CP
  `nodeCores` - number of CPU cores for the node running CP

#### setSparkClusterResourceConfigs

    public static void setSparkClusterResourceConfigs​(long driverMemory,
                                                      int driverCores,
                                                      int numExecutors,
                                                      long executorMemory,
                                                      int executorCores)

Sets resource configurations for executions in hybrid mode including the hardware configurations for the node running the CP and the worker nodes running Spark executors

Parameters:
  `driverMemory` - memory budget for the node running CP
  `driverCores` - number of CPU cores for the node running CP
  `numExecutors` - number of nodes in cluster
  `executorMemory` - memory budget for the nodes running executors
  `executorCores` - number of CPU cores for the nodes running executors