Package org.apache.sysds.resource.enumeration

## Class Enumerator

- java.lang.Object
- - org.apache.sysds.resource.enumeration.Enumerator

Direct Known Subclasses:
  `GridBasedEnumerator`, `InterestBasedEnumerator`, `PruneBasedEnumerator`

    public abstract class Enumerator
    extends Object

### Nested Class Summary

Nested Classes
`static class` `Enumerator.Builder`
`static class` `Enumerator.EnumerationStrategy`
`static class` `Enumerator.OptimizationStrategy`

### Field Summary

Fields
`static double` `COST_DELTA_FRACTION` Time/Monetary delta for considering optimal solutions as fraction
`static int` `DEFAULT_MAX_EXECUTORS` A reasonable upper bound for the possible number of executors is required to set limits for the search space and to avoid evaluating cluster configurations that most probably would have too high distribution overhead
`static int` `DEFAULT_MIN_EXECUTORS`

### Constructor Summary

Constructors
`Enumerator​(Enumerator.Builder builder)`

### Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
`abstract ArrayList<Integer>` `estimateRangeExecutors​(int driverCores, long executorMemory, int executorCores)` Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations
`abstract boolean` `evaluateSingleNodeExecution​(long driverMemory, int cores)`
`double` `getCostsWeightFactor()` Meant to be used for testing purposes
`EnumerationUtils.InstanceSearchSpace` `getDriverSpace()` Meant to be used for testing purposes
`Enumerator.EnumerationStrategy` `getEnumStrategy()` Meant to be used for testing purposes
`EnumerationUtils.InstanceSearchSpace` `getExecutorSpace()` Meant to be used for testing purposes
`HashMap<String,​CloudInstance>` `getInstances()` Meant to be used for testing purposes
`double` `getMaxPrice()` Meant to be used for testing purposes
`double` `getMaxTime()` Meant to be used for testing purposes
`EnumerationUtils.SolutionPoint` `getOptimalSolution()` Meant to be used for testing purposes
`Enumerator.OptimizationStrategy` `getOptStrategy()` Meant to be used for testing purposes
`EnumerationUtils.SolutionPoint` `postprocessing()` Retrieving the estimated optimal configurations after processing.
`abstract void` `preprocessing()` Called once to enumerate the search space for VM instances for driver or executor nodes.
`void` `processing()` Called once after preprocessing to fill the pool with optimal solutions by parsing the enumerated search space.
`static void` `setCostsWeightFactor​(double newFactor)`
`static void` `setCpuQuota​(int newQuotaValue)`
`void` `setDriverSpace​(EnumerationUtils.InstanceSearchSpace inputSpace)` Meant to be used for testing purposes
`void` `setExecutorSpace​(EnumerationUtils.InstanceSearchSpace inputSpace)` Meant to be used for testing purposes
`static void` `setMinPrice​(double maxPrice)`
`static void` `setMinTime​(double maxTime)`
`void` `updateOptimalSolution​(double newTimeEstimate, double newMonetaryEstimate, EnumerationUtils.ConfigurationPoint newPoint)` Invokes the estimation of the time and monetary cost based on the compiled program and the given cluster configurations.

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### DEFAULT_MIN_EXECUTORS

    public static final int DEFAULT_MIN_EXECUTORS

See Also:
  Constant Field Values

#### DEFAULT_MAX_EXECUTORS

    public static final int DEFAULT_MAX_EXECUTORS

A reasonable upper bound for the possible number of executors is required to set limits for the search space and to avoid evaluating cluster configurations that most probably would have too high distribution overhead

See Also:
  Constant Field Values

#### COST_DELTA_FRACTION

    public static final double COST_DELTA_FRACTION

Time/Monetary delta for considering optimal solutions as fraction

See Also:
  Constant Field Values

### Constructor Detail

#### Enumerator

    public Enumerator​(Enumerator.Builder builder)

### Method Detail

#### setCostsWeightFactor

    public static void setCostsWeightFactor​(double newFactor)

#### setMinTime

    public static void setMinTime​(double maxTime)

#### setMinPrice

    public static void setMinPrice​(double maxPrice)

#### setCpuQuota

    public static void setCpuQuota​(int newQuotaValue)

#### preprocessing

    public abstract void preprocessing()

Called once to enumerate the search space for VM instances for driver or executor nodes. These instances are being represented as

#### processing

    public void processing()

Called once after preprocessing to fill the pool with optimal solutions by parsing the enumerated search space. Within its execution the number of potential executor nodes is being estimated (enumerated) dynamically for each parsed executor instance.

#### postprocessing

    public EnumerationUtils.SolutionPoint postprocessing()

Retrieving the estimated optimal configurations after processing.

Returns:
  optimal cluster configuration and corresponding costs

#### evaluateSingleNodeExecution

    public abstract boolean evaluateSingleNodeExecution​(long driverMemory,
                                                        int cores)

#### estimateRangeExecutors

    public abstract ArrayList<Integer> estimateRangeExecutors​(int driverCores,
                                                              long executorMemory,
                                                              int executorCores)

Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations

Parameters:
  `driverCores` - CPU cores for the currently evaluated driver node
  `executorMemory` - memory of currently evaluated executor node
  `executorCores` - CPU cores of currently evaluated executor node
Returns:
  - [min, max]

#### updateOptimalSolution

    public void updateOptimalSolution​(double newTimeEstimate,
                                      double newMonetaryEstimate,
                                      EnumerationUtils.ConfigurationPoint newPoint)

Invokes the estimation of the time and monetary cost based on the compiled program and the given cluster configurations. Following the optimization strategy, the given current optimal solution and the new cost estimation, it decides if the given cluster configuration can be potential optimal solution having lower cost or such a cost that is negligibly higher than the current lowest one.

Parameters:
  `newTimeEstimate` - estimated time cost for the given configurations
  `newMonetaryEstimate` - estimated monetary cost for the given configurations
  `newPoint` - new cluster configuration for estimation

#### getInstances

    public HashMap<String,​CloudInstance> getInstances()

Meant to be used for testing purposes

Returns:
  the available instances for enumeration

#### getDriverSpace

    public EnumerationUtils.InstanceSearchSpace getDriverSpace()

Meant to be used for testing purposes

Returns:
  the object representing the driver search space

#### setDriverSpace

    public void setDriverSpace​(EnumerationUtils.InstanceSearchSpace inputSpace)

Meant to be used for testing purposes

Parameters:
  `inputSpace` - the object representing the driver search space

#### getExecutorSpace

    public EnumerationUtils.InstanceSearchSpace getExecutorSpace()

Meant to be used for testing purposes

Returns:
  the object representing the executor search space

#### setExecutorSpace

    public void setExecutorSpace​(EnumerationUtils.InstanceSearchSpace inputSpace)

Meant to be used for testing purposes

Parameters:
  `inputSpace` - the object representing the executor search space

#### getEnumStrategy

    public Enumerator.EnumerationStrategy getEnumStrategy()

Meant to be used for testing purposes

Returns:
  applied enumeration strategy

#### getOptStrategy

    public Enumerator.OptimizationStrategy getOptStrategy()

Meant to be used for testing purposes

Returns:
  applied optimization strategy

#### getCostsWeightFactor

    public double getCostsWeightFactor()

Meant to be used for testing purposes

Returns:
  configured weight factor optimization function 'costs'

#### getMaxTime

    public double getMaxTime()

Meant to be used for testing purposes

Returns:
  configured max time for consideration (seconds)

#### getMaxPrice

    public double getMaxPrice()

Meant to be used for testing purposes

Returns:
  configured max price for consideration (dollars)

#### getOptimalSolution

    public EnumerationUtils.SolutionPoint getOptimalSolution()

Meant to be used for testing purposes

Returns:
  current optimal solution