Package org.apache.sysds.resource.enumeration

## Class InterestBasedEnumerator

- java.lang.Object
- - org.apache.sysds.resource.enumeration.Enumerator
  - - org.apache.sysds.resource.enumeration.InterestBasedEnumerator

    public class InterestBasedEnumerator
    extends Enumerator

### Nested Class Summary

### Nested classes/interfaces inherited from class org.apache.sysds.resource.enumeration.Enumerator

`Enumerator.Builder, Enumerator.EnumerationStrategy, Enumerator.OptimizationStrategy`

### Field Summary

Fields
`static double` `BROADCAST_MEMORY_FACTOR`
`static double` `CACHE_MEMORY_FACTOR`
`static double` `MEMORY_DELTA_FRACTION`
`static double` `MEMORY_FACTOR`
`static long` `MINIMUM_RELEVANT_MEM_ESTIMATE`
`static boolean` `USE_MEMORY_DELTA` different instance families can have slightly different memory characteristics and using memory delta allows not ignoring equivalent instances of different families (e.g.

### Fields inherited from class org.apache.sysds.resource.enumeration.Enumerator

`COST_DELTA_FRACTION, DEFAULT_MAX_EXECUTORS, DEFAULT_MIN_EXECUTORS`

### Constructor Summary

Constructors
`InterestBasedEnumerator​(Enumerator.Builder builder, boolean interestLargestEstimate, boolean fitDriverMemory, boolean interestBroadcastVars, boolean interestOutputCaching)`

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`ArrayList<Integer>` `estimateRangeExecutors​(int driverCores, long executorMemory, int executorCores)` Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations
`boolean` `evaluateSingleNodeExecution​(long driverMemory, int cores)`
`static TreeSet<Long>` `getMemoryEstimates​(Program currentProgram, boolean outputOnly, double memoryFactor)` Extracts the memory estimates which original size is larger than `MINIMUM_RELEVANT_MEM_ESTIMATE`
`boolean` `interestBroadcastVars()`
`boolean` `interestEstimatesInCPEnabled()`
`boolean` `interestLargestEstimateEnabled()`
`boolean` `interestOutputCachingEnabled()`
`void` `preprocessing()` Called once to enumerate the search space for VM instances for driver or executor nodes.

### Methods inherited from class org.apache.sysds.resource.enumeration.Enumerator

`getCostsWeightFactor, getDriverSpace, getEnumStrategy, getExecutorSpace, getInstances, getMaxPrice, getMaxTime, getOptimalSolution, getOptStrategy, postprocessing, processing, setCostsWeightFactor, setCpuQuota, setDriverSpace, setExecutorSpace, setMinPrice, setMinTime, updateOptimalSolution`

### Methods inherited from class java.lang.Object

`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### MINIMUM_RELEVANT_MEM_ESTIMATE

    public static final long MINIMUM_RELEVANT_MEM_ESTIMATE

See Also:
  Constant Field Values

#### USE_MEMORY_DELTA

    public static final boolean USE_MEMORY_DELTA

different instance families can have slightly different memory characteristics and using memory delta allows not ignoring equivalent instances of different families (e.g. newer generation use 7.5/15.25/30.5/... GB memory instead of 8/16/32/...)

See Also:
  Constant Field Values

#### MEMORY_DELTA_FRACTION

    public static final double MEMORY_DELTA_FRACTION

See Also:
  Constant Field Values

#### MEMORY_FACTOR

    public static final double MEMORY_FACTOR

#### BROADCAST_MEMORY_FACTOR

    public static final double BROADCAST_MEMORY_FACTOR

See Also:
  Constant Field Values

#### CACHE_MEMORY_FACTOR

    public static final double CACHE_MEMORY_FACTOR

See Also:
  Constant Field Values

### Constructor Detail

#### InterestBasedEnumerator

    public InterestBasedEnumerator​(Enumerator.Builder builder,
                                   boolean interestLargestEstimate,
                                   boolean fitDriverMemory,
                                   boolean interestBroadcastVars,
                                   boolean interestOutputCaching)

### Method Detail

#### preprocessing

    public void preprocessing()

Description copied from class: `Enumerator`
Called once to enumerate the search space for VM instances for driver or executor nodes. These instances are being represented as

Specified by:
  `preprocessing` in class `Enumerator`

#### evaluateSingleNodeExecution

    public boolean evaluateSingleNodeExecution​(long driverMemory,
                                               int cores)

Specified by:
  `evaluateSingleNodeExecution` in class `Enumerator`

#### estimateRangeExecutors

    public ArrayList<Integer> estimateRangeExecutors​(int driverCores,
                                                     long executorMemory,
                                                     int executorCores)

Description copied from class: `Enumerator`
Estimates the minimum and maximum number of executors based on given VM instance characteristics, the enumeration strategy and the user-defined configurations

Specified by:
  `estimateRangeExecutors` in class `Enumerator`
Parameters:
  `driverCores` - CPU cores for the currently evaluated driver node
  `executorMemory` - memory of currently evaluated executor node
  `executorCores` - CPU cores of currently evaluated executor node
Returns:
  - [min, max]

#### getMemoryEstimates

    public static TreeSet<Long> getMemoryEstimates​(Program currentProgram,
                                                   boolean outputOnly,
                                                   double memoryFactor)

Extracts the memory estimates which original size is larger than `MINIMUM_RELEVANT_MEM_ESTIMATE`

Parameters:
  `currentProgram` - program for extracting the memory estimates from
  `outputOnly` - `true` - output estimate only; `false` - sum of input, intermediate and output estimates
  `memoryFactor` - factor for reverse scaling the estimates to avoid scaling the search space parameters representing the nodes' memory budget
Returns:
  memory estimates in ascending order ensured by the `TreeSet` data structure

#### interestEstimatesInCPEnabled

    public boolean interestEstimatesInCPEnabled()

#### interestBroadcastVars

    public boolean interestBroadcastVars()

#### interestLargestEstimateEnabled

    public boolean interestLargestEstimateEnabled()

#### interestOutputCachingEnabled

    public boolean interestOutputCachingEnabled()