# Package org.apache.sysds.runtime.controlprogram.parfor

Class Summary
CachedReuseVariables
DataPartitioner: This is the base class for all data partitioner.
DataPartitionerLocal: Partitions a given matrix into row or column partitions with a two pass-approach.
DataPartitionerRemoteSpark: MR job class for submitting parfor remote partitioning MR jobs.
DataPartitionerRemoteSparkMapper: NOTE: for the moment we only support binary block here TODO extend impl for binarycell and textcell Interface of Writable output in order to support both PairWritableBlock and PairWritableCell.
DataPartitionerRemoteSparkReducer
LocalParWorker: Instances of this class can be used to execute tasks in parallel.
LocalTaskQueue<T>: This class provides a way of dynamic task distribution to multiple workers in local multi-threaded environments.
ParForBody: Wrapper for exchanging parfor body data structures.
ParWorker: Super class for master/worker pattern implementations.
RemoteDPParForSpark: TODO heavy hitter maintenance TODO data partitioning with binarycell
RemoteDPParForSparkWorker
RemoteParForJobReturn: Wrapper for job return of ParFor REMOTE for transferring statistics and result symbol table.
RemoteParForSpark: This class serves two purposes: (1) isolating Spark imports to enable running in environments where no Spark libraries are available, and (2) to follow the same structure as the parfor remote_mr job submission.
RemoteParForSparkWorker
RemoteParForUtils: Common functionalities for parfor workers in MR jobs.
ResultMerge<T extends CacheableData<?>>
ResultMergeFrameLocalMemory
ResultMergeLocalAutomatic
ResultMergeLocalFile
ResultMergeLocalMemory: Local in-memory realization of result merge.
ResultMergeMatrix: Due to independence of all iterations, any result has the following properties:
ResultMergeRemoteSpark
ResultMergeRemoteSparkWCompare
Task: A task is a logical group of one or multiple iterations (each iteration is assigned to exactly one task).
TaskPartitioner: This is the base class for all task partitioner.
TaskPartitionerFactoring: This factoring task partitioner virtually iterates over the given FOR loop (from, to, incr), creates iterations and group them to tasks.
TaskPartitionerFactoringCmax: Factoring with maximum constraint (e.g., if LIX matrix out-of-core and we need to bound the maximum number of iterations per map task -> memory bounds)
TaskPartitionerFactoringCmin: Factoring with minimum constraint (e.g., if communication is expensive)
TaskPartitionerFactory
TaskPartitionerFixedsize: This naive task partitioner virtually iterates over the given FOR loop (from, to, incr), creates iterations and group them to tasks according to the given task size.
TaskPartitionerNaive: This static task partitioner virtually iterates over the given FOR loop (from, to, incr), creates iterations and group them to tasks according to a task size of numIterations/numWorkers.
TaskPartitionerStatic: This static task partitioner virtually iterates over the given FOR loop (from, to, incr), creates iterations and group them to tasks according to a task size of numIterations/numWorkers.

Enum Summary
Task.TaskType