com.cloudera.impala.planner.PipelinedPlanNodeSet Class Reference

Collaboration diagram for com.cloudera.impala.planner.PipelinedPlanNodeSet:

Public Member Functions
void	add (PlanNode node)
void	addSink (DataSink sink)
boolean	computeResourceEstimates (boolean excludeUnpartitionedFragments, TQueryOptions queryOptions)
long	getPerHostMem ()
int	getPerHostVcores ()

Static Public Member Functions
static ArrayList < PipelinedPlanNodeSet >	computePlanNodeSets (PlanNode root)

Static Public Attributes
static final long	MIN_PER_HOST_MEM = 10 * 1024 * 1024
static final int	MIN_PER_HOST_VCORES = 1

Static Private Member Functions
static void	computePlanNodeSets (PlanNode node, PipelinedPlanNodeSet lhsSet, PipelinedPlanNodeSet rhsSet, ArrayList< PipelinedPlanNodeSet > planNodeSets)

Private Attributes
final ArrayList< PlanNode >	planNodes = Lists.newArrayList()
final List< DataSink >	dataSinks = Lists.newArrayList()
long	perHostMem = MIN_PER_HOST_MEM
int	perHostVcores = MIN_PER_HOST_VCORES

Static Private Attributes
static final Logger	LOG = LoggerFactory.getLogger(PipelinedPlanNodeSet.class)

Detailed Description

Represents a set of PlanNodes and DataSinks that execute and consume resources concurrently. PlanNodes and DataSinks in such a pipelined plan node set may belong to different plan fragments because data is streamed across fragments.

For example, a series of left-deep joins consists of two plan node sets. The first set contains all build-side nodes. The second set contains the leftmost scan. Both sets contain all join nodes because they execute and consume resources during the build and probe phases. Similarly, all nodes below a 'blocking' node (e.g, an AggregationNode) are placed into a differnet plan node set than the nodes above it, but the blocking node itself belongs to both sets.

Definition at line 41 of file PipelinedPlanNodeSet.java.

Member Function Documentation

void com.cloudera.impala.planner.PipelinedPlanNodeSet.add ( PlanNode  node )

inline

Definition at line 61 of file PipelinedPlanNodeSet.java.

void com.cloudera.impala.planner.PipelinedPlanNodeSet.addSink ( DataSink  sink )

inline

Definition at line 66 of file PipelinedPlanNodeSet.java.

static ArrayList<PipelinedPlanNodeSet> com.cloudera.impala.planner.PipelinedPlanNodeSet.computePlanNodeSets ( PlanNode  root )

inlinestatic

Computes and returns the pipelined plan node sets of the given plan.

Definition at line 157 of file PipelinedPlanNodeSet.java.

Referenced by com.cloudera.impala.planner.PipelinedPlanNodeSet.computePlanNodeSets().

static void com.cloudera.impala.planner.PipelinedPlanNodeSet.computePlanNodeSets ( PlanNode  node, PipelinedPlanNodeSet  lhsSet, PipelinedPlanNodeSet  rhsSet, ArrayList< PipelinedPlanNodeSet >  planNodeSets  )

inlinestaticprivate

Populates 'planNodeSets' by recursively traversing the plan tree rooted at 'node' The plan node sets are computed top-down. As a result, the plan node sets are added in reverse order of their runtime execution.

Nodes are generally added to lhsSet. Joins are treated specially in that their left child is added to lhsSet and their right child to rhsSet to make sure that concurrent join builds end up in the same plan node set.

Definition at line 173 of file PipelinedPlanNodeSet.java.

References com.cloudera.impala.planner.PipelinedPlanNodeSet.computePlanNodeSets(), com.cloudera.impala.planner.PlanNode.getFragment(), and com.cloudera.impala.planner.PlanNode.isBlockingNode().

boolean com.cloudera.impala.planner.PipelinedPlanNodeSet.computeResourceEstimates ( boolean  excludeUnpartitionedFragments, TQueryOptions  queryOptions  )

inline

Computes the estimated per-host memory and CPU requirements of this plan node set. Optionally excludes unpartitioned fragments from the estimation. Returns true if at least one plan node was included in the estimation. Otherwise returns false indicating the estimates are invalid.

Definition at line 77 of file PipelinedPlanNodeSet.java.

References com.cloudera.impala.planner.PlanNode.getNumNodes(), com.cloudera.impala.planner.DataSink.getPerHostMemCost(), com.cloudera.impala.planner.PlanNode.getPerHostMemCost(), com.cloudera.impala.planner.PipelinedPlanNodeSet.perHostMem, and com.cloudera.impala.planner.PipelinedPlanNodeSet.perHostVcores.

long com.cloudera.impala.planner.PipelinedPlanNodeSet.getPerHostMem ( )

inline

Definition at line 151 of file PipelinedPlanNodeSet.java.

References com.cloudera.impala.planner.PipelinedPlanNodeSet.perHostMem.

int com.cloudera.impala.planner.PipelinedPlanNodeSet.getPerHostVcores ( )

inline

Definition at line 152 of file PipelinedPlanNodeSet.java.

References com.cloudera.impala.planner.PipelinedPlanNodeSet.perHostVcores.

Member Data Documentation

final List<DataSink> com.cloudera.impala.planner.PipelinedPlanNodeSet.dataSinks = Lists.newArrayList()

private

Definition at line 54 of file PipelinedPlanNodeSet.java.

final Logger com.cloudera.impala.planner.PipelinedPlanNodeSet.LOG = LoggerFactory.getLogger(PipelinedPlanNodeSet.class)

staticprivate

Definition at line 42 of file PipelinedPlanNodeSet.java.

final long com.cloudera.impala.planner.PipelinedPlanNodeSet.MIN_PER_HOST_MEM = 10 * 1024 * 1024

static

Definition at line 46 of file PipelinedPlanNodeSet.java.

final int com.cloudera.impala.planner.PipelinedPlanNodeSet.MIN_PER_HOST_VCORES = 1

static

Definition at line 47 of file PipelinedPlanNodeSet.java.

long com.cloudera.impala.planner.PipelinedPlanNodeSet.perHostMem = MIN_PER_HOST_MEM

private

Definition at line 58 of file PipelinedPlanNodeSet.java.

Referenced by com.cloudera.impala.planner.PipelinedPlanNodeSet.computeResourceEstimates(), and com.cloudera.impala.planner.PipelinedPlanNodeSet.getPerHostMem().

int com.cloudera.impala.planner.PipelinedPlanNodeSet.perHostVcores = MIN_PER_HOST_VCORES

private

Definition at line 59 of file PipelinedPlanNodeSet.java.

Referenced by com.cloudera.impala.planner.PipelinedPlanNodeSet.computeResourceEstimates(), and com.cloudera.impala.planner.PipelinedPlanNodeSet.getPerHostVcores().

final ArrayList<PlanNode> com.cloudera.impala.planner.PipelinedPlanNodeSet.planNodes = Lists.newArrayList()

private

Definition at line 50 of file PipelinedPlanNodeSet.java.

---

The documentation for this class was generated from the following file:

• fe/src/main/java/com/cloudera/impala/planner/PipelinedPlanNodeSet.java