Collaboration diagram for com.cloudera.impala.planner.SingleNodePlanner:

Public Member Functions
	SingleNodePlanner (PlannerContext ctx)

PlanNode	createSingleNodePlan () throws ImpalaException

void	migrateConjunctsToInlineView (Analyzer analyzer, InlineViewRef inlineViewRef)

Private Member Functions
PlanNode	createEmptyNode (QueryStmt stmt, Analyzer analyzer) throws InternalException

PlanNode	createQueryPlan (QueryStmt stmt, Analyzer analyzer, boolean disableTopN) throws ImpalaException

PlanNode	addUnassignedConjuncts (Analyzer analyzer, List< TupleId > tupleIds, PlanNode root) throws InternalException

PlanNode	createCheapestJoinPlan (Analyzer analyzer, List< Pair< TableRef, PlanNode >> refPlans) throws ImpalaException

PlanNode	createJoinPlan (Analyzer analyzer, TableRef leftmostRef, List< Pair< TableRef, PlanNode >> refPlans) throws ImpalaException

PlanNode	createFromClauseJoinPlan (Analyzer analyzer, List< Pair< TableRef, PlanNode >> refPlans) throws ImpalaException

PlanNode	createSelectPlan (SelectStmt selectStmt, Analyzer analyzer) throws ImpalaException

PlanNode	createAggregationPlan (SelectStmt selectStmt, Analyzer analyzer, PlanNode root) throws InternalException

PlanNode	createConstantSelectPlan (SelectStmt selectStmt, Analyzer analyzer) throws InternalException

ValueRange	createHBaseValueRange (SlotDescriptor d, List< Expr > conjuncts)

PlanNode	createInlineViewPlan (Analyzer analyzer, InlineViewRef inlineViewRef) throws ImpalaException

boolean	canMigrateConjuncts (InlineViewRef inlineViewRef)

PlanNode	createScanNode (Analyzer analyzer, TableRef tblRef) throws InternalException

void	getHashLookupJoinConjuncts (Analyzer analyzer, List< TupleId > planIds, TableRef joinedTblRef, List< BinaryPredicate > joinConjuncts, List< Expr > joinPredicates)

PlanNode	createJoinNode (Analyzer analyzer, PlanNode outer, PlanNode inner, TableRef outerRef, TableRef innerRef) throws ImpalaException

PlanNode	createTableRefNode (Analyzer analyzer, TableRef tblRef) throws ImpalaException

UnionNode	createUnionPlan (Analyzer analyzer, UnionStmt unionStmt, List< UnionOperand > unionOperands, PlanNode unionDistinctPlan) throws ImpalaException

PlanNode	createUnionPlan (UnionStmt unionStmt, Analyzer analyzer) throws ImpalaException

Private Attributes
final PlannerContext	ctx_

Static Private Attributes
static final Logger	LOG = LoggerFactory.getLogger(SingleNodePlanner.class)

Detailed Description

Constructs a non-executable single-node plan from an analyzed parse tree. The single-node plan does not contain data exchanges or data-reduction optimizations such as local aggregations that are important for distributed execution. The single-node plan needs to be wrapped in a plan fragment for it to be executable.

Definition at line 71 of file SingleNodePlanner.java.

Constructor & Destructor Documentation

com.cloudera.impala.planner.SingleNodePlanner.SingleNodePlanner ( PlannerContext ctx )

inline

Definition at line 76 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.ctx_.

Member Function Documentation

PlanNode com.cloudera.impala.planner.SingleNodePlanner.addUnassignedConjuncts	(	Analyzer	analyzer,
		List< TupleId >	tupleIds,
		PlanNode	root
	)		throws InternalException

inlineprivate

If there are unassigned conjuncts that are bound by tupleIds or if there are slot equivalences for tupleIds that have not yet been enforced, returns a SelectNode on top of root that evaluates those conjuncts; otherwise returns root unchanged. TODO: change this to assign the unassigned conjuncts to root itself, if that is semantically correct

Definition at line 212 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.ctx_, and com.cloudera.impala.planner.PlannerContext.getNextNodeId().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan(), and com.cloudera.impala.planner.SingleNodePlanner.createUnionPlan().

boolean com.cloudera.impala.planner.SingleNodePlanner.canMigrateConjuncts ( InlineViewRef inlineViewRef )

inlineprivate

Checks if conjuncts can be migrated into an inline view.

Definition at line 838 of file SingleNodePlanner.java.

References com.cloudera.impala.analysis.InlineViewRef.getViewStmt().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan(), and com.cloudera.impala.planner.SingleNodePlanner.migrateConjunctsToInlineView().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createAggregationPlan	(	SelectStmt	selectStmt,
		Analyzer	analyzer,
		PlanNode	root
	)		throws InternalException

inlineprivate

Returns a new AggregationNode that materializes the aggregation of the given stmt. Assigns conjuncts from the Having clause to the returned node.

Definition at line 564 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.ctx_, com.cloudera.impala.planner.PlannerContext.getNextNodeId(), and com.cloudera.impala.analysis.AggregateInfo.isDistinctAgg().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createCheapestJoinPlan	(	Analyzer	analyzer,
		List< Pair< TableRef, PlanNode >>	refPlans
	)		throws ImpalaException

inlineprivate

Return the cheapest plan that materializes the joins of all TblRefs in refPlans. Assumes that refPlans are in the order as they originally appeared in the query. For this plan:

the plan is executable, ie, all non-cross joins have equi-join predicates
the leftmost scan is over the largest of the inputs for which we can still construct an executable plan
all rhs's are in decreasing order of selectiveness (percentage of rows they eliminate)
outer/cross/semi joins: rhs serialized size is < lhs serialized size; enforced via join inversion, if necessary Returns null if we can't create an executable plan.

Definition at line 246 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createJoinPlan(), com.cloudera.impala.planner.PlanNode.getAvgRowSize(), com.cloudera.impala.planner.PlanNode.getCardinality(), com.cloudera.impala.analysis.JoinOperator.isCrossJoin(), com.cloudera.impala.analysis.JoinOperator.isNullAwareLeftAntiJoin(), and com.cloudera.impala.analysis.JoinOperator.isOuterJoin().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createConstantSelectPlan	(	SelectStmt	selectStmt,
		Analyzer	analyzer
	)		throws InternalException

inlineprivate

Returns a UnionNode that materializes the exprs of the constant selectStmt. Replaces the resultExprs of the selectStmt with SlotRefs into the materialized tuple.

Definition at line 592 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.ctx_, and com.cloudera.impala.planner.PlannerContext.getNextNodeId().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createEmptyNode	(	QueryStmt	stmt,
		Analyzer	analyzer
	)		throws InternalException

inlineprivate

Creates an EmptyNode that 'materializes' the tuples of the given stmt.

Definition at line 127 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.ctx_, and com.cloudera.impala.planner.PlannerContext.getNextNodeId().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan(), and com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createFromClauseJoinPlan	(	Analyzer	analyzer,
		List< Pair< TableRef, PlanNode >>	refPlans
	)		throws ImpalaException

inlineprivate

Return a plan with joins in the order of refPlans (= FROM clause order).

Definition at line 465 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createJoinNode().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan().

ValueRange com.cloudera.impala.planner.SingleNodePlanner.createHBaseValueRange	(	SlotDescriptor	d,
		List< Expr >	conjuncts
	)

inlineprivate

Transform '=', '<[=]' and '>[=]' comparisons for given slot into ValueRange. Also removes those predicates which were used for the construction of ValueRange from 'conjuncts_'. Only looks at comparisons w/ string constants (ie, the bounds of the result can be evaluated with Expr::GetValue(NULL)). HBase row key filtering works only if the row key is mapped to a string column and the expression is a string constant expression. If there are multiple competing comparison predicates that could be used to construct a ValueRange, only the first one from each category is chosen.

Definition at line 632 of file SingleNodePlanner.java.

References com.cloudera.impala.analysis.BinaryPredicate.getOp(), com.cloudera.impala.analysis.Expr.isConstant(), com.cloudera.impala.analysis.BinaryPredicate.Operator.Operator(), and com.cloudera.impala.catalog.Type.STRING.

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createScanNode().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan	(	Analyzer	analyzer,
		InlineViewRef	inlineViewRef
	)		throws ImpalaException

inlineprivate

Returns plan tree for an inline view ref:

predicates from the enclosing scope that can be evaluated directly within the inline-view plan are pushed down
predicates that cannot be evaluated directly within the inline-view plan but only apply to the inline view are evaluated in a SelectNode placed on top of the inline view plan
all slots that are referenced by predicates from the enclosing scope that cannot be pushed down are marked as materialized (so that when computeMemLayout() is called on the base table descriptors materialized by the inline view it has a complete picture)

Definition at line 684 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.addUnassignedConjuncts(), com.cloudera.impala.planner.SingleNodePlanner.canMigrateConjuncts(), com.cloudera.impala.planner.SingleNodePlanner.createEmptyNode(), com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan(), com.cloudera.impala.planner.SingleNodePlanner.ctx_, com.cloudera.impala.analysis.QueryStmt.getAnalyzer(), com.cloudera.impala.planner.PlannerContext.getNextNodeId(), com.cloudera.impala.analysis.SelectStmt.getTableRefs(), com.cloudera.impala.planner.PlanNode.getTupleIds(), and com.cloudera.impala.planner.SingleNodePlanner.migrateConjunctsToInlineView().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createTableRefNode().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createJoinNode	(	Analyzer	analyzer,
		PlanNode	outer,
		PlanNode	inner,
		TableRef	outerRef,
		TableRef	innerRef
	)		throws ImpalaException

inlineprivate

Create a node to join outer with inner. Either the outer or the inner may be a plan created from a table ref (but not both), and the corresponding outer/innerRef should be non-null.

Definition at line 977 of file SingleNodePlanner.java.

References com.cloudera.impala.analysis.JoinOperator.CROSS_JOIN, com.cloudera.impala.planner.SingleNodePlanner.getHashLookupJoinConjuncts(), com.cloudera.impala.analysis.JoinOperator.INNER_JOIN, and com.cloudera.impala.analysis.BinaryPredicate.isNullMatchingEq().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createFromClauseJoinPlan(), and com.cloudera.impala.planner.SingleNodePlanner.createJoinPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createJoinPlan	(	Analyzer	analyzer,
		TableRef	leftmostRef,
		List< Pair< TableRef, PlanNode >>	refPlans
	)		throws ImpalaException

inlineprivate

Returns a plan with leftmostRef's plan as its leftmost input; the joins are in decreasing order of selectiveness (percentage of rows they eliminate). The leftmostRef's join will be inverted if it is an outer/semi/cross join.

Definition at line 311 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createJoinNode(), com.cloudera.impala.planner.PlanNode.getCardinality(), com.cloudera.impala.analysis.TableRef.getJoinOp(), com.cloudera.impala.analysis.TableRef.getLeftTblRef(), and com.cloudera.impala.analysis.TableRef.getUniqueAlias().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createCheapestJoinPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan	(	QueryStmt	stmt,
		Analyzer	analyzer,
		boolean	disableTopN
	)		throws ImpalaException

inlineprivate

Create plan tree for single-node execution. Generates PlanNodes for the Select/Project/Join/Union [All]/Group by/Having/Order by clauses of the query stmt.

Definition at line 145 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createEmptyNode(), com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan(), com.cloudera.impala.planner.SingleNodePlanner.createUnionPlan(), com.cloudera.impala.planner.SingleNodePlanner.ctx_, and com.cloudera.impala.planner.PlannerContext.getNextNodeId().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan(), com.cloudera.impala.planner.SingleNodePlanner.createSingleNodePlan(), and com.cloudera.impala.planner.SingleNodePlanner.createUnionPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createScanNode	(	Analyzer	analyzer,
		TableRef	tblRef
	)		throws InternalException

inlineprivate

Create node for scanning all data files of a particular table.

Definition at line 848 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createHBaseValueRange(), com.cloudera.impala.planner.SingleNodePlanner.ctx_, com.cloudera.impala.planner.PlannerContext.getNextNodeId(), and com.cloudera.impala.analysis.SlotDescriptor.getType().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createTableRefNode().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan	(	SelectStmt	selectStmt,
		Analyzer	analyzer
	)		throws ImpalaException

inlineprivate

Create tree of PlanNodes that implements the Select/Project/Join/Group by/Having of the selectStmt query block.

Definition at line 484 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createAggregationPlan(), com.cloudera.impala.planner.SingleNodePlanner.createCheapestJoinPlan(), com.cloudera.impala.planner.SingleNodePlanner.createConstantSelectPlan(), com.cloudera.impala.planner.SingleNodePlanner.createFromClauseJoinPlan(), com.cloudera.impala.planner.SingleNodePlanner.createTableRefNode(), com.cloudera.impala.planner.SingleNodePlanner.ctx_, and com.cloudera.impala.planner.PlannerContext.getNextNodeId().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createSingleNodePlan ( ) throws ImpalaException

inline

Generates and returns the root of the single-node plan for the analyzed parse tree in the planner context. The planning process recursively walks the parse tree and performs the following actions. In the top-down phase over query statements:

materialize the slots required for evaluating expressions of that statement
migrate conjuncts from parent blocks into inline views and union operands In the bottom-up phase generate the plan tree for every query statement:
perform join-order optimization when generating the plan of the FROM clause of a select statement; requires that all materialized slots are known for an accurate estimate of row sizes needed for cost-based join ordering
assign conjuncts that can be evaluated at that node and compute the stats of that node (cardinality, etc.)
apply combined expression substitution map of child plan nodes; if a plan node re-maps its input, set a substitution map to be applied by parents

Definition at line 96 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan(), com.cloudera.impala.planner.SingleNodePlanner.ctx_, com.cloudera.impala.analysis.QueryStmt.getBaseTblResultExprs(), and com.cloudera.impala.planner.PlannerContext.getQueryOptions().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createTableRefNode	(	Analyzer	analyzer,
		TableRef	tblRef
	)		throws ImpalaException

inlineprivate

Create a tree of PlanNodes for the given tblRef, which can be a BaseTableRef, CollectionTableRef or an InlineViewRef.

Definition at line 1080 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan(), and com.cloudera.impala.planner.SingleNodePlanner.createScanNode().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan().

UnionNode com.cloudera.impala.planner.SingleNodePlanner.createUnionPlan	(	Analyzer	analyzer,
		UnionStmt	unionStmt,
		List< UnionOperand >	unionOperands,
		PlanNode	unionDistinctPlan
	)		throws ImpalaException

inlineprivate

Create a plan tree corresponding to 'unionOperands' for the given unionStmt. The individual operands' plan trees are attached to a single UnionNode. If unionDistinctPlan is not null, it is expected to contain the plan for the distinct portion of the given unionStmt. The unionDistinctPlan is then added as a child of the returned UnionNode.

Definition at line 1098 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan(), com.cloudera.impala.planner.SingleNodePlanner.ctx_, com.cloudera.impala.planner.PlannerContext.getNextNodeId(), and com.cloudera.impala.analysis.SelectStmt.getTableRefs().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan(), and com.cloudera.impala.planner.SingleNodePlanner.createUnionPlan().

PlanNode com.cloudera.impala.planner.SingleNodePlanner.createUnionPlan	(	UnionStmt	unionStmt,
		Analyzer	analyzer
	)		throws ImpalaException

inlineprivate

Returns plan tree for unionStmt:

distinctOperands' plan trees are collected in a single UnionNode and duplicates removed via distinct aggregation
the output of that plus the allOperands' plan trees are collected in another UnionNode which materializes the result of unionStmt
if any of the union operands contains analytic exprs, we avoid pushing predicates directly into the operands and instead evaluate them after the final UnionNode (see createInlineViewPlan() for the reasoning) TODO: optimize this by still pushing predicates into the union operands that don't contain analytic exprs and evaluating the conjuncts in Select directly above the AnalyticEvalNodes

Definition at line 1141 of file SingleNodePlanner.java.

References com.cloudera.impala.planner.SingleNodePlanner.addUnassignedConjuncts(), com.cloudera.impala.planner.SingleNodePlanner.createUnionPlan(), com.cloudera.impala.planner.SingleNodePlanner.ctx_, and com.cloudera.impala.planner.PlannerContext.getNextNodeId().

void com.cloudera.impala.planner.SingleNodePlanner.getHashLookupJoinConjuncts	(	Analyzer	analyzer,
		List< TupleId >	planIds,
		TableRef	joinedTblRef,
		List< BinaryPredicate >	joinConjuncts,
		List< Expr >	joinPredicates
	)

inlineprivate

Return all applicable conjuncts for join between a plan tree and a single TableRef; the conjuncts can be used for hash table lookups.

for inner joins, those are equi-join predicates in which one side is fully bound by planIds and the other by joinedTblRef.id_;
for outer joins: same type of conjuncts_ as inner joins, but only from the JOIN clause Returns the conjuncts_ in 'joinConjuncts' (in which "<lhs> = <rhs>" is returned as BinaryPredicate and also in their original form in 'joinPredicates'. Each lhs is bound by planIds, and each rhs by the tuple id of joinedTblRef. Predicates that are redundant based on equivalences classes are intentionally returneded by this function because the removal of redundant predicates and the creation of new predicates for enforcing slot equivalences go hand-in-hand (see analyzer.createEquivConjuncts()).

Definition at line 912 of file SingleNodePlanner.java.

References com.cloudera.impala.analysis.TableRef.getDesc().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createJoinNode().

void com.cloudera.impala.planner.SingleNodePlanner.migrateConjunctsToInlineView	(	Analyzer	analyzer,
		InlineViewRef	inlineViewRef
	)

inline

Migrates unassigned conjuncts into an inline view. Conjuncts are not migrated into the inline view if the view has a LIMIT/OFFSET clause or if the view's stmt computes analytic functions (see IMPALA-1243/IMPALA-1900). The reason is that analytic functions compute aggregates over their entire input, and applying filters from the enclosing scope before the aggregate computation would alter the results. This is unlike regular aggregate computation, which only makes the output of the computation visible to the enclosing scope, so that filters from the enclosing scope can be safely applied (to the grouping cols, say).

Definition at line 769 of file SingleNodePlanner.java.

References com.cloudera.impala.analysis.Analyzer.canEvalPredicate(), com.cloudera.impala.planner.SingleNodePlanner.canMigrateConjuncts(), com.cloudera.impala.analysis.TableRef.getId(), and com.cloudera.impala.analysis.Expr.isRegisteredPredicate().

Referenced by com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan().

Member Data Documentation

final PlannerContext com.cloudera.impala.planner.SingleNodePlanner.ctx_

private

Definition at line 74 of file SingleNodePlanner.java.

final Logger com.cloudera.impala.planner.SingleNodePlanner.LOG = LoggerFactory.getLogger(SingleNodePlanner.class)

staticprivate

Definition at line 72 of file SingleNodePlanner.java.

The documentation for this class was generated from the following file:

fe/src/main/java/com/cloudera/impala/planner/SingleNodePlanner.java

Public Member Functions

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation