Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros
com.cloudera.impala.planner.UnionNode Class Reference
Inheritance diagram for com.cloudera.impala.planner.UnionNode:
Collaboration diagram for com.cloudera.impala.planner.UnionNode:

Public Member Functions

void addConstExprList (List< Expr > exprs)
 
void addChild (PlanNode node, List< Expr > baseTblResultExprs)
 
void computeStats (Analyzer analyzer)
 
void reorderOperands (Analyzer analyzer)
 
void init (Analyzer analyzer) throws InternalException
 
PlanNodeId getId ()
 
void setId (PlanNodeId id)
 
long getLimit ()
 
boolean hasLimit ()
 
long getPerHostMemCost ()
 
long getCardinality ()
 
int getNumNodes ()
 
float getAvgRowSize ()
 
void setFragment (PlanFragment fragment)
 
PlanFragment getFragment ()
 
List< ExprgetConjuncts ()
 
ExprSubstitutionMap getOutputSmap ()
 
void setOutputSmap (ExprSubstitutionMap smap)
 
Set< ExprIdgetAssignedConjuncts ()
 
void setAssignedConjuncts (Set< ExprId > conjuncts)
 
void setLimit (long limit)
 
void unsetLimit ()
 
ArrayList< TupleIdgetTupleIds ()
 
ArrayList< TupleIdgetTblRefIds ()
 
void setTblRefIds (ArrayList< TupleId > ids)
 
Set< TupleIdgetNullableTupleIds ()
 
void addConjuncts (List< Expr > conjuncts)
 
void transferConjuncts (PlanNode recipient)
 
String getExplainString ()
 
TPlan treeToThrift ()
 
boolean isBlockingNode ()
 
void computeCosts (TQueryOptions queryOptions)
 
long getInputCardinality ()
 

Static Public Member Functions

static long addCardinalities (long a, long b)
 
static long multiplyCardinalities (long a, long b)
 

Protected Member Functions

 UnionNode (PlanNodeId id, TupleId tupleId)
 
void toThrift (TPlanNode msg)
 
String getNodeExplainString (String prefix, String detailPrefix, TExplainLevel detailLevel)
 
final String getExplainString (String rootPrefix, String prefix, TExplainLevel detailLevel)
 
String getExplainString (List<?extends Expr > exprs)
 
void setDisplayName (String s)
 
final String getDisplayLabel ()
 
String getDisplayLabelDetail ()
 
String getOffsetExplainString (String prefix)
 
void assignConjuncts (Analyzer analyzer)
 
ExprSubstitutionMap getCombinedChildSmap ()
 
void createDefaultSmap (Analyzer analyzer)
 
long capAtLimit (long cardinality)
 
void markSlotsMaterialized (Analyzer analyzer, List< Expr > exprs)
 
void computeMemLayout (Analyzer analyzer)
 
double computeSelectivity ()
 
String debugString ()
 
boolean hasValidStats ()
 

Protected Attributes

List< List< Expr > > resultExprLists_ = Lists.newArrayList()
 
List< List< Expr > > constExprLists_ = Lists.newArrayList()
 
List< List< Expr > > materializedResultExprLists_ = Lists.newArrayList()
 
List< List< Expr > > materializedConstExprLists_ = Lists.newArrayList()
 
final TupleId tupleId_
 
String displayName_
 
PlanNodeId id_
 
long limit_
 
ArrayList< TupleIdtupleIds_
 
ArrayList< TupleIdtblRefIds_
 
Set< TupleIdnullableTupleIds_ = Sets.newHashSet()
 
List< Exprconjuncts_ = Lists.newArrayList()
 
PlanFragment fragment_
 
ExprSubstitutionMap outputSmap_
 
Set< ExprIdassignedConjuncts_
 
long cardinality_
 
int numNodes_
 
float avgRowSize_
 
long perHostMemCost_ = -1
 

Static Protected Attributes

static final int DEFAULT_BATCH_SIZE = 1024
 

Static Private Attributes

static final Logger LOG = LoggerFactory.getLogger(UnionNode.class)
 

Detailed Description

Node that merges the results of its child plans by materializing the corresponding result exprs into a new tuple.

Definition at line 43 of file UnionNode.java.

Constructor & Destructor Documentation

com.cloudera.impala.planner.UnionNode.UnionNode ( PlanNodeId  id,
TupleId  tupleId 
)
inlineprotected

Definition at line 62 of file UnionNode.java.

References com.cloudera.impala.planner.UnionNode.tupleId_.

Member Function Documentation

static long com.cloudera.impala.planner.PlanNode.addCardinalities ( long  a,
long  b 
)
inlinestaticinherited
void com.cloudera.impala.planner.UnionNode.addChild ( PlanNode  node,
List< Expr baseTblResultExprs 
)
inline

Add a child tree plus its corresponding resolved resultExprs.

Definition at line 72 of file UnionNode.java.

References com.cloudera.impala.planner.PlanNode.tupleIds_.

void com.cloudera.impala.planner.PlanNode.addConjuncts ( List< Expr conjuncts)
inlineinherited

Definition at line 209 of file PlanNode.java.

void com.cloudera.impala.planner.UnionNode.addConstExprList ( List< Expr exprs)
inline

Definition at line 67 of file UnionNode.java.

void com.cloudera.impala.planner.PlanNode.computeCosts ( TQueryOptions  queryOptions)
inlineinherited

Estimates the cost of executing this PlanNode. Currently only sets perHostMemCost_. May only be called after this PlanNode has been placed in a PlanFragment because the cost computation is dependent on the enclosing fragment's data partition.

Definition at line 562 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.perHostMemCost_.

void com.cloudera.impala.planner.UnionNode.computeStats ( Analyzer  analyzer)
inline
void com.cloudera.impala.planner.PlanNode.createDefaultSmap ( Analyzer  analyzer)
inlineprotectedinherited
String com.cloudera.impala.planner.PlanNode.debugString ( )
inlineprotectedinherited
Set<ExprId> com.cloudera.impala.planner.PlanNode.getAssignedConjuncts ( )
inlineinherited
ExprSubstitutionMap com.cloudera.impala.planner.PlanNode.getCombinedChildSmap ( )
inlineprotectedinherited
List<Expr> com.cloudera.impala.planner.PlanNode.getConjuncts ( )
inlineinherited
String com.cloudera.impala.planner.PlanNode.getDisplayLabelDetail ( )
inlineprotectedinherited

Subclasses can override to provide a node specific detail string that is displayed to the user. e.g. scan can return the table name.

Definition at line 234 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

final String com.cloudera.impala.planner.PlanNode.getExplainString ( String  rootPrefix,
String  prefix,
TExplainLevel  detailLevel 
)
inlineprotectedinherited
String com.cloudera.impala.planner.PlanNode.getExplainString ( List<?extends Expr exprs)
inlineprotectedinherited

Definition at line 506 of file PlanNode.java.

PlanFragment com.cloudera.impala.planner.PlanNode.getFragment ( )
inlineinherited
long com.cloudera.impala.planner.PlanNode.getInputCardinality ( )
inlineinherited

The input cardinality is the sum of output cardinalities of its children. For scan nodes the input cardinality is the expected number of rows scanned.

Definition at line 570 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.addCardinalities().

long com.cloudera.impala.planner.PlanNode.getLimit ( )
inlineinherited
String com.cloudera.impala.planner.UnionNode.getNodeExplainString ( String  prefix,
String  detailPrefix,
TExplainLevel  detailLevel 
)
inlineprotected
Set<TupleId> com.cloudera.impala.planner.PlanNode.getNullableTupleIds ( )
inlineinherited
String com.cloudera.impala.planner.PlanNode.getOffsetExplainString ( String  prefix)
inlineprotectedinherited

Return the offset_ details, if applicable. This is available separately from 'getNodeExplainString' because we want to output 'limit: ...' (which can be printed from PlanNode) before 'offset: ...', which is only printed from SortNodes right now.

Definition at line 336 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.getExplainString().

ExprSubstitutionMap com.cloudera.impala.planner.PlanNode.getOutputSmap ( )
inlineinherited

Definition at line 178 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.outputSmap_.

long com.cloudera.impala.planner.PlanNode.getPerHostMemCost ( )
inlineinherited
ArrayList<TupleId> com.cloudera.impala.planner.PlanNode.getTblRefIds ( )
inlineinherited
boolean com.cloudera.impala.planner.PlanNode.hasValidStats ( )
inlineprotectedinherited
void com.cloudera.impala.planner.UnionNode.init ( Analyzer  analyzer) throws InternalException
inline

Must be called after addChild()/addConstExprList(). Computes the materialized result/const expr lists based on the materialized slots of this UnionNode's produced tuple. The UnionNode doesn't need an smap: like a ScanNode, it materializes an original tuple. There is no need to call assignConjuncts() because all non-constant conjuncts have already been assigned to the union operands, and all constant conjuncts have been evaluated during registration to set analyzer.hasEmptyResultSet_.

Definition at line 160 of file UnionNode.java.

References com.cloudera.impala.planner.PlanNode.computeMemLayout(), com.cloudera.impala.planner.UnionNode.computeStats(), com.cloudera.impala.planner.UnionNode.constExprLists_, and com.cloudera.impala.planner.UnionNode.tupleId_.

boolean com.cloudera.impala.planner.PlanNode.isBlockingNode ( )
inlineinherited

Returns true if this plan node can output its first row only after consuming all rows of all its children. This method is used to group plan nodes into pipelined units for resource estimation.

Definition at line 555 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PipelinedPlanNodeSet.computePlanNodeSets().

void com.cloudera.impala.planner.PlanNode.markSlotsMaterialized ( Analyzer  analyzer,
List< Expr exprs 
)
inlineprotectedinherited

Marks all slots referenced in exprs as materialized.

Definition at line 464 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.HdfsScanNode.init().

static long com.cloudera.impala.planner.PlanNode.multiplyCardinalities ( long  a,
long  b 
)
inlinestaticinherited

Computes and returns the product of two cardinalities. If an overflow occurs, the maximum Long value is returned (Long.MAX_VALUE).

Definition at line 541 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeStats(), and com.cloudera.impala.planner.HashJoinNode.getJoinCardinality().

void com.cloudera.impala.planner.UnionNode.reorderOperands ( Analyzer  analyzer)
inline

Re-order the union's operands descending by their estimated per-host memory, such that parent nodes can gauge the peak memory consumption of this MergeNode after opening it during execution (a MergeNode opens its first operand in Open()). Scan nodes are always ordered last because they can dynamically scale down their memory usage, whereas many other nodes cannot (e.g., joins, aggregations). One goal is to decrease the likelihood of a SortNode parent claiming too much memory in its Open(), possibly causing the mem limit to be hit when subsequent union operands are executed. Can only be called on a fragmented plan because this function calls computeCosts() on this node's children. TODO: Come up with a good way of handing memory out to individual operators so that they don't trip each other up. Then remove this function.

Definition at line 115 of file UnionNode.java.

References com.cloudera.impala.planner.PlanNode.fragment_, and com.cloudera.impala.planner.UnionNode.resultExprLists_.

void com.cloudera.impala.planner.PlanNode.setAssignedConjuncts ( Set< ExprId conjuncts)
inlineinherited
void com.cloudera.impala.planner.PlanNode.setDisplayName ( String  s)
inlineprotectedinherited

Definition at line 223 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.displayName_.

void com.cloudera.impala.planner.PlanNode.setFragment ( PlanFragment  fragment)
inlineinherited

Definition at line 175 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.fragment_.

void com.cloudera.impala.planner.PlanNode.setId ( PlanNodeId  id)
inlineinherited

Definition at line 165 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.id_.

void com.cloudera.impala.planner.PlanNode.setLimit ( long  limit)
inlineinherited

Set the limit_ to the given limit_ only if the limit_ hasn't been set, or the new limit_ is lower.

Parameters
limit_

Definition at line 190 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.limit_.

void com.cloudera.impala.planner.PlanNode.setOutputSmap ( ExprSubstitutionMap  smap)
inlineinherited

Definition at line 179 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.outputSmap_.

void com.cloudera.impala.planner.PlanNode.setTblRefIds ( ArrayList< TupleId ids)
inlineinherited

Definition at line 202 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.tblRefIds_.

void com.cloudera.impala.planner.UnionNode.toThrift ( TPlanNode  msg)
inlineprotected
void com.cloudera.impala.planner.PlanNode.transferConjuncts ( PlanNode  recipient)
inlineinherited

Definition at line 214 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.conjuncts_.

TPlan com.cloudera.impala.planner.PlanNode.treeToThrift ( )
inlineinherited
void com.cloudera.impala.planner.PlanNode.unsetLimit ( )
inlineinherited

Definition at line 194 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.limit_.

Member Data Documentation

List<List<Expr> > com.cloudera.impala.planner.UnionNode.constExprLists_ = Lists.newArrayList()
protected
final int com.cloudera.impala.planner.PlanNode.DEFAULT_BATCH_SIZE = 1024
staticprotectedinherited

Definition at line 63 of file PlanNode.java.

PlanNodeId com.cloudera.impala.planner.PlanNode.id_
protectedinherited
final Logger com.cloudera.impala.planner.UnionNode.LOG = LoggerFactory.getLogger(UnionNode.class)
staticprivate

Definition at line 44 of file UnionNode.java.

List<List<Expr> > com.cloudera.impala.planner.UnionNode.materializedConstExprLists_ = Lists.newArrayList()
protected

Definition at line 58 of file UnionNode.java.

Referenced by com.cloudera.impala.planner.UnionNode.toThrift().

List<List<Expr> > com.cloudera.impala.planner.UnionNode.materializedResultExprLists_ = Lists.newArrayList()
protected

Definition at line 57 of file UnionNode.java.

Referenced by com.cloudera.impala.planner.UnionNode.toThrift().

Set<TupleId> com.cloudera.impala.planner.PlanNode.nullableTupleIds_ = Sets.newHashSet()
protectedinherited
List<List<Expr> > com.cloudera.impala.planner.UnionNode.resultExprLists_ = Lists.newArrayList()
protected
ArrayList<TupleId> com.cloudera.impala.planner.PlanNode.tblRefIds_
protectedinherited
final TupleId com.cloudera.impala.planner.UnionNode.tupleId_
protected

The documentation for this class was generated from the following file: