com.cloudera.impala.planner.DataSourceScanNode Class Reference

Inheritance diagram for com.cloudera.impala.planner.DataSourceScanNode:

Collaboration diagram for com.cloudera.impala.planner.DataSourceScanNode:

Public Member Functions
	DataSourceScanNode (PlanNodeId id, TupleDescriptor desc)
void	init (Analyzer analyzer) throws InternalException
void	computeStats (Analyzer analyzer)
void	computeCosts (TQueryOptions queryOptions)
TupleDescriptor	getTupleDesc ()
List< TScanRangeLocations >	getScanRangeLocations ()
boolean	isTableMissingStats ()
boolean	isTableMissingTableStats ()
boolean	isTableMissingColumnStats ()
long	getInputCardinality ()
PlanNodeId	getId ()
void	setId (PlanNodeId id)
long	getLimit ()
boolean	hasLimit ()
long	getPerHostMemCost ()
long	getCardinality ()
int	getNumNodes ()
float	getAvgRowSize ()
void	setFragment (PlanFragment fragment)
PlanFragment	getFragment ()
List< Expr >	getConjuncts ()
ExprSubstitutionMap	getOutputSmap ()
void	setOutputSmap (ExprSubstitutionMap smap)
Set< ExprId >	getAssignedConjuncts ()
void	setAssignedConjuncts (Set< ExprId > conjuncts)
void	setLimit (long limit)
void	unsetLimit ()
ArrayList< TupleId >	getTupleIds ()
ArrayList< TupleId >	getTblRefIds ()
void	setTblRefIds (ArrayList< TupleId > ids)
Set< TupleId >	getNullableTupleIds ()
void	addConjuncts (List< Expr > conjuncts)
void	transferConjuncts (PlanNode recipient)
String	getExplainString ()
TPlan	treeToThrift ()
boolean	isBlockingNode ()

Static Public Member Functions
static TColumnValue	literalToColumnValue (LiteralExpr expr)
static long	getPerHostMemUpperBound ()
static long	addCardinalities (long a, long b)
static long	multiplyCardinalities (long a, long b)

Protected Member Functions
String	debugString ()
void	toThrift (TPlanNode msg)
String	getNodeExplainString (String prefix, String detailPrefix, TExplainLevel detailLevel)
String	getStatsExplainString (String prefix, TExplainLevel detailLevel)
final String	getExplainString (String rootPrefix, String prefix, TExplainLevel detailLevel)
String	getExplainString (List<?extends Expr > exprs)
void	setDisplayName (String s)
final String	getDisplayLabel ()
String	getDisplayLabelDetail ()
String	getOffsetExplainString (String prefix)
void	assignConjuncts (Analyzer analyzer)
ExprSubstitutionMap	getCombinedChildSmap ()
void	createDefaultSmap (Analyzer analyzer)
long	capAtLimit (long cardinality)
void	markSlotsMaterialized (Analyzer analyzer, List< Expr > exprs)
void	computeMemLayout (Analyzer analyzer)
double	computeSelectivity ()
boolean	hasValidStats ()

Static Protected Member Functions
static TNetworkAddress	addressToTNetworkAddress (String address)

Protected Attributes
long	inputCardinality_ = -1
int	numPartitionsMissingStats_ = 0
List< TScanRangeLocations >	scanRanges_
String	displayName_
PlanNodeId	id_
long	limit_
ArrayList< TupleId >	tupleIds_
ArrayList< TupleId >	tblRefIds_
Set< TupleId >	nullableTupleIds_ = Sets.newHashSet()
List< Expr >	conjuncts_ = Lists.newArrayList()
PlanFragment	fragment_
ExprSubstitutionMap	outputSmap_
Set< ExprId >	assignedConjuncts_
long	cardinality_
int	numNodes_
float	avgRowSize_
long	perHostMemCost_ = -1

Static Protected Attributes
static final int	DEFAULT_BATCH_SIZE = 1024

Private Member Functions
void	prepareDataSource () throws InternalException
List< TBinaryPredicate >	getDisjuncts (Expr conjunct)
boolean	getDisjunctsHelper (Expr conjunct, List< TBinaryPredicate > predicates)
void	removeAcceptedConjuncts (List< Integer > acceptedPredicatesIdx, List< Integer > conjunctsIdx)
void	computeScanRangeLocations (Analyzer analyzer)

Private Attributes
final TupleDescriptor	desc_
final DataSourceTable	table_
List< List< TBinaryPredicate > >	acceptedPredicates_
List< Expr >	acceptedConjuncts_
long	numRowsEstimate_

Static Private Attributes
static final Logger	LOG = LoggerFactory.getLogger(DataSourceScanNode.class)

Detailed Description

Scan of a table provided by an external data source.

Definition at line 64 of file DataSourceScanNode.java.

Constructor & Destructor Documentation

com.cloudera.impala.planner.DataSourceScanNode.DataSourceScanNode ( PlanNodeId  id, TupleDescriptor  desc  )

inline

Definition at line 82 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.DataSourceScanNode.acceptedConjuncts_, com.cloudera.impala.planner.DataSourceScanNode.acceptedPredicates_, com.cloudera.impala.planner.DataSourceScanNode.desc_, com.cloudera.impala.analysis.TupleDescriptor.getTable(), and com.cloudera.impala.planner.DataSourceScanNode.table_.

Member Function Documentation

static long com.cloudera.impala.planner.PlanNode.addCardinalities ( long  a, long  b  )

inlinestaticinherited

Computes and returns the sum of two cardinalities. If an overflow occurs, the maximum Long value is returned (Long.MAX_VALUE).

Definition at line 528 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.ExchangeNode.computeStats(), com.cloudera.impala.planner.UnionNode.computeStats(), com.cloudera.impala.planner.HashJoinNode.computeStats(), com.cloudera.impala.planner.HdfsScanNode.computeStats(), and com.cloudera.impala.planner.PlanNode.getInputCardinality().

void com.cloudera.impala.planner.PlanNode.addConjuncts ( List< Expr >  conjuncts )

inlineinherited

Definition at line 209 of file PlanNode.java.

static TNetworkAddress com.cloudera.impala.planner.ScanNode.addressToTNetworkAddress ( String  address )

inlinestaticprotectedinherited

Helper function to parse a "host:port" address string into TNetworkAddress This is called with ipaddress:port when doing scan range assignment.

Definition at line 132 of file ScanNode.java.

Referenced by com.cloudera.impala.planner.HBaseScanNode.computeScanRangeLocations(), and com.cloudera.impala.planner.DataSourceScanNode.computeScanRangeLocations().

void com.cloudera.impala.planner.PlanNode.assignConjuncts ( Analyzer  analyzer )

inlineprotectedinherited

Assign remaining unassigned conjuncts.

Definition at line 401 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.init(), com.cloudera.impala.planner.SortNode.init(), com.cloudera.impala.planner.AggregationNode.init(), com.cloudera.impala.planner.HBaseScanNode.init(), com.cloudera.impala.planner.HdfsScanNode.init(), com.cloudera.impala.planner.HashJoinNode.init(), and com.cloudera.impala.planner.PlanNode.init().

long com.cloudera.impala.planner.PlanNode.capAtLimit ( long  cardinality )

inlineprotectedinherited

Definition at line 450 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.hasLimit(), and com.cloudera.impala.planner.PlanNode.limit_.

Referenced by com.cloudera.impala.planner.SortNode.computeStats(), com.cloudera.impala.planner.HBaseScanNode.computeStats(), com.cloudera.impala.planner.DataSourceScanNode.computeStats(), and com.cloudera.impala.planner.HdfsScanNode.computeStats().

void com.cloudera.impala.planner.DataSourceScanNode.computeCosts ( TQueryOptions  queryOptions )

inline

Definition at line 324 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.PlanNode.perHostMemCost_.

void com.cloudera.impala.planner.PlanNode.computeMemLayout ( Analyzer  analyzer )

inlineprotectedinherited

Call computeMemLayout() for all materialized tuples.

Definition at line 475 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.tupleIds_.

Referenced by com.cloudera.impala.planner.EmptySetNode.init(), com.cloudera.impala.planner.DataSourceScanNode.init(), com.cloudera.impala.planner.SortNode.init(), com.cloudera.impala.planner.AggregationNode.init(), com.cloudera.impala.planner.AnalyticEvalNode.init(), com.cloudera.impala.planner.HBaseScanNode.init(), com.cloudera.impala.planner.HdfsScanNode.init(), and com.cloudera.impala.planner.UnionNode.init().

void com.cloudera.impala.planner.DataSourceScanNode.computeScanRangeLocations ( Analyzer  analyzer )

inlineprivate

Create a single scan range for the localhost.

Definition at line 314 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.ScanNode.addressToTNetworkAddress(), and com.cloudera.impala.planner.ScanNode.scanRanges_.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.init().

double com.cloudera.impala.planner.PlanNode.computeSelectivity ( )

inlineprotectedinherited

Compute the product of the selectivies of all conjuncts.

Definition at line 484 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.conjuncts_.

Referenced by com.cloudera.impala.planner.SelectNode.computeStats(), com.cloudera.impala.planner.CrossJoinNode.computeStats(), com.cloudera.impala.planner.AggregationNode.computeStats(), com.cloudera.impala.planner.HBaseScanNode.computeStats(), com.cloudera.impala.planner.DataSourceScanNode.computeStats(), and com.cloudera.impala.planner.HdfsScanNode.computeStats().

void com.cloudera.impala.planner.DataSourceScanNode.computeStats ( Analyzer  analyzer )

inline

Definition at line 258 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.PlanNode.capAtLimit(), com.cloudera.impala.planner.PlanNode.cardinality_, com.cloudera.impala.planner.PlanNode.computeSelectivity(), com.cloudera.impala.planner.PlanNode.getNumNodes(), com.cloudera.impala.planner.ScanNode.inputCardinality_, com.cloudera.impala.planner.PlanNode.numNodes_, and com.cloudera.impala.planner.DataSourceScanNode.numRowsEstimate_.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.init().

void com.cloudera.impala.planner.PlanNode.createDefaultSmap ( Analyzer  analyzer )

inlineprotectedinherited

Sets outputSmap_ to compose(existing smap, combined child smap). Also substitutes conjuncts_ using the combined child smap.

Definition at line 425 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.conjuncts_, com.cloudera.impala.planner.PlanNode.getCombinedChildSmap(), and com.cloudera.impala.planner.PlanNode.outputSmap_.

Referenced by com.cloudera.impala.planner.SelectNode.init(), com.cloudera.impala.planner.AnalyticEvalNode.init(), com.cloudera.impala.planner.HashJoinNode.init(), and com.cloudera.impala.planner.PlanNode.init().

String com.cloudera.impala.planner.DataSourceScanNode.debugString ( )

inlineprotected

Definition at line 273 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.DataSourceScanNode.desc_, com.cloudera.impala.analysis.TupleDescriptor.getId(), com.cloudera.impala.catalog.DataSourceTable.getInitString(), and com.cloudera.impala.planner.DataSourceScanNode.table_.

Set<ExprId> com.cloudera.impala.planner.PlanNode.getAssignedConjuncts ( )

inlineinherited

Definition at line 180 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.assignedConjuncts_.

float com.cloudera.impala.planner.PlanNode.getAvgRowSize ( )

inlineinherited

Definition at line 174 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.avgRowSize_.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeCosts(), com.cloudera.impala.planner.HashJoinNode.computeCosts(), and com.cloudera.impala.planner.SingleNodePlanner.createCheapestJoinPlan().

long com.cloudera.impala.planner.PlanNode.getCardinality ( )

inlineinherited

Definition at line 172 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.cardinality_.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeCosts(), com.cloudera.impala.planner.HashJoinNode.computeCosts(), com.cloudera.impala.planner.AggregationNode.computeStats(), com.cloudera.impala.planner.SingleNodePlanner.createCheapestJoinPlan(), and com.cloudera.impala.planner.SingleNodePlanner.createJoinPlan().

ExprSubstitutionMap com.cloudera.impala.planner.PlanNode.getCombinedChildSmap ( )

inlineprotectedinherited

Returns an smap that combines the childrens' smaps.

Definition at line 410 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.createDefaultSmap(), com.cloudera.impala.planner.SortNode.init(), com.cloudera.impala.planner.AggregationNode.init(), com.cloudera.impala.planner.AnalyticEvalNode.init(), and com.cloudera.impala.planner.HashJoinNode.init().

List<Expr> com.cloudera.impala.planner.PlanNode.getConjuncts ( )

inlineinherited

Definition at line 177 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.conjuncts_.

Referenced by com.cloudera.impala.planner.ScanNode.getInputCardinality().

List<TBinaryPredicate> com.cloudera.impala.planner.DataSourceScanNode.getDisjuncts ( Expr  conjunct )

inlineprivate

Converts the conjunct to a list of TBinaryPredicates if it contains only disjunctive predicates of the form {slotref} {op} {constant} that can be represented by TBinaryPredicates. If the Expr cannot be converted, null is returned. TODO: Move this to Expr.

Definition at line 210 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.DataSourceScanNode.getDisjunctsHelper().

Referenced by com.cloudera.impala.planner.DataSourceScanNode.prepareDataSource().

boolean com.cloudera.impala.planner.DataSourceScanNode.getDisjunctsHelper ( Expr  conjunct, List< TBinaryPredicate >  predicates  )

inlineprivate

Definition at line 217 of file DataSourceScanNode.java.

References com.cloudera.impala.analysis.CompoundPredicate.getOp(), com.cloudera.impala.analysis.SlotRef.getResolvedPath(), com.cloudera.impala.analysis.Expr.getType(), and com.cloudera.impala.planner.DataSourceScanNode.literalToColumnValue().

Referenced by com.cloudera.impala.planner.DataSourceScanNode.getDisjuncts().

final String com.cloudera.impala.planner.PlanNode.getDisplayLabel ( )

inlineprotectedinherited

Definition at line 225 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.displayName_.

Referenced by com.cloudera.impala.planner.ExchangeNode.getNodeExplainString(), com.cloudera.impala.planner.AnalyticEvalNode.getNodeExplainString(), com.cloudera.impala.planner.AggregationNode.getNodeExplainString(), com.cloudera.impala.planner.HashJoinNode.getNodeExplainString(), com.cloudera.impala.planner.HdfsScanNode.getNodeExplainString(), and com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

String com.cloudera.impala.planner.PlanNode.getDisplayLabelDetail ( )

inlineprotectedinherited

Subclasses can override to provide a node specific detail string that is displayed to the user. e.g. scan can return the table name.

Definition at line 234 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

String com.cloudera.impala.planner.PlanNode.getExplainString ( )

inlineinherited

Definition at line 219 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.ExchangeNode.getDisplayLabelDetail(), com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.planner.SelectNode.getNodeExplainString(), com.cloudera.impala.planner.CrossJoinNode.getNodeExplainString(), com.cloudera.impala.planner.AnalyticEvalNode.getNodeExplainString(), com.cloudera.impala.planner.UnionNode.getNodeExplainString(), com.cloudera.impala.planner.AggregationNode.getNodeExplainString(), com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString(), com.cloudera.impala.planner.HBaseScanNode.getNodeExplainString(), com.cloudera.impala.planner.HashJoinNode.getNodeExplainString(), and com.cloudera.impala.planner.HdfsScanNode.getNodeExplainString().

final String com.cloudera.impala.planner.PlanNode.getExplainString ( String  rootPrefix, String  prefix, TExplainLevel  detailLevel  )

inlineprotectedinherited

Generate the explain plan tree. The plan will be in the form of:

root | |-—child 3 | limit:1 | |-—child 2 | limit:2 | child 1

The root node header line will be prefixed by rootPrefix and the remaining plan output will be prefixed by prefix.

Definition at line 252 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.avgRowSize_, com.cloudera.impala.planner.PlanNode.cardinality_, com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.planner.PlanNode.getNodeExplainString(), com.cloudera.impala.planner.PlanNode.getOffsetExplainString(), com.cloudera.impala.planner.PlanNode.limit_, com.cloudera.impala.planner.PlanNode.numNodes_, com.cloudera.impala.planner.PlanNode.perHostMemCost_, and com.cloudera.impala.planner.PlanNode.tupleIds_.

String com.cloudera.impala.planner.PlanNode.getExplainString ( List<?extends Expr >  exprs )

inlineprotectedinherited

Definition at line 506 of file PlanNode.java.

PlanFragment com.cloudera.impala.planner.PlanNode.getFragment ( )

inlineinherited

Definition at line 176 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.fragment_.

Referenced by com.cloudera.impala.planner.ExchangeNode.addChild(), and com.cloudera.impala.planner.PipelinedPlanNodeSet.computePlanNodeSets().

PlanNodeId com.cloudera.impala.planner.PlanNode.getId ( )

inlineinherited

Definition at line 164 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.id_.

Referenced by com.cloudera.impala.planner.DataStreamSink.getExplainString(), com.cloudera.impala.planner.AggregationNode.init(), and com.cloudera.impala.planner.DataStreamSink.toThrift().

long com.cloudera.impala.planner.ScanNode.getInputCardinality ( )

inlineinherited

Definition at line 141 of file ScanNode.java.

References com.cloudera.impala.planner.PlanNode.getConjuncts(), com.cloudera.impala.planner.PlanNode.getLimit(), com.cloudera.impala.planner.PlanNode.hasLimit(), and com.cloudera.impala.planner.ScanNode.inputCardinality_.

long com.cloudera.impala.planner.PlanNode.getLimit ( )

inlineinherited

Definition at line 169 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.limit_.

Referenced by com.cloudera.impala.planner.ScanNode.getInputCardinality().

String com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString ( String  prefix, String  detailPrefix, TExplainLevel  detailLevel  )

inlineprotected

Definition at line 339 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.DataSourceScanNode.acceptedConjuncts_, com.cloudera.impala.planner.PlanNode.conjuncts_, com.cloudera.impala.planner.DataSourceScanNode.desc_, com.cloudera.impala.planner.PlanNode.displayName_, com.cloudera.impala.analysis.TupleDescriptor.getAlias(), com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.catalog.Table.getFullName(), com.cloudera.impala.planner.ScanNode.getStatsExplainString(), and com.cloudera.impala.planner.DataSourceScanNode.table_.

Set<TupleId> com.cloudera.impala.planner.PlanNode.getNullableTupleIds ( )

inlineinherited

Definition at line 204 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.nullableTupleIds_.

int com.cloudera.impala.planner.PlanNode.getNumNodes ( )

inlineinherited

Definition at line 173 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.numNodes_.

Referenced by com.cloudera.impala.planner.PipelinedPlanNodeSet.computeResourceEstimates(), com.cloudera.impala.planner.HBaseScanNode.computeStats(), and com.cloudera.impala.planner.DataSourceScanNode.computeStats().

String com.cloudera.impala.planner.PlanNode.getOffsetExplainString ( String  prefix )

inlineprotectedinherited

Return the offset_ details, if applicable. This is available separately from 'getNodeExplainString' because we want to output 'limit: ...' (which can be printed from PlanNode) before 'offset: ...', which is only printed from SortNodes right now.

Definition at line 336 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.getExplainString().

ExprSubstitutionMap com.cloudera.impala.planner.PlanNode.getOutputSmap ( )

inlineinherited

Definition at line 178 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.outputSmap_.

long com.cloudera.impala.planner.PlanNode.getPerHostMemCost ( )

inlineinherited

Definition at line 171 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.perHostMemCost_.

Referenced by com.cloudera.impala.planner.PipelinedPlanNodeSet.computeResourceEstimates().

static long com.cloudera.impala.planner.DataSourceScanNode.getPerHostMemUpperBound ( )

inlinestatic

Returns the per-host upper bound of memory that any number of concurrent scan nodes will use. Used for estimating the per-host memory requirement of queries.

Definition at line 333 of file DataSourceScanNode.java.

List<TScanRangeLocations> com.cloudera.impala.planner.ScanNode.getScanRangeLocations ( )

inlineinherited

Returns all scan ranges plus their locations.

Definition at line 54 of file ScanNode.java.

References com.cloudera.impala.planner.ScanNode.scanRanges_.

String com.cloudera.impala.planner.ScanNode.getStatsExplainString ( String  prefix, TExplainLevel  detailLevel  )

inlineprotectedinherited

Returns the explain string for table and columns stats to be included into the a ScanNode's explain string. The given prefix is prepended to each of the lines. The prefix is used for proper formatting when the string returned by this method is embedded in a query's explain plan.

Definition at line 75 of file ScanNode.java.

References com.cloudera.impala.planner.ScanNode.desc_, com.cloudera.impala.analysis.TupleDescriptor.getSlots(), com.cloudera.impala.analysis.TupleDescriptor.getTable(), and com.cloudera.impala.planner.ScanNode.numPartitionsMissingStats_.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString(), com.cloudera.impala.planner.HBaseScanNode.getNodeExplainString(), and com.cloudera.impala.planner.HdfsScanNode.getNodeExplainString().

ArrayList<TupleId> com.cloudera.impala.planner.PlanNode.getTblRefIds ( )

inlineinherited

Definition at line 201 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.tblRefIds_.

Referenced by com.cloudera.impala.analysis.Analyzer.canEvalPredicate().

TupleDescriptor com.cloudera.impala.planner.ScanNode.getTupleDesc ( )

inlineinherited

Definition at line 49 of file ScanNode.java.

References com.cloudera.impala.planner.ScanNode.desc_.

ArrayList<TupleId> com.cloudera.impala.planner.PlanNode.getTupleIds ( )

inlineinherited

Definition at line 196 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.tupleIds_.

Referenced by com.cloudera.impala.planner.AnalyticEvalNode.AnalyticEvalNode(), com.cloudera.impala.planner.SingleNodePlanner.createInlineViewPlan(), com.cloudera.impala.planner.AnalyticPlanner.createSortInfo(), and com.cloudera.impala.planner.SelectNode.SelectNode().

boolean com.cloudera.impala.planner.PlanNode.hasLimit ( )

inlineinherited

Definition at line 170 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.limit_.

Referenced by com.cloudera.impala.planner.PlanNode.capAtLimit(), com.cloudera.impala.planner.ExchangeNode.computeStats(), com.cloudera.impala.planner.ScanNode.getInputCardinality(), and com.cloudera.impala.planner.SortNode.getNodeExplainDetail().

boolean com.cloudera.impala.planner.PlanNode.hasValidStats ( )

inlineprotectedinherited

Returns true if stats-related variables are valid.

Definition at line 519 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.cardinality_, and com.cloudera.impala.planner.PlanNode.numNodes_.

Referenced by com.cloudera.impala.planner.SortNode.computeCosts(), and com.cloudera.impala.planner.HashJoinNode.computeStats().

void com.cloudera.impala.planner.DataSourceScanNode.init ( Analyzer  analyzer ) throws InternalException

inline

Definition at line 91 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.PlanNode.assignConjuncts(), com.cloudera.impala.planner.PlanNode.computeMemLayout(), com.cloudera.impala.planner.DataSourceScanNode.computeScanRangeLocations(), com.cloudera.impala.planner.DataSourceScanNode.computeStats(), com.cloudera.impala.planner.PlanNode.conjuncts_, and com.cloudera.impala.planner.DataSourceScanNode.prepareDataSource().

boolean com.cloudera.impala.planner.PlanNode.isBlockingNode ( )

inlineinherited

Returns true if this plan node can output its first row only after consuming all rows of all its children. This method is used to group plan nodes into pipelined units for resource estimation.

Definition at line 555 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PipelinedPlanNodeSet.computePlanNodeSets().

boolean com.cloudera.impala.planner.ScanNode.isTableMissingColumnStats ( )

inlineinherited

Definition at line 120 of file ScanNode.java.

References com.cloudera.impala.planner.ScanNode.desc_, and com.cloudera.impala.analysis.TupleDescriptor.getSlots().

Referenced by com.cloudera.impala.planner.ScanNode.isTableMissingStats().

boolean com.cloudera.impala.planner.ScanNode.isTableMissingStats ( )

inlineinherited

Returns true if the table underlying this scan is missing table stats or column stats relevant to this scan node.

Definition at line 111 of file ScanNode.java.

References com.cloudera.impala.planner.ScanNode.isTableMissingColumnStats(), and com.cloudera.impala.planner.ScanNode.isTableMissingTableStats().

boolean com.cloudera.impala.planner.ScanNode.isTableMissingTableStats ( )

inlineinherited

Definition at line 115 of file ScanNode.java.

References com.cloudera.impala.planner.ScanNode.desc_, com.cloudera.impala.analysis.TupleDescriptor.getTable(), and com.cloudera.impala.planner.ScanNode.numPartitionsMissingStats_.

Referenced by com.cloudera.impala.planner.ScanNode.isTableMissingStats().

static TColumnValue com.cloudera.impala.planner.DataSourceScanNode.literalToColumnValue ( LiteralExpr  expr )

inlinestatic

Returns a thrift TColumnValue representing the literal from a binary predicate, or null if the type cannot be represented.

Definition at line 106 of file DataSourceScanNode.java.

References com.cloudera.impala.analysis.Expr.getType().

Referenced by com.cloudera.impala.planner.DataSourceScanNode.getDisjunctsHelper().

void com.cloudera.impala.planner.PlanNode.markSlotsMaterialized ( Analyzer  analyzer, List< Expr >  exprs  )

inlineprotectedinherited

Marks all slots referenced in exprs as materialized.

Definition at line 464 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.HdfsScanNode.init().

static long com.cloudera.impala.planner.PlanNode.multiplyCardinalities ( long  a, long  b  )

inlinestaticinherited

Computes and returns the product of two cardinalities. If an overflow occurs, the maximum Long value is returned (Long.MAX_VALUE).

Definition at line 541 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeStats(), and com.cloudera.impala.planner.HashJoinNode.getJoinCardinality().

void com.cloudera.impala.planner.DataSourceScanNode.prepareDataSource ( ) throws InternalException

inlineprivate

Calls prepare() on the data source to determine accepted predicates and get stats. The accepted predicates are moved from conjuncts_ into acceptedConjuncts_ and the associated TBinaryPredicates are set in acceptedPredicates_.

Definition at line 144 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.DataSourceScanNode.acceptedPredicates_, com.cloudera.impala.catalog.DataSource.debugString(), com.cloudera.impala.catalog.DataSourceTable.getDataSource(), com.cloudera.impala.planner.DataSourceScanNode.getDisjuncts(), com.cloudera.impala.planner.DataSourceScanNode.numRowsEstimate_, impala.OK, com.cloudera.impala.planner.DataSourceScanNode.removeAcceptedConjuncts(), and com.cloudera.impala.planner.DataSourceScanNode.table_.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.init().

void com.cloudera.impala.planner.DataSourceScanNode.removeAcceptedConjuncts ( List< Integer >  acceptedPredicatesIdx, List< Integer >  conjunctsIdx  )

inlineprivate

Removes the predicates from conjuncts_ that were accepted by the data source. Stores the accepted conjuncts in acceptedConjuncts_.

Definition at line 287 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.DataSourceScanNode.acceptedConjuncts_.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.prepareDataSource().

void com.cloudera.impala.planner.PlanNode.setAssignedConjuncts ( Set< ExprId >  conjuncts )

inlineinherited

Definition at line 181 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.assignedConjuncts_.

void com.cloudera.impala.planner.PlanNode.setDisplayName ( String  s )

inlineprotectedinherited

Definition at line 223 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.displayName_.

void com.cloudera.impala.planner.PlanNode.setFragment ( PlanFragment  fragment )

inlineinherited

Definition at line 175 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.fragment_.

void com.cloudera.impala.planner.PlanNode.setId ( PlanNodeId  id )

inlineinherited

Definition at line 165 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.id_.

void com.cloudera.impala.planner.PlanNode.setLimit ( long  limit )

inlineinherited

Set the limit_ to the given limit_ only if the limit_ hasn't been set, or the new limit_ is lower.

Parameters

limit_

Definition at line 190 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.limit_.

void com.cloudera.impala.planner.PlanNode.setOutputSmap ( ExprSubstitutionMap  smap )

inlineinherited

Definition at line 179 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.outputSmap_.

void com.cloudera.impala.planner.PlanNode.setTblRefIds ( ArrayList< TupleId >  ids )

inlineinherited

Definition at line 202 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.tblRefIds_.

void com.cloudera.impala.planner.DataSourceScanNode.toThrift ( TPlanNode  msg )

inlineprotected

Definition at line 304 of file DataSourceScanNode.java.

References com.cloudera.impala.planner.DataSourceScanNode.acceptedPredicates_, com.cloudera.impala.planner.DataSourceScanNode.desc_, com.cloudera.impala.catalog.DataSourceTable.getDataSource(), com.cloudera.impala.analysis.TupleDescriptor.getId(), and com.cloudera.impala.planner.DataSourceScanNode.table_.

void com.cloudera.impala.planner.PlanNode.transferConjuncts ( PlanNode  recipient )

inlineinherited

Definition at line 214 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.conjuncts_.

TPlan com.cloudera.impala.planner.PlanNode.treeToThrift ( )

inlineinherited

Definition at line 341 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

void com.cloudera.impala.planner.PlanNode.unsetLimit ( )

inlineinherited

Definition at line 194 of file PlanNode.java.

References com.cloudera.impala.planner.PlanNode.limit_.

Member Data Documentation

List<Expr> com.cloudera.impala.planner.DataSourceScanNode.acceptedConjuncts_

private

Definition at line 77 of file DataSourceScanNode.java.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.DataSourceScanNode(), com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString(), and com.cloudera.impala.planner.DataSourceScanNode.removeAcceptedConjuncts().

List<List<TBinaryPredicate> > com.cloudera.impala.planner.DataSourceScanNode.acceptedPredicates_

private

Definition at line 72 of file DataSourceScanNode.java.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.DataSourceScanNode(), com.cloudera.impala.planner.DataSourceScanNode.prepareDataSource(), and com.cloudera.impala.planner.DataSourceScanNode.toThrift().

Set<ExprId> com.cloudera.impala.planner.PlanNode.assignedConjuncts_

protectedinherited

Definition at line 99 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.getAssignedConjuncts(), com.cloudera.impala.planner.CrossJoinNode.init(), com.cloudera.impala.planner.HdfsScanNode.init(), com.cloudera.impala.planner.HashJoinNode.init(), and com.cloudera.impala.planner.PlanNode.setAssignedConjuncts().

float com.cloudera.impala.planner.PlanNode.avgRowSize_

protectedinherited

Definition at line 110 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeCosts(), com.cloudera.impala.planner.SortNode.computeCosts(), com.cloudera.impala.planner.AggregationNode.computeCosts(), com.cloudera.impala.planner.HashJoinNode.computeCosts(), com.cloudera.impala.planner.EmptySetNode.computeStats(), com.cloudera.impala.planner.ExchangeNode.computeStats(), com.cloudera.impala.planner.PlanNode.computeStats(), com.cloudera.impala.planner.PlanNode.getAvgRowSize(), and com.cloudera.impala.planner.PlanNode.getExplainString().

long com.cloudera.impala.planner.PlanNode.cardinality_

protectedinherited

Definition at line 103 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeCosts(), com.cloudera.impala.planner.SortNode.computeCosts(), com.cloudera.impala.planner.AggregationNode.computeCosts(), com.cloudera.impala.planner.HashJoinNode.computeCosts(), com.cloudera.impala.planner.EmptySetNode.computeStats(), com.cloudera.impala.planner.SelectNode.computeStats(), com.cloudera.impala.planner.CrossJoinNode.computeStats(), com.cloudera.impala.planner.ExchangeNode.computeStats(), com.cloudera.impala.planner.UnionNode.computeStats(), com.cloudera.impala.planner.SortNode.computeStats(), com.cloudera.impala.planner.AnalyticEvalNode.computeStats(), com.cloudera.impala.planner.AggregationNode.computeStats(), com.cloudera.impala.planner.HBaseScanNode.computeStats(), com.cloudera.impala.planner.DataSourceScanNode.computeStats(), com.cloudera.impala.planner.HashJoinNode.computeStats(), com.cloudera.impala.planner.HdfsScanNode.computeStats(), com.cloudera.impala.planner.PlanNode.getCardinality(), com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.planner.HashJoinNode.getJoinCardinality(), com.cloudera.impala.planner.HashJoinNode.getSemiJoinCardinality(), com.cloudera.impala.planner.PlanNode.hasValidStats(), com.cloudera.impala.planner.PlanNode.PlanNode(), and com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

List<Expr> com.cloudera.impala.planner.PlanNode.conjuncts_ = Lists.newArrayList()

protectedinherited

Definition at line 86 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.computeSelectivity(), com.cloudera.impala.planner.PlanNode.createDefaultSmap(), com.cloudera.impala.planner.HBaseScanNode.createHBaseFilters(), com.cloudera.impala.planner.PlanNode.debugString(), com.cloudera.impala.planner.PlanNode.getConjuncts(), com.cloudera.impala.planner.SelectNode.getNodeExplainString(), com.cloudera.impala.planner.CrossJoinNode.getNodeExplainString(), com.cloudera.impala.planner.AnalyticEvalNode.getNodeExplainString(), com.cloudera.impala.planner.UnionNode.getNodeExplainString(), com.cloudera.impala.planner.AggregationNode.getNodeExplainString(), com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString(), com.cloudera.impala.planner.HBaseScanNode.getNodeExplainString(), com.cloudera.impala.planner.HashJoinNode.getNodeExplainString(), com.cloudera.impala.planner.HdfsScanNode.getNodeExplainString(), com.cloudera.impala.planner.SelectNode.init(), com.cloudera.impala.planner.DataSourceScanNode.init(), com.cloudera.impala.planner.AggregationNode.init(), com.cloudera.impala.planner.HBaseScanNode.init(), com.cloudera.impala.planner.HdfsScanNode.init(), com.cloudera.impala.planner.PlanNode.PlanNode(), com.cloudera.impala.planner.PlanNode.transferConjuncts(), and com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

final int com.cloudera.impala.planner.PlanNode.DEFAULT_BATCH_SIZE = 1024

staticprotectedinherited

Definition at line 63 of file PlanNode.java.

final TupleDescriptor com.cloudera.impala.planner.DataSourceScanNode.desc_

private

Definition at line 66 of file DataSourceScanNode.java.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.DataSourceScanNode(), com.cloudera.impala.planner.DataSourceScanNode.debugString(), com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString(), and com.cloudera.impala.planner.DataSourceScanNode.toThrift().

String com.cloudera.impala.planner.PlanNode.displayName_

protectedinherited

Definition at line 66 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.getDisplayLabel(), com.cloudera.impala.planner.EmptySetNode.getNodeExplainString(), com.cloudera.impala.planner.SelectNode.getNodeExplainString(), com.cloudera.impala.planner.CrossJoinNode.getNodeExplainString(), com.cloudera.impala.planner.SortNode.getNodeExplainString(), com.cloudera.impala.planner.UnionNode.getNodeExplainString(), com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString(), com.cloudera.impala.planner.HBaseScanNode.getNodeExplainString(), com.cloudera.impala.planner.PlanNode.PlanNode(), com.cloudera.impala.planner.PlanNode.setDisplayName(), and com.cloudera.impala.planner.ExchangeNode.setMergeInfo().

PlanFragment com.cloudera.impala.planner.PlanNode.fragment_

protectedinherited

Definition at line 90 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.AnalyticEvalNode.computeCosts(), com.cloudera.impala.planner.AggregationNode.computeCosts(), com.cloudera.impala.planner.ExchangeNode.getDisplayLabelDetail(), com.cloudera.impala.planner.PlanNode.getFragment(), com.cloudera.impala.planner.HdfsScanNode.getNodeExplainString(), com.cloudera.impala.planner.UnionNode.reorderOperands(), and com.cloudera.impala.planner.PlanNode.setFragment().

PlanNodeId com.cloudera.impala.planner.PlanNode.id_

protectedinherited

Definition at line 69 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.getId(), com.cloudera.impala.planner.PlanNode.PlanNode(), and com.cloudera.impala.planner.PlanNode.setId().

long com.cloudera.impala.planner.ScanNode.inputCardinality_ = -1

protectedinherited

Definition at line 36 of file ScanNode.java.

Referenced by com.cloudera.impala.planner.HBaseScanNode.computeStats(), com.cloudera.impala.planner.DataSourceScanNode.computeStats(), com.cloudera.impala.planner.HdfsScanNode.computeStats(), and com.cloudera.impala.planner.ScanNode.getInputCardinality().

long com.cloudera.impala.planner.PlanNode.limit_

protectedinherited

Definition at line 71 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.ExchangeNode.addChild(), com.cloudera.impala.planner.PlanNode.capAtLimit(), com.cloudera.impala.planner.ExchangeNode.computeStats(), com.cloudera.impala.planner.PlanNode.debugString(), com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.planner.PlanNode.getLimit(), com.cloudera.impala.planner.SortNode.getNodeExplainDetail(), com.cloudera.impala.planner.PlanNode.hasLimit(), com.cloudera.impala.planner.PlanNode.PlanNode(), com.cloudera.impala.planner.PlanNode.setLimit(), com.cloudera.impala.planner.PlanNode.treeToThriftHelper(), and com.cloudera.impala.planner.PlanNode.unsetLimit().

final Logger com.cloudera.impala.planner.DataSourceScanNode.LOG = LoggerFactory.getLogger(DataSourceScanNode.class)

staticprivate

Definition at line 65 of file DataSourceScanNode.java.

Set<TupleId> com.cloudera.impala.planner.PlanNode.nullableTupleIds_ = Sets.newHashSet()

protectedinherited

Definition at line 84 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.ExchangeNode.addChild(), com.cloudera.impala.planner.PlanNode.getNullableTupleIds(), and com.cloudera.impala.planner.PlanNode.PlanNode().

int com.cloudera.impala.planner.PlanNode.numNodes_

protectedinherited

Definition at line 107 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeCosts(), com.cloudera.impala.planner.HashJoinNode.computeCosts(), com.cloudera.impala.planner.HdfsScanNode.computeCosts(), com.cloudera.impala.planner.EmptySetNode.computeStats(), com.cloudera.impala.planner.ExchangeNode.computeStats(), com.cloudera.impala.planner.UnionNode.computeStats(), com.cloudera.impala.planner.HBaseScanNode.computeStats(), com.cloudera.impala.planner.DataSourceScanNode.computeStats(), com.cloudera.impala.planner.PlanNode.computeStats(), com.cloudera.impala.planner.HdfsScanNode.computeStats(), com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.planner.PlanNode.getNumNodes(), com.cloudera.impala.planner.PlanNode.hasValidStats(), and com.cloudera.impala.planner.PlanNode.PlanNode().

int com.cloudera.impala.planner.ScanNode.numPartitionsMissingStats_ = 0

protectedinherited

Definition at line 39 of file ScanNode.java.

Referenced by com.cloudera.impala.planner.HdfsScanNode.computeStats(), com.cloudera.impala.planner.ScanNode.getStatsExplainString(), and com.cloudera.impala.planner.ScanNode.isTableMissingTableStats().

long com.cloudera.impala.planner.DataSourceScanNode.numRowsEstimate_

private

Definition at line 80 of file DataSourceScanNode.java.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.computeStats(), and com.cloudera.impala.planner.DataSourceScanNode.prepareDataSource().

ExprSubstitutionMap com.cloudera.impala.planner.PlanNode.outputSmap_

protectedinherited

Definition at line 93 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.createDefaultSmap(), com.cloudera.impala.planner.PlanNode.getOutputSmap(), com.cloudera.impala.planner.SortNode.init(), com.cloudera.impala.planner.AggregationNode.init(), com.cloudera.impala.planner.AnalyticEvalNode.init(), and com.cloudera.impala.planner.PlanNode.setOutputSmap().

long com.cloudera.impala.planner.PlanNode.perHostMemCost_ = -1

protectedinherited

Definition at line 114 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.CrossJoinNode.computeCosts(), com.cloudera.impala.planner.SortNode.computeCosts(), com.cloudera.impala.planner.AnalyticEvalNode.computeCosts(), com.cloudera.impala.planner.AggregationNode.computeCosts(), com.cloudera.impala.planner.DataSourceScanNode.computeCosts(), com.cloudera.impala.planner.HashJoinNode.computeCosts(), com.cloudera.impala.planner.HBaseScanNode.computeCosts(), com.cloudera.impala.planner.PlanNode.computeCosts(), com.cloudera.impala.planner.HdfsScanNode.computeCosts(), com.cloudera.impala.planner.EmptySetNode.computeStats(), com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.planner.PlanNode.getPerHostMemCost(), and com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

List<TScanRangeLocations> com.cloudera.impala.planner.ScanNode.scanRanges_

protectedinherited

Definition at line 42 of file ScanNode.java.

Referenced by com.cloudera.impala.planner.HdfsScanNode.computeCosts(), com.cloudera.impala.planner.HdfsScanNode.computeScanRangeLocations(), com.cloudera.impala.planner.HBaseScanNode.computeScanRangeLocations(), com.cloudera.impala.planner.DataSourceScanNode.computeScanRangeLocations(), and com.cloudera.impala.planner.ScanNode.getScanRangeLocations().

final DataSourceTable com.cloudera.impala.planner.DataSourceScanNode.table_

private

Definition at line 67 of file DataSourceScanNode.java.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.DataSourceScanNode(), com.cloudera.impala.planner.DataSourceScanNode.debugString(), com.cloudera.impala.planner.DataSourceScanNode.getNodeExplainString(), com.cloudera.impala.planner.DataSourceScanNode.prepareDataSource(), and com.cloudera.impala.planner.DataSourceScanNode.toThrift().

ArrayList<TupleId> com.cloudera.impala.planner.PlanNode.tblRefIds_

protectedinherited

Definition at line 79 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.PlanNode.getTblRefIds(), com.cloudera.impala.planner.PlanNode.PlanNode(), and com.cloudera.impala.planner.PlanNode.setTblRefIds().

ArrayList<TupleId> com.cloudera.impala.planner.PlanNode.tupleIds_

protectedinherited

Definition at line 74 of file PlanNode.java.

Referenced by com.cloudera.impala.planner.ExchangeNode.addChild(), com.cloudera.impala.planner.UnionNode.addChild(), com.cloudera.impala.planner.PlanNode.computeMemLayout(), com.cloudera.impala.planner.PlanNode.computeStats(), com.cloudera.impala.planner.PlanNode.getExplainString(), com.cloudera.impala.planner.PlanNode.getTupleIds(), com.cloudera.impala.planner.EmptySetNode.init(), com.cloudera.impala.planner.AggregationNode.init(), com.cloudera.impala.planner.PlanNode.PlanNode(), com.cloudera.impala.planner.HdfsScanNode.prunePartitions(), com.cloudera.impala.planner.ExchangeNode.toThrift(), and com.cloudera.impala.planner.PlanNode.treeToThriftHelper().

---

The documentation for this class was generated from the following file:

• fe/src/main/java/com/cloudera/impala/planner/DataSourceScanNode.java