Inheritance diagram for com.cloudera.impala.catalog.HBaseTable:

Collaboration diagram for com.cloudera.impala.catalog.HBaseTable:

Public Member Functions
void	load (Table oldValue, HiveMetaStoreClient client, org.apache.hadoop.hive.metastore.api.Table msTbl) throws TableLoadingException

synchronized Pair< Long, Long >	getEstimatedRowStats (byte[] startRowKey, byte[] endRowKey)

long	getHdfsSize (HRegionInfo info) throws IOException

ArrayList< Column >	getColumnsInHiveOrder ()

TTableDescriptor	toThriftDescriptor (Set< Long > referencedPartitions)

String	getHBaseTableName ()

HTable	getHTable ()

int	getNumNodes ()

TCatalogObjectType	getCatalogObjectType ()

TTable	toThrift ()

String	getStorageHandlerClassName ()

TResultSet	getTableStats ()

void	addColumn (Column col)

void	clearColumns ()

void	updateLastDdlTime (long ddlTime)

void	validate () throws TableLoadingException

TCatalogObject	toTCatalogObject ()

Db	getDb ()

String	getName ()

String	getFullName ()

TableName	getTableName ()

String	getOwner ()

ArrayList< Column >	getColumns ()

List< String >	getColumnNames ()

List< Column >	getNonClusteringColumns ()

Column	getColumn (String name)

org.apache.hadoop.hive.metastore.api.Table	getMetaStoreTable ()

int	getNumClusteringCols ()

TableId	getId ()

long	getNumRows ()

ArrayType	getType ()

long	getCatalogVersion ()

void	setCatalogVersion (long catalogVersion)

boolean	isLoaded ()

Static Public Member Functions
static Path	getRootDir (final Configuration c) throws IOException

static Configuration	getHBaseConf ()

static List< HRegionLocation >	getRegionsInRange (HTable hbaseTbl, final byte[] startKey, final byte[] endKey) throws IOException

static boolean	isHBaseTable (org.apache.hadoop.hive.metastore.api.Table msTbl)

static Table	fromMetastoreTable (TableId id, Db db, org.apache.hadoop.hive.metastore.api.Table msTbl)

static Table	fromThrift (Db parentDb, TTable thriftTable) throws TableLoadingException

Static Public Attributes
static final String	DEFAULT_PREFIX = "default."

static final int	ROW_COUNT_ESTIMATE_BATCH_SIZE = 10

Protected Member Functions
	HBaseTable (TableId id, org.apache.hadoop.hive.metastore.api.Table msTbl, Db db, String name, String owner)

void	loadFromThrift (TTable table) throws TableLoadingException

List< String >	getColumnNamesWithHmsStats ()

void	loadAllColumnStats (HiveMetaStoreClient client)

Type	parseColumnType (FieldSchema fs) throws TableLoadingException

Static Protected Member Functions
static long	getRowCount (Map< String, String > parameters)

Protected Attributes
HBaseColumn	rowKey_

String	hbaseTableName_

final org.apache.hadoop.hive.metastore.api.Table	msTable_

final TableId	id_

final Db	db_

final String	name_

final String	owner_

TTableDescriptor	tableDesc_

List< FieldSchema >	fields_

TAccessLevel	accessLevel_ = TAccessLevel.READ_WRITE

int	numClusteringCols_

long	numRows_ = -1

final ArrayType	type_ = new ArrayType(new StructType())

long	lastDdlTime_

Static Protected Attributes
static EnumSet< TableType >	SUPPORTED_TABLE_TYPES

Private Member Functions
void	parseColumnMapping (boolean tableDefaultStorageIsBinary, String columnsMappingSpec, List< FieldSchema > fieldSchemas, List< String > columnFamilies, List< String > columnQualifiers, List< Boolean > colIsBinaryEncoded) throws SerDeException

boolean	supportsBinaryEncoding (FieldSchema fs)

String	getHBaseTableName (org.apache.hadoop.hive.metastore.api.Table tbl)

Pair< Long, Long >	getEstimatedRowStatsForRegion (HRegionLocation location, boolean isCompressed) throws IOException

THBaseTable	getTHBaseTable ()

Private Attributes
HTable	hTable_ = null

HColumnDescriptor[]	columnFamilies_ = null

Static Private Attributes
static final double	DELTA_FROM_AVERAGE = 0.15

static final Logger	LOG = Logger.getLogger(HBaseTable.class)

static final int	MIN_NUM_REGIONS_TO_CHECK = 5

static final String	HBASE_INPUT_FORMAT

static final String	HBASE_SERIALIZATION_LIB

static final String	HBASE_STORAGE_HANDLER

static final String	ROW_KEY_COLUMN_FAMILY = ":key"

static final Configuration	hbaseConf_ = HBaseConfiguration.create()

Detailed Description

Impala representation of HBase table metadata, as loaded from Hive's metastore. This implies that we inherit the metastore's limitations related to HBase, for example the lack of support for composite HBase row keys. We sort the HBase columns (cols) by family/qualifier to simplify the retrieval logic in the backend, since HBase returns data ordered by family/qualifier. This implies that a "select *"-query on an HBase table will not have the columns ordered as they were declared in the DDL. They will be ordered by family/qualifier.

Definition at line 76 of file HBaseTable.java.

Constructor & Destructor Documentation

com.cloudera.impala.catalog.HBaseTable.HBaseTable	(	TableId	id,
		org.apache.hadoop.hive.metastore.api.Table	msTbl,
		Db	db,
		String	name,
		String	owner
	)

inlineprotected

Definition at line 123 of file HBaseTable.java.

Member Function Documentation

void com.cloudera.impala.catalog.Table.addColumn ( Column col )

inlineinherited

Definition at line 114 of file Table.java.

References com.cloudera.impala.catalog.ArrayType.getItemType(), and com.cloudera.impala.catalog.Table.type_.

Referenced by com.cloudera.impala.catalog.View.load(), com.cloudera.impala.catalog.HBaseTable.load(), com.cloudera.impala.catalog.DataSourceTable.loadColumns(), and com.cloudera.impala.catalog.HdfsTable.loadColumns().

void com.cloudera.impala.catalog.Table.clearColumns ( )

inlineinherited

Definition at line 121 of file Table.java.

References com.cloudera.impala.catalog.ArrayType.getItemType(), and com.cloudera.impala.catalog.Table.type_.

Referenced by com.cloudera.impala.catalog.HBaseTable.load().

static Table com.cloudera.impala.catalog.Table.fromMetastoreTable	(	TableId	id,
		Db	db,
		org.apache.hadoop.hive.metastore.api.Table	msTbl
	)

inlinestaticinherited

Creates a table of the appropriate type based on the given hive.metastore.api.Table object.

Definition at line 207 of file Table.java.

References com.cloudera.impala.catalog.DataSourceTable.isDataSourceTable(), com.cloudera.impala.catalog.HBaseTable.isHBaseTable(), and com.cloudera.impala.catalog.HdfsFileFormat.isHdfsFormatClass().

static Table com.cloudera.impala.catalog.Table.fromThrift	(	Db	parentDb,
		TTable	thriftTable
	)		throws TableLoadingException

inlinestaticinherited

Factory method that creates a new Table from its Thrift representation. Determines the type of table to create based on the Thrift table provided.

Definition at line 231 of file Table.java.

TCatalogObjectType com.cloudera.impala.catalog.HBaseTable.getCatalogObjectType ( )

inline

Implements com.cloudera.impala.catalog.CatalogObject.

Definition at line 605 of file HBaseTable.java.

long com.cloudera.impala.catalog.Table.getCatalogVersion ( )

inlineinherited

Implements com.cloudera.impala.catalog.CatalogObject.

Definition at line 408 of file Table.java.

References com.cloudera.impala.catalog.Table.catalogVersion_.

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.dropPartition(), com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), com.cloudera.impala.catalog.ImpaladCatalog.removeTable(), com.cloudera.impala.catalog.CatalogServiceCatalog.replaceTableIfUnchanged(), and com.cloudera.impala.catalog.Table.toTCatalogObject().

Column com.cloudera.impala.catalog.Table.getColumn ( String name )

inlineinherited

Case-insensitive lookup.

Definition at line 392 of file Table.java.

Referenced by com.cloudera.impala.catalog.Table.loadAllColumnStats(), and com.cloudera.impala.catalog.CatalogTest.testColStatsColTypeMismatch().

List<String> com.cloudera.impala.catalog.Table.getColumnNames ( )

inlineinherited

Returns a list of the column names ordered by position.

Definition at line 354 of file Table.java.

References com.cloudera.impala.catalog.Table.colsByPos_.

Referenced by com.cloudera.impala.analysis.ColumnLineageGraph.addTargetColumnLabels(), com.cloudera.impala.catalog.HdfsTable.getTHdfsTable(), com.cloudera.impala.catalog.DataSourceTable.toThriftDescriptor(), com.cloudera.impala.catalog.HBaseTable.toThriftDescriptor(), and com.cloudera.impala.catalog.HdfsTable.toThriftDescriptor().

List<String> com.cloudera.impala.catalog.Table.getColumnNamesWithHmsStats ( )

inlineprotectedinherited

Definition at line 144 of file Table.java.

References com.cloudera.impala.catalog.Table.colsByName_, and impala.name.

Referenced by com.cloudera.impala.catalog.Table.loadAllColumnStats().

ArrayList<Column> com.cloudera.impala.catalog.Table.getColumns ( )

inlineinherited

Definition at line 349 of file Table.java.

References com.cloudera.impala.catalog.Table.colsByPos_.

ArrayList<Column> com.cloudera.impala.catalog.HBaseTable.getColumnsInHiveOrder ( )

inline

Hive returns the columns in order of their declaration for HBase tables.

Definition at line 572 of file HBaseTable.java.

References com.cloudera.impala.catalog.Table.getColumns().

Db com.cloudera.impala.catalog.Table.getDb ( )

inlineinherited

Definition at line 341 of file Table.java.

References com.cloudera.impala.catalog.Table.db_.

Referenced by com.cloudera.impala.service.CatalogOpExecutor.alterTableUpdateStats(), com.cloudera.impala.service.CatalogOpExecutor.createHiveColStats(), com.cloudera.impala.analysis.Analyzer.getTable(), com.cloudera.impala.analysis.InsertStmt.setTargetTable(), com.cloudera.impala.analysis.ShowStatsStmt.toThrift(), com.cloudera.impala.analysis.ComputeStatsStmt.toThrift(), and com.cloudera.impala.service.CatalogOpExecutor.updateCatalog().

synchronized Pair<Long, Long> com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats	(	byte[]	startRowKey,
		byte[]	endRowKey
	)

inline

Get an estimate of the number of rows and bytes per row in regions between startRowKey and endRowKey.

This number is calculated by incrementally checking as many region servers as necessary until we observe a relatively constant row size per region on average. Depending on the skew of data in the regions this can either mean that we need to check only a minimal number of regions or that we will scan all regions.

The accuracy of this number is determined by the number of rows that are written and kept in the memstore and have not been flushed until now. A large number of key-value pairs in the memstore will lead to bad estimates as this number is not reflected in the file size on HDFS that is used to estimate this number.

Currently, the algorithm does not consider the case that the key range used as a parameter might be generally of different size than the rest of the region.

The values computed here should be cached so that in high qps workloads the nn is not overwhelmed. Could be done in load(); Synchronized to make sure that only one thread at a time is using the htable.

Parameters

startRowKey	First row key in the range
endRowKey	Last row key in the range

Returns: The estimated number of rows in the regions between the row keys (first) and the estimated row size in bytes (second).

Definition at line 488 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.columnFamilies_, com.cloudera.impala.catalog.HBaseTable.DELTA_FROM_AVERAGE, com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStatsForRegion(), com.cloudera.impala.catalog.HBaseTable.getHdfsSize(), com.cloudera.impala.catalog.HBaseTable.getRegionsInRange(), com.cloudera.impala.catalog.HBaseTable.hTable_, and com.cloudera.impala.catalog.HBaseTable.MIN_NUM_REGIONS_TO_CHECK.

Pair<Long, Long> com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStatsForRegion	(	HRegionLocation	location,
		boolean	isCompressed
	)		throws IOException

inlineprivate

Estimates the number of rows for a single region and returns a pair with the estimated row count and the estimated size in bytes per row.

Definition at line 399 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.getHdfsSize(), and com.cloudera.impala.catalog.HBaseTable.ROW_COUNT_ESTIMATE_BATCH_SIZE.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats(), and com.cloudera.impala.catalog.HBaseTable.getTableStats().

String com.cloudera.impala.catalog.Table.getFullName ( )

inlineinherited

Definition at line 343 of file Table.java.

References com.cloudera.impala.catalog.Table.db_, com.cloudera.impala.catalog.Db.getName(), and com.cloudera.impala.catalog.Table.name_.

static Configuration com.cloudera.impala.catalog.HBaseTable.getHBaseConf ( )

inlinestatic

Definition at line 594 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.hbaseConf_.

String com.cloudera.impala.catalog.HBaseTable.getHBaseTableName ( org.apache.hadoop.hive.metastore.api.Table tbl )

inlineprivate

Definition at line 377 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.DEFAULT_PREFIX.

Referenced by com.cloudera.impala.catalog.CatalogTest.checkHBaseTableCols(), com.cloudera.impala.planner.HBaseScanNode.computeScanRangeLocations(), and com.cloudera.impala.planner.HBaseScanNode.toThrift().

String com.cloudera.impala.catalog.HBaseTable.getHBaseTableName ( )

inline

Definition at line 586 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.hbaseTableName_.

Referenced by com.cloudera.impala.catalog.HBaseTable.load(), and com.cloudera.impala.catalog.HBaseTable.loadFromThrift().

long com.cloudera.impala.catalog.HBaseTable.getHdfsSize ( HRegionInfo info ) throws IOException

inline

Returns the Hdfs size of the given region in bytes. NULL can be passed as a parameter to retrieve the size of the complete table.

Definition at line 545 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.getRootDir(), com.cloudera.impala.catalog.HBaseTable.hbaseConf_, and com.cloudera.impala.catalog.HBaseTable.hbaseTableName_.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats(), com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStatsForRegion(), and com.cloudera.impala.catalog.HBaseTable.getTableStats().

HTable com.cloudera.impala.catalog.HBaseTable.getHTable ( )

inline

Definition at line 590 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.hTable_.

TableId com.cloudera.impala.catalog.Table.getId ( )

inlineinherited

Definition at line 403 of file Table.java.

References com.cloudera.impala.catalog.Table.id_.

Referenced by com.cloudera.impala.planner.HBaseTableSink.toThrift(), and com.cloudera.impala.planner.HdfsTableSink.toThrift().

org.apache.hadoop.hive.metastore.api.Table com.cloudera.impala.catalog.Table.getMetaStoreTable ( )

inlineinherited

Returns the metastore.api.Table object this Table was created from. Returns null if the derived Table object was not created from a metastore Table (ex. InlineViews).

Definition at line 398 of file Table.java.

References com.cloudera.impala.catalog.Table.msTable_.

String com.cloudera.impala.catalog.Table.getName ( )

inlineinherited

Implements com.cloudera.impala.catalog.CatalogObject.

Definition at line 342 of file Table.java.

References com.cloudera.impala.catalog.Table.name_.

List<Column> com.cloudera.impala.catalog.Table.getNonClusteringColumns ( )

inlineinherited

Returns the list of all columns excluding any partition columns.

Definition at line 385 of file Table.java.

References com.cloudera.impala.catalog.Table.numClusteringCols_.

Referenced by com.cloudera.impala.analysis.ComputeStatsStmt.analyze(), and com.cloudera.impala.catalog.Table.getColumnsInHiveOrder().

int com.cloudera.impala.catalog.Table.getNumClusteringCols ( )

inlineinherited

Definition at line 402 of file Table.java.

References com.cloudera.impala.catalog.Table.numClusteringCols_.

Referenced by com.cloudera.impala.analysis.ShowPartitionsStmt.analyze(), com.cloudera.impala.analysis.ComputeStatsStmt.analyze(), com.cloudera.impala.catalog.CatalogTest.checkTableCols(), com.cloudera.impala.service.CatalogOpExecutor.dropTableOrView(), com.cloudera.impala.analysis.ComputeStatsStmt.getBaseColumnStatsQuerySelectList(), com.cloudera.impala.planner.HdfsScanNode.getNodeExplainString(), com.cloudera.impala.catalog.HdfsTable.getNonPartitionFieldSchemas(), com.cloudera.impala.planner.HdfsScanNode.prunePartitions(), and com.cloudera.impala.service.CatalogOpExecutor.updateCatalog().

int com.cloudera.impala.catalog.HBaseTable.getNumNodes ( )

inline

Definition at line 599 of file HBaseTable.java.

long com.cloudera.impala.catalog.Table.getNumRows ( )

inlineinherited

Definition at line 404 of file Table.java.

References com.cloudera.impala.catalog.Table.numRows_.

Referenced by com.cloudera.impala.analysis.IsNullPredicate.analyze().

String com.cloudera.impala.catalog.Table.getOwner ( )

inlineinherited

Definition at line 348 of file Table.java.

References com.cloudera.impala.catalog.Table.owner_.

static List<HRegionLocation> com.cloudera.impala.catalog.HBaseTable.getRegionsInRange	(	HTable	hbaseTbl,
		final byte[]	startKey,
		final byte[]	endKey
	)		throws IOException

inlinestatic

This is copied from org.apache.hadoop.hbase.client.HTable. The only difference is that it does not use cache when calling getRegionLocation. TODO: Remove this function and use HTable.getRegionsInRange when the non-cache version has been ported to CDH (DISTRO-477). Get the corresponding regions for an arbitrary range of keys.

Parameters

startRow	Starting row in range, inclusive
endRow	Ending row in range, exclusive

Returns: A list of HRegionLocations corresponding to the regions that contain the specified range

Exceptions

IOException if a remote or network exception occurs

Definition at line 650 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats().

static Path com.cloudera.impala.catalog.HBaseTable.getRootDir ( final Configuration c ) throws IOException

inlinestatic

Returns hbase's root directory: i.e. hbase.rootdir from the given configuration as a qualified Path. Method copied from HBase FSUtils.java to avoid depending on HBase server.

Definition at line 562 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getHdfsSize().

static long com.cloudera.impala.catalog.Table.getRowCount ( Map< String, String > parameters )

inlinestaticprotectedinherited

Returns the value of the ROW_COUNT constant, or -1 if not found.

Definition at line 191 of file Table.java.

Referenced by com.cloudera.impala.catalog.DataSourceTable.load(), com.cloudera.impala.catalog.HBaseTable.load(), com.cloudera.impala.catalog.HdfsTable.load(), and com.cloudera.impala.catalog.HdfsTable.loadPartitions().

String com.cloudera.impala.catalog.HBaseTable.getStorageHandlerClassName ( )

inline

Returns the storage handler class for HBase tables read by Hive.

Definition at line 676 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.HBASE_STORAGE_HANDLER.

TableName com.cloudera.impala.catalog.Table.getTableName ( )

inlineinherited

Definition at line 344 of file Table.java.

References com.cloudera.impala.catalog.Table.db_, com.cloudera.impala.catalog.Db.getName(), and com.cloudera.impala.catalog.Table.name_.

Referenced by com.cloudera.impala.analysis.ComputeStatsStmt.analyze(), com.cloudera.impala.service.CatalogOpExecutor.dropStats(), and com.cloudera.impala.analysis.InlineViewRef.InlineViewRef().

TResultSet com.cloudera.impala.catalog.HBaseTable.getTableStats ( )

inline

Returns statistics on this table as a tabular result set. Used for the SHOW TABLE STATS statement. The schema of the returned TResultSet is set inside this method.

Definition at line 685 of file HBaseTable.java.

References com.cloudera.impala.catalog.Type.BIGINT, com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStatsForRegion(), com.cloudera.impala.catalog.HBaseTable.getHdfsSize(), com.cloudera.impala.catalog.HBaseTable.hTable_, com.cloudera.impala.catalog.Type.STRING, and com.cloudera.impala.catalog.ScalarType.toThrift().

THBaseTable com.cloudera.impala.catalog.HBaseTable.getTHBaseTable ( )

inlineprivate

Definition at line 617 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseColumn.getColumnQualifier(), com.cloudera.impala.catalog.Table.getColumns(), and com.cloudera.impala.catalog.HBaseTable.hbaseTableName_.

Referenced by com.cloudera.impala.catalog.HBaseTable.toThrift(), and com.cloudera.impala.catalog.HBaseTable.toThriftDescriptor().

ArrayType com.cloudera.impala.catalog.Table.getType ( )

inlineinherited

Definition at line 405 of file Table.java.

References com.cloudera.impala.catalog.Table.type_.

Referenced by com.cloudera.impala.catalog.HdfsTable.load().

static boolean com.cloudera.impala.catalog.HBaseTable.isHBaseTable ( org.apache.hadoop.hive.metastore.api.Table msTbl )

inlinestatic

Returns true if the given Metastore Table represents an HBase table. Versions of Hive/HBase are inconsistent which HBase related fields are set (e.g., HIVE-6548 changed the input format to null). For maximum compatibility consider all known fields that indicate an HBase table.

Definition at line 739 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.HBASE_INPUT_FORMAT, com.cloudera.impala.catalog.HBaseTable.HBASE_SERIALIZATION_LIB, and com.cloudera.impala.catalog.HBaseTable.HBASE_STORAGE_HANDLER.

Referenced by com.cloudera.impala.catalog.Table.fromMetastoreTable().

boolean com.cloudera.impala.catalog.Table.isLoaded ( )

inlineinherited

Implements com.cloudera.impala.catalog.CatalogObject.

Definition at line 416 of file Table.java.

Referenced by com.cloudera.impala.service.CatalogOpExecutor.getExistingTable(), com.cloudera.impala.service.Frontend.getMissingTbls(), com.cloudera.impala.catalog.CatalogServiceCatalog.getOrLoadTable(), com.cloudera.impala.testutil.ImpaladTestCatalog.getTable(), com.cloudera.impala.catalog.ImpaladCatalog.getTable(), and com.cloudera.impala.analysis.Analyzer.getTable().

void com.cloudera.impala.catalog.HBaseTable.load	(	Table	oldValue,
		HiveMetaStoreClient	client,
		org.apache.hadoop.hive.metastore.api.Table	msTbl
	)		throws TableLoadingException

inline

For hbase tables, we can support tables with columns we don't understand at all (e.g. map) as long as the user does not select those. This is in contrast to hdfs tables since we typically need to understand all columns to make sense of the file at all.

Definition at line 266 of file HBaseTable.java.

void com.cloudera.impala.catalog.Table.loadAllColumnStats ( HiveMetaStoreClient client )

inlineprotectedinherited

Loads column statistics for all columns in this table from the Hive metastore. Any errors are logged and ignored, since the absence of column stats is not critical to the correctness of the system.

Definition at line 155 of file Table.java.

References com.cloudera.impala.catalog.Table.getColumn(), com.cloudera.impala.catalog.Table.getColumnNamesWithHmsStats(), com.cloudera.impala.catalog.Table.getFullName(), com.cloudera.impala.catalog.Column.getType(), com.cloudera.impala.catalog.ColumnStats.isSupportedColType(), com.cloudera.impala.catalog.Table.name_, and com.cloudera.impala.catalog.Column.updateStats().

Referenced by com.cloudera.impala.catalog.HBaseTable.load(), and com.cloudera.impala.catalog.HdfsTable.loadColumns().

void com.cloudera.impala.catalog.HBaseTable.loadFromThrift ( TTable table ) throws TableLoadingException

inlineprotected

Definition at line 364 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.columnFamilies_, com.cloudera.impala.catalog.HBaseTable.getHBaseTableName(), com.cloudera.impala.catalog.Table.getMetaStoreTable(), com.cloudera.impala.catalog.HBaseTable.hbaseConf_, com.cloudera.impala.catalog.HBaseTable.hbaseTableName_, com.cloudera.impala.catalog.HBaseTable.hTable_, and com.cloudera.impala.catalog.Table.name_.

void com.cloudera.impala.catalog.HBaseTable.parseColumnMapping	(	boolean	tableDefaultStorageIsBinary,
		String	columnsMappingSpec,
		List< FieldSchema >	fieldSchemas,
		List< String >	columnFamilies,
		List< String >	columnQualifiers,
		List< Boolean >	colIsBinaryEncoded
	)		throws SerDeException

inlineprivate

Definition at line 137 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.supportsBinaryEncoding().

Referenced by com.cloudera.impala.catalog.HBaseTable.load().

Type com.cloudera.impala.catalog.Table.parseColumnType ( FieldSchema fs ) throws TableLoadingException

inlineprotectedinherited

Gets the ColumnType from the given FieldSchema by using Impala's SqlParser. Throws a TableLoadingException if the FieldSchema could not be parsed. The type can either be:

Supported by Impala, in which case the type is returned.
A type Impala understands but is not yet implemented (e.g. date), the type is returned but type.IsSupported() returns false.
A type Impala can't understand at all, and a TableLoadingException is thrown.

Definition at line 331 of file Table.java.

References com.cloudera.impala.catalog.Table.getName().

Referenced by com.cloudera.impala.catalog.View.load(), com.cloudera.impala.catalog.HBaseTable.load(), com.cloudera.impala.catalog.DataSourceTable.loadColumns(), com.cloudera.impala.catalog.HdfsTable.loadColumns(), and com.cloudera.impala.catalog.HBaseTable.supportsBinaryEncoding().

void com.cloudera.impala.catalog.Table.setCatalogVersion ( long catalogVersion )

inlineinherited

Implements com.cloudera.impala.catalog.CatalogObject.

Definition at line 411 of file Table.java.

References com.cloudera.impala.catalog.Table.catalogVersion_.

boolean com.cloudera.impala.catalog.HBaseTable.supportsBinaryEncoding ( FieldSchema fs )

inlineprivate

Definition at line 248 of file HBaseTable.java.

References com.cloudera.impala.catalog.Type.isIntegerType(), and com.cloudera.impala.catalog.Table.parseColumnType().

Referenced by com.cloudera.impala.catalog.HBaseTable.parseColumnMapping().

TCatalogObject com.cloudera.impala.catalog.Table.toTCatalogObject ( )

inlineinherited

Definition at line 314 of file Table.java.

References com.cloudera.impala.catalog.Table.getCatalogObjectType(), com.cloudera.impala.catalog.Table.getCatalogVersion(), and com.cloudera.impala.catalog.Table.toThrift().

TTable com.cloudera.impala.catalog.HBaseTable.toThrift ( )

inline

Definition at line 610 of file HBaseTable.java.

References com.cloudera.impala.catalog.HBaseTable.getTHBaseTable().

TTableDescriptor com.cloudera.impala.catalog.HBaseTable.toThriftDescriptor ( Set< Long > referencedPartitions )

inline

Definition at line 577 of file HBaseTable.java.

References com.cloudera.impala.catalog.Table.db_, com.cloudera.impala.catalog.Table.getColumnNames(), com.cloudera.impala.catalog.Table.getColumns(), com.cloudera.impala.catalog.Db.getName(), com.cloudera.impala.catalog.HBaseTable.getTHBaseTable(), com.cloudera.impala.catalog.HBaseTable.hbaseTableName_, com.cloudera.impala.catalog.Table.id_, and com.cloudera.impala.catalog.Table.numClusteringCols_.

void com.cloudera.impala.catalog.Table.updateLastDdlTime ( long ddlTime )

inlineinherited

Updates the lastDdlTime for this Table, if the new value is greater than the existing value. Does nothing if the new value is less than or equal to the existing value.

Definition at line 132 of file Table.java.

References com.cloudera.impala.catalog.Table.lastDdlTime_.

void com.cloudera.impala.catalog.Table.validate ( ) throws TableLoadingException

inlineinherited

Checks preconditions for this table to function as expected. Currently only checks that all entries in colsByName_ use lower case keys.

Definition at line 279 of file Table.java.

References com.cloudera.impala.catalog.Table.colsByName_.

Member Data Documentation

TAccessLevel com.cloudera.impala.catalog.Table.accessLevel_ = TAccessLevel.READ_WRITE

protectedinherited

Definition at line 66 of file Table.java.

Referenced by com.cloudera.impala.catalog.HdfsTable.getFirstLocationWithoutWriteAccess(), com.cloudera.impala.catalog.HdfsTable.hasWriteAccess(), com.cloudera.impala.catalog.Table.loadFromThrift(), com.cloudera.impala.catalog.HdfsTable.loadPartitions(), and com.cloudera.impala.catalog.Table.toThrift().

HColumnDescriptor [] com.cloudera.impala.catalog.HBaseTable.columnFamilies_ = null

private

Definition at line 121 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats(), com.cloudera.impala.catalog.HBaseTable.load(), and com.cloudera.impala.catalog.HBaseTable.loadFromThrift().

final Db com.cloudera.impala.catalog.Table.db_

protectedinherited

Definition at line 61 of file Table.java.

final String com.cloudera.impala.catalog.HBaseTable.DEFAULT_PREFIX = "default."

static

Definition at line 84 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getHBaseTableName().

final double com.cloudera.impala.catalog.HBaseTable.DELTA_FROM_AVERAGE = 0.15

staticprivate

Definition at line 79 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats().

List<FieldSchema> com.cloudera.impala.catalog.Table.fields_

protectedinherited

Definition at line 65 of file Table.java.

Referenced by com.cloudera.impala.catalog.HdfsTable.getFieldSchemas(), com.cloudera.impala.catalog.HdfsTable.loadColumns(), and com.cloudera.impala.catalog.Table.loadFromThrift().

final String com.cloudera.impala.catalog.HBaseTable.HBASE_INPUT_FORMAT

staticprivate

Initial value:

=

"org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat"

Definition at line 101 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.isHBaseTable().

final String com.cloudera.impala.catalog.HBaseTable.HBASE_SERIALIZATION_LIB

staticprivate

Initial value:

=

"org.apache.hadoop.hive.hbase.HBaseSerDe"

Definition at line 105 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.isHBaseTable().

final String com.cloudera.impala.catalog.HBaseTable.HBASE_STORAGE_HANDLER

staticprivate

Initial value:

=

"org.apache.hadoop.hive.hbase.HBaseStorageHandler"

Definition at line 109 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getStorageHandlerClassName(), and com.cloudera.impala.catalog.HBaseTable.isHBaseTable().

final Configuration com.cloudera.impala.catalog.HBaseTable.hbaseConf_ = HBaseConfiguration.create()

staticprivate

Definition at line 116 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getHBaseConf(), com.cloudera.impala.catalog.HBaseTable.getHdfsSize(), com.cloudera.impala.catalog.HBaseTable.load(), and com.cloudera.impala.catalog.HBaseTable.loadFromThrift().

String com.cloudera.impala.catalog.HBaseTable.hbaseTableName_

protected

Definition at line 98 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getHBaseTableName(), com.cloudera.impala.catalog.HBaseTable.getHdfsSize(), com.cloudera.impala.catalog.HBaseTable.getTHBaseTable(), com.cloudera.impala.catalog.HBaseTable.load(), com.cloudera.impala.catalog.HBaseTable.loadFromThrift(), and com.cloudera.impala.catalog.HBaseTable.toThriftDescriptor().

HTable com.cloudera.impala.catalog.HBaseTable.hTable_ = null

private

Definition at line 118 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats(), com.cloudera.impala.catalog.HBaseTable.getHTable(), com.cloudera.impala.catalog.HBaseTable.getTableStats(), com.cloudera.impala.catalog.HBaseTable.load(), and com.cloudera.impala.catalog.HBaseTable.loadFromThrift().

final TableId com.cloudera.impala.catalog.Table.id_

protectedinherited

Definition at line 60 of file Table.java.

Referenced by com.cloudera.impala.catalog.Table.getId(), com.cloudera.impala.catalog.Table.Table(), com.cloudera.impala.catalog.DataSourceTable.toThriftDescriptor(), com.cloudera.impala.catalog.HBaseTable.toThriftDescriptor(), and com.cloudera.impala.catalog.HdfsTable.toThriftDescriptor().

long com.cloudera.impala.catalog.Table.lastDdlTime_

protectedinherited

Definition at line 85 of file Table.java.

Referenced by com.cloudera.impala.catalog.HdfsTable.load(), com.cloudera.impala.catalog.Table.Table(), and com.cloudera.impala.catalog.Table.updateLastDdlTime().

final Logger com.cloudera.impala.catalog.HBaseTable.LOG = Logger.getLogger(HBaseTable.class)

staticprivate

Definition at line 81 of file HBaseTable.java.

final int com.cloudera.impala.catalog.HBaseTable.MIN_NUM_REGIONS_TO_CHECK = 5

staticprivate

Definition at line 90 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStats().

final org.apache.hadoop.hive.metastore.api.Table com.cloudera.impala.catalog.Table.msTable_

protectedinherited

Definition at line 58 of file Table.java.

Referenced by com.cloudera.impala.catalog.Table.getMetaStoreTable(), and com.cloudera.impala.catalog.Table.Table().

final String com.cloudera.impala.catalog.Table.name_

protectedinherited

Definition at line 62 of file Table.java.

int com.cloudera.impala.catalog.Table.numClusteringCols_

protectedinherited

Definition at line 69 of file Table.java.

long com.cloudera.impala.catalog.Table.numRows_ = -1

protectedinherited

Definition at line 72 of file Table.java.

final String com.cloudera.impala.catalog.Table.owner_

protectedinherited

Definition at line 63 of file Table.java.

Referenced by com.cloudera.impala.catalog.Table.getOwner(), and com.cloudera.impala.catalog.Table.Table().

final int com.cloudera.impala.catalog.HBaseTable.ROW_COUNT_ESTIMATE_BATCH_SIZE = 10

static

Definition at line 87 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.getEstimatedRowStatsForRegion().

final String com.cloudera.impala.catalog.HBaseTable.ROW_KEY_COLUMN_FAMILY = ":key"

staticprivate

Definition at line 113 of file HBaseTable.java.

Referenced by com.cloudera.impala.catalog.HBaseTable.load().

HBaseColumn com.cloudera.impala.catalog.HBaseTable.rowKey_

protected

Definition at line 94 of file HBaseTable.java.

EnumSet<TableType> com.cloudera.impala.catalog.Table.SUPPORTED_TABLE_TYPES

staticprotectedinherited

Initial value:

= EnumSet.of(

TableType.EXTERNAL_TABLE, TableType.MANAGED_TABLE, TableType.VIRTUAL_VIEW)

Definition at line 88 of file Table.java.

TTableDescriptor com.cloudera.impala.catalog.Table.tableDesc_

protectedinherited

Definition at line 64 of file Table.java.

final ArrayType com.cloudera.impala.catalog.Table.type_ = new ArrayType(new StructType())

protectedinherited

Definition at line 82 of file Table.java.

Referenced by com.cloudera.impala.catalog.Table.addColumn(), com.cloudera.impala.catalog.Table.clearColumns(), com.cloudera.impala.catalog.Table.getType(), and com.cloudera.impala.catalog.Table.loadFromThrift().

The documentation for this class was generated from the following file:

fe/src/main/java/com/cloudera/impala/catalog/HBaseTable.java

Public Member Functions

Static Public Member Functions

Static Public Attributes

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Static Protected Attributes

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation