Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros
com.cloudera.impala.catalog.DataSourceTable Class Reference
Inheritance diagram for com.cloudera.impala.catalog.DataSourceTable:
Collaboration diagram for com.cloudera.impala.catalog.DataSourceTable:

Public Member Functions

TDataSource getDataSource ()
 
String getInitString ()
 
int getNumNodes ()
 
TCatalogObjectType getCatalogObjectType ()
 
void load (Table oldValue, HiveMetaStoreClient client, org.apache.hadoop.hive.metastore.api.Table msTbl) throws TableLoadingException
 
TResultSet getTableStats ()
 
TTableDescriptor toThriftDescriptor (Set< Long > referencedPartitions)
 
TTable toThrift ()
 
void addColumn (Column col)
 
void clearColumns ()
 
void updateLastDdlTime (long ddlTime)
 
void validate () throws TableLoadingException
 
TCatalogObject toTCatalogObject ()
 
Db getDb ()
 
String getName ()
 
String getFullName ()
 
TableName getTableName ()
 
String getOwner ()
 
ArrayList< ColumngetColumns ()
 
List< String > getColumnNames ()
 
String getStorageHandlerClassName ()
 
ArrayList< ColumngetColumnsInHiveOrder ()
 
List< ColumngetNonClusteringColumns ()
 
Column getColumn (String name)
 
org.apache.hadoop.hive.metastore.api.Table getMetaStoreTable ()
 
int getNumClusteringCols ()
 
TableId getId ()
 
long getNumRows ()
 
ArrayType getType ()
 
long getCatalogVersion ()
 
void setCatalogVersion (long catalogVersion)
 
boolean isLoaded ()
 

Static Public Member Functions

static boolean isSupportedColumnType (Type colType)
 
static boolean isSupportedPrimitiveType (PrimitiveType primitiveType)
 
static boolean isDataSourceTable (org.apache.hadoop.hive.metastore.api.Table msTbl)
 
static Table fromMetastoreTable (TableId id, Db db, org.apache.hadoop.hive.metastore.api.Table msTbl)
 
static Table fromThrift (Db parentDb, TTable thriftTable) throws TableLoadingException
 

Static Public Attributes

static final String TBL_PROP_DATA_SRC_NAME = "__IMPALA_DATA_SOURCE_NAME"
 
static final String TBL_PROP_INIT_STRING = "__IMPALA_DATA_SOURCE_INIT_STRING"
 
static final String TBL_PROP_LOCATION = "__IMPALA_DATA_SOURCE_LOCATION"
 
static final String TBL_PROP_CLASS = "__IMPALA_DATA_SOURCE_CLASS"
 
static final String TBL_PROP_API_VER = "__IMPALA_DATA_SOURCE_API_VERSION"
 

Protected Member Functions

 DataSourceTable (TableId id, org.apache.hadoop.hive.metastore.api.Table msTable, Db db, String name, String owner)
 
void loadFromThrift (TTable thriftTable) throws TableLoadingException
 
List< String > getColumnNamesWithHmsStats ()
 
void loadAllColumnStats (HiveMetaStoreClient client)
 
Type parseColumnType (FieldSchema fs) throws TableLoadingException
 

Static Protected Member Functions

static long getRowCount (Map< String, String > parameters)
 

Protected Attributes

final
org.apache.hadoop.hive.metastore.api.Table 
msTable_
 
final TableId id_
 
final Db db_
 
final String name_
 
final String owner_
 
TTableDescriptor tableDesc_
 
List< FieldSchema > fields_
 
TAccessLevel accessLevel_ = TAccessLevel.READ_WRITE
 
int numClusteringCols_
 
long numRows_ = -1
 
final ArrayType type_ = new ArrayType(new StructType())
 
long lastDdlTime_
 

Static Protected Attributes

static EnumSet< TableType > SUPPORTED_TABLE_TYPES
 

Private Member Functions

void loadColumns (List< FieldSchema > fieldSchemas, HiveMetaStoreClient client) throws TableLoadingException
 
String getRequiredTableProperty (org.apache.hadoop.hive.metastore.api.Table msTbl, String key, String dataSourceName) throws TableLoadingException
 
TDataSourceTable getDataSourceTable ()
 

Private Attributes

String initString_
 
TDataSource dataSource_
 

Static Private Attributes

static final Logger LOG = LoggerFactory.getLogger(DataSourceTable.class)
 

Detailed Description

Represents a table backed by an external data source. All data source properties are stored as table properties (persisted in the metastore) because the DataSource catalog object is not persisted so the DataSource catalog object will not exist if the catalog server is restarted, but the table does not need the DataSource catalog object in order to scan the table. Tables that contain the TBL_PROP_DATA_SRC_NAME table parameter are assumed to be backed by an external data source.

Definition at line 46 of file DataSourceTable.java.

Constructor & Destructor Documentation

com.cloudera.impala.catalog.DataSourceTable.DataSourceTable ( TableId  id,
org.apache.hadoop.hive.metastore.api.Table  msTable,
Db  db,
String  name,
String  owner 
)
inlineprotected

Definition at line 77 of file DataSourceTable.java.

Member Function Documentation

void com.cloudera.impala.catalog.Table.clearColumns ( )
inlineinherited
static Table com.cloudera.impala.catalog.Table.fromMetastoreTable ( TableId  id,
Db  db,
org.apache.hadoop.hive.metastore.api.Table  msTbl 
)
inlinestaticinherited

Creates a table of the appropriate type based on the given hive.metastore.api.Table object.

Definition at line 207 of file Table.java.

References com.cloudera.impala.catalog.DataSourceTable.isDataSourceTable(), com.cloudera.impala.catalog.HBaseTable.isHBaseTable(), and com.cloudera.impala.catalog.HdfsFileFormat.isHdfsFormatClass().

static Table com.cloudera.impala.catalog.Table.fromThrift ( Db  parentDb,
TTable  thriftTable 
) throws TableLoadingException
inlinestaticinherited

Factory method that creates a new Table from its Thrift representation. Determines the type of table to create based on the Thrift table provided.

Definition at line 231 of file Table.java.

TCatalogObjectType com.cloudera.impala.catalog.DataSourceTable.getCatalogObjectType ( )
inline

Implements com.cloudera.impala.catalog.CatalogObject.

Definition at line 97 of file DataSourceTable.java.

Column com.cloudera.impala.catalog.Table.getColumn ( String  name)
inlineinherited
List<String> com.cloudera.impala.catalog.Table.getColumnNamesWithHmsStats ( )
inlineprotectedinherited
ArrayList<Column> com.cloudera.impala.catalog.Table.getColumnsInHiveOrder ( )
inlineinherited

Returns the list of all columns, but with partition columns at the end of the list rather than the beginning. This is equivalent to the order in which Hive enumerates columns.

Definition at line 373 of file Table.java.

References com.cloudera.impala.catalog.Table.colsByPos_, com.cloudera.impala.catalog.Table.getNonClusteringColumns(), and com.cloudera.impala.catalog.Table.numClusteringCols_.

Referenced by com.cloudera.impala.service.DescribeResultFactory.describeTableMinimal(), com.cloudera.impala.service.Frontend.getColumnStats(), and com.cloudera.impala.analysis.InsertStmt.prepareExpressions().

TDataSource com.cloudera.impala.catalog.DataSourceTable.getDataSource ( )
inline
TDataSourceTable com.cloudera.impala.catalog.DataSourceTable.getDataSourceTable ( )
inlineprivate
TableId com.cloudera.impala.catalog.Table.getId ( )
inlineinherited
String com.cloudera.impala.catalog.DataSourceTable.getInitString ( )
inline

Gets the table init string passed to the data source.

Definition at line 91 of file DataSourceTable.java.

References com.cloudera.impala.catalog.DataSourceTable.initString_.

Referenced by com.cloudera.impala.planner.DataSourceScanNode.debugString().

List<Column> com.cloudera.impala.catalog.Table.getNonClusteringColumns ( )
inlineinherited

Returns the list of all columns excluding any partition columns.

Definition at line 385 of file Table.java.

References com.cloudera.impala.catalog.Table.numClusteringCols_.

Referenced by com.cloudera.impala.analysis.ComputeStatsStmt.analyze(), and com.cloudera.impala.catalog.Table.getColumnsInHiveOrder().

int com.cloudera.impala.catalog.DataSourceTable.getNumNodes ( )
inline

Definition at line 94 of file DataSourceTable.java.

long com.cloudera.impala.catalog.Table.getNumRows ( )
inlineinherited
String com.cloudera.impala.catalog.Table.getOwner ( )
inlineinherited

Definition at line 348 of file Table.java.

References com.cloudera.impala.catalog.Table.owner_.

String com.cloudera.impala.catalog.DataSourceTable.getRequiredTableProperty ( org.apache.hadoop.hive.metastore.api.Table  msTbl,
String  key,
String  dataSourceName 
) throws TableLoadingException
inlineprivate
static long com.cloudera.impala.catalog.Table.getRowCount ( Map< String, String >  parameters)
inlinestaticprotectedinherited
String com.cloudera.impala.catalog.Table.getStorageHandlerClassName ( )
inlineinherited

Subclasses should override this if they provide a storage handler class. Currently only HBase tables need to provide a storage handler.

Definition at line 366 of file Table.java.

TResultSet com.cloudera.impala.catalog.DataSourceTable.getTableStats ( )
inline

Returns statistics on this table as a tabular result set. Used for the SHOW TABLE STATS statement. The schema of the returned TResultSet is set inside this method.

Definition at line 208 of file DataSourceTable.java.

References com.cloudera.impala.catalog.Type.BIGINT, com.cloudera.impala.catalog.Table.numRows_, and com.cloudera.impala.catalog.ScalarType.toThrift().

ArrayType com.cloudera.impala.catalog.Table.getType ( )
inlineinherited
static boolean com.cloudera.impala.catalog.DataSourceTable.isDataSourceTable ( org.apache.hadoop.hive.metastore.api.Table  msTbl)
inlinestatic

True if the Hive org.apache.hadoop.hive.metastore.api.Table is a data source table by checking for the existance of the TBL_PROP_DATA_SRC_NAME table property.

Definition at line 252 of file DataSourceTable.java.

References com.cloudera.impala.catalog.DataSourceTable.TBL_PROP_DATA_SRC_NAME.

Referenced by com.cloudera.impala.catalog.Table.fromMetastoreTable().

static boolean com.cloudera.impala.catalog.DataSourceTable.isSupportedColumnType ( Type  colType)
inlinestatic
static boolean com.cloudera.impala.catalog.DataSourceTable.isSupportedPrimitiveType ( PrimitiveType  primitiveType)
inlinestatic

Returns true if the primitive type is supported.

Definition at line 110 of file DataSourceTable.java.

References impala.INVALID_TYPE.

Referenced by com.cloudera.impala.catalog.DataSourceTable.isSupportedColumnType(), and com.cloudera.impala.analysis.AnalyzeDDLTest.TestCreateTable().

void com.cloudera.impala.catalog.Table.loadAllColumnStats ( HiveMetaStoreClient  client)
inlineprotectedinherited
void com.cloudera.impala.catalog.DataSourceTable.loadColumns ( List< FieldSchema >  fieldSchemas,
HiveMetaStoreClient  client 
) throws TableLoadingException
inlineprivate
void com.cloudera.impala.catalog.DataSourceTable.loadFromThrift ( TTable  thriftTable) throws TableLoadingException
inlineprotected
Type com.cloudera.impala.catalog.Table.parseColumnType ( FieldSchema  fs) throws TableLoadingException
inlineprotectedinherited

Gets the ColumnType from the given FieldSchema by using Impala's SqlParser. Throws a TableLoadingException if the FieldSchema could not be parsed. The type can either be:

  • Supported by Impala, in which case the type is returned.
  • A type Impala understands but is not yet implemented (e.g. date), the type is returned but type.IsSupported() returns false.
  • A type Impala can't understand at all, and a TableLoadingException is thrown.

Definition at line 331 of file Table.java.

References com.cloudera.impala.catalog.Table.getName().

Referenced by com.cloudera.impala.catalog.View.load(), com.cloudera.impala.catalog.HBaseTable.load(), com.cloudera.impala.catalog.DataSourceTable.loadColumns(), com.cloudera.impala.catalog.HdfsTable.loadColumns(), and com.cloudera.impala.catalog.HBaseTable.supportsBinaryEncoding().

void com.cloudera.impala.catalog.Table.setCatalogVersion ( long  catalogVersion)
inlineinherited
TCatalogObject com.cloudera.impala.catalog.Table.toTCatalogObject ( )
inlineinherited
TTable com.cloudera.impala.catalog.DataSourceTable.toThrift ( )
inline

Returns a thrift structure representing the table.

Definition at line 233 of file DataSourceTable.java.

References com.cloudera.impala.catalog.DataSourceTable.getDataSourceTable().

void com.cloudera.impala.catalog.Table.updateLastDdlTime ( long  ddlTime)
inlineinherited

Updates the lastDdlTime for this Table, if the new value is greater than the existing value. Does nothing if the new value is less than or equal to the existing value.

Definition at line 132 of file Table.java.

References com.cloudera.impala.catalog.Table.lastDdlTime_.

void com.cloudera.impala.catalog.Table.validate ( ) throws TableLoadingException
inlineinherited

Checks preconditions for this table to function as expected. Currently only checks that all entries in colsByName_ use lower case keys.

Definition at line 279 of file Table.java.

References com.cloudera.impala.catalog.Table.colsByName_.

Member Data Documentation

List<FieldSchema> com.cloudera.impala.catalog.Table.fields_
protectedinherited
long com.cloudera.impala.catalog.Table.lastDdlTime_
protectedinherited
final Logger com.cloudera.impala.catalog.DataSourceTable.LOG = LoggerFactory.getLogger(DataSourceTable.class)
staticprivate

Definition at line 47 of file DataSourceTable.java.

final org.apache.hadoop.hive.metastore.api.Table com.cloudera.impala.catalog.Table.msTable_
protectedinherited
final String com.cloudera.impala.catalog.Table.owner_
protectedinherited
EnumSet<TableType> com.cloudera.impala.catalog.Table.SUPPORTED_TABLE_TYPES
staticprotectedinherited
Initial value:
= EnumSet.of(
TableType.EXTERNAL_TABLE, TableType.MANAGED_TABLE, TableType.VIRTUAL_VIEW)

Definition at line 88 of file Table.java.

TTableDescriptor com.cloudera.impala.catalog.Table.tableDesc_
protectedinherited

Definition at line 64 of file Table.java.

final String com.cloudera.impala.catalog.DataSourceTable.TBL_PROP_API_VER = "__IMPALA_DATA_SOURCE_API_VERSION"
static

Table property key for the API version implemented by the data source.

Definition at line 72 of file DataSourceTable.java.

Referenced by com.cloudera.impala.catalog.DataSourceTable.load().

final String com.cloudera.impala.catalog.DataSourceTable.TBL_PROP_CLASS = "__IMPALA_DATA_SOURCE_CLASS"
static

Table property key for the class implementing ExternalDataSource.

Definition at line 67 of file DataSourceTable.java.

Referenced by com.cloudera.impala.catalog.DataSourceTable.load().

final String com.cloudera.impala.catalog.DataSourceTable.TBL_PROP_DATA_SRC_NAME = "__IMPALA_DATA_SOURCE_NAME"
static
final String com.cloudera.impala.catalog.DataSourceTable.TBL_PROP_INIT_STRING = "__IMPALA_DATA_SOURCE_INIT_STRING"
static

Table property key for the table init string.

Definition at line 57 of file DataSourceTable.java.

Referenced by com.cloudera.impala.catalog.DataSourceTable.load().

final String com.cloudera.impala.catalog.DataSourceTable.TBL_PROP_LOCATION = "__IMPALA_DATA_SOURCE_LOCATION"
static

Table property key for the data source library HDFS path.

Definition at line 62 of file DataSourceTable.java.

Referenced by com.cloudera.impala.catalog.DataSourceTable.load().


The documentation for this class was generated from the following file: