Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros
Public Member Functions | Static Public Member Functions | Static Public Attributes | Protected Attributes | Private Member Functions | Private Attributes | Static Private Attributes | List of all members
com.cloudera.impala.catalog.ImpaladCatalog Class Reference
Inheritance diagram for com.cloudera.impala.catalog.ImpaladCatalog:
Collaboration diagram for com.cloudera.impala.catalog.ImpaladCatalog:

Public Member Functions

 ImpaladCatalog ()
 
synchronized
TUpdateCatalogCacheResponse 		updateCatalog (TUpdateCatalogCacheRequest req) throws CatalogException
 
void waitForCatalogUpdate (long timeoutMs)
 
Table getTable (String dbName, String tableName) throws CatalogException
 
Path getTablePath (org.apache.hadoop.hive.metastore.api.Table msTbl) throws NoSuchObjectException, MetaException, TException
 
boolean isReady ()
 
void setIsReady ()
 
AuthorizationPolicy getAuthPolicy ()
 
Db getBuiltinsDb ()
 
Db addDb (Db db)
 
Db getDb (String dbName)
 
Db removeDb (String dbName)
 
List< String > getDbNames (String dbPattern)
 
Table removeTable (TTableName tableName)
 
List< String > getTableNames (String dbName, String tablePattern) throws DatabaseNotFoundException
 
boolean containsTable (String dbName, String tableName)
 
boolean addDataSource (DataSource dataSource)
 
DataSource removeDataSource (String dataSourceName)
 
DataSource getDataSource (String dataSourceName)
 
List< DataSource > getDataSources ()
 
List< DataSource > getDataSources (String pattern)
 
List< String > getDataSourceNames (String pattern)
 
boolean addFunction (Function fn)
 
Function getFunction (Function desc, Function.CompareMode mode)
 
Function removeFunction (Function desc)
 
boolean containsFunction (FunctionName name)
 
boolean addHdfsCachePool (HdfsCachePool cachePool)
 
HdfsCachePool getHdfsCachePool (String poolName)
 
void close ()
 
MetaStoreClient getMetaStoreClient ()
 
HdfsPartition getHdfsPartition (String dbName, String tableName, List< TPartitionKeyValue > partitionSpec) throws CatalogException
 
boolean containsHdfsPartition (String dbName, String tableName, List< TPartitionKeyValue > partitionSpec) throws CatalogException
 
TCatalogObject getTCatalogObject (TCatalogObject objectDesc) throws CatalogException
 

Static Public Member Functions

static Function getBuiltin (Function desc, Function.CompareMode mode)
 

Static Public Attributes

static final long INITIAL_CATALOG_VERSION = 0L
 
static final String DEFAULT_DB = "default"
 
static final String BUILTINS_DB = "_impala_builtins"
 

Protected Attributes

final MetaStoreClientPool metaStoreClientPool_ = new MetaStoreClientPool(0)
 
AuthorizationPolicy authPolicy_ = new AuthorizationPolicy()
 
AtomicReference
< ConcurrentHashMap< String,
Db > > 		dbCache_
 
final CatalogObjectCache
< DataSource > 		dataSources_
 
final CatalogObjectCache
< HdfsCachePool > 		hdfsCachePools_
 

Private Member Functions

void addCatalogObject (TCatalogObject catalogObject) throws TableLoadingException, DatabaseNotFoundException
 
void removeCatalogObject (TCatalogObject catalogObject, long currentCatalogUpdateVersion)
 
void addDb (TDatabase thriftDb, long catalogVersion)
 
void addTable (TTable thriftTable, long catalogVersion) throws TableLoadingException
 
void addFunction (TFunction fn, long catalogVersion)
 
void addDataSource (TDataSource thrift, long catalogVersion)
 
void removeDataSource (TDataSource thrift, long dropCatalogVersion)
 
void removeDb (TDatabase thriftDb, long dropCatalogVersion)
 
void removeTable (TTable thriftTable, long dropCatalogVersion)
 
void removeFunction (TFunction thriftFn, long dropCatalogVersion)
 
void removeRole (TRole thriftRole, long dropCatalogVersion)
 
void removePrivilege (TPrivilege thriftPrivilege, long dropCatalogVersion)
 

Private Attributes

TUniqueId catalogServiceId_ = INITIAL_CATALOG_SERVICE_ID
 
long lastSyncedCatalogVersion_ = Catalog.INITIAL_CATALOG_VERSION
 
final AtomicBoolean isReady_ = new AtomicBoolean(false)
 
final CatalogDeltaLog catalogDeltaLog_ = new CatalogDeltaLog()
 
final Object catalogUpdateEventNotifier_ = new Object()
 

Static Private Attributes

static final Logger LOG = Logger.getLogger(ImpaladCatalog.class)
 
static final TUniqueId INITIAL_CATALOG_SERVICE_ID = new TUniqueId(0L, 0L)
 

Detailed Description

Thread safe Catalog for an Impalad. The Impalad catalog can be updated either via a StateStore heartbeat or by directly applying the result of a catalog operation to the CatalogCache. All updates are applied using the updateCatalog() function. Table metadata is loaded lazily. The CatalogServer initially broadcasts (via the statestore) the known table names (as IncompleteTables). These table names are added to the Impalad catalog cache and when one of the tables is accessed, the impalad will make an RPC to the CatalogServer to request loading the complete table metadata. In both cases, we need to ensure that work from one update is not "undone" by another update. To handle this the ImpaladCatalog does the following:

  • Tracks the overall catalog version last received in a state store heartbeat, this version is maintained by the catalog server and it is always guaranteed that this impalad's catalog will never contain any objects < than this version (any updates with a lower version number are ignored).
  • For updated/new objects, check if the object already exists in the catalog cache. If it does, only apply the update if the catalog version is > the existing object's catalog version. Also keep a log of all dropped catalog objects (and the version they were dropped in). Before updating any object, check if it was dropped in a later version. If so, ignore the update.
  • Before dropping any catalog object, see if the object already exists in the catalog cache. If it does, only drop the object if the version of the drop is > that object's catalog version. The CatalogServiceId is also tracked to detect if a different instance of the catalog service has been started, in which case a full topic update is required.

Definition at line 64 of file ImpaladCatalog.java.

Constructor & Destructor Documentation

com.cloudera.impala.catalog.ImpaladCatalog.ImpaladCatalog ( )
inline

C'tor used by tests that need to validate the ImpaladCatalog outside of the CatalogServer.

Definition at line 91 of file ImpaladCatalog.java.

Member Function Documentation

void com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject ( TCatalogObject  catalogObject) throws TableLoadingException, DatabaseNotFoundException
inlineprivate

Adds the given TCatalogObject to the catalog cache. The update may be ignored (considered out of date) if: 1) An item exists in the catalog cache with a version > than the given TCatalogObject's version. 2) The catalogDeltaLog_ contains an entry for this object with a version

than the given TCatalogObject's version.

Definition at line 231 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.ImpaladCatalog.addDataSource(), com.cloudera.impala.catalog.ImpaladCatalog.addDb(), com.cloudera.impala.catalog.ImpaladCatalog.addFunction(), com.cloudera.impala.catalog.ImpaladCatalog.addTable(), com.cloudera.impala.catalog.ImpaladCatalog.catalogDeltaLog_, and com.cloudera.impala.catalog.CatalogDeltaLog.wasObjectRemovedAfter().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog().

boolean com.cloudera.impala.catalog.Catalog.addDataSource ( DataSource  dataSource)
inlineinherited

Adds a data source to the in-memory map of data sources. It is not persisted to the metastore.

Returns
true if this item was added or false if the existing value was preserved.

Definition at line 197 of file Catalog.java.

void com.cloudera.impala.catalog.ImpaladCatalog.addDataSource ( TDataSource  thrift,
long  catalogVersion 
)
inlineprivate

Definition at line 377 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject().

Db com.cloudera.impala.catalog.Catalog.addDb ( Db  db)
inlineinherited

Adds a new database to the catalog, replacing any existing database with the same name. Returns the previous database with this name, or null if there was no previous database.

Definition at line 107 of file Catalog.java.

References com.cloudera.impala.catalog.Db.getName().

Referenced by com.cloudera.impala.catalog.Catalog.Catalog().

void com.cloudera.impala.catalog.ImpaladCatalog.addDb ( TDatabase  thriftDb,
long  catalogVersion 
)
inlineprivate

Definition at line 338 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Db.getCatalogVersion(), and com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject(), and com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog().

boolean com.cloudera.impala.catalog.Catalog.addFunction ( Function  fn)
inlineinherited

Adds a function to the catalog. Returns true if the function was successfully added. Returns false if the function already exists. TODO: allow adding a function to a global scope. We probably want this to resolve after the local scope. e.g. if we had fn() and db.fn(). If the current database is 'db', fn() would resolve first to db.fn().

Definition at line 259 of file Catalog.java.

References com.cloudera.impala.catalog.Function.dbName(), and com.cloudera.impala.catalog.Catalog.getDb().

void com.cloudera.impala.catalog.ImpaladCatalog.addFunction ( TFunction  fn,
long  catalogVersion 
)
inlineprivate

Definition at line 362 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Function.getCatalogVersion(), and com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject().

boolean com.cloudera.impala.catalog.Catalog.addHdfsCachePool ( HdfsCachePool  cachePool)
inlineinherited

Adds a new HdfsCachePool to the catalog.

Definition at line 304 of file Catalog.java.

void com.cloudera.impala.catalog.ImpaladCatalog.addTable ( TTable  thriftTable,
long  catalogVersion 
) throws TableLoadingException
inlineprivate

Definition at line 348 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject().

void com.cloudera.impala.catalog.Catalog.close ( )
inlineinherited

Release the Hive Meta Store Client resources. Can be called multiple times (additional calls will be no-ops).

Definition at line 320 of file Catalog.java.

boolean com.cloudera.impala.catalog.Catalog.containsFunction ( FunctionName  name)
inlineinherited

Returns true if there is a function with this function name. Parameters are ignored.

Definition at line 295 of file Catalog.java.

References com.cloudera.impala.analysis.FunctionName.getDb(), and com.cloudera.impala.catalog.Catalog.getDb().

boolean com.cloudera.impala.catalog.Catalog.containsHdfsPartition ( String  dbName,
String  tableName,
List< TPartitionKeyValue >  partitionSpec 
) throws CatalogException
inlineinherited

Returns true if the table contains the given partition spec, otherwise false. This may trigger a metadata load if the table metadata is not yet cached.

Exceptions
DatabaseNotFoundException- If the database does not exist.
TableNotFoundException- If the table does not exist.
TableLoadingException- If there is an error loading the table metadata.

Definition at line 384 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getHdfsPartition().

Referenced by com.cloudera.impala.service.CatalogOpExecutor.alterTableAddPartition(), and com.cloudera.impala.service.CatalogOpExecutor.alterTableDropPartition().

boolean com.cloudera.impala.catalog.Catalog.containsTable ( String  dbName,
String  tableName 
)
inlineinherited

Returns true if the table and the database exist in the Impala Catalog. Returns false if either the table or the database do not exist.

Definition at line 187 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.service.CatalogOpExecutor.createTable(), com.cloudera.impala.service.CatalogOpExecutor.createTableLike(), and com.cloudera.impala.service.CatalogOpExecutor.createView().

AuthorizationPolicy com.cloudera.impala.catalog.ImpaladCatalog.getAuthPolicy ( )
inline

Definition at line 447 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Catalog.authPolicy_.

Referenced by com.cloudera.impala.service.Frontend.Frontend(), and com.cloudera.impala.service.Frontend.updateCatalogCache().

static Function com.cloudera.impala.catalog.Catalog.getBuiltin ( Function  desc,
Function.CompareMode  mode 
)
inlinestaticinherited

Definition at line 276 of file Catalog.java.

Db com.cloudera.impala.catalog.Catalog.getBuiltinsDb ( )
inlineinherited

Definition at line 100 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.builtinsDb_.

DataSource com.cloudera.impala.catalog.Catalog.getDataSource ( String  dataSourceName)
inlineinherited

Gets the specified data source.

Definition at line 213 of file Catalog.java.

Referenced by com.cloudera.impala.service.CatalogOpExecutor.createDataSource(), and com.cloudera.impala.catalog.Catalog.getTCatalogObject().

List<String> com.cloudera.impala.catalog.Catalog.getDataSourceNames ( String  pattern)
inlineinherited

Returns a list of data sources names that match pattern. See filterStringsByPattern for details of the pattern match semantics.

pattern may be null (and thus matches everything).

Definition at line 231 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.dataSources_, and com.cloudera.impala.catalog.Catalog.filterStringsByPattern().

List<DataSource> com.cloudera.impala.catalog.Catalog.getDataSources ( )
inlineinherited

Gets a list of all data sources.

Definition at line 221 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects().

List<DataSource> com.cloudera.impala.catalog.Catalog.getDataSources ( String  pattern)
inlineinherited

Returns a list of data sources that match pattern. See filterStringsByPattern for details of the pattern match semantics.

pattern may be null (and thus matches everything).

Definition at line 241 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.dataSources_, com.cloudera.impala.catalog.Catalog.filterStringsByPattern(), and impala.name.

Db com.cloudera.impala.catalog.Catalog.getDb ( String  dbName)
inlineinherited

Gets the Db object from the Catalog using a case-insensitive lookup on the name. Returns null if no matching database is found.

Definition at line 115 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.addDb(), com.cloudera.impala.catalog.Catalog.addFunction(), com.cloudera.impala.catalog.ImpaladCatalog.addFunction(), com.cloudera.impala.catalog.CatalogServiceCatalog.addFunction(), com.cloudera.impala.catalog.CatalogServiceCatalog.addPartition(), com.cloudera.impala.catalog.ImpaladCatalog.addTable(), com.cloudera.impala.catalog.CatalogServiceCatalog.addTable(), com.cloudera.impala.catalog.Catalog.containsFunction(), com.cloudera.impala.catalog.Catalog.containsTable(), com.cloudera.impala.service.CatalogOpExecutor.createDatabase(), com.cloudera.impala.catalog.TableLoadingMgr.LoadRequest.get(), com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), com.cloudera.impala.catalog.Catalog.getFunction(), com.cloudera.impala.catalog.CatalogServiceCatalog.getFunctions(), com.cloudera.impala.testutil.ImpaladTestCatalog.getTable(), com.cloudera.impala.catalog.Catalog.getTable(), com.cloudera.impala.catalog.Catalog.getTableNames(), com.cloudera.impala.catalog.Catalog.getTCatalogObject(), com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog(), com.cloudera.impala.catalog.CatalogServiceCatalog.invalidateTable(), com.cloudera.impala.catalog.ImpaladCatalog.removeDb(), com.cloudera.impala.catalog.Catalog.removeFunction(), com.cloudera.impala.catalog.ImpaladCatalog.removeFunction(), com.cloudera.impala.catalog.Catalog.removeTable(), com.cloudera.impala.catalog.ImpaladCatalog.removeTable(), com.cloudera.impala.catalog.CatalogServiceCatalog.removeTable(), com.cloudera.impala.catalog.CatalogServiceCatalog.renameTable(), com.cloudera.impala.catalog.CatalogServiceCatalog.replaceTableIfUnchanged(), com.cloudera.impala.catalog.CatalogTest.testStats(), com.cloudera.impala.analysis.AuthorizationTest.TestTPCHCleanup(), and com.cloudera.impala.catalog.CatalogServiceCatalog.updateLastDdlTime().

List<String> com.cloudera.impala.catalog.Catalog.getDbNames ( String  dbPattern)
inlineinherited

Returns a list of databases that match dbPattern. See filterStringsByPattern for details of the pattern match semantics.

dbPattern may be null (and thus matches everything).

Definition at line 136 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.dbCache_, and com.cloudera.impala.catalog.Catalog.filterStringsByPattern().

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog(), and com.cloudera.impala.testutil.BlockIdGenerator.main().

Function com.cloudera.impala.catalog.Catalog.getFunction ( Function  desc,
Function.CompareMode  mode 
)
inlineinherited

Returns the function that best matches 'desc' that is registered with the catalog using 'mode' to check for matching. If desc matches multiple functions in the catalog, it will return the function with the strictest matching mode.

Definition at line 270 of file Catalog.java.

References com.cloudera.impala.catalog.Function.dbName(), and com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.catalog.Catalog.getTCatalogObject().

HdfsCachePool com.cloudera.impala.catalog.Catalog.getHdfsCachePool ( String  poolName)
inlineinherited

Gets a HdfsCachePool given a cache pool name. Returns null if the cache pool does not exist.

Definition at line 312 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.Catalog.getTCatalogObject().

HdfsPartition com.cloudera.impala.catalog.Catalog.getHdfsPartition ( String  dbName,
String  tableName,
List< TPartitionKeyValue >  partitionSpec 
) throws CatalogException
inlineinherited

Returns the HdfsPartition object for the given dbName/tableName and partition spec. This will trigger a metadata load if the table metadata is not yet cached.

Exceptions
DatabaseNotFoundException- If the database does not exist.
TableNotFoundException- If the table does not exist.
PartitionNotFoundException- If the partition does not exist.
TableLoadingException- If there is an error loading the table metadata.

Definition at line 361 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getTable().

Referenced by com.cloudera.impala.catalog.Catalog.containsHdfsPartition().

MetaStoreClient com.cloudera.impala.catalog.Catalog.getMetaStoreClient ( )
inlineinherited

Returns a managed meta store client from the client connection pool.

Definition at line 326 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.getTablePath(), and com.cloudera.impala.catalog.CatalogServiceCatalog.invalidateTable().

Table com.cloudera.impala.catalog.ImpaladCatalog.getTable ( String  dbName,
String  tableName 
) throws CatalogException
inline

Returns the Table object for the given dbName/tableName. Returns null if the table does not exist. Will throw a TableLoadingException if the table's metadata was not able to be loaded successfully and DatabaseNotFoundException if the parent database does not exist.

Definition at line 181 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Table.isLoaded().

List<String> com.cloudera.impala.catalog.Catalog.getTableNames ( String  dbName,
String  tablePattern 
) throws DatabaseNotFoundException
inlineinherited

Returns a list of tables in the supplied database that match tablePattern. See filterStringsByPattern for details of the pattern match semantics.

dbName must not be null, but tablePattern may be null (and thus matches everything).

Table names are returned unqualified.

Definition at line 173 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.filterStringsByPattern(), and com.cloudera.impala.catalog.Catalog.getDb().

Path com.cloudera.impala.catalog.ImpaladCatalog.getTablePath ( org.apache.hadoop.hive.metastore.api.Table  msTbl) throws NoSuchObjectException, MetaException, TException
inline

Returns the HDFS path where the metastore would create the given table. If the table has a "location" set, that will be returned. Otherwise the path will be resolved based on the location of the parent database. The metastore folder hierarchy is: <warehouse directory>="">/<db name>="">.db/

for items in the default database which will be: <warehouse directory>="">/

method handles both of these cases.

Definition at line 205 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Catalog.getMetaStoreClient().

TCatalogObject com.cloudera.impala.catalog.Catalog.getTCatalogObject ( TCatalogObject  objectDesc) throws CatalogException
inlineinherited

Gets the thrift representation of a catalog object, given the "object description". The object description is just a TCatalogObject with only the catalog object type and object name set. If the object is not found, a CatalogException is thrown.

Definition at line 399 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getDataSource(), com.cloudera.impala.catalog.Catalog.getDb(), com.cloudera.impala.catalog.Catalog.getFunction(), com.cloudera.impala.catalog.Catalog.getHdfsCachePool(), com.cloudera.impala.catalog.Role.getName(), com.cloudera.impala.catalog.Role.getPrivileges(), com.cloudera.impala.catalog.Catalog.getTable(), and pool.

boolean com.cloudera.impala.catalog.ImpaladCatalog.isReady ( )
inline

Returns true if the ImpaladCatalog is ready to accept requests (has received and processed a valid catalog topic update from the StateStore), false otherwise.

Definition at line 443 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.analysis.Analyzer.getCatalog().

void com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject ( TCatalogObject  catalogObject,
long  currentCatalogUpdateVersion 
)
inlineprivate

Removes the matching TCatalogObject from the catalog, if one exists and its catalog version is < the catalog version of this drop operation. Note that drop operations that come from statestore heartbeats always have a version of 0. To determine the drop version for statestore updates, the catalog version from the current update is used. This is okay because there can never be a catalog update from the statestore that contains a drop and an addition of the same object. For more details on how drop versioning works, see CatalogServerCatalog.java

Definition at line 290 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.HdfsCachePool.getCatalogVersion(), com.cloudera.impala.catalog.ImpaladCatalog.lastSyncedCatalogVersion_, com.cloudera.impala.catalog.ImpaladCatalog.removeDataSource(), com.cloudera.impala.catalog.ImpaladCatalog.removeDb(), com.cloudera.impala.catalog.ImpaladCatalog.removeFunction(), com.cloudera.impala.catalog.ImpaladCatalog.removePrivilege(), com.cloudera.impala.catalog.ImpaladCatalog.removeRole(), and com.cloudera.impala.catalog.ImpaladCatalog.removeTable().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog().

DataSource com.cloudera.impala.catalog.Catalog.removeDataSource ( String  dataSourceName)
inlineinherited

Removes a data source from the in-memory map of data sources.

Returns
the item that was removed if it existed in the cache, null otherwise.

Definition at line 205 of file Catalog.java.

void com.cloudera.impala.catalog.ImpaladCatalog.removeDataSource ( TDataSource  thrift,
long  dropCatalogVersion 
)
inlineprivate

Definition at line 383 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

Db com.cloudera.impala.catalog.Catalog.removeDb ( String  dbName)
inlineinherited

Removes a database from the metadata cache. Returns the value removed or null if not database was removed as part of this operation. Used by DROP DATABASE statements.

Definition at line 126 of file Catalog.java.

void com.cloudera.impala.catalog.ImpaladCatalog.removeDb ( TDatabase  thriftDb,
long  dropCatalogVersion 
)
inlineprivate

Definition at line 387 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Db.getCatalogVersion(), com.cloudera.impala.catalog.Catalog.getDb(), and com.cloudera.impala.catalog.Db.getName().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

Function com.cloudera.impala.catalog.Catalog.removeFunction ( Function  desc)
inlineinherited

Removes a function from the catalog. Increments the catalog version and returns the Function object that was removed if the function existed, otherwise returns null.

Definition at line 285 of file Catalog.java.

References com.cloudera.impala.catalog.Function.dbName(), and com.cloudera.impala.catalog.Catalog.getDb().

void com.cloudera.impala.catalog.ImpaladCatalog.removeFunction ( TFunction  thriftFn,
long  dropCatalogVersion 
)
inlineprivate

Definition at line 405 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Function.getCatalogVersion(), and com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

void com.cloudera.impala.catalog.ImpaladCatalog.removePrivilege ( TPrivilege  thriftPrivilege,
long  dropCatalogVersion 
)
inlineprivate

Definition at line 426 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.RolePrivilege.getCatalogVersion().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

void com.cloudera.impala.catalog.ImpaladCatalog.removeRole ( TRole  thriftRole,
long  dropCatalogVersion 
)
inlineprivate

Definition at line 418 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Role.getCatalogVersion().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

Table com.cloudera.impala.catalog.Catalog.removeTable ( TTableName  tableName)
inlineinherited

Removes a table from the catalog and returns the table that was removed, or null if the table/database does not exist.

Definition at line 157 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getDb().

void com.cloudera.impala.catalog.ImpaladCatalog.removeTable ( TTable  thriftTable,
long  dropCatalogVersion 
)
inlineprivate

Definition at line 394 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Table.getCatalogVersion(), and com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

void com.cloudera.impala.catalog.ImpaladCatalog.setIsReady ( )
inline

Definition at line 446 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog().

synchronized TUpdateCatalogCacheResponse com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog ( TUpdateCatalogCacheRequest  req) throws CatalogException
inline

Updates the internal Catalog based on the given TCatalogUpdateReq. This method: 1) Updates all databases in the Catalog 2) Updates all tables, views, and functions in the Catalog 3) Removes all dropped tables, views, and functions 4) Removes all dropped databases

This method is called once per statestore heartbeat and is guaranteed the same object will not be in both the "updated" list and the "removed" list (it is a detail handled by the statestore). Catalog updates are ordered by the object type with the dependent objects coming first. That is, database "foo" will always come before table "foo.bar". Synchronized because updateCatalog() can be called by during a statestore update or during a direct-DDL operation and catalogServiceId_ and lastSyncedCatalogVersion_ must be protected.

Definition at line 112 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject(), com.cloudera.impala.catalog.ImpaladCatalog.catalogServiceId_, com.cloudera.impala.catalog.ImpaladCatalog.catalogUpdateEventNotifier_, com.cloudera.impala.catalog.ImpaladCatalog.INITIAL_CATALOG_SERVICE_ID, com.cloudera.impala.catalog.ImpaladCatalog.lastSyncedCatalogVersion_, and com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

void com.cloudera.impala.catalog.ImpaladCatalog.waitForCatalogUpdate ( long  timeoutMs)
inline

Causes the calling thread to wait until a catalog update notification has been sent or the given timeout has been reached. A timeout value of 0 indicates an indefinite wait. Does not protect against spurious wakeups, so this should be called in a loop.

Definition at line 164 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.ImpaladCatalog.catalogUpdateEventNotifier_.

Member Data Documentation

AuthorizationPolicy com.cloudera.impala.catalog.Catalog.authPolicy_ = new AuthorizationPolicy()
protectedinherited

Definition at line 67 of file Catalog.java.

Referenced by com.cloudera.impala.testutil.CatalogServiceTestCatalog.getAuthPolicy(), com.cloudera.impala.catalog.ImpaladCatalog.getAuthPolicy(), com.cloudera.impala.catalog.CatalogServiceCatalog.getAuthPolicy(), com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), and com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog().

final String com.cloudera.impala.catalog.Catalog.BUILTINS_DB = "_impala_builtins"
staticinherited

Definition at line 61 of file Catalog.java.

Referenced by com.cloudera.impala.analysis.ExtractFromExpr.analyze(), com.cloudera.impala.analysis.FunctionName.analyze(), com.cloudera.impala.analysis.CreateFunctionStmtBase.analyze(), com.cloudera.impala.analysis.CastExpr.analyze(), com.cloudera.impala.catalog.Catalog.Catalog(), com.cloudera.impala.catalog.ScalarFunction.createBuiltin(), com.cloudera.impala.catalog.ScalarFunction.createBuiltinSearchDesc(), and com.cloudera.impala.analysis.Expr.getBuiltinFunction().

final CatalogDeltaLog com.cloudera.impala.catalog.ImpaladCatalog.catalogDeltaLog_ = new CatalogDeltaLog()
private

Definition at line 82 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject().

TUniqueId com.cloudera.impala.catalog.ImpaladCatalog.catalogServiceId_ = INITIAL_CATALOG_SERVICE_ID
private

Definition at line 70 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog().

final Object com.cloudera.impala.catalog.ImpaladCatalog.catalogUpdateEventNotifier_ = new Object()
private

Definition at line 85 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog(), and com.cloudera.impala.catalog.ImpaladCatalog.waitForCatalogUpdate().

final CatalogObjectCache<DataSource> com.cloudera.impala.catalog.Catalog.dataSources_
protectedinherited

Definition at line 80 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.addDataSource(), com.cloudera.impala.catalog.Catalog.Catalog(), com.cloudera.impala.catalog.Catalog.getDataSourceNames(), and com.cloudera.impala.catalog.Catalog.getDataSources().

AtomicReference<ConcurrentHashMap<String, Db> > com.cloudera.impala.catalog.Catalog.dbCache_
protectedinherited
Initial value:
=
new AtomicReference<ConcurrentHashMap<String, Db>>(
new ConcurrentHashMap<String, Db>())

Definition at line 72 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.Catalog.getDbNames(), and com.cloudera.impala.catalog.CatalogServiceCatalog.reset().

final String com.cloudera.impala.catalog.Catalog.DEFAULT_DB = "default"
staticinherited

Definition at line 58 of file Catalog.java.

Referenced by com.cloudera.impala.analysis.AnalyzerTest.AnalysisError(), com.cloudera.impala.analysis.UseStmt.analyze(), com.cloudera.impala.analysis.AuditingTest.AnalyzeAccessEvents(), com.cloudera.impala.analysis.AnalyzeAuthStmtsTest.AnalyzeAuthStmtsTest(), com.cloudera.impala.analysis.AnalyzeAuthStmtsTest.AnalyzeCreateDropRole(), com.cloudera.impala.analysis.AnalyzeAuthStmtsTest.AnalyzeGrantRevokePriv(), com.cloudera.impala.analysis.AnalyzeAuthStmtsTest.AnalyzeGrantRevokeRole(), com.cloudera.impala.analysis.AnalyzeAuthStmtsTest.AnalyzeShowGrantRole(), com.cloudera.impala.analysis.AnalyzeAuthStmtsTest.AnalyzeShowRoles(), com.cloudera.impala.analysis.AnalyzerTest.AnalyzesOk(), com.cloudera.impala.analysis.AuthorizationTest.AuthorizationTest(), com.cloudera.impala.analysis.AnalyzerTest.createAnalyzerUsingHiveColLabels(), com.cloudera.impala.testutil.TestUtils.createQueryContext(), com.cloudera.impala.analysis.AuthorizationTest.TestFunction(), com.cloudera.impala.analysis.AuthorizationTest.TestLocalGroupPolicyProvider(), com.cloudera.impala.analysis.AuthorizationTest.TestShortUsernameUsed(), and com.cloudera.impala.analysis.AuthorizationTest.TestWithIncorrectConfig().

final CatalogObjectCache<HdfsCachePool> com.cloudera.impala.catalog.Catalog.hdfsCachePools_
protectedinherited
Initial value:
=
new CatalogObjectCache<HdfsCachePool>(false)

Definition at line 84 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), and com.cloudera.impala.catalog.CatalogServiceCatalog.CachePoolReader.run().

final TUniqueId com.cloudera.impala.catalog.ImpaladCatalog.INITIAL_CATALOG_SERVICE_ID = new TUniqueId(0L, 0L)
staticprivate

Definition at line 66 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog().

final long com.cloudera.impala.catalog.Catalog.INITIAL_CATALOG_VERSION = 0L
staticinherited

Definition at line 57 of file Catalog.java.

Referenced by com.cloudera.impala.service.CatalogOpExecutor.createDatabase(), com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), and com.cloudera.impala.service.CatalogOpExecutor.TableToTCatalogObject().

final AtomicBoolean com.cloudera.impala.catalog.ImpaladCatalog.isReady_ = new AtomicBoolean(false)
private

Definition at line 79 of file ImpaladCatalog.java.

long com.cloudera.impala.catalog.ImpaladCatalog.lastSyncedCatalogVersion_ = Catalog.INITIAL_CATALOG_VERSION
private

Definition at line 76 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject(), and com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog().

final Logger com.cloudera.impala.catalog.ImpaladCatalog.LOG = Logger.getLogger(ImpaladCatalog.class)
staticprivate

Definition at line 65 of file ImpaladCatalog.java.

final MetaStoreClientPool com.cloudera.impala.catalog.Catalog.metaStoreClientPool_ = new MetaStoreClientPool(0)
protectedinherited

Definition at line 63 of file Catalog.java.

The documentation for this class was generated from the following file: