Inheritance diagram for com.cloudera.impala.testutil.ImpaladTestCatalog:

Collaboration diagram for com.cloudera.impala.testutil.ImpaladTestCatalog:

Public Member Functions
	ImpaladTestCatalog ()

	ImpaladTestCatalog (AuthorizationConfig authzConfig)

HdfsCachePool	getHdfsCachePool (String poolName)

void	reset () throws CatalogException

Table	getTable (String dbName, String tableName) throws CatalogException

synchronized TUpdateCatalogCacheResponse	updateCatalog (TUpdateCatalogCacheRequest req) throws CatalogException

void	waitForCatalogUpdate (long timeoutMs)

Path	getTablePath (org.apache.hadoop.hive.metastore.api.Table msTbl) throws NoSuchObjectException, MetaException, TException

Db	addDb (Db db)

boolean	addFunction (Function fn)

boolean	addDataSource (DataSource dataSource)

DataSource	removeDataSource (String dataSourceName)

Db	removeDb (String dbName)

Table	removeTable (TTableName tableName)

Function	removeFunction (Function desc)

boolean	isReady ()

void	setIsReady ()

AuthorizationPolicy	getAuthPolicy ()

Db	getBuiltinsDb ()

Db	getDb (String dbName)

List< String >	getDbNames (String dbPattern)

List< String >	getTableNames (String dbName, String tablePattern) throws DatabaseNotFoundException

boolean	containsTable (String dbName, String tableName)

DataSource	getDataSource (String dataSourceName)

List< DataSource >	getDataSources ()

List< DataSource >	getDataSources (String pattern)

List< String >	getDataSourceNames (String pattern)

Function	getFunction (Function desc, Function.CompareMode mode)

boolean	containsFunction (FunctionName name)

boolean	addHdfsCachePool (HdfsCachePool cachePool)

void	close ()

MetaStoreClient	getMetaStoreClient ()

HdfsPartition	getHdfsPartition (String dbName, String tableName, List< TPartitionKeyValue > partitionSpec) throws CatalogException

boolean	containsHdfsPartition (String dbName, String tableName, List< TPartitionKeyValue > partitionSpec) throws CatalogException

TCatalogObject	getTCatalogObject (TCatalogObject objectDesc) throws CatalogException

Static Public Member Functions
static Function	getBuiltin (Function desc, Function.CompareMode mode)

Static Public Attributes
static final long	INITIAL_CATALOG_VERSION = 0L

static final String	DEFAULT_DB = "default"

static final String	BUILTINS_DB = "_impala_builtins"

Protected Attributes
final MetaStoreClientPool	metaStoreClientPool_ = new MetaStoreClientPool(0)

AuthorizationPolicy	authPolicy_ = new AuthorizationPolicy()

AtomicReference < ConcurrentHashMap< String, Db > >	dbCache_

final CatalogObjectCache < DataSource >	dataSources_

final CatalogObjectCache < HdfsCachePool >	hdfsCachePools_

Private Attributes
final CatalogServiceCatalog	srcCatalog_

Detailed Description

Mock catalog used for running FE tests that allows lazy-loading of tables without a running catalogd/statestored.

Definition at line 30 of file ImpaladTestCatalog.java.

Constructor & Destructor Documentation

com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog ( )

inline

Definition at line 34 of file ImpaladTestCatalog.java.

com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog ( AuthorizationConfig authzConfig )

inline

Takes an AuthorizationConfig to bootstrap the backing CatalogServiceCatalog.

Definition at line 41 of file ImpaladTestCatalog.java.

References com.cloudera.impala.catalog.ImpaladCatalog.addDb(), com.cloudera.impala.catalog.Catalog.authPolicy_, com.cloudera.impala.catalog.Catalog.getDb(), com.cloudera.impala.catalog.Catalog.getDbNames(), com.cloudera.impala.catalog.ImpaladCatalog.setIsReady(), and com.cloudera.impala.testutil.ImpaladTestCatalog.srcCatalog_.

Member Function Documentation

boolean com.cloudera.impala.catalog.Catalog.addDataSource ( DataSource dataSource )

inlineinherited

Adds a data source to the in-memory map of data sources. It is not persisted to the metastore.

Returns: true if this item was added or false if the existing value was preserved.

Definition at line 197 of file Catalog.java.

Db com.cloudera.impala.catalog.Catalog.addDb ( Db db )

inlineinherited

Adds a new database to the catalog, replacing any existing database with the same name. Returns the previous database with this name, or null if there was no previous database.

Definition at line 107 of file Catalog.java.

References com.cloudera.impala.catalog.Db.getName().

Referenced by com.cloudera.impala.catalog.Catalog.Catalog().

boolean com.cloudera.impala.catalog.Catalog.addFunction ( Function fn )

inlineinherited

Adds a function to the catalog. Returns true if the function was successfully added. Returns false if the function already exists. TODO: allow adding a function to a global scope. We probably want this to resolve after the local scope. e.g. if we had fn() and db.fn(). If the current database is 'db', fn() would resolve first to db.fn().

Definition at line 259 of file Catalog.java.

References com.cloudera.impala.catalog.Function.dbName(), and com.cloudera.impala.catalog.Catalog.getDb().

boolean com.cloudera.impala.catalog.Catalog.addHdfsCachePool ( HdfsCachePool cachePool )

inlineinherited

Adds a new HdfsCachePool to the catalog.

Definition at line 304 of file Catalog.java.

void com.cloudera.impala.catalog.Catalog.close ( )

inlineinherited

Release the Hive Meta Store Client resources. Can be called multiple times (additional calls will be no-ops).

Definition at line 320 of file Catalog.java.

boolean com.cloudera.impala.catalog.Catalog.containsFunction ( FunctionName name )

inlineinherited

Returns true if there is a function with this function name. Parameters are ignored.

Definition at line 295 of file Catalog.java.

References com.cloudera.impala.analysis.FunctionName.getDb(), and com.cloudera.impala.catalog.Catalog.getDb().

boolean com.cloudera.impala.catalog.Catalog.containsHdfsPartition	(	String	dbName,
		String	tableName,
		List< TPartitionKeyValue >	partitionSpec
	)		throws CatalogException

inlineinherited

Returns true if the table contains the given partition spec, otherwise false. This may trigger a metadata load if the table metadata is not yet cached.

Exceptions

DatabaseNotFoundException	- If the database does not exist.
TableNotFoundException	- If the table does not exist.
TableLoadingException	- If there is an error loading the table metadata.

Definition at line 384 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getHdfsPartition().

Referenced by com.cloudera.impala.service.CatalogOpExecutor.alterTableAddPartition(), and com.cloudera.impala.service.CatalogOpExecutor.alterTableDropPartition().

boolean com.cloudera.impala.catalog.Catalog.containsTable	(	String	dbName,
		String	tableName
	)

inlineinherited

Returns true if the table and the database exist in the Impala Catalog. Returns false if either the table or the database do not exist.

Definition at line 187 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.service.CatalogOpExecutor.createTable(), com.cloudera.impala.service.CatalogOpExecutor.createTableLike(), and com.cloudera.impala.service.CatalogOpExecutor.createView().

AuthorizationPolicy com.cloudera.impala.catalog.ImpaladCatalog.getAuthPolicy ( )

inlineinherited

Definition at line 447 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Catalog.authPolicy_.

Referenced by com.cloudera.impala.service.Frontend.Frontend(), and com.cloudera.impala.service.Frontend.updateCatalogCache().

static Function com.cloudera.impala.catalog.Catalog.getBuiltin	(	Function	desc,
		Function.CompareMode	mode
	)

inlinestaticinherited

Definition at line 276 of file Catalog.java.

Db com.cloudera.impala.catalog.Catalog.getBuiltinsDb ( )

inlineinherited

Definition at line 100 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.builtinsDb_.

DataSource com.cloudera.impala.catalog.Catalog.getDataSource ( String dataSourceName )

inlineinherited

Gets the specified data source.

Definition at line 213 of file Catalog.java.

Referenced by com.cloudera.impala.service.CatalogOpExecutor.createDataSource(), and com.cloudera.impala.catalog.Catalog.getTCatalogObject().

List<String> com.cloudera.impala.catalog.Catalog.getDataSourceNames ( String pattern )

inlineinherited

Returns a list of data sources names that match pattern. See filterStringsByPattern for details of the pattern match semantics.

pattern may be null (and thus matches everything).

Definition at line 231 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.dataSources_, and com.cloudera.impala.catalog.Catalog.filterStringsByPattern().

List<DataSource> com.cloudera.impala.catalog.Catalog.getDataSources ( )

inlineinherited

Gets a list of all data sources.

Definition at line 221 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects().

List<DataSource> com.cloudera.impala.catalog.Catalog.getDataSources ( String pattern )

inlineinherited

Returns a list of data sources that match pattern. See filterStringsByPattern for details of the pattern match semantics.

pattern may be null (and thus matches everything).

Definition at line 241 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.dataSources_, com.cloudera.impala.catalog.Catalog.filterStringsByPattern(), and impala.name.

Db com.cloudera.impala.catalog.Catalog.getDb ( String dbName )

inlineinherited

Gets the Db object from the Catalog using a case-insensitive lookup on the name. Returns null if no matching database is found.

Definition at line 115 of file Catalog.java.

List<String> com.cloudera.impala.catalog.Catalog.getDbNames ( String dbPattern )

inlineinherited

Returns a list of databases that match dbPattern. See filterStringsByPattern for details of the pattern match semantics.

dbPattern may be null (and thus matches everything).

Definition at line 136 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.dbCache_, and com.cloudera.impala.catalog.Catalog.filterStringsByPattern().

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog(), and com.cloudera.impala.testutil.BlockIdGenerator.main().

Function com.cloudera.impala.catalog.Catalog.getFunction	(	Function	desc,
		Function.CompareMode	mode
	)

inlineinherited

Returns the function that best matches 'desc' that is registered with the catalog using 'mode' to check for matching. If desc matches multiple functions in the catalog, it will return the function with the strictest matching mode.

Definition at line 270 of file Catalog.java.

References com.cloudera.impala.catalog.Function.dbName(), and com.cloudera.impala.catalog.Catalog.getDb().

Referenced by com.cloudera.impala.catalog.Catalog.getTCatalogObject().

HdfsCachePool com.cloudera.impala.testutil.ImpaladTestCatalog.getHdfsCachePool ( String poolName )

inline

Definition at line 56 of file ImpaladTestCatalog.java.

HdfsPartition com.cloudera.impala.catalog.Catalog.getHdfsPartition	(	String	dbName,
		String	tableName,
		List< TPartitionKeyValue >	partitionSpec
	)		throws CatalogException

inlineinherited

Returns the HdfsPartition object for the given dbName/tableName and partition spec. This will trigger a metadata load if the table metadata is not yet cached.

Exceptions

DatabaseNotFoundException	- If the database does not exist.
TableNotFoundException	- If the table does not exist.
PartitionNotFoundException	- If the partition does not exist.
TableLoadingException	- If there is an error loading the table metadata.

Definition at line 361 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getTable().

Referenced by com.cloudera.impala.catalog.Catalog.containsHdfsPartition().

MetaStoreClient com.cloudera.impala.catalog.Catalog.getMetaStoreClient ( )

inlineinherited

Returns a managed meta store client from the client connection pool.

Definition at line 326 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.getTablePath(), and com.cloudera.impala.catalog.CatalogServiceCatalog.invalidateTable().

Table com.cloudera.impala.testutil.ImpaladTestCatalog.getTable	(	String	dbName,
		String	tableName
	)		throws CatalogException

inline

Overrides ImpaladCatalog.getTable to load the table metadata if it is missing.

Definition at line 71 of file ImpaladTestCatalog.java.

References com.cloudera.impala.catalog.Catalog.getDb(), and com.cloudera.impala.catalog.Table.isLoaded().

List<String> com.cloudera.impala.catalog.Catalog.getTableNames	(	String	dbName,
		String	tablePattern
	)		throws DatabaseNotFoundException

inlineinherited

Returns a list of tables in the supplied database that match tablePattern. See filterStringsByPattern for details of the pattern match semantics.

dbName must not be null, but tablePattern may be null (and thus matches everything).

Table names are returned unqualified.

Definition at line 173 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.filterStringsByPattern(), and com.cloudera.impala.catalog.Catalog.getDb().

Path com.cloudera.impala.catalog.ImpaladCatalog.getTablePath ( org.apache.hadoop.hive.metastore.api.Table msTbl ) throws NoSuchObjectException, MetaException, TException

inlineinherited

Returns the HDFS path where the metastore would create the given table. If the table has a "location" set, that will be returned. Otherwise the path will be resolved based on the location of the parent database. The metastore folder hierarchy is: <warehouse directory>="">/<db name>="">.db/

for items in the default database which will be: <warehouse directory>="">/

method handles both of these cases.

Definition at line 205 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.Catalog.getMetaStoreClient().

TCatalogObject com.cloudera.impala.catalog.Catalog.getTCatalogObject ( TCatalogObject objectDesc ) throws CatalogException

inlineinherited

Gets the thrift representation of a catalog object, given the "object description". The object description is just a TCatalogObject with only the catalog object type and object name set. If the object is not found, a CatalogException is thrown.

Definition at line 399 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getDataSource(), com.cloudera.impala.catalog.Catalog.getDb(), com.cloudera.impala.catalog.Catalog.getFunction(), com.cloudera.impala.catalog.Catalog.getHdfsCachePool(), com.cloudera.impala.catalog.Role.getName(), com.cloudera.impala.catalog.Role.getPrivileges(), com.cloudera.impala.catalog.Catalog.getTable(), and pool.

boolean com.cloudera.impala.catalog.ImpaladCatalog.isReady ( )

inlineinherited

Returns true if the ImpaladCatalog is ready to accept requests (has received and processed a valid catalog topic update from the StateStore), false otherwise.

Definition at line 443 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.analysis.Analyzer.getCatalog().

DataSource com.cloudera.impala.catalog.Catalog.removeDataSource ( String dataSourceName )

inlineinherited

Removes a data source from the in-memory map of data sources.

Returns: the item that was removed if it existed in the cache, null otherwise.

Definition at line 205 of file Catalog.java.

Db com.cloudera.impala.catalog.Catalog.removeDb ( String dbName )

inlineinherited

Removes a database from the metadata cache. Returns the value removed or null if not database was removed as part of this operation. Used by DROP DATABASE statements.

Definition at line 126 of file Catalog.java.

Function com.cloudera.impala.catalog.Catalog.removeFunction ( Function desc )

inlineinherited

Removes a function from the catalog. Increments the catalog version and returns the Function object that was removed if the function existed, otherwise returns null.

Definition at line 285 of file Catalog.java.

References com.cloudera.impala.catalog.Function.dbName(), and com.cloudera.impala.catalog.Catalog.getDb().

Table com.cloudera.impala.catalog.Catalog.removeTable ( TTableName tableName )

inlineinherited

Removes a table from the catalog and returns the table that was removed, or null if the table/database does not exist.

Definition at line 157 of file Catalog.java.

References com.cloudera.impala.catalog.Catalog.getDb().

void com.cloudera.impala.testutil.ImpaladTestCatalog.reset ( ) throws CatalogException

inline

Reloads all metadata from the source catalog.

Definition at line 63 of file ImpaladTestCatalog.java.

void com.cloudera.impala.catalog.ImpaladCatalog.setIsReady ( )

inlineinherited

Definition at line 446 of file ImpaladCatalog.java.

Referenced by com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog().

synchronized TUpdateCatalogCacheResponse com.cloudera.impala.catalog.ImpaladCatalog.updateCatalog ( TUpdateCatalogCacheRequest req ) throws CatalogException

inlineinherited

Updates the internal Catalog based on the given TCatalogUpdateReq. This method: 1) Updates all databases in the Catalog 2) Updates all tables, views, and functions in the Catalog 3) Removes all dropped tables, views, and functions 4) Removes all dropped databases

This method is called once per statestore heartbeat and is guaranteed the same object will not be in both the "updated" list and the "removed" list (it is a detail handled by the statestore). Catalog updates are ordered by the object type with the dependent objects coming first. That is, database "foo" will always come before table "foo.bar". Synchronized because updateCatalog() can be called by during a statestore update or during a direct-DDL operation and catalogServiceId_ and lastSyncedCatalogVersion_ must be protected.

Definition at line 112 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject(), com.cloudera.impala.catalog.ImpaladCatalog.catalogServiceId_, com.cloudera.impala.catalog.ImpaladCatalog.catalogUpdateEventNotifier_, com.cloudera.impala.catalog.ImpaladCatalog.INITIAL_CATALOG_SERVICE_ID, com.cloudera.impala.catalog.ImpaladCatalog.lastSyncedCatalogVersion_, and com.cloudera.impala.catalog.ImpaladCatalog.removeCatalogObject().

void com.cloudera.impala.catalog.ImpaladCatalog.waitForCatalogUpdate ( long timeoutMs )

inlineinherited

Causes the calling thread to wait until a catalog update notification has been sent or the given timeout has been reached. A timeout value of 0 indicates an indefinite wait. Does not protect against spurious wakeups, so this should be called in a loop.

Definition at line 164 of file ImpaladCatalog.java.

References com.cloudera.impala.catalog.ImpaladCatalog.catalogUpdateEventNotifier_.

Member Data Documentation

AuthorizationPolicy com.cloudera.impala.catalog.Catalog.authPolicy_ = new AuthorizationPolicy()

protectedinherited

Definition at line 67 of file Catalog.java.

Referenced by com.cloudera.impala.testutil.CatalogServiceTestCatalog.getAuthPolicy(), com.cloudera.impala.catalog.ImpaladCatalog.getAuthPolicy(), com.cloudera.impala.catalog.CatalogServiceCatalog.getAuthPolicy(), com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), and com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog().

final String com.cloudera.impala.catalog.Catalog.BUILTINS_DB = "_impala_builtins"

staticinherited

Definition at line 61 of file Catalog.java.

Referenced by com.cloudera.impala.analysis.ExtractFromExpr.analyze(), com.cloudera.impala.analysis.FunctionName.analyze(), com.cloudera.impala.analysis.CreateFunctionStmtBase.analyze(), com.cloudera.impala.analysis.CastExpr.analyze(), com.cloudera.impala.catalog.Catalog.Catalog(), com.cloudera.impala.catalog.ScalarFunction.createBuiltin(), com.cloudera.impala.catalog.ScalarFunction.createBuiltinSearchDesc(), and com.cloudera.impala.analysis.Expr.getBuiltinFunction().

final CatalogObjectCache<DataSource> com.cloudera.impala.catalog.Catalog.dataSources_

protectedinherited

Definition at line 80 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.addDataSource(), com.cloudera.impala.catalog.Catalog.Catalog(), com.cloudera.impala.catalog.Catalog.getDataSourceNames(), and com.cloudera.impala.catalog.Catalog.getDataSources().

AtomicReference<ConcurrentHashMap<String, Db> > com.cloudera.impala.catalog.Catalog.dbCache_

protectedinherited

Initial value:

=
      new AtomicReference<ConcurrentHashMap<String, Db>>(
          new ConcurrentHashMap<String, Db>())

Definition at line 72 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.Catalog.getDbNames(), and com.cloudera.impala.catalog.CatalogServiceCatalog.reset().

final String com.cloudera.impala.catalog.Catalog.DEFAULT_DB = "default"

staticinherited

Definition at line 58 of file Catalog.java.

final CatalogObjectCache<HdfsCachePool> com.cloudera.impala.catalog.Catalog.hdfsCachePools_

protectedinherited

Initial value:

=

new CatalogObjectCache<HdfsCachePool>(false)

Definition at line 84 of file Catalog.java.

Referenced by com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), and com.cloudera.impala.catalog.CatalogServiceCatalog.CachePoolReader.run().

final long com.cloudera.impala.catalog.Catalog.INITIAL_CATALOG_VERSION = 0L

staticinherited

Definition at line 57 of file Catalog.java.

Referenced by com.cloudera.impala.service.CatalogOpExecutor.createDatabase(), com.cloudera.impala.catalog.CatalogServiceCatalog.getCatalogObjects(), and com.cloudera.impala.service.CatalogOpExecutor.TableToTCatalogObject().

final MetaStoreClientPool com.cloudera.impala.catalog.Catalog.metaStoreClientPool_ = new MetaStoreClientPool(0)

protectedinherited

Definition at line 63 of file Catalog.java.

final CatalogServiceCatalog com.cloudera.impala.testutil.ImpaladTestCatalog.srcCatalog_

private

Definition at line 32 of file ImpaladTestCatalog.java.

Referenced by com.cloudera.impala.testutil.ImpaladTestCatalog.ImpaladTestCatalog().

The documentation for this class was generated from the following file:

fe/src/test/java/com/cloudera/impala/testutil/ImpaladTestCatalog.java

Public Member Functions

Static Public Member Functions

Static Public Attributes

Protected Attributes

Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation