com.cloudera.impala.catalog.CatalogDeltaLog Class Reference

Collaboration diagram for com.cloudera.impala.catalog.CatalogDeltaLog:

Public Member Functions
synchronized void	addRemovedObject (TCatalogObject catalogObject)
synchronized void	garbageCollect (long currentCatalogVersion)
synchronized boolean	wasObjectRemovedAfter (TCatalogObject catalogObject)

Private Member Functions
boolean	objectNamesMatch (TCatalogObject first, TCatalogObject second)

Private Attributes
SortedMap< Long, TCatalogObject >	removedCatalogObjects_

Detailed Description

The impalad catalog cache can be modified by either a state store update or by a direct ("fast") update that applies the result of a catalog operation to the cache out-of-band of a state store update. This thread safe log tracks the divergence (due to direct updates to the cache) of this impalad's cache from the last state store update. This log is needed to ensure work is never undone. For example, consider the following sequence of events: t1: [Direct Update] - Add item A - (Catalog Version 9) t2: [Direct Update] - Drop item A - (Catalog Version 10) t3: [StateStore Update] - (From Catalog Version 9) This log is used to ensure the state store update in t3 does not undo the drop in t2.

Currently this only tracks objects that were dropped, since the catalog cache can be queried to check if an object was added. TODO: Also track object additions from async operations. This could be used to to "replay" the log in the case of a catalog reset ("invalidate metadata"). Currently, the catalog may briefly go back in time if "invalidate metadata" is run concurrently with async catalog operations.

Definition at line 43 of file CatalogDeltaLog.java.

Member Function Documentation

synchronized void com.cloudera.impala.catalog.CatalogDeltaLog.addRemovedObject ( TCatalogObject  catalogObject )

inline

Adds a new item to the map of removed catalog objects.

Definition at line 52 of file CatalogDeltaLog.java.

synchronized void com.cloudera.impala.catalog.CatalogDeltaLog.garbageCollect ( long  currentCatalogVersion )

inline

Given the current catalog version, removes all items with catalogVersion < currectCatalogVersion. Such objects do not need to be tracked in the delta log anymore because they are consistent with the state store's view of the catalog.

Definition at line 63 of file CatalogDeltaLog.java.

References com.cloudera.impala.catalog.CatalogDeltaLog.removedCatalogObjects_.

boolean com.cloudera.impala.catalog.CatalogDeltaLog.objectNamesMatch ( TCatalogObject  first, TCatalogObject  second  )

inlineprivate

Returns true if the two objects have the same object type and name. TODO: Use global object IDs everywhere instead of tracking catalog objects by name.

Definition at line 94 of file CatalogDeltaLog.java.

Referenced by com.cloudera.impala.catalog.CatalogDeltaLog.wasObjectRemovedAfter().

synchronized boolean com.cloudera.impala.catalog.CatalogDeltaLog.wasObjectRemovedAfter ( TCatalogObject  catalogObject )

inline

Checks if a matching catalog object was removed in a catalog version after this object's catalog version. Returns true if there was a matching object that was removed after this object, false otherwise.

Definition at line 77 of file CatalogDeltaLog.java.

References com.cloudera.impala.catalog.CatalogDeltaLog.objectNamesMatch(), and com.cloudera.impala.catalog.CatalogDeltaLog.removedCatalogObjects_.

Referenced by com.cloudera.impala.catalog.ImpaladCatalog.addCatalogObject().

Member Data Documentation

SortedMap<Long, TCatalogObject> com.cloudera.impala.catalog.CatalogDeltaLog.removedCatalogObjects_

private

Initial value:

=
      new TreeMap<Long, TCatalogObject>()

Definition at line 46 of file CatalogDeltaLog.java.

Referenced by com.cloudera.impala.catalog.CatalogDeltaLog.garbageCollect(), and com.cloudera.impala.catalog.CatalogDeltaLog.wasObjectRemovedAfter().

---

The documentation for this class was generated from the following file:

• fe/src/main/java/com/cloudera/impala/catalog/CatalogDeltaLog.java