Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros
Public Member Functions | Private Attributes | List of all members
impala::Statestore::Topic Class Reference
Collaboration diagram for impala::Statestore::Topic:

Public Member Functions

 Topic (const TopicId &topic_id, IntGauge *key_size_metric, IntGauge *value_size_metric, IntGauge *topic_size_metric)
 
TopicEntry::Version Put (const TopicEntryKey &key, const TopicEntry::Value &bytes)
 Must be called holding the topic lock. More...
 
void DeleteIfVersionsMatch (TopicEntry::Version version, const TopicEntryKey &key)
 Must be called holding the topic lock. More...
 
const TopicId & id () const
 
const TopicEntryMap & entries () const
 
TopicEntry::Version last_version () const
 
const TopicUpdateLog & topic_update_log () const
 
int64_t total_key_size_bytes () const
 
int64_t total_value_size_bytes () const
 

Private Attributes

TopicEntryMap entries_
 Map from topic entry key to topic entry. More...
 
const TopicId topic_id_
 Unique identifier for this topic. Should be human-readable. More...
 
TopicEntry::Version last_version_
 
TopicUpdateLog topic_update_log_
 
int64_t total_key_size_bytes_
 Total memory occupied by the key strings, in bytes. More...
 
int64_t total_value_size_bytes_
 Total memory occupied by the value byte strings, in bytes. More...
 
IntGauge * key_size_metric_
 Metrics shared across all topics to sum the size in bytes of keys, values and both. More...
 
IntGauge * value_size_metric_
 
IntGauge * topic_size_metric_
 

Detailed Description

A Topic is logically a map between a string key and a sequence of bytes. A <string, bytes> pair is a TopicEntry. Each topic has a unique version number that tracks the number of updates made to the topic. This is to support sending only the delta of changes on every update.

Definition at line 178 of file statestore.h.

Constructor & Destructor Documentation

impala::Statestore::Topic::Topic ( const TopicId &  topic_id,
IntGauge *  key_size_metric,
IntGauge *  value_size_metric,
IntGauge *  topic_size_metric 
)
inline

Definition at line 180 of file statestore.h.

Member Function Documentation

void Statestore::Topic::DeleteIfVersionsMatch ( TopicEntry::Version  version,
const TopicEntryKey &  key 
)

Must be called holding the topic lock.

Utility method to support removing transient entries. We track the version numbers of entries added by subscribers, and remove entries with the same version number when that subscriber fails (the same entry may exist, but may have been updated by another subscriber giving it a new version number) Deletion means setting the entry's value to NULL and incrementing its version number.

Definition at line 158 of file statestore.cc.

References NULL_VALUE.

const TopicEntryMap& impala::Statestore::Topic::entries ( ) const
inline

Definition at line 205 of file statestore.h.

References entries_.

Referenced by impala::Statestore::GatherTopicUpdates().

const TopicId& impala::Statestore::Topic::id ( ) const
inline

Definition at line 204 of file statestore.h.

References topic_id_.

TopicEntry::Version impala::Statestore::Topic::last_version ( ) const
inline

Definition at line 206 of file statestore.h.

References last_version_.

Referenced by impala::Statestore::GatherTopicUpdates().

Statestore::TopicEntry::Version Statestore::Topic::Put ( const TopicEntryKey &  key,
const TopicEntry::Value &  bytes 
)

Must be called holding the topic lock.

Adds an entry with the given key. If bytes == NULL_VALUE, the entry is considered deleted, and may be garbage collected in the future. The entry is assigned a new version number by the Topic, and that version number is returned.

Definition at line 127 of file statestore.cc.

Referenced by impala::Statestore::SendTopicUpdate().

const TopicUpdateLog& impala::Statestore::Topic::topic_update_log ( ) const
inline

Definition at line 207 of file statestore.h.

References topic_update_log_.

Referenced by impala::Statestore::GatherTopicUpdates().

int64_t impala::Statestore::Topic::total_key_size_bytes ( ) const
inline

Definition at line 208 of file statestore.h.

References total_key_size_bytes_.

Referenced by impala::Statestore::GatherTopicUpdates().

int64_t impala::Statestore::Topic::total_value_size_bytes ( ) const
inline

Definition at line 209 of file statestore.h.

References total_value_size_bytes_.

Referenced by impala::Statestore::GatherTopicUpdates().

Member Data Documentation

TopicEntryMap impala::Statestore::Topic::entries_
private

Map from topic entry key to topic entry.

Definition at line 213 of file statestore.h.

Referenced by entries().

IntGauge* impala::Statestore::Topic::key_size_metric_
private

Metrics shared across all topics to sum the size in bytes of keys, values and both.

Definition at line 239 of file statestore.h.

TopicEntry::Version impala::Statestore::Topic::last_version_
private

Tracks the last version that was assigned to an entry in this Topic. Incremented on every Put() so each TopicEntry is tagged with a unique version value.

Definition at line 220 of file statestore.h.

Referenced by last_version().

const TopicId impala::Statestore::Topic::topic_id_
private

Unique identifier for this topic. Should be human-readable.

Definition at line 216 of file statestore.h.

Referenced by id().

IntGauge* impala::Statestore::Topic::topic_size_metric_
private

Definition at line 241 of file statestore.h.

TopicUpdateLog impala::Statestore::Topic::topic_update_log_
private

Contains a history of updates to this Topic, with each key being a Version and the value being a TopicEntryKey. Used to look up the TopicEntry in the entries_ map corresponding to each version update. TODO: Looking up a TopicEntry from the topic_update_log_ requires two reads - one to get the TopicEntryKey and another to use that key to look up the corresponding TopicEntry in the entries_ map. Based on performance needs, we may need to revisit this to find a way to do the look up using a single read.

Definition at line 230 of file statestore.h.

Referenced by topic_update_log().

int64_t impala::Statestore::Topic::total_key_size_bytes_
private

Total memory occupied by the key strings, in bytes.

Definition at line 233 of file statestore.h.

Referenced by total_key_size_bytes().

int64_t impala::Statestore::Topic::total_value_size_bytes_
private

Total memory occupied by the value byte strings, in bytes.

Definition at line 236 of file statestore.h.

Referenced by total_value_size_bytes().

IntGauge* impala::Statestore::Topic::value_size_metric_
private

Definition at line 240 of file statestore.h.

The documentation for this class was generated from the following files: