#include <periodic-counter-updater.h>

Collaboration diagram for impala::PeriodicCounterUpdater:

Classes
struct	BucketCountersInfo

struct	RateCounterInfo

struct	SamplingCounterInfo

Public Types
enum	PeriodicCounterType { RATE_COUNTER = 0, SAMPLING_COUNTER }

Public Member Functions
	~PeriodicCounterUpdater ()
	Tears down the update thread. More...

Static Public Member Functions
static void	RegisterPeriodicCounter (RuntimeProfile::Counter src_counter, RuntimeProfile::DerivedCounterFunction sample_fn, RuntimeProfile::Counter dst_counter, PeriodicCounterType type)

static void	RegisterBucketingCounters (RuntimeProfile::Counter src_counter, std::vector< RuntimeProfile::Counter > *buckets)
	Adds a bucketing counter to be updated at regular intervals. More...

static void	RegisterTimeSeriesCounter (RuntimeProfile::TimeSeriesCounter *counter)
	Adds counter to be sampled and updated at regular intervals. More...

static void	StopRateCounter (RuntimeProfile::Counter *counter)
	Stops updating the value of 'counter'. More...

static void	StopSamplingCounter (RuntimeProfile::Counter *counter)
	Stops updating the value of 'counter'. More...

static void	StopBucketingCounters (std::vector< RuntimeProfile::Counter * > *buckets, bool convert)

static void	StopTimeSeriesCounter (RuntimeProfile::TimeSeriesCounter *counter)
	Stops 'counter' from receiving any more samples. More...

Private Types
typedef boost::unordered_map < RuntimeProfile::Counter *, RateCounterInfo >	RateCounterMap
	A map of the dst (rate) counter to the src counter and elapsed time. More...

typedef boost::unordered_map < RuntimeProfile::Counter *, SamplingCounterInfo >	SamplingCounterMap

typedef boost::unordered_map < std::vector < RuntimeProfile::Counter * > *, BucketCountersInfo >	BucketCountersMap
	Map from a bucket of counters to the src counter. More...

typedef boost::unordered_set < RuntimeProfile::TimeSeriesCounter * >	TimeSeriesCounters
	Set of time series counters that need to be updated. More...

Private Member Functions
	PeriodicCounterUpdater ()

void	UpdateLoop ()

Private Attributes
boost::scoped_ptr< boost::thread >	update_thread_
	Thread performing asynchronous updates. More...

SpinLock	rate_lock_
	Spinlock that protects the map of rate counters. More...

RateCounterMap	rate_counters_

SpinLock	sampling_lock_
	Spinlock that protects the map of averages over samples of counters. More...

SamplingCounterMap	sampling_counters_

SpinLock	bucketing_lock_
	Spinlock that protects the map of buckets of counters. More...

BucketCountersMap	bucketing_counters_

SpinLock	time_series_lock_
	Spinlock that protects the map of time series counters. More...

TimeSeriesCounters	time_series_counters_

AtomicInt< uint32_t >	done_
	If 1, tear down the update thread. More...

Static Private Attributes
static PeriodicCounterUpdater	state_

Detailed Description

Singleton utility class that updates counter values. This is used to sample some metric (e.g. memory used) at regular intervals. The samples can be summarized in a few ways (e.g. averaged, stored as histogram, kept as a time series data, etc). This class has one thread that will wake up at a regular period and update all the registered counters. Typically, the counter updates should be stopped as early as possible to prevent future stale samples from polluting the useful values.

Definition at line 37 of file periodic-counter-updater.h.

Member Typedef Documentation

typedef boost::unordered_map<std::vector<RuntimeProfile::Counter*>*, BucketCountersInfo> impala::PeriodicCounterUpdater::BucketCountersMap

private

Map from a bucket of counters to the src counter.

Definition at line 129 of file periodic-counter-updater.h.

typedef boost::unordered_map<RuntimeProfile::Counter*, RateCounterInfo> impala::PeriodicCounterUpdater::RateCounterMap

private

A map of the dst (rate) counter to the src counter and elapsed time.

Definition at line 112 of file periodic-counter-updater.h.

typedef boost::unordered_map<RuntimeProfile::Counter*, SamplingCounterInfo> impala::PeriodicCounterUpdater::SamplingCounterMap

private

A map of the dst (averages over samples) counter to the src counter (to be sampled) and number of samples taken.

Definition at line 121 of file periodic-counter-updater.h.

typedef boost::unordered_set<RuntimeProfile::TimeSeriesCounter*> impala::PeriodicCounterUpdater::TimeSeriesCounters

private

Set of time series counters that need to be updated.

Definition at line 136 of file periodic-counter-updater.h.

Member Enumeration Documentation

enum impala::PeriodicCounterUpdater::PeriodicCounterType

Enumerator
RATE_COUNTER
SAMPLING_COUNTER

Definition at line 39 of file periodic-counter-updater.h.

Constructor & Destructor Documentation

impala::PeriodicCounterUpdater::~PeriodicCounterUpdater ( )

Tears down the update thread.

Definition at line 39 of file periodic-counter-updater.cc.

References done_, impala::AtomicInt< T >::Swap(), and update_thread_.

impala::PeriodicCounterUpdater::PeriodicCounterUpdater ( )

private

Definition at line 33 of file periodic-counter-updater.cc.

References state_, update_thread_, and UpdateLoop().

Member Function Documentation

void impala::PeriodicCounterUpdater::RegisterBucketingCounters	(	RuntimeProfile::Counter *	src_counter,
		std::vector< RuntimeProfile::Counter * > *	buckets
	)

static

Adds a bucketing counter to be updated at regular intervals.

Definition at line 85 of file periodic-counter-updater.cc.

References bucketing_counters_, bucketing_lock_, impala::PeriodicCounterUpdater::BucketCountersInfo::num_sampled, impala::PeriodicCounterUpdater::BucketCountersInfo::src_counter, and state_.

Referenced by impala::RuntimeProfile::RegisterBucketingCounters().

void impala::PeriodicCounterUpdater::RegisterPeriodicCounter	(	RuntimeProfile::Counter *	src_counter,
		RuntimeProfile::DerivedCounterFunction	sample_fn,
		RuntimeProfile::Counter *	dst_counter,
		PeriodicCounterType	type
	)

static

Registers a periodic counter to be updated by the update thread. Either sample_fn or dst_counter must be non-NULL. When the periodic counter is updated, it either gets the value from the dst_counter or calls the sample function to get the value. dst_counter/sample fn is assumed to be compatible types with src_counter.

Definition at line 44 of file periodic-counter-updater.cc.

References impala::PeriodicCounterUpdater::RateCounterInfo::elapsed_ms, impala::PeriodicCounterUpdater::SamplingCounterInfo::num_sampled, RATE_COUNTER, rate_counters_, rate_lock_, impala::PeriodicCounterUpdater::RateCounterInfo::sample_fn, impala::PeriodicCounterUpdater::SamplingCounterInfo::sample_fn, SAMPLING_COUNTER, sampling_counters_, sampling_lock_, impala::PeriodicCounterUpdater::RateCounterInfo::src_counter, impala::PeriodicCounterUpdater::SamplingCounterInfo::src_counter, state_, and impala::PeriodicCounterUpdater::SamplingCounterInfo::total_sampled_value.

void impala::PeriodicCounterUpdater::RegisterTimeSeriesCounter ( RuntimeProfile::TimeSeriesCounter * counter )

static

Adds counter to be sampled and updated at regular intervals.

Definition at line 116 of file periodic-counter-updater.cc.

References state_, time_series_counters_, and time_series_lock_.

void impala::PeriodicCounterUpdater::StopBucketingCounters	(	std::vector< RuntimeProfile::Counter * > *	buckets,
		bool	convert
	)

static

Stops updating the bucket counter. If convert is true, convert the buckets from count to percentage. Sampling counters are updated periodically so should be removed as soon as the underlying counter is no longer going to change.

Definition at line 94 of file periodic-counter-updater.cc.

References bucketing_counters_, bucketing_lock_, impala::RuntimeProfile::Counter::Set(), state_, and impala::RuntimeProfile::Counter::value().

Referenced by impala::HdfsScanNode::StopAndFinalizeCounters(), impala::TEST(), and impala::RuntimeProfile::~RuntimeProfile().

void impala::PeriodicCounterUpdater::StopRateCounter ( RuntimeProfile::Counter * counter )

static

Stops updating the value of 'counter'.

Definition at line 75 of file periodic-counter-updater.cc.

References rate_counters_, rate_lock_, and state_.

Referenced by impala::HBaseScanNode::Close(), impala::HdfsScanNode::StopAndFinalizeCounters(), impala::TEST(), and impala::RuntimeProfile::~RuntimeProfile().

void impala::PeriodicCounterUpdater::StopSamplingCounter ( RuntimeProfile::Counter * counter )

static

Stops updating the value of 'counter'.