Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros
impala::ExecNode::RowBatchQueue Class Reference

#include <exec-node.h>

Inheritance diagram for impala::ExecNode::RowBatchQueue:
Collaboration diagram for impala::ExecNode::RowBatchQueue:

Public Member Functions

 RowBatchQueue (int max_batches)
 
 ~RowBatchQueue ()
 
void AddBatch (RowBatch *batch)
 Adds a batch to the queue. This is blocking if the queue is full. More...
 
RowBatchGetBatch ()
 
int Cleanup ()
 
bool BlockingGet (RowBatch **out)
 
bool BlockingPut (const RowBatch *&val)
 
void Shutdown ()
 Shut down the queue. Wakes up all threads waiting on BlockingGet or BlockingPut. More...
 
uint32_t GetSize () const
 
uint64_t total_get_wait_time () const
 Returns the total amount of time threads have blocked in BlockingGet. More...
 
uint64_t total_put_wait_time () const
 Returns the total amount of time threads have blocked in BlockingPut. More...
 

Private Attributes

SpinLock lock_
 Lock protecting cleanup_queue_. More...
 
std::list< RowBatch * > cleanup_queue_
 Queue of orphaned row batches. More...
 

Detailed Description

Extends blocking queue for row batches. Row batches have a property that they must be processed in the order they were produced, even in cancellation paths. Preceding row batches can contain ptrs to memory in subsequent row batches and we need to make sure those ptrs stay valid. Row batches that are added after Shutdown() are queued in another queue, which can be cleaned up during Close(). All functions are thread safe.

Definition at line 181 of file exec-node.h.

Constructor & Destructor Documentation

impala::ExecNode::RowBatchQueue::RowBatchQueue ( int  max_batches)

max_batches is the maximum number of row batches that can be queued. When the queue is full, producers will block.

Definition at line 66 of file exec-node.cc.

impala::ExecNode::RowBatchQueue::~RowBatchQueue ( )

Definition at line 70 of file exec-node.cc.

Member Function Documentation

void impala::ExecNode::RowBatchQueue::AddBatch ( RowBatch batch)

Adds a batch to the queue. This is blocking if the queue is full.

Definition at line 74 of file exec-node.cc.

References impala::lock_.

bool impala::BlockingQueue< RowBatch * >::BlockingGet ( RowBatch * *  out)
inlineinherited

Get an element from the queue, waiting indefinitely for one to become available. Returns false if we were shut down prior to getting the element, and there are no more elements available.

Definition at line 46 of file blocking-queue.h.

bool impala::BlockingQueue< RowBatch * >::BlockingPut ( const RowBatch * &  val)
inlineinherited

Puts an element into the queue, waiting indefinitely until there is space. If the queue is shut down, returns false.

Definition at line 69 of file blocking-queue.h.

int impala::ExecNode::RowBatchQueue::Cleanup ( )

Deletes all row batches in cleanup_queue_. Not valid to call AddBatch() after this is called. Returns the number of io buffers that were released (for debug tracking)

Definition at line 87 of file exec-node.cc.

References impala::lock_, and impala::RowBatch::num_io_buffers().

RowBatch * impala::ExecNode::RowBatchQueue::GetBatch ( )

Gets a row batch from the queue. Returns NULL if there are no more. This function blocks. Returns NULL after Shutdown().

Definition at line 81 of file exec-node.cc.

uint32_t impala::BlockingQueue< RowBatch * >::GetSize ( ) const
inlineinherited

Definition at line 99 of file blocking-queue.h.

void impala::BlockingQueue< RowBatch * >::Shutdown ( )
inlineinherited

Shut down the queue. Wakes up all threads waiting on BlockingGet or BlockingPut.

Definition at line 89 of file blocking-queue.h.

uint64_t impala::BlockingQueue< RowBatch * >::total_get_wait_time ( ) const
inlineinherited

Returns the total amount of time threads have blocked in BlockingGet.

Definition at line 105 of file blocking-queue.h.

uint64_t impala::BlockingQueue< RowBatch * >::total_put_wait_time ( ) const
inlineinherited

Returns the total amount of time threads have blocked in BlockingPut.

Definition at line 111 of file blocking-queue.h.

Member Data Documentation

std::list<RowBatch*> impala::ExecNode::RowBatchQueue::cleanup_queue_
private

Queue of orphaned row batches.

Definition at line 206 of file exec-node.h.

SpinLock impala::ExecNode::RowBatchQueue::lock_
private

Lock protecting cleanup_queue_.

Definition at line 203 of file exec-node.h.


The documentation for this class was generated from the following files: