Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
|
#include <exec-node.h>
Public Member Functions | |
RowBatchQueue (int max_batches) | |
~RowBatchQueue () | |
void | AddBatch (RowBatch *batch) |
Adds a batch to the queue. This is blocking if the queue is full. More... | |
RowBatch * | GetBatch () |
int | Cleanup () |
bool | BlockingGet (RowBatch **out) |
bool | BlockingPut (const RowBatch *&val) |
void | Shutdown () |
Shut down the queue. Wakes up all threads waiting on BlockingGet or BlockingPut. More... | |
uint32_t | GetSize () const |
uint64_t | total_get_wait_time () const |
Returns the total amount of time threads have blocked in BlockingGet. More... | |
uint64_t | total_put_wait_time () const |
Returns the total amount of time threads have blocked in BlockingPut. More... | |
Private Attributes | |
SpinLock | lock_ |
Lock protecting cleanup_queue_. More... | |
std::list< RowBatch * > | cleanup_queue_ |
Queue of orphaned row batches. More... | |
Extends blocking queue for row batches. Row batches have a property that they must be processed in the order they were produced, even in cancellation paths. Preceding row batches can contain ptrs to memory in subsequent row batches and we need to make sure those ptrs stay valid. Row batches that are added after Shutdown() are queued in another queue, which can be cleaned up during Close(). All functions are thread safe.
Definition at line 181 of file exec-node.h.
impala::ExecNode::RowBatchQueue::RowBatchQueue | ( | int | max_batches | ) |
max_batches is the maximum number of row batches that can be queued. When the queue is full, producers will block.
Definition at line 66 of file exec-node.cc.
impala::ExecNode::RowBatchQueue::~RowBatchQueue | ( | ) |
Definition at line 70 of file exec-node.cc.
void impala::ExecNode::RowBatchQueue::AddBatch | ( | RowBatch * | batch | ) |
Adds a batch to the queue. This is blocking if the queue is full.
Definition at line 74 of file exec-node.cc.
References impala::lock_.
|
inlineinherited |
Get an element from the queue, waiting indefinitely for one to become available. Returns false if we were shut down prior to getting the element, and there are no more elements available.
Definition at line 46 of file blocking-queue.h.
|
inlineinherited |
Puts an element into the queue, waiting indefinitely until there is space. If the queue is shut down, returns false.
Definition at line 69 of file blocking-queue.h.
int impala::ExecNode::RowBatchQueue::Cleanup | ( | ) |
Deletes all row batches in cleanup_queue_. Not valid to call AddBatch() after this is called. Returns the number of io buffers that were released (for debug tracking)
Definition at line 87 of file exec-node.cc.
References impala::lock_, and impala::RowBatch::num_io_buffers().
RowBatch * impala::ExecNode::RowBatchQueue::GetBatch | ( | ) |
Gets a row batch from the queue. Returns NULL if there are no more. This function blocks. Returns NULL after Shutdown().
Definition at line 81 of file exec-node.cc.
|
inlineinherited |
Definition at line 99 of file blocking-queue.h.
|
inlineinherited |
Shut down the queue. Wakes up all threads waiting on BlockingGet or BlockingPut.
Definition at line 89 of file blocking-queue.h.
|
inlineinherited |
Returns the total amount of time threads have blocked in BlockingGet.
Definition at line 105 of file blocking-queue.h.
|
inlineinherited |
Returns the total amount of time threads have blocked in BlockingPut.
Definition at line 111 of file blocking-queue.h.
|
private |
Queue of orphaned row batches.
Definition at line 206 of file exec-node.h.
|
private |
Lock protecting cleanup_queue_.
Definition at line 203 of file exec-node.h.