Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros
Public Member Functions | Private Attributes | List of all members
impala::HdfsPartitionDescriptor Class Reference

Metadata for a single partition inside an Hdfs table. More...

#include <descriptors.h>

Collaboration diagram for impala::HdfsPartitionDescriptor:

Public Member Functions

 HdfsPartitionDescriptor (const THdfsPartition &thrift_partition, ObjectPool *pool)
 
char line_delim () const
 
char field_delim () const
 
char collection_delim () const
 
char escape_char () const
 
THdfsFileFormat::type file_format () const
 
const std::vector< ExprContext * > & partition_key_value_ctxs () const
 
int block_size () const
 
const std::string & location () const
 
int64_t id () const
 
Status PrepareExprs (RuntimeState *state)
 
Status OpenExprs (RuntimeState *state)
 
void CloseExprs (RuntimeState *state)
 
std::string DebugString () const
 

Private Attributes

char line_delim_
 
char field_delim_
 
char collection_delim_
 
char escape_char_
 
int block_size_
 
std::string location_
 
int64_t id_
 
bool exprs_prepared_
 True if PrepareExprs has been called, to prevent repeating expensive codegen. More...
 
bool exprs_opened_
 
bool exprs_closed_
 
std::vector< ExprContext * > partition_key_value_ctxs_
 
THdfsFileFormat::type file_format_
 The format (e.g. text, sequence file etc.) of data in the files in this partition. More...
 
ObjectPool * object_pool_
 

Detailed Description

Metadata for a single partition inside an Hdfs table.

Definition at line 177 of file descriptors.h.

Constructor & Destructor Documentation

impala::HdfsPartitionDescriptor::HdfsPartitionDescriptor ( const THdfsPartition &  thrift_partition,
ObjectPool *  pool 
)

Definition at line 109 of file descriptors.cc.

References impala::Expr::CreateExprTree(), object_pool_, impala::Status::ok(), and partition_key_value_ctxs_.

Member Function Documentation

int impala::HdfsPartitionDescriptor::block_size ( ) const
inline

Definition at line 188 of file descriptors.h.

References block_size_.

Referenced by impala::HdfsTableSink::CreateNewTmpFile().

void impala::HdfsPartitionDescriptor::CloseExprs ( RuntimeState *  state)

Definition at line 151 of file descriptors.cc.

References impala::Expr::Close(), exprs_closed_, and partition_key_value_ctxs_.

Referenced by impala::HdfsScanNode::Close(), and impala::HdfsTableSink::Close().

char impala::HdfsPartitionDescriptor::collection_delim ( ) const
inline

Definition at line 182 of file descriptors.h.

References collection_delim_.

Referenced by impala::HdfsTextScanner::InitNewRange(), and impala::HdfsSequenceScanner::InitNewRange().

string impala::HdfsPartitionDescriptor::DebugString ( ) const

Definition at line 157 of file descriptors.cc.

References collection_delim_, escape_char_, field_delim_, file_format_, and line_delim_.

char impala::HdfsPartitionDescriptor::escape_char ( ) const
inline

Definition at line 183 of file descriptors.h.

References escape_char_.

Referenced by impala::HdfsSequenceTableWriter::HdfsSequenceTableWriter(), impala::HdfsTextTableWriter::HdfsTextTableWriter(), impala::HdfsScanner::InitializeWriteTuplesFn(), impala::HdfsTextScanner::InitNewRange(), and impala::HdfsSequenceScanner::InitNewRange().

char impala::HdfsPartitionDescriptor::field_delim ( ) const
inline

Definition at line 181 of file descriptors.h.

References field_delim_.

Referenced by impala::HdfsSequenceTableWriter::HdfsSequenceTableWriter(), impala::HdfsTextTableWriter::HdfsTextTableWriter(), impala::HdfsTextScanner::InitNewRange(), and impala::HdfsSequenceScanner::InitNewRange().

THdfsFileFormat::type impala::HdfsPartitionDescriptor::file_format ( ) const
inline

Definition at line 184 of file descriptors.h.

References file_format_.

Referenced by impala::HdfsScanNode::CreateAndPrepareScanner(), impala::HdfsTableSink::InitOutputPartition(), impala::HdfsScanNode::Prepare(), and impala::HdfsScanNode::ScannerThread().

int64_t impala::HdfsPartitionDescriptor::id ( ) const
inline

Definition at line 190 of file descriptors.h.

References id_.

Referenced by impala::HdfsTableSink::GetOutputPartition().

char impala::HdfsPartitionDescriptor::line_delim ( ) const
inline

Definition at line 180 of file descriptors.h.

References line_delim_.

Referenced by impala::HdfsTextTableWriter::HdfsTextTableWriter(), and impala::HdfsTextScanner::InitNewRange().

const std::string& impala::HdfsPartitionDescriptor::location ( ) const
inline

Definition at line 189 of file descriptors.h.

References location_.

Referenced by impala::HdfsTableSink::BuildHdfsFileNames(), and impala::HdfsScanNode::Prepare().

Status impala::HdfsPartitionDescriptor::OpenExprs ( RuntimeState *  state)

Definition at line 145 of file descriptors.cc.

References exprs_opened_, impala::Status::OK, impala::Expr::Open(), and partition_key_value_ctxs_.

Referenced by impala::HdfsScanNode::Open(), and impala::HdfsTableSink::Open().

const std::vector<ExprContext*>& impala::HdfsPartitionDescriptor::partition_key_value_ctxs ( ) const
inline

Definition at line 185 of file descriptors.h.

References partition_key_value_ctxs_.

Referenced by impala::HdfsTableSink::Open(), and impala::HdfsScanner::Prepare().

Status impala::HdfsPartitionDescriptor::PrepareExprs ( RuntimeState *  state)

Calls Prepare()/Open()/Close() on all partition key exprs. Idempotent (this is because both HdfsScanNode and HdfsTableSink may both use the same partition desc).

Definition at line 134 of file descriptors.cc.

References exprs_prepared_, impala::RuntimeState::instance_mem_tracker(), impala::Status::OK, partition_key_value_ctxs_, impala::Expr::Prepare(), and RETURN_IF_ERROR.

Referenced by impala::HdfsScanNode::Prepare(), and impala::HdfsTableSink::PrepareExprs().

Member Data Documentation

int impala::HdfsPartitionDescriptor::block_size_
private

Definition at line 205 of file descriptors.h.

Referenced by block_size().

char impala::HdfsPartitionDescriptor::collection_delim_
private

Definition at line 203 of file descriptors.h.

Referenced by collection_delim(), and DebugString().

char impala::HdfsPartitionDescriptor::escape_char_
private

Definition at line 204 of file descriptors.h.

Referenced by DebugString(), and escape_char().

bool impala::HdfsPartitionDescriptor::exprs_closed_
private

Definition at line 212 of file descriptors.h.

Referenced by CloseExprs().

bool impala::HdfsPartitionDescriptor::exprs_opened_
private

Definition at line 211 of file descriptors.h.

Referenced by OpenExprs().

bool impala::HdfsPartitionDescriptor::exprs_prepared_
private

True if PrepareExprs has been called, to prevent repeating expensive codegen.

Definition at line 210 of file descriptors.h.

Referenced by PrepareExprs().

char impala::HdfsPartitionDescriptor::field_delim_
private

Definition at line 202 of file descriptors.h.

Referenced by DebugString(), and field_delim().

THdfsFileFormat::type impala::HdfsPartitionDescriptor::file_format_
private

The format (e.g. text, sequence file etc.) of data in the files in this partition.

Definition at line 219 of file descriptors.h.

Referenced by DebugString(), and file_format().

int64_t impala::HdfsPartitionDescriptor::id_
private

Definition at line 207 of file descriptors.h.

Referenced by id().

char impala::HdfsPartitionDescriptor::line_delim_
private

Definition at line 201 of file descriptors.h.

Referenced by DebugString(), and line_delim().

std::string impala::HdfsPartitionDescriptor::location_
private

Definition at line 206 of file descriptors.h.

Referenced by location().

ObjectPool* impala::HdfsPartitionDescriptor::object_pool_
private

For allocating expression objects in partition_key_values_ Owned by DescriptorTbl, supplied in constructor.

Definition at line 223 of file descriptors.h.

Referenced by HdfsPartitionDescriptor().

std::vector<ExprContext*> impala::HdfsPartitionDescriptor::partition_key_value_ctxs_
private

List of literal (and therefore constant) expressions for each partition key. Their order corresponds to the first num_clustering_cols of the parent table.

Definition at line 216 of file descriptors.h.

Referenced by CloseExprs(), HdfsPartitionDescriptor(), OpenExprs(), partition_key_value_ctxs(), and PrepareExprs().

The documentation for this class was generated from the following files: