Useful utility functions for runtime values (which are passed around as void*). More...

#include <raw-value.h>

Collaboration diagram for impala::RawValue:

Static Public Member Functions
static void	PrintValue (const void value, const ColumnType &type, int scale, std::stringstream stream)

static void	PrintValue (const void value, const ColumnType &type, int scale, std::string str)
	Write ascii value to string instead of stringstream. More...

static void	PrintValueAsBytes (const void value, const ColumnType &type, std::stringstream stream)
	Writes the byte representation of a value to a stringstream character-by-character. More...

static uint32_t	GetHashValue (const void *v, const ColumnType &type, uint32_t seed=0)

static uint32_t	GetHashValueFnv (const void *v, const ColumnType &type, uint32_t seed)

static int	Compare (const void v1, const void v2, const ColumnType &type)

static void	Write (const void value, Tuple tuple, const SlotDescriptor slot_desc, MemPool pool)

static void	Write (const void src, void dst, const ColumnType &type, MemPool *pool)

static void	Write (const void src, const ColumnType &type, void dst, uint8_t **buf)

static bool	Eq (const void v1, const void v2, const ColumnType &type)

Static Public Attributes
static const int	ASCII_PRECISION = 16
	Ascii output precision for double/float. More...

Static Private Member Functions
static uint32_t	HashCombine32 (uint32_t value, uint32_t seed)

Static Private Attributes
static const uint32_t	HASH32_COMBINE_SEED = 0x9e3779b9
	The magic number (used in hash_combine()) 0x9e3779b9 = 2^32 / (golden ratio). More...

Detailed Description

Useful utility functions for runtime values (which are passed around as void*).

Definition at line 37 of file raw-value.h.

Member Function Documentation

int impala::RawValue::Compare	(	const void *	v1,
		const void *	v2,
		const ColumnType &	type
	)

static

Compares both values. Return value is < 0 if v1 < v2, 0 if v1 == v2, > 0 if v1 > v2.

Definition at line 109 of file raw-value.cc.

References impala::StringValue::CharSlotToPtr(), impala::StringValue::Compare(), impala::ColumnType::DebugString(), impala::ColumnType::GetByteSize(), impala::ColumnType::len, impala::StringCompare(), impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_NULL, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, impala::TYPE_VARCHAR, and impala::StringValue::UnpaddedCharLength().

Referenced by impala::TupleRowComparator::Compare(), and impala::TEST_F().

bool impala::RawValue::Eq	(	const void *	v1,
		const void *	v2,
		const ColumnType &	type
	)

inlinestatic

Returns true if v1 == v2. This is more performant than Compare() == 0 for string equality, mostly because of the length comparison check.

Definition at line 106 of file raw-value.h.

References impala::StringValue::CharSlotToPtr(), impala::StringValue::Eq(), impala::ColumnType::GetByteSize(), impala::ColumnType::len, impala::StringCompare(), impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, impala::TYPE_VARCHAR, and impala::StringValue::UnpaddedCharLength().

Referenced by impala::HashTableCtx::Equals(), impala::OldHashTable::Equals(), impala::HdfsTableSink::Open(), and impala::TupleEqualityChecker::operator()().

uint32_t impala::RawValue::GetHashValue	(	const void *	v,
		const ColumnType &	type,
		uint32_t	seed = `0`
	)

inlinestatic

Returns hash value for 'v' interpreted as 'type'. The resulting hash value is combined with the seed value.

Use boost::hash_combine for corner cases. boost::hash_combine is reimplemented here to use uint32_t's (instead of size_t)

Definition at line 168 of file raw-value.h.

References impala::StringValue::CharSlotToPtr(), impala::ColumnType::GetByteSize(), impala::HashUtil::Hash(), HashCombine32(), impala::StringValue::len, impala::ColumnType::len, impala::StringValue::ptr, impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, and impala::TYPE_VARCHAR.

Referenced by impala::OldHashTable::AddBitmapFilters(), impala::PartitionedHashJoinNode::Partition::BuildHashTableInternal(), impala::HdfsParquetScanner::ColumnReader< T >::ReadSlot(), and impala::TEST().

uint32_t impala::RawValue::GetHashValueFnv	(	const void *	v,
		const ColumnType &	type,
		uint32_t	seed
	)

inlinestatic

Get a 32-bit hash value using the FNV hash function. Using different seeds with FNV results in different hash functions. GetHashValue() does not have this property and cannot be safely used as the first step in data repartitioning. However, GetHashValue() can be significantly faster. TODO: fix GetHashValue

Definition at line 196 of file raw-value.h.

References impala::StringValue::CharSlotToPtr(), impala::HashUtil::FnvHash64to32(), impala::ColumnType::GetByteSize(), HashCombine32(), impala::StringValue::len, impala::ColumnType::len, impala::StringValue::ptr, impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, and impala::TYPE_VARCHAR.

Referenced by impala::DataStreamTest::CheckReceivers(), and impala::DataStreamSender::Send().

static uint32_t impala::RawValue::HashCombine32	(	uint32_t	value,
		uint32_t	seed
	)

inlinestaticprivate

Combine hashes 'value' and 'seed' to get a new hash value. Similar to boost::hash_combine(), but for uint32_t. Used for NULL and boolean inputs in GetHashValue().

Definition at line 101 of file raw-value.h.

References HASH32_COMBINE_SEED.

Referenced by GetHashValue(), and GetHashValueFnv().

void impala::RawValue::PrintValue	(	const void *	value,
		const ColumnType &	type,
		int	scale,
		std::stringstream *	stream
	)

inlinestatic

Convert 'value' into ascii and write to 'stream'. NULL turns into "NULL". 'scale' determines how many digits after the decimal are printed for floating point numbers, -1 indicates to use the stream's current formatting. TODO: for string types, we just print the result regardless of whether or not it ascii. This could be undesirable.

Definition at line 224 of file raw-value.h.

References impala::StringValue::CharSlotToPtr(), impala::ColumnType::GetByteSize(), impala::StringValue::len, impala::ColumnType::len, LIKELY, impala::StringValue::ptr, impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, and impala::TYPE_VARCHAR.

Referenced by impala::ExprContext::GetValue(), and impala::TEST_F().

static void impala::RawValue::PrintValue	(	const void *	value,
		const ColumnType &	type,
		int	scale,
		std::string *	str
	)

static

Write ascii value to string instead of stringstream.

void impala::RawValue::PrintValueAsBytes	(	const void *	value,
		const ColumnType &	type,
		std::stringstream *	stream
	)

static

Writes the byte representation of a value to a stringstream character-by-character.

Definition at line 28 of file raw-value.cc.

References impala::StringValue::CharSlotToPtr(), impala::ColumnType::DebugString(), impala::ColumnType::GetByteSize(), impala::StringValue::len, impala::ColumnType::len, impala::StringValue::ptr, impala::TimestampValue::Size(), impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, and impala::TYPE_VARCHAR.

Referenced by impala::HdfsTableSink::GetHashTblKey().

void impala::RawValue::Write	(	const void *	value,
		Tuple *	tuple,
		const SlotDescriptor *	slot_desc,
		MemPool *	pool
	)

static

Writes the bytes of a given value into the slot of a tuple. For string values, the string data is copied into memory allocated from 'pool' only if pool is non-NULL.

Definition at line 303 of file raw-value.cc.

References impala::Tuple::GetSlot(), impala::SlotDescriptor::null_indicator_offset(), pool, impala::Tuple::SetNull(), impala::SlotDescriptor::tuple_offset(), and impala::SlotDescriptor::type().

Referenced by impala::AggregationNode::ConstructIntermediateTuple(), impala::PartitionedAggregationNode::ConstructIntermediateTuple(), impala::UnionNode::EvalAndMaterializeExprs(), impala::HashTableCtx::EvalRow(), impala::OldHashTable::EvalRow(), impala::AggregationNode::FinalizeTuple(), impala::PartitionedAggregationNode::GetOutputTuple(), impala::HdfsScanNode::InitTemplateTuple(), impala::Tuple::MaterializeExprs(), impala::HdfsAvroScanner::ResolveSchemas(), and impala::AggFnEvaluator::SerializeOrFinalize().

void impala::RawValue::Write	(	const void *	src,
		void *	dst,
		const ColumnType &	type,
		MemPool *	pool
	)

static

Writes 'src' into 'dst' for type. For string values, the string data is copied into 'pool' if pool is non-NULL. src must be non-NULL.

Definition at line 189 of file raw-value.cc.

References impala::MemPool::Allocate(), impala::StringValue::CharSlotToPtr(), impala::ColumnType::DebugString(), impala::ColumnType::GetByteSize(), impala::ColumnType::IsVarLen(), impala::StringValue::len, impala::ColumnType::len, impala::StringValue::ptr, impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_NULL, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, and impala::TYPE_VARCHAR.

void impala::RawValue::Write	(	const void *	src,
		const ColumnType &	type,
		void *	dst,
		uint8_t **	buf
	)

static

Writes 'src' into 'dst' for type. String values are copied into *buffer and *buffer is updated by the length. *buf must be preallocated to be large enough.

Definition at line 249 of file raw-value.cc.

References impala::ColumnType::DebugString(), impala::ColumnType::GetByteSize(), impala::ColumnType::IsVarLen(), impala::StringValue::len, impala::ColumnType::len, impala::StringValue::ptr, impala::ColumnType::type, impala::TYPE_BIGINT, impala::TYPE_BOOLEAN, impala::TYPE_CHAR, impala::TYPE_DECIMAL, impala::TYPE_DOUBLE, impala::TYPE_FLOAT, impala::TYPE_INT, impala::TYPE_SMALLINT, impala::TYPE_STRING, impala::TYPE_TIMESTAMP, impala::TYPE_TINYINT, and impala::TYPE_VARCHAR.

Member Data Documentation

const int impala::RawValue::ASCII_PRECISION = 16

static

Ascii output precision for double/float.

Definition at line 40 of file raw-value.h.

Referenced by impala::HdfsTextTableWriter::HdfsTextTableWriter().

const uint32_t impala::RawValue::HASH32_COMBINE_SEED = 0x9e3779b9

staticprivate

The magic number (used in hash_combine()) 0x9e3779b9 = 2^32 / (golden ratio).

Definition at line 96 of file raw-value.h.

Referenced by HashCombine32().

The documentation for this class was generated from the following files:

be/src/runtime/raw-value.h
be/src/runtime/raw-value.cc

Static Public Member Functions

Static Public Attributes

Static Private Member Functions

Static Private Attributes

Detailed Description

Member Function Documentation

Member Data Documentation