#include <hbase-table-writer.h>

Collaboration diagram for impala::HBaseTableWriter:

Public Member Functions
	HBaseTableWriter (HBaseTableDescriptor table_desc, const std::vector< ExprContext > &output_expr_ctxs, RuntimeProfile *profile)

Status	AppendRowBatch (RowBatch *batch)

void	Close (RuntimeState *state)
	Calls to Close release the HBaseTable. More...

Status	Init (RuntimeState *state)

Static Public Member Functions
static Status	InitJNI ()
	Grab all of the Java classes needed to get data into and out of HBase. More...

Private Member Functions
Status	CreatePut (JNIEnv env, const void rk, int rk_len, jobject *put)

Status	CreateByteArray (JNIEnv env, const std::string &s, jbyteArray j_array)
	Create a byte array containing the string's chars. More...

Status	CreateByteArray (JNIEnv env, const void data, int data_len, jbyteArray *j_array)
	Create a byte array containing the input bytes. More...

Status	CreatePutList (JNIEnv *env, int num_puts)

Status	CleanUpJni ()

Private Attributes
HBaseTableDescriptor *	table_desc_
	Owned by RuntimeState not by this object. More...

boost::scoped_ptr< HBaseTable >	table_

const std::vector< ExprContext * >	output_expr_ctxs_
	The expressions that are run to create tuples to be written to hbase. More...

std::vector< int >	output_exprs_byte_sizes_
	output_exprs_byte_sizes_[i] is the byte size of output_expr_ctxs_[i]->root()'s type. More...

jobject	put_list_
	jni ArrayList<Put> More...

std::vector< jbyteArray >	cf_arrays_
	cf_arrays_[i-1] is the column family jbyteArray for column i. More...

std::vector< jbyteArray >	qual_arrays_
	qual_arrays_[i-1] is the column family qualifier jbyteArray for column i. More...

RuntimeProfile *	runtime_profile_
	Parent table sink's profile. More...

RuntimeProfile::Counter *	encoding_timer_

RuntimeProfile::Counter *	htable_put_timer_

Static Private Attributes
static jclass	put_cl_ = NULL
	org.apache.hadoop.hbase.client.Put More...

static jmethodID	put_ctor_ = NULL
	new Put(byte[]) More...

static jmethodID	put_add_id_ = NULL
	Put::add(byte[], byte[], byte[]) More...

static jclass	list_cl_ = NULL
	java.util.ArrayList More...

static jmethodID	list_ctor_ = NULL
	new ArrayList<V>(starting_capacity) More...

static jmethodID	list_add_id_ = NULL
	ArrayList::add(V);. More...

Detailed Description

Class to write RowBatches to an HBase table using the java HTable client. This class should only be called from a single sink and should not be shared. Sample usage (Must happen in order): HBaseTableWriter::InitJni(); writer = new HBaseTableWriter(state, table_desc_, output_exprs_); writer.Init(state); writer.AppendRowBatch(batch);

Definition at line 41 of file hbase-table-writer.h.

Constructor & Destructor Documentation

impala::HBaseTableWriter::HBaseTableWriter	(	HBaseTableDescriptor *	table_desc,
		const std::vector< ExprContext * > &	output_expr_ctxs,
		RuntimeProfile *	profile
	)

Definition at line 42 of file hbase-table-writer.cc.

Member Function Documentation

Status impala::HBaseTableWriter::AppendRowBatch ( RowBatch * batch )

Status impala::HBaseTableWriter::CleanUpJni ( )

private

Clean up the jni global ref in put_list_, allowing the jni to garbage collect all of the puts that are created by a writer.

Definition at line 199 of file hbase-table-writer.cc.

References cf_arrays_, getJNIEnv(), impala::Status::OK, put_list_, qual_arrays_, and RETURN_ERROR_IF_EXC.

Referenced by Close().

void impala::HBaseTableWriter::Close ( RuntimeState * state )

Calls to Close release the HBaseTable.

Definition at line 263 of file hbase-table-writer.cc.

References CleanUpJni(), impala::Status::GetDetail(), impala::RuntimeState::LogError(), impala::Status::ok(), and table_.

Status impala::HBaseTableWriter::CreateByteArray	(	JNIEnv *	env,
		const std::string &	s,
		jbyteArray *	j_array
	)

private

Create a byte array containing the string's chars.

Referenced by AppendRowBatch(), CreatePut(), and Init().

Status impala::HBaseTableWriter::CreateByteArray	(	JNIEnv *	env,
		const void *	data,
		int	data_len,
		jbyteArray *	j_array
	)

private

Create a byte array containing the input bytes.

Definition at line 254 of file hbase-table-writer.cc.

References impala::Status::OK, and RETURN_ERROR_IF_EXC.

Status impala::HBaseTableWriter::CreatePut	(	JNIEnv *	env,
		const void *	rk,
		int	rk_len,
		jobject *	put
	)

private

Methods used to create JNI objects. Create a Put using the supplied row key

Definition at line 229 of file hbase-table-writer.cc.

References CreateByteArray(), list_add_id_, impala::Status::OK, put_cl_, put_ctor_, put_list_, RETURN_ERROR_IF_EXC, and RETURN_IF_ERROR.

Referenced by AppendRowBatch().

Status impala::HBaseTableWriter::CreatePutList	(	JNIEnv *	env,
		int	num_puts
	)

private

Create an ArrayList<Put> to be passed to put.put(list); Should be used like: CreatePutList(env, limit);

Definition at line 219 of file hbase-table-writer.cc.

References list_cl_, list_ctor_, impala::JniUtil::LocalToGlobalRef(), impala::Status::OK, put_list_, RETURN_ERROR_IF_EXC, and RETURN_IF_ERROR.

Referenced by AppendRowBatch().

Status impala::HBaseTableWriter::Init ( RuntimeState * state )

Create all needed java side objects. This call may cause connections to HBase and Zookeeper to be created.

Definition at line 52 of file hbase-table-writer.cc.

References ADD_TIMER, cf_arrays_, impala::HBaseTableDescriptor::cols(), CreateByteArray(), encoding_timer_, impala::HBaseTableDescriptor::HBaseColumnDescriptor::family, getJNIEnv(), impala::HBaseTableFactory::GetTable(), impala::RuntimeState::htable_factory(), htable_put_timer_, impala::JniUtil::LocalToGlobalRef(), impala::TableDescriptor::name(), impala::TableDescriptor::num_cols(), impala::Status::OK, output_expr_ctxs_, output_exprs_byte_sizes_, qual_arrays_, impala::HBaseTableDescriptor::HBaseColumnDescriptor::qualifier, RETURN_IF_ERROR, runtime_profile_, table_, and table_desc_.

Status impala::HBaseTableWriter::InitJNI ( )

static

Grab all of the Java classes needed to get data into and out of HBase.

Definition at line 89 of file hbase-table-writer.cc.

References impala::JniUtil::GetGlobalClassRef(), getJNIEnv(), list_add_id_, list_cl_, list_ctor_, impala::Status::OK, put_add_id_, put_cl_, put_ctor_, RETURN_ERROR_IF_EXC, and RETURN_IF_ERROR.

Referenced by main().

Member Data Documentation

std::vector<jbyteArray> impala::HBaseTableWriter::cf_arrays_

private

cf_arrays_[i-1] is the column family jbyteArray for column i.

Definition at line 116 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), CleanUpJni(), and Init().

RuntimeProfile::Counter* impala::HBaseTableWriter::encoding_timer_

private

Definition at line 123 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), and Init().

RuntimeProfile::Counter* impala::HBaseTableWriter::htable_put_timer_

private

Definition at line 124 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), and Init().

jmethodID impala::HBaseTableWriter::list_add_id_ = NULL

staticprivate

ArrayList::add(V);.

Definition at line 113 of file hbase-table-writer.h.

Referenced by CreatePut(), and InitJNI().

jclass impala::HBaseTableWriter::list_cl_ = NULL

staticprivate

java.util.ArrayList

Definition at line 107 of file hbase-table-writer.h.

Referenced by CreatePutList(), and InitJNI().

jmethodID impala::HBaseTableWriter::list_ctor_ = NULL

staticprivate

new ArrayList<V>(starting_capacity)

Definition at line 110 of file hbase-table-writer.h.

Referenced by CreatePutList(), and InitJNI().

const std::vector<ExprContext*> impala::HBaseTableWriter::output_expr_ctxs_

private

The expressions that are run to create tuples to be written to hbase.

Definition at line 89 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), and Init().

std::vector<int> impala::HBaseTableWriter::output_exprs_byte_sizes_

private

output_exprs_byte_sizes_[i] is the byte size of output_expr_ctxs_[i]->root()'s type.

Definition at line 92 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), and Init().

jmethodID impala::HBaseTableWriter::put_add_id_ = NULL

staticprivate

Put::add(byte[], byte[], byte[])

Definition at line 104 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), and InitJNI().

jclass impala::HBaseTableWriter::put_cl_ = NULL

staticprivate

org.apache.hadoop.hbase.client.Put

Definition at line 98 of file hbase-table-writer.h.

Referenced by CreatePut(), and InitJNI().

jmethodID impala::HBaseTableWriter::put_ctor_ = NULL

staticprivate

new Put(byte[])

Definition at line 101 of file hbase-table-writer.h.

Referenced by CreatePut(), and InitJNI().

jobject impala::HBaseTableWriter::put_list_

private

jni ArrayList<Put>

Definition at line 95 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), CleanUpJni(), CreatePut(), and CreatePutList().

std::vector<jbyteArray> impala::HBaseTableWriter::qual_arrays_

private

qual_arrays_[i-1] is the column family qualifier jbyteArray for column i.

Definition at line 119 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), CleanUpJni(), and Init().

RuntimeProfile* impala::HBaseTableWriter::runtime_profile_

private

Parent table sink's profile.

Definition at line 122 of file hbase-table-writer.h.

Referenced by Init().

boost::scoped_ptr<HBaseTable> impala::HBaseTableWriter::table_

private

The wrapper around a Java HTable. This instance is owned by this object and must be cleaned up using close before the table can be discarded.

Definition at line 86 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), Close(), and Init().

HBaseTableDescriptor* impala::HBaseTableWriter::table_desc_

private

Owned by RuntimeState not by this object.

Definition at line 81 of file hbase-table-writer.h.

Referenced by AppendRowBatch(), and Init().

The documentation for this class was generated from the following files:

be/src/exec/hbase-table-writer.h
be/src/exec/hbase-table-writer.cc

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation