impala::WriteStream Class Reference

#include <write-stream.h>

Collaboration diagram for impala::WriteStream:

Public Member Functions
	WriteStream ()
int	WriteBytes (int length, const uint8_t *buf)
	Writes bytes to the buffer, returns the number of bytes written. More...
int	WriteBytes (int length, const char *buf)
int	WriteVInt (int32_t val)
int	WriteInt (uint32_t val)
int	WriteByte (uint8_t val)
int	WriteByte (char val)
int	WriteVLong (int64_t val)
int	WriteBoolean (bool val)
int	WriteZInt (int32_t val)
	Writes a zig-zag encoded integer. More...
int	WriteZLong (int64_t val)
int	WriteText (int32_t len, const uint8_t *buf)
	Writes the length as a VLong follows by the byte string. More...
int	WriteEmptyText ()
	Writes an empty string to the buffer (encoded as 1 byte) More...
void	Clear ()
size_t	Size ()
std::string	String ()
	returns the contents of this stream as a string More...

Private Attributes
std::stringstream	buffer_
	TODO consider making this like the parquet writer to avoid extra copy. More...
uint64_t	len_

Detailed Description

An append-only buffer to stage output from file writers. The buffer is backed by a stringstream and uses the ReadWriteUtil to encode data. Append operations will never fail, and will grow the backing buffer using stringstream semantics. Each write function returns the number of bytes written

Definition at line 29 of file write-stream.h.

Constructor & Destructor Documentation

impala::WriteStream::WriteStream ( )

inline

Definition at line 32 of file write-stream.h.

Member Function Documentation

void impala::WriteStream::Clear ( )

inline

Definition at line 100 of file write-stream.inline.h.

References buffer_, and len_.

Referenced by impala::HdfsSequenceTableWriter::ConsumeRow(), impala::HdfsSequenceTableWriter::Flush(), impala::HdfsAvroTableWriter::Flush(), impala::HdfsSequenceTableWriter::WriteFileHeader(), and impala::HdfsAvroTableWriter::WriteFileHeader().

size_t impala::WriteStream::Size ( )

inline

Definition at line 96 of file write-stream.inline.h.

References len_.

Referenced by impala::HdfsSequenceTableWriter::AppendRowBatch(), impala::HdfsAvroTableWriter::AppendRowBatch(), impala::HdfsSequenceTableWriter::ConsumeRow(), and impala::HdfsAvroTableWriter::Flush().

string impala::WriteStream::String ( )

inline

returns the contents of this stream as a string

Definition at line 92 of file write-stream.inline.h.

References buffer_.

Referenced by impala::HdfsSequenceTableWriter::ConsumeRow(), impala::HdfsSequenceTableWriter::Flush(), impala::HdfsAvroTableWriter::Flush(), impala::HdfsSequenceTableWriter::WriteCompressedBlock(), impala::HdfsSequenceTableWriter::WriteFileHeader(), and impala::HdfsAvroTableWriter::WriteFileHeader().

int impala::WriteStream::WriteBoolean ( bool  val )

inline

Definition at line 87 of file write-stream.inline.h.

References WriteBytes().

Referenced by impala::HdfsSequenceTableWriter::WriteFileHeader().

int impala::WriteStream::WriteByte ( uint8_t  val )

inline

Definition at line 28 of file write-stream.inline.h.

Referenced by impala::HdfsAvroTableWriter::AppendField(), impala::HdfsSequenceTableWriter::EncodeRow(), and impala::HdfsSequenceTableWriter::WriteEscapedString().

int impala::WriteStream::WriteByte ( char  val )

inline

Definition at line 32 of file write-stream.inline.h.

References WriteBytes().

int impala::WriteStream::WriteBytes ( int  length, const uint8_t *  buf  )

inline

Writes bytes to the buffer, returns the number of bytes written.

Definition at line 66 of file write-stream.inline.h.

Referenced by impala::HdfsAvroTableWriter::AppendField(), impala::HdfsSequenceTableWriter::AppendRowBatch(), impala::HdfsSequenceTableWriter::ConsumeRow(), impala::HdfsSequenceTableWriter::EncodeRow(), WriteBoolean(), WriteByte(), impala::HdfsSequenceTableWriter::WriteCompressedBlock(), impala::HdfsSequenceTableWriter::WriteFileHeader(), impala::HdfsAvroTableWriter::WriteFileHeader(), WriteInt(), WriteText(), WriteVInt(), WriteVLong(), WriteZInt(), and WriteZLong().

int impala::WriteStream::WriteBytes ( int  length, const char *  buf  )

inline

Definition at line 70 of file write-stream.inline.h.

References buffer_, and len_.

int impala::WriteStream::WriteEmptyText ( )

inline

Writes an empty string to the buffer (encoded as 1 byte)

Definition at line 83 of file write-stream.inline.h.

References WriteVInt().

Referenced by impala::HdfsSequenceTableWriter::WriteCompressedBlock(), and impala::HdfsSequenceTableWriter::WriteFileHeader().

int impala::WriteStream::WriteInt ( uint32_t  val )

inline

Definition at line 48 of file write-stream.inline.h.

References impala::ReadWriteUtil::PutInt(), and WriteBytes().

Referenced by impala::HdfsSequenceTableWriter::ConsumeRow(), and impala::HdfsSequenceTableWriter::WriteFileHeader().

int impala::WriteStream::WriteText ( int32_t  len, const uint8_t *  buf  )

inline

Writes the length as a VLong follows by the byte string.

Definition at line 76 of file write-stream.inline.h.

References WriteBytes(), and WriteVInt().

Referenced by impala::HdfsSequenceTableWriter::ConsumeRow(), and impala::HdfsSequenceTableWriter::WriteFileHeader().

int impala::WriteStream::WriteVInt ( int32_t  val )

inline

Definition at line 42 of file write-stream.inline.h.

References impala::ReadWriteUtil::MAX_VINT_LEN, impala::ReadWriteUtil::PutVInt(), and WriteBytes().

Referenced by impala::HdfsSequenceTableWriter::WriteCompressedBlock(), WriteEmptyText(), and WriteText().

int impala::WriteStream::WriteVLong ( int64_t  val )

inline

Definition at line 36 of file write-stream.inline.h.

References impala::ReadWriteUtil::MAX_VINT_LEN, impala::ReadWriteUtil::PutVLong(), and WriteBytes().

Referenced by impala::HdfsSequenceTableWriter::ConsumeRow(), and impala::HdfsSequenceTableWriter::WriteCompressedBlock().

int impala::WriteStream::WriteZInt ( int32_t  val )

inline

Writes a zig-zag encoded integer.

Definition at line 54 of file write-stream.inline.h.

References impala::ReadWriteUtil::MAX_ZINT_LEN, impala::ReadWriteUtil::PutZInt(), and WriteBytes().

Referenced by impala::HdfsAvroTableWriter::AppendField().

int impala::WriteStream::WriteZLong ( int64_t  val )

inline

Definition at line 60 of file write-stream.inline.h.

References impala::ReadWriteUtil::MAX_ZLONG_LEN, impala::ReadWriteUtil::PutZLong(), and WriteBytes().

Referenced by impala::HdfsAvroTableWriter::AppendField(), impala::HdfsAvroTableWriter::Flush(), and impala::HdfsAvroTableWriter::WriteFileHeader().

Member Data Documentation

std::stringstream impala::WriteStream::buffer_

private

TODO consider making this like the parquet writer to avoid extra copy.

Definition at line 60 of file write-stream.h.

Referenced by Clear(), String(), and WriteBytes().

uint64_t impala::WriteStream::len_

private

Definition at line 61 of file write-stream.h.

Referenced by Clear(), Size(), and WriteBytes().

---

The documentation for this class was generated from the following files:

• be/src/exec/write-stream.h
• be/src/exec/write-stream.inline.h