impala::DiskIoMgr::WriteRange Class Reference

#include <disk-io-mgr.h>

Inheritance diagram for impala::DiskIoMgr::WriteRange:

Collaboration diagram for impala::DiskIoMgr::WriteRange:

Public Types
typedef boost::function< void(const Status &)>	WriteDoneCallback

Public Member Functions
	WriteRange (const std::string &file, int64_t file_offset, int disk_id, WriteDoneCallback callback)
void	SetData (const uint8_t *buffer, int64_t len)
hdfsFS	fs () const
const char *	file () const
int64_t	offset () const
int64_t	len () const
int	disk_id () const
RequestType::type	request_type () const
T *	Next () const
	Returns the Next/Prev node or NULL if this is the end/front. More...
T *	Prev () const

Protected Attributes
hdfsFS	fs_
	Hadoop filesystem that contains file_, or set to NULL for local filesystem. More...
std::string	file_
	Path to file being read or written. More...
int64_t	offset_
	Offset within file_ being read or written. More...
int64_t	len_
	Length of data read or written. More...
int	disk_id_
	Id of disk containing file_;. More...
RequestType::type	request_type_
	The type of IO request, READ or WRITE. More...

Private Attributes
const uint8_t *	data_
WriteDoneCallback	callback_
	Callback to invoke after the write is complete. More...

Friends
class	DiskIoMgr

Detailed Description

Used to specify data to be written to a file and offset. It is the responsibility of the client to ensure that the data to be written is valid and that the file to be written to exists until the callback is invoked. A callback is invoked to inform the client when the write is done.

Definition at line 455 of file disk-io-mgr.h.

Member Typedef Documentation

typedef boost::function<void (const Status&)> impala::DiskIoMgr::WriteRange::WriteDoneCallback

This callback is invoked on each WriteRange after the write is complete or the context is cancelled. The status returned by the callback parameter indicates if the write was successful (i.e. Status::OK), if there was an error TStatusCode::RUNTIME_ERROR) or if the context was cancelled (TStatusCode::CANCELLED). The callback is only invoked if this WriteRange was successfully added (i.e. AddWriteRange() succeeded). No locks are held while the callback is invoked.

Definition at line 464 of file disk-io-mgr.h.

Constructor & Destructor Documentation

DiskIoMgr::WriteRange::WriteRange	(	const std::string &	file,
		int64_t	file_offset,
		int	disk_id,
		WriteDoneCallback	callback
	)

Definition at line 196 of file disk-io-mgr.cc.

References impala::DiskIoMgr::RequestType::WRITE.

Member Function Documentation

int impala::DiskIoMgr::RequestRange::disk_id ( ) const

inlineinherited

Definition at line 269 of file disk-io-mgr.h.

References impala::DiskIoMgr::RequestRange::disk_id_.

Referenced by impala::DiskIoMgr::RequestContext::AddRequestRange(), impala::HdfsParquetScanner::InitColumns(), impala::HdfsTextScanner::IssueInitialRanges(), impala::HdfsParquetScanner::IssueInitialRanges(), impala::BufferedBlockMgr::PinBlock(), impala::HdfsParquetScanner::ProcessFooter(), and impala::DiskIoMgr::RequestContext::ScheduleScanRange().

const char* impala::DiskIoMgr::RequestRange::file ( ) const

inlineinherited

Definition at line 266 of file disk-io-mgr.h.

References impala::DiskIoMgr::RequestRange::file_.

Referenced by impala::ScannerContext::Stream::filename(), impala::HdfsParquetScanner::InitColumns(), impala::BufferedBlockMgr::PinBlock(), impala::HdfsParquetScanner::ProcessFooter(), impala::HdfsScanNode::ScannerThread(), impala::HdfsParquetScanner::ValidateColumn(), and impala::DiskIoMgr::Write().

hdfsFS impala::DiskIoMgr::RequestRange::fs ( ) const

inlineinherited

Definition at line 265 of file disk-io-mgr.h.

References impala::DiskIoMgr::RequestRange::fs_.

Referenced by impala::HdfsParquetScanner::InitColumns(), and impala::HdfsParquetScanner::ProcessFooter().

int64_t impala::DiskIoMgr::RequestRange::len ( ) const

inlineinherited

Definition at line 268 of file disk-io-mgr.h.

References impala::DiskIoMgr::RequestRange::len_.

Referenced by impala::DiskIoMgr::AddWriteRange(), impala::ScannerContext::Stream::bytes_left(), impala::ScannerContext::Stream::eosr(), impala::BufferedBlockMgr::PinBlock(), impala::DiskIoMgr::Read(), impala::HdfsScanNode::ScannerThread(), and impala::DiskIoMgrTest::ValidateSyncRead().

template<typename T>

T* impala::InternalQueue< T >::Node::Next ( ) const

inlineinherited

Returns the Next/Prev node or NULL if this is the end/front.

Definition at line 48 of file internal-queue.h.

References impala::InternalQueue< T >::lock_, impala::InternalQueue< T >::Node::next, and impala::InternalQueue< T >::Node::parent_queue.

Referenced by impala::TEST(), and impala::BufferedBlockMgr::Validate().

int64_t impala::DiskIoMgr::RequestRange::offset ( ) const

inlineinherited

Definition at line 267 of file disk-io-mgr.h.

References impala::DiskIoMgr::RequestRange::offset_.

Referenced by impala::DiskIoMgrStress::ClientThread(), impala::ScannerContext::Stream::file_offset(), impala::HdfsTextScanner::FindFirstTuple(), impala::HdfsTextScanner::IssueInitialRanges(), impala::HdfsParquetScanner::IssueInitialRanges(), impala::BufferedBlockMgr::PinBlock(), impala::HdfsScanNode::ScannerThread(), impala::TEST_F(), impala::DiskIoMgrTest::ValidateScanRange(), and impala::DiskIoMgr::WriteRangeHelper().

template<typename T>

T* impala::InternalQueue< T >::Node::Prev ( ) const

inlineinherited

Definition at line 52 of file internal-queue.h.

References impala::InternalQueue< T >::lock_, impala::InternalQueue< T >::Node::parent_queue, and impala::InternalQueue< T >::Node::prev.

Referenced by impala::TEST().

RequestType::type impala::DiskIoMgr::RequestRange::request_type ( ) const

inlineinherited

Definition at line 270 of file disk-io-mgr.h.

References impala::DiskIoMgr::RequestRange::request_type_.

Referenced by impala::DiskIoMgr::RequestContext::AddRequestRange(), impala::DiskIoMgr::RequestContext::Cancel(), and impala::DiskIoMgr::WorkLoop().

void DiskIoMgr::WriteRange::SetData	(	const uint8_t *	buffer,
		int64_t	len
	)

Set the data and number of bytes to be written for this WriteRange. File data can be over-written by calling SetData() and AddWriteRange().

Definition at line 205 of file disk-io-mgr.cc.

Referenced by impala::TEST_F(), and impala::BufferedBlockMgr::WriteUnpinnedBlock().

Friends And Related Function Documentation

friend class DiskIoMgr

friend

Definition at line 473 of file disk-io-mgr.h.

Member Data Documentation

WriteDoneCallback impala::DiskIoMgr::WriteRange::callback_

private

Callback to invoke after the write is complete.

Definition at line 480 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::RequestContext::Cancel(), and impala::DiskIoMgr::HandleWriteFinished().

const uint8_t* impala::DiskIoMgr::WriteRange::data_

private

Data to be written. RequestRange::len_ contains the length of data to be written.

Definition at line 477 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::WriteRangeHelper().

int impala::DiskIoMgr::RequestRange::disk_id_

protectedinherited

Id of disk containing file_;.

Definition at line 286 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::RequestRange::disk_id(), impala::DiskIoMgr::HandleWriteFinished(), and impala::DiskIoMgr::ValidateScanRange().

std::string impala::DiskIoMgr::RequestRange::file_

protectedinherited

Path to file being read or written.

Definition at line 277 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::RequestRange::file(), impala::DiskIoMgr::Write(), and impala::DiskIoMgr::WriteRangeHelper().

hdfsFS impala::DiskIoMgr::RequestRange::fs_

protectedinherited

Hadoop filesystem that contains file_, or set to NULL for local filesystem.

Definition at line 274 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::RequestRange::fs().

int64_t impala::DiskIoMgr::RequestRange::len_

protectedinherited

Length of data read or written.

Definition at line 283 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::RequestRange::len(), impala::DiskIoMgr::ReadRange(), and impala::DiskIoMgr::WriteRangeHelper().

int64_t impala::DiskIoMgr::RequestRange::offset_

protectedinherited

Offset within file_ being read or written.

Definition at line 280 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::RequestRange::offset().

RequestType::type impala::DiskIoMgr::RequestRange::request_type_

protectedinherited

The type of IO request, READ or WRITE.

Definition at line 289 of file disk-io-mgr.h.

Referenced by impala::DiskIoMgr::RequestRange::request_type(), and impala::DiskIoMgr::ScanRange::ScanRange().

---

The documentation for this class was generated from the following files:

• be/src/runtime/disk-io-mgr.h
• be/src/runtime/disk-io-mgr.cc