impala::HdfsFileDesc Struct Reference

#include <hdfs-scan-node.h>

Collaboration diagram for impala::HdfsFileDesc:

Public Member Functions
	HdfsFileDesc (const std::string &filename)

Public Attributes
hdfsFS	fs
	Connection to the filesystem containing the file. More...
std::string	filename
	File name including the path. More...
int64_t	file_length
int64_t	mtime
	Last modified time. More...
THdfsCompression::type	file_compression
std::vector < DiskIoMgr::ScanRange * >	splits
	Splits (i.e. raw byte ranges) for this file, assigned to this scan node. More...

Detailed Description

Maintains per file information for files assigned to this scan node. This includes all the splits for the file. Note that it is not thread-safe.

Definition at line 53 of file hdfs-scan-node.h.

Constructor & Destructor Documentation

impala::HdfsFileDesc::HdfsFileDesc ( const std::string &  filename )

inline

Definition at line 71 of file hdfs-scan-node.h.

Member Data Documentation

THdfsCompression::type impala::HdfsFileDesc::file_compression

Definition at line 67 of file hdfs-scan-node.h.

Referenced by impala::HdfsTextScanner::Close(), impala::HdfsScanNode::CreateAndPrepareScanner(), impala::HdfsTextScanner::InitNewRange(), impala::HdfsScanNode::Prepare(), and impala::HdfsTextScanner::ProcessSplit().

int64_t impala::HdfsFileDesc::file_length

Length of the file. This is not related to which parts of the file have been assigned to this node.

Definition at line 62 of file hdfs-scan-node.h.

Referenced by impala::ScannerContext::AddStream(), impala::HdfsTextScanner::FillByteBufferCompressedFile(), impala::HdfsParquetScanner::InitColumns(), impala::HdfsScanNode::Prepare(), and impala::HdfsParquetScanner::ProcessFooter().

std::string impala::HdfsFileDesc::filename

File name including the path.

Definition at line 58 of file hdfs-scan-node.h.

Referenced by impala::HdfsParquetScanner::InitColumns(), and impala::HdfsScanNode::Prepare().

hdfsFS impala::HdfsFileDesc::fs

Connection to the filesystem containing the file.

Definition at line 55 of file hdfs-scan-node.h.

Referenced by impala::HdfsScanNode::Prepare().

int64_t impala::HdfsFileDesc::mtime

Last modified time.

Definition at line 65 of file hdfs-scan-node.h.

Referenced by impala::HdfsParquetScanner::InitColumns(), impala::HdfsScanNode::Prepare(), and impala::HdfsParquetScanner::ProcessFooter().

std::vector<DiskIoMgr::ScanRange*> impala::HdfsFileDesc::splits

Splits (i.e. raw byte ranges) for this file, assigned to this scan node.

Definition at line 70 of file hdfs-scan-node.h.

Referenced by impala::HdfsScanNode::AddDiskIoRanges(), impala::BaseSequenceScanner::CloseFileRanges(), and impala::HdfsScanNode::Prepare().

---

The documentation for this struct was generated from the following file:

• be/src/exec/hdfs-scan-node.h