impala::HdfsAvroScanner::AvroFileHeader Struct Reference

Inheritance diagram for impala::HdfsAvroScanner::AvroFileHeader:

Collaboration diagram for impala::HdfsAvroScanner::AvroFileHeader:

Public Attributes
std::vector< SchemaElement >	schema
	List of SchemaElements corresponding to the fields of the file schema. More...
Tuple *	template_tuple
bool	use_codegend_decode_avro_data
uint8_t	sync [SYNC_HASH_SIZE]
	The sync hash for this file. More...
bool	is_compressed
	true if the file is compressed More...
std::string	codec
	Codec name if it is compressed. More...
THdfsCompression::type	compression_type
	Enum for compression type. More...
int64_t	header_size

Detailed Description

Definition at line 145 of file hdfs-avro-scanner.h.

Member Data Documentation

std::string impala::BaseSequenceScanner::FileHeader::codec

inherited

Codec name if it is compressed.

Definition at line 63 of file base-sequence-scanner.h.

Referenced by impala::HdfsSequenceScanner::InitNewRange(), impala::HdfsRCFileScanner::InitNewRange(), impala::HdfsAvroScanner::ParseMetadata(), impala::HdfsSequenceScanner::ReadFileHeader(), and impala::HdfsRCFileScanner::ReadFileHeader().

THdfsCompression::type impala::BaseSequenceScanner::FileHeader::compression_type

inherited

Enum for compression type.

Definition at line 66 of file base-sequence-scanner.h.

Referenced by impala::BaseSequenceScanner::Close(), impala::HdfsAvroScanner::InitNewRange(), impala::HdfsAvroScanner::ParseMetadata(), impala::HdfsAvroScanner::ProcessRange(), impala::HdfsSequenceScanner::ReadFileHeader(), and impala::HdfsRCFileScanner::ReadFileHeader().

int64_t impala::BaseSequenceScanner::FileHeader::header_size

inherited

Byte size of header. This must not include the sync directly preceding the data (even if the sync is considered part of the header in the file format spec).

Definition at line 70 of file base-sequence-scanner.h.

Referenced by impala::BaseSequenceScanner::ProcessSplit(), impala::HdfsAvroScanner::ReadFileHeader(), impala::HdfsSequenceScanner::ReadFileHeader(), and impala::HdfsRCFileScanner::ReadFileHeader().

bool impala::BaseSequenceScanner::FileHeader::is_compressed

inherited

true if the file is compressed

Definition at line 60 of file base-sequence-scanner.h.

Referenced by impala::HdfsSequenceScanner::GetRecord(), impala::HdfsAvroScanner::InitNewRange(), impala::HdfsSequenceScanner::InitNewRange(), impala::HdfsRCFileScanner::InitNewRange(), impala::HdfsAvroScanner::ParseMetadata(), impala::HdfsSequenceScanner::ProcessBlockCompressedScanRange(), impala::HdfsAvroScanner::ProcessRange(), impala::HdfsSequenceScanner::ProcessRange(), impala::BaseSequenceScanner::ProcessSplit(), impala::HdfsRCFileScanner::ReadColumnBuffers(), impala::HdfsSequenceScanner::ReadFileHeader(), impala::HdfsRCFileScanner::ReadFileHeader(), and impala::HdfsRCFileScanner::ReadKeyBuffers().

std::vector<SchemaElement> impala::HdfsAvroScanner::AvroFileHeader::schema

List of SchemaElements corresponding to the fields of the file schema.

Definition at line 147 of file hdfs-avro-scanner.h.

Referenced by impala::HdfsAvroScanner::MaterializeTuple(), impala::HdfsAvroScanner::ParseMetadata(), and impala::HdfsAvroScanner::ResolveSchemas().

uint8_t impala::BaseSequenceScanner::FileHeader::sync[SYNC_HASH_SIZE]

inherited

The sync hash for this file.

Definition at line 57 of file base-sequence-scanner.h.

Referenced by impala::BaseSequenceScanner::ProcessSplit(), impala::HdfsAvroScanner::ReadFileHeader(), impala::HdfsSequenceScanner::ReadFileHeader(), impala::HdfsRCFileScanner::ReadFileHeader(), and impala::BaseSequenceScanner::ReadSync().

Tuple* impala::HdfsAvroScanner::AvroFileHeader::template_tuple

Template tuple for this file containing partition key values and default values. NULL if there are no materialized partition keys and no default values are necessary (i.e., all materialized fields are present in the file schema). template_tuple_ is set to this value.

Definition at line 153 of file hdfs-avro-scanner.h.

Referenced by impala::HdfsAvroScanner::AllocateFileHeader(), impala::HdfsAvroScanner::InitNewRange(), and impala::HdfsAvroScanner::ResolveSchemas().

bool impala::HdfsAvroScanner::AvroFileHeader::use_codegend_decode_avro_data

True if this file can use the codegen'd version of DecodeAvroData() (i.e. its schema matches the table schema), false otherwise.

Definition at line 157 of file hdfs-avro-scanner.h.

Referenced by impala::HdfsAvroScanner::InitNewRange(), and impala::HdfsAvroScanner::ParseMetadata().

---

The documentation for this struct was generated from the following file:

• be/src/exec/hdfs-avro-scanner.h