impala::DiskIoMgrStress Class Reference

#include <disk-io-mgr-stress.h>

Collaboration diagram for impala::DiskIoMgrStress:

Classes
struct	Client
struct	File

Public Member Functions
	DiskIoMgrStress (int num_disks, int num_threads_per_disk, int num_clients, bool includes_cancellation)
void	Run (int sec)
	Run the test for 'sec'. If 0, run forever. More...

Private Member Functions
void	NewClient (int i)
void	ClientThread (int client_id)
void	CancelRandomReader ()
	Possibly cancels a random reader. More...

Private Attributes
std::vector< File >	files_
MemTracker	dummy_tracker_
	Dummy mem tracker. More...
boost::scoped_ptr< DiskIoMgr >	io_mgr_
	io manager More...
boost::thread_group	readers_
	Thread group for reader threads. More...
int	num_clients_
	Array of clients. More...
Client *	clients_
bool	includes_cancellation_
	If true, tests cancelling readers. More...
volatile bool	shutdown_
	Flag to signal that client reader threads should exit. More...

Detailed Description

Test utility to stress the disk io mgr. It allows for a configurable number of clients. The clients continuously issue work to the io mgr and asynchronously get cancelled. The stress test can be run forever or for a fixed duration. The unit test runs this for a fixed duration.

Definition at line 36 of file disk-io-mgr-stress.h.

Constructor & Destructor Documentation

DiskIoMgrStress::DiskIoMgrStress	(	int	num_disks,
		int	num_threads_per_disk,
		int	num_clients,
		bool	includes_cancellation
	)

Definition at line 64 of file disk-io-mgr-stress.cc.

References clients_, CreateTempFile(), dummy_tracker_, files_, GenerateRandomData(), io_mgr_, MAX_READ_BUFFER_SIZE, MIN_READ_BUFFER_SIZE, NewClient(), num_clients_, and impala::Status::ok().

Member Function Documentation

void DiskIoMgrStress::CancelRandomReader ( )

private

Possibly cancels a random reader.

Definition at line 165 of file disk-io-mgr-stress.cc.

References clients_, includes_cancellation_, io_mgr_, and num_clients_.

Referenced by Run().

void DiskIoMgrStress::ClientThread ( int  client_id )

private

Thread running the reader. When the current reader is done (either normally or cancelled), it picks up a new reader

Definition at line 94 of file disk-io-mgr-stress.cc.

References impala::DiskIoMgrStress::Client::abort_at_byte, impala::DiskIoMgr::BufferDescriptor::buffer(), clients_, impala::DiskIoMgrStress::Client::file_idx, files_, impala::DiskIoMgr::ScanRange::GetNext(), io_mgr_, impala::Status::IsCancelled(), impala::DiskIoMgr::BufferDescriptor::len(), impala::DiskIoMgrStress::Client::lock, MAX_FILE_LEN, NewClient(), impala::DiskIoMgr::RequestRange::offset(), impala::Status::ok(), impala::DiskIoMgrStress::Client::reader, impala::DiskIoMgr::BufferDescriptor::Return(), impala::DiskIoMgr::BufferDescriptor::scan_range_offset(), and shutdown_.

Referenced by Run().

void DiskIoMgrStress::NewClient ( int  i )

private

Helper to initialize a new reader client, registering a new reader with the io mgr and initializing the scan ranges

Definition at line 204 of file disk-io-mgr-stress.cc.

References impala::DiskIoMgrStress::Client::abort_at_byte, ABORT_CHANCE, clients_, impala::DiskIoMgrStress::Client::file_idx, files_, impala::DiskIoMgrStress::Client::files_processed, includes_cancellation_, io_mgr_, MAX_READ_LEN, MIN_READ_LEN, impala::DiskIoMgr::ScanRange::NEVER_CACHE, impala::Status::ok(), impala::DiskIoMgrStress::Client::reader, impala::DiskIoMgr::ScanRange::Reset(), and impala::DiskIoMgrStress::Client::scan_ranges.

Referenced by ClientThread(), and DiskIoMgrStress().

void DiskIoMgrStress::Run

(

int

sec

)

Run the test for 'sec'. If 0, run forever.

Definition at line 174 of file disk-io-mgr-stress.cc.

References CANCEL_READER_PERIOD_MS, CancelRandomReader(), clients_, ClientThread(), io_mgr_, num_clients_, readers_, shutdown_, and impala::SleepForMs().

Referenced by main(), and impala::TEST_F().

Member Data Documentation

Client* impala::DiskIoMgrStress::clients_

private

Definition at line 68 of file disk-io-mgr-stress.h.

Referenced by CancelRandomReader(), ClientThread(), DiskIoMgrStress(), NewClient(), and Run().

MemTracker impala::DiskIoMgrStress::dummy_tracker_

private

Dummy mem tracker.

Definition at line 58 of file disk-io-mgr-stress.h.

Referenced by DiskIoMgrStress().

std::vector<File> impala::DiskIoMgrStress::files_

private

Files used for testing. These are created at startup and recycled during the test

Definition at line 55 of file disk-io-mgr-stress.h.

Referenced by ClientThread(), DiskIoMgrStress(), and NewClient().

bool impala::DiskIoMgrStress::includes_cancellation_

private

If true, tests cancelling readers.

Definition at line 71 of file disk-io-mgr-stress.h.

Referenced by CancelRandomReader(), and NewClient().

boost::scoped_ptr<DiskIoMgr> impala::DiskIoMgrStress::io_mgr_

private

io manager

Definition at line 61 of file disk-io-mgr-stress.h.

Referenced by CancelRandomReader(), ClientThread(), DiskIoMgrStress(), NewClient(), and Run().

int impala::DiskIoMgrStress::num_clients_

private

Array of clients.

Definition at line 67 of file disk-io-mgr-stress.h.

Referenced by CancelRandomReader(), DiskIoMgrStress(), and Run().

boost::thread_group impala::DiskIoMgrStress::readers_

private

Thread group for reader threads.

Definition at line 64 of file disk-io-mgr-stress.h.

Referenced by Run().

volatile bool impala::DiskIoMgrStress::shutdown_

private

Flag to signal that client reader threads should exit.

Definition at line 74 of file disk-io-mgr-stress.h.

Referenced by ClientThread(), and Run().

---

The documentation for this class was generated from the following files:

• be/src/runtime/disk-io-mgr-stress.h
• be/src/runtime/disk-io-mgr-stress.cc