impala::ParallelExecutor Class Reference

#include <parallel-executor.h>

Collaboration diagram for impala::ParallelExecutor:

Public Types
typedef boost::function < Status(void *arg)>	Function

Static Public Member Functions
static Status	Exec (Function function, void **args, int num_args, StatsMetric< double > *latencies=NULL)
	Callers may pass a StatsMetric to gather the latency distribution of task execution. More...

Static Private Member Functions
static void	Worker (Function function, void *arg, boost::mutex *lock, Status *status, StatsMetric< double > *latencies)

Detailed Description

This is a class that executes multiple functions in parallel with different arguments using a thread pool. TODO: look into an API for this. Boost has one that is in review but not yet official. TODO: use a shared pool? Thread creation is pretty cheap so this might not be worth it TODO: Consider rewriting in terms of ThreadPool

Definition at line 33 of file parallel-executor.h.

Member Typedef Documentation

typedef boost::function<Status (void* arg)> impala::ParallelExecutor::Function

Typedef for the underlying function for the work. The function must be thread safe. The function must return a Status indicating if it was successful or not. An example of how this function should be defined would be: static Status Foo::IssueRpc(void* arg); TODO: there might some magical template way to do this with boost that is more type safe.

Definition at line 42 of file parallel-executor.h.

Member Function Documentation

Status ParallelExecutor::Exec ( Function  function, void **  args, int  num_args, StatsMetric< double > *  latencies = NULL  )

static

Callers may pass a StatsMetric to gather the latency distribution of task execution.

Calls function(args[i]) num_args times in parallel using num_args threads. If any of the work item fails, returns the Status of the first failed work item. Otherwise, returns Status::OK when all work items have been executed.

Definition at line 26 of file parallel-executor.cc.

References impala::ThreadGroup::AddThread(), impala::ThreadGroup::JoinAll(), and Worker().

Referenced by impala::TEST().

void ParallelExecutor::Worker ( Function  function, void *  arg, boost::mutex *  lock, Status *  status, StatsMetric< double > *  latencies  )

staticprivate

Worker thread function which calls function(arg). This function updates *status taking *lock to synchronize results from different threads. If 'latencies' is not NULL, it is updated with the time elapsed while executing 'function'.

Definition at line 43 of file parallel-executor.cc.

References impala::MonotonicStopWatch::ElapsedTime(), impala::Status::ok(), impala::MonotonicStopWatch::Start(), and impala::StatsMetric< T >::Update().

Referenced by Exec().

---

The documentation for this class was generated from the following files:

• be/src/runtime/parallel-executor.h
• be/src/runtime/parallel-executor.cc