Impala
Impalaistheopensource,nativeanalyticdatabaseforApacheHadoop.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros
Public Types | Public Member Functions | List of all members
impala::Scheduler Class Referenceabstract

#include <scheduler.h>

Inheritance diagram for impala::Scheduler:
Collaboration diagram for impala::Scheduler:

Public Types

typedef std::vector
< TBackendDescriptor > 		BackendList
 List of server descriptors. More...
 

Public Member Functions

virtual ~Scheduler ()
 
virtual Status GetBackends (const std::vector< TNetworkAddress > &data_locations, BackendList *backends)=0
 
virtual Status GetBackend (const TNetworkAddress &data_location, TBackendDescriptor *backend)=0
 
virtual bool HasLocalBackend (const TNetworkAddress &data_location)=0
 Return true if there is a backend located on the given data_location. More...
 
virtual void GetAllKnownBackends (BackendList *backends)=0
 Return a list of all backends known to the scheduler. More...
 
virtual Status Schedule (Coordinator *coord, QuerySchedule *schedule)=0
 
virtual Status Release (QuerySchedule *schedule)=0
 Releases the reserved resources (if any) from the given schedule. More...
 
virtual void HandlePreemptedReservation (const TUniqueId &reservation_id)=0
 
virtual void HandlePreemptedResource (const TUniqueId &client_resource_id)=0
 
virtual void HandleLostResource (const TUniqueId &client_resource_id)=0
 
virtual Status Init ()=0
 

Detailed Description

Abstract scheduler and nameservice class. Given a list of resources and locations returns a list of hosts on which to execute plan fragments requiring those resources. At the moment, this simply returns a list of registered backends running on those hosts.

Definition at line 40 of file scheduler.h.

Member Typedef Documentation

typedef std::vector<TBackendDescriptor> impala::Scheduler::BackendList

List of server descriptors.

Definition at line 45 of file scheduler.h.

Constructor & Destructor Documentation

virtual impala::Scheduler::~Scheduler ( )
inlinevirtual

Definition at line 42 of file scheduler.h.

Member Function Documentation

virtual void impala::Scheduler::GetAllKnownBackends ( BackendList *  backends)
pure virtual

Return a list of all backends known to the scheduler.

Implemented in impala::SimpleScheduler.

virtual Status impala::Scheduler::GetBackend ( const TNetworkAddress &  data_location,
TBackendDescriptor *  backend 
)
pure virtual

Return a host/port pair of known ImpalaInternalServices that is running on or nearby the given data location

Implemented in impala::SimpleScheduler.

virtual Status impala::Scheduler::GetBackends ( const std::vector< TNetworkAddress > &  data_locations,
BackendList *  backends 
)
pure virtual

Given a list of host / port pairs that represent data locations, fills in hostports with host/port pairs of known ImpalaInternalServices (that are running on those hosts or nearby).

Implemented in impala::SimpleScheduler.

virtual void impala::Scheduler::HandleLostResource ( const TUniqueId &  client_resource_id)
pure virtual

Notifies this scheduler that a single resource with the given client resource id has been lost by the central scheduler (Yarn via Llama). All affected queries are cancelled via their coordinator.

Implemented in impala::SimpleScheduler.

virtual void impala::Scheduler::HandlePreemptedReservation ( const TUniqueId &  reservation_id)
pure virtual

Notifies this scheduler that a resource reservation has been preempted by the central scheduler (Yarn via Llama). All affected queries are cancelled via their coordinator.

Implemented in impala::SimpleScheduler.

Referenced by impala::ResourceBroker::AMNotification().

virtual void impala::Scheduler::HandlePreemptedResource ( const TUniqueId &  client_resource_id)
pure virtual

Notifies this scheduler that a single resource with the given client resource id has been preempted by the central scheduler (Yarn via Llama). All affected queries are cancelled via their coordinator.

Implemented in impala::SimpleScheduler.

Referenced by impala::ResourceBroker::AMNotification().

virtual bool impala::Scheduler::HasLocalBackend ( const TNetworkAddress &  data_location)
pure virtual

Return true if there is a backend located on the given data_location.

Implemented in impala::SimpleScheduler.

virtual Status impala::Scheduler::Init ( )
pure virtual

Initialises the scheduler, acquiring all resources needed to make scheduling decisions once this method returns.

Implemented in impala::SimpleScheduler.

virtual Status impala::Scheduler::Release ( QuerySchedule *  schedule)
pure virtual

Releases the reserved resources (if any) from the given schedule.

Implemented in impala::SimpleScheduler.

Referenced by impala::ImpalaServer::QueryExecState::Done().

virtual Status impala::Scheduler::Schedule ( Coordinator *  coord,
QuerySchedule *  schedule 
)
pure virtual

Populates given query schedule whose execution is to be coordinated by coord. Assigns fragments to hosts based on scan ranges in the query exec request. If resource management is enabled, also reserves resources from the central resource manager (Yarn via Llama) to run the query in. This function blocks until the reservation request has been granted or denied.

Implemented in impala::SimpleScheduler.

Referenced by impala::ImpalaServer::QueryExecState::ExecQueryOrDmlRequest().

The documentation for this class was generated from the following file: