Resource Management for Impala
The use of the Llama component for integrated resource management within YARN is no longer supported with Impala 2.3 and higher. The Llama support code is removed entirely in Impala 2.8 and higher.
For clusters running Impala alongside other data management components, you define static service pools to define the resources available to Impala and other components. Then within the area allocated for Impala, you can create dynamic service pools, each with its own settings for the Impala admission control feature.
You can limit the CPU and memory resources used by Impala, to manage and prioritize workloads on clusters that run jobs from many Hadoop components.
How Resource Limits Are Enforced
Limits on memory usage are enforced by Impala's process memory limit (the MEM_LIMIT
query option setting). The admission control feature checks this setting to decide how many queries
can be safely run at the same time. Then the Impala daemon enforces the limit by activating the
spill-to-disk mechanism when necessary, or cancelling a query altogether if the limit is exceeded at runtime.
impala-shell Query Options for Resource Management
Before issuing SQL statements through the impala-shell interpreter, you can use the
SET command to configure the following parameters related to resource management:
Limitations of Resource Management for Impala
The MEM_LIMIT query option, and the other resource-related query options, are settable
through the ODBC or JDBC interfaces in Impala 2.0 and higher. This is a former limitation that is now
lifted.