BUFFER_POOL_LIMIT Query Option

Defines a limit on the amount of memory that a query can allocate from the internal buffer pool. The value for this limit applies to the memory on each host, not the aggregate memory across the cluster. Typically not changed by users, except during diagnosis of out-of-memory errors during queries.

Type: integer

Default:

The default setting for this option is the lower of 80% of the MEM_LIMIT setting, or the MEM_LIMIT setting minus 100 MB.

Added in: Impala 2.10.0

Usage notes:

If queries encounter out-of-memory errors, consider decreasing the BUFFER_POOL_LIMIT setting to less than 80% of the MEM_LIMIT setting.

Examples:

-- Set an absolute value.
set buffer_pool_limit=8GB;

-- Set a relative value based on the MEM_LIMIT setting.
set buffer_pool_limit=80%;

Related information:

DEFAULT_SPILLABLE_BUFFER_SIZE Query Option, MAX_ROW_SIZE Query Option, MIN_SPILLABLE_BUFFER_SIZE Query Option, Scalability Considerations for Impala