Enable this option if you prefer to have queries fail when they exceed the Impala memory limit, rather than write temporary data to disk.
Queries that "spill" to disk typically complete successfully, when in earlier Impala releases they would have failed. However, queries with exorbitant memory requirements due to missing statistics or inefficient join clauses could become so slow as a result that you would rather have them cancelled automatically and reduce the memory usage through standard Impala tuning techniques.
This option prevents only "unsafe" spill operations, meaning that one or more tables are missing
statistics or the query does not include a hint to set the most efficient mechanism for a join or
INSERT ... SELECT into a partitioned table. These are the tables most likely to result in
suboptimal execution plans that could cause unnecessary spilling. Therefore, leaving this option enabled is a
good way to find tables on which to run the COMPUTE STATS statement.
See SQL Operations that Spill to Disk for information about the "spill to disk"
feature for queries processing large result sets with joins, ORDER BY, GROUP
BY, DISTINCT, aggregation functions, or analytic functions.
Type: Boolean; recognized values are 1 and 0, or true and
false; any other value interpreted as false
Default: false (shown as 0 in output of SET
statement)
Added in: Impala 2.0.0