When the LIVE_SUMMARY query option is set
to TRUE, Impala displays the same output as the
SUMMARY command for queries submitted through the
impala-shell command, with the measurements updated
in real time as the query progresses. When the query finishes, the final
SUMMARY output remains visible in the
impala-shell console output.
Starting in Impala 3.1, the summary output also includes the queuing status consisting of whether the query was queued and what was the latest queuing reason.
Type:
Boolean
Default:
FALSE (0)
Command-line equivalent:
You can enable this query option within impala-shell
by starting the shell with the --live_summary
command-line option. You can still turn this setting off and on again
within the shell through the SET command.
Usage notes:
The live summary output can be useful for evaluating long-running queries, to evaluate which phase of execution takes up the most time, or if some hosts take much longer than others for certain operations, dragging overall performance down. By making the information available in real time, this feature lets you decide what action to take even before you cancel a query that is taking much longer than normal.
For example, you might see the HDFS scan phase taking a long time, and
therefore revisit performance-related aspects of your schema design such
as constructing a partitioned table, switching to the Parquet file format,
running the COMPUTE STATS statement for the table, and so
on. Or you might see a wide variation between the average and maximum
times for all hosts to perform some phase of the query, and therefore
investigate if one particular host needed more memory or was experiencing
a network problem.
The output from this query option is printed to standard
error. The output is displayed only in interactive mode, not when the -q
or -f options are used.
For a simple and concise way of tracking the progress of an interactive query, see LIVE_PROGRESS Query Option (Impala 2.3 or higher only).
Restrictions:
The LIVE_PROGRESS and LIVE_SUMMARY query options
currently do not produce any output during COMPUTE STATS operations.
LIVE_PROGRESS and LIVE_SUMMARY query
options are available only within the impala-shell interpreter:
You cannot change these query options through the SQL SET
statement using the JDBC or ODBC interfaces. The SET command in
impala-shell recognizes these names as shell-only options.
Be careful when using impala-shell on a pre-Impala 2.3 system to connect to a system running Impala 2.3 or higher. The older impala-shell does not recognize these query option names. Upgrade impala-shell on the systems where you intend to use these query options.
Likewise, the impala-shell command relies on some information
only available in Impala 2.3 and higher to prepare live
progress reports and query summaries. The LIVE_PROGRESS and
LIVE_SUMMARY query options have no effect when
impala-shell connects to a cluster running an older version of
Impala.
Added in: Impala 2.3.0
To see how the LIVE_PROGRESS and LIVE_SUMMARY query
options work in real time, see
this
animated demo.