impala-shell Configuration Options
You can specify the following options when starting the
impala-shell
command to change how shell commands are
executed. The table shows the format to use when specifying each option on
the command line, or through the impala-shell
configuration file.
These options are different than the configuration options for the impalad
daemon itself.
For the impalad
options, see Modifying Impala Startup Options.
Summary of impala-shell Configuration Options
The following table shows the names and allowed arguments for the impala-shell configuration options. You can specify options on the command line, or in a configuration file as described in impala-shell Configuration File.
Command-Line Option | Configuration File Setting | Explanation |
---|---|---|
-B or --delimited |
write_delimited=true |
Causes all query results to be printed in plain format as a delimited text file. Useful for
producing data files to be used with other Hadoop components. Also useful for avoiding the
performance overhead of pretty-printing all output, especially when running benchmark tests using
queries returning large result sets. Specify the delimiter character with the
|
-b or --kerberos_host_fqdn |
kerberos_host_fqdn= load-balancer-hostname |
If set, the setting overrides the expected hostname of the
Impala daemon's Kerberos service principal.
impala-shell will check that the server's
principal matches this hostname. This may be used when
|
--print_header |
print_header=true |
|
-o filename or --output_file filename |
output_file=filename |
Stores all query results in the specified file. Typically used to store the results of a single
query issued from the command line with the |
--output_delimiter=character |
output_delimiter=character |
Specifies the character to use as a delimiter between fields when query results are printed in
plain format by the |
-E or --vertical |
vertical=true |
Print the output of a query (rows) vertically, no effects if |
-p or --show_profiles |
show_profiles=true |
Displays the query execution plan (same output as the |
-h or --help |
N/A |
Displays help information. |
N/A |
history_max=1000 |
Sets the maximum number of queries to store in the history file. |
-i hostname or --impalad=hostname[:portnum] |
impalad=hostname[:portnum] |
Connects to the |
-q query or --query=query |
query=query |
Passes a query or other impala-shell command from the command line. The
impala-shell interpreter immediately exits after processing the statement. It
is limited to a single statement, which could be a |
-f query_file or --query_file=query_file |
query_file=path_to_query_file |
Passes a SQL query from a file. Multiple statements must be semicolon (;) delimited.
In Impala 2.3 and higher, you can specify a filename of |
--query_option= "option= value" -Q "option= value" |
Header line |
Sets default query options for an invocation of the impala-shell command. To set multiple query options at once, use more than one instance of this command-line option. The query option names are not case-sensitive. |
-k or --kerberos |
use_kerberos=true |
Kerberos authentication is used when the shell connects to See Enabling Kerberos Authentication for Impala for the steps to set up and use Kerberos authentication in Impala. |
-s kerberos_service_name or --kerberos_service_name=name |
kerberos_service_name=name |
Instructs |
-V or --verbose |
verbose=true |
Enables verbose output. |
--quiet |
verbose=false |
Disables verbose output. |
-v or --version |
version=true |
Displays version information. |
-c |
ignore_query_failure=true |
Continues on query failure. |
-d default_db or --database=default_db |
default_db=default_db |
Specifies the database to be used on startup. Same as running the
|
-ssl | ssl=true | Enables TLS/SSL for impala-shell. |
--ca_cert=path_to_certificate | ca_cert=path_to_certificate |
The local pathname pointing to the third-party CA certificate, or to a copy of the server
certificate for self-signed server certificates. If --ca_cert is not set,
impala-shell enables TLS/SSL, but does not validate the server certificate. This is
useful for connecting to a known-good Impala that is only running over TLS/SSL, when a copy of the
certificate is not available (such as when debugging customer installations).
|
-l | use_ldap=true | Enables LDAP authentication. |
-u | user=user_name |
Supplies the username, when LDAP authentication is enabled by the -l option.
(Specify the short username, not the full LDAP distinguished name.) The shell then prompts
interactively for the password.
|
--ldap_password_cmd=command | N/A |
Specifies a command to run to retrieve the LDAP password,
when LDAP authentication is enabled by the -l option.
If the command includes space-separated arguments, enclose the command and
its arguments in quotation marks.
|
--config_file=path_to_config_file | N/A | Specifies the path of the file containing impala-shell configuration settings. The default is /etc/impalarc. This setting can only be specified on the command line. |
--live_progress | live_progress=true | Prints a progress bar showing roughly the percentage complete for each query. The information is updated interactively as the query progresses. See LIVE_PROGRESS Query Option (Impala 2.3 or higher only). |
--disable_live_progress | live_progress=false | A command line flag allows users to disable live_progress in the interactive mode. |
--live_summary | N/A | Prints a detailed report, similar to the SUMMARY command, showing progress details for each phase of query execution.
The information is updated interactively as the query progresses.
See LIVE_SUMMARY Query Option (Impala 2.3 or higher only). |
--var=variable_name=value | N/A |
Defines a substitution variable that can be used within the impala-shell session.
The variable can be substituted into statements processed by the -q or -f options,
or in an interactive shell session.
Within a SQL statement, you substitute the value by using the notation ${var:variable_name} .
This feature is available in Impala 2.5 and higher.
|
--auth_creds_ok_in_clear | N/A | Allows LDAP authentication to be used with an insecure connection to the shell. WARNING: This will allow authentication credentials to be sent unencrypted, and hence may be vulnerable to an attack. |
--protocol=protocol | N/A | Protocol to use for the connection to Impala. Valid
protocol values are:
You cannot connect to the 3.2 or earlier versions
of Impala using the Beeswax support is deprecated and will be removed in the future. |
--hs2_fp_format=HS2_FP_FORMAT | hs2_fp_format=HS2_FP_FORMAT | Sets the printing format specification for floating point values when using HS2 protocol. The default behaviour makes the values handled by Python's str() built-in method. Use '16G' to match Beeswax protocol's floating-point output format This feature is available in Impala 4.2 and higher. |
impala-shell Configuration File
You can store a set of default settings for
impala-shell in the impala-shell
configuration file.
The global impala-shell
configuration file is located
in /etc/impalarc
.
The user-level impala-shell
configuration file is
located in ~/.impalarc
.
Note that the global-level file name is different from the user-level file name. The global-level file name does not include a dot (.) in the file name.
The default path of the global configuration file can be changed by
setting the $IMPALA_SHELL_GLOBAL_CONFIG_FILE
environment variable.
To specify a different file name or path for the user-level
configuration file, start impala-shell with the
--config_file
impala-shell option set to the path of the
configuration file.
Typically, an administrator creates the global configuration file for the impala-shell, and if the user-level configuration file exists, the options set in the user configuration file take precedence over those in the global configuration file.
In turn, any options you specify on the impala-shell command line override any corresponding options within the configuration file.
The impala-shell
configuration file (global or user)
configuration file must contain a header label
[impala]
, followed by the options specific to
impala-shell.
The impala-shell
configuration file consists of
key-value pairs, one option per line. Everything after the
#
character on a line is treated as a comment and
ignored.
The names of the options in the configuration file are similar (although not necessarily identical) to the long-form command-line arguments to the impala-shell command. For the names to use, see Summary of impala-shell Configuration Options.
You can specify key-value pair options using keyval
,
similar to the --var
command-line option. For example,
keyval=
variable1=value1.
The query options specified in the [impala]
section
override the options specified in the
[impala.query_options]
section.
The following example shows a configuration file that you might use
during benchmarking tests. It sets verbose mode, so that the output from
each SQL query is followed by timing information.
impala-shell starts inside the database containing
the tables with the benchmark data, avoiding the need to issue a
USE
statement or use fully qualified table names.
In this example, the query output is formatted as delimited text
rather than enclosed in ASCII art boxes, and is stored in a file rather
than printed to the screen. Those options are appropriate for benchmark
situations, so that the overhead of impala-shell
formatting and printing the result set does not factor into the timing
measurements. It also enables the show_profiles
option.
That option prints detailed performance information after each query,
which might be valuable in understanding the performance of benchmark
queries.
[impala]
verbose=true
default_db=tpc_benchmarking
write_delimited=true
output_delimiter=,
output_file=/home/tester1/benchmark_results.csv
show_profiles=true
keyval=msg1=hello,keyval=msg2=world
The following example shows a configuration
file that connects to a specific remote Impala node, runs a single query
within a particular database, then exits. Any query options predefined
under the [impala.query_options]
section in the
configuration file take effect during the session.
You would typically use this kind of single-purpose configuration
setting with the impala-shell command-line option
--config_file=path_to_config_file
,
to easily select between many predefined queries that could be run
against different databases, hosts, or even different clusters. To run a
sequence of statements instead of a single query, specify the
configuration option
query_file=path_to_query_file
instead.
[impala]
impalad=impala-test-node1.example.com
default_db=site_stats
# Issue a predefined query and immediately exit.
query=select count(*) from web_traffic where event_date = trunc(now(),'dd')
[impala.query_options]
mem_limit=32g