REFRESH_UPDATED_HMS_PARTITIONS Query Option

As the name implies the query option REFRESH_UPDATED_HMS_PARTITIONS is used to refresh any updated HMS partitions.

This option is disabled by default so that the performance is not compromised when refreshing a table. However, for certain corner case scenarios refresh table command does not detect changed partitions. In case of the default refresh, catalogd detects, adds any new partitions and removes any partitions which are not present in HMS anymore. However, it does not update any partitions that changed (eg. change of location). When this query option is enabled, the refresh table command will detect certain changes to the partitions and update them accordingly. Currently, catalogd will update the partitions if any of the following StorageDescriptor properties have been modified. 1. Partition Location 2. Partition Fileformat. 3 SerdeInfo. 4. Partition schema changes. 5. Partition bucketInfo.

Type:BOOLEAN; recognized values are 1 and 0, or true and false; any other value will be interpreted as false.

Default:FALSE (shown as 0 in output of SET statement).

Added in: Impala 4.0