[
https://issues.apache.org/jira/browse/IMPALA-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18044488#comment-18044488
]
ASF subversion and git services commented on IMPALA-14131:
----------------------------------------------------------
Commit 1684c2d9da20cd27dcfc30dbaf940ac8144b496f in impala's branch
refs/heads/master from Sai Hemanth Gantasala
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1684c2d9d ]
IMPALA-14131: Add flag to configure the default value of
'impala.disableHmsSync'
FEATURE: Implement global 'disable_hms_sync_by_default' flag for event
processing. This change introduces a new catalogd startup flag,
`disable_hms_sync_by_default`, to simplify skipping/processing events.
Problem: Disabling event processing globally requires tedious process
of setting 'impala.disableHmsSync' property on every database and table,
especially if few specific tables requires sync up of events.
Solution: The new flag provides a global default for the
'impala.disableHmsSync' property.
Behavior:
- If `disable_hms_sync_by_default` is true (the intended default-off
state), event processing is skipped for all tables/databases unless
the property "impala.disableHmsSync"="false" is explicitly set.
- This allows users to easily keep event processing off by default
and opt-in specific databases or tables to start syncing.
- The check order is: table-property > db-property > global default.
- HMS polling remains independent and unaffected by this flag.
Change-Id: I4ee617aed48575502d9cf5cf2cbea6ec897d6839
Reviewed-on: http://gerrit.cloudera.org:8080/23487
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add flag to configure the default value of "impala.disableHmsSync"
> ------------------------------------------------------------------
>
> Key: IMPALA-14131
> URL: https://issues.apache.org/jira/browse/IMPALA-14131
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Sai Hemanth Gantasala
> Priority: Major
>
> Users might want to roll out EventProcessor job by job and turn it off on all
> db/tables by default. It's tedious to set "impala.disableHmsSync" to "true"
> on all dbs. If a db has some tables that want to turn event processing on,
> users should set "impala.disableHmsSync" to "true" on all other tables in
> that db, which is also tedious.
> We need a catalogd startup flag like disable_hms_sync_by_default which is the
> default value of "impala.disableHmsSync" when it's missing in db/table
> properties. It's independent to hms_event_polling_interval_s that catalogds
> will still poll HMS events based on hms_event_polling_interval_s, but if
> disable_hms_sync_by_default is true, events on all db/tables will be skipped
> unless the db/table has property "impala.disableHmsSync"="false".
> CC [~hemanth619], [~VenuReddy]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
