[
https://issues.apache.org/jira/browse/HIVE-29035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18105254#comment-18105254
]
Henri Biestro commented on HIVE-29035:
--------------------------------------
[~okumin] thanks for your review.
As it stands, HIVE-29035 alone even with no L1 cache, has a variation of the
authz flaw you pointed our early.
On a cold miss it calls hiveCatalog.loadTable → HMS get_table_req, which fires
PreReadTableEvent and runs HiveMetaStoreAuthorizer (QUERY/TABLE_OR_VIEW) — so
cold loads are authorized by HMS.
But on a cache hit no such call happens:
• L2 hit (location matches): the only HMS call is MetadataLocator.getLocation →
getTables(..., GetProjectionsSpec) → RPC get_table_objects_by_name_req →
GetTableHandler.getTableObjectsInternal, which reads RawStore directly and
fires no PreReadTableEvent. The cached Table is returned unauthorized.
• L1 hit (recency guard): returns the cached Table with no HMS call at all.
So read authorization is bypassed on every cache hit — the bypass is a property
of caching in general, not of the L1 guard specifically. The pre-event
authorizer only ever runs on the cold get_table_req path.
Consequence. Removing or relocating L1 does not close the hole: pure-L2 still
returns unauthorized reads on the (common) location-match hit. The fix has to
authorize reads on the REST layer independently of cache state.
This is why #6702 (HIVE-29817) is self-sufficient: it carries the cache and the
authz that makes the cache safe.
#6702 includes all of #6441. #6441 head ab7f86470d is an ancestor of #6702 head
a495197940 — #6702 is exactly #6441 plus one commit (the read/list authz). That
one extra commit adds the unconditional authorizeLoadTable (and
authorizeLoadView + list filters) at the adapter, which runs on every read
regardless of L1/L2/cold cache state.
On a cold miss, we will indeed check authz twice, once at the adapter level and
another at HMS level; it is a cheap in-memory policy eval but building the
authorizer is not. I'll amend #6702 to further reduce the cost.
> Fixing cache handling for REST catalog
> --------------------------------------
>
> Key: HIVE-29035
> URL: https://issues.apache.org/jira/browse/HIVE-29035
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Standalone Metastore
> Reporter: Shohei Okumiya
> Assignee: Henri Biestro
> Priority: Major
> Labels: pull-request-available
>
> We initially implemented the server-side caching with CachingCatalog, but
> disabled it in HIVE-29016.
> We have some unknowns about the caching architecture.
> * Where should we cache? Client? REST Catalog? Thrift?
> * What should we cache? HMS's records? Manifest?
> * What strategy is needed to ensure the cache coherency?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
