ealeonraz opened a new pull request, #13106:
URL: https://github.com/apache/gravitino/pull/13106
### What changes were proposed in this pull request?
Add an optional `redis` entity cache behind the existing `EntityCache` SPI,
selected with `gravitino.cache.implementation=redis`. The default stays
`caffeine`; no caller changes.
- `RedisEntityCache` reports `Coherence.SHARED` and keeps one copy of every
cached entity in Redis. Keys are `<ns>:{<metalake>}:D:<identifier>:<TYPE>` for
values, `<ns>:{<metalake>}:F:<identifier>` for version fences, and
`<ns>:{<metalake>}:IDX` for a lex-ordered index ZSet. The metalake is the Redis
Cluster hash tag, so every multi-key script is single-slot at any depth (no
`CROSSSLOT`).
- Container drop is one Lua script: bump the dropped identifier's fence,
delete its value, then `ZRANGEBYLEX` the index over `<identifier>.` (and
`<identifier><schema separator>` for a schema, mirroring #12416) and delete
each descendant. Readers never see a torn drop.
- Stale-write guard: a read miss records the fences of the identifier and of
every ancestor for the calling thread; the write that fills the entry is a Lua
script that re-reads those fences and refuses to write if any moved. A load
that began before a drop of the entity or of any container above it (including
a nested-schema parent) cannot refill the key, whether or not it was indexed.
Fences outlive the value TTL (`gravitino.cache.redis.fenceTtlMs`, default 2x
`expireTimeInMs`), so a slow reader cannot win by outliving the value.
- Failure policy: reads and fills degrade to a miss on any Redis error or
timeout; an invalidation failure is propagated so it is never silently dropped;
an unreachable Redis at startup fails fast.
- `KryoEntitySerializer`: Kryo with `CompatibleFieldSerializer` (field names
written, so rolling upgrades tolerate added/removed fields), Objenesis
instantiation, `Type` values written in the REST JSON form so primitive type
singletons come back as the same instances, and Guava immutable collections
rebuilt on read. An undecodable entry is discarded and treated as a miss.
- `CacheFactory` registers `"redis"`; new `gravitino.cache.redis.*` config
entries; docs and LICENSE.bin updated for Jedis (MIT), Kryo/ReflectASM/MinLog
(BSD-3), Objenesis (Apache-2), org.json (public domain).
Scope, as agreed on the issue: this is a shared cache, not a strongly
consistent database/cache pair. The cache is only touched after the store
commits, and a crash between commit and invalidation leaves staleness bounded
by the value TTL. Consistency-sensitive call sites still need the fresh-read
path from #12150. No dependency on #12151: the writing node invalidates the
single shared copy directly.
### Why are the changes needed?
Phase 2 of #11737. With per-node Caffeine caches, a write on one node leaves
the other nodes' copies stale until the change-log poller reaches them; a
shared cache removes that cross-node divergence for deployments that accept a
network hop per read.
Fix: #12020
### Does this PR introduce _any_ user-facing change?
New optional configuration, all additive:
| Key | Default |
|-----|---------|
| `gravitino.cache.redis.address` | (required when `implementation=redis`) |
| `gravitino.cache.redis.cluster` | `false` |
| `gravitino.cache.redis.namespace` | `gravitino` |
| `gravitino.cache.redis.fenceTtlMs` | `0` (twice `expireTimeInMs`) |
| `gravitino.cache.redis.serializer` | `kryo` |
| `gravitino.cache.redis.timeoutMs` | `1000` |
| `gravitino.cache.redis.username` / `password` | (none) |
`gravitino.cache.expireTimeInMs` is reused as the Redis value TTL. New
bundled dependencies in `core`: `redis.clients:jedis` and
`com.esotericsoftware:kryo`.
### How was this patch tested?
Unit tests (no Redis needed): `TestRedisKeyspace` (key layout, hash tag,
ancestor fence paths incl. nested schemas, descendant prefixes),
`TestKryoEntitySerializer` (round trip of every cacheable entity type, type
singleton identity, garbage rejection), `TestRedisEntityCache` (config
validation, factory registration, fail-fast startup, failure policy with a
mocked client, fence bookkeeping between miss and write, single-slot hashing
via `JedisClusterCRC16`).
Integration tests tagged `gravitino-docker-test`: `RedisEntityCacheIT`
(Testcontainers `redis:7.2-alpine`) and `RedisClusterEntityCacheIT`
(`grokzen/redis-cluster`, 3 masters) share `RedisEntityCacheTestBase`:
cross-node freshness, catalog/metalake/schema drops incl. nested schemas and
prefix siblings, stale write rejected after same-key invalidation, after an
ancestor drop of a never-indexed key, and after a nested-schema parent drop,
reload after invalidation accepted, unconditional write without a preceding
miss, the store's lock/miss/load/put pattern with a concurrent invalidation,
concurrent readers and writers across two nodes, TTL expiry, clear/size across
metalakes and cluster nodes, undecodable entry discarded, non-cacheable types
not stored, namespace isolation, and a 40-metalake drop sweep on the cluster.
Both suites also accept an external server via `GRAVITINO_REDIS_ADDRESS` /
`GRAVITINO_REDIS_CLUSTER_ADDRESS`; I ran them that way against a local Redis
7.2.7 standalon
e and a local 3-master cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
