[
https://issues.apache.org/jira/browse/KAFKA-20749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Bejeck updated KAFKA-20749:
--------------------------------
Description: R{{{}ocksDBTransactionBuffer{}}}'s staging map
({{{}AbstractTransactionBuffer.pendingWrites{}}}) is keyed by key bytes only,
with no column-family dimension. Writes to the offsets column family (the
"position" key, written when the store persists its Position) are staged into
that shared map and then surface in data-column-family range()/all() scans. The
outer-join ListValueStore tries to deserialize the 40-byte serialized Position
as a value list and fails. The {{get()}} methods was already gated against the
offsets CF; the iterator paths were not. Non-transactional stores write
straight to the offsets CF and never leak, which is why it's
transactional-only. (was: RocksDBTransactionBuffer's staging map
(AbstractTransactionBuffer.pendingWrites) is keyed by key bytes only, with no
column-family dimension. Writes to the offsets column family (the "position"
key, written when the store persists its Position) are staged into that shared
map and then surface in data-column-family range()/all() scans. The outer-join
ListValueStore tries to deserialize the 40-byte serialized Position as a value
list and fails. get() was already gated against the offsets CF; the iterator
paths were not. Non-transactional stores write straight to the offsets CF and
never leak, which is why it's transactional-only. )
> KStream-KStream outer/left join crashes under EOS with transactional state
> stores
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-20749
> URL: https://issues.apache.org/jira/browse/KAFKA-20749
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.4.0
> Reporter: Bill Bejeck
> Assignee: Bill Bejeck
> Priority: Critical
> Fix For: 4.4.0
>
>
> R{{{}ocksDBTransactionBuffer{}}}'s staging map
> ({{{}AbstractTransactionBuffer.pendingWrites{}}}) is keyed by key bytes only,
> with no column-family dimension. Writes to the offsets column family (the
> "position" key, written when the store persists its Position) are staged into
> that shared map and then surface in data-column-family range()/all() scans.
> The outer-join ListValueStore tries to deserialize the 40-byte serialized
> Position as a value list and fails. The {{get()}} methods was already gated
> against the offsets CF; the iterator paths were not. Non-transactional stores
> write straight to the offsets CF and never leak, which is why it's
> transactional-only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
