Hi Bill, Thanks for the thoughtful question and for pointing this out.
There isn’t a strong technical requirement to change the ordering. I originally proposed `headers | payload` mainly because it is convenient to prefix the blob with a length and then read headers in one pass; from the state store’s point of view, everything is just a single opaque byte array, so both `headers | payload` and `payload | headers` are equivalent in terms of storage and performance. I agree that consistency with the Kafka Record format (where headers come after the value) is a valid concern, but I'm not convinced of the benefit yet. Our current storage method for the SS value is optimized for semi-lazy deserialization during large range scans. If we decide to re-incorporate the header length, I’ll update the ordering then. Thanks, Alieh On Thu, Jan 15, 2026 at 3:03 AM Chia-Ping Tsai <[email protected]> wrote: > hi > > chia_03: should we provide a more effective way to load the value without > scanning the header bytes? (e.g., by storing the total size of headers) > > chia_04: Do we need to allow separate Rocksdb configuration for the new > column family > > Best, > Chia-Ping > > On 2026/01/09 22:14:18 Alieh Saeedi via dev wrote: > > Hi all, > > > > I’d like to start a discussion on KIP-1271, which proposes allowing Kafka > > Streams state stores to preserve record headers. > > This would let header-based metadata like schema IDs, tracing info, and > > feature flags be stored and restored alongside values. > > The KIP introduces header-aware store types and a small config to cap the > > size of headers written into state. > > Details are in the KIP: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1271%3A+Allow+to+Store+Record+Headers+in+State+Stores > > . > > I’d appreciate your feedback and questions on the proposal. > > > > Thanks, > > Alieh > > >
