Ah! Yes. Thank you! That make sense.
Anyway, I _think_ that's not what I was doing given that all items were
being routed to and then read from a partition identified by one key.
On Fri, Mar 17, 2017 at 12:50 PM, Damian Guy <damian....@gmail.com> wrote:
> > When you use Queryable State you are actually querying multiple
>
> > underlying stores, i.e., one per partition.
> >
> > Huh? I was only querying one partition. In my example, I have a user's
> > posts. Upon creation, they are routed to a particular partition using a
> > partitioner that hashes the post's user ID. The posts are then indexed on
> > that partition by prefixed keys using the method described above. When
> > querying, I am only querying the one partition that has all of the user's
> > posts. As far as I know, I am not querying across multiple partitions.
> > Furthermore, I did not even think this was possible, given the fact that
> > Interactive Queries require you to manually forward requests that should
> go
> > to other partitions.
> >
> >
> Each KafkaStreams instance is potentially responsible for multiple
> partitions, so when you use Queryable State on a particular instance you
> are querying all partitions for that store on the given instance.
>
>
>
> >
> >
> >
> >
> >
> > On Thu, Mar 16, 2017 at 2:11 PM, Damian Guy <damian....@gmail.com>
> wrote:
> >
> > > I think what you are seeing is that the order is not guaranteed across
> > > partitions. When you use Queryable State you are actually querying
> > multiple
> > > underlying stores, i.e., one per partition. The implementation iterates
> > > over one store/partition at a time, so the ordering will appear random.
> > > This could be improved
> > >
> > > The tombstone records appearing in the results seems like a bug.
> > >
> > > Thanks,
> > > Damian
> > >
> > > On Thu, 16 Mar 2017 at 17:37 Matthias J. Sax <matth...@confluent.io>
> > > wrote:
> > >
> > > > Can you check if the problem exist for 0.10.2, too? (0.10.2 is
> > > > compatible to 0.10.1 broker -- so you can upgrade your Streams code
> > > > independently from the brokers).
> > > >
> > > > About the range: I did double check this, and I guess my last answer
> > was
> > > > not correct, and range() should return ordered data, but I got a
> follow
> > > > up question: what the key type and serializer you use? Internally,
> data
> > > > is stored in serialized form and ordered according to
> > > > `LexicographicByteArrayComparator` -- thus, if the serialized bytes
> > > > don't reflect the order of the deserialized data, it returned range
> > > > shows up unordered to you.
> > > >
> > > >
> > > > -Matthias
> > > >
> > > >
> > > >
> > > >
> > > > On 3/16/17 10:14 AM, Dmitry Minkovsky wrote:
> > > > > Hi Matthias. Thank you for your response.
> > > > >
> > > > > Yes, I was able to reproduce the null issue reliably. I can't open
> a
> > > JIRA
> > > > > at this time, but I can say I was using 0.10.1.0 and it was trivial
> > to
> > > > > reproduce. Just send records and the tombstones to a table topic.
> > Then
> > > > scan
> > > > > the range. You'll see the tombstones.
> > > > >
> > > > > Indeed, ranges are returned with no specific order. I'm not sure
> what
> > > you
> > > > > mean that default stores are hash-based, but this ordering thing
> is a
> > > > shame
> > > > > because it kind of kills the ability to use KS as a full fledged DB
> > > that
> > > > > lets you index things like HBase (composite keys for lists of
> items).
> > > Is
> > > > > that how RocksDB works? Just returns range scans in random order? I
> > > don't
> > > > > know C++ so the documentation is a bit opaque to me. But what's the
> > > point
> > > > > of scanning a range if the data comes in some random order? That
> > being
> > > > the
> > > > > case, the number of possible use-case scenarios seem to become
> > > > > significantly limited.
> > > > >
> > > > >
> > > > > Thank you!
> > > > > Dmitry
> > > > >
> > > > > On Tue, Mar 14, 2017 at 1:12 PM, Matthias J. Sax <
> > > matth...@confluent.io>
> > > > > wrote:
> > > > >
> > > > >>> However,
> > > > >>>> for keys that have been tombstoned, it does return null for me.
> > > > >>
> > > > >> Sound like a bug. Can you reliable reproduce this? Would you mind
> > > > >> opening a JIRA?
> > > > >>
> > > > >> Can you check if this happens for both cases: caching enabled and
> > > > >> disabled? Or only for once case?
> > > > >>
> > > > >>
> > > > >>> "No ordering guarantees are provided."
> > > > >>
> > > > >> That is correct. Internally, default stores are hash-based --
> thus,
> > we
> > > > >> don't give a sorted list/iterator back. You could replace RocksDB
> > > with a
> > > > >> custom store though.
> > > > >>
> > > > >>
> > > > >> -Matthias
> > > > >>
> > > > >>
> > > > >> On 3/13/17 3:56 PM, Dmitry Minkovsky wrote:
> > > > >>> I am using interactive streams to query tables:
> > > > >>>
> > > > >>> ReadOnlyKeyValueStore<Messages.ByUserAndDate,
> > > > >>> Messages.UserLetter> store
> > > > >>> = streams.store("view-user-drafts",
> > > > >>> QueryableStoreTypes.keyValueStore());
> > > > >>>
> > > > >>> Documentation says that #range() should not return null values.
> > > > However,
> > > > >>> for keys that have been tombstoned, it does return null for me.
> > > > >>>
> > > > >>> Also, I noticed only just now that "No ordering guarantees are
> > > > >> provided." I
> > > > >>> haven't done enough testing or looked at the code carefully
> enough
> > > yet
> > > > >> and
> > > > >>> wonder if someone who knows could confirm: is this true? Is this
> > > common
> > > > >> to
> > > > >>> all store implementations? I was hoping to use interactive
> streams
> > > like
> > > > >>> HBase to scan ranges. It appears this is not possible.
> > > > >>>
> > > > >>> Thank you,
> > > > >>> Dmitry
> > > > >>>
> > > > >>
> > > > >>
> > > > >
> > > >
> > > >
> > >
> >
>
