> By the way - do I understand correctly that when a state store is persistent, it is logged by default?
Yes. > So enableLogging(Map) only is a way to provide default configuration to the default logging? Yes. That is, any configs that should be applied to the state store's changelog topic. > And, logging can be disabled with .disableLogging(), in which case the state store is persisted > on disk using RocksDB, but is not backed by a Kafka topic for recovery in case of loosing the rocksdb file? Yes. On Tue, Feb 14, 2017 at 12:57 AM, Adam Warski <a...@warski.org> wrote: > Hello, > > > Can you try this out with 0.10.2 branch or current trunk? > > With 0.10.2 this works fine! The state-store changelog is created with 3 > partitions if the source topic has 3 partitions. > I checked with client from https://repository.apache.org/ > content/groups/staging/org/apache/kafka/kafka_2.11/0.10.2.0/ < > https://repository.apache.org/content/groups/staging/ > org/apache/kafka/kafka_2.11/0.10.2.0/> & server from > http://home.apache.org/~ewencp/kafka-0.10.2.0-rc1/ < > http://home.apache.org/~ewencp/kafka-0.10.2.0-rc1/> > > Also, using 0.10.2 made these warnings go away: > 2017-02-13 19:57:12 WARN ConsumerConfig:188 - The configuration > 'replication.factor' was supplied but isn't a known config. > 2017-02-13 19:57:12 WARN ConsumerConfig:188 - The configuration ' > windowstore.changelog.additional.retention.ms' was supplied but isn't a > known config. > > Thanks! > > >> Or the auto-created topic should use the partition count of the > incoming streams (though I’m not sure that’s always possible - a store > might be used multiple times). > > > > This should happen. If not, there is a bug. Can you reliable reproduce > > this issue? > > Yes, I’m was testing using a fresh 0.10.1.1 download, one local broker, > default configuration. > > >> Finally, is there a way to specify how many partitions should > auto-created topics have when creating a state store? I tried with: > > > > The number of partitions are computed based on the number of tasks, that > > depend on the number of input partitions. Thus, you cannot configure it. > > Because the topic must have a specific number of partitions go guarantee > > correct results, there is nothing to "tune" and thus you cannot > > configure anything. > > Ah, makes sense. > > By the way - do I understand correctly that when a state store is > persistent, it is logged by default? So enableLogging(Map) only is a way to > provide default configuration to the default logging? > > And, logging can be disabled with .disableLogging(), in which case the > state store is persisted on disk using RocksDB, but is not backed by a > Kafka topic for recovery in case of loosing the rocksdb file? > > Thanks again, > Adam > > -- > Adam Warski > > http://twitter.com/#!/adamwarski <http://twitter.com/#!/adamwarski> > http://www.softwaremill.com <http://www.softwaremill.com/> > http://www.warski.org <http://www.warski.org/> >
