Hello, > Can you try this out with 0.10.2 branch or current trunk?
With 0.10.2 this works fine! The state-store changelog is created with 3 partitions if the source topic has 3 partitions. I checked with client from https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.11/0.10.2.0/ <https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.11/0.10.2.0/> & server from http://home.apache.org/~ewencp/kafka-0.10.2.0-rc1/ <http://home.apache.org/~ewencp/kafka-0.10.2.0-rc1/> Also, using 0.10.2 made these warnings go away: 2017-02-13 19:57:12 WARN ConsumerConfig:188 - The configuration 'replication.factor' was supplied but isn't a known config. 2017-02-13 19:57:12 WARN ConsumerConfig:188 - The configuration 'windowstore.changelog.additional.retention.ms' was supplied but isn't a known config. Thanks! >> Or the auto-created topic should use the partition count of the incoming >> streams (though I’m not sure that’s always possible - a store might be used >> multiple times). > > This should happen. If not, there is a bug. Can you reliable reproduce > this issue? Yes, I’m was testing using a fresh 0.10.1.1 download, one local broker, default configuration. >> Finally, is there a way to specify how many partitions should auto-created >> topics have when creating a state store? I tried with: > > The number of partitions are computed based on the number of tasks, that > depend on the number of input partitions. Thus, you cannot configure it. > Because the topic must have a specific number of partitions go guarantee > correct results, there is nothing to "tune" and thus you cannot > configure anything. Ah, makes sense. By the way - do I understand correctly that when a state store is persistent, it is logged by default? So enableLogging(Map) only is a way to provide default configuration to the default logging? And, logging can be disabled with .disableLogging(), in which case the state store is persisted on disk using RocksDB, but is not backed by a Kafka topic for recovery in case of loosing the rocksdb file? Thanks again, Adam -- Adam Warski http://twitter.com/#!/adamwarski <http://twitter.com/#!/adamwarski> http://www.softwaremill.com <http://www.softwaremill.com/> http://www.warski.org <http://www.warski.org/>
