Hi Igor, see inline: On Sat, May 28, 2016 at 8:14 AM, Igor Kravzov <igork.ine...@gmail.com> wrote:
> I need some clarification on subject. > In Kafka documentations I found the following: > > Kafka only provides a total order over messages *within* a partition, not > between different partitions in a topic. Per-partition ordering combined > with the ability to partition data by key is sufficient for most > applications. However, if you require a total order over messages this can > be achieved with a topic that has only one partition, though this will mean > only one consumer process per consumer group. > > So here are my questions: > 1. Does it mean if i want to have more than 1 consumer (from the same > group) reading from one topic I need to have more than 1 partition? > Yes. > > 2. Does it mean I need same amount of partitions as amount of consumers for > the same group? > No. If you have more partitions than consumers, consumers will consume from more than one topic. > > 3. How many consumers can read from one partition? > Only one. > > Also have some questions regarding relationship between keys and partitions > with regard to API. I only looked at .net APIs (especially one from MS) > but looks like the mimic Java API. > I see when using a producer to send a message to a topic there is a key > parameter. But when consumer reads from a topic there is a partition > number. > > 1. How are partitions numbered? Starting from 0 or 1? > 2. What exactly relationship between a key and partition? > Using the default partitioner, and assuming you don't add new partitions, all messages with the same key are guaranteed to land in the same partition. > As I understand some function on key will determine a partition. is that > correct? > 3. If I have 2 partitions in a topic and want some particular messages go > to one partition and other messages go to another I should use a specific > key for one specific partition, and the rest for another? > If you always want one key to go to a specific partition, and another key to go to a different partition, you can use a custom partitioner or the Java API: http://kafka.apache.org/090/javadoc/org/apache/kafka/clients/producer/ProducerRecord.html > 4. What if I have 3 partitions and one type of messages to one particular > partition and the rest to other 2? > Same as above. Although, in most cases, all messages in a topic are the same "type" of message. A topic is like a database table in this way. > 5. How in general I send messages to a particular partition in order to > know for a consumer from where to read? > See above. > Or I better off with multiple topics? > This will depend on your application. > > Thanks in advance. >
