Dear Developers,
I am 2nd year masters student at IIT. I am using Kafka for one of my
research projects.My question is the following:
1. I have a producer, consumer and a broker(that contains 1st partition of
my topic) on node1
2. I have a producer, consumer, zookeeper and a broker(that contains
Yes, they are on the same consumer group, but I have two partitions.
On Thu, Nov 13, 2014 at 5:04 PM, Jagat Singh wrote:
> Are both of them in same Consumer Group?
>
> On Fri, Nov 14, 2014 at 9:12 AM, Palur Sandeep
> wrote:
>
> > Dear Developers,
> >
> > I am 2
umer section from the
> > documentation.
> > >
> > > https://kafka.apache.org/documentation.html
> > >
> > >
> > >
> > > On Fri, Nov 14, 2014 at 10:09 AM, Palur Sandeep >
> > > wrote:
> > >
> > > > Yes, they
Dear Developers,
I am using the default partitioning logic(Random Partitioning) to produce
messages into brokers. That is I don't use a partitioner.class.
My requirement is If I produce 10 messages using the below code for a
broker that has 8 partitions across 8 nodes. How many messages will
sages is on each broker then you can use a simple consumer which allows
> you to specify the partition id you want to consume from.
>
> In the new consumer (0.9), each of the consumed message will contain the
> partition id as part of its message metadata.
>
> Guozhang
>
> On
rom.
> >
> >In the new consumer (0.9), each of the consumed message will contain the
> >partition id as part of its message metadata.
> >
> >Guozhang
> >
> >On Tue, Nov 25, 2014 at 7:47 AM, Palur Sandeep
> >wrote:
> >
> >> Dear Dev
fferent
> brokers, you will still see messages roughly evenly distributed.
> If you are using high level consumer (ZookeeperConsumerConnector), after
> getting the message from stream, you can simply call message.Partition to
> get the partition id.
>
> Jiangjie (Becket) Qin
>
Hi all,
I am using kafka version 0.8.1.1. In the consumer side, can I find out the
partition id of each message I am consuming using high level consumer API?
Thanks,
Sandeep
--
Regards,
Sandeep Palur
Data-Intensive Distributed Systems Laboratory, CS/IIT
Department of Computer Science, Illinois
t does not partition member or
> MessageAndMetadata.partition give you nothing?
>
> Jiangjie (Becket) Qin
>
> On 11/26/14, 10:31 AM, "Palur Sandeep" wrote:
>
> >Hi Jiangjie,
> >
> >I am using the high level consumer (ZookeeperConsumerConnector), after
> >getting th
Thank you. but I dont see that method. I dont know whats wrong
On Wed, Nov 26, 2014 at 6:31 PM, Jun Rao wrote:
> Yes, the consumer iterator returns MessageAndMetadata which contains a
> method partition().
>
> Thanks,
>
> Jun
>
> On Wed, Nov 26, 2014 at 10:40 AM
t;
> Jiangjie (Becket) Qin
>
> On 11/26/14, 12:56 PM, "Palur Sandeep" wrote:
>
> >Hi Jiangle,
> >
> >
> >Thanks for the information. This is what I get when I print
> >MessageandMetadata
> >
> >*Thread 0: Message(ma
Hi all,
Consumer doesnt receive message if it is big: When the producer sends
256kb messages to broker, consumer is able to retrieve it, but when
producer sends 10MB messages to the broker, the consumer doesn’t receive
any message.
Please tell me how to make the consumer receive 10MB messages.
-
; -Harsha
>
> On Mon, Dec 1, 2014, at 07:27 PM, Palur Sandeep wrote:
> > Hi all,
> >
> > Consumer doesnt receive message if it is big: When the producer sends
> > 256kb messages to broker, consumer is able to retrieve it, but when
> > producer sends 10MB message
> controls the maximum number of bytes a consumer issues in one fetch.
>
>
> Guozhang
>
>
> On Mon, Dec 1, 2014 at 7:25 PM, Palur Sandeep
> wrote:
>
> > Thank you so much Jiangle. I got it working.
> >
> > I have another problem the consumer doesnt receive
14 matches
Mail list logo
