Sorry, I mean KafkaProducer!
On Sat, Jun 11, 2016 at 4:18 PM, Saeed Ansari <saeed.ans...@gmail.com>
wrote:
> When I read from stream, I create AKKA actors for processing events. I am
> not processing them in the stream, as the result
> I do not have KStream to write into it. So I use KafkaConsumer instead.
>
> On Sat, Jun 11, 2016 at 7:13 AM, Matthias J. Sax <matth...@confluent.io>
> wrote:
>
>> Do you instantiate KafkaProduer in your user code?
>>
>> Why no use KStream.to("topic-name") ?
>>
>>
>> -Matthias
>>
>> On 06/10/2016 12:28 AM, Saeed Ansari wrote:
>> > Thank you Eno,
>> > Adding more threads extremely increased the throughput of stream. As I
>> said
>> > after processing I send the event to another topic. For that I was
>> opening
>> > a connection via KafkaProducer to the cluster and I think that was the
>> > issue. Now there is just one producer for sending events to output
>> topic.
>> >
>> > Do you have any recommendation how that part can get better?
>> >
>> > Thank you so much,
>> > Saeed
>> >
>> > On Thu, Jun 9, 2016 at 3:33 PM, Eno Thereska <eno.there...@gmail.com>
>> wrote:
>> >
>> >> Hi Saeed,
>> >>
>> >> There could be several reasons why things appear slow and it is
>> difficult
>> >> to say without knowing the exact details of the setup and the results
>> you
>> >> are observing.
>> >> One thing to check is the number of threads you have assigned to the
>> Kafka
>> >> Stream application. By default just one thread is used. Perhaps you
>> want
>> >> more (depending on number of cores you have). An example way to change
>> that
>> >> in your app:
>> >>
>> >> props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 4 /* four threads
>> */);
>> >>
>> >> Thanks
>> >> Eno
>> >>
>> >>> On 9 Jun 2016, at 18:08, Saeed Ansari <saeed.ans...@gmail.com> wrote:
>> >>>
>> >>> Hi Eno,
>> >>> Thank you for the response. Actually I did not know it automatically
>> >>> assigns partitions to consumers. Now I have one Kafkastream reading
>> from
>> >>> 12 partitions, like below:
>> >>> Controller is an actor that I am sending the message to and then it
>> >> creates
>> >>> child actors to send messages out.
>> >>>
>> >>> builder.stream("test").foreach((x, y) -> {
>> >>> controller.tell(y, controller.noSender());
>> >>> });
>> >>>
>> >>>
>> >>> The msg/sec rate I get from receiving messages to sending them out is
>> >>> really slow!
>> >>>
>> >>> Do you think it is about how consume messages?
>> >>>
>> >>> Thank you,
>> >>> Saeed
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Jun 8, 2016 at 3:08 AM, Eno Thereska <eno.there...@gmail.com>
>> >> wrote:
>> >>>
>> >>>> Hi Saeed,
>> >>>>
>> >>>> Kafka Streams takes care of assigning partitions to consumers
>> >>>> automatically for you. You don't have to write anything explicit to
>> do
>> >>>> that. See WordCountDemo.java as an example. Was there another reason
>> you
>> >>>> wanted control over partition assignment?
>> >>>>
>> >>>> Thanks
>> >>>> Eno
>> >>>>
>> >>>>> On 7 Jun 2016, at 20:02, Saeed Ansari <saeed.ans...@gmail.com>
>> wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>> I have started a small Kafka streaming application. I need to assign
>> >>>>> partitions to consumers in a consumer group. I am not sure where to
>> >> start
>> >>>>> and how to structure consumer groups in KafkaStreams.
>> >>>>>
>> >>>>> I found that there is a StreamPartitionAssignor that can be added to
>> >>>>> config, but still I do not know how to configure it and what
>> parameters
>> >>>> to
>> >>>>> config.
>> >>>>>
>> >>>>> Any help is really appreciated.
>> >>>>>
>> >>>>> Thank you,
>> >>>>> Saeed
>> >>>>
>> >>>>
>> >>
>> >>
>> >
>>
>>
>
