Not sure if I understand (1).

For (2), you don't need to worry about it. The consumer will handle all this for you internally, and only return a full message from `poll()`.

-Matthias

On 10/16/24 00:16, Sathishkumar Palanisamy wrote:
Thanks for response , understood,

   1.
Exception classes isn't available for any fetching related i believe in javadocs
TOPIC -> testing, partition count =1, offset count = 2
   2.
Assuming the message 1st topics 1st message we are trying to read
  "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor 
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud 
exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure 
dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. 
Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit 
anim id est laborum."

Under network latency of 1 kbps bandwidth, 1mb offset message with value of 
fetch.min.bytes = 1 bytes along with fetch.max.wait.ms = 0.5 seconds ,  it will fetch 1 
kilobytes of data but that data if is in avro serialization it will be in binary right, 
so the fetched data in binary message is upto "Lorem ipsum dolor sit amet, 
consectetur adipiscing elit, sed do " for example

Then how data is assembled together for remaining potion ? how data is made as 
whole ?




________________________________
From: Matthias J. Sax <mj...@apache.org>
Sent: Wednesday, October 16, 2024 06:23
To: users@kafka.apache.org <users@kafka.apache.org>
Subject: [EXT MSG] Re: KAFKA POLL

EXTERNAL source. Be CAREFUL with links / attachments

1. As pointed out in the JavaDocs

@param timeoutMs The time, in milliseconds, spent waiting in poll if data is 
not available in the buffer.



2. If there is no data with the timeout, `poll()` return an "empty"
`ConsumerRecords` object.

For error/exception I cannot say, as it depends on the
error/exception... The exception which could be thrown are documented in
the JavaDocs, so maybe you can start there?


consumerRecords<String, String> records = consumer.poll(10000L);
      *   whether this records count will be 0 in this case ?

Yes, the `ConsumerRecords` object will be empty.



if poll gets 500kb from topic in first poll within 10s after processing 
assuming no errors comes up and records count is 0 then in next poll whether 
next 500kb is fetched or it will attempt to again fetch full message ?

If the message is 1MB in size, it will either be fetched completely not
not at all. There will only be one fetch request and one fetch response.
If the first poll() send the fetch request, the second poll() would see
that a fetch request is already "in progress" and just wait for it to
complete (or again just let `poll()` return with empty result if the
timeout hits before the fetch request completes)


HTH.

-Matthias


On 10/10/24 5:27 AM, Sathishkumar Palanisamy wrote:
Hi Team,
                Does anyone know in the experimental scenario,
                                             JAVA runtime 8 or 11

    1.  Kafka's topic  -> Testing_topic
    2.  Kafka's topic partition count -> 1
    3.  Kafka's topic partition offset -> 2 [two messages are available, each 
weight about 1 mb ]
    4.  Bandwidth of 1 kb and Kafka's topic offset's message of 1 mb is 
available each
    5.  In consumer scenario
       *   While polling the consumer using consumer.poll(10000L) deprecated 
method supporting the long argument
       *   With properties (exactly used properties)

                                                                             i. 
     props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, 
bootstrap.server.config.url);

                                                                           ii.  
    props.put(ConsumerConfig.GROUP_ID_CONFIG, group-1);

                                                                         iii.      
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, 
"org.apache.kafka.common.serialization.StringDeserializer");

                                                                          iv.      
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, 
"org.apache.kafka.common.serialization.StringDeserializer");

                                                                           v.   
   props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, 1000);

                                                                          vi.   
   props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);

                                                                        vii.      
props.put("request.timeout.ms", "60000");
Question

    1.  in poll(10000L ) given , whether this is in seconds or milliseconds or 
minutes (Deprecated method) ?
    2.  so in bandwidth of 1 kb for experimental purpose -> while polling 
assuming the 10000L is milliseconds so if I kafka consumer doesn't get the offset 
message which of 1 mb
       *   what will happens in brief details ?
       *   whether any errors/exceptions comes up ?
       *   consumerRecords<String, String> records = consumer.poll(10000L);
       *   whether this records count will be 0 in this case ?
       *   if poll gets 500kb from topic in first poll within 10s after 
processing assuming no errors comes up and records count is 0 then in next poll 
whether next 500kb is fetched or it will attempt to again fetch full message ?




The information in this e-mail and any attachments is confidential and may be 
legally privileged. It is intended solely for the addressee or addressees. Any 
use or disclosure of the contents of this e-mail/attachments by a not intended 
recipient is unauthorized and may be unlawful. If you have received this e-mail 
in error please notify the sender. Please note that any views or opinions 
presented in this e-mail are solely those of the author and do not necessarily 
represent those of TEMENOS. We recommend that you check this e-mail and any 
attachments against viruses. TEMENOS accepts no liability for any damage caused 
by any malicious code or virus transmitted by this e-mail.

Reply via email to