Sorry for reposting the previous message as the images didn’t come thru..
pasting as text.
I have changed both system and user limits.
To completely isolate the problem, I have tried the application in Centos 7
environment. Set the ulimit to 1million and system limits to 10million open
files
Also, when you say you changed the OS limit are you referring to the system
limit or the user limit or both? If you increased one but not the other you
may still be hitting the lower limit.
On Wed, Jul 3, 2019 at 1:53 AM Patrik Kleindl wrote:
> Hi
> Try to set it really low like Sophie suggested
https://github.com/apache/kafka/pull/6771
Bouncing both users and dev to get some activity going. We are waiting for
a while to get this KIP pr merged.
Could someone please review?
Thanks,
On Sun, 30 Jun 2019 at 08:59, M. Manna wrote:
> https://github.com/apache/kafka/pull/6771
>
> Hello,
>
>
Hi
Try to set it really low like Sophie suggested.
You can verify if the settings take effect by checking the files in the rocksdb
directories, I think it should be somewhere in OPTIONS or LOG
br, Patrik
> Am 03.07.2019 um 09:37 schrieb Thameem Ansari :
>
> Tried setting the open files to 100 a
Yes I tried with console-producer , it is working fine.
But its a plain message i.e. String .. but i have to send a 20 field json
as message.
It depends on the use case. Look at lingerMs (10ms) So the latency is 10ms.
` Received invalid metadata error in produce request on partition
inbound_topic-1 due to org.apache.kafka.common.errors.NetworkException: The
server disconnected before a response was received.. Going to request
metadata u
thank you.
Senthil ,
Why batch.size=65536 & retries=100, its big number right ? do we need this
much for streaming applications ?
Regards,
Shyam
On Wed, Jul 3, 2019 at 1:30 PM SenthilKumar K
wrote:
> `*Partition = -1` - *This explains why are you getting timeout error.
>
> Why dont you use Def
Hi,
I fixed the earlier issue using returning a valid partition number.
Now I am facing a different error
2019-07-03 04:16:44.334 WARN 1524 --- [ad | producer-1]
o.a.k.clients.producer.internals.Sender : [Producer clientId=producer-1]
Received invalid metadata error in produce request on partit
`*Partition = -1` - *This explains why are you getting timeout error.
Why dont you use Default Partitioner ?:
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/DefaultPartitioner.java
Try only with below producer properties :
bootstrap.se
Hi SenthilKumar,
thanks a lot .
Yeah I set up local set up and print the log with partition info which
shows as below
2019-07-03 02:48:28.822 INFO 7092 --- [lt-dispatcher-2]
c.s.c.p.p.CompanyInfoPartitioner : Topic : inbound_topic Key = 597736248-
Entropy Cayman Solar Ltd.-null-null-null *Partiti
Tried setting the open files to 100 and 50 but the results are same. I checked
the total open files while the streaming application was busy running just
before getting the “too many open files” message it was around 41756 which is
same as what we have got when we set to -1.
VisualVM shows tha
11 matches
Mail list logo
