RE: Log Retention Period of Kafka Messages

2018-03-12 Thread Kyle Tinker
trols https://kafka.apache.org/08/documentation/#brokerconfigs Thanks, -- KYLE TINKER Lead Architect/Team Lead | WorkForce Software T: +1 734 742 2616 | E: ktin...@workforcesoftware.com -Original Message- From: Muruganandham, Ashokkumar [mailto:ashokkumar.muruganand...@fluke.com]

RE: Excessive Memory Usage with Compression enabled and possible resolutions

2018-01-31 Thread Kyle Tinker
Hi Kyle, Are you interested in submitting a pull request? Ismael On Wed, Jan 31, 2018 at 3:00 PM, Kyle Tinker wrote: > Ismael, > > I have filed > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue > s.apache.org%2Fjira%2Fbrowse%2FKAFKA-6512&

RE: Excessive Memory Usage with Compression enabled and possible resolutions

2018-01-31 Thread Kyle Tinker
2. Can you please file a JIRA with 1.1.0 as the target version? Ismael On 30 Jan 2018 5:46 pm, "Kyle Tinker" wrote: > I'm using Kafka 1.0.0 and the Java producer. > > I've noticed high memory usage in the producer when enabling > compression (gzip or lz4). I don&

[jira] [Created] (KAFKA-6512) Java Producer: Excessive memory usage with compression enabled

2018-01-31 Thread Kyle Tinker (JIRA)
Kyle Tinker created KAFKA-6512: -- Summary: Java Producer: Excessive memory usage with compression enabled Key: KAFKA-6512 URL: https://issues.apache.org/jira/browse/KAFKA-6512 Project: Kafka

Excessive Memory Usage with Compression enabled and possible resolutions

2018-01-30 Thread Kyle Tinker
I'm using Kafka 1.0.0 and the Java producer. I've noticed high memory usage in the producer when enabling compression (gzip or lz4). I don't observe the behavior with compression off, but with it on I'll run out of heap (2GB). Using a Java profiler, I see the data is in the KafkaLZ4BlockOutpu