MaoWei Tian created KAFKA-18337:
---
Summary: KafkaProducer Memory Leak in JMX Reporter
Key: KAFKA-18337
URL: https://issues.apache.org/jira/browse/KAFKA-18337
Project: Kafka
Issue Type: Bug
come up. Here is the line of code where we create a kafka
producer:
*Properties props;*
*KafkaProducer producer = new KafkaProducer(props); *
This line is the problem. I set a breakpoint here and then tried to enter
inside the KafkaProducer function. (This is of no use as I don't have the
s
IN_FLIGHT_REQUESTS_PER_CONNECTION, "5");
>props.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, "1");
>
> Producer producer = new KafkaProducer<>(props);
>
>
> Thankyou,
> Giri
>
--
Okada Haruki
ocadar...@gmail.com
08864);
props.put("retries", 3);
props.put("retry.backoff.ms", 5000);
props.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, "5");
props.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, "1");
Producer producer = new KafkaProducer<>(props);
Thankyou,
Giri
ers lead to NPE in KafkaProducer
> constructor
>
>
> Key: KAFKA-17478
> URL: https://issues.apache.org/jira/browse/KAFKA-17478
> Project: Kafka
>
Hongshun Wang created KAFKA-17025:
-
Summary: KafkaThread and KafkaProducer expose method to set
setUncaughtExceptionHandler
Key: KAFKA-17025
URL: https://issues.apache.org/jira/browse/KAFKA-17025
Hi,
my name is Pere, Software Engineer turned Consultant for the last few
years, :-/ , but keep doing Kafka quite a lot of my day-to-day.
I know not a sexy issue, but last few days, I worked with customers who
again hit https://issues.apache.org/jira/browse/KAFKA-4169 when using
producing w
bug.
> KafkaProducer: IllegalMonitorStateException
>
>
> Key: KAFKA-10637
> URL: https://issues.apache.org/jira/browse/KAFKA-10637
> Project: Kafka
> Issue Type: Bug
>
Artem Livshits created KAFKA-14085:
--
Summary: Clean up usage of asserts in KafkaProducer
Key: KAFKA-14085
URL: https://issues.apache.org/jira/browse/KAFKA-14085
Project: Kafka
Issue Type
to everything, but if
there's something worth mentioning like the behavior proposed in item 2,
it's nice to include in a Javadoc for the relevant class/method.
5. Just for aesthetics, I wonder if we can add static builder() methods to
the KafkaProducer, KafkaConsumer, KafkaStreams, and
nks,
> Kirk
>
> On Wed, May 18, 2022, at 10:11 AM, François Rosière wrote:
> > Hi all,
> >
> > KIP to create builders for
> >
> >- KafkaProducer
> >- KafkaConsumer
> >- KafkaStreams
> >
> >
> > This KIP can be seen as th
t; KIP to create builders for
>
>- KafkaProducer
>- KafkaConsumer
>- KafkaStreams
>
>
> This KIP can be seen as the continuity of the KIP-832.
>
> KIP details:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211884640
> Jira issue: h
Hi all,
KIP to create builders for
- KafkaProducer
- KafkaConsumer
- KafkaStreams
This KIP can be seen as the continuity of the KIP-832.
KIP details:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211884640
Jira issue: https://issues.apache.org/jira/browse/KAFKA
François Rosière created KAFKA-13913:
Summary: Provide builders for KafkaProducer/KafkaConsumer and
KafkaStreams
Key: KAFKA-13913
URL: https://issues.apache.org/jira/browse/KAFKA-13913
Project
[
https://issues.apache.org/jira/browse/KAFKA-13864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
François Rosière resolved KAFKA-13864.
--
Resolution: Won't Do
> Change the visibility of a KafkaProducer and Kafka
François Rosière created KAFKA-13864:
Summary: Change the visibility of a KafkaProducer and
KafkaConsumer constructor
Key: KAFKA-13864
URL: https://issues.apache.org/jira/browse/KAFKA-13864
[
https://issues.apache.org/jira/browse/KAFKA-12791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lucas Bradstreet resolved KAFKA-12791.
--
Resolution: Fixed
> ConcurrentModificationException in KafkaProducer construc
Blocking in the constructor to fetch metadata doesn't solve the problem, since
metadata still needs to be periodically refreshed over time.
best,
Colin
On Wed, Jun 16, 2021, at 09:51, Jesse Feinman wrote:
> Hi Moses,
>
> Specifically, on blocking in the constructor to fetch metadata, while I
Hi Moses,
Specifically, on blocking in the constructor to fetch metadata, while I like
the idea of specifying the topics in the constructor and fetching the metadata
then, I think it leads to a few scenarios that could be unexpected. First is if
you try to use a topic that wasn't included in t
at 2:21 PM Colin McCabe wrote:
> On Tue, Jun 1, 2021, at 12:22, Nakamura wrote:
> > I think we're talking past each other a bit. I know about non-blocking
> > I/O. The problem I'm facing is how to preserve the existing semantics
> > without blocking. Right now calle
On Tue, Jun 1, 2021, at 12:22, Nakamura wrote:
> I think we're talking past each other a bit. I know about non-blocking
> I/O. The problem I'm facing is how to preserve the existing semantics
> without blocking. Right now callers assume their work is enqueued in-order
>
Since we loop until the I/O is complete, you can consider this behavior
"blocking on I/O".
> > How are the ordering semantics of `KafkaProducer#send` related to the
> > metadata fetch?
> > I already proposed a solution (returning an error)
>
> There is a s
Hi Colin,
> KafkaProducer#send is supposed to initiate non-blocking I/O, but not wait
for it to complete.
>
> There's more information about non-blocking I/O in Java here:
> https://en.wikipedia.org/wiki/Non-blocking_I/O_%28Java%29
I think we're talking past each other a
ering semantics of `KafkaProducer#send` related to the
metadata fetch?
> I already proposed a solution (returning an error)
There is a subtle difference between failing immediately vs blocking for
metadata, related to ordering in the face of retries. Say we set the send
timeout to max-long (or
On Tue, Jun 1, 2021, at 07:00, Nakamura wrote:
> Hi Colin,
>
> Sorry, I still don't follow.
>
> Right now `KafkaProducer#send` seems to trigger a metadata fetch. Today,
> we block on that before returning. Is your proposal that we move the
> metadata fetch out of `Ka
Hi Colin,
Sorry, I still don't follow.
Right now `KafkaProducer#send` seems to trigger a metadata fetch. Today,
we block on that before returning. Is your proposal that we move the
metadata fetch out of `KafkaProducer#send` entirely?
Even if the metadata fetch moves to be non-blocki
> > > Throwing an exception to the caller directly (not via the
> > future) is
> > > > > > > another option with precedent in Kafka clients, tho it doesn't
> > seem
> > > > as
> > > > > > > ergonomic to me.
> > > > > > &
t; > > > another option with precedent in Kafka clients, tho it doesn't
> > seem
> > > > as
> > > > > > > ergonomic to me.
> > > > > > >
> > > > > > > It would be interesting to analyze some exi
> > for
> > >> > > > different
> > >> > > > > > > > types of tasks, for serialization which is CPU bound you
> > >> > > definitely
> > >> > > > > > would
> > >> > > > > > > &g
; > > > ThreadPool this is more of an ergonomics question for the
> >> API.
> >> > > > > > Especially
> >> > > > > > > > when it gets to monitoring/tracing, giving the ability for
> >> > users
> >> > > to
> >> > > > > > >
ool for IO tasks and another for CPU
>> > > tasks
>> > > > > > makes
>> > > > > > > > their lives a lot easier. However due to the large amount of
>> > > > changes to
>> > > > > > > the
>> > >
> > I was thinking the sender would typically wrap send() in a
> > > > > > > backoff/retry
> > > > > > > > > loop, or else ignore any failures and drop sends on the
> floor
> > > > > > > > > (fire-and-forget), and in b
t; > asynchronously
> > > > > > > > signal the caller of failure (either by asynchronously
> failing
> > > the
> > > > > > future
> > > > > > > > or invoking a callback off-thread or something) we'd force
>
x27;t
> > seem
> > > > as
> > > > > > > ergonomic to me.
> > > > > > >
> > > > > > > It would be interesting to analyze some existing usage and
> > determine
> > > > > how
> > > > > > > difficult
; > > > >
> > > > > > > Hmm, that's an interesting idea. Basically it would mean that
> > > after
> > > > > > > calling send, you would also have to check whether the returned
> > > > future
> > > > > > had
>
; > >
> > > > > > > Hmm, that's an interesting idea. Basically it would mean that
> > > after
> > > > > > > calling send, you would also have to check whether the returned
> > > > future
> > > > > > had
> > > > &
gt; > which
> > > > > case
> > > > > > it will be failed immediately if it failed to enqueue) or whether
> > we
> > > > > > succeeded in sending or not".
> > > > > >
> > > > > > But you're right, it should be on t
> ryannedo...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Moses, in the case of a full queue, could we just return a failed
> > > > future
> > > > > > immediately?
> > > > > >
uld use some feedback in
> > > this
> > > > > > area. There are two mechanisms here, one for slowing down when
> we
> > > > don't
> > > > > > have the relevant metadata, and the other for slowing down when a
> > > queue
gt; > queue
> > > > has
> > > > > filled up. Although the first one applies backpressure somewhat
> > > > > inadvertently, we could still get in trouble if we're not providing
> > > > > information to the mechanism that monitors whether we&
27;t drop that ability.
> > > >
> > > > Right now backpressure is applied by blocking, which is a natural way
> > to
> > > > apply backpressure in synchronous systems, but can lead to
> unnecessary
> > > > slowdowns in asynchronous syst
y opinion, the safest way to
> > apply
> > > backpressure in an asynchronous model is to have an explicit
> backpressure
> > > signal. A good example would be returning an exception, and providing
> an
> > > optional hook to add a callback onto so that you can b
g an
> > optional hook to add a callback onto so that you can be notified when
> it's
> > ready to accept more messages.
> >
> > However, this would be a really big change to how users use
> > KafkaProducer#send, so I don't know how much appetite we have
al hook to add a callback onto so that you can be notified when it's
> ready to accept more messages.
>
> However, this would be a really big change to how users use
> KafkaProducer#send, so I don't know how much appetite we have for making
> that kind of change. Maybe it w
A good example would be returning an exception, and providing an
optional hook to add a callback onto so that you can be notified when it's
ready to accept more messages.
However, this would be a really big change to how users use
KafkaProducer#send, so I don't know how much appetite we
rrently are thrown to the caller (before the
> > > future is returned). In the new API, the send() wouldn't throw any
> > > exceptions, and instead the future would fail. I think that might mean
> > that
> > > a new method signature is required.
> > >
Lucas Bradstreet created KAFKA-12791:
Summary: ConcurrentModificationException in KafkaProducer
constructor
Key: KAFKA-12791
URL: https://issues.apache.org/jira/browse/KAFKA-12791
Project: Kafka
ons, and instead the future would fail. I think that might mean
> that
> > a new method signature is required.
> >
> > Ryanne
> >
> >
> > On Thu, May 13, 2021, 2:57 PM Nakamura wrote:
> >
> > > Hey Ryanne,
> > >
> > > I agree we s
ad the future would fail. I think that might mean that
> a new method signature is required.
>
> Ryanne
>
>
> On Thu, May 13, 2021, 2:57 PM Nakamura wrote:
>
> > Hey Ryanne,
> >
> > I agree we should add an additional constructor (or else an additional
> > o
constructor (or else an additional
> overload in KafkaProducer#send, but the new constructor would be easier to
> understand) if we're targeting the "user provides the thread" approach.
>
> From looking at the code, I think we can keep record serialization on the
> user
Hey Ryanne,
I agree we should add an additional constructor (or else an additional
overload in KafkaProducer#send, but the new constructor would be easier to
understand) if we're targeting the "user provides the thread" approach.
>From looking at the code, I think
ery
easy for clients to switch.
Ryanne
On Thu, May 13, 2021, 9:00 AM Nakamura wrote:
> Hey Folks,
> I just posted a new proposal
> <
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181306446
> >
> in the wiki. I think we have an opportunity to impr
Hey Folks,
I just posted a new proposal
<https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181306446>
in the wiki. I think we have an opportunity to improve the
KafkaProducer#send user experience. It would certainly make our lives
easier. Please take a look! There a
Rohan Desai created KAFKA-12707:
---
Summary: KafkaProducer should have a clearer error message on
sasl.mechanism misconfiguration
Key: KAFKA-12707
URL: https://issues.apache.org/jira/browse/KAFKA-12707
Boyang Chen created KAFKA-10733:
---
Summary: Enforce exception thrown for KafkaProducer txn APIs
Key: KAFKA-10733
URL: https://issues.apache.org/jira/browse/KAFKA-10733
Project: Kafka
Issue Type
Lefteris Katiforis created KAFKA-10637:
--
Summary: KafkaProducer: IllegalMonitorStateException
Key: KAFKA-10637
URL: https://issues.apache.org/jira/browse/KAFKA-10637
Project: Kafka
Luis Ashurei created KAFKA-10620:
Summary: Ensure Topic Metadata loaded just after create
KafkaProducer
Key: KAFKA-10620
URL: https://issues.apache.org/jira/browse/KAFKA-10620
Project: Kafka
[
https://issues.apache.org/jira/browse/KAFKA-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantine Karantasis resolved KAFKA-9830.
---
Resolution: Fixed
> DeadLetterQueueReporter leaks KafkaProducer insta
Greg Harris created KAFKA-9830:
--
Summary: DeadLetterQueueReporter leaks KafkaProducer instance
Key: KAFKA-9830
URL: https://issues.apache.org/jira/browse/KAFKA-9830
Project: Kafka
Issue Type
available in the 2.3.0 release.
> KafkaProducer possible deadlock when sending to different topics
>
>
> Key: KAFKA-8623
> URL: https://issues.apache.org/jira/browse/KAFKA-8623
>
Boyang Chen created KAFKA-9439:
--
Summary: Add more public API tests for KafkaProducer
Key: KAFKA-9439
URL: https://issues.apache.org/jira/browse/KAFKA-9439
Project: Kafka
Issue Type
[
https://issues.apache.org/jira/browse/KAFKA-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomoyuki Saito resolved KAFKA-9301.
---
Resolution: Duplicate
> KafkaProducer#flush should block until all the sent records
Lucas Bradstreet created KAFKA-9312:
---
Summary: KafkaProducer flush behavior does not guarantee send
completion under record batch splitting
Key: KAFKA-9312
URL: https://issues.apache.org/jira/browse/KAFKA-9312
Tomoyuki Saito created KAFKA-9301:
-
Summary: KafkaProducer#flush should block until all the sent
records get completed
Key: KAFKA-9301
URL: https://issues.apache.org/jira/browse/KAFKA-9301
Project
Myeonghyeon Lee created KAFKA-8996:
--
Summary: Support health check method of TransactionManager and
KafkaProducer
Key: KAFKA-8996
URL: https://issues.apache.org/jira/browse/KAFKA-8996
Project: Kafka
the KafkaProducer clients
> --
>
> Key: KAFKA-5712
> URL: https://issues.apache.org/jira/browse/KAFKA-5712
> Project: Kafka
> Issue Type: Improvement
> Components: clients
>
Alexander Bagiev created KAFKA-8623:
---
Summary: KafkaProducer possible deadlock when sending to different
topics
Key: KAFKA-8623
URL: https://issues.apache.org/jira/browse/KAFKA-8623
Project: Kafka
Sergei Egorov created KAFKA-8178:
Summary: KafkaProducer#send(ProducerRecord,Callback) may block for
up to 60 seconds
Key: KAFKA-8178
URL: https://issues.apache.org/jira/browse/KAFKA-8178
Project
[
https://issues.apache.org/jira/browse/KAFKA-8111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajini Sivaram resolved KAFKA-8111.
---
Resolution: Fixed
Reviewer: Manikumar
Fix Version/s: 2.3.0
> KafkaProdu
John Roesler created KAFKA-8111:
---
Summary: KafkaProducer can't produce data
Key: KAFKA-8111
URL: https://issues.apache.org/jira/browse/KAFKA-8111
Project: Kafka
Issue Type
Margaret Figura created KAFKA-8054:
--
Summary: KafkaProducer(Properties, Serializer, Serializer)
fails when serializers are lambdas
Key: KAFKA-8054
URL: https://issues.apache.org/jira/browse/KAFKA-8054
[
https://issues.apache.org/jira/browse/KAFKA-7763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson resolved KAFKA-7763.
Resolution: Fixed
Fix Version/s: 2.3.0
> KafkaProducer with transactionId endl
[
https://issues.apache.org/jira/browse/KAFKA-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson resolved KAFKA-6833.
Resolution: Fixed
Fix Version/s: 2.1.1
2.2.0
> KafkaProdu
weasker created KAFKA-7763:
--
Summary: KafkaProducer with transactionId endless waits when
network is disconnection
Key: KAFKA-7763
URL: https://issues.apache.org/jira/browse/KAFKA-7763
Project: Kafka
Sherwin Chiu created KAFKA-7585:
---
Summary: Avoid classloader when passing serializers to
KafkaProducer constructors
Key: KAFKA-7585
URL: https://issues.apache.org/jira/browse/KAFKA-7585
Project: Kafka
.
> Missing Property in ProducerConfig.java - KafkaProducer API 0.9.0.0
> ---
>
> Key: KAFKA-4065
> URL: https://issues.apache.org/jira/browse/KAFKA-4065
> Project: Kafka
>
Magnus Jungsbluth created KAFKA-7205:
Summary: KafkaConsumer / KafkaProducer should allow
Reconfiguration of SSL Configuration
Key: KAFKA-7205
URL: https://issues.apache.org/jira/browse/KAFKA-7205
Arjun Satish created KAFKA-6833:
---
Summary: KafkaProducer throws "Invalid partition given with
record" exception
Key: KAFKA-6833
URL: https://issues.apache.org/jira/browse/KAFKA-6833
Proj
[
https://issues.apache.org/jira/browse/KAFKA-6446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson resolved KAFKA-6446.
Resolution: Fixed
Fix Version/s: 1.2.0
> KafkaProducer with transactionId endl
[
https://issues.apache.org/jira/browse/KAFKA-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mickael Maison resolved KAFKA-6633.
---
Resolution: Not A Problem
> Is KafkaProducer still thread safe in version 1.
Herbert Koelman created KAFKA-6633:
--
Summary: Is KafkaProducer still thread safe in version 1.0.1
Key: KAFKA-6633
URL: https://issues.apache.org/jira/browse/KAFKA-6633
Project: Kafka
Issue
[
https://issues.apache.org/jira/browse/KAFKA-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edoardo Comar resolved KAFKA-6516.
--
Resolution: Won't Fix
thanks, [~rsivaram] so my expectation was invalid then
> Kafka
Edoardo Comar created KAFKA-6516:
Summary: KafkaProducer retries indefinitely to authenticate on
SaslAuthenticationException
Key: KAFKA-6516
URL: https://issues.apache.org/jira/browse/KAFKA-6516
Eduardo Sciullo created KAFKA-6446:
--
Summary: KafkaProducer with transactionId endless waits when
bootstrap server is down
Key: KAFKA-6446
URL: https://issues.apache.org/jira/browse/KAFKA-6446
[
https://issues.apache.org/jira/browse/KAFKA-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huxihx resolved KAFKA-4767.
---
Resolution: Fixed
already fixed in another jira.
> KafkaProducer is not joining its IO thread prope
Github user asfgit closed the pull request at:
https://github.com/apache/kafka/pull/4094
---
GitHub user hmcl opened a pull request:
https://github.com/apache/kafka/pull/4094
MINOR: Correct KafkaProducer Javadoc spelling of property
'max.in.flight.requests.per.connection'
Currently, in branches _trunk_, _0.11.0_, and _1.0_ th
Matthias J. Sax created KAFKA-5936:
--
Summary: KafkaProducer should not wrap InterruptedException in
close() with KafkaException
Key: KAFKA-5936
URL: https://issues.apache.org/jira/browse/KAFKA-5936
Github user asfgit closed the pull request at:
https://github.com/apache/kafka/pull/3697
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Yingji Zhang created KAFKA-5803:
---
Summary: KafkaProducer should ignore invalid record from
ProducerInterceptors
Key: KAFKA-5803
URL: https://issues.apache.org/jira/browse/KAFKA-5803
Project: Kafka
Github user asfgit closed the pull request at:
https://github.com/apache/kafka/pull/3703
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
GitHub user huxihx opened a pull request:
https://github.com/apache/kafka/pull/3703
KAFKA-5755: KafkaProducer should be refactored to use LogContext
With LogContext, each producer log item is automatically prefixed with
client id and transactional id.
@hachikuji
Since a
[
https://issues.apache.org/jira/browse/KAFKA-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar resolved KAFKA-2289.
--
Resolution: Fixed
This has been fixed.
> KafkaProducer logs erroneous warning on star
GitHub user omkreddy opened a pull request:
https://github.com/apache/kafka/pull/3697
KAFKA-2105: add topic null check to KafkaProducer.partitionsFor method
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/omkreddy/kafka KAFKA-210
0.11.0.0. Pl reopen if you think the issue still exists
> KafkaProducer may send duplicated message sometimes
> ---
>
> Key: KAFKA-4951
> URL: https://issues.apache.org/jira/browse/KAFKA-4951
>
Leo Xuzhang Lin created KAFKA-5712:
--
Summary: Expose ProducerConfig in the KafkaProducer clients
Key: KAFKA-5712
URL: https://issues.apache.org/jira/browse/KAFKA-5712
Project: Kafka
Issue
GitHub user evis opened a pull request:
https://github.com/apache/kafka/pull/3450
KAFKA-5529 Use only KafkaProducer in ConsoleProducer
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/evis/kafka console-producer-remove-deprecated
GitHub user 10110346 opened a pull request:
https://github.com/apache/kafka/pull/3440
MINOR: Close producer in all KafkaProducer tests
After running KafkaProducer tests, we should close producer for free
resource
You can merge this pull request into a Git repository by running
oker will be 32 KB in 0.11.0.0.
Similarly, the buffer size for Gzip was increased to 8KB.
> Improve mechanism for compression stream block size selection in KafkaProducer
> --
>
>
e to continue to work on this?
If not, I will provide a patch based on latest trunk code.
> KafkaProducer logs erroneous warning on startup
> ---
>
> Key: KAFKA-2289
> URL: https://issues.apache.org/jir
1 - 100 of 451 matches
Mail list logo
