Does anyone have any thoughts on this? Should I create a proper KIP to
better facilitate discussion?
On Tue, Mar 21, 2017 at 11:05 AM, Kevin Conaway <kevin.a.cona...@gmail.com>
wrote:
> Greetings all,
>
> Please correct me if this is not the right place to discuss this. I filed
> a jira about this but I figured the mailing list might encourage better
> collaboration.
>
> I'd like to propose adding the ProducerRecord field to the
> onAcknowledgement callback in the ProducerInterceptor interface that was
> added in KAFKA-3162
>
> The issue I'm having is that its not possible to determine if a specific
> record was sent (based on the record content or key), only that some record
> was sent based on the RecordMetadata. From reading the KIP and the initial
> mailing list discussion, it seems that the design was modeled after the
> existing Producer.Callback interface. This model makes sense given that
> callbacks are typically anonymous functions where you would have access to
> the producer record:
>
> ProducerRecord record = ...;
> producer.send(record, new Callback() {
> @Override
> public void onCompletion(RecordMetadata recordMetadata, Exception e)
> {
> // do something with record
> }
> })
>
> However, based on my understanding of the interceptor design, the
> interceptor should be thread safe and thus stateless. Even if you wanted to
> make it stateful, its not easy to tie together a record from onSend to
> onAcknowledgement. Exception handling also suffers from this if you want
> to tie the exception to the specific record that failed
>
> With that in mind, is there a way that the producer interceptor API can be
> modified so that implementers can be aware of the key & content of the
> record that was acknowledged or failed?
>
> Thanks,
>
> --
> Kevin Conaway
> http://www.linkedin.com/pub/kevin-conaway/7/107/580/
> https://github.com/kevinconaway
>
--
Kevin Conaway
http://www.linkedin.com/pub/kevin-conaway/7/107/580/
https://github.com/kevinconaway
