Hi Jason,
Thanks for the correction. See inline.
On Wed, Aug 9, 2017 at 5:13 PM, Jason Gustafson <ja...@confluent.io> wrote:
> Minor correction: the OutOfOrderSequenceException is not fatal for the
> idempotent producer and it is not necessarily tied to the acks setting
> (though it is more likely to be thrown with acks=1).
Right, it would be worth expanding on the specifics of this. My
understanding is that common failure scenarios could trigger it.
> It is used to signal
> the user that there was a gap in the delivery of messages. You can hit this
> if there is a pause on the producer and the topic retention kicks in and
> deletes the last records the producer had written. However, it is possible
> for the user to catch it and simply keep producing (internally the producer
> will generate a new ProducerId).
I see, our documentation states that it's fatal in the following example
and in the `send` method. I had overlooked that this was mentioned in the
context of transactions. If we were to enable idempotence by default, we'd
want to flesh out the docs for idempotence without transactions.
* try {
* producer.beginTransaction();
* for (int i = 0; i < 100; i++)
* producer.send(new ProducerRecord<>("my-topic",
Integer.toString(i), Integer.toString(i)));
* producer.commitTransaction();
* } catch (ProducerFencedException | OutOfOrderSequenceException |
AuthorizationException e) {
* // We can't recover from these exceptions, so our only option is
to close the producer and exit.
* producer.close();
* } catch (KafkaException e) {
* // For all other exceptions, just abort the transaction and try again.
* producer.abortTransaction();
* }
* producer.close();
Nevertheless, pre-idempotent-producer code
> won't be expecting this exception, and that may cause it to break in cases
> where it previously wouldn't. This is probably the biggest risk of the
> change.
>
This is a good point and we should include it in the KIP.
Ismael
