Hi Malcolm,
The warning is not particularly related to your restarts. It happens when
the offset requested by the consumer no longer exists on the broker.
It can typically happen if you are consuming from a topic with TTL enabled
and the time retention kicked in and broker purged older offsets.
You can find the starting offset from the container logs for that specific
topic and partition. Look for
"Registering ssp: {} with offset: {}"
Once you get the starting offset, you can verify if the above theory is
true by looking at the metadata of the topic in zookeeper to get the
beginning and end offsets for a particular partition.
Let me know if you have any other questions.
Thanks,
Bharath
On Thu, Oct 31, 2019 at 6:09 PM Malcolm McFarland <mmcfarl...@cavulus.com>
wrote:
> Hey folks,
>
> We're running Samza 0.14.1 on Hadoop 2.7.6. Every once in a while while
> restarting an application, the process will come up with some variation on
> this error:
>
> INFO Validating offset <offset> for topic and partition
> [<topic>,<partition>]
> WARN While refreshing brokers for [<topic>,<partition>]:
> org.apache.kafka.common.errors.OffsetOutOfRangeException: The requested
> offset is not within the range of offsets maintained by the server..
> Retrying
>
> This error is really mystifying us. We're not doing anything severe here,
> just using YARN's kill command to stop the application and then submitting
> it via a normal mechanism. Are there any best practices or gotchas
> surrounding restarting Samza applications on YARN that could help here?
>
> Cheers,
> Malcolm McFarland
> Cavulus
>
>
> This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any
> unauthorized or improper disclosure, copying, distribution, or use of the
> contents of this message is prohibited. The information contained in this
> message is intended only for the personal and confidential use of the
> recipient(s) named above. If you have received this message in error,
> please notify the sender immediately and delete the original message.
>
