The last offset is stored in file system you specified , how does it
expire? I don't understand. I haven't met that condition.
Srinivas V 于2020年3月19日周四 下午10:18写道：
> 1. How would a prod admin user/other engineers understand which process
> is this random groupid which is consuming a specific
1. How would a prod admin user/other engineers understand which process is
this random groupid which is consuming a specific topic? why is it
designed this way?
2. I don't see the groupid changing all the time. It is repeating on
restarts. Not able to understand when and how it changes. I know it
1.Maybe we can't use customized group id in structured streaming.
2.When restarting from failure or killing , the group id changes, but the
starting offset will be the last one you consumed last time .
Srinivas V 于2020年3月19日周四 下午12:36写道：
> Hello,
> 1. My Kafka consumer name is randomly being
Hello,
1. My Kafka consumer name is randomly being generated by spark structured
streaming. Can I override this?
2. When testing in development, when I stop my streaming job for Kafka
consumer job for couple of days and try to start back again, the job keeps
failing for missing offsets as the offse
