Hi, I am seeing weird behavior in our spark structured streaming application where the offerts are not getting picked by the streaming job.
If I delete the checkpoint directory and run the job again, I can see the data for the first batch but it is not picking up new offsets again from the next job when the job is running. FYI, job is still running but it is not picking up new offsets. I am not able to figure out where the issue is in this case. Thanks, Asmath