Could it be possible that this is a bug? I hate to throw that word around,
but this is definitely not expected behavior (as far as I can tell). If
anyone has a suggestion for a work around or better way to accomplish
handling a global value in UpdateStateByKey, that would be fantastic.
Thanks
On
I've run into an issue where a global variable used within an
UpdateStateByKey function isn't being assigned after the application
restarts from a checkpoint. Using ForEachRDD I have a global variable 'A'
that is propagated from a file every time a batch runs, and A is then used
in an UpdateStateBy
