Hi Nathan,
You’re right, it looks like we don’t currently provide a method to unregister
accumulators. I’ve opened a JIRA to discuss a fix:
https://issues.apache.org/jira/browse/SPARK-3885
In the meantime, here’s a workaround that might work: Accumulators have a
public setValue() method that
I notice that accumulators register themselves with a private Accumulators
object.
I don't notice any way to unregister them when one is done.
Am I missing something? If not, is there any plan for how to free up that
memory?
I've a case where we're gathering data from repeated queries using some
