On Spark 1.2: I am trying to capture # records read from a kafka topic:
val inRecords = ssc.sparkContext.accumulator(0, "InRecords")
..
kInStreams.foreach( k =>
{
k.foreachRDD ( rdd => inRecords += rdd.count().toInt )
inRecords.value
Question is how do I get the accumulator to show up in the UI? I tried
"inRecords.value" but that didn't help. Pretty sure it isn't showing up in
Stage metrics.
What's the trick here? collect?
Thanks,
Tim
