Which version of spark are you on? I thought that was added to the spark UI in recent versions.
DIrect api doesn't have any inherent interaction with zookeeper. If you need number of messages per batch and aren't on a recent enough version of spark to see them in the ui, you can get them programatically from the offset ranges. See the definition of count() in recent versions of KafkaRDD for an example. On Wed, Dec 9, 2015 at 9:39 AM, Dan Dutrow <dan.dut...@gmail.com> wrote: > Is there documentation for how to update the metrics (#messages per batch) > in the Spark Streaming tab when using the Direct API? Does the Streaming > tab get its information from Zookeeper or something else internally? > -- > Dan ✆ >
