I was debugging why a Kafka-based streaming app doesn't seem to find Kafka-related integration classes when run standalone from our latest 2.3.1 release, and noticed that there doesn't seem to be any Kafka-related jars from Spark in the distro. In jars/, I see:
spark-catalyst_2.11-2.3.1.jar spark-core_2.11-2.3.1.jar spark-graphx_2.11-2.3.1.jar spark-hive-thriftserver_2.11-2.3.1.jar spark-hive_2.11-2.3.1.jar spark-kubernetes_2.11-2.3.1.jar spark-kvstore_2.11-2.3.1.jar spark-launcher_2.11-2.3.1.jar spark-mesos_2.11-2.3.1.jar spark-mllib-local_2.11-2.3.1.jar spark-mllib_2.11-2.3.1.jar spark-network-common_2.11-2.3.1.jar spark-network-shuffle_2.11-2.3.1.jar spark-repl_2.11-2.3.1.jar spark-sketch_2.11-2.3.1.jar spark-sql_2.11-2.3.1.jar spark-streaming_2.11-2.3.1.jar spark-tags_2.11-2.3.1.jar spark-unsafe_2.11-2.3.1.jar spark-yarn_2.11-2.3.1.jar I checked make-distribution.sh, and it copies a bunch of JARs into the distro, but does not seem to touch the kafka modules. Am I crazy or missing something obvious -- those should be in the release, right?
