Kafka Support new topic subscriptions without requiring restart of the streaming context

2016-08-07 Thread r7raul1...@163.com
How to add new topic to kafka without requiring restart of the streaming context? r7raul1...@163.com

[SPARK-2.0][SQL] UDF containing non-serializable object does not work as expected

2016-08-07 Thread Hao Ren
I am playing with spark 2.0 What I tried to test is: Create a UDF in which there is a non serializable object. What I expected is when this UDF is called during materializing the dataFrame where the UDF is used in "select", an task non serializable exception should be thrown. It depends also which