Re: Java Implementation of StreamingContext.fileStream

2014-09-23 Thread Michael Quinlan
Thanks very much for the pointer, which validated my initial approach. It turns out that I was creating a tag for the abstract class "InputFormat.class". Using "TextInputFormat.class" instead fixed my issue. Regards, Mike -- View this message in context: http://apache-spark-user-list.1001560.

Re: Java Implementation of StreamingContext.fileStream

2014-09-22 Thread Akhil Das
Here's a working version that we have. > DStream> hadoopDStream = > streamingContext.fileStream("/akhld/lookhere/", new Function Object>(){ > @Override > public Object call(Path path) throws Exception { > // TODO Auto-generated method stub > return !path.getName().startsWith("."); > } }, true, Sp

Java Implementation of StreamingContext.fileStream

2014-09-22 Thread Michael Quinlan
I'm attempting to code a Java only implementation accessing the StreamingContext.fileStream method and am especially interested in setting the boolean "newFilesOnly" to false. Unfortunately my code throws exceptions: Exception in thread "main" java.lang.InstantiationException at sun.reflec