Re: k-means for text mining in a streaming context

2015-06-17 Thread Xiangrui Meng
Yes. You can apply HashingTF on your input stream and then use StreamingKMeans for training and prediction. -Xiangrui On Mon, Jun 8, 2015 at 11:05 AM, Ruslan Dautkhanov wrote: > Hello, > > https://spark.apache.org/docs/latest/mllib-feature-extraction.html > would Feature Extraction and Transforma

k-means for text mining in a streaming context

2015-06-08 Thread Ruslan Dautkhanov
Hello, https://spark.apache.org/docs/latest/mllib-feature-extraction.html would Feature Extraction and Transformation work in a streaming context? Wanted to extract text features, build K-means clusters for streaming context to detect anomalies on a continuous text stream. Would it be possible?