Load the textFile as an RDD. Something like this:
>
> > val file = sc.textFile("/path/to/file")
After this you can manipulate this RDD to filter texts the way you want
them :
> > val a1 = file.filter( line => line.contains("[ERROR]") )
> > val a2 = file.filter( line => line.contains("[WARN]") )
I think there is already an example for this shipped with Spark. However,
you do not benefit really from any spark functionality for this scenario.
If you want to do something more advanced you should look at Elasticsearch
or Solr
Le ven. 28 août 2015 à 16:15, Darksu a écrit :
> Hello,
>
> T
