Re: Spark & NLP

2014-09-10 Thread Oleksandr Olgashko
Factorie (https://github.com/factorie/factorie) might be what you need (i'd suggest conditional random field/maximum entropy Markov mode for NER). Also you can use Java's libraries (for example, i'm using opennlp's implementation of MEMM for close-to-NER-problem in Scala) Chalk (https://github.com/

Re: Spark & NLP

2014-09-10 Thread andy petrella
never tried but might fit your need: http://www.scalanlp.org/ It's the parent project of both breeze (already part of spark) and epic. However you'll have to train for IT (not part of the supported list) (actually I never used it because for my very small needs, I generally just perform a small n

Spark & NLP

2014-09-10 Thread Paolo Platter
Hi all, What is your preferred scala NLP lib ? why ? Is there any items on the spark’s road map to integrate NLP features ? I basically need to perform NER line by line, so I don’t need a deep integration with the distributed engine. I only want simple dependencies and the chance to build a dict