Hi, Petr,
Thank you for the pointer to the site. Indeed a treasure trove of ideas on
stemmer algorithms.
Tuba
On Thu, Aug 11, 2011 at 8:45 AM, Petr Gladkikh <petrg...@gmail.com> wrote:
> On Mon, Aug 8, 2011 at 1:46 PM, Tuba Lambanog <tuba.lamba...@gmail.com>
> wrote:
> > Hello,
> >
> > I’m doing a word stemmer for a non-English language. A stemmer parses
> > a word into its word parts: prefixes, roots, suffixes. The input word
> > is at least a root word (English example would be ‘cloud’), but can be
> > any combination of prefix(es) and a root (e.g., 'pre-nuptial'), or a
> > root and suffix(es) (‘cloudy’), or all three ('unidirection'). A
> > sequence of more than one prefix in a word is considered one
> > occurrence of a prefix, and similarly for complex prefixes, thus,
> > ‘directional’ is considered to have the ‘single’ suffix ‘ional’. The
> > prefixes, roots, and suffixes are in their own set data structure.
> >
> > The approach I am pursuing is to create a set of potential suffixes
> > that the input word contains. Asssume, for simplicity, that the suffix
> > set consists of #{-or, -er, -al, -ion, -ional, able}. The input
> > ‘directional’ would have the candidate suffix set #{-al –ional}. Now,
> > drop the longest suffix (‘ional’) from the input then check the
> > remaining string (‘direct’) if it is a root; if it is, done. If not,
> > try the next suffix (‘-al’) in the potential suffix set. Prefixes
> > will be similarly processed. Input words with both prefixes and
> > affixes will be fun to do ;)
> >
> > I’m having a hard time thinking through the process of generating the
> > candidate suffix set using set forms, and I’m beginning to think I
> > have selected an arduous path (for me).
> >
> > Thoughts?
> >
>
> Somehow offtopic maybe, but have you looked at Snowball
> http://snowball.tartarus.org/ ?
> Algorithm is different but language that is used to describe stemmers
> there is almost lisp and may be useful at least as reference.
>
> --
> Petr Gladkikh
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
