On Tue, Nov 8, 2011 at 7:15 AM, <riak-users-requ...@lists.basho.com> wrote:
> Date: Tue, 8 Nov 2011 07:08:59 -0500 > From: Hector Castro <hectcas...@gmail.com> > > I'm currently in the process of evaluating solutions to index the contents > of ~1TB of SEC (Securities and Exchange Commission) documents. File sizes > vary between a few KB to a couple hundred KB. I started evaluating Riak > first because ease of setting up and expanding a cluster are primary > requirements (ElasticSearch is also probably going to get evaluated, along > with Solr). > > Below I have a few specific questions that I was hoping people could help > with: > > * In going through the search querying documentation, I haven't > found a way to extract a section of a result containing matches. Something > similar to Google's search results page where you see an excerpt of the > webpage contents that match your query. Is something like this built-in so > that it doesn't have to be done by the application? > * Given that the documents total ~1TB of storage (not including the > generated indexes), does something like decreasing the n_val make sense? > Mostly the documents are bulk inserted on a daily or weekly basis ? other > than that all of the operations are read-only. > > Other than these specific questions, if anyone can provide general insight > on issues that would arise from a dataset like this within Riak, please > feel free to mention them. > I would opt for ElasticSearch. ES is document sharded, whereas RS is term sharded. The later means your query time is bounded by the performance of the worst term within a single vnode. In our experience this has been a problem. RS also has some rough edges. E.g. the Solr API does not expose a timeout parameter, which means if you query takes longer that the hardcoded timeout you'll have to modify your query or resort to using search inputs into a MR job, which does not expose the Solr API options. Which has some issues of its own, like the fact that the sort option sorts after slicing, not before, which means its useless for paging results by anything other than sorting by score. I think you'll find that ES, being based on Lucene, will give you more tokenizer options. RS is great to add some search to your data stored within Riak, but if you want a search engine, use something more specialized like ES. Elias Levy
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
