Re: How should I avoid words that are effectively stopwords in Riak Search?
Rob, Riak Search doesn't have a traditional term-frequency count. It has something similar but it's an estimate and it is much more expensive than a simple table lookup. Even if it did have term-frequency it doesn't really expose it to the outside world. Not only that, but the standard analyzer provides no way to specify additional stop words. You'd have to keep track of this data externally and do some pre-processing to remove stopwords before. For the last 9 months I've been working on a project called Yokozuna with the goal to replace Riak Search [1]. It's like Riak Search except much better because the underlying engine is actually Solr/Lucene, not an inferior clone written in Erlang. In that case you could add new stopwords, exploit query caching, and use newer features like LUCENE-4628 [2] to help combat high frequency terms. You'd also have an easy way to get frequency count for a given term to determine if you should make it a stopword. [1] https://github.com/basho/yokozuna [2] https://issues.apache.org/jira/browse/LUCENE-4628 On Fri, Mar 22, 2013 at 2:21 PM, Rob Speer wrote: > My company is starting to use Riak for document storage. I'm pretty happy > about how it has been working so far, but I see the messages of foreboding > and doom out there about Riak Search and I've encountered a problem myself. > > I can't really avoid using Riak Search, as full text indexing is a key > feature we need to provide. If Riak Search is suboptimal, so is basically > every other text index out there. We've just been burned by ElasticSearch's > ineffective load balancing (who would have guessed, consistent hashing is > kind of important). > > I know that performing searches in Riak Search that return many thousands > of documents is discouraged for performance reasons, and the developers > encourage removing stopwords to help with this. There's additionally, I > have seen, a hard limit on the number of documents that can be examined by > a search query; if any term matches more than 100,000 documents, the query > will return a too_many_results error (and, incidentally, things will get so > confused that, in the Python client, the *next* query will also fail with > an HTTP error 400). > > The question is, what should I actually do to avoid this case? I've > already removed the usual stopwords, but any particular set of documents > might have its own personal stopwords. For example, in a database of > millions of hotel reviews, the word 'hotel' could easily appear in more > than 100,000 documents. > > If we need to search for '5-star hotel', it's wasteful and probably > crash-prone to retrieve all the 'hotel' results. What I'd really like to do > is just search for '5-star', which because of IDF scoring will have about > the same effect. That requires knowing somehow that the word 'hotel' > appears in too many documents. > > Is there a way to determine, via Riak, which terms are overused so I can > remove them from search queries? Or do I need to keep track of this > entirely on the client end so I can avoid searching for those terms? > > Thanks, > -- Rob Speer > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Unexpected Riak 1.3 crash
I've been able to log a better error message regarding this:
2013-03-28 22:08:07 =ERROR REPORT
** Generic server disksup terminating
** Last message in was {'EXIT',#Port<0.1415>,normal}
** When Server state ==
[{data,[{"OS",{unix,linux}},{"Timeout",180},{"Threshold",80},{"DiskData",[{"/",206424760,32},{"/dev/shm",3559560,0}]}]}]
** Reason for termination ==
** {port_died,normal}
2013-03-28 22:08:07 =CRASH REPORT
crasher:
initial call: disksup:init/1
pid: <0.88.0>
registered_name: disksup
exception exit:
{{port_died,normal},[{gen_server,terminate,6,[{file,"gen_server.erl"},{line,747}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}
ancestors: [os_mon_sup,<0.86.0>]
messages: []
links: [<0.87.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 59654
neighbours:
2013-03-28 22:08:07 =SUPERVISOR REPORT
Supervisor: {local,os_mon_sup}
Context:child_terminated
Reason: {port_died,normal}
Offender:
[{pid,<0.88.0>},{name,disksup},{mfargs,{disksup,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]
--
View this message in context:
http://riak-users.197444.n3.nabble.com/Unexpected-Riak-1-3-crash-tp4027359p4027405.html
Sent from the Riak Users mailing list archive at Nabble.com.
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Unexpected Riak 1.3 crash
I'm using Riak 1.3 and levelDB. I found out this thread, I'm not sure if this is somehow related: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2012-September/009425.html -- View this message in context: http://riak-users.197444.n3.nabble.com/Unexpected-Riak-1-3-crash-tp4027359p4027406.html Sent from the Riak Users mailing list archive at Nabble.com. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Unexpected Riak 1.3 crash
More info here: https://github.com/basho/riak/issues/301 -- View this message in context: http://riak-users.197444.n3.nabble.com/Unexpected-Riak-1-3-crash-tp4027359p4027407.html Sent from the Riak Users mailing list archive at Nabble.com. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Rebalancing issue -- failure to hand-off partitions
That's an interesting result. Once it's fully rebalanced, I'd turn it back on and see if the fallback handoffs still fail. If they do, I'd recommend using memory limits, rather than TTL to limit growth (also, remember that memory limits are *per vnode*, rather than per node). They're slower, but they don't seem to have this problem. I'll do my best to figure out what is going on and get a patch (if one is needed) into the next version. On Fri, Mar 29, 2013 at 3:15 PM, Giri Iyengar wrote: > Evan, > > As recommended by you, I disabled the TTL on the memory backends and did a > rolling restart of the cluster. Now, things are rebalancing quite nicely. > Do you think I can turn the TTL back on once the rebalancing completes? I'd > like to ensure that the vnodes in memory don't keep growing forever. > > -giri > > > On Thu, Mar 28, 2013 at 6:50 PM, Giri Iyengar > wrote: >> >> Evan, >> >> This has been happening for a while now (about 3.5 weeks now), even prior >> to our upgrade to 1.3. >> >> -giri >> >> On Thu, Mar 28, 2013 at 6:36 PM, Evan Vigil-McClanahan >> wrote: >>> >>> No. AAE is unrelated to the handoff subsystem. I am not familiar >>> enough with the lowest level of it's working to know if it'd reproduce >>> the TTL stuff across on nodes that don't have it. >>> >>> I am not totally sure about your timeline here. >>> >>> When did you start seeing these errors, before or after your upgrade >>> to 1.3? When did you start your cluster transition? What cluster >>> transitions have you initiated? >>> >>> If these errors started with 1.3, an interesting experiment would be >>> to disable AAE and do a rolling restart of the cluster, which should >>> lead to empty memory backends that won't be populated by AAE with >>> anything suspicious. That said: if you've had cluster balance >>> problems for a while, it's possible that these messages (even this >>> whole issue) is just masking some other problem. >>> >>> On Thu, Mar 28, 2013 at 3:24 PM, Giri Iyengar >>> wrote: >>> > Evan, >>> > >>> > All nodes have been restarted (more than once, in fact) after the >>> > config >>> > changes. Using riak-admin aae-status, I noticed that the anti-entropy >>> > repair >>> > is still proceeding across the cluster. >>> > It has been less than 24 hours since I upgraded to 1.3 and maybe I have >>> > to >>> > wait till the first complete build of the index trees happens for the >>> > cluster to start rebalancing itself. >>> > Could that be the case? >>> > >>> > -giri >>> > >>> > >>> > On Thu, Mar 28, 2013 at 5:49 PM, Evan Vigil-McClanahan >>> > wrote: >>> >> >>> >> Giri, >>> >> >>> >> if all of the nodes are using identical app.config files (including >>> >> the joining node) and have been restarted since those files changed, >>> >> it may be some other, related issue. >>> >> >>> >> On Thu, Mar 28, 2013 at 2:46 PM, Giri Iyengar >>> >> wrote: >>> >> > Evan, >>> >> > >>> >> > I reconfirmed that all the servers are using identical app.configs. >>> >> > They >>> >> > all >>> >> > use multi-backend schema. Are you saying that some of the vnodes are >>> >> > in >>> >> > memory backend in one physical node and in eleveldb backend in >>> >> > another >>> >> > physical node? >>> >> > If so, how can I fix the offending vnodes? >>> >> > >>> >> > Thanks, >>> >> > >>> >> > -giri >>> >> > >>> >> > On Thu, Mar 28, 2013 at 5:18 PM, Evan Vigil-McClanahan >>> >> > wrote: >>> >> >> >>> >> >> it would if some of the nodes weren't migrated to the new >>> >> >> multi-backend schema; if a memory node was trying to hand off to a >>> >> >> eleveldb backed node, you'd see this. >>> >> >> >>> >> >> On Thu, Mar 28, 2013 at 2:05 PM, Giri Iyengar >>> >> >> wrote: >>> >> >> > Evan, >>> >> >> > >>> >> >> > I verified that all of the memory backends have the same ttl >>> >> >> > settings >>> >> >> > and >>> >> >> > have done rolling restarts but it doesn't seem to make a >>> >> >> > difference. >>> >> >> > One >>> >> >> > thing to note though -- I remember this problem starting roughly >>> >> >> > around >>> >> >> > the >>> >> >> > time I migrated a bucket from being backed by leveldb to being >>> >> >> > backed >>> >> >> > by >>> >> >> > memory. I did this by setting the bucket properties via curl and >>> >> >> > let >>> >> >> > Riak do >>> >> >> > the migration of the objects in that bucket. Would that cause >>> >> >> > such >>> >> >> > issues? >>> >> >> > >>> >> >> > Thanks for your help. >>> >> >> > >>> >> >> > -giri >>> >> >> > >>> >> >> > >>> >> >> > On Thu, Mar 28, 2013 at 4:55 PM, Evan Vigil-McClanahan >>> >> >> > wrote: >>> >> >> >> >>> >> >> >> Giri, I've seen similar issues in the past when someone was >>> >> >> >> adjusting >>> >> >> >> their ttl setting on the memory backend. Because one memory >>> >> >> >> backend >>> >> >> >> has it and the other does not, it fails on handoff. The >>> >> >> >> solution >>> >> >> >> then was to make sure that all memory backend settings are the >>> >> >> >> same >>> >> >> >> and then do a rolling
Re: Rebalancing issue -- failure to hand-off partitions
Evan, As recommended by you, I disabled the TTL on the memory backends and did a rolling restart of the cluster. Now, things are rebalancing quite nicely. Do you think I can turn the TTL back on once the rebalancing completes? I'd like to ensure that the vnodes in memory don't keep growing forever. -giri On Thu, Mar 28, 2013 at 6:50 PM, Giri Iyengar wrote: > Evan, > > This has been happening for a while now (about 3.5 weeks now), even prior > to our upgrade to 1.3. > > -giri > > On Thu, Mar 28, 2013 at 6:36 PM, Evan Vigil-McClanahan < > emcclana...@basho.com> wrote: > >> No. AAE is unrelated to the handoff subsystem. I am not familiar >> enough with the lowest level of it's working to know if it'd reproduce >> the TTL stuff across on nodes that don't have it. >> >> I am not totally sure about your timeline here. >> >> When did you start seeing these errors, before or after your upgrade >> to 1.3? When did you start your cluster transition? What cluster >> transitions have you initiated? >> >> If these errors started with 1.3, an interesting experiment would be >> to disable AAE and do a rolling restart of the cluster, which should >> lead to empty memory backends that won't be populated by AAE with >> anything suspicious. That said: if you've had cluster balance >> problems for a while, it's possible that these messages (even this >> whole issue) is just masking some other problem. >> >> On Thu, Mar 28, 2013 at 3:24 PM, Giri Iyengar >> wrote: >> > Evan, >> > >> > All nodes have been restarted (more than once, in fact) after the config >> > changes. Using riak-admin aae-status, I noticed that the anti-entropy >> repair >> > is still proceeding across the cluster. >> > It has been less than 24 hours since I upgraded to 1.3 and maybe I have >> to >> > wait till the first complete build of the index trees happens for the >> > cluster to start rebalancing itself. >> > Could that be the case? >> > >> > -giri >> > >> > >> > On Thu, Mar 28, 2013 at 5:49 PM, Evan Vigil-McClanahan >> > wrote: >> >> >> >> Giri, >> >> >> >> if all of the nodes are using identical app.config files (including >> >> the joining node) and have been restarted since those files changed, >> >> it may be some other, related issue. >> >> >> >> On Thu, Mar 28, 2013 at 2:46 PM, Giri Iyengar >> >> wrote: >> >> > Evan, >> >> > >> >> > I reconfirmed that all the servers are using identical app.configs. >> They >> >> > all >> >> > use multi-backend schema. Are you saying that some of the vnodes are >> in >> >> > memory backend in one physical node and in eleveldb backend in >> another >> >> > physical node? >> >> > If so, how can I fix the offending vnodes? >> >> > >> >> > Thanks, >> >> > >> >> > -giri >> >> > >> >> > On Thu, Mar 28, 2013 at 5:18 PM, Evan Vigil-McClanahan >> >> > wrote: >> >> >> >> >> >> it would if some of the nodes weren't migrated to the new >> >> >> multi-backend schema; if a memory node was trying to hand off to a >> >> >> eleveldb backed node, you'd see this. >> >> >> >> >> >> On Thu, Mar 28, 2013 at 2:05 PM, Giri Iyengar >> >> >> wrote: >> >> >> > Evan, >> >> >> > >> >> >> > I verified that all of the memory backends have the same ttl >> settings >> >> >> > and >> >> >> > have done rolling restarts but it doesn't seem to make a >> difference. >> >> >> > One >> >> >> > thing to note though -- I remember this problem starting roughly >> >> >> > around >> >> >> > the >> >> >> > time I migrated a bucket from being backed by leveldb to being >> backed >> >> >> > by >> >> >> > memory. I did this by setting the bucket properties via curl and >> let >> >> >> > Riak do >> >> >> > the migration of the objects in that bucket. Would that cause such >> >> >> > issues? >> >> >> > >> >> >> > Thanks for your help. >> >> >> > >> >> >> > -giri >> >> >> > >> >> >> > >> >> >> > On Thu, Mar 28, 2013 at 4:55 PM, Evan Vigil-McClanahan >> >> >> > wrote: >> >> >> >> >> >> >> >> Giri, I've seen similar issues in the past when someone was >> >> >> >> adjusting >> >> >> >> their ttl setting on the memory backend. Because one memory >> backend >> >> >> >> has it and the other does not, it fails on handoff. The >> solution >> >> >> >> then was to make sure that all memory backend settings are the >> same >> >> >> >> and then do a rolling restart of the cluster (ignoring a lot of >> >> >> >> errors >> >> >> >> along the way). I am not sure that this is applicable to your >> case, >> >> >> >> but it's something to look at. >> >> >> >> >> >> >> >> On Thu, Mar 28, 2013 at 10:22 AM, Giri Iyengar >> >> >> >> wrote: >> >> >> >> > Godefroy: >> >> >> >> > >> >> >> >> > Thanks. Your email exchange on the mailing list was what >> prompted >> >> >> >> > me >> >> >> >> > to >> >> >> >> > consider switching to Riak 1.3. I do see repair messages in the >> >> >> >> > console >> >> >> >> > logs >> >> >> >> > and so some healing is happening. However, there are a bunch of >> >> >> >> > hinted >> >> >> >> > handoffs and ownership handoffs that are simply
Austin-area Riak users: Data Day tomorrow, meetup coming 4/17
For anyone in or near Austin: Mark Phillips, Seth Thomas and I will be at Data Day tomorrow, and Seth and I will be hosting a meetup in three weeks at Mass Relevance. Please feel free to stop by either event and say hi. http://www.meetup.com/Austin-Riak-Meetup/ I will be hosting more meetups in Texas this year, and I'll definitely be looking for users to talk about their experiences with Riak. Please let me know if you'd be interested in sharing your pain points and successes (not necessarily in that order). And, of course, the essentials: free food and swag. -John Daily Technical Evangelist ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
