For a work around you could use streaming and pagination. Request smaller pages of data (i.e. sub 60 seconds worth) and use streaming to get the results to your client sooner.
In HTTP this would look like http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?max_results=10000&stream=true your results will include a continuation like {"continuation":"g2gCYgAAFXttAAAABDU0OTk="} and you can use that to get the next N results. Breaking your query up that way should duck the timeout. Furthermore, adding &stream=true will mean the first results is received very rapidly. I don't think the Ruby client is up to date for the new 2i features, but you could monkeypatch as before. Cheers Russell On 26 Jul 2013, at 19:00, Sean McKibben <grap...@graphex.com> wrote: > Thank you for looking in to this. This is a major problem for our production > cluster, and we're in a bit of a bind right now trying to figure out a > workaround in the interim. It sounds like maybe a mapreduce might handle the > timeout properly, so hopefully we can do that in the meantime. > If there is any way we can have a hotfix ASAP though, that would be > preferable. Certainly would not be a problem for us to edit a value in the > config file (and given the lack of support in the ruby client for the timeout > setting, the ability to edit the global default would be preferred). > In the ruby client i had to monkeypatch it like this to even submit the > timeout value, which is not ideal: > > module Riak > class Client > class HTTPBackend > def get_index(bucket, index, query) > bucket = bucket.name if Bucket === bucket > path = case query > when Range > raise ArgumentError, t('invalid_index_query', :value => > query.inspect) unless String === query.begin || Integer === query.end > index_range_path(bucket, index, query.begin, query.end) > when String, Integer > index_eq_path(bucket, index, query, 'timeout' => '260000') > else > raise ArgumentError, t('invalid_index_query', :value => > query.inspect) > end > response = get(200, path) > JSON.parse(response[:body])['keys'] > end > end > end > end > > Thanks for the update, > Sean > > > > On Jul 26, 2013, at 4:49 PM, Russell Brown <russell.br...@mac.com> wrote: > >> Hi Sean, >> I'm very sorry to say that you've found a featurebug. >> >> There was a fix put in here https://github.com/basho/riak_core/pull/332 >> >> But that means that the default timeout of 60 seconds is now honoured. In >> the past it was not. >> >> As far as I can see the 2i endpoint never accepted a timeout argument, and >> it still does not. >> >> The fix would be to add the timeout to the 2i API endpoints, and I'll do >> that straight away. >> >> In the meantime, I wonder if streaming the results would help, or if you'd >> still hit the overall timeout? >> >> Very sorry that you've run into this. Let me know if streaming helps, I've >> raised an issue here[1] if you want to track this bug >> >> Cheers >> >> Russell >> >> [1] https://github.com/basho/riak_kv/issues/610 >> >> >> On 26 Jul 2013, at 17:59, Sean McKibben <grap...@graphex.com> wrote: >> >>> I should have mentioned that I also tried: >>> curl -H "X-Riak-Timeout: 260000" >>> "http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?timeout=260000"; >>> -i >>> but still receive the 500 error below exactly at the 60 second mark. Is >>> this a bug? >>> >>> Secondary to getting this working at all, is this documented anywhere? and >>> any way to set this timeout using the ruby riak client? >>> >>> Stream may well work, but I'm going to have to make a number of changes on >>> the client side to deal with the results. >>> >>> Sean >>> >>> On Jul 26, 2013, at 3:53 PM, Brian Roach <ro...@basho.com> wrote: >>> >>>> Sean - >>>> >>>> The timeout isn't via a header, it's a query param -> &timeout=xxxx >>>> >>>> You can also use stream=true to stream the results. >>>> >>>> - Roach >>>> >>>> Sent from my iPhone >>>> >>>> On Jul 26, 2013, at 3:43 PM, Sean McKibben <grap...@graphex.com> wrote: >>>> >>>>> We just upgraded to 1.4 and are having a big problem with some of our >>>>> larger 2i queries. We have a few key queries that takes longer than 60 >>>>> seconds (usually about 110 seconds) to execute, but after going to 1.4 we >>>>> can't seem to get around a 60 second timeout. >>>>> >>>>> I've tried: >>>>> curl -H "X-Riak-Timeout: 260000" >>>>> "http://127.0.0.1:8098/buckets/mybucket/index/test_bin/myval?x-riak-timeout=260000"; >>>>> -i >>>>> >>>>> But I always get >>>>> HTTP/1.1 500 Internal Server Error >>>>> Vary: Accept-Encoding >>>>> Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact) >>>>> Date: Fri, 26 Jul 2013 21:41:28 GMT >>>>> Content-Type: text/html >>>>> Content-Length: 265 >>>>> Connection: close >>>>> >>>>> <html><head><title>500 Internal Server >>>>> Error</title></head><body><h1>Internal Server Error</h1>The server >>>>> encountered an error while processing this >>>>> request:<br><pre>{error,{error,timeout}}</pre><P><HR><ADDRESS>mochiweb+webmachine >>>>> web server</ADDRESS></body></html> >>>>> >>>>> Right at the 60 second mark. What can I set to give my secondary index >>>>> queries more time?? >>>>> >>>>> This is causing major problems for us :( >>>>> >>>>> Sean >>>>> _______________________________________________ >>>>> riak-users mailing list >>>>> riak-users@lists.basho.com >>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> _______________________________________________ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
