Range Loop Timeout Error
We keep getting the following error in production:
@riak_search_backend:collect_info_response:55 range_loop timed out!
@riak_solr_searcher_wm:malformed_request:82 Unable to parse request:
{timeout,range_loop}
We don't any feedback as to what the query was that generated the problem, nor
do we have a consistent repro.
Has anyone else seen this error or can anyone help us with figuring what this
is and how to prevent it?
Thanks!
--Ken
clipboard, inc.
we're hiring!: www.clipboard.com/jobs
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: reg. storageon Localfile system
First of all, when dealing with large csv files I always use split to
shrink them down to manageable size. That way you can do all sorts of
parallel loading tricks via bash. Google search for "unix split".
@siculars
http://siculars.posterous.com
Sent from my rotary phone.
On Jul 12, 2012 2:23 AM, wrote:
>
>
>
>
>
>
> Thanks Sicular,
>
>
>
> Supervisor riak_core_vnode_sup had child undefined started with
> riak_core_vnode:start_link() at undefined exit with reason
> {timeout,{gen_server,call,[<0.1164.0>,stop]}} in context shutdown_error…
>
>
>
>
>
> trying to insert csv data of 434 MB onto 2 node riak cluster….iam getting
> the following error ..could you please share the reason for the same
>
> …Thanks In advance . attached app.config for your reference .
>
>
>
>
>
>
>
> This is my another quest need your valuable suggestion please
>
> Dear team,
>
> >
>
> >
>
> >
>
> >
>
> > FYI:we have a 4 quad core intel processor on each server on 2 node
> cluster with more than 1 TB of storage
>
> >
>
> >
>
> > I Ihave constructed the 2 node physical machine riak cluster with n_val
> 2 and my app.config ,vm.args are attached for your reference..
>
> >
>
> >
>
> >
>
> >
>
> > Please tell me where the bulk inserted data onto riak db gets stored on
> Local file system…its taking huge time to load small size itself…how to
> tune it to perform to large scale since we deal wit hbigdata of in few
> hungred GB’s?
>
> >
>
> >
>
> >
>
> >
>
> > Cmd used:time ./load_data1m Customercalls1m.csv
>
> >
>
> >
>
> >
>
> >
>
> > ./load_data100m CustomerCalls100m(got this error so changed default
> config of app.config…from 8 MB to 3072 MB
>
> >
>
> >
>
> > escript: exception error: no match of right hand side value
> {error,enoent}
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > size
>
> >
>
> >
>
> >
>
> > Load time
>
> >
>
> >
>
> >
>
> > No of mappersonapp.config
>
> >
>
> >
>
> >
>
> > Js-max-vm-mem on app.config
>
> >
>
> >
>
> >
>
> > Js-thread-stack
>
> >
>
> >
>
> *> y this much time for small data *
>
> >
>
> > 100k(10,lakhrows)—5 MB takes
>
> >
>
> >
>
> >
>
> > 20m39.625 seconds
>
> >
>
> >
>
> >
>
> > 48
>
> >
>
> >
>
> >
>
> > 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
> >
>
> >
>
> >
>
> > 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
> >
>
> >
>
> >
>
> >
>
> > 1millionrows---54 MB takes
>
> >
>
> >
>
> >
>
> > 198m42.375seconds
>
> >
>
> >
>
> >
>
> > 48
>
> >
>
> >
>
> >
>
> > 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
> >
>
> >
>
> >
>
> > 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > .
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > ./load_data script used:
>
> >
>
> >
>
> >
>
> >
>
> > #!/usr/local/bin/escript
>
> >
>
> >
>
> > main([Filename]) ->
>
> >
>
> >
>
> > {ok, Data} = file:read_file(Filename),
>
> >
>
> >
>
> > Lines = tl(re:split(Data, "\r?\n", [{return, binary},trim])),
>
> >
>
> >
>
> > lists:foreach(fun(L) -> LS = re:split(L, ","), format_and_insert(LS)
> end, Lines).
>
> >
>
> >
>
> >
>
> >
>
> > format_and_insert(Line) ->
>
> >
>
> >
>
> > JSON =
> io_lib:format("{\"id\":\"~s\",\"phonenumber\":~s,\"callednumber\":~s,\"starttime\":~s,\"endtime\":~s,\"status\":~s}",
> Line),
>
> >
>
> >
>
> > Command = io_lib:format("curl -X PUT
> http://10.232.5.169:8098/riak/CustomerCalls100k/~s -d '~s' -H
> 'content-type: application/json'", [hd(Line),JSON]),
>
> >
>
> >
>
> > io:format("Inserting: ~s~n", [hd(Line)]),
>
> >
>
> >
>
> > os:cmd(Command).
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > Thanks in advance!!waiting fr the reply…plz anyone help..struck
> u pwit hbulk loading…..and make me clear how riak splits the data and gets
> loaded on cluster
>
> >
>
> >
>
> > Thanks & regards
>
> >
>
> >
>
> > sangeetha
>
> >
>
> >
>
> >
>
> >
>
> > This e-mail and any files transmitted with it are for the sole use of
> the intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful.
>
> >
>
> >
>
>
>
>
>
> Regards
>
> Sangeetha
>
>
>
> *From:* Alexander Sicular [mailto:sicul...@gmail.com]
> *Sent:* Thursday, July 12, 2012 11:30 AM
> *To:* Pattabi Raman, Sangeetha (Cognizant)
> *Cc:* riak-users@lists.basho.com
> *Subject:* Re: reg. storageon Localfile system
>
>
>
> check your installations app.config file in the etc folder.
>
> @siculars
> http://siculars.posterous.com
>
> Sent from my rotary phone.
>
> On Jul 11, 2012 11:18 PM, wrote:
>
>
>
>
>
>
>
> Hi team,
>
>
>
> Iam very new to riak ,can anyone say on loading onto riak
> db 2 node cluster …where does it puts the data on the Local file
> system.like we see o
Riak Recap for June 29 - July 11
Morning, Afternoon, Evening to All OK. A lot to cover today. Grab a glass/mug of your favorite $DRINK and settle in. Happy Thursday. Mark twitter.com/pharkmillups --- Riak Recap for June 29 - July 11 1) Basho Hacker Russell Brown wrote a blog post about the new stats code that will be in Riak 1.2. * Read here ---> http://basho.com/blog/technical/2012/07/02/folsom-backed-stats-riak-1-2/ 2) Basho Hacker Bryce Kerley wrote some CRDT code called Hanover that will eventually target Riak. * Code here ---> https://github.com/bkerley/hanover 3) Data-Riak was bumped to v0.3 * Code here ---> http://search.cpan.org/~anelson/Data-Riak-0.3/ 4) Mathias Meyer wrote a great blog post all about his experiences selling the Riak Handbook. * Read here ---> http://www.paperplanes.de/2012/7/5/six-ish-months-of-ebook-sales-riak-handbook.html 5) Lots of great stuff coming via the team at Boundary: * Ray Jenkins wrote a blog post about how they use Scala with Riak ---> http://blog.boundary.com/2012/07/09/reusable-patterns-for-riak-in-scala/ * He also released their Scala/Riak code (or at least made its existence known) ---> https://github.com/rjenkins/riak-scala-dao * AND they announced a new round of funding. Congrats to the team! ---> http://blog.boundary.com/2012/07/10/big-day-for-boundary-15m-in-new-funding/ 6) GoGrid now has a Riak image built for their platform. They wrote a great blog post about how (and why) to use it. * Read here --> http://blog.gogrid.com/2012/07/09/create-a-basho-riak-cluster-on-gogrid/ 7) OJ Reeves delivered the long-awaited 5th installment of his Riak, Webmachine, and ErlyDTL blog series. This one did not disappoint. * Read here ---> http://buffered.io/posts/webmachine-erlydtl-and-riak-part-5/ 8) Basho Hacker Reid Draper gave a talk at Clojure/West about Knockbox and it's now online for viewing. * Watch here ---> http://www.infoq.com/presentations/Knockbox-an-Eventual-Consistency-Toolkit 9) We added a Riak/Couchbase comparison to the wiki. (h/t to Seth Benton and new-minted Basho Dev Eric Redmond for putting this together. ) * Read here ---> http://wiki.basho.com/Riak-Compared-to-Couchbase.html 10) mbk (from the Engine Yard crew) got started on some Ruby code called riak-shim that generates "sensible bucket names" based on config data. * Code here ---> https://github.com/mkb/riak-shim 11) We released the videos of Voxer's talk from the last Riak Meetup in San Francisco. * Watch here ---> http://basho.com/blog/technical/2012/06/27/Riak-at-Voxer/ 12) Sebastian Röbke shared a pretty amazing photo of what they do at they do at the Xing.com offices. * Check it out here ---> https://twitter.com/boosty/status/220997388754632704 13) We got started on a Riak Charm for Juju. * Details here on Launchpad ---> https://bugs.launchpad.net/charms/+bug/1022591 14) Some Q & A from Stack Overflow * Does Riak have any synchronization features to help sych. mobile and web apps? ---> http://stackoverflow.com/questions/11351866/does-riak-have-any-sychronization-features-to-help-sych-mobile-and-web-apps * Riak Failure Scenarios ---> http://stackoverflow.com/questions/11272276/riak-failure-scenarios * LinkWalk in RIAK ---> http://stackoverflow.com/questions/11384068/linkwalk-in-riak 15) We released the first of two talks from the last BashoChats meetup. This one of from Jeff Hodges and it's all about Go. * Watch here ---> http://basho.com/blog/technical/2012/07/10/escape-hatches-in-go-basho-chats/ # Contributions * Add crash_log to TestServer app_config ---> https://github.com/basho/riak-python-client/pull/137 * Throw an exception if the TestServer fails to start ---> https://github.com/basho/riak-python-client/pull/136 * Test server fixes ---> https://github.com/basho/riak-python-client/pull/132 * Allow commented out "-name" lines. ---> https://github.com/basho/riak/pull/169 * Flatten MaxScore string so that it is consistent in all interfaces. ---> https://github.com/basho/riak_search/issues/118 # Issues * Compression unit test fails with latest eleveldb ---> https://github.com/basho/eleveldb/issues/31 * Support the head parameter if pbc transport is used ---> https://github.com/basho/riak-python-client/issues/134 ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Range Loop Timeout Error
I forgot to reply-all. My response to Ken below for anyone else who is
curious.
-Z
On Thu, Jul 12, 2012 at 6:08 PM, Ryan Zezeski wrote:
> Ken,
>
> Not that this error msg is descriptive at all but it means that a range
> query is running and it timed out because merge index took too long to
> respond.
>
> 1. Is this happening for all queries?
>
> 2. If not, do you by any chance log the queries you run in your
> application? If so it would be nice to correlate a query to a timeout.
> Barring this, can you correlate the timeouts with anything?
>
> 3. What does iostat -x 1 look like during these timeouts? What about
> other things like vmstat?
>
> 4. Any other errors in the logs?
>
> -Z
>
> On Thu, Jul 12, 2012 at 12:39 PM, Ken Perkins wrote:
>
>> We keep getting the following error in production:
>>
>> @riak_search_backend:collect_info_response:55 range_loop timed out!
>> @riak_solr_searcher_wm:malformed_request:82 Unable to parse request:
>> {timeout,range_loop}
>>
>> We don't any feedback as to what the query was that generated the
>> problem, nor do we have a consistent repro.
>>
>> Has anyone else seen this error or can anyone help us with figuring what
>> this is and how to prevent it?
>>
>> Thanks!
>>
>> --Ken
>> clipboard, inc.
>> we're hiring!: www.clipboard.com/jobs
>>
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
About second parameter of riakc_pb_socket:mapred
Hi guys.
To test mapred_inputs() in the program, These parameters
([*{**bucket()
,
key()}
|
{bucket()
,
key(),
term()}] |
bucket()
| {index,
bucket(),
Index::binary(),
key()}
|
{index,bucket(),
Index::binary(),
StartKey::key(),
EndKey::key()
**} *) is ok.
The "Inputs" parameter({modfun, Module::atom(), Function::atom(), [term()]}
) {modfun, riak_search, mapred_search, [ <<"Bucket">>,
<<"SearchQuery">>]} is ok.
But myself function is can not work(Inputs = {modfun, trend_riak, test,
[<<"a">>, <<"b">>]}). The "trend_riak" module is added riak path.
There are errors in the following:
{error,<<"Error sending inputs: [{<<\"my\">>,<<\"bu\">>}]">>}
The trend_riak module code.
1 -module(trend_riak).
2
3 -export([test/3]).
4 [{<<"my">>, <<"bu">>}].
Where is wrong?
Best wish ,thank you.
Jason
mapred_inputs()
mapred_inputs() =
[{bucket()
,
key()}
|
{bucket()
,
key(),
term()}] | {modfun, Module::atom(), Function::atom(), [term()]} |
bucket()
|
{index,
bucket(),
Index::binary(),
key()}
|
{index,bucket(),
Index::binary(),
StartKey::key(),
EndKey::key()
}
mapred/3
mapred(Pid::pid(),
Inputs::mapred_inputs(),
Query::[mapred_queryterm()])
-> {ok,
mapred_result()}
| {error, {badqterm,
mapred_queryterm()}}
| {error, timeout} | {error, term()}
Equivalent to mapred(Inputs, Query,
default_timeout(mapred))
.
--
只为成功找方法，不为失败找理由
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
