Thanks Sicular,
Supervisor riak_core_vnode_sup had child undefined started with
riak_core_vnode:start_link() at undefined exit with reason
{timeout,{gen_server,call,[<0.1164.0>,stop]}} in context shutdown_error...
trying to insert csv data of 434 MB onto 2 node riak cluster....iam getting the
following error ..could you please share the reason for the same
...Thanks In advance . attached app.config for your reference .
This is my another quest need your valuable suggestion please
Dear team,
>
>
>
>
> FYI:we have a 4 quad core intel processor on each server on 2 node cluster
> with more than 1 TB of storage
>
>
> I Ihave constructed the 2 node physical machine riak cluster with n_val 2 and
> my app.config ,vm.args are attached for your reference..
>
>
>
>
> Please tell me where the bulk inserted data onto riak db gets stored on Local
> file system...its taking huge time to load small size itself...how to tune it
> to perform to large scale since we deal wit hbigdata of in few hungred
> GB's?????????????????
>
>
>
>
> Cmd used:time ./load_data1m Customercalls1m.csv
>
>
>
>
> ./load_data100m CustomerCalls100m(got this error so changed default config of
> app.config...from 8 MB to 3072 MB
>
>
> escript: exception error: no match of right hand side value {error,enoent}
>
>
>
>
>
>
> size
>
>
>
> Load time
>
>
>
> No of mappersonapp.config
>
>
>
> Js-max-vm-mem on app.config
>
>
>
> Js-thread-stack
>
>
> y this much time for small data
>
> 100k(10,lakhrows)-5 MB takes
>
>
>
> 20m39.625 seconds
>
>
>
> 48
>
>
>
> 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
>
>
> 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
>
>
>
> 1millionrows---54 MB takes
>
>
>
> 198m42.375seconds
>
>
>
> 48
>
>
>
> 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
>
>
> 3 GB 3072MB(changedfromdefault 8MB)since i/p data is large)
>
>
>
>
>
>
> .
>
>
>
>
>
>
> ./load_data script used:
>
>
>
>
> #!/usr/local/bin/escript
>
>
> main([Filename]) ->
>
>
> {ok, Data} = file:read_file(Filename),
>
>
> Lines = tl(re:split(Data, "\r?\n", [{return, binary},trim])),
>
>
> lists:foreach(fun(L) -> LS = re:split(L, ","), format_and_insert(LS) end,
> Lines).
>
>
>
>
> format_and_insert(Line) ->
>
>
> JSON =
> io_lib:format("{\"id\":\"~s\",\"phonenumber\":~s,\"callednumber\":~s,\"starttime\":~s,\"endtime\":~s,\"status\":~s}",
> Line),
>
>
> Command = io_lib:format("curl -X PUT
> http://10.232.5.169:8098/riak/CustomerCalls100k/~s -d '~s' -H 'content-type:
> application/json'", [hd(Line),JSON]),
>
>
> io:format("Inserting: ~s~n", [hd(Line)]),
>
>
> os:cmd(Command).
>
>
>
>
>
>
>
>
> Thanks in advance!!!!!!!!!!waiting fr the reply...plz anyone help..struck u
> pwit hbulk loading.....and make me clear how riak splits the data and gets
> loaded on cluster
>
>
> Thanks & regards
>
>
> sangeetha
>
>
>
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful.
>
>
Regards
Sangeetha
From: Alexander Sicular [mailto:sicul...@gmail.com]
Sent: Thursday, July 12, 2012 11:30 AM
To: Pattabi Raman, Sangeetha (Cognizant)
Cc: riak-users@lists.basho.com
Subject: Re: reg. storageon Localfile system
check your installations app.config file in the etc folder.
@siculars
http://siculars.posterous.com
Sent from my rotary phone.
On Jul 11, 2012 11:18 PM,
<sangeetha.pattabiram...@cognizant.com<mailto:sangeetha.pattabiram...@cognizant.com>>
wrote:
Hi team,
Iam very new to riak ,can anyone say on loading onto riak db 2
node cluster ...where does it puts the data on the Local file system.like we
see on autosharding of mongodb or hadoop-hdfs over shards in the cluster mode
....as in which of riak........................
FYI:
I use curl in loading the same
-module(time).
-export([starttime/0]).
-export([main/1]).
-export([format_and_insert/1]).
-export([endtime/0]).
starttime() ->
starttime = {{Year,Month,Day},{Hour,Min,Sec}} = erlang:localtime().
main([Filename]) ->
{ok, Data} = file:read_file(Filename),
Lines = tl(re:split(Data, "\r?\n", [{return, binary},trim])),
lists:foreach(fun(L) -> LS = re:split(L, ","), format_and_insert(LS) end,
Lines).
format_and_insert(Line) ->
JSON =
io_lib:format("{\"id\":\"~s\",\"phonenumber\":~s,\"callednumber\":~s,\"starttime\":~s,\"endtime\":~s,\"status\":~s}",
Line),
Command = io_lib:format("curl -X PUT http://10.232.5.169:8098/riak/goog1/~s
-d '~s' -H 'content-type: application/json'", [hd(Line),JSON]),
io:format("Inserting: ~s~n", [hd(Line)]),
os:cmd(Command).
Thanks & regards
sangeetha
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
If you are not the intended recipient(s), please reply to the sender and
destroy all copies of the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email,
and/or any action taken in reliance on the contents of this e-mail is strictly
prohibited and may be unlawful.
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com<mailto:riak-users@lists.basho.com>
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
If you are not the intended recipient(s), please reply to the sender and
destroy all copies of the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email,
and/or any action taken in reliance on the contents of this e-mail is strictly
prohibited and may be unlawful.
app.config
Description: app.config
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
