One thing I am doing different than the benchmark is creating a lot of buckets…
Paul Ingalls Founder & CEO Fanzo p...@fanzo.me @paulingalls http://www.linkedin.com/in/paulingalls On Aug 1, 2013, at 11:52 PM, Paul Ingalls <p...@fanzo.me> wrote: > The objects are either tweet jsons, so between 1-2k, or simple strings with > some links. > > I am using secondary indexes with timestamps so I can do range queries. > Would this significantly impact performance? I am not indexing for search... > > Here are some more bench results. Obviously using the azure load balancer is > a bad idea. Fortunately I'm not. > > This was run from my laptop using 8 concurrent connections and the pb client, > using the load balancer in azure. Lots of errors. > > <summary.png> > > > > this was run on a vm in the same cloud service using 8 concurrent workers and > the pb client, but clustered directly (i input all the IP addresses to > riakc_pb_ips). Much better ops. So it must be something with my usage that > isn't getting captured by the benchmark. > > <summary.png> > > Paul Ingalls > Founder & CEO Fanzo > p...@fanzo.me > @paulingalls > http://www.linkedin.com/in/paulingalls > > > > On Aug 1, 2013, at 10:46 PM, Jeremy Ong <jer...@quarkgames.com> wrote: > >> What is the average size of one of your objects? Are you indexing? >> >> >> On Thu, Aug 1, 2013 at 10:23 PM, Paul Ingalls <p...@fanzo.me> wrote: >> Running them on separate machines. >> >> I just ran basho bench against the service from my laptop. here is the >> result. in this case I am just hitting one node. will try it again shortly >> using a load balancer to hit all the nodes. >> >> <summary.png> >> >> Paul Ingalls >> Founder & CEO Fanzo >> p...@fanzo.me >> @paulingalls >> http://www.linkedin.com/in/paulingalls >> >> >> >> On Aug 1, 2013, at 8:57 PM, Jared Morrow <ja...@basho.com> wrote: >> >>> Also, are you generating the load from the same VMs that Riak is running on >>> or do you have separate machines generating load? >>> >>> >>> >>> On Thursday, August 1, 2013, Jeremy Ong wrote: >>> What erlang version did you build with? How are you load balancing >>> between the nodes? What kind of disks are you using? >>> >>> On Thu, Aug 1, 2013 at 7:53 PM, Paul Ingalls <p...@fanzo.me> wrote: >>> > FYI, 2 more nodes died with the end of the last test. Storm, which I'm >>> > using to put data in, kills the topology a bit abruptly, perhaps the nodes >>> > don't like a client going away like that? >>> > >>> > log from one of the nodes: >>> > >>> > 2013-08-02 02:27:23 =ERROR REPORT==== >>> > Error in process <0.4959.0> on node 'riak@riak004' with exit value: >>> > {badarg,[{riak_core_stat,vnodeq_len,1,[{file,"src/riak_core_stat.erl"},{line,181}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[{file,"src/riak_core_stat.erl"},{line,172}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[... >>> > >>> > 2013-08-02 02:27:33 =ERROR REPORT==== >>> > Error in process <0.5055.0> on node 'riak@riak004' with exit value: >>> > {badarg,[{riak_core_stat,vnodeq_len,1,[{file,"src/riak_core_stat.erl"},{line,181}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[{file,"src/riak_core_stat.erl"},{line,172}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[... >>> > >>> > 2013-08-02 02:27:51 =ERROR REPORT==== >>> > Error in process <0.5228.0> on node 'riak@riak004' with exit value: >>> > {badarg,[{riak_core_stat,vnodeq_len,1,[{file,"src/riak_core_stat.erl"},{line,181}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[{file,"src/riak_core_stat.erl"},{line,172}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[... >>> > >>> > and the log from the other node: >>> > >>> > 2013-08-02 00:09:39 =ERROR REPORT==== >>> > Error in process <0.4952.0> on node 'riak@riak007' with exit value: >>> > {badarg,[{riak_core_stat,vnodeq_len,1,[{file,"src/riak_core_stat.erl"},{line,181}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[{file,"src/riak_core_stat.erl"},{line,172}]},{riak_core_stat,'-vnodeq_stats/0-lc$^0/1-0-',1,[... >>> > >>> > 2013-08-02 00:09:44 =ERROR REPORT==== >>> > ** State machine <0.2368.0> terminating >>> > ** Last event in was unregistered >>> > ** When State == active >>> > ** Data == >>> > {state,114179815416476790484662877555959610910619729920,riak_kv_vnode,{deleted,{state,114179815416476790484662877555959610910619729920,riak_kv_eleveldb_backend,{state,<<>>,"/mnt/datadrive/riak/data/leveldb/114179815416476790484662877555959610910619729920",[{create_if_missing,true},{max_open_files,128},{use_bloomfilter,true},{write_buffer_size,58858594}],[{add_paths,[]},{allow_strfun,false},{anti_entropy,{on,[]}},{anti_entropy_build_limit,{1,3600000}},{anti_entropy_concurrency,2},{anti_entropy_data_dir,"/mnt/datadrive/riak/data/anti_entropy"},{anti_entropy_expire,604800000},{anti_entropy_leveldb_opts,[{write_buffer_size,4194304},{max_open_files,20}]},{anti_entropy_tick,15000},{create_if_missing,true},{data_root,"/mnt/datadrive/riak/data/leveldb"},{fsm_limit,50000},{hook_js_vm_count,2},{http_url_encoding,on},{included_applications,[]},{js_max_vm_mem,8},{js_thread_stack,16},{legacy_stats,true},{listkeys_backpressure,true},{map_js_vm_count,8},{mapred_2i_pipe,true},{mapred_name,"mapred"},{max_open_files,128},{object_format,v1},{reduce_js_vm_count,6},{stats_urlpath,"stats"},{storage_backend,riak_kv_eleveldb_backend},{use_bloomfilter,true},{vnode_vclocks,true},{write_buffer_size,58858594}],[],[],[{fill_cache,false}],true,false},{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},undefined,3000,1000,100,100,true,true,undefined}},riak@riak003,none,undefined,undefined,undefined,{pool,riak_kv_worker,10,[]},undefined,107615} >>> > ** Reason for termination = >>> > ** >>> > {badarg,[{eleveldb,close,[<<>>],[]},{riak_kv_eleveldb_backend,stop,1,[{file,"src/riak_kv_eleveldb_backend.erl"},{line,149}]},{riak_kv_vnode,terminate,2,[{file,"src/riak_kv_vnode.erl"},{line,836}]},{riak_core_vnode,terminate,3,[{file,"src/riak_core_vnode.erl"},{line,847}]},{gen_fsm,terminate,7,[{file,"gen_fsm.erl"},{line,586}]},{proc_lib,init_p_do_apply,3,[{file," >> >> >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
