I get this error when I increase the number of threads accessing a single
connection. Something related to getting unexpected number of bytes on the
connection.
On Wednesday, June 27, 2012, Reid Draper wrote:
>
> On Jun 27, 2012, at 8:41 AM, Yousuf Fauzan wrote:
>
> So I created an array of clien
LevelDB is a nice option with a key space that will not fit in memory. Whether
or not bitcask will work for you depends on total memory capacity of the
cluster and N value. Recommend using the bitcask capacity planner to see if it
is a suitable backed for your hardware+data combination.
http://
Hi all,
I did a quick computation, and figured that I need to be able to manage a
minimum 5 billion keys roughly 80bytes each. Since Bitcask needs all keys in
memory, that's a lot of memory. How does it handle swapping? I typically only
need a portion of those keys available at a given time,
On Jun 26, 2012, at 9:37 AM, Jeremiah Peschka
wrote:
> Scale out - zerg rush your data.
>
> Many smaller systems means that, in theory, you're going to be affected less
> by poor performance of any single instance.
>
> When you're building out instances in EC2 always remember that the larges
We're running smoothly behind nginx in a round robin config. One thing
to remember is eventual consistency... So, if you have two very quick
serial operations, one dependent on the results of the other, be sure
to make sure both requests are hitting the same node. Also, it was
necessary to bypass t
On Jun 27, 2012, at 8:41 AM, Yousuf Fauzan wrote:
> So I created an array of clients using the following code
>
> Clients = [riak.RiakClient(e, port=8087,
> transport_class=riak.RiakPbcTransport) for e in NODES]
Sounds like you're bringing your concurrency back down to 3 (because you have
thr
So I created an array of clients using the following code
Clients = [riak.RiakClient(e, port=8087,
transport_class=riak.RiakPbcTransport) for e in NODES]
After this I assigned each thread a particular id ranging from 0 to Number
of Nodes
So each thread now communicates with a single node.
Even
Oh! I think that may be an issue with my code then.
Let me make some changes and get back to you.
On Wed, Jun 27, 2012 at 5:25 PM, Reid Draper wrote:
>
> On Jun 27, 2012, at 7:48 AM, Yousuf Fauzan wrote:
>
> This is great.
>
> I was loading data using Python. My code would spawn 10 threads and
Hi Jason,
Riak's model is that of a simple key-value store, you typically can only
look things up by key. You will have to structure your data and keys
differently so they can satisfy the access pattern you're looking for. For
example, maybe you have a key which is just {node, itemid} (although ma
On Jun 27, 2012, at 7:48 AM, Yousuf Fauzan wrote:
> This is great.
>
> I was loading data using Python. My code would spawn 10 threads and put data
> in a queue. All threads would read data from this queue.
> However, all threads were hitting the same server/load balancer.
>
> I tried a differ
This is great.
I was loading data using Python. My code would spawn 10 threads and put
data in a queue. All threads would read data from this queue.
However, all threads were hitting the same server/load balancer.
I tried a different setup too. Where I spawned processes with each process
having i
On 27 Jun 2012, at 12:36, Yousuf Fauzan wrote:
> So I changed concurrency to 10 and put all the IPs of the nodes in basho
> bench config.
> Throughput is now around 1500.
>
I guess you can now try 5 or 15 concurrent workers and see which is optimal for
that set up to get a good feel for the s
So I changed concurrency to 10 and put all the IPs of the nodes in basho
bench config.
Throughput is now around 1500.
On Wed, Jun 27, 2012 at 4:40 PM, Russell Brown wrote:
>
> On 27 Jun 2012, at 12:09, Yousuf Fauzan wrote:
>
> I used examples/riakc_pb.config
>
> {mode, max}.
>
> {duration, 10}.
On 27 Jun 2012, at 12:09, Yousuf Fauzan wrote:
> I used examples/riakc_pb.config
>
> {mode, max}.
>
> {duration, 10}.
>
> {concurrent, 1}.
Try upping this. On my local 3 node cluster with 8gb ram and an old, cheap quad
core per box I'd set concurrency to 10 workers.
>
> {driver, basho_benc
I used examples/riakc_pb.config
{mode, max}.
{duration, 10}.
{concurrent, 1}.
{driver, basho_bench_driver_riakc_pb}.
{key_generator, {int_to_bin, {uniform_int, 1}}}.
{value_generator, {fixed_bin, 1}}.
{riakc_pb_ips, [{}]}.
{riakc_pb_replies, 1}.
{operations, [{get, 1}, {update, 1}]
On 27 Jun 2012, at 12:05, Yousuf Fauzan wrote:
> I did use basho bench on my clusters. It should throughput of around 150
Could you share the config you used, please?
>
> On Wed, Jun 27, 2012 at 4:24 PM, Russell Brown wrote:
>
> On 27 Jun 2012, at 11:50, Yousuf Fauzan wrote:
>
>> Its not ab
I did use basho bench on my clusters. It should throughput of around 150
On Wed, Jun 27, 2012 at 4:24 PM, Russell Brown wrote:
>
> On 27 Jun 2012, at 11:50, Yousuf Fauzan wrote:
>
> Its not about the difference in throughput in the two approaches I took.
> Rather, the issue is that even 200 write
On 27 Jun 2012, at 11:50, Yousuf Fauzan wrote:
> Its not about the difference in throughput in the two approaches I took.
> Rather, the issue is that even 200 writes/sec is a bit on the lower side.
> I could be doing something wrong with the configuration because people are
> reporting throughp
Its not about the difference in throughput in the two approaches I took.
Rather, the issue is that even 200 writes/sec is a bit on the lower side.
I could be doing something wrong with the configuration because people are
reporting throughputs of 2-3k ops/sec
If anyone here could guide me in setti
On Jun 27, 2012, at 5:13 AM, Yousuf Fauzan wrote:
> Hi,
>
> I setup a 3 machine riak SM cluster. Each machine used 4GB Ram and riak
> OpenSource SmartMachine Image.
>
> Afterwards I tried loading data by following two methods
> 1. Bash script
> #!/bin/bash
> echo $(date)
> for (( c=1; c<=1000;
Hi,
I setup a 3 machine riak SM cluster. Each machine used 4GB Ram and riak
OpenSource SmartMachine Image.
Afterwards I tried loading data by following two methods
1. Bash script
#!/bin/bash
echo $(date)
for (( c=1; c<=1000; c++ ))
do
curl -s -d 'this is a test' -H "Content-Type: text/plain"
http
21 matches
Mail list logo
