I have an application that pushes a large amount of small updates (usually
below 1KB). Instead of reading massive numbers of keys, they are aggregated
into keys of roughly 1MB. The problem is as these keys near the 1MB limit the
throughput obviously drops both from a disk and network perspecti
I am currently forcing siblings for time series data. The maximum bucket sizes are very predictable due to the nature of the data. I originally used the get/update/set cycle, but as I approach the end of the interval, reading and writing 1MB+ objects at a high frequency kills network bandwidth. So
, how does it handle siblings since Level DB doesn't support them? Is there an intermediate object that stores links to all siblings?I think this information would allow people to better utilise siblings and understand why certain workloads can crash Riak or make it dreadfully slow.Thanks,
- Original Message -
From: "Sean Cribbs"
To: "Jason Campbell"
Sent: Saturday, 21 December, 2013 3:17:57 AM
Subject: Re: May allow_mult cause DoS?
> No, the behavior in LevelDB is no different than the behavior of any of our
> other backends, namel
- Original Message -
From: "Andrew Stone"
To: "Jason Campbell"
Cc: "Sean Cribbs" , "riak-users" ,
"Viable Nisei"
Sent: Saturday, 21 December, 2013 10:01:29 AM
Subject: Re: May allow_mult cause DoS?
> Think of an object with thous
- Original Message -
From: "Russell Brown"
To: "Jason Campbell"
Cc: "riak-users" , "Viable Nisei"
Sent: Saturday, 21 December, 2013 8:25:01 PM
Subject: Re: May allow_mult cause DoS?
>Hi,
>What is a "sibling with no parent object”?
testing purposes)
- Ensure ZFS has some RAM to play with
Hope this helps,
Jason Campbell
- Original Message -
From: "Jared Morrow"
To: ejh...@gmail.com
Cc: "riak-users"
Sent: Wednesday, 22 January, 2014 7:51:19 AM
Subject: Re: Performance Tuning in OmniOS
Oh I
. It will
work, but there is other software that will work much better.
I hope this helps,
Jason Campbell
- Original Message -
From: "Edgar Veiga"
To: "Russell Brown"
Cc: "riak-users"
Sent: Friday, 31 January, 2014 3:20:42 AM
Subject: Re: last_write_wins
on
has to take advantage of the extra consistency and reliability for it to make
sense.
Sorry again,
Jason Campbell
- Original Message -
From: "Edgar Veiga"
To: "Eric Redmond"
Cc: "Jason Campbell" , "riak-users"
, "Russell Brown"
Sent:
My understanding of what Elias wanted was a counter that simply stored the
minimum and maximum values it has ever reached, an optional reset would
probably be nice as well. It would be quite helpful when dealing with
statistics counters that can decrement.
Then again, I could be wrong.
-
Hi Bryce,
I have code that does something similar to this, and it works well.
In my case, the value is a JSON array, with a JSON object per event.
Siblings are easily resolved by merging the two arrays.
In Riak 2.0, sets using JSON-encoded strings would probably do this
automatically and more c
Riak CS is designed to be the same interface as Amazon S3. It can be run
anywhere you can run Riak.
Think of it as a way to run your own S3.
I think the hard limit on Riak objects is 50MB, but the recommended size is
less than 1MB per object. The main reason for that is latency. It's faster
Hi Guido
For Sydney hosting, I would recommend either Equinix or Global Switch. There
is also another NextDC datacenter in Melbourne.
For larger installations, deal with the datacenters themselves. If you are
looking for smaller installations, Servers Australia resells partial rack colo
spac
I like UUIDs for everything as well, although I expected compatibility issues with something. Base 64 encoding the binary value is a nice compromise for me, and takes 22 characters (if you drop the padding) instead of the usual 36 for the hyphenated hex format.It would still require re encoding all
I currently maintain my own indexes for some things, and use natural keys where
I can, but a question has been nagging me lately.
Why is key listing slow? Specifically, why is bitcask key listing slow?
One of the biggest issues with bitcask is all keys (including the bucket name
and some overh
-
From: "Kelly McLaughlin"
To: "Jason Campbell" , "riak-users"
Sent: Wednesday, 20 August, 2014 1:26:36 AM
Subject: Re: Bitcask Key Listing
Jason,
There are two aspects to to a key listing operation that make it
expensive relative to normal gets or puts.
T
Riak is binary storage, if there is an injection attack, it will be done
against the Ruby portion, not against Riak.
The example you listed gives a ParserError when trying to load it via JSON, so
I don’t understand how it could even get far enough in your code to reach Riak,
unless you are not
Exactly, SQL injection happens because people construct SQL statements
themselves and aren’t aware of correct escaping edge cases. Use a library to
convert between Ruby and JSON, and you’ll be fine on that front. The same
applies to SQL really. SQL injection is impossible if you pass Ruby obj
I have the same questions as Christopher.
Does this data need to change, or is it write-once?
What information do you have when querying?
- Will you already have timestamp and msg-id?
- If not, you may want to consider aggregating everything into a single key.
This is easier of the data isn’t
rare cases I expect to be able to
> simply run map-reduce jobs for custom queries.
>
> Given that, does my current model look reasonable?
>
> Thanks.
> AM
>
>
> On 2/21/15 6:40 PM, Jason Campbell wrote:
>> I have the same questions as Christopher.
>>
ies (Riak
MR jobs), and anything you know you will need should ideally be generated when
loading the data initially.
Hope this helps, let me know if you have any other questions.
Jason
> On 24 Feb 2015, at 05:24, AM wrote:
>
> On 2/22/15 6:16 PM, Jason Campbell wrote:
>> C
Hello,
I'm currently trying to debug slow YZ queries, and I've narrowed down the
issue, but not sure how to solve it.
First off, we have about 80 million records in Riak (and YZ), but the queries
return relatively few (a thousand or so at most). Our query times are anywhere
from 800ms to 1.5s
. member/founder of @papers_we_love | paperswelove.org
> twitter => @zeeshanlakhani
>
>> On Apr 21, 2015, at 1:06 AM, Jason Campbell wrote:
>>
>> Hello,
>>
>> I'm currently trying to debug slow YZ queries, and I've narrowed down the
>> issu
> No real workaround other than what you described or looking into
>>> config/fq-no-cache settings as mentioned in
>>> http://lucidworks.com/blog/advanced-filter-caching-in-solr/ and playing
>>> around with those.
>>>
>>> Riak is now at 2.1.0. I hope that one of
g 0 rows
though, and also takes almost 2 seconds.
Now I'm not sure these last 2 queries are actually required to complete the
yokozuna request, but they are run every time I make a new request. Although
each of these queries takes over 1.8 seconds, the yokozuna request completed in
1.2 se
After some comparisons with tcpdump, I think the strange results are related to
a too aggressive redirect rule.
Please ignore. Sorry for the confusion.
> On 22 Apr 2015, at 12:35, Jason Campbell wrote:
>
> Sorry, still running into issues.
>
> So I've disabled cache on
I've just done a backup and restore of our production Riak cluster, and
Yokozuna has dropped from around 125 million records to 25million. Obviously
the IPs have changed, and although the Riak cluster is stable, I'm not sure
Solr handled the transition as nicely.
Is there a way to force Solr t
opped their Solr indexes.
Is this expected? If so, it should really be in the docs, and there should be
another way to restore a cluster keeping Solr intact.
Also, is there a way to rebuild a Solr index?
Thanks,
Jason
> On 24 Apr 2015, at 15:16, Jason Campbell wrote:
>
> I've jus
> programmer |
> software engineer at @basho |
> org. member/founder of @papers_we_love | paperswelove.org
> twitter => @zeeshanlakhani
>
>> On Apr 24, 2015, at 1:34 AM, Jason Campbell wrote:
>>
>> I think I figured it out.
>>
>> I followed this
rce-replaced? Unless someone else wants to chime in, I’ll gather
> more info on what occurred from the reip vs the force-replace.
>
> Zeeshan Lakhani
> programmer |
> software engineer at @basho |
> org. member/founder of @papers_we_love | paperswelove.org
> twitter =&g
start riak node works fine.
I still have the cluster available if there is anything else you want me to
test.
Jason Campbell
> On 1 May 2015, at 22:51, Matthew Brender wrote:
>
> Hi Jason,
>
> Did you and Zeeshan have time to follow up on your experiments? I'm cur
31 matches
Mail list logo
