Hi Jon,
Just to let you know, I saw your email and have planned over the last 2
days to sit down and gather this info for you, but other issues keep
arising to keep me from getting the time to dedicate to it.
I'm going to try to get some time at home tonight, and get back to you with
this info.
Thanks for the advice in advance
On Sat, Aug 8, 2015 at 1:45 PM, Jon Meredith <jmered...@basho.com> wrote:
> Hi Sean,
>
> Thank you for the info. I had exchanged a couple of emails with Russell
> Brown (who implemented the maps/sets) and he was interested how
> enabled/disabling compression affected your results. The shipping
> implementation keeps a fair bit of metadata around which we compress to try
> and improve size, but it may be impacting performance.
>
> If you have the time, you could try adding this to your advanced.config
> file and restarting,
> [{riak_dt, [{binary_compression, false}]}]
>
> Or on a running cluster you could execute
>
> (dev1@127.0.0.1)2> rpc:multicall(application, set_env, [riak_dt,
> binary_compression, false]).
>
> {[ok],[]}
>
> To revert back to the default behavior
>
> (dev1@127.0.0.1)2> rpc:multicall(application, set_env, [riak_dt,
> binary_compression, 1]).
>
> {[ok],[]}
>
>
> The compression uses gzip - and is currently set for least compression
> time. Increasing from 1 through 9 will increase CPU load and could provide
> a counter example in the other direction.
>
> If you find anything interesting, please share, we'd really like the
> feedback on real world data.
>
> Jon
>
> On Thu, Aug 6, 2015 at 8:41 AM Sean Kelly <sean.ke...@tapjoy.com> wrote:
>
>> Because it's important to note, the version of Riak is 2.0.5.
>>
>> AAE was actually the first thing I checked - it was on, but turning it
>> off had no impact.
>>
>> Interestingly enough, I was able to pinpoint the source of the higher
>> than expected CPU to a service that was frequently polling a set of map and
>> set CRDTs from Riak once every second (which does not really seem all that
>> frequent) to keep up with any changes that may have occurred since the last
>> time it cached the information in process.
>>
>> I'm trying to get some time to look more into if we're doing something
>> dumb in our service (always possible) or if there are settings for these
>> CRDTs that we're not making proper use of in Riak itself.
>>
>> For now, I've tuned down the polling frequency for the test environment
>> (which for some reason was higher than the production environment), and the
>> problem appears to have gone away. We didn't seem to see any noticeable
>> rise or fall in the CPU utilization of the service itself, just in Riak
>>
>> If anyone has any insights into poor performance with the map / set CRDT
>> types (maps and sets, and maps containing sets), any input would be welcome
>> to help hunt down why that seemed to be such a issue.
>>
>> On Thu, Aug 6, 2015 at 8:19 AM, Matthew Von-Maszewski <matth...@basho.com
>> > wrote:
>>
>>> Two ideas about lots of cpu with nothing happening:
>>>
>>> - is Riak entropy setting "active". Weekly it will rebuild hash trees
>>>
>>> - does this activity occur after a handoff? Sometimes leveldb needs
>>> time to catch up afterward in current releases (fix coming soon).
>>>
>>> Matthew
>>>
>>> Sent from my iPad
>>>
>>> > On Aug 5, 2015, at 7:16 PM, Matthew Brender <mbren...@basho.com>
>>> wrote:
>>> >
>>> > Hey all, I'm practicing what I preach and asking a question to others
>>> > here. A community member pinged me with a couple questions:
>>> >
>>> > "I'm looking into an issue with Riak on our test environment (we
>>> > basically let devs spin up a copy of our prod stack on a giant box in
>>> > ec2).
>>> >
>>> > We've gotten some complaints that riak seems to be using a lot of CPU,
>>> > even when "nothing" is happening (something is always happening with
>>> > the # of apps and services on the box, even when no one is poking it).
>>> >
>>> > Is there a way to turn on an "access" log, so I could see what
>>> > requests are being made against riak, to try and correlate with? It
>>> > would have to be able to tell me about pbuffs usage, not just http."
>>> >
>>> > Could anyone help with these questions. I see a couple separate ones:
>>> >
>>> > 1. How to monitor what is causing higher CPU usage on a Riak cluster
>>> > 2. How to view an access log of protobuff and http requests, or just
>>> > the *most helpful* counters for them
>>> >
>>> > I know a ton of statistics are visible [1] through riak-admin, but I'd
>>> > like to hear what others recommend in particular.
>>> >
>>> > [1]
>>> http://docs.basho.com/riak/latest/ops/running/stats-and-monitoring/
>>> >
>>> > Thanks,
>>> >
>>> > Matt Brender | Developer Advocacy Lead
>>> > Basho Technologies
>>> > t: @mjbrender
>>> >
>>> > _______________________________________________
>>> > riak-users mailing list
>>> > riak-users@lists.basho.com
>>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
