I've tested this on the latest Kraken RC (installed on RHEL from the el7
repo) and it seemed promising at first but the OSDs still gradually consume
all available memory until OOM killed, they just do so slower. It takes
them a couple of hours to go from 500M each to >2G each. After they're
restart
Just to update, this is still an issue as of the latest Git commit (
64bcf92e87f9fbb3045de49b7deb53aca1989123).
On Fri, Nov 11, 2016 at 1:31 PM, bobobo1...@gmail.com
wrote:
> Here's another: http://termbin.com/smnm
>
> On Fri, Nov 11, 2016 at 1:28 PM, Sage Weil wrote:
> > On Fri, 11 Nov 2016, b
Here's another: http://termbin.com/smnm
On Fri, Nov 11, 2016 at 1:28 PM, Sage Weil wrote:
> On Fri, 11 Nov 2016, bobobo1...@gmail.com wrote:
>> Any more data needed?
>>
>> On Wed, Nov 9, 2016 at 9:29 AM, bobobo1...@gmail.com
>> wrote:
>> > Here it is after running overnight (~9h): http://ix.io/1
On Fri, 11 Nov 2016, bobobo1...@gmail.com wrote:
> Any more data needed?
>
> On Wed, Nov 9, 2016 at 9:29 AM, bobobo1...@gmail.com
> wrote:
> > Here it is after running overnight (~9h): http://ix.io/1DNi
I'm getting a 500 on that URL...
sage
> >
> > On Tue, Nov 8, 2016 at 11:00 PM, bobobo1...@
Any more data needed?
On Wed, Nov 9, 2016 at 9:29 AM, bobobo1...@gmail.com
wrote:
> Here it is after running overnight (~9h): http://ix.io/1DNi
>
> On Tue, Nov 8, 2016 at 11:00 PM, bobobo1...@gmail.com
> wrote:
>> Ah, I was actually mistaken. After running without Valgrind, it seems
>> I just es
Here it is after running overnight (~9h): http://ix.io/1DNi
On Tue, Nov 8, 2016 at 11:00 PM, bobobo1...@gmail.com
wrote:
> Ah, I was actually mistaken. After running without Valgrind, it seems
> I just estimated how slowed down it was. I'll leave it to run
> overnight as suggested.
>
> On Tue, No
Ah, I was actually mistaken. After running without Valgrind, it seems
I just estimated how slowed down it was. I'll leave it to run
overnight as suggested.
On Tue, Nov 8, 2016 at 10:44 PM, bobobo1...@gmail.com
wrote:
> Okay, I left it for 3h and it seemed to actually stabilise at around
> 2.3G: h
Okay, I left it for 3h and it seemed to actually stabilise at around
2.3G: http://ix.io/1DEK
This was only after disabling other services on the system however.
Generally this much RAM isn't available to Ceph (hence the OOM
previously).
On Tue, Nov 8, 2016 at 9:00 AM, Mark Nelson wrote:
> It sho
It should be running much slower through valgrind so probably won't
accumulate very quickly. That was the problem with the earlier trace,
there wasn't enough memory used yet to really get us out of the weeds.
If it's still accumulating quickly, try to wait until the OSD is up to
4+GB RSS if yo
Unfortunately I don't think overnight is possible. The OOM will kill it in
hours, if not minutes. Will the output be preserved/usable if the process
is uncleanly terminated?
On 8 Nov 2016 08:33, "Mark Nelson" wrote:
> Heya,
>
> Sorry got distracted with other stuff yesterday. Any chance you cou
Heya,
Sorry got distracted with other stuff yesterday. Any chance you could
run this for longer? It's tough to tell what's going on from this run
unfortunately. Maybe overnight if possible.
Thanks!
Mark
On 11/08/2016 01:10 AM, bobobo1...@gmail.com wrote:
Just bumping this and CCing dir
Just bumping this and CCing directly since I foolishly broke the threading
on my reply.
On 4 Nov. 2016 8:40 pm, "bobobo1...@gmail.com" wrote:
> > Then you can view the output data with ms_print or with
> massif-visualizer. This may help narrow down where in the code we are
> using the memory.
>
> Then you can view the output data with ms_print or with massif-visualizer.
> This may help narrow down where in the code we are using the memory.
Done! I've dumped the output from ms_print here: http://ix.io/1CrS
It seems most of the memory comes from here:
92.78% (998,248,799B) (heap alloca
Hi Lucas!
On 11/02/2016 09:07 PM, bobobo1...@gmail.com wrote:
I'm running Kraken built from Git right now and I've found that my OSDs
eat as much memory as they can before they're killed by OOM. I
understand that Bluestore is experimental but thought the fact that it
does this should be known.
I'm running Kraken built from Git right now and I've found that my OSDs eat
as much memory as they can before they're killed by OOM. I understand that
Bluestore is experimental but thought the fact that it does this should be
known.
My setup:
- Xeon D-1540, 32GB DDR4 ECC RAM
- Arch Linux
- Single
15 matches
Mail list logo
