Re: KVM memory overprovision breaking me

2014-01-28 Thread Marcus Sorensen
just send an email requesting that it be cherry-picked, with the reasoning. I don't feel strongly that it's a bug, but if you really want it in 4.3 it's worth a shot to make your case. On Tue, Jan 28, 2014 at 10:57 AM, Nux! wrote: > On 28.01.2014 17:49, Marcus Sorensen wrote: >> >> Commit 1530c16

Re: KVM memory overprovision breaking me

2014-01-28 Thread Nux!
On 28.01.2014 17:49, Marcus Sorensen wrote: Commit 1530c162e58abd8e572a89d2a2706de00bd49e26 in branch refs/heads/master from [~mlsorensen] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=1530c16 ] CLOUDSTACK-5968 create vm.memballoon.disable agent parameter How do I lobby for it?

Re: KVM memory overprovision breaking me

2014-01-28 Thread Marcus Sorensen
Commit 1530c162e58abd8e572a89d2a2706de00bd49e26 in branch refs/heads/master from [~mlsorensen] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=1530c16 ] CLOUDSTACK-5968 create vm.memballoon.disable agent parameter This is just a tuneable that you have to explicitly set, otherwise eve

Re: KVM memory overprovision breaking me

2014-01-28 Thread Marcus Sorensen
They don't seem to be on a per-vm basis. It seems to be service offering size divided by cluster overprovision setting. I do get what is trying to be accomplished, the best is to have both KSM and balloon. Ideally the balloon settings would all be maxed by default, and then if KSM is insufficient

Re: KVM memory overprovision breaking me

2014-01-28 Thread Nux!
On 28.01.2014 09:56, Bharat Kumar wrote: Hi, The calculations when overcommitting are on a per VM basis. I am not sure if these will be valid when using KSM. IMO KSM is applicable to a set of VMs not a single VM. so how do we keep track of how much free memory is actually available on the host.

Re: KVM memory overprovision breaking me

2014-01-28 Thread Bharat Kumar
Hi, The calculations when overcommitting are on a per VM basis. I am not sure if these will be valid when using KSM. IMO KSM is applicable to a set of VMs not a single VM. so how do we keep track of how much free memory is actually available on the host. This was not problem in case of balloo

Re: KVM memory overprovision breaking me

2014-01-28 Thread Nux!
On 28.01.2014 06:57, Marcus Sorensen wrote: Actually, does anyone have objections to adding an agent.properties tunable to disable balloon? That would allow anyone who wants to overcommit to do so, without messing with the existing FS, and the calculations for allocation at least are all accurate

Re: KVM memory overprovision breaking me

2014-01-28 Thread Bharat Kumar
Hi Marcus, if you want to get what is assigned in the service offering then we can simply disable the overcommit feature. Thanks, Bharat. On 28-Jan-2014, at 12:37 pm, Marcus Sorensen wrote: > They're both very specific. As mentioned, ballooning only works if > you're basically just making vm

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
They're both very specific. As mentioned, ballooning only works if you're basically just making vms for yourself. And even then you have to add your own script to make it work, hence my suggestion to enable it via agent.properties. Again, my main concern is that it messes with the service offering

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
One last comment... since you mention that it doesn't actually work for KVM without an admin writing or adding a script, maybe it would make more sense for the agent.properties tunable to *enable* balloon, basically saying that they have done that scripting portion and want to enable the balloon fo

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
Actually, does anyone have objections to adding an agent.properties tunable to disable balloon? That would allow anyone who wants to overcommit to do so, without messing with the existing FS, and the calculations for allocation at least are all accurate. And I won't have to maintain another local p

Re: KVM memory overprovision breaking me

2014-01-27 Thread Bharat Kumar
Hi Marcus, KSM or memory de-duplication on KVM can only be used when the memory pages are identical. IMO this is a huge constraint which is true only for specific use cases. using KSM to implement overprovisioning will limit this feature to a specific use case and hence memory ballooning was

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
Yeah, I'm a little disappointed that the functional spec doesn't really address memory deduplication, which is the real version of overcommit, IMO. Since it looks like the feature is already fully implemented, I'm not sure I have much of a leg to stand on in trying to change it. I'll just patch it

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
I should add that I wouldn't expect scaling to work as a combination of a cluster setting and a service offering, I'd expect it to be handled entirely by service offerings. The former just becomes confusing... I define my service offerings with a certain level of resources, and then that's divided

Re: KVM memory overprovision breaking me

2014-01-27 Thread Bharat Kumar
Hi Marcus, in case of KVM the guest memory is not dynamically adjusted by hypervisor, this is a hypervisor limitation. we have documented this in the FS in prerequisites for KVM. https://cwiki.apache.org/confluence/display/CLOUDSTACK/CPU+and+RAM+Overcommit One way to make this work automatical

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
Yeah, that's not overprovisioning, though. That's scaling. The way it's implemented now is just ... provisioning. It allocates exactly what's available at the host. Also, the vm in your example can't go to 4GB. There is nothing that changes it's 'currentMemory' setting. Without a scaling feature i

Re: KVM memory overprovision breaking me

2014-01-27 Thread Harikrishna Patnala
Hi, I think the way it was done is to guarantee minimum memory to that VM and upon demand it can get upto the memory defined in service offering. Say a vm with service offering 4Gb is deployed with overprovisioing factor 2, we guarantee that vm should get minimum of 2GB (4GB/2) and if that VM is

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
Its an easy fix on the KVM side, just waiting to hear any objections. On Jan 27, 2014 6:11 PM, "Nux!" wrote: > On 28.01.2014 00:49, Marcus Sorensen wrote: > >> So... I tried to use memory overcommit on KVM this week, and it blew >> up in my face. Apparently it's configured such that if I have a >

Re: KVM memory overprovision breaking me

2014-01-27 Thread Nux!
On 28.01.2014 00:49, Marcus Sorensen wrote: So... I tried to use memory overcommit on KVM this week, and it blew up in my face. Apparently it's configured such that if I have a Service Offering of 4G, and I set memory overprovisioning to 2:1, the guest only actually gets configured with 2G. That'

Re: KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
I see that the same approach has been taken for CPU overprovisioning, but it actually works there, because cpu is allocated in 'shares', which are abstract and relative to all shares allocated. With memory, we can't just divide the raw size allocated and call it good. I think this was an oversight

KVM memory overprovision breaking me

2014-01-27 Thread Marcus Sorensen
So... I tried to use memory overcommit on KVM this week, and it blew up in my face. Apparently it's configured such that if I have a Service Offering of 4G, and I set memory overprovisioning to 2:1, the guest only actually gets configured with 2G. That's not how overprovisioning is supposed to work