Well guys, yesterday I took a look at the CS source code and how it was getting those values. I think that I cracked it.
It seems that there is a whole misunderstanding about the used memory value and It would be nice an update on the Cloudstack API documentation. Let's start: Here is a link for the API doc: http://cloudstack.apache.org/docs/api/apidocs-4.2/root_admin/listHosts.html "listHosts" is the method which returns those values about the host (mem, cpu and etc), the confusion is there. One looking at the list of values that it can return would find this: *memoryallocated* the amount of the host's memory currently allocated *memorytotal* the memory total of the host *memoryused* the amount of the host's memory currently used The first time I looked at it I thought; well, nice I can have the values about the amount of the total memory available to allocate VMs, the amount allocated and the amount used by those VMs.However, it turns out that it is a little more complicated than that. First of all, the memoryTotal, is the total amount of memory available to allocate directly to VMs, it is neither the total amount of memory on DOM0, nor the total amount of memory of the host. I will give an example; I am use Xen hypervisor and XCP. Well, let's suppose I have a host with 8GB of RAM, if I let the auto ballooning driver on it would look like the DOM0 would have available 8GB of RAM (at first sight), and one could use the "free" command to find out the amount of memory of that host. However, that is not the proper way to check the total amount of RAM to allocate to VMs, and each time you instantiate a new VM that value would decrease and would not appear on used memory. BTW, I do not use the auto ballooning driver, so, if I use the free command on DOM0 it would present a much smaller amount o RAM. The total amount of memory available to allocate VMs is calculated like this, "total host's RAM - DOM0 RAM". Cloudstack does that math perfectly; my host that has 8GB of RAM on CS it says that it has 7.5GB, since I set static values of RAM and VCPUs to my DOM0 (512MB of RAM and 2 VCPUs which are more than enough to that DOM0). To find out those values, you could you "xentop", and check where your hosts memory and CPUs are being used. Second, the memoryAllocated value is the amount of memory allocated to VMs and CS does that properly. No explanations needed, the value could be calculated using "xentop" or "xe host-compute-free-memory" and then decrease that value from the value of total memory, anyways it would give the exactly amount of memory with no problems. Third, the memoryused value, here is the misunderstanding. This value represents the memory used by the DOM0 (the amount that is allocated to it) and not the memory used by deployed VMs on that Host. However, the way CS API doc is written can make one think that those values are all about deployed VMs on hosts, when one of them is not. Thus, there is also another problem, there are no info about the metric used to represent such values, memoryTotal and memoryAllocated are given in bytes while memoryused is given in kilobytes and that is why the CS UI is presenting the those pretty weird values for used memory on infrastructure > Hosts view. At the end I do believe that the documentation needs update in order to state clearly what those values means and in what metric they are given. Thus, the CS UI needs a bug fix since the used memory value is being converted in a wrong way. BTW, I also noticed that it those values are taking a while to be updated, what is the time interval between updates of hosts statistics? I saw a configuration parameter called "host.stats.interval" that is set to 1(6000 milliseconds) minute. However, on table "op_host_capacity" and it has been two days that those values are not being updated. Could be a misconfiguration?A problem? On Thu, Jan 30, 2014 at 10:57 AM, Rafael Weingartner < rafaelweingart...@gmail.com> wrote: > I am using xen-hypervisor 4.1 and XCP 1.6 if I am not mistaken. > > I have also never noticed this used memory values until we deployed Zenoss > to monitor the cloud. I just noticed that because the Zenoss was generating > a graph that presented whole cloud memory usage and it was a little weird. > With high allocated values, but pretty low used values. > > As in your case, it seems pretty weird to me that we have this huge amount > of memory allocated (26GB), but just 28MB are being used. It almost does > not make sense. I also should point out that you used the command "free > -g", it shows the memory of the DOM0, in your case it just presents the > whole amount of memory of your hosts because you are using the auto > ballooning option. But in my case, we disabled this, and set static amount > of memory and cpus for the DOM0. > > Does anyone know from where the CS is getting those used memory values? > > > On Thu, Jan 30, 2014 at 5:48 AM, Marty Sweet <msweet....@gmail.com> wrote: > >> I should also point out that this is on CS 4.2.0. >> >> This issue may have been fixed since then. >> >> Marty >> >> >> On Thu, Jan 30, 2014 at 7:41 AM, Marty Sweet <msweet....@gmail.com> >> wrote: >> >> > What hypervisor are you using? >> > >> > My CS System Capacity on the dashboard shows *Memory 157.05 GB / 1.37 >> TB *(which >> > works + overprovisioning). >> > >> > I think this issue is slightly different to CLOUDSTACK-3048 as it should >> > not effect VM allocation. >> > I have never noticed about the memory used section: >> > >> > Memory Allocated 68.50 GB Memory Used 70.65 MB >> > $ free -g >> > total used free shared buffers >> cached >> > Mem: 251 250 1 0 0 >> 179 >> > -/+ buffers/cache: 70 181 >> > Swap: 5 2 2 >> > >> > >> > >> > Memory Allocated 22.05 GB Memory Used 24.06 MB >> > $ free -g >> > total used free shared buffers >> cached >> > Mem: 251 27 223 0 0 >> 3 >> > -/+ buffers/cache: 24 227 >> > Swap: 2 0 2 >> > >> > >> > Memory Allocated 39.53 GB Memory Used 41.51 MB >> > $ free -g >> > total used free shared buffers >> cached >> > Mem: 251 42 209 0 0 >> 0 >> > -/+ buffers/cache: 41 210 >> > Swap: 5 0 5 >> > >> > >> > Memory Allocated 26.97 GB Memory Used 28.46 MB >> > $ free -g >> > total used free shared buffers >> cached >> > Mem: 141 29 111 0 0 >> 1 >> > -/+ buffers/cache: 28 113 >> > Swap: 2 0 2 >> > >> > >> > It appears that the units used for 'Memory used' are simply a few >> factors >> > of 10 out. >> > >> > Can you have a look to see if there is a bug specifically for this? If >> not >> > feel free to create one using the information I have provided above. >> > >> > Thanks, >> > Marty >> > >> > >> > >> > On Wed, Jan 29, 2014 at 6:31 PM, Rafael Weingartner < >> > rafaelweingart...@gmail.com> wrote: >> > >> >> Doesn't anyone think that those memory values presented on >> >> infrastructure>hosts are a little odd? >> >> >> >> >> >> On Tue, Jan 28, 2014 at 2:54 PM, Rafael Weingartner < >> >> rafaelweingart...@gmail.com> wrote: >> >> >> >> > Hi all, >> >> > >> >> > I am mailing this to the dev and users list, since I am not sure >> which >> >> one >> >> > would be the best to make such question. >> >> > >> >> > >> >> > So, I deployed Zenoss to monitor my Cloud, and I noticed something >> weird >> >> > on a graph that it was generating about the Cloud's memory. The graph >> >> was >> >> > showing total memory as 26GB, allocated as 16GB and used as 19MB. >> >> > >> >> > It was not making sense, so I was looking around and I noticed that >> this >> >> > 19MB is the value that the Cloudstack presents as the memory used on >> >> > infrastructure>Hosts. >> >> > >> >> > >> >> > I think that this problem is related to this ticket opened last year. >> >> > >> >> > https://issues.apache.org/jira/browse/CLOUDSTACK-3048 >> >> > Sadly, it seems that no one has taken a look at it. >> >> > >> >> > >> >> > Does anyone know anything about this problem/bug? >> >> > >> >> > >> >> > -- >> >> > Rafael Weingärtner >> >> > >> >> >> >> >> >> >> >> -- >> >> Rafael Weingärtner >> >> >> > >> > >> > >> > > > > -- > Rafael Weingärtner > -- Rafael Weingärtner
