Re: [pve-devel] Count monthly traffic
> > It is still not clear if we want a single file, or multiple files. I > > guess this requires careful evaluation. How does such system behave > > when we have 3000VMs? We need to test that before we go further. > > Oh i thought a single file containing 32 net in/out values. I've no idea how > to test > i'm not really familiar with rrd. It is simply a requirement that you make yourself familiar with the required tools before you start any development - it is not that hard ;-) ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 09:08, schrieb Dietmar Maurer: >>> It is still not clear if we want a single file, or multiple files. I >>> guess this requires careful evaluation. How does such system behave >>> when we have 3000VMs? We need to test that before we go further. >> >> Oh i thought a single file containing 32 net in/out values. I've no idea how >> to test >> i'm not really familiar with rrd. > > It is simply a requirement that you make yourself familiar with the required > tools > before you start any development - it is not that hard ;-) > Sure but how to benchmark? Just create a file with 32 values and create 2 files (this but be the avg of network interfaces) may be it is just 1.5. And then update both each second and look at the disk i/o? Or how did you imagine this. I think it makes no sense to compare 32 values agains 32 files as most VMs won't have 32 nics. Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> Sure but how to benchmark? Just create a file with 32 values and create > 2 files (this but be the avg of network interfaces) may be it is just 1.5. > > And then update both each second and look at the disk i/o? Or how did you > imagine this. I think it makes no sense to compare 32 values agains > 32 files as most VMs won't have 32 nics. I want to know how large a file with 32 values is, and how the system behaves when we update 3000 files every 10 seconds. You can easily write a test program to write files at that rate. I am particularly interested in IO load (on normal ide disks) and memory requirements of rrdcached. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 09:28, schrieb Dietmar Maurer: >> Sure but how to benchmark? Just create a file with 32 values and create >> 2 files (this but be the avg of network interfaces) may be it is just 1.5. >> >> And then update both each second and look at the disk i/o? Or how did you >> imagine this. I think it makes no sense to compare 32 values agains >> 32 files as most VMs won't have 32 nics. > > I want to know how large a file with 32 values is, and how the system behaves > when we > update 3000 files every 10 seconds. You can easily write a test program to > write files > at that rate. > > I am particularly interested in IO load (on normal ide disks) and memory > requirements of rrdcached. Don't have IDE Disks at all. Can only provide SSD or SAS or SATA2 Disks. Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> > I am particularly interested in IO load (on normal ide disks) and memory > requirements of rrdcached. > > Don't have IDE Disks at all. Can only provide SSD or SAS or SATA2 Disks. I know that you have quite fast hardware. But we need to test on 'normal' hardware. Testing IO on SSDs makes no sense, so simply use the slowest SAS or SATA Disk you have ;-) ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 09:44, schrieb Dietmar Maurer: >>> I am particularly interested in IO load (on normal ide disks) and memory >> requirements of rrdcached. >> >> Don't have IDE Disks at all. Can only provide SSD or SAS or SATA2 Disks. > > I know that you have quite fast hardware. But we need to test on 'normal' > hardware. > Testing IO on SSDs makes no sense, so simply use the slowest SAS or SATA Disk > you have ;-) OK File size is 416k: ~# du -h myrrdfile.rrd 416Kmyrrdfile.rrd But on IDE with 3000 Files it doesn't work every 10s. At least without rrdcache. But that sounds in general a way to much for such a HW. I've no idea how to enable rrdcache in this case. Testscript is here: http://pastebin.com/raw.php?i=vQtYeHXc Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> But on IDE with 3000 Files it doesn't work every 10s. At least without > rrdcache. > But that sounds in general a way to much for such a HW. So maybe it is a bad idea to use rrd for that purpose - any other suggestions? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 13:52, schrieb Dietmar Maurer: >> But on IDE with 3000 Files it doesn't work every 10s. At least without >> rrdcache. >> But that sounds in general a way to much for such a HW. > > So maybe it is a bad idea to use rrd for that purpose - any other suggestions? How have you calculated your 3000 Files? 3000 VMs all with 32 nics seems unrealistic to me. Another way could be to just allow to query actual tap counters through API per NIC. We then just have no SUM of this data and just a current value. Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> How have you calculated your 3000 Files? > > 3000 VMs all with 32 nics seems unrealistic to me. I simply want to show you that this can overload all! servers inside a cluster by just writing rrd files. I am not keen to spend any resources on that task. > Another way could be to just allow to query actual tap counters through API > per > NIC. > > We then just have no SUM of this data and just a current value. Where do you plan to store that data? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> How have you calculated your 3000 Files? > > 3000 VMs all with 32 nics seems unrealistic to me. BTW, we have several users with >500VMs, so I guess we can easily reach that limit. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 14:09, schrieb Dietmar Maurer: >> How have you calculated your 3000 Files? >> >> 3000 VMs all with 32 nics seems unrealistic to me. > > I simply want to show you that this can overload all! servers inside a > cluster by > just writing rrd files. I am not keen to spend any resources on that task. > >> Another way could be to just allow to query actual tap counters through API >> per >> NIC. >> >> We then just have no SUM of this data and just a current value. > > Where do you plan to store that data? Nowhere ;-) how about just return the counter values for the correct tap device through API? So it is basically: 1.) a wrapper from netX to correct tap 2.) query tap counter inout / output values 3.) allow to query this through API So it is at least possible to implement traffic account in external software. You jsut have to query the API every X seconds and detect resets yourself. It acts than basically like SNMP traffic counters in switches. Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> Nowhere ;-) how about just return the counter values for the correct tap > device > through API? > > So it is basically: > 1.) a wrapper from netX to correct tap > 2.) query tap counter inout / output values > 3.) allow to query this through API > > So it is at least possible to implement traffic account in external software. > You > jsut have to query the API every X seconds and detect resets yourself. It acts > than basically like SNMP traffic counters in switches. sounds reasonable. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 14:11, schrieb Dietmar Maurer: >> How have you calculated your 3000 Files? >> >> 3000 VMs all with 32 nics seems unrealistic to me. > > BTW, we have several users with >500VMs, so I guess we can easily reach that > limit. Sure but those won't have 32 NICs. So if we go the way one file per NIC. We might have 3800 files for 3000 VMs. (800 VMs with 2 Nics so they have 1600 files and the rest with just one nic) Updating those works pretty good even on IDE and even without rrdcached. It generates nearly no load at all just 10% I/O Wait. But running so many VMs shouldn't be done on IDE anyway. Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 14:17, schrieb Dietmar Maurer:
>> Nowhere ;-) how about just return the counter values for the correct tap
>> device
>> through API?
>>
>> So it is basically:
>> 1.) a wrapper from netX to correct tap
>> 2.) query tap counter inout / output values
>> 3.) allow to query this through API
>>
>> So it is at least possible to implement traffic account in external
>> software. You
>> jsut have to query the API every X seconds and detect resets yourself. It
>> acts
>> than basically like SNMP traffic counters in switches.
>
> sounds reasonable.
Then let's go this way. It's much simpler than adding RRD.
So the question is should this be a completely new call or do you want
to add a new hash key to sub vmstatus { ?
this could be
my $netdev = PVE::ProcFSTools::read_proc_net_dev();
foreach my $dev (keys %$netdev) {
next if $dev !~ m/^tap([1-9]\d*)i/;
my $vmid = $1;
my $d = $res->{$vmid};
next if !$d;
$d->{netout} += $netdev->{$dev}->{receive};
$d->{netin} += $netdev->{$dev}->{transmit};
}
converted to:
my $netdev = PVE::ProcFSTools::read_proc_net_dev();
foreach my $dev (keys %$netdev) {
next if $dev !~ m/^tap([1-9]\d*)i(\d+)/;
my $vmid = $1;
my $netid = $2;
my $d = $res->{$vmid};
next if !$d;
$d->{netout} += $netdev->{$dev}->{receive};
$d->{netin} += $netdev->{$dev}->{transmit};
$d->{traffic}{'net'.$netid}{netout} = $netdev->{$dev}{receive};
$d->{traffic}{'net'.$netid}{netin} = $netdev->{$dev}{transmit};
}
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
I'm sorry to dropping in, but isn't Proxmox already counting traffic in
it's internal RRD? Couldn't the "rrddata" API call be used to retrieve the
data in the RRD externally and process it to count the total bandwidth used
in 1 month?
Maybe I've misunderstood what the issue is here...
Best regards,
Adrian Costin
On Wed, Apr 17, 2013 at 3:29 PM, Stefan Priebe - Profihost AG <
s.pri...@profihost.ag> wrote:
> Am 17.04.2013 14:17, schrieb Dietmar Maurer:
> >> Nowhere ;-) how about just return the counter values for the correct
> tap device
> >> through API?
> >>
> >> So it is basically:
> >> 1.) a wrapper from netX to correct tap
> >> 2.) query tap counter inout / output values
> >> 3.) allow to query this through API
> >>
> >> So it is at least possible to implement traffic account in external
> software. You
> >> jsut have to query the API every X seconds and detect resets yourself.
> It acts
> >> than basically like SNMP traffic counters in switches.
> >
> > sounds reasonable.
>
> Then let's go this way. It's much simpler than adding RRD.
>
> So the question is should this be a completely new call or do you want
> to add a new hash key to sub vmstatus { ?
>
> this could be
>
> my $netdev = PVE::ProcFSTools::read_proc_net_dev();
> foreach my $dev (keys %$netdev) {
> next if $dev !~ m/^tap([1-9]\d*)i/;
> my $vmid = $1;
> my $d = $res->{$vmid};
> next if !$d;
>
> $d->{netout} += $netdev->{$dev}->{receive};
> $d->{netin} += $netdev->{$dev}->{transmit};
> }
>
> converted to:
>
> my $netdev = PVE::ProcFSTools::read_proc_net_dev();
> foreach my $dev (keys %$netdev) {
> next if $dev !~ m/^tap([1-9]\d*)i(\d+)/;
> my $vmid = $1;
> my $netid = $2;
> my $d = $res->{$vmid};
> next if !$d;
>
> $d->{netout} += $netdev->{$dev}->{receive};
> $d->{netin} += $netdev->{$dev}->{transmit};
> $d->{traffic}{'net'.$netid}{netout} = $netdev->{$dev}{receive};
> $d->{traffic}{'net'.$netid}{netin} = $netdev->{$dev}{transmit};
> }
>
>
> Stefan
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Hi, Am 17.04.2013 14:42, schrieb Adrian Costin: > I'm sorry to dropping in, but isn't Proxmox already counting traffic in > it's internal RRD? Couldn't the "rrddata" API call be used to retrieve > the data in the RRD externally and process it to count the total > bandwidth used in 1 month? > > Maybe I've misunderstood what the issue is here... yes but just as a sum of all nics. If you just want to count for some nics or for EACH nic - you can't as all NICS are summed up. Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> >> 3000 VMs all with 32 nics seems unrealistic to me. > > > > BTW, we have several users with >500VMs, so I guess we can easily reach that > limit. > > Sure but those won't have 32 NICs. So if we go the way one file per NIC. > We might have 3800 files for 3000 VMs. (800 VMs with 2 Nics so they have > 1600 files and the rest with just one nic) > > Updating those works pretty good even on IDE and even without rrdcached. > It generates nearly no load at all just 10% I/O Wait. But running so many VMs > shouldn't be done on IDE anyway. I am confused now. If we have 500 VMs, we have at least 500 rrd files (currently). If you add one additional file per VM we have 1000 rrd files. If we assume a VM has 2nics on average we are at 1500 files... You wrote that the benchmark does not work at all with 3000 files, so I assume it also generates serious load with 1600 files. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 17.04.2013 16:39, schrieb Dietmar Maurer: 3000 VMs all with 32 nics seems unrealistic to me. BTW, we have several users with >500VMs, so I guess we can easily reach that limit. Sure but those won't have 32 NICs. So if we go the way one file per NIC. We might have 3800 files for 3000 VMs. (800 VMs with 2 Nics so they have 1600 files and the rest with just one nic) Updating those works pretty good even on IDE and even without rrdcached. It generates nearly no load at all just 10% I/O Wait. But running so many VMs shouldn't be done on IDE anyway. I am confused now. If we have 500 VMs, we have at least 500 rrd files (currently). If you add one additional file per VM we have 1000 rrd files. If we assume a VM has 2nics on average we are at 1500 files... Sorry i was just testing the netin / netout part. A main point seems to be the rows in file - so my tests don't cover this scenario i was just testing 32 in/out values vs. 1 file per nic for in/out. But right now i would prefer the API query SNMP like way anyway. Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] Add "sound card" to VM's from WebUI
Is this feature planned ? And if not, would patches be accepted to add this feature ? Use cases: -Virtual Terminal Services (Required for RDS Remote Sound) -Virtual Desktop Infrastructure Based on the guest OS the default sound card could be selected, e.g. ac97 for 32bit WinXP, hda for 64bit Windows/Linux Regards, Andrew Thrift ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Process to submit patches
Hi Dietmar, We have been running this successfully for a few weeks now on the default PVE kernel with no issues. I searched the list archives, and the issues I could see looked like they were related to the MTU on the parent physical interface being too small. Linux does not clearly differentiate between L2 (interface) and L3 (IP) MTU, and also has a hidden 4 byte allowance on the interface MTU, making it confusing. When running QinQ the parent physical interface needs a MTU of at least 1504 bytes (allows a 1508 byte frame), any sub vlan's and bridges will then inherit this MTU allowing for the "inner" tag to pass externally correctly. Obviously if the users are not doing QinQ the parent physical interface can be the default 1500byte MTU and will work perfectly. We have only tested with KVM, so maybe there are other issues with OpenVZ... If there are not, it would be good to see this patch merged. Regards, Andrew On 4/2/2013 5:32 PM, Dietmar Maurer wrote: Our patches creates the vlan sub-if on the parent VM bridge, rather than on the parent interface. This works with both QinQ and non QinQ configurations. AFAIR this create serious problem. We already tried that several times and always run into problems. Search the list for details. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Add "sound card" to VM's from WebUI
> Is this feature planned ? no, not really. > And if not, would patches be accepted to add this feature ? I do not think that is needed. > Use cases: > > -Virtual Terminal Services (Required for RDS Remote Sound) > -Virtual Desktop Infrastructure All known VDI solutions provide some kind of virtual sound device (RDP, spice), so there is no need to add any sound card. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Process to submit patches
> We have been running this successfully for a few weeks now on the default PVE > kernel with no issues. > > I searched the list archives, and the issues I could see looked like they were > related to the MTU on the parent physical interface being too small. > > Linux does not clearly differentiate between L2 (interface) and L3 (IP) MTU, > and > also has a hidden 4 byte allowance on the interface MTU, making it confusing. > When running QinQ the parent physical interface needs a MTU of at least 1504 > bytes (allows a 1508 byte frame), any sub vlan's and bridges will then > inherit this > MTU allowing for the "inner" tag to pass externally correctly. > > Obviously if the users are not doing QinQ the parent physical interface can be > the default 1500byte MTU and will work perfectly. @Alexandre: Do you have some spare time to test that again using MTU 1504? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Process to submit patches
> Linux does not clearly differentiate between L2 (interface) and L3 (IP) MTU, > and > also has a hidden 4 byte allowance on the interface MTU, making it confusing. > When running QinQ the parent physical interface needs a MTU of at least 1504 > bytes (allows a 1508 byte frame), any sub vlan's and bridges will then > inherit this > MTU allowing for the "inner" tag to pass externally correctly. And what are the correct settings for jumbo frames (MTU 9000)? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> Then let's go this way. It's much simpler than adding RRD.
>
> So the question is should this be a completely new call
Yes, I think this should be a new call:
GET /nodes//netstat
[
{vmid => 100, dev => net0, in => XXX, out => YYY},
{vmid => 100, dev => net1, in => XXX, out => YYY},
{vmid => 200, dev => net0, in => XXX, out => YYY},
]
So that we can get all data with one API call.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Add "sound card" to VM's from WebUI
RDP still requires that the Terminal Server has a sound card. It only configures a virtual sound device for the RDS client if one exists on the server. On 4/18/2013 3:49 PM, Dietmar Maurer wrote: Is this feature planned ? no, not really. And if not, would patches be accepted to add this feature ? I do not think that is needed. Use cases: -Virtual Terminal Services (Required for RDS Remote Sound) -Virtual Desktop Infrastructure All known VDI solutions provide some kind of virtual sound device (RDP, spice), so there is no need to add any sound card. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Process to submit patches
Hi Dietmar, It depends on what size frame you are trying to pass. Anything over 1504 bytes is considered "Jumbo". An MTU of 9000 would allow you to pass a 9004byte frame. If you were doing "jumbo" frames to one of your VM's and wanted double tagging you would set your physical interface with an MTU of 9004 to allow for a 9008byte frame. On 4/18/2013 4:19 PM, Dietmar Maurer wrote: Linux does not clearly differentiate between L2 (interface) and L3 (IP) MTU, and also has a hidden 4 byte allowance on the interface MTU, making it confusing. When running QinQ the parent physical interface needs a MTU of at least 1504 bytes (allows a 1508 byte frame), any sub vlan's and bridges will then inherit this MTU allowing for the "inner" tag to pass externally correctly. And what are the correct settings for jumbo frames (MTU 9000)? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Process to submit patches
> It depends on what size frame you are trying to pass. Anything over > 1504 bytes is considered "Jumbo". An MTU of 9000 would allow you to pass a > 9004byte frame. If you were doing "jumbo" frames to one of your VM's and > wanted double tagging you would set your physical interface with an MTU of > 9004 to allow for a 9008byte frame. But AFAIK MTU >9000 is not possible with most network hardware? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Add "sound card" to VM's from WebUI
I'm not sure about spice, I think we need a sound card. I'll do test this week. - Mail original - De: "Dietmar Maurer" À: "Andrew Thrift" , pve-devel@pve.proxmox.com Envoyé: Jeudi 18 Avril 2013 05:49:33 Objet: Re: [pve-devel] Add "sound card" to VM's from WebUI > Is this feature planned ? no, not really. > And if not, would patches be accepted to add this feature ? I do not think that is needed. > Use cases: > > - Virtual Terminal Services (Required for RDS Remote Sound) > - Virtual Desktop Infrastructure All known VDI solutions provide some kind of virtual sound device (RDP, spice), so there is no need to add any sound card. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH] implement node netstat call to get current tap network counters
Signed-off-by: Stefan Priebe
---
API2/Nodes.pm | 40
1 file changed, 40 insertions(+)
diff --git a/API2/Nodes.pm b/API2/Nodes.pm
index 0dac6af..8aebae0 100644
--- a/API2/Nodes.pm
+++ b/API2/Nodes.pm
@@ -123,6 +123,7 @@ __PACKAGE__->register_method ({
{ name => 'aplinfo' },
{ name => 'startall' },
{ name => 'stopall' },
+ { name => 'netstat' },
];
return $result;
@@ -273,6 +274,45 @@ __PACKAGE__->register_method({
}});
__PACKAGE__->register_method({
+name => 'netstat',
+path => 'netstat',
+method => 'GET',
+permissions => {
+ check => ['perm', '/nodes/{node}', [ 'Sys.Audit' ]],
+},
+description => "Read tap/vm network device interface counters",
+proxyto => 'node',
+parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ },
+},
+returns => {
+ type => "object",
+ properties => {
+
+ },
+},
+code => sub {
+ my ($param) = @_;
+
+ my $res = { };
+
+ my $netdev = PVE::ProcFSTools::read_proc_net_dev();
+ foreach my $dev (keys %$netdev) {
+ next if $dev !~ m/^tap([1-9]\d*)i(\d+)$/;
+ my $vmid = $1;
+ my $netid = $2;
+
+ $res->{$vmid}{'net'.$netid}{out} = $netdev->{$dev}->{receive};
+ $res->{$vmid}{'net'.$netid}{in} = $netdev->{$dev}->{transmit};
+ }
+
+ return $res;
+}});
+
+__PACKAGE__->register_method({
name => 'node_cmd',
path => 'status',
method => 'POST',
--
1.7.10.4
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 18.04.2013 06:32, schrieb Dietmar Maurer:
Then let's go this way. It's much simpler than adding RRD.
So the question is should this be a completely new call
Yes, I think this should be a new call:
GET /nodes//netstat
[
{vmid => 100, dev => net0, in => XXX, out => YYY},
{vmid => 100, dev => net1, in => XXX, out => YYY},
{vmid => 200, dev => net0, in => XXX, out => YYY},
]
So that we can get all data with one API call.
Patch sent. I used a different output format which has the vmid as a key
so it is easier to access the right information as nobody needs to loop
through the array.
Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] new https server
Thanks Dietmar, I'll test it this week. BTW, I finally get tls working with spice, I'll submit patches this week. Do you think it's possible to use this new http server as CONNECT proxy ? - Mail original - De: "Dietmar Maurer" À: pve-devel@pve.proxmox.com Envoyé: Mercredi 17 Avril 2013 07:58:13 Objet: [pve-devel] new https server I just committed the code for the new http server. You basically need to update pve-manager and pve-access-control packages. We do not longer need apache2 for the api, so you can safely stop and remove apache2. The new service is called pveproxy, and only listens to port 8006 (no redirect for port 80 and 443). The new http server code in PVE::HTTPServer is event based using AnyEvent (fully asynchronous). So we can open many connection and support keep-alive. The whole server is about 1000 LOC, but code is a bit hard to read because of the ‘continuation passing’ style ( http://en.wikipedia.org/wiki/Continuation-passing_style ) In future, it would be even possible to use Coro ( http://search.cpan.org/~mlehmann/Coro-6.23/ ) to further improve performance. Please review the code, and report bugs and missing features. - Dietmar ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> Patch sent. I used a different output format which has the vmid as a key so > it is > easier to access the right information as nobody needs to loop through the > array. But that format does not work with an ExtJS grid widget (in case someone wants to display it on the GUI)? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
Am 18.04.2013 08:35, schrieb Dietmar Maurer: Patch sent. I used a different output format which has the vmid as a key so it is easier to access the right information as nobody needs to loop through the array. But that format does not work with an ExtJS grid widget (in case someone wants to display it on the GUI)? Oh ok sorry didn't know that. I can change it but i don't see any useful usage to display these values. They're just counters rising up to 64bit and then reset through the lifetime of a VM. I think without useful deltas and history there is no usage. But i can still change it if you want. Just tell me. Greets, Stefan ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] new https server
> BTW, I finally get tls working with spice, I'll submit patches this week. > Do you think it's possible to use this new http server as CONNECT proxy ? Honestly, I never implemented HTTP Connect method. But we now have full control on the web server, so we can do anything we want ;-) I guess it is possible. BTW, does spice support websockets, or only HTTP Connect? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Count monthly traffic
> Oh ok sorry didn't know that. I can change it but i don't see any useful > usage to > display these values. They're just counters rising up to 64bit and then reset > through the lifetime of a VM. I think without useful deltas and history there > is no > usage. But i can still change it if you want. Just tell me. yes, I would prefer the other format. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Process to submit patches
jumbo frame on cisco is 9216 and normal size is 1518 If you set 1500 or 9000 in your vm and you do vlan tagging, it's not a problem as we have some extra bytes. - Mail original - De: "Dietmar Maurer" À: "Andrew Thrift" , pve-devel@pve.proxmox.com Envoyé: Jeudi 18 Avril 2013 07:19:19 Objet: Re: [pve-devel] Process to submit patches > It depends on what size frame you are trying to pass. Anything over > 1504 bytes is considered "Jumbo". An MTU of 9000 would allow you to pass a > 9004byte frame. If you were doing "jumbo" frames to one of your VM's and > wanted double tagging you would set your physical interface with an MTU of > 9004 to allow for a 9008byte frame. But AFAIK MTU >9000 is not possible with most network hardware? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] implement node netstat call to get current tap network counters
> + $res->{$vmid}{'net'.$netid}{out} = $netdev->{$dev}->{receive};
> + $res->{$vmid}{'net'.$netid}{in} = $netdev->{$dev}->{transmit};
I also prefer "net$netid" instead of 'net'.$netid
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] implement node netstat call to get current tap network counters
Am 18.04.2013 08:41, schrieb Dietmar Maurer:
+ $res->{$vmid}{'net'.$netid}{out} = $netdev->{$dev}->{receive};
+ $res->{$vmid}{'net'.$netid}{in} = $netdev->{$dev}->{transmit};
I also prefer "net$netid" instead of 'net'.$netid
Is this a bug?
The code in vmstatus seems to be wrong for netout and netin
$d->{netout} += $netdev->{$dev}->{receive};
$d->{netin} += $netdev->{$dev}->{transmit};
netin is receive and netout is transmit... isn't it?
Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] new https server
>>BTW, does spice support websockets, or only HTTP Connect? Natively for the spice server support only http connect. We need to use websockets (for spice-html5), using websocky (python or C implementation) https://github.com/kanaka/websockify I have redone tests with spice-html5 last week, it's works better than last yeay, but they are some display bugs, and it's slower than spice-gtk. - Mail original - De: "Dietmar Maurer" À: "Alexandre DERUMIER" Cc: pve-devel@pve.proxmox.com Envoyé: Jeudi 18 Avril 2013 08:37:51 Objet: RE: [pve-devel] new https server > BTW, I finally get tls working with spice, I'll submit patches this week. > Do you think it's possible to use this new http server as CONNECT proxy ? Honestly, I never implemented HTTP Connect method. But we now have full control on the web server, so we can do anything we want ;-) I guess it is possible. BTW, does spice support websockets, or only HTTP Connect? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH] implement node netstat call to get current tap network counters
Changes since V1:
- new return format (use an arrayref instead of a hash to be JS compatible)
- swap in / out / transmit / receive
Signed-off-by: Stefan Priebe
---
API2/Nodes.pm | 48
1 file changed, 48 insertions(+)
diff --git a/API2/Nodes.pm b/API2/Nodes.pm
index 0dac6af..9bda85c 100644
--- a/API2/Nodes.pm
+++ b/API2/Nodes.pm
@@ -123,6 +123,7 @@ __PACKAGE__->register_method ({
{ name => 'aplinfo' },
{ name => 'startall' },
{ name => 'stopall' },
+ { name => 'netstat' },
];
return $result;
@@ -273,6 +274,53 @@ __PACKAGE__->register_method({
}});
__PACKAGE__->register_method({
+name => 'netstat',
+path => 'netstat',
+method => 'GET',
+permissions => {
+ check => ['perm', '/nodes/{node}', [ 'Sys.Audit' ]],
+},
+description => "Read tap/vm network device interface counters",
+proxyto => 'node',
+parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ },
+},
+returns => {
+type => "array",
+items => {
+type => "object",
+properties => {},
+},
+},
+code => sub {
+ my ($param) = @_;
+
+ my $res = [ ];
+
+ my $netdev = PVE::ProcFSTools::read_proc_net_dev();
+ foreach my $dev (keys %$netdev) {
+ next if $dev !~ m/^tap([1-9]\d*)i(\d+)$/;
+ my $vmid = $1;
+ my $netid = $2;
+
+push(
+@$res,
+{
+vmid => $vmid,
+dev => "net$netid",
+in => $netdev->{$dev}->{receive},
+out => $netdev->{$dev}->{transmit},
+}
+);
+ }
+
+ return $res;
+}});
+
+__PACKAGE__->register_method({
name => 'node_cmd',
path => 'status',
method => 'POST',
--
1.7.10.4
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] implement node netstat call to get current tap network counters
> Is this a bug?
>
> The code in vmstatus seems to be wrong for netout and netin
>
> $d->{netout} += $netdev->{$dev}->{receive};
> $d->{netin} += $netdev->{$dev}->{transmit};
>
>
> netin is receive and netout is transmit... isn't it?
Should be easy to test by downloading a file inside the VM - please can you
test that?
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
