Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Dietmar Maurer
> Test case inspired by Linbit: > http://lists.linbit.com/pipermail/drbd-user/2008-May/009282.html > > We already know that the swap code has similar behavior. In case > a page gets touched while it is under write out IO, the swap allows > the modification to the page although it is under IO by th

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Eric Blevins
Suggestion for another test case. Create a linux VM that has tons of swap and just enough RAM to boot. Run a single threaded program that allocates and modifies more RAM than the VM has ensuring many of the modifications are to swap. This should result in out of sync blocks in the swap area. Test

[pve-devel] should we use systemd journal for logging

2015-05-29 Thread Dietmar Maurer
Hi all, in debian jessie, we have systemd journal on /tmp and rsyslog to log into /var/log/syslog. That means: - two different locations for logs - additional memory requirements to keep journal in mem - journal is lost at reboot, which is very bad for debugging. also see: https://bugs.d

Re: [pve-devel] [PATCH v2] move preparations into prepare()

2015-05-29 Thread Dietmar Maurer
applied, thanks! ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v2] prepare functions number two

2015-05-29 Thread Wolfgang Bumiller
So as reading a $nodename in the beginning isn't an issue the patch is simpler. I'm still moving initlog() calls to prepare() as well though as there's no point in having this occur on every `make`. Wolfgang Bumiller (1): move preparations into prepare() bin/pveceph | 25 ++

[pve-devel] [PATCH v2] move preparations into prepare()

2015-05-29 Thread Wolfgang Bumiller
Moving some preparation code found at the top level of some binaries into a prepare() sub. --- bin/pveceph | 25 ++--- bin/pvedaemon | 10 ++ bin/pveproxy| 18 ++ bin/pvestatd| 17 ++--- bin/pvesubscription | 23

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Dietmar Maurer
> >> 1) On any guest OS (Linux or Windows) where swap is used actively So maybe the bug can be triggered using mmap? Unfortunately, I have no real idea how mmap work internally, but I can imagine that it produces a similar pattern? Any mmap expert here? ___

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Stanislav German-Evtushenko
On Fri, May 29, 2015 at 4:03 PM, Dietmar Maurer wrote: > >> > So we still not understand the problem 100% :-/ >> > >> > You mention that it only happens on some special guest OS? Did you >> > also test different drivers (virtio, ide, scsi)? >> > >> >> 1) On any guest OS (Linux or Windows) where sw

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Dietmar Maurer
> > So we still not understand the problem 100% :-/ > > > > You mention that it only happens on some special guest OS? Did you > > also test different drivers (virtio, ide, scsi)? > > > > 1) On any guest OS (Linux or Windows) where swap is used actively > 2) As I remember I've seen the same issue

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Stanislav German-Evtushenko
On Fri, May 29, 2015 at 3:51 PM, Dietmar Maurer wrote: > >> > Come to think of it, you could probably say inconsistent garbage is >> > also just garbage. >> > >> > Does it _really_ fix the problems if you use a different cache option >> > in qemu? I'm just asking because it seems weird that a prog

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Dietmar Maurer
> > Come to think of it, you could probably say inconsistent garbage is > > also just garbage. > > > > Does it _really_ fix the problems if you use a different cache option > > in qemu? I'm just asking because it seems weird that a program (or > > OS for that matter) is _relying_ on bad data to be

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Stanislav German-Evtushenko
> Come to think of it, you could probably say inconsistent garbage is > also just garbage. > > Does it _really_ fix the problems if you use a different cache option > in qemu? I'm just asking because it seems weird that a program (or > OS for that matter) is _relying_ on bad data to be, well, not s

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Wolfgang Bumiller
Come to think of it, you could probably say inconsistent garbage is also just garbage. Does it _really_ fix the problems if you use a different cache option in qemu? I'm just asking because it seems weird that a program (or OS for that matter) is _relying_ on bad data to be, well, not so bad. ___

Re: [pve-devel] [PATCH 2/2] moving more binary initialization code into prepare()

2015-05-29 Thread Wolfgang Bumiller
> On May 29, 2015 at 1:59 PM Dietmar Maurer wrote: > > diff --git a/bin/pveceph b/bin/pveceph > > index 18a4e8b..092c569 100755 > > --- a/bin/pveceph > > +++ b/bin/pveceph > > @@ -24,19 +24,34 @@ use PVE::CLIHandler; > > > > use base qw(PVE::CLIHandler); > > > > -$ENV{'PATH'} = '/sbin:/bin:/u

Re: [pve-devel] [PATCH 2/2] moving more binary initialization code into prepare()

2015-05-29 Thread Dietmar Maurer
> diff --git a/bin/pveceph b/bin/pveceph > index 18a4e8b..092c569 100755 > --- a/bin/pveceph > +++ b/bin/pveceph > @@ -24,19 +24,34 @@ use PVE::CLIHandler; > > use base qw(PVE::CLIHandler); > > -$ENV{'PATH'} = '/sbin:/bin:/usr/sbin:/usr/bin'; Sorry, but why do you want to move this inside pre

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Wolfgang Bumiller
> > I can see that it is bad if RAID devices contains different data (not > > really acceptable for me). But in this case, it is due to wrong behavior > > inside the application, so the application cannot expect to read back > > "correct" data. Even if the application generates garbage writes, it'

[pve-devel] [PATCH 2/2] moving more binary initialization code into prepare()

2015-05-29 Thread Wolfgang Bumiller
These two are less trivial cases: They used to initialize the nodename right away and use it in the $cmddef command description. I have left the references to $nodename there for clarity for now. The printmanpod action speicifically ignores this section of the $cmddef structure, and prepare gets ca

[pve-devel] [PATCH 0/2] Start making use of daemon initialization changes

2015-05-29 Thread Wolfgang Bumiller
I've split this patch up into two parts because the second part needs some reviewing. The issue I encountered in these two files is that the $nodename is not only initialized early but also used in $cmddef. I'm now updating the $cmddef in prepare(). The functions where it matters (like RESTHandler:

[pve-devel] [PATCH 1/2] move trivial preparations into prepare()

2015-05-29 Thread Wolfgang Bumiller
Moving some preparation code found at the top level of some binaries into a prepare() sub. --- bin/pvedaemon | 10 ++ bin/pveproxy | 18 ++ bin/pvestatd | 18 +++--- bin/spiceproxy | 18 ++ 4 files changed, 37 insertions(+), 27 deletions(-)

Re: [pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-29 Thread Dietmar Maurer
> >>Great. I guess we can also hotplug that? > > Yes sure. it's just a write in /sys/fs/cgroups. > > This could be done online too in lxc. OK > BTW, I see that in lxc gui, the cpus number field = > lxc.cgroup.cpu.cfs_quota_us, so it's more cpulimit. > (all cores are present in container) Sor

Re: [pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-29 Thread Alexandre DERUMIER
>>Great. I guess we can also hotplug that? Yes sure. it's just a write in /sys/fs/cgroups. This could be done online too in lxc. BTW, I see that in lxc gui, the cpus number field = lxc.cgroup.cpu.cfs_quota_us, so it's more cpulimit. (all cores are present in container) - Mail original --

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Stanislav German-Evtushenko
On Fri, May 29, 2015 at 1:21 PM, Dietmar Maurer wrote: >> > It is also quite curious that DRBD layer has checks that detect this >> > problem (but doesn't attempt to fix it). >> > >> >> DRBD does not catch this by default but: >> - it detects checksum mismatch if data-integrity-alg is enabled >> (

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Dietmar Maurer
> > It is also quite curious that DRBD layer has checks that detect this > > problem (but doesn't attempt to fix it). > > > > DRBD does not catch this by default but: > - it detects checksum mismatch if data-integrity-alg is enabled > (disabled by default and recommended to be disabled in producti

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Stanislav German-Evtushenko
> Date: Fri, 29 May 2015 09:58:29 +0200 > From: Eneko Lacunza > To: Dietmar Maurer , pve-devel@pve.proxmox.com > Subject: Re: [pve-devel] Default cache mode for VM hard drives > > It is also quite curious that DRBD layer has checks that detect this > problem (but doesn't attempt to fix it). > DRB

Re: [pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-29 Thread Dietmar Maurer
> >>Answering myself, I found we can set this with > >> > >>CPUQuota=XX% > >> > >>So we can directly map this to the 'cpulimit' parameter? > > yes indeed. I'll send a patch monday for this. Great. I guess we can also hotplug that? ___ pve-devel mail

Re: [pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-29 Thread Alexandre DERUMIER
>>Answering myself, I found we can set this with >> >>CPUQuota=XX% >> >>So we can directly map this to the 'cpulimit' parameter? yes indeed. I'll send a patch monday for this. - Mail original - De: "dietmar" À: "aderumier" , "pve-devel" Envoyé: Vendredi 29 Mai 2015 08:38:09 Objet:

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Eneko Lacunza
On 28/05/15 17:54, Dietmar Maurer wrote: If you provide a buffer to the kernel, that you change while it is working with it, I don't know why you expect a reliable/predictable result? Specially (but not only) if you tell it not to make a copy!! Note that without O_DIRECT you won't get a "correct

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-29 Thread Wolfgang Bumiller
> AFAIK all disk IO is done by a single, dedicated thread. The same is true for the C testcase. One thread is writing, the other one is manipulating. > I tried to read qemu-kvm code but it is difficult for me as I have > never written C code. (...) > Meanwhile > if somebody implemented the "defau

Re: [pve-devel] High Performance SSH

2015-05-29 Thread Martin Waschbüsch
> Am 28.05.2015 um 12:55 schrieb dea : > > >> I don't think it is wise to play with security-related software in >> the stack. If OpenBSD and Debian (or for the matter all the other >> distros) haven't applied those patches, I'm sure there is some >> reason, although maybe it being only "uncerta