[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Just out of interest tried how far the timeout hackery can go working around the issue. Well, looks like it goes quite far: having previously reproduced the hang in 4-5 runs and in under a minute, now have had this running without a hang for an hour. I will also test the patch under OBS worker(s) and if it solves the issue there as well, I will attach it as a workaround for time being for those interested. However, Peter is right and this is not a final solution of any kind: just a workaround. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Some kind of semi-workaround patch attached. It seems to leave this kind of race window for me (for select which is worse): 0x6004bf98 <+136>: xor%r8d,%r8d 0x6004bf9b <+139>: test %eax,%eax 0x6004bf9d <+141>: jne0x6004c2b7 0x6004bfa3 <+147>: mov0x20(%rsp),%r14 0x6004bfa8 <+152>: mov0x246d8(%r14),%esi 0x6004bfaf <+159>: test %esi,%esi 0x6004bfb1 <+161>: je 0x6004bfb8 0x6004bfb3 <+163>: lea0x40(%rsp),%r8 0x6004bfb8 <+168>: mov0x28(%rsp),%rdx 0x6004bfbd <+173>: mov%r11,%rsi 0x6004bfc0 <+176>: mov%ebx,%edi 0x6004bfc2 <+178>: callq 0x6012df90 I think it could still be narrowed some, but this makes it unlikely enough for me for time being... ** Patch added: "racy workaround patch" https://bugs.launchpad.net/qemu/+bug/955379/+attachment/3452139/+files/cmake.patch -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
LK: Ok, good catch, that might be more suitable option. Thanks, -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Peter, I have qemu chrootable test case under which you could fire one command to hit the bug reliably. Only issue is, are you willing to take a peek at 100M extractable tarball? If not, I'll try to create a smaller one. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: New Status in Linaro QEMU: New Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Ok, test case attached (80M tar). This hugely stripped one is not 100% reproducer, but do few loops and you will hit it. Instructions for using: - extract, chroot - cd /home/abuild/rpmbuild - su abuild - export RPM_BUILD_ROOT=$PWD - rpmbuild -ba SOURCES/libshortcut.spec ** Attachment added: "Tizen chroot" https://bugs.launchpad.net/qemu/+bug/955379/+attachment/3446633/+files/root5-small.tar.bz2 -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: New Status in Linaro QEMU: New Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Mind you, when you hit the bug it just hangs and cmake test errors are just to speed up the process of hitting the bug (if cmake just fails you did not hit the bug). Feel free to try with any qemu variant, they all hang similarly when bug is hit. I think that root had some suse 1.2 one inside. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: New Status in Linaro QEMU: New Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
If that is the case for you (for me it reproduces it every 4-5 runs or so), there are two options: 1) put while(true) loop around the rpmbuild and you will hit it always, or 2) I can wrap up a bit bigger cmake usecase that systematically hits it. Warn you though, size will jump to 200M. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: New Status in Linaro QEMU: New Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
> this blocks forever, because the thing that would wake it up is the signal handler writing to the pipe we're selecting on, but we will never run the signal handler until select exits Duh, makes sense, have to think about this. Thank you for great analysis :) Apparently have to dig into qemu's code to understand this better, but first thought was that do you think it would be possible to add some crude hack bit in qemu's signal handler which we could 'almost atomically' check prior to entering system poll/select/read/whatnot ? This bit would tell there are user signals queued and handlers should be executed first.. ? -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: New Status in Linaro QEMU: New Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
So I guess 'raciness' of my proposed patch would only depend on how small I could squeeze the section between 'sigpending' flag comparison and actual syscall entering? -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
And what would break if we make poll timeout instantly in case there are signals pending and restart the given syscall after handlers run? -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Moreover, is there even a need to restart anything, just make it async call in case signals were pending? -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Never mind, async/zero timeout call would suffer from same (albeit now tiny) race. It would make this far less invasive as a change though. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [Bug 955379] Re: cmake hangs with qemu-arm-static
Luke Kim: quite unlikely that above patch would cause the issue you see.. are you sure something else did not break in your environment? Can you execute that same make manually? -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/955379 Title: cmake hangs with qemu-arm-static Status in QEMU: Confirmed Status in Linaro QEMU: Confirmed Status in “qemu-linaro” package in Ubuntu: Confirmed Bug description: I'm using git commit 3e7ecd976b06f... configured with --target-list =arm-linux-user --static in a chroot environment to compile some things. I ran into this problem with both pcl and opencv-2.3.1. cmake consistently freezes at some point during its execution, though in a different spot each time, usually during a step when it's searching for some libraries. For instance, pcl most commonly stops after: [snip] -- Boost version: 1.46.1 -- Found the following Boost libraries: -- system -- filesystem -- thread -- date_time -- checking for module 'eigen3' -- found eigen3, version 3.0.1 which is perplexing because it freezes after finding what it wants, not during the search. When it does get past that point, it does so almost immediately but freezes somewhere else. I'm using 64-bit Ubuntu 11.10 with kernel release 3.0.0-16-generic with an Intel i5. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/955379/+subscriptions
[Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
From: Janne Karhunen
Replace hardcoded vmdk description file hardware version flag with
a user definable version.
Signed-off-by: Janne Karhunen
---
block/vmdk.c | 20 ++--
include/block/block_int.h | 3 +--
qemu-doc.texi | 4 ++--
3 files changed, 13 insertions(+), 14 deletions(-)
diff --git a/block/vmdk.c b/block/vmdk.c
index 45f9d3c..1436785 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -1829,8 +1829,8 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
int64_t total_size = 0, filesize;
char *adapter_type = NULL;
char *backing_file = NULL;
+char *hw_version = NULL;
char *fmt = NULL;
-int flags = 0;
int ret = 0;
bool flat, split, compress;
GString *ext_desc_lines;
@@ -1861,7 +1861,7 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
"# The Disk Data Base\n"
"#DDB\n"
"\n"
-"ddb.virtualHWVersion = \"%d\"\n"
+"ddb.virtualHWVersion = \"%s\"\n"
"ddb.geometry.cylinders = \"%" PRId64 "\"\n"
"ddb.geometry.heads = \"%" PRIu32 "\"\n"
"ddb.geometry.sectors = \"63\"\n"
@@ -1878,9 +1878,8 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
BDRV_SECTOR_SIZE);
adapter_type = qemu_opt_get_del(opts, BLOCK_OPT_ADAPTER_TYPE);
backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
-if (qemu_opt_get_bool_del(opts, BLOCK_OPT_COMPAT6, false)) {
-flags |= BLOCK_FLAG_COMPAT6;
-}
+hw_version = qemu_opt_get_del(opts, BLOCK_OPT_HWVERSION);
+
fmt = qemu_opt_get_del(opts, BLOCK_OPT_SUBFMT);
if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ZEROED_GRAIN, false)) {
zeroed_grain = true;
@@ -2001,7 +2000,7 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
fmt,
parent_desc_line,
ext_desc_lines->str,
- (flags & BLOCK_FLAG_COMPAT6 ? 6 : 4),
+ hw_version,
total_size /
(int64_t)(63 * number_heads * BDRV_SECTOR_SIZE),
number_heads,
@@ -2047,6 +2046,7 @@ exit:
}
g_free(adapter_type);
g_free(backing_file);
+g_free(hw_version);
g_free(fmt);
g_free(desc);
g_free(path);
@@ -2292,10 +2292,10 @@ static QemuOptsList vmdk_create_opts = {
.help = "File name of a base image"
},
{
-.name = BLOCK_OPT_COMPAT6,
-.type = QEMU_OPT_BOOL,
-.help = "VMDK version 6 image",
-.def_value_str = "off"
+.name = BLOCK_OPT_HWVERSION,
+.type = QEMU_OPT_STRING,
+.help = "VMDK hardware version",
+.def_value_str = "4"
},
{
.name = BLOCK_OPT_SUBFMT,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 10d8759..0852316 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -38,12 +38,11 @@
#include "qemu/throttle.h"
#define BLOCK_FLAG_ENCRYPT 1
-#define BLOCK_FLAG_COMPAT6 4
#define BLOCK_FLAG_LAZY_REFCOUNTS 8
#define BLOCK_OPT_SIZE "size"
#define BLOCK_OPT_ENCRYPT "encryption"
-#define BLOCK_OPT_COMPAT6 "compat6"
+#define BLOCK_OPT_HWVERSION "hwversion"
#define BLOCK_OPT_BACKING_FILE "backing_file"
#define BLOCK_OPT_BACKING_FMT "backing_fmt"
#define BLOCK_OPT_CLUSTER_SIZE "cluster_size"
diff --git a/qemu-doc.texi b/qemu-doc.texi
index 79141d3..79f4168 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -691,8 +691,8 @@ Supported options:
@table @code
@item backing_file
File name of a base image (see @option{create} subcommand).
-@item compat6
-Create a VMDK version 6 image (instead of version 4)
+@item hwversion
+Specify vmdk hardware version (default 4)
@item subformat
Specifies which VMDK subformat to use. Valid options are
@code{monolithicSparse} (default),
--
1.9.1
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
On Wed, Apr 20, 2016 at 7:22 PM, Fam Zheng wrote: >> Replace hardcoded vmdk description file hardware version flag with >> a user definable version. > > Hi Janne, > > Since this is a new feature, please explain why this change is desired (i.e. > the necessity.) See here what it controls for vmware: https://kb.vmware.com/selfservice/microsites/search.do?languag e=en_US&cmd=displayKC&externalId=1003746 Seems that the vmware cluster I'm using is configured to eat vmdks with only certain virtual hardware versions specified as being supported, so this patch allows me to define that version. Besides that, I've read that to mean that it's a hint from me (as the vmdk author) to the cluster that this is the vmware virtual hardware version I have tested this vmdk to run with. It's actually relatively useful metadata as such. In other words, without the patch not a single qemu-img generated vmdk is 'good' for the system I'm using. -- Janne
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
On Wed, Apr 20, 2016 at 7:22 PM, Fam Zheng wrote: > For backward compatibility reason, we shouldn't remove a pre-existing option > without solid justification and probably a grace period. I suggest adding the > hwversion option in addition, and document and check it as exclusive with the > compat6 flag. That's certainly doable and kind of makes sense, but I'm not entirely sure that compat6 flag makes any sense to begin with. Does it? -- Janne
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
On Wed, Apr 20, 2016 at 8:26 PM, Fam Zheng wrote: >> In other words, without the patch not a single qemu-img generated vmdk >> is 'good' for the system I'm using. > > Thanks for the explanation, please add this information into the commit > message > in next submission and also fix the option code so I can ack your patch. Ack, will do. Thanks -- Janne
[Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
From: Janne Karhunen
Vmdk images have metadata to indicate the vmware virtual
hardware version image was created/tested to run with.
Allow users to specify that version via new 'hwversion'
option.
Signed-off-by: Janne Karhunen
---
block/vmdk.c | 23 +++
include/block/block_int.h | 2 +-
qemu-doc.texi | 3 +++
3 files changed, 23 insertions(+), 5 deletions(-)
diff --git a/block/vmdk.c b/block/vmdk.c
index 45f9d3c..4b5cac8 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -1829,8 +1829,8 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
int64_t total_size = 0, filesize;
char *adapter_type = NULL;
char *backing_file = NULL;
+char *hw_version = NULL;
char *fmt = NULL;
-int flags = 0;
int ret = 0;
bool flat, split, compress;
GString *ext_desc_lines;
@@ -1861,7 +1861,7 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
"# The Disk Data Base\n"
"#DDB\n"
"\n"
-"ddb.virtualHWVersion = \"%d\"\n"
+"ddb.virtualHWVersion = \"%s\"\n"
"ddb.geometry.cylinders = \"%" PRId64 "\"\n"
"ddb.geometry.heads = \"%" PRIu32 "\"\n"
"ddb.geometry.sectors = \"63\"\n"
@@ -1878,8 +1878,16 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
BDRV_SECTOR_SIZE);
adapter_type = qemu_opt_get_del(opts, BLOCK_OPT_ADAPTER_TYPE);
backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
+hw_version = qemu_opt_get_del(opts, BLOCK_OPT_HWVERSION);
if (qemu_opt_get_bool_del(opts, BLOCK_OPT_COMPAT6, false)) {
-flags |= BLOCK_FLAG_COMPAT6;
+if (strcmp(hw_version, "4")) {
+error_setg(errp,
+ "compat6 and hwversion are mutually exclusive");
+ret = -EINVAL;
+goto exit;
+}
+g_free(hw_version);
+hw_version = g_strdup("6");
}
fmt = qemu_opt_get_del(opts, BLOCK_OPT_SUBFMT);
if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ZEROED_GRAIN, false)) {
@@ -2001,7 +2009,7 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
fmt,
parent_desc_line,
ext_desc_lines->str,
- (flags & BLOCK_FLAG_COMPAT6 ? 6 : 4),
+ hw_version,
total_size /
(int64_t)(63 * number_heads * BDRV_SECTOR_SIZE),
number_heads,
@@ -2047,6 +2055,7 @@ exit:
}
g_free(adapter_type);
g_free(backing_file);
+g_free(hw_version);
g_free(fmt);
g_free(desc);
g_free(path);
@@ -2298,6 +2307,12 @@ static QemuOptsList vmdk_create_opts = {
.def_value_str = "off"
},
{
+.name = BLOCK_OPT_HWVERSION,
+.type = QEMU_OPT_STRING,
+.help = "VMDK hardware version",
+.def_value_str = "4"
+},
+{
.name = BLOCK_OPT_SUBFMT,
.type = QEMU_OPT_STRING,
.help =
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 10d8759..931a412 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -38,12 +38,12 @@
#include "qemu/throttle.h"
#define BLOCK_FLAG_ENCRYPT 1
-#define BLOCK_FLAG_COMPAT6 4
#define BLOCK_FLAG_LAZY_REFCOUNTS 8
#define BLOCK_OPT_SIZE "size"
#define BLOCK_OPT_ENCRYPT "encryption"
#define BLOCK_OPT_COMPAT6 "compat6"
+#define BLOCK_OPT_HWVERSION "hwversion"
#define BLOCK_OPT_BACKING_FILE "backing_file"
#define BLOCK_OPT_BACKING_FMT "backing_fmt"
#define BLOCK_OPT_CLUSTER_SIZE "cluster_size"
diff --git a/qemu-doc.texi b/qemu-doc.texi
index 79141d3..d3a5f94 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -693,6 +693,9 @@ Supported options:
File name of a base image (see @option{create} subcommand).
@item compat6
Create a VMDK version 6 image (instead of version 4)
+@item hwversion
+Specify vmdk virtual hardware version. Option is mutually exclusive
+with the compat6 flag.
@item subformat
Specifies which VMDK subformat to use. Valid options are
@code{monolithicSparse} (default),
--
1.9.1
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
On Wed, Apr 20, 2016 at 9:44 PM, Fam Zheng wrote: >> That's certainly doable and kind of makes sense, but I'm not entirely >> sure that compat6 flag makes any sense to begin with. Does it? > > I don't know VMware products well enough to tell, but it has been available > since forever, therefore my suggestion is to just keep it which cannot hurt. I sent the new patch yesterday with this option still present and it works the same as it always has. I also tested it to work on all option pairings, so it should be good to go AFAIK. Hope you have some time to check that. -- Janne
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
On Fri, Apr 29, 2016 at 12:41 AM, Fam Zheng wrote:
>> Signed-off-by: Janne Karhunen
>
> Sorry for the late reply, I was distracted..
Like wise, was traveling..
>> if (qemu_opt_get_bool_del(opts, BLOCK_OPT_COMPAT6, false)) {
>> -flags |= BLOCK_FLAG_COMPAT6;
>
> Please remove BLOCK_FLAG_COMPAT6 from include/block/block_int.h| as well.
Wasn't it removed?
-#define BLOCK_FLAG_COMPAT6 4
>> +if (strcmp(hw_version, "4")) {
>
> Doesn't compat6 mean hw_version is 6? I think you want s/4/6/ here.
Yes and that's the functionality here. So the logic is that if
BLOCK_OPT_COMPAT6 was defined and hw_version had something other than
"4" we fail saying can't do both.
Test case:
qemu-img convert -f vmdk -O vmdk img.vmdk img.vmdk-output
ddb.virtualHWVersion = "4" (default)
qemu-img convert -f vmdk -O vmdk -o compat6=on img.vmdk img.vmdk-output
ddb.virtualHWVersion = "6" (compat6 default)
qemu-img convert -f vmdk -O vmdk -o hwversion=9 img.vmdk img.vmdk-output
ddb.virtualHWVersion = "9" (the new feature)
>> +@item hwversion
>> +Specify vmdk virtual hardware version. Option is mutually exclusive
>> +with the compat6 flag.
>
> The code is not really treating them exclusively, it only checks for
> contradiction of values. So I suggest either fix the document or the code.
It fails instantly if you try to pass both so what do you suggest it
should do (returns -EINVAL)?
qemu-img convert -f vmdk -O vmdk -o compat6=on -o hwversion=9 img.vmdk
img.vmdk.out
qemu-img: img.vmdk-output: error while converting vmdk: compat6 and
hwversion are mutually exclusive
--
Janne
[Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
From: Janne Karhunen
Vmdk images have metadata to indicate the vmware virtual
hardware version image was created/tested to run with.
Allow users to specify that version via new 'hwversion'
option.
Signed-off-by: Janne Karhunen
---
block/vmdk.c | 27 +++
include/block/block_int.h | 2 +-
qemu-doc.texi | 3 +++
3 files changed, 27 insertions(+), 5 deletions(-)
diff --git a/block/vmdk.c b/block/vmdk.c
index 45f9d3c..955c6b3 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -1829,8 +1829,8 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
int64_t total_size = 0, filesize;
char *adapter_type = NULL;
char *backing_file = NULL;
+char *hw_version = NULL;
char *fmt = NULL;
-int flags = 0;
int ret = 0;
bool flat, split, compress;
GString *ext_desc_lines;
@@ -1861,7 +1861,7 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
"# The Disk Data Base\n"
"#DDB\n"
"\n"
-"ddb.virtualHWVersion = \"%d\"\n"
+"ddb.virtualHWVersion = \"%s\"\n"
"ddb.geometry.cylinders = \"%" PRId64 "\"\n"
"ddb.geometry.heads = \"%" PRIu32 "\"\n"
"ddb.geometry.sectors = \"63\"\n"
@@ -1878,8 +1878,20 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
BDRV_SECTOR_SIZE);
adapter_type = qemu_opt_get_del(opts, BLOCK_OPT_ADAPTER_TYPE);
backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
+hw_version = qemu_opt_get_del(opts, BLOCK_OPT_HWVERSION);
if (qemu_opt_get_bool_del(opts, BLOCK_OPT_COMPAT6, false)) {
-flags |= BLOCK_FLAG_COMPAT6;
+if (strcmp(hw_version, "undefined")) {
+error_setg(errp,
+ "compat6 cannot be enabled with hwversion set");
+ret = -EINVAL;
+goto exit;
+}
+g_free(hw_version);
+hw_version = g_strdup("6");
+}
+if (strcmp(hw_version, "undefined") == 0) {
+g_free(hw_version);
+hw_version = g_strdup("4");
}
fmt = qemu_opt_get_del(opts, BLOCK_OPT_SUBFMT);
if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ZEROED_GRAIN, false)) {
@@ -2001,7 +2013,7 @@ static int vmdk_create(const char *filename, QemuOpts
*opts, Error **errp)
fmt,
parent_desc_line,
ext_desc_lines->str,
- (flags & BLOCK_FLAG_COMPAT6 ? 6 : 4),
+ hw_version,
total_size /
(int64_t)(63 * number_heads * BDRV_SECTOR_SIZE),
number_heads,
@@ -2047,6 +2059,7 @@ exit:
}
g_free(adapter_type);
g_free(backing_file);
+g_free(hw_version);
g_free(fmt);
g_free(desc);
g_free(path);
@@ -2298,6 +2311,12 @@ static QemuOptsList vmdk_create_opts = {
.def_value_str = "off"
},
{
+.name = BLOCK_OPT_HWVERSION,
+.type = QEMU_OPT_STRING,
+.help = "VMDK hardware version",
+.def_value_str = "undefined"
+},
+{
.name = BLOCK_OPT_SUBFMT,
.type = QEMU_OPT_STRING,
.help =
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 10d8759..931a412 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -38,12 +38,12 @@
#include "qemu/throttle.h"
#define BLOCK_FLAG_ENCRYPT 1
-#define BLOCK_FLAG_COMPAT6 4
#define BLOCK_FLAG_LAZY_REFCOUNTS 8
#define BLOCK_OPT_SIZE "size"
#define BLOCK_OPT_ENCRYPT "encryption"
#define BLOCK_OPT_COMPAT6 "compat6"
+#define BLOCK_OPT_HWVERSION "hwversion"
#define BLOCK_OPT_BACKING_FILE "backing_file"
#define BLOCK_OPT_BACKING_FMT "backing_fmt"
#define BLOCK_OPT_CLUSTER_SIZE "cluster_size"
diff --git a/qemu-doc.texi b/qemu-doc.texi
index 79141d3..f37fd31 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -693,6 +693,9 @@ Supported options:
File name of a base image (see @option{create} subcommand).
@item compat6
Create a VMDK version 6 image (instead of version 4)
+@item hwversion
+Specify vmdk virtual hardware version. Compat6 flag cannot be enabled
+if hwversion is specified.
@item subformat
Specifies which VMDK subformat to use. Valid options are
@code{monolithicSparse} (default),
--
1.9.1
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
On Mon, May 2, 2016 at 5:49 PM, Fam Zheng wrote: > Yes. > > But you get: Fair enough, sent another one. Thanks, -- Janne
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
Fam,
Any objections to this one?
--
Janne
On Tue, May 3, 2016 at 12:43 PM, Janne Karhunen
wrote:
> From: Janne Karhunen
>
> Vmdk images have metadata to indicate the vmware virtual
> hardware version image was created/tested to run with.
> Allow users to specify that version via new 'hwversion'
> option.
>
> Signed-off-by: Janne Karhunen
> ---
> block/vmdk.c | 27 +++
> include/block/block_int.h | 2 +-
> qemu-doc.texi | 3 +++
> 3 files changed, 27 insertions(+), 5 deletions(-)
>
> diff --git a/block/vmdk.c b/block/vmdk.c
> index 45f9d3c..955c6b3 100644
> --- a/block/vmdk.c
> +++ b/block/vmdk.c
> @@ -1829,8 +1829,8 @@ static int vmdk_create(const char *filename, QemuOpts
> *opts, Error **errp)
> int64_t total_size = 0, filesize;
> char *adapter_type = NULL;
> char *backing_file = NULL;
> +char *hw_version = NULL;
> char *fmt = NULL;
> -int flags = 0;
> int ret = 0;
> bool flat, split, compress;
> GString *ext_desc_lines;
> @@ -1861,7 +1861,7 @@ static int vmdk_create(const char *filename, QemuOpts
> *opts, Error **errp)
> "# The Disk Data Base\n"
> "#DDB\n"
> "\n"
> -"ddb.virtualHWVersion = \"%d\"\n"
> +"ddb.virtualHWVersion = \"%s\"\n"
> "ddb.geometry.cylinders = \"%" PRId64 "\"\n"
> "ddb.geometry.heads = \"%" PRIu32 "\"\n"
> "ddb.geometry.sectors = \"63\"\n"
> @@ -1878,8 +1878,20 @@ static int vmdk_create(const char *filename, QemuOpts
> *opts, Error **errp)
>BDRV_SECTOR_SIZE);
> adapter_type = qemu_opt_get_del(opts, BLOCK_OPT_ADAPTER_TYPE);
> backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
> +hw_version = qemu_opt_get_del(opts, BLOCK_OPT_HWVERSION);
> if (qemu_opt_get_bool_del(opts, BLOCK_OPT_COMPAT6, false)) {
> -flags |= BLOCK_FLAG_COMPAT6;
> +if (strcmp(hw_version, "undefined")) {
> +error_setg(errp,
> + "compat6 cannot be enabled with hwversion set");
> +ret = -EINVAL;
> +goto exit;
> +}
> +g_free(hw_version);
> +hw_version = g_strdup("6");
> +}
> +if (strcmp(hw_version, "undefined") == 0) {
> +g_free(hw_version);
> +hw_version = g_strdup("4");
> }
> fmt = qemu_opt_get_del(opts, BLOCK_OPT_SUBFMT);
> if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ZEROED_GRAIN, false)) {
> @@ -2001,7 +2013,7 @@ static int vmdk_create(const char *filename, QemuOpts
> *opts, Error **errp)
> fmt,
> parent_desc_line,
> ext_desc_lines->str,
> - (flags & BLOCK_FLAG_COMPAT6 ? 6 : 4),
> + hw_version,
> total_size /
> (int64_t)(63 * number_heads *
> BDRV_SECTOR_SIZE),
> number_heads,
> @@ -2047,6 +2059,7 @@ exit:
> }
> g_free(adapter_type);
> g_free(backing_file);
> +g_free(hw_version);
> g_free(fmt);
> g_free(desc);
> g_free(path);
> @@ -2298,6 +2311,12 @@ static QemuOptsList vmdk_create_opts = {
> .def_value_str = "off"
> },
> {
> +.name = BLOCK_OPT_HWVERSION,
> +.type = QEMU_OPT_STRING,
> +.help = "VMDK hardware version",
> +.def_value_str = "undefined"
> +},
> +{
> .name = BLOCK_OPT_SUBFMT,
> .type = QEMU_OPT_STRING,
> .help =
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index 10d8759..931a412 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -38,12 +38,12 @@
> #include "qemu/throttle.h"
>
> #define BLOCK_FLAG_ENCRYPT 1
> -#define BLOCK_FLAG_COMPAT6 4
> #define BLOCK_FLAG_LAZY_REFCOUNTS 8
>
> #define BLOCK_OPT_SIZE "size"
> #define BLOCK_OPT_ENCRYPT "encryption"
> #define BLOCK_OPT_COMPAT6 "compat6"
> +#define BLOCK_OPT_HWVERSION "hwversion"
> #define BLOCK_OPT_BACKING_FILE "backing_file"
> #define BLOCK_OPT_BACKING_FMT "backing_fmt"
> #define BLOCK_OPT_CLUSTER_SIZE "cluster_size"
> diff --git a/qemu-doc.texi b/qemu-doc.texi
> index 79141d3..f37fd31 100644
> --- a/qemu-doc.texi
> +++ b/qemu-doc.texi
> @@ -693,6 +693,9 @@ Supported options:
> File name of a base image (see @option{create} subcommand).
> @item compat6
> Create a VMDK version 6 image (instead of version 4)
> +@item hwversion
> +Specify vmdk virtual hardware version. Compat6 flag cannot be enabled
> +if hwversion is specified.
> @item subformat
> Specifies which VMDK subformat to use. Valid options are
> @code{monolithicSparse} (default),
> --
> 1.9.1
>
Re: [Qemu-devel] [PATCH] Allow users to specify the vmdk virtual hardware version.
On Fri, May 6, 2016 at 2:05 PM, Kevin Wolf wrote: >> >> Reviewed-by: Fam Zheng > > Thanks, applied to block-next. Great, thanks everyone! -- Janne
virtio via external shared ram
Hi,
I have created an experimental setup for Linux where all the virtio
data structures and traffic can be allocated by the guest from a ram
blob outside of the guest default ram space. That ram blob can be
hotplugged to the guest or defined via the guests device tree.This is
done as some hypervisors, including tdx/sev/pkvm and others, would
probably benefit from a simple security policy that removes all
set_memory_{encrypted,decrypted} calls to open up the guest dma memory
in fragments that are not only likely to leak information due to the
widespread use of the DMA API but also slow things down for no obvious
reason. From the hypervisors point of view the fragmented shadow page
table space is also an unnecessary slowdown and a source of memory
waste.
I have seen forks of SWIOTLB that do similar things, but fundamentally
they are still SWIOTLB behind the curtains and as such unusable for
low latency / high bandwidth applications due to bouncing (copying)
data back and forth into those external buffers. The setup I have
created can act as virtio as it was designed to be, a zero copy data
transport path.
A trial integration into QEMU could probably look something like this
(in virt.c):
..
emem_map = mmap(NULL, EMEM_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED
| MAP_SYNC, fd, 0);
memory_region_init_ram_ptr(sr, OBJECT(machine), "ext-mem",
EMEM_SIZE, emem_map);
..
emem = g_new(MemoryRegion, 1);
memory_region_add_subregion_overlap(sysmem, emem_physaddr, emem, 1000);
..
So the question I have is that did I understand the qemu RAM model
correctly and would something like that lead to known issues
somewhere?
--
Janne
