Re: [Qemu-discuss] Live migration, different instance RAM allocations?
On 1/3/2014 4:33 PM, Scott Sullivan wrote: Hello, Is it possible to perform a live migration where SOURCE and DESTINATION have different RAM sizes (using QEMU/KVM)? I know you can live migrate from hostA to hostB without downtime, when the instances RAM allocation is the same on both ends. Is there anyway to perform the live migration, but with the RAM size on the DESTINATION to be larger than it was on the SOURCE? To clarify, so what I'm wondering is if you can live migrate instance 'blah' from physical hostA with 1GB of RAM allocated, to physical hostB with 2GB of RAM allocated? The closest thing I have thought of is doing a normal live migration, then ballooning the instance up to the new RAM allocation on the DESTINATION. However you can't balloon up past the instances maxMemory it appears when the instance is live: virsh setmaxmem blah --live5242880k error: Unable to change MaxMemorySize error: Requested operation is not valid: cannot resize the maximum memory on an active domain So it doesn't appear ballooning would work for my use case here. It's fine if its not possible, I was just wondering if anyone knew of a way to do this, or has experiences doing something similar, as it seems like a somewhat common desire. The issue is much more fundamental: At bootup, the OS inside the Guest VM asks the BIOS (SEABIOS) how much (simulated) physical ram the virtual machine has. The optional balloon driver can then tell the guest OS that it is using X MB of that memory and temporarily hand that back to the host via qemu, the guest OS will still think the memory is there, it just happens to be used by the "balloon" process. But to tell the guest OS about additional memory beyond what it was told about at boot, the guest OS needs to support "hot-plugging" extra RAM into a computer without reboot, and qemu needs to simulate the hardware that tells the guest OS this has been done. So the real things to check are: - does the OS in your virtual machine support hot-plugging additional memory? - does qemu simulate a form of memory hot-plugging supported by that guest OS? - what is the qemu command to actually simulate such a hot-plugging event? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Overload system function through LD_PRELOAD / Stracing a program in a virtual machine
On 1/10/2014 4:39 PM, Alexandre LAURENT wrote: Hello everyone, I have a library overloading system functions (like connect/open ...) that I am loading with qemu using LD_PRELOAD : LD_PRELOAD=./mylibofoverload.so qemu $qemu_params Would my overloads be used in the program runned in the virtual machine ? No, but you should see the system calls made by qemu to access the files and data on the host system (where the LD_PRELOAD is done). You might not see calls that are optimized inside the kvm kernel module to not go through the qemu user mode code, the workaround for that is to tell qemu not to use kvm for that particular invocation (see man qemu for the correct option to do that). The second part of the question, which I guess is related is : does strace can see and trace the function runned in the virtual machine programs ? No, same situation, an strace of qemu will show you what qemu does to the host system inorder to simulate whatever is happening inside the guest system. Actually, I am not able to do that. I think that is because everything is runned through kvm, but is there a trick/hack allowing me to strace inner programs ? If you want to see the system or library callshappening inside the VM, you need to run LD_PRELOAD / strace inside the VM too. For system calls you might also get away by using the "user space" as opposed to "system" variant of qemu to run a program with the emulated CPU but the system calls and kernel of the host, that should make the system calls made by the program inside qemu appear as nearly identical system calls in an strace of qemu. Because the current documentation link from http://wiki.qemu.org/Manual is dead, I had to base this answer on the copy of the ancient qemu 0.12 documentation which is still online. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] QEMU ARM test image: modifying arm_root.img
On 1/11/2014 2:18 PM, jmf list wrote: Hello, I'm trying to modify the initrd (arm_root.img) for the QEMU ARM test image (http://wiki.qemu.org/Testing#QEMU_disk_images). I am able to extract it using gunzip and cpio, but I can't recreate the archive so that the VM will boot properly (I get a kernel panic). I believe my problem is with how I am invoking cpio: if I simply gunzip/zip the existing initrd cpio archive the VM boots properly, but if I extract and re-create the cpio archive (without modification) the boot fails. So, I was wondering: could you tell me what cpio options you used to create the arm_root.img archive? Pardon for any obvious issues I might be overlooking. I've never delved into initrd before; I am actually just trying to set up automated testing for some ARM code that was contributed to one of my open source projects. I am a bit far down the rabbit hole, I'm afraid. I've googled and searched the mailing lists but wasn't able to find a solution to my problem. Thanks for all the great work regardless, Jon PS Here is how I am extracting the archive, after gunzip: $ cpio -i -vd -H newc < ../arm_root Here is how I am creating it, before gzipping it: $ rm ../arm_root $ find . -depth -print | cpio -o --format=newc > ../arm_root Hmm, have you tried without "-depth" ?, I am not sure the kernel can successfully unpack rootfs files to a directory which has not yet been created Here is a snippet from the console when the kernel panic occurs: ... 802.1Q VLAN Support v1.8 Ben Greear <mailto:gree...@candelatech.com>> All bugs added by David S. Miller <mailto:da...@redhat.com>> VFP support v0.3: implementor 41 architecture 1 part 10 variant 9 rev 0 Root-NFS: No NFS server available, giving up. VFS: Unable to mount root fs via NFS, trying floppy. VFS: Cannot open root device "" or unknown-block(2,0) Please append a correct "root=" boot option Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(2,0) Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] troubleshooting live migration
Just one small idea: Could it be that the network access is lost because some (physical?) networkswitch is not accepting that packets to/from the guest VMs virtual MACaddress shall now be sent to the physical switch port connected to host2,and not the one connected to host1? This obviously wold not directly explain the CPU thrashing, but that might bea secondary effect caused by the loss of network connectivity. Of cause the converse might also be true: the network being fine, but the thrashing CPU preventing the VM from answering pings etc. On 1/14/2014 8:23 AM, Marcus Sorensen wrote: I suppose I can capture a core while it's spinning, if someone is willing to look at it. On Tue, Jan 14, 2014 at 12:20 AM, Marcus Sorensen wrote: Does anyone have tips on troubleshooting live migration? I've got several E5-2650 servers running in test environment, kernel 3.10.26 and qemu 1.7.0. If I start a VM guest (say ubuntu, debian, or centos), I can migrate it around from host to host to host just fine, but if I wait awhile (say 1 hour), I try to migrate and it succeeds but the guest is hosed. No longer pings, cpu is thrashing. I've tried to strace it and don't see anything that other working hosts aren't doing, and I've tried gdb but I'm not entirely sure what I'm doing. I tried downgrading to qemu 1.6.1. I've found dozens of reports of such behavior, but they're all due to other things (migrating between different host CPUs, someone thinking it's virtio or memballoon only to later find a fix like changing machine type, etc). I'm at a loss. This seems to work just fine with stock CentOS builds. Here's an example xml: VM12 dd25acfc-e24d-4de6-814c-72ac465bc208 4194304 4194304 2 2000 /machine hvm destroy restart destroy /usr/bin/qemu-kvm Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Live migration of VM
On 5/23/2014 8:19 AM, Prashant Upadhyaya wrote: Hi, Is it possible to live migrate a VM (emulating AMD) running on native AMD to a machine running native ARM and back with QEMU ? I believe a clarification is needed here. What is it you want to do: A) Migrate the VM from qemu-system-x86_64 software emulating an AMD64 guest on an AMD64 host to qemu-system-x86_64 software emulating the same AMD64 guest on an ARM host. B) Migrate the VM from qemu-system-x86_64 KVM emulating an AMD64 guest on an AMD64 host to qemu-system-x86_64 software emulating the same AMD64 guest on an ARM host C) Migrate the VM from qemu-system-x86_64 emulating an AMD64 guest on an AMD64 host to qemu-system-arm emulating an ARM guest on an ARM host. C) Is mostly impossible, because the Guest memory needs to contain completely different code when the virtual machine suddenly swaps its (virtual) CPU from AMD64 to ARM. A) Might be possible, unless there is some oddity in the qemu code which transmits the virtual machine state differently depending on the Host architecture. B) Might be more difficult, if the qemu code transmits KVM virtual machine state differently from equivalent software emulated virtual machine state. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How do I optimize qemu for old VxWorks 5.2 Guest[was: (no subject)]
On 6/2/2014 11:30 PM, p_j_r_m wrote: Hi I´ve an old 5.2 vxWorks x86 based system running on qemu over Linux.It compiles with Tornado , so it´s quite old.My doubt is ,can this system take benefit of some kind of acceleration?. (Linux goes up to 100% in all CPU cores). Or, being it compiled with such an old "features" is no way to accelerate it? Should i recompile and relink my binaries with newer libraries to take benefit of it ,or this has nothing to do?. I´m quite confused about what type of acceleration,if any, use in this system. The first optimization to try is to use kvm: On the physical machine, use a recent AMD or Intel CPU with the hardware virtualization instruction set extensions (AMD-V or Intel VT-x). Then use a 64 bit Linux kernel with the kvm modules loaded or compiled in. Finally invoke qemu with the -enable-kvm option to take advantage of this hardware accelleration. The second optimization is to find a VxWorks boot/run/configuration option which executes the HLT instruction whenever the VxWorks kernel is just idle waiting for an interrupt. On a physical hardware CPU (any age), this will save electricity and heat by making the CPU use less power when idle. On a virtual CPU, this will release the CPU for use by other Linux processes (or allow Linux to save power by using the HLT instruction on the real CPU). If you do not find a VxWorks option to invoke HLT, there might be an option to call some legacy BIOS function in its place, there is a good (but not perfect) chance that the SeaBIOS provided by qemu will map that legacy BIOS function to a similar release of the CPU back to Linux. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How do I optimize qemu for old VxWorks 5.2 Guest[was: (no subject)]
On 6/4/2014 8:25 PM, p_j_r_m wrote: So it seems acceleration is possible... why do you say "use a 64 bit Linux kernel"? Why can´t I use a i386 kernel ? would be a better performance on x86_64 machine ? Thank you very much. The kvm module uses a 64 bit mechanism in the x86_64 instruction set, which I don't think (I am not certain though) is available at all in the 32 bit x86 instruction set. So I strongly suspect that it is simply not available in i386 kernels. Others on this list: Please correct me if I am wrong. El Miércoles 4 de junio de 2014 19:10, Jakob Bohm escribió: On 6/2/2014 11:30 PM, p_j_r_m wrote: > Hi > I´ve an old 5.2 vxWorks x86 based system running on qemu over Linux.It > compiles with Tornado , so it´s quite old.My doubt is ,can this system > take benefit of some kind of acceleration?. (Linux goes up to 100% in > all CPU cores). Or, being it compiled with such an old "features" is no > way to accelerate it? Should i recompile and relink my binaries with > newer libraries to take benefit of it ,or this has nothing to do?. > I´m quite confused about what type of acceleration,if any, use in this > system. The first optimization to try is to use kvm: On the physical machine, use a recent AMD or Intel CPU with the hardware virtualization instruction set extensions (AMD-V or Intel VT-x). Then use a 64 bit Linux kernel with the kvm modules loaded or compiled in. Finally invoke qemu with the -enable-kvm option to take advantage of this hardware accelleration. The second optimization is to find a VxWorks boot/run/configuration option which executes the HLT instruction whenever the VxWorks kernel is just idle waiting for an interrupt. On a physical hardware CPU (any age), this will save electricity and heat by making the CPU use less power when idle. On a virtual CPU, this will release the CPU for use by other Linux processes (or allow Linux to save power by using the HLT instruction on the real CPU). If you do not find a VxWorks option to invoke HLT, there might be an option to call some legacy BIOS function in its place, there is a good (but not perfect) chance that the SeaBIOS provided by qemu will map that legacy BIOS function to a similar release of the CPU back to Linux. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] qemu -kernel u-boot.bin
On 6/10/2014 6:57 PM, Matwey V. Kornilov wrote: 2014-06-10 20:54 GMT+04:00 Dale R. Worley : │0x600c <_start+12> ldrpc, [pc, #6397080] ; 0x6018 <_start+24> │ That last instruction looks to me as if it is a Load Register into the PC, which will cause the next instruction to be fetched from whatever the new value of PC points to. And this value is no way 0x1. Actually, this instruction reads the goto address from memory at address 0x6018 (or more precisely: the address you loaded it at + 0x0018). So offset 0x0018 into the u-boot image is probably 0x6001, and if that is the wrong place, then the load address was wrong. Maybe the initial ELF relocation is being done in an unexpected way, given that u-boot is meant to be loaded directly from flash without the benefit of an ELF loader loading before it. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] qemu-img convert
On 7/1/2014 9:20 AM, Upendra Sahu wrote: Hi, When I am using to convert VMDK to VHD (format vpc), 1.Why its creating new file with new format, can I change original file itself to new format, if its creating new file it means its copying all the block of disk image and also changing the metadata, headers? Because if anything goes wrong, it would be a disaster to be left with a file that is not valid in any format whatsoever. Also converting between arbitrary file formats in place is non-trivial because one format may want two pieces of data in the opposite order of what the other format wants. 2.When I converting 8GB vmdk to vhd (format vpc) I see converted image size as around 4GB, how its changing the size. Many of the formats handled by qemu-img include special efficient methods of storing sectors that contain nothing but zeroes. This is what VMWare refers to as "thin" disks. 3.In conversion process, does qemu-img removes free sectors and defragment the virtual disk? Can I disable that and just create new metadata for output format image to reduce the conversion time? No, it doesn't touch the file system inside the virtual disk, but it does copy all the sectors and regenerate the tables that say which sector is stored where in the file. There are other speed problems inside the qemu tools, which are most annoying to those of us who use them every day, but that is a different discussion. Thanks & Regards, Upendra Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Any one have successfully make window 7 guest in kvm use more than 2 cpus/cores?
On 7/23/2014 6:22 PM, Sai Prajeeth wrote: Try setting sockets=6,cores=1? I had this issue when running Solaris as guest. For some reason Solaris never detected any of the cores. It could only detect the sockets. Unlikely to work. Windows 7 (unlike the more expensive server licenses for the same kernel) enforces a license restriction of max 2 physical CPUs, each with unlimited cores (subject to the kernel design maximum of at least 32). But try looking at the command line of the running qemu process. Does it actually specify 2 sockets with 4 cores each, or were the settings somehow mistranslated from libvirt XML to qemu command line? Also, if you are using a computer with more than 2 physical sockets, you may want to check if kvm passes through the physical or the command line CPU topology to the guest. On Wed, Jul 23, 2014 at 9:38 PM, Scott Zhang mailto:macromars...@gmail.com>> wrote: __ __ Dear all: I have been trying this for whole afternoon and whole night. I have installed windows 7 as guest in kvm using virt-manager which use qemu. And after I alloc it 6 vcpus, I noticed win7 only see 2 cpus in task manager. But in device manager, it shows 6 cpus. After google a lot, looks many people say they workaround this by setting the topology of CPU, which is technical correct. But when I try to set sockets=2,cores=4 and several vairable pairs. None is working. Win7 only see 2 at most. I am using fedora 20 with kernel 3.15 and the default qemu. Can any one help? Thanks Regards 2014-07-24 Scott Zhang __ Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Any one have successfully make window 7 guest in kvm use more than 2 cpus/cores?
On 7/24/2014 4:24 AM, Tony Su wrote: Aside from verifying/supporting the previous posts which state that a Desktop OS like Win7 only supports 2 CPUs, I wonder why the effort to even deploy on multiple CPU. Today's OS (all I know of) fully take advantage of all physical CPUs no matter how many (or few) virtual CPUs are configured. The only reason I can conceive of to configure multiple CPUs is to attempt to partition running processes on physical CPUs (not virtual) which I highly doubt would be an objective running any app on Win7 (or similar desktop). The point isn't to partition. It is to make the guest OS take advantage of more CPUs, thus running faster. The fact that different guest OS-es have different topology restrictions causes frequent problems configuring the virtual topology presented to them by virtual machines such as qemu-tc, qemu-xen, qemu-kvm, VirtualBox, VMware and HyperV. On Wed, Jul 23, 2014 at 7:23 PM, Tony Su wrote: http://social.technet.microsoft.com/Forums/windows/en-US/970c999c-8ffc-4611-968c-7d0ceffbedd4/max-number-of-cpu-cores-that-windows-7-64-bit-will-recognize?forum=w7itprohardware Depending on your version of Windows 7, your license supports either one or two CPU and any number of cores. Tony On Wed, Jul 23, 2014 at 6:45 PM, Scott Zhang wrote: that is interesting. i use my opensuse 13.1 with i7 4cores.cpu and run win7 as guest. win7 show 4 cpus correctly. guess the problem with fedora kvm or the xeon cpu. will try later to confirm On Jul 24, 2014 12:35 AM, "Jakob Bohm" wrote: On 7/23/2014 6:22 PM, Sai Prajeeth wrote: Try setting sockets=6,cores=1? I had this issue when running Solaris as guest. For some reason Solaris never detected any of the cores. It could only detect the sockets. Unlikely to work. Windows 7 (unlike the more expensive server licenses for the same kernel) enforces a license restriction of max 2 physical CPUs, each with unlimited cores (subject to the kernel design maximum of at least 32). But try looking at the command line of the running qemu process. Does it actually specify 2 sockets with 4 cores each, or were the settings somehow mistranslated from libvirt XML to qemu command line? Also, if you are using a computer with more than 2 physical sockets, you may want to check if kvm passes through the physical or the command line CPU topology to the guest. On Wed, Jul 23, 2014 at 9:38 PM, Scott Zhang mailto:macromars...@gmail.com>> wrote: __ __ Dear all: I have been trying this for whole afternoon and whole night. I have installed windows 7 as guest in kvm using virt-manager which use qemu. And after I alloc it 6 vcpus, I noticed win7 only see 2 cpus in task manager. But in device manager, it shows 6 cpus. After google a lot, looks many people say they workaround this by setting the topology of CPU, which is technical correct. But when I try to set sockets=2,cores=4 and several vairable pairs. None is working. Win7 only see 2 at most. I am using fedora 20 with kernel 3.15 and the default qemu. Can any one help? Thanks Regards 2014-07-24 Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Is SSE2 implemented with hardware support ?
On 7/30/2014 9:14 AM, 邓尧 wrote: Hello, I need to optimize a Linux program running under qemu-system-i386. qemu is compiled with KVM support. File /proc/cpuinfo in the guest shows that SSE2 is supported. If SSE2 is backed with host hardware, using SSE2 may improve my program's performance significantly. The host is x86_64 Linux. My question is: whether sse2 is backed with host hardware ? If it is, how much overhead does an SSE2 instruction have ? (compared with an SSE2 instruction on the host) In KVM and XEN modes, almost every CPU instruction is executed directly by the physical host CPU. Only parts of memory management, and a few system administration instructions and (on some CPUs) virtual real mode 16 bit (DOS) modes involve the emulation software. So mathematical SSE2 operations should run at full hardware speed. Cache preload SSE/MMX operations may have different performance than on the host. Exact relationships depend on the specific CPU chip design and how the chip designers implemented the VM chip extensions inside each CPU core. P.S. Note that even if qemu-system-i386 is compiled with KVM support, KVM is only active if specified on the command line AND the kvm modules are loaded in the host kernel AND VM extensions are enabled in the host CPU AND it actually works. Otherwise qemu-system-i386 falls back to its usual software emulation. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] manipulating raw disk image as non-root user
es. Jenkins won't allow it. I am afraid that the last few suggestions above have all been about really getting (partial) root privileges in various roundabout ways. That is a bad idea. The whole reason why tools such as Jenkins don't allow root is to ensure that an evil patch to some component (local or upstream) cannot compromise the build server itself, and from there all machines that run software built by that build server. So here are some secure ways to create and manipulate disk images without any kind of root or other special privileges: 1. Formatting tools such as mkfs can be told to format a raw partition image file directly by simply passing the filename where a disk device is normally specified, plus sometimes adding an option not to complain about that. 2. Partition images can be combined into disk images by clever use of dd. 3. For some file systems there are programs that can create disk images from a file tree. The classic being mkisofs for CD images, but others have been made. For FAT file systems, the classic mtools can read and write files in a raw image without root privileges. 4. The other option is to use qemu: Using the other techniques above, create a bootable file system which will format and populate the second hard drive of a virtual machine. Then run qemu with the creator file system as hard drive 1 and an empty raw disk image as hard drive 2. When the creator system completes, it should create some kind of "OK" file and shut down its machine, causing qemu to exit. Then your build system (Jenkins) can check if the OK file was created or not before proceeding to compress or otherwise package the image file. Be sure to configure qemu to redirect the "console tty" of the creator system to qemu's own stdout, so you can see the output in the build log. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] disk image: self-organized format or raw file
On 8/12/2014 1:51 AM, 吴兴博 wrote: Hello, The introduction in the wiki page present several advantages of qcow2 [1]. But I'm a little confused. I really appreciate if any one can give me some help on this :). (1) Currently the raw format doesn't support COW. In other words, a raw image cannot have a backing file. COW depends on the mapping table on which we it knows whether each block/cluster is present (has been modified) in the current image file. Modern file-systems like xfs/ext4/etc. provide extent/block allocation information to user-level. Like what 'filefrag' does with ioctl 'FIBMAP' and 'FIEMAP'. I guess the raw file driver (maybe block/raw-posix.c) may obtain correct 'present information about blocks. However this information may be limited to be aligned with file allocation unit size. Maybe it's just because a raw file has no space to store the "backing file name"? I don't think this could hinder the useful feature. The ability to store a backing file name or other per-image metadata (such as emulated disk geometry) is not that critical, as this could simply be provided in a config file or command line option. In fact there are scenarios where the storage of this data inside the diskimage files is actually a problem. However the present/not-present feature of "sparse" files (on almost any file system) is subject to the rule that both the file system and basic file utilities (/bin/cp, /bin/tar etc. etc.) are allowed to arbitrarily convert byte ranges back and forth between all-zeroes and "bytes absent". This in term means that there will be no reliable way for qemu etc. to distinguish between a hole that should be mapped to the content of the next cow level down and byte ranges where non-zero bytes in the next cow level down have been overridden by bytes with the value zero. Thus to reliably maintain a COW overlay which lists some byte ranges as being "whatever the next level closer to the base file says" and others as being "some value (maybe zero) stored in the cow overlay". All the various COW file formats supported by qemu have been designed as different solutions to this requirement. qcow2 is the "native" qemu format and as such is supposed to always have the most complete feature set and the most optimised and tested implementation details. (2) As most popular filesystems support delay-allocation/on-demand allocation/holes, whatever, a raw image is also thin provisioned as other formats. It doesn't consume much disk space by storing useless zeros. However, I don't know if there is any concern on whether fragmented extents would become a burden of the host filesystem. A much more significant issue is the lack of a standard POSIX API for changing a byte range in a file from "non-hole allocated space" to "sparse hole with no allocated disk space". This makes it hard to release disk space previously consumed by the guest filesystem. When a COW file format is used, that key opearation is now an internal detail independent of the underlying host file system, host kernel version etc. (3) For compression and encryption, I'm not an export on these topics at all but I think these features may not be vital to a image format as both guest/host's filesystem can also provide similar functionality. I agree with you there. (4) I don't have too much understanding on how snapshot works but I think theoretically it would be using the techniques no more than that used in COW and backing file. A full featured snapshot (as supported by qcow2 and several non-qemu VM systems) consists of two things representing a single point in time: A split between a (possible itself COW) base image representing the virtual disk content at time T and a COW overlay representing any later changes to the virtual disk content. AND a stored copy of the virtual physical memory and virtual hardware device states at time T. Together these two items allow a virtual machine to be "instantaneously" reverted to whatever it was at time T and continue running from that point as if nothing had happened. After all these thoughts, I still found no reason to not using a 'raw' file image (engineering efforts in Qemu should not count as we don't ask for more features from outside world). I would be very sorry if my ignorance wasted your time. references: [1] http://en.wikibooks.org/wiki/QEMU/Images#Image_types "QEMU supports several image types. The "native" and most flexible type is /qcow2/, which supports copy on write <http://en.wikibooks.org/wiki/QEMU/Images#Copy_on_write>, encryption, compression, and VM snapshots. Cheers! 吴兴博 Wu, Xingbo mailto:wux...@gmail.com>> Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Guests don't start with qemu 2.1.0: smbios_get_tables: Assertion `smbios_smp_sockets >= 1' failed
On 8/12/2014 8:40 PM, Brian Rak wrote:
I've been testing 2.1.0, and qemu will no longer start up with our CPU
configuration. I get the error:
qemu-kvm: /root/qemu/src/hw/i386/smbios.c:825: smbios_get_tables:
Assertion `smbios_smp_sockets >= 1' failed.
The relevant parts of my command line:
-smp 4,sockets=2,cores=12,threads=2
This is ultimately coming from my libvirt config of:
4
Is this configuration no longer supported? Up until 2.1.0, this was
handled well.. for any CPU count < 12, I got one socket with the correct
number of cores. For anything over 12 vcpus, I got two sockets. This
makes Windows licensing happy.
I took a look at the code, and it seems like it should just cap the
minimum number of sockets at 1, rather then this assert.
--- ../src_clean/hw/i386/smbios.c 2014-08-01 10:12:17.0 -0400
+++ hw/i386/smbios.c2014-08-12 14:39:40.560485901 -0400
@@ -822,7 +822,7 @@
smbios_build_type_3_table();
smbios_smp_sockets = smp_cpus / (smp_cores * smp_threads);
-assert(smbios_smp_sockets >= 1);
+ if (smbios_smp_sockets < 1) smbios_smp_sockets = 1;
for (i = 0; i < smbios_smp_sockets; i++) {
smbios_build_type_4_table(i);
Is there a downside to this patch?
Have you checked why the variable is 0 and not 1 (or 2), maybe there is
a deeper bug which leaves it at 0, even when it should be 2 (or more).
Remember, the place that reports an error is rarely the cause of the
problem.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com
Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Guests don't start with qemu 2.1.0: smbios_get_tables: Assertion `smbios_smp_sockets >= 1' failed
On 8/13/2014 3:22 PM, Brian Rak wrote:
On 8/13/2014 7:33 AM, Jakob Bohm wrote:
On 8/12/2014 8:40 PM, Brian Rak wrote:
I've been testing 2.1.0, and qemu will no longer start up with our CPU
configuration. I get the error:
qemu-kvm: /root/qemu/src/hw/i386/smbios.c:825: smbios_get_tables:
Assertion `smbios_smp_sockets >= 1' failed.
The relevant parts of my command line:
-smp 4,sockets=2,cores=12,threads=2
This is ultimately coming from my libvirt config of:
4
Is this configuration no longer supported? Up until 2.1.0, this was
handled well.. for any CPU count < 12, I got one socket with the correct
number of cores. For anything over 12 vcpus, I got two sockets. This
makes Windows licensing happy.
I took a look at the code, and it seems like it should just cap the
minimum number of sockets at 1, rather then this assert.
--- ../src_clean/hw/i386/smbios.c 2014-08-01 10:12:17.0
-0400
+++ hw/i386/smbios.c2014-08-12 14:39:40.560485901 -0400
@@ -822,7 +822,7 @@
smbios_build_type_3_table();
smbios_smp_sockets = smp_cpus / (smp_cores * smp_threads);
-assert(smbios_smp_sockets >= 1);
+ if (smbios_smp_sockets < 1) smbios_smp_sockets = 1;
for (i = 0; i < smbios_smp_sockets; i++) {
smbios_build_type_4_table(i);
Is there a downside to this patch?
Have you checked why the variable is 0 and not 1 (or 2), maybe there is
a deeper bug which leaves it at 0, even when it should be 2 (or more).
Remember, the place that reports an error is rarely the cause of the
problem.
Enjoy
Jakob
My command line is this:
-smp 4,sockets=2,cores=12,threads=2
Which means, we get:
smbios_smp_sockets = 4 / (12 * 2) = 0.1666
Which means the assert is correct. This has been a valid configuration
of CPUs/cores up until 2.1.0. I'd have to go update the config on all
of our instances if I wanted to upgrade without this patch :/
Ah, the bug is that the handling of fractional results is wrong, and
your code just hides it for the case where the result is between 0 and
1, while doing nothing for fractional results above 1.0.
If smbios_smp_sockets is a floating point type (float or double), then
the correct fix is to change the assert to
assert(smbios_smp_sockets > 0.0);
If smbios_smp_sockets is an integer type (int, unsigned, etc.), then the
correct fix is to make the division round up and keep the assert(), like
this:
assert(smp_cpus >= 1); // Next calculation assumes this
smbios_smp_sockets = 1 + (smp_cpus - 1) / (smp_cores * smp_threads);
assert(smbios_smp_sockets >= 1);
Otherwise, consider what happens with the following option settings:
-smp 28,sockets=2,cores=12,threads=2
Here the original division would result in 1.1666. Which needs to be
treated as floating point or rounded up to 2 sockets.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com
Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Guest execution statistics from QEMU
On 8/18/2014 2:26 PM, Mie Ite wrote: Hi everyone, I would like to know the following information of the guest application executed in QEMU: - the number of guest instructions executed per each function call - the number of memory access instructions - the address and size of each executed load / store It it possible to extract any of the above from stock QEMU currently or should I start to look at the source code? Probably not. Qemu (like most other virtual machine engines) are trying to run the guest code as fast as possible without letting it touch the host system, so counting individual instructions would be unwanted overhead. Also, to qemu, the function calls in the guest application don't really exist, only the various instructions, some of which happen to implement the function call mechanism of a higher level language. Also these particular statistics are so rarely wanted for modern CPU architectures, where the execution "cost" (time, energy) of individual instructions is highly variable due to hardware features such as superscalar execution, pipelining, microcode, caching etc. All that said, I don't know if the TCG emulator in qemu (its slowest mode) can be easily configured or patched to collect some statistics about each emulated instruction. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Write serial in qemu and read in target
On 28/08/2014 19:05, Brandon Williams wrote: Hi there, I'm trying to figure out a way that I can write to the serial port in a thread that I have made in mac_oldworld and be able to read that serial data from my target program (currently a modified OpenBIOS). I am writing to the essc in my mac_oldworld thread with this: io_mem_write(escc_mem, 48, 36, 1); I would like to be able to read that byte on the target (OpenBIOS) side. Does anyone have any suggestions? You are not being clear which of those "worlds" are running in qemu virtual machines, and which are not. But assuming the two programs are running in two different qemu virtual machines, there are qemu options to connect virtual serial ports over TCP/IP. For example at one end (the virtual machine started first): -chardev socket,id=charserial0,host=127.0.0.1,port=12345,server,nowait -device isa-serial,chardev=charserial0,id=serial0 And at the other end: -chardev socket,id=charserial0,host=127.0.0.1,port=12345 -device isa-serial,chardev=charserial0,id=serial0 (Choose your own port number in place of 12345). My example -device options specifies PC style serial ports, replace with the equivalent option for Mac serial ports. Enjoy Jakob
Re: [Qemu-discuss] What does "COW" mean?
On 02/09/2014 10:33, shhuiw wrote: Hi, I'm new to qemu community, and I'm trying the COW image format (old but simple:-). I have read through its source code, and didn't find anything about 'copy on write'. I wonder wthat "COW" stands for? COW = Copy On Write is a general computing technique where something (in this case a disk image file) doesn't really copy data until someone actual tries to write to (change) their copy, then at that very moment, it makes a real copy for that someone to write to. In qemu context this is used when you make a "snapshot" copy of an entire virtual disk and tells qemu that the running virtual machine should only write to one of the copies while the other copies are kept unchanged for any reason (such as backup). When you tell qemu to do this with a COW disk image format, it just makes copies some data about where different parts of the disk image are stored in the file, and sets some flags such that when the virtual machine actually writes to the disk image, it is given its own copy (in the COW file) of the part of the disk it has written to. For the (usually at least 90%) of the disk that the virtual machine doesn't write to, it just reads back from the original copy without wasting time and disk space on an actual copy. As a bonus feature, some recent versions of the qcow2 file format also uses this technique for any part of the disk that contains all zeroes, which saves space as long as the virtual machine doesn't completely fill its virtual disk. Reading back zeroes from nowhere is faster and cheaper than reading zeroes from a real disk, making this a win-win in most situations. Enjoy Jakob
Re: [Qemu-discuss] Fwd: My progress with compiling QEMU Native on W32 Win7 Pro.
On 16/09/2014 22:19, Paul Gydos wrote: -- Forwarded message -- From: *Stefan Weil*mailto:s...@weilnetz.de>> Date: Tue, Sep 16, 2014 at 10:19 AM Subject: Re: My progress with compiling QEMU Native on W32 Win7 Pro. To: Paul Gydos mailto:p...@gydos.com>> Am I already a root user? On Windows? No. You are "paul" or whatever your Windows account is. Actually, on Windows there are various degrees on being "root" 1. The real "root" is the account called "Local System" or just "SYSTEM" uid S-1-1-18. This has almost the same powers as root on Linux, for instance it runs the equivalents of /sbin/init (smss.exe), /usr/sbin/login (winlogon.exe) and the X server "csrss.exe" . This is very rarely needed and is only for real experts. 2. Having "Administrators" gid S-1-5-32-544 as an active gid for your process. This grants you root equivalent write permissions to most system settings and install locactions, including the equivalents of /etc (Registry branch HKLM), /opt (C:\Program Files), /usr (C:\Program Files\Common Files), /usr/local (ditto), /var (C:\ProgramData), /bin, /lib and /sbin (C:\Windows\System32), /lib/modules (C:\Windows\System32\Drivers), /usr/share (not separate from /usr), /usr/bin32 (C:\Program Files (x86)), /bin32 (C:\Windows\SysWow64) etc. etc. 3. Having "Administrators" gid as a "passive" gid for your process. This is like being in the wheel group on some POSIX systems: You can run processes with "Administrators" enabled by either explicitly selecting "Run As Administrator" or by choosing program files with a special attribute in an embedded XML manifest (like being suid root on POSIX, but you are usually prompted first to check if it was really you, and not some virus, that launched it). 4. Being reported as "root" (unix uid 0) by whatever POSIX emulation layer (mingw32, MSYS, Cygwin, etc.) you run a ported unix tool under. This could be for everyone, for those in one of the above 3 categories or configurable. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] (no subject)
On 17/09/2014 18:20, Priyanka Ranjan wrote: Hello Experts, I am using CentOS 6.5. I am getting an issue with libguestfs (qemu-kvm) # /usr/libexec/qemu-kvm -nographic -machine accel=kvm:tcg -device \? open /dev/kvm: No such file or directory failed to initialize KVM: Operation not permitted Back to tcg accelerator. Could not allocate dynamic translator buffer Check your kernel and dev dir. # lsmod | grep kvm # Output should look like this, execpt for the numbers and # intel may be amd: kvm_intel 134469 19 kvm 392084 1 kvm_intel # ls -l /dev/kvm # Output should look like this, but the date is not important: crw-rw---T 1 root kvm 10, 232 Sep 18 2014 /dev/kvm I saw this issue listed in libguestfs faq "http://libguestfs.org/guestfs-faq.1.html"; and executed "setsebool -P virt_use_execmem=on" as a solution but that also did not help. I confirmed this solution with libguestfs mailing list. They asked me to to write to you. Could you please help me. Thanks a lot in advance, Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] need help in setting up tap networking for using qemu virtual machine
#!/bin/sh # # Script to bring down and delete bridge br0 when QEMU exits # # Bring down eth0 and br0 # set -x /sbin/ifdown eth0 /sbin/ifdown br0 /sbin/ifconfig br0 down # # Delete the bridge # /usr/sbin/brctl delbr br0 # # bring up eth0 in "normal" mode # /sbin/ifconfig eth0 -promisc /sbin/ifup eth0 # # delete the tap device # /usr/sbin/openvpn --rmtun --dev $1 # # start firewall again # #/sbin/service firestarter start end -- and I start my virtual machine with this command : qemu-system-sparc -M ALDEBARAN -serial stdio -kernel telnetd/o-optimize/telnetd.exe -net nic -net tap,ifname=tap0 (this makes /etc/qemu-ifup tap0 and /etc/qemu-ifdown tap0 run before and after qemu run.) When I run above command, I can see the virtual machine's code is running. (it's RTEMS network-demo program. telnetd.exe) From outside the host, I can ping to host and the guest. So I thought everything is fine But I then I found something is wrong.. When I telnet to 129.254.132.47, it is centOS as expected. But when I telnet to 129.254.132.48, it takes unusually long (like 10 seonds?) then I see the loging prompt coming from CentOS.(the host) I can log into it, and it's my host machine. (By the way, both telnetting looked hampered by something, like interfering one another by wrong setting..) I can run the same telnetd program using actual board the qemu emulator is emulating, and I can telnet to the board (and of course it's RTEMS). Only, when using actual board, I use physical network cable between my host and the board. (host 10.10.10.1 and the board 10.10.10.11) I tried setting ETH0IP to 129.254.132.47 in qemu-ifup script but it didn't work. Please anyone tell me what the problem is.. Kind regards, Chan Kim Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Help with -kernel option (unable to mount root fs)
ot tainted 3.17.0-rc6+ #1 [2.282071] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140617_173321-var-lib-archbuild-testing-x86_64-tobias 04/01/2014 [2.282071] task: 88003daa ti: 88003daa8000 task.ti: 88003daa8000 [2.282071] RIP: 0010:[] [] panic+0x1c5/0x209 [2.282071] RSP: :88003daabd70 EFLAGS: 0282 [2.282071] RAX: 0057 RBX: 818ba3c0 RCX: 0057 [2.282071] RDX: RSI: 0046 RDI: 0046 [2.282071] RBP: 88003daabde8 R08: 67b8 R09: 0096 [2.282071] R10: 81b177c0 R11: 018b R12: 88003daabdf8 [2.282071] R13: R14: R15: 88003d71 [2.282071] FS: () GS:88003fc0() knlGS: [2.282071] CS: 0010 DS: ES: CR0: 8005003b [2.282071] CR2: CR3: 019c4000 CR4: 06f0 [2.282071] DR0: DR1: DR2: [2.282071] DR3: DR6: DR7: [2.282071] Stack: [2.282071] 0010 88003daabdf8 88003daabd90 724949f3 [2.282071] 724949f3 88003daabe08 0012 [2.282071] 000a fffb 88003d71 [2.282071] Call Trace: [2.282071] [] mount_block_root+0x2a4/0x2b3 [2.282071] [] mount_root+0x53/0x56 [2.282071] [] prepare_namespace+0x13c/0x174 [2.282071] [] kernel_init_freeable+0x23d/0x261 [2.282071] [] ? rest_init+0x80/0x80 [2.282071] [] kernel_init+0xe/0xf0 [2.282071] [] ret_from_fork+0x7c/0xb0 [2.282071] [] ? rest_init+0x80/0x80 [2.282071] Code: c3 64 eb b1 83 3d 15 35 70 00 00 74 05 e8 1e 32 a1 ff 48 c7 c6 40 71 d8 81 48 c7 c7 60 be 8c 81 31 c0 e8 64 02 00 00 fb 66 66 90 <66> 66 90 45 31 e4 e8 3a 49 aa ff 4d 39 ec 7c 18 41 83 f6 01 44 [2.282071] RIP [] panic+0x1c5/0x209 [2.282071] RSP [2.282071] ---[ end trace 5c54a652a799b4be ]--- Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Virtio-blk and trim
It makes a huge amount of sense for any backend that has sparse storage, including: - qcow2 (and all other cow formats). - raw files on any host file system that supports sparse files. - anything which caches the (virtual) sectors host side. - anything which supports discard/trim/puncture/whateveryoucallit at a lower level This pretty much covers all the qemu storage backends. And what makes it so hard to add commands to virtio-blk, given that any host not issuingdiscard commands will continue to function normally, and presuming there is a way todetect the qemu version from inside the host and/or a way to issue commands that the host qemu might ignore if unimplemented. On 01/10/2014 22:24, Tony Su wrote: I don't know that it makes any sense to run a TRIM/discard function in a virtual file system. It only makes sense for the command to be applied directly to the disk at the hardware I/O level. So, if you run the command on your HostOS regularly, it should addres the needs for all GuestOS as well as the HostOS. Tony On Wed, Oct 1, 2014 at 1:19 PM, George Shuklin wrote: It's better to name it 'discard', because TRIM a very specific ATA command. It done completely different in SCSI and slightly different in SD cards. But linux call it 'discard', or, to be precise, BLKDISCARD (name of ioctl). On 10/01/2014 02:21 AM, Brian Rak wrote: Are there any plans to bring trim support to Virtio-blk? We've got a couple things that would be made nicer by having working trim support. I see that it's implemented in virtio-scsi, but there's a decent performance hit for switching over to that. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Installed Ubuntu 14.04 and now getting "unknown OS type hvm" on my old VMs.
On 06/10/2014 04:13, Hans Deragon wrote: Greetings, I installed Ubuntu 14.04 64 bits and am now getting "unknown OS type hvm" when I try 'virsh define ' on my old VMs. A search trough the web suggest that kernel modules are not loaded. I my case, they are: # lsmod | fgrep kvm kvm_intel 143109 0 kvm 451511 1 kvm_intel Let me take a wild guess: your .xml file specifies the use of kvm in an oldway which is no longer supported by your new versions of libvirt and qemu. The phrase "hvm" (if that is how it is spelled in the XML) may refer to a XEN hvm setup, not a Linux kvm setup. -- Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] passthrough keyboard/mouse
On 31/10/2014 16:13, Tobias Specht wrote: Hi, I'm running Fedora 21 Alpha with qemu version 2.1.2 and linux kernel 3.16 an a notebook with intel VT-d. Forwarding the graphics card is working really well. The guest system is Windows 7 x64. The only devices that are missing inside the VM are the internal keyboard and mouse. These devices are NOT connected via USB to the host instead they are connected via PS/2 [1.045386] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12 [1.047179] i8042: Detected active multiplexing controller, rev 1.1 [1.048002] serio: i8042 KBD port at 0x60,0x64 irq 1 [1.048006] serio: i8042 AUX0 port at 0x60,0x64 irq 12 [1.048025] serio: i8042 AUX1 port at 0x60,0x64 irq 12 [1.048040] serio: i8042 AUX2 port at 0x60,0x64 irq 12 [1.048055] serio: i8042 AUX3 port at 0x60,0x64 irq 12 [1.048186] mousedev: PS/2 mouse device common for all mice Disabling legacy USB support in the BIOS doesn't change anything keyboard/mouse are still connected via PS/2. The PS/2 controller is directly connected to the chipset which appears on the host as an PCI device: 00:1f.0 ISA bridge: Intel Corporation QM77 Express Chipset LPC Controller (rev 04) 00:1f.2 SATA controller: Intel Corporation 7 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04) I could imagine that it is not possible to passthrough the chipset to the virtual machine. I want to use the internal keyboard/mouse inside the virtual machine because I wont to use the notebook without the need for an external keyboard. Does qemu provide a possibility to share the host keyboard and mouse with a guest when not using a virtual display? As a more obvious alternative: Does qemu provide a way to pass the host attachedPS/2 keyboard/mouse through to the virtual PS/2 keyboard/mouse of the virtualPC chipset? Note that these areessentially character devices, although only the mouse may bedirectly exposed (as /dev/psaux) by the host kernel. Perhaps some of the options used when qemu powers the XEN Dom0 handling of XEN DomUI/O can do this? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
[Qemu-discuss] Why are qcow2 internal snapshots (savevm and otherwise) still so slow
This is just a burp of frustration, after years of running qemu+kvm. I am currently running qemu-system version 2.1.0 (Debian package 2.1+dfsg-2~bpo70+2), and after several years of ongoing bug reports (and various patches)from others, snapshots are still painfully slow. Even simple snapshot operations take large amounts of time. Creating and removing snapshots from a running VM makes the VM unresponsive (pings etc. fail), often for as much as 30 minutes to an hour. Note that this is happening on a production system, so experimentation is limited and unsafe cache settings are just not an option. In one qemu bug tracker (not sure if it is the current one), I found reports and patches for at least 3 internal issues related to this: 1. Unnecessary disk flushes for each update of each entry in some qcow2 file format tables(I think it was the L1 or L2 tables). It seems that functions intended to perform and flush single allocations are being misused by calling them in a loop for the bulk allocations involved in snapshots. 2. Failure to remove a temporary copy of the savevm memory image from the HEAD of the snapshot tree (not sure why it was written there and not in the snapshot itself). 3. Metadata pre-allocation not surviving snapshot creation/removal. In addition to these bug reports, I have noticed in other documents, that snapshot-related features, such as streaming blocks to combine snapshots are inexplicably designed only to cover non-typical cases in terms of the direction blocks are copied, compared to what is actually needed by the snapshot commands exposed in the user interfaces. There is also a lack of clear rules as to how the qcow2 format handles being backed up while "live" and later restored, then jumped to a snapshot made just before the backup (this is the standard scenario for snapshot-based backups and restores). For instance this may cause the "reference count" fields in a restored file to be out of sync with the referring tables, if one or more blocks were written while the backup program was reading the qcow2 file sequentially. It seems strange that such a basic operation, using the native qemu file format, isn't considered apriority in terms of reliability and performance. P.S. In case you didn't know, the standard way to backup virtual machines (qemu orotherwise) is this sequence: - Create snapshot named "Backup #"using savevm. - Sequentially copy disk image file using a tool such as gnu tar. (Any byte ranges that change due to the running VM may get backed up with their values at any time during this copy operation). - Remove snapshot named "Backup #" After the disaster: - Restore disk image file as it was seen by the backup tool. -Restart virtual machine from the disk image, memory image etc. represented by thesnapshot named "Backup #" (loadvm). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Merging blocks from an overlay file containing vmstate to a backing file
On 20/11/2014 06:36, 정복득 wrote: > Hi, > > I have a question about block commit and internal vm snapshot. > > I am using an overlay file to run a virtual machine. > I took a snapshot with the 'vmsave' qemu monitor command, then commited > blocks in the overlay file into the backing file after playing with my > virtual machine a while and then shutting down it. > Because every block in the overlay file merged to the backing file, I deleted > the overlay file. > After I started virtual machine with the backing file later on, I checked the > snapshot with 'info snapshots' at qemu monitor but couldn't see any snapshots > at all. Meanwhile, block commit seemed successful as I could see files that I > added while the VM was using the overlay file. > > Internal snapshots with vmstate in an overlay file are not merged to the > backing file? > What should I do if I want to preserve internal snapshots into the backing > file? > Is there a way that I can take a external snapshot? I am not using libvirt or > virsh. vmsave memory snapshots can only be stored in qcow2 files. If you somehow set up one qcow2 file to use another qcow2 file as its backing store, I don't think there is any command to transfer snapshots (disk or memory) from the overlay qcow2 to the backing qcow2 and stillappear as snapshots in the backing qcow2. When creating an internal snapshot in an existing qcow2 backing file, there is some vagueness in the documentation as to where the saved memory is stored in the qcow2 file structure: In the frozen snapshot or in the internal overlay. If the vmsave memory/register image is stored in the part of the qcow2 file which represents the disk contents at snapshot time, it would be an interesting issue to look at the ability to commit this part to a qcow2 backing store when committing the block changes in a non-running snapshot to that same qcow2 backing store. However in your case it sounds like you committed the "running" disk blocks to the backing store, essentially removing the snapshot (and making any references to disk contents in the vmsave memory snapshot invalid anyway). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Booting a standalone with qemu-microblaze
On 17/12/2014 14:36, Kevin Grandemange wrote: I am trying to boot my own "kernel" with qemu-microblaze but I can't get it running. I tried the following cmdline: qemu-system-microblaze img.elf qemu-system-microblaze -M petalogix-s3adsp1800 -kernel img.elf -net none -s -S img.elf: ELF 32-bit MSB executable, version 1 (SYSV), statically linked, not stripped Base adress is 0x0. I get the following error: IN: PC=0 rmsr=0 resr=0 rear=0 debug=0 imm=0 iflags=0 fsr=0 btaken=0 btarget=0 mode=kernel(saved=kernel) eip=0 ie=0 r00= r01= r02= r03= r04= r05= r06= r07= r08= r09= r10= r11= r12= r13= r14= r15= r16= r17= r18= r19= r20= r21= r22= r23= r24= r25= r26= r27= r28= r29= r30= r31= qemu: fatal: fetching nop sequence IN: PC=0 rmsr=0 resr=0 rear=0 debug=0 imm=0 iflags=0 fsr=0 btaken=0 btarget=0 mode=kernel(saved=kernel) eip=0 ie=0 r00= r01= r02= r03= r04= r05= r06= r07= r08= r09= r10= r11= r12= r13= r14= r15= r16= r17= r18= r19= r20= r21= r22= r23= r24= r25= r26= r27= r28= r29= r30= r31= Aborted How can I get the code of my elf to be emulated ? I believe the "-kernel" option expects the file to be invoked according to the conventions for invoking a Linux kernel. I don't know what those conventions are for a microblaze, but on most architectures, it involves something beyond a simple ELF file. As an alternative, you may want to look at options for loading your kernel as a "ROM" firmware image, which would involve a different set of conventions. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] sti triggers strange behavior
On 19/01/2015 23:03, e...@sloth.life wrote: I've been working on a small kernel, but I've found that any useage of the sti instruction makes the virtual system restart endlessly very quickly running the same program over and over. When I have an IDT installed it registers interrupt 1. I posted a question here https://stackoverflow.com/questions/27994889/sti-triggers-debug-exception-in-qemu , but got nothing. Does anyone know what could cause this behavior? First the obvious possibility: Are you handling the debug exceptions, (on x86: IDT entries 1 and 3, possibly also other entries from 0 to 0x0F). If not, emulating simple debugging of your code would trigger emulated calls to those exceptions, which then degenerate to double faults or triple faults, which would crash or reset real CPUs. Next the more tricky case: What interrupts are the virtual hardware generating?, and are any of the corresponding IDT entries left blank/disabled/invalid? If so, even a real machine would reboot (actually, crash) every time the CPU (with interrupts enabled) receives an interrupt for which the IDT does not tell it what to do. On a real machine, this is avoided by first (before the first sti) programming some piece of real/virtual hardware to not pass the unwanted interrupts to the real/virtual CPU, thus avoiding the problem. On historic PCs, these were one or two 8259 8-input interrupt controller chips. On modern x86/x64 CPUs this would be the APIC "chip" (physically on the CPU die). On non-x86 architectures, there are often similar interrupt controller devices on the mainboard or inside the SoC. As an additional complication, I am unsure if qemu fully emulates unusual configurations of the virtual interrupt controller, for example, qemu might be passing on some rarely disabled interrupt sources unconditionally. One way around this (for debugging) would be to fill your IDT with dummy handlers that tell you which one was called. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] debian wheezy which kvm version
On 21/01/2015 09:16, Thomas Manninger wrote: Hi, next month, i start a new kvm installation with 4 servers and debian wheezy as os. The current version in debian wheezy is qemu kvm 1.1.2. Should i use this version, or is better, when i compile a newer version of kvm? Which version should i use? Add the wheezy backports archive to /etc/apt/sources.list and enjoy qemu 2.1. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Block format 'qcow2' used by device 'drive-virtio0' doesn't support the option 'physical_block_size'
On 27/01/2015 17:52, Hendrik Visage wrote: Good day, Anybody able to guide me the combinations, etc. when and where these are supported? I'm having performance issues on 512e advance format/4Kb disks, and the symptoms for me tells me that I'm hitting the read-modify-write problems as the VMs report 512/512 for the virtio block devices, and I want to modify that to 4k sector sizes. Hendrik Visage qcow2 cannot store the "physical block size" as an explicit property. But what you can do is the following: 1. Make sure the host physical disk partition system that stores the qcow2 file is aligned to a multiple of 4K (or the RAID block size if on a RAID system). 2. Make sure the host file system that stores the qcow2 file has a block size of 4K or a multiple of 4K. 3. Make sure the internal qcow2 cluster_size is 4K or a multiple of 4K (I think this is the default). Otherwise this is set using the "-o" "cluster_size=4096" option to qemu-img create/convert. 4. Make sure the guest partition on the virtual disk (backed by the qcow2 file) is aligned on a multiple of the qcow2 cluster_size. 5. Make sure the guest file system of the guest partition on the virtual disk has a block size which is a multiple of the qcow2 cluster_size. In other words, the usual "4K issue" procedures, but on both the physical and virtual machine. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] About ftp
On 29/01/2015 02:19, z_axis wrote: hi, friends After `ftp.exe 10.0.2.2` successfully, why cannot i `ls` the host(OpenBSD) ? Even passive mode doesnot work too. The guest is WindowsXP running in qemu-2.x on OpenBSD. Regards! Three possibilities: 1. Is this one of those ftp servers (traditional) where the ftp ls command actually invokes /bin/ls on the server? In that case it could be a server issue. To test this try to ftp 127.0.0.1 from the server itself (with the same username/password) to make sure it is actually working. 2. Is this a situation where the secondary (data) channel doesn't get through. To test this, try to simply "get" a file which you know is there. The error message should be more revealing in such cases, as it now only reflects the transmission (and possible file access errors). For extra diagnostics, turn on verbose mode first. If this reveals a problem, check if you are using some kind of NAT logic between the virtual network inside qemu and the host (I seem to recall that some qemu options do this). 3. Are you sure you are in passive mode? Not all Microsoft implementations of ftp.exe actually include the code for passive mode, even when the corresponding copy of Internet Explorer does. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] A different way of networking
On 29/01/2015 05:01, Gregor Szaktilla wrote: Hi there! When I was looking around for some documentation on how to get qemu networking I only found guides where bridges and/or VPNs were involved. Actually, the "bridge" involved is also part of Linux itself. In fact on many moderndistributions (such as Debian 6/7/8), the needed bridge commands are preinstalledand simply need to be enabled in the network config file. Since I didn‘t want to read another manpage concerning something I probably won‘t need anywhere else and since I had the time to think about how things usually work, I created „my“ setup and wrote down what I did. Starting with my notes I wrote the documentation I was missing when I was looking around for something helpful. My setup doesn‘t need anything else than what comes with probably any Linux installation (no bridge, no VPN etc.). Another advantage of my setup is that all things are tied together with iptables rules only. A disadvantage may be that every new independent virtual machine sits in its own (sub)net. For example, on Debian, all you need to do is change the file /etc/network/interfaceson the host machine from something like: auto eth0 iface eth0 inet ... ... Tosomething like (the ... parts are unchanged): auto br0 iface br0 inet ... ... bridge_ports eth0 bridge_stp on bridge_maxwait 0 bridge_fd 0 Then qemu instances can simply be told to join br0, and it'll be like they are plugged in next tothe host computer in extra ports in your home router. No need for special iptables rules, tun/tap devices etc. (though you can still use iptables/ebtables if you want to block some traffic). I have heard that Red Hat people like much more difficult explanations, and have even included those in their GUI designs, but with Debian (and probably Ubuntu) it is really this simple. If you want to make your qemu networking without having to set up bridging and such, my (german) guide may be helpful: http://html.szaktilla.de/qemu If you have any comment or suggestion please let me know. Have fun! That‘s an order :-) Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Dear Qemu-Discuss Team - How can i setting Setting of 8G RAM Host
On 30-01-2015 03:29, 여승준 wrote: Oops, I didn’t describe my qemu kvm version. I resend mail. Hello. I’m Qemu KVM User for testing Virtualization. I was installed Qemu KVM Virt-manager Package in Linux Centos Version 6.6. And I Adding Virtual Host For Windows 2003. I successed install Windows 2003 Host in qemu-kvm But I can’t setting memory 8GB. If I setting 8G RMA in qemu virt-manager’s Host Information, Can’t install Virtual Host. So I setting 3072MB, then I can install virtual host ( windows 2003 and any host ) I need over the Virtualized Host of over 4GB RAM. (exactly 8G RAM) How Can I setting Virtualized Host of 8GB RAM? Please reply to Mail Address string_st...@hostcenter.co.kr <mailto:string_st...@hostcenter.co.kr> [root@n-ato ~]# rpm -qa virt-manager libvirt qemu-system-x86 openssh-askpass libcanberra-devel libcanberra-devel-0.22-1.el6.centos.x86_64 libvirt-0.10.2-46.el6_6.3.x86_64 openssh-askpass-5.3p1-104.el6_6.1.x86_64 virt-manager-0.9.0-28.el6.x86_64 You still haven't listed the version of qemu (qemu-system-x86) and kvm (part of the Linux kernel or a patch), but it looks like you may be using some old stuff. A few tips: I cannot find it now, but I seem to recall that some older Windows version had a bug where the OS installer would crash if there was too much RAM, but Windows would be fine if you had less RAM while installing Windows and the needed update before increasing the RAM. On the other hand 32 bit editions of XP and 2003 with their latest servicepacks are *artificially* limited to 4GiB RAM to work around a few broken pieces of PCI hardware/drivers that hadn't been tested properly with DMA addresses larger than 32 bit. Stupidly, the original versions (before the big "security features" servicepack) could use the larger memory just fine if you didn't have one of the bad pieces of hardware and just added the "/PAE" switch in BOOT.INI, just like you could in Windows 2000 and NT 4.0 SP3+. But when "/PAE" became the default they saw that it broke some hardware, panicked and added a hard 4G limit. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] cannot use Up/Down key
On 02/02/2015 01:55, z_axis wrote: As there are some errors in WindowXP guest, it wants me choose "the last time righ configuration" to boot normally. However, I cannot use the Up/Down key, i can use C-A-2 to enter the console though. Sincerely! Make sure your qemu options indicate that you want to emulate a PS/2 keyboard. Emulating USB keyboards in this situation may trigger unfortunate corner cases in the qemu USB keyboard emulation, specifically related to the transition between "boot mode emulation" and "full HID mode" (Those mode may be called something else, I don't remember right now). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Network Migration During Live Migration
On 02-02-2015 23:33, Azizul Hakim wrote: Does QEMU supports network migration during live migration? By network migration I mean migrating open connections inside the VM so when the VM moves to a new physical host, it can still communicate with outside world at the same fashion as it did before migration? I'm mainly interested in this issue as I've a Web Server running inside the VM which I use using port forwarding and I would like to know what should I do to keep the server working after the migration? I am not sure if this will be helpful: The general concept is that the virtual machine (including its OS and network software) should be completely unaware that the physical computer host has been replaced beneath it. Thus for networking, it should have the exact same MAC address(es), IP address(es) and connectivity before and after. Whatever tool, script or procedure outside qemu is used to command live migration is expected/required to make this happen by specifying identical "inside" network behavior (same virtual NIC model, MAC address and virtual PCI bus location) and compatible "outside" network behavior (e.g. bridging or other hookup to the same VLAN). Typically the your web server VM would be connected to a bridged network which is available on both physical machines and the effect would be as if you instantly moved a physical web servers network cable from one position in the outside network switch to another. This way, even ongoing downloads from the web server should not be disrupted, except for a short pause while the migration slows down the VM and the physical network switch learns the new path to its MAC address. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2730 Herlev, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] vm live storage migration with snapshots
On 11/02/2015 18:08, Edward Young wrote: Hi all, I'm investigating the ways to improve the live migration performance in libvirt. I have a question about the vm live storage migration. the platform is libvirt + qemu + kvm If we want to migrate a running vm with its virtual disk images to another node. we can use 'virsh migrate ' commands. What if this vm has a number of disk-only external snapshots? In the current version, how can live migrate this vm? Is it possible to blockcommit the snapshots to the base image and later perform the migration, without shutting down the running vm? Or is it possible to iteratively transfer all the snapshots to the destination and later live migrate only the latest new data? Thanks in advance. Best, Ed Just a quick answer, others can answer on clever ways to transfer disk deltas between systems: - The usual way is to store both the base images and the deltas on some kind of shared storage accessible from both hosts involved in the migration, thus eliminating the need to transfer any disk contents. - To the extent that the (not changing at that point in time) base image and deltas for the non-current snapshot are stored in separate files, they could simply be transferred with ordinary tools such as cp, dd or rsync. - If you freeze execution of the guest during migration (not usually a preferred thing when wanting to do live migration), remaining deltas could be transferred with a simple rsync. - If you can find a way to make the snapshot commands store all further deltas around the time (just before/after) the migration in a separate delta file on a shared drive (NFS etc.), you could do that, then rsync all the (now stable) underlying files, then live migrate the running VM from using the shared delta and one hosts underlying files to using the same shared delta with the other hosts (identical) underlying files, then commit the snapshot delta from the NFS drive to the new host's copy of the next lower snapshot or base image. ASCII art illustration of the final tip above: ++ | VM | ++ / \ /4Migrate\ +-++-+ | Host1 || Host2 | +-++-+ | \ / | | 2Snap \/5Commit| |+--+| || TempSnap || On NFS etc. |+--+| | / \ | | / 3rsync \ | +-++-+ |Snapshots||Snapshots| +-++-+ |1copy | +-++-+ | Base || Base | +-++-+ Note that disks are only identical during step 4 Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] vm live storage migration with snapshots
On 11/02/2015 20:42, Edward Young wrote:
Hello Jakob,
Thank you very much for quick and valuable answers to my question!
I have follow up questions about this topic:
1. Is there any other way we can finish this process without the shared
storage? I see the last tip you mentioned, we still use a tempsnap in
shared nfs server. If we consider the long distance VM migration
scenario, we do not have any shared storage resource between the source
and destination.
One way would be to make the top snapshot a qcow2 snapshot
created with a vmsave memory snapshot included, force off
the machine (virsh destroy), rsync any leftover changes to
the new locations, then resume the new machine from the
snapshot. This means downtime.
Another way would be to set up some form of simple file
sharing (NFS, SMB, or similar) for just enough storage to
store the tempsnap small snapshot. This would typically
be much smaller than the capacity needed to store a full
disk even for a single VM, provided you do the procedure
steps 2-5 (temporary snap, final rsync, migrate, commit)
on one VM at a time. Assuming of course that nothing
inside the VM decides to do a full disk rewrite (defrag,
zerofree, read-write disk test etc.) in the middle of
the process, as that would explode the temporary
snapshot to full disk size.
2. I noticed in the qemu implementation, we have drive_mirror function.
The drive_mirror will mirror the write requests to both the source and
the destination. Can we use drive_mirror to transfer the new generated
data in the migration. In the background, we copy all the snapshots to
the destination. After the transformation is done, we reconstruct the
disk images in the destination. I'm not sure whether this is feasible or
not.
I don't know that option, so cannot advise on its abilities.
*From: *Jakob Bohm mailto:jb-gnumli...@wisemo.com>>
*Date: *February 11, 2015 11:38:15 AM
*To: *qemu-discuss@nongnu.org <mailto:qemu-discuss@nongnu.org>
*Subject: **Re: [Qemu-discuss] vm live storage migration with
snapshots*
On 11/02/2015 18:08, Edward Young wrote:
Hi all,
I'm investigating the ways to improve the live migration
performance in libvirt. I have a question about the vm live
storage migration. the platform is libvirt + qemu + kvm
If we want to migrate a running vm with its virtual disk images
to another node.
we can use 'virsh migrate ' commands.
What if this vm has a number of disk-only external snapshots? In
the current version, how can live migrate this vm?
Is it possible to blockcommit the snapshots to the base image and
later perform the migration, without shutting down the running vm?
Or is it possible to iteratively transfer all the snapshots to
the destination and later live migrate only the latest new data?
Thanks in advance.
Best,
Ed
Just a quick answer, others can answer on clever ways to
transfer disk deltas between systems:
- The usual way is to store both the base images and the
deltas on some kind of shared storage accessible from
both hosts involved in the migration, thus eliminating
the need to transfer any disk contents.
- To the extent that the (not changing at that point in
time) base image and deltas for the non-current
snapshot are stored in separate files, they could
simply be transferred with ordinary tools such as cp, dd
or rsync.
- If you freeze execution of the guest during migration
(not usually a preferred thing when wanting to do live
migration), remaining deltas could be transferred with a
simple rsync.
- If you can find a way to make the snapshot commands
store all further deltas around the time (just
before/after) the migration in a separate delta file on
a shared drive (NFS etc.), you could do that, then rsync
all the (now stable) underlying files, then live migrate
the running VM from using the shared delta and one hosts
underlying files to using the same shared delta with the
other hosts (identical) underlying files, then commit
the snapshot delta from the NFS drive to the new host's
copy of the next lower snapshot or base image.
ASCII art illustration of the final tip above:
++
| VM |
++
/ \
/4Migrate\
+-++-+
| Host1 || Host2 |
+-++-+
| \ / |
| 2Snap \/5Commit|
|+--+|
|| TempSnap || On NFS etc.
|+--+|
| / \ |
| / 3rsync \ |
+-++-+
|Snapshots||Snapshots|
+-++-+
|1copy |
Re: [Qemu-discuss] libvirt live migration, qcow2 image, nbd server
On 13/02/2015 03:41, Edward Young wrote: Hi all, When I live migrate a vm using "migrate --live --copy-storage-all mig-vm qemu+ssh://192.168.1.3/system <http://192.168.1.3/system> tcp://192.168.1.3 <http://192.168.1.3>" I got the following error WARNING: Image format was not specified for 'nbd://node2:49155/drive-virtio-disk0' and probing guessed raw. 258 Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. 259 Specify the 'raw' format explicitly to remove the restrictions. From the error itself, I can tell that nbd server treat this image as raw, rather than qcow2. My platform is libvirt+qemu. the image type of this vm is indeed qcow2. I do not know how to let nbd know this image format. Thanks! Ed Are you sure it is not saying that the image served by qemu (on the destination) based on *decoding* the qcow2 disk is treated as a raw image, which it probably is, with the serving qemu instance mapping the "raw" read/writes to proper qcow2 operations? Just guessing of cause. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] network at 100MB
On 23/02/2015 15:59, Colin Durrant wrote: Hi I have scientific Linux 6.x running with *libvirt* and KVM / QEMU. Basically my windows VM servers are connected at 100MB instead of 1000MB. As they all have pretty high network traffic, is there a way to resolve this? Is this a common issue? We have some IBM 3750 servers with two network adapters so pretty high end. Which virtual network adapter are you presenting to Windows inside the VM (e.g. E1000, virtio, ...) Which method of connecting the virtual network to the real network have you specified to qemu (e.g. linux kernel bridging, VDE daemon based bridging, ...) Which additional options have you specified to qemu regarding this virtual network connection? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] linux: ioctl(KVM_CREATE_VM) failed: 12 Cannot allocate memory
On 27/02/2015 05:22, Ken Moffat wrote: On Fri, Feb 27, 2015 at 02:12:28AM +, Ken Moffat wrote: I had kvm working fine (i686 guest) from a new x86_64 host system - after fixing up some Xorg server issues - and then I decided to create a new x86_64 guest (this is linuxfromscratch, so using an old guest to build a new one). But now when I try to start qemu with any of my images, even one which worked a couple of hours ago after I last reran 'startx', I get: ioctl(KVM_CREATE_VM) failed: 12 Cannot allocate memory failed to initialize KVM: Cannot allocate memory and google produced nothing useful about where I need to look. This is linux-3.19.0 and qemu-2.2.0 (and linuxfromscratch-7.7-rc / current BLFS). Any pointers, pretty please with sugar ? Eventually, I finished the application tests I had been doing, and decided to reboot to see if that would fix it. But before rebooting, I killed Xorg with ctrl-alt-backspace (yes, I'm old school ;) and then ran 'startx' again : now qemu is working once more. Very weird, I was starting to think it might be a kernel regression, but apparently not. Any pointers for what the error message means would still be appreciated. I think it is self-explanatory: Not enough free memory in the system, mostprobably because your X server had allocated too much. The command "free" of the top lines in "top" output should tell you how much is left. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Starting troubles in running stm32f4 on an i386
On 04/03/2015 15:36, Peter Maydell wrote: On 4 March 2015 at 23:32, Srinath M wrote: The binary is for a very custom board. http://autoquad.org/ I have replaced the sensor reading tasks with an onboard simulation that feeds 'virtual sensor values' to the STM32F4. This way the executable is now entirely self contained and doesn't call any physical sensors etc I would rather alter qemu to add the functionality needed if there was some documentation on how to go about it, if that's the way to go about it It can be done, but it's a big and complicated job (think of it as about as much effort as porting an OS like Linux to the board would be: each device needs a device model to be written, and that's about as much work as writing a device driver). There's no documentation, you have to look at the existing QEMU source code to see how to write new device models. To Srinath M: Maybe the easiest solution is to use an existing -M parameter most closelyresembling the relevant parts of the hardware (a Cortex M3 or similar CPU,and other on-chip components as close as possible to what is on the chipyou use, if there is more than one choice). Somewhat confusingly, Linux on Arm people have a nasty habit of conflatingthe SoC chip with the machine/board, resulting in lots of absurd situationswhen compiling kernels for real world arm machines, such as the millions(billions soon?) of phones and tablets running Linux kernels on Arm coresinside mass produced SoC chips soldered onto different boards whose very existence cannot be known by the free software community until thefinished product, complete with a running Linux kernel and production OShas been mass produced and distributed to warehouses around the globefor a surprise commercial release. You, as someone soldering an Arm-based microcontroller onto a custom project board and trying to emulate it using qemu are now in the same situation as a Samsung/LG/HTC/Motorola/ OPPO/Acer/etc. engineer designing a new Smartphone around a commercially available higher end Arm SoC connected to a new secret (until released) mix of peripherals and trying to get Android firmware up and running without revealing your new design to the competitors until it is already arriving in the shops while being announced to the world in a multi- million dollar advertising campaign. To the qemu maintainers: Life would be so much easier if the Arm "machines" supported by the Linux kernel, qemu etc. were more like the PC architectures: A command line/config file specified arbitrary combination of CPU cores and common peripherials. During its short stint as an open source OS, the Symbian project was trying to create this under the code name "Syborg", maybe someone could resurrect that as a general mechanism for emulating different Arm (etc.) machines without explicit code development for each board. In other words, one should be able to do something like qemu-system-arm --cpu M3 --vfp=no --ram 32M \ --device=sdcard1,sdcard4wire,0xabcd,disk1 \ --disk=... --device=watchdog1,wdfoo,0xabcd0010 \ --device=gpio,iobar,0xabcd0020 \ --device=tty1,serial8250,0xabcd0030 ... \ --bootrom=0xf000,0xf0001000,myrom.rom Causing qemu to emulate that CPU without a floating point unit, with the specified peripherals (out of those supported by qemu, many shared with other architectures like the classic 8250-style UART) at the specified bus addresses, mapping in a virtual ROM with the specified file contents at the specified address (0xf000, size is file size) then start executing at 0xf0001000 and leaving it to the supplied ROM to find a bootloader and tell it about where the I/O devices are, just like what happens when someone builds a new board from existing chips or assembles a new SoC design from ARM inc. licensed mask blobs. But for now, it seems we will all have to make do with the ugly hack of choosing a "similar" machine from a short list of what others have created, and then adapt our test software configurations to that unrealistic model, until we are ready to test on the real hardware with a differently compiled system. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to read memory dump?
On 09/03/2015 17:21, Binh Q Pham wrote: Hi folks, Could you suggest me a way to extract information from Virtual Machine's memory dump (I used 'pmemsave' to get this memory dump)? Thanks for your help. -Binh I read somewhere offline that there is a project called "volatility", which provides tools and scripts to examine machine states found in such memory dumps. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] execve failing inside of a qemu chroot with argument list too long
On 18/03/2015 10:27, Dave Flogeras wrote: On Thu, Oct 23, 2014 at 5:03 PM, Dave Flogeras wrote: I am trying to compile binutils-2.24 within a qemu chroot (well 3 Sorry to dig up an old thread, but I finally found a workaround. This author of this blog post (a Gentoo developer) posted this and bumping up MAX_ARG_PAGES worked for me. Hopefully this helps someone else googling. https://blogs.gentoo.org/calchan/2015/03/11/binsh-argument-list-too-long/ I don't know what a proper long term solution to this would be, since the binutils unit test trying to pass a huge arg list is certainly exceptional. Hmm, would a gcc invocation to link a real world large program (like, say, LibreOffice)cause gcc topass a similarly huge command line to /usr/bin/ld from binutils? If so, this might affect using qemu(not qemu-system) for cross compiles. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] 答复: How to connect megasas to its backend?
On 07/04/2015 09:05, Yu Hou (侯雨) wrote: Hi, all, may I ask a question? How to write the -drive and -device parameter to specify two files to work as two partitions/drivers in qemu? I am hoping to use virtio-blk-device, thanks! Please start your own thread next time. Simply provide more than one pair of --device and --drive options, one for each virtual drive. If using virtio-blk-device, there is no option for a virtual SCSI adapter, otherwise multiple drives can share a single SCSI adapter. Hannes Reinecke 发送时间: 2015年4月7日 14:50 收件人: Ming Lin 抄送: qemu-discuss@nongnu.org 主题: Re: [Qemu-discuss] How to connect megasas to its backend? On 04/03/2015 08:40 AM, Ming Lin wrote: Hi, For nvme, it uses "drive=" property to connect to backend. -drive file=/path/to/image,if=none,id=foo -device nvme,drive=foo,serial=1234 But megasas doesn't have "drive" property. Could you share qemu commands to connect megasas device to its backend drive? Of course megasas doesn't. Megasas is a SCSI HBA emulation, not a SCSI device emulation. You'd have to use two device specifications here: -drive file=/path/to/image,if=none,id=foo -device megasas,id=raid -device scsi-disk,drive=foo,serial=1234,bus=raid.0 or -device scsi-disk,drive=foo,serial=1234,scsi-id=0 Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Monitor qemu process termination ...
On 09/04/2015 12:17, cluster lab wrote: Hi, I want to recognize qemu process termination, After some test and try, developed this scenario: "Receiving of EOF from opened monitoring channel (return of zero from read) would mean qemu process termination. " (monitor means JSON monitor channel), Is it a correct way? Best regards. You could also look at the actual OS level termination of the process id of the relevant qemu process. Unfortunately, POSIX is less helpful in waiting for processes you did not start yourself than some non-POSIX systems. But maybe you are lucky and can get this from some channel to the process that actually started qemu, e.g. libvirt-bin or your own shell scripts. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Inter-guest communication with Virtio
On 13/04/2015 16:43, Mauricio Vásquez wrote: Hi All, I was wondering if there is an implementation of inter-guest communication using virtio (specially networking), I found that there is an implementation using lguest (lists.ozlabs.org/pipermail/lguest/2008-March/001064.html) but I can not find anything similar in KVM/QEMU) Thank you very much for your help! Well, at least there is the virtio-net network adapter. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Possible to uniquely identify KVM guests?
On 16/04/2015 13:45, Caspar Smit wrote: Hi All, I searched the mailinglist archives for an answer but couldn't find any so I hope this question wasn't asked already. I'm looking for a way to uniquely identify a KVM guest on different hosts (to prevent running a single license multiple times). For instance I have a Windows 7 KVM Guest running an application which requires a license. Once the license is installed it is possible to clone the VM to a different identical host (keeping all UUID's and MAC adresses of the vNIC the same) and the license still works because there is no difference. Is there a way to distinguish clones of the same VM from inside the Windows Guest? Maybe some unique Registry key? Kind regards and thanks in advance, Caspar Smit Sounds like you are the one writing the license enforcing code, sosome people around here will refuse to help on principle. However, as a more pragmatic person, I can offer these tidbits ofhelp: 1. The Windows Registry is just some files on the disk and won'tidentify cloning (except as a side effect of other code detectingit somehow. 2. Cloning the MAC address while staying on the same (virtual)network segment is going to cause severe communicationsproblems, so is less likely to be practical. But people mightwork around this by placing each clone behind its own NATrouter, each giving it the same RFC1918 IPv4 address, while letting an outer NAT router map all these clones to the samepublic IPv4 address. This will be less workable for IPv6,IPXand other non-NAT protocols. It would also limit the boxesability to communicate with each otherdirectly. 3. It is potentially possible that someof the virtualizationspecific CPUID or virtio interfaces will provide some perinstance identifiers, but it is also likely that those maychange when doing a virtual power off/on (i.e. stopping andstarting the qemu process itself). 4. If your license checking code is talking to one of youroutside servers anyway, it could itself change a uniquevaluestored inside the virtual machine (on disk or registry)each time it does that, and your server could then objectif the unique values start deviating in ways that cannot be explained by snapshot rollbacks/ restores of computerbackups. 5. If your license checking code is not phoning home (whichis the morally right thing to avoid), but is talking toother locally networked copies of your software, youcould try to detect if two licensed copies with the samepermanent unique install time ID (randomly generated during installation or license issuance) start directly orindirectly talking to each other. 6. If your license checking code and the application beinglicense checked is not networked at all, then the taskbecomes near impossible, and you will have to rely onhuman/legal means of preventing bad behavior. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Android in Qemu on an ARMv7 host
Inc. fuse init (API version 7.22) msgmni has been set to 1490 io scheduler noop registered (default) loop: module loaded vda: unknown partition table vdb: unknown partition table vdc: unknown partition table tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <mailto:m...@qualcomm.com>> usbcore: registered new interface driver usb-storage mousedev: PS/2 mouse device common for all mice input: qwerty2 as /devices/904.goldfish-events/input/input0 usbcore: registered new interface driver usbhid usbhid: USB HID core driver ashmem: initialized logger: created 256K log 'log_main' logger: created 256K log 'log_events' logger: created 256K log 'log_radio' logger: created 256K log 'log_system' oprofile: no performance counters oprofile: using timer interrupt. ip_tables: (C) 2000-2006 Netfilter Core Team TCP: cubic registered NET: Registered protocol family 10 ip6_tables: (C) 2000-2006 Netfilter Core Team sit: IPv6 over IPv4 tunneling driver NET: Registered protocol family 17 Bridge firewalling registered VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0 drivers/rtc/hctosys.c: unable to open rtc device (rtc0) ALSA device list: No soundcards found. Freeing unused kernel memory: 216K (c0564000 - c059a000) init: init started! init: (Loading properties from /default.prop took 0.02s.) init: /init.rc: 492: inval0d option 'start' init: (Parsing /init.environ.rc took 0.01s.) init: (Parsing /init.usb.rc took 0.02s.) init: could not import file '/init.unknown.rc' from '/init.rc' init: (Parsing /init.zygote32.rc took 0.00s.) init: (Parsing /init.trace.rc took 0.01s.) init: (Parsing /init.rc took 0.30s.) init: Starting service 'ueventd'... ueventd: ueventd started! init: Timed out waiting for /dev/.coldboot_done init: /dev/hw_random not found init: /dev/hw_random not found init: cannot find '/system/bin/logd', disabling 'logd' ext4_utils: Failed to insert key into keyring with error Function not implemented init: Not bootcharting. init: cannot find '/system/bin/tzdatacheck', disabling 'exec 0 (/system/bin/tzdatacheck)' init: (Loading properties from /system/build.prop took 0.00s.) init: (Loading properties from /system/default.prop took 0.00s.) init: (Loading properties from /vendor/build.prop took 0.01s.) init: (Loading properties from /build.prop took 0.02s.) init: (Loading properties from /factory/factory.prop took 0.00s.) init: (Loading properties from /data/local.prop took 0.01s.) init: Unable to open persistent property directory "/data/property": No such file or directory init: cannot find '/system/bin/logd', disabling 'logd-reinit' init: Starting service 'healthd'... init: cannot find '/system/bin/lmkd', disabling 'lmkd' init: cannot find '/system/bin/servicemanager', disabling 'servicemanager' init: cannot find '/system/bin/vold', disabling 'vold' init: cannot find '/system/bin/surfaceflinger', disabling 'surfaceflinger' init: cannot find '/system/bin/sh', disabling 'console' init: Starting service 'adbd'... init: Service 'healthd' (pid 784) killed by signal 6 init: Service 'healthd' (pid 784) killing any children in process group ueventd: Coldboot took 9.44s. init: Starting service 'healthd'... binder: 856:856 transaction failed 29189, size 0-0 binder: 856:856 transaction failed 29189, size 0-0 binder: 856:856 transaction failed 29189, size 0-0 <…> The binder messages go on forever and the UI stays black. Seems to me like the init process has some major trouble. I could not find out why or how to resolve this. Any ideas, anyone? On 15 Apr 2015, at 21:12 , Jan Teske <mailto:jan.te...@gmail.com>> wrote: Hey, I’m trying to boot an Android L guest (32 bit) using Qemu on my Cubietruck, which sports a Cortex-A7 CPU. It doesn’t really work out. I get a Linux shell but the UI does not load. I have now prior experience in using Qemu. I don’t know if I might need a patch for Qemu, a special Android image/kernel or just the right command line parameters. Instead of describing my setup and trying to work out were my mistakes lie, I would rather like to know if anyone has done this before. I guess it would be too optimistic to hope for some kind of tutorial (although there is one for 64-bit Android: https://www.linaro.org/blog/core-dump/running-64bit-android-l-qemu/). I already searched for one and found nothing. But any pointers
Re: [Qemu-discuss] x86 Linux (RHAT) on RaspPi?
On 25/04/2015 19:38, autostu...@online.de wrote: Hi, i am wondering if it is possible a x86 Linux to run on a rasp? I have an application, which is x86 (64bit) compiled and I am looking for a solution to run this somehow. So I’ve installed qemu but I don’t know how could I create for example a RedHat image (I know odx or vmdisk formats) and how to emulate this. Thanks for the help. The standard formats used by qemu are qcow2 (qemu-copy- on-write-2) and raw (just a plain file holding all the sectors of the disk). Both formats can take advantage of the sparse file feature in ext2/ext3/ext4 (etc.) to not allocate disk/sdcard space for sectors that are all 0. Blank disk images are created using the tool qemu-img, run qemu-img --help for a list of operations. For raw images, manipulation is done using basic tools such as dd directly. ISO files (such as Radhat install CD images) are a special case of "raw" files and can be used directly. Disk images can be converted, checked for qcow2-level errors, copied etc. using other qemu-img subcommands. The virtual hardware configuration of your virtual x86 PC is specified using command line options to the qemu-system-i386 or qemu-system-x86_64 program each time you "power on" the virtual machine. qemu-system exits when the virtual machine powers off. "BIOS" settings, such as selecting a boot drive are command line options too. qemu-system also has a "monitor" interface, which is a separate command line where you can do stuff such as virtually inserting/removing CDs (ISO files), simulate a press of the power button, create snapshots of the virtual machine etc. There are various GUIs on top of qemu, which provide other (sometimes easier) ways to do all this, but they all map back to the raw commands described above. The most popular GUI is Red Hat's libvirt, which is in Debian, and probably in Raspian too. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Virtual Smartcard GPG
On 29/04/2015 13:22, r...@openmailbox.org wrote: Hi. I am trying to get a virtual smartcard attached to a vm but I want it to use GPG instead of NSS. RedHat focuses on NSS because of PKCS#11 requirements and FIPS approval, but for most of the community its GPG that matters for smartcards. Is is possible to use GPG on the host instead of NSS with virtual smartcards? Please document how or add support for it. I don't know, I have not yet studied that feature. However I have always been disappointed that the gnupg code chose to invent its own smart card API rather than using the standard PKCS#11 API. Maybe (just maybe) later versions of gnupg added support for using PKCS11 drivers. As a datapoint, gnupg 1.4.10 seems to support accessing non-usb card readers via pcsclite, as well as built in support to access CCID card readers via libusb but apparently not the use of pkcs11 card specific drivers. Is using a virtual smartcard make the host less secure from a rogue vm? If there are bugs in GPG/NSS backend on the host can they be abused by untrusted code in the vm? This is true for all emulated devices, no exceptions. If the device emulation code in qemu (or worse, in the KVM host kernel part) has a bug in checking arguments received from the VM, this can be used to "escape from VM" by anything using that device. Sometimes (depending on the bug, not the design), such bugs are only exploitable from the Guest Kernel mode device driver, while in other cases such bugs can be exploited from Guest user mode code. For example (random guess), if the virtual smart card feature is exposed to the Guest as a virtual USB CCID card reader, then the ability to exploit any bugs in it may be restricted to: a) Anything allowed to access the virtual reader at the PC/SC like API level. b) Anything allowed to access the virtual reader at the libusb API level. c) Only from the actual kernel mode USB code in the Guest kernel. d) Nothing, if there is no bug. Case d) is of cause the best case, a), the worst. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Networking: Questions about Host to Guest interal traffic management
(Summary for other readers of this list) There are two highly relevant questions that I could not answer, someone else should answer these: Q1: Depending on configuration, how are network packets transferred between the networking stack in the host computer and the virtual NIC emulated by qemu for the guest? Q2: Depending on configuration, does the host networking stack see the virtual machine's NIC as a virtual NIC on the host or as a local network application? This would (for instance) determine if iptables processes guest traffic using its INPUT/OUTPUT or its FORWARD rule chains. I presume the answers would depend of the networking type selected with the --net option, and that some cases may use special fast paths with KVM and/or xen, but I do not know of a document giving definitive answers for all of the cases. The OP would prefer pointers to specific qemu documentation pages for the answers, if such exist. The most interesting cases are probably these: --net tap --enable-kvm --net l2tpv3 --enable-kvm --net tap --xen-domid --net l2tpv3 --xen-domid --net tap (with software emulation) --net l2tpv3 (with software emulation) The answers are of cause obvious when using any of the following documented options: --net user --net socket --net vde --net none On 07/05/2015 10:37, David Borman wrote: Hi, I have a Instance running and iam curious about the internals and how the packets are routed from the physical (hostsystem) layer to the Internal, virtual Guest interface (virtio, e1000, rtl8139 ect) (IP/TCP/UDP/ICMP Data only). "How" is an inboud packet, reaching the hosts physical ethernetcard hardware forwarded to the virtual nic inside the guest os and what is happening if the guest os firewall drops/reject/accept the packet? 1) Will the Hostsystem drop this packet physically? Since network packets are logical phenomena rather than physical objects, they are not as such "physically dropped". However if the host is running any sane kind of firewall (such as the Linux iptables), when the host firewall drops the packet, it will not progress any further, and the correspondingmemory will be freed and reused soon thereafter (where "soon" may be anything from a few picoseconds to billions of years, mostly by chance). As for the freeing of memory, this should be no different from how the host would free the same memory if there was no virtual machine involved. It would either follow the code path for packets that were going to be forwarded out another (physical or virtual) network card if they had not been dropped, or the code path for packets that were going to be returned to a locally running program if they had not been dropped. This part would be better asked in a forum about the host OS and its firewall. However what someone else on this list should be able to answer is how the packets get from the Host OS kernel to the virtual machines virtual hardware, because that is very specific to how qemu is implemented in different situations and configurations. Someone else on this list should also be able to answer if the Host kernel will act as if the packets are going out another (virtual) network card representing the virtual machine, or if it will act as if the packets are being returned to an ordinary program running on the host, again, this is very specific to how qemu is implemented in different situations and configurations. In particular, the answers to the previous two questions is going to depend on the following qemu command line options: --net --enable-kvm --xen-domid 2) Is the Guest OS dropping the packet at the virtual guest network adapter? (and if so, what data still remains inside the Hostsystem memory structures and what event triggers the memory cleanup?) The Guest OS will operate the same way as it would on a physical machine. This includes any firewalls in the guest OS. This part would be better asked in a forum about the Guest OS and its firewall. ps: If there is any good docs out there, then please let me now. On google i only find very generic stuff. Thx, David Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] High CPU in idle Windows guest
(top posting to keep thread consistent) Then you should remember to pass the qemu options to start the VM as a XEN DomU, not as a software VM inside your current (Dom0) VM. I think (from reading qemu --help) that you may need to do that via something called xend, there is probably a manual somewhere. On 10/05/2015 17:14, Terry Glanfield wrote: Hi Dave, You do know that I'm running a Xen hypervisor and not KVM? I'm not wedded to Xen and would consider using KVM if it's more likely to solve my problem, but it would be good to get the Xen solution working properly. Regards, Terry. Dave Gosselin writes: Hi Terry, It appears that the kernel driver hasn't been started. You can do this manually, before starting qemu, by running this as the root user: modprobe kvm You can confirm that the modprobe command started via the lsmod command: lsmod | grep -i kvm If you see any entries, it's loaded; if the command returns with nothing, then it did not load. Afterwards, you can run qemu but you may have to run it as root for it to take advantage of the KVM module you just started. Dave On 5/9/15 8:44 PM, Terry Glanfield wrote: Hi Dave, With -enable-kvm I get. Could not access KVM kernel module: No such file or directory failed to initialize KVM: No such file or directory Where to next? I'm willing to help diagnose it in whatever way I can. Regards, Terry. Dave Gosselin writes: Try supplying the -enable-kvm option to your qemu arguments. If it doesn't work then the support may not be compiled in to qemu or the driver may not be loaded into the kernel. Let me know. Dave Gosselin Sent from my iPhone On May 9, 2015, at 18:52, Terry Glanfield wrote: Hi, I'm running a freshly installed Windows 7 domU on an up-to-date Debian jessie machine running Xen 4.4.1-9. When the Windows machine is idle, I'm seeing upwards of 10% CPU usage from the qemu-system-i386 instance. Other Linux and FreeBSD machines register negligable CPU usage (<0.5%). The server is an HP Proliant DL360 G7. PV drivers don't seem to make any difference. Data from perf attacthed to the process might give the best clues. Any information as to why this processes is comsuming so much CPU would be much appreciated. Regards, Terry. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Cross Building QEMU for Aarch64
On 27/05/2015 17:21, Yehuda Yitschak wrote: Hello i am trying to cross build qemu v2.3.0 to Aarch64 on a i386 host i am using "gcc-linaro-aarch64-linux-gnu-4.9-2014.05_linux" toolchain the qemu configure scripts complains: ERROR: zlib check failed Make sure to have the zlib libs and headers installed. i believe this is a problems with the toolchain. Does anyone know a way around this ? Make sure there is an aarch64 compiled zlib *and its headers* installed in the cross toolchain's aarch64 include and lib directories. Look at the failing lines in the configure scripts and try to figure out where it looks for zlib.h and ( zlib.so.* or zlib.a.*), or if it simply does a test compile/link of a dummy program that referenceszlib. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Cross Building QEMU for Aarch64
On 28/05/2015 14:13, Yehuda Yitschak wrote: Hi Jacob -Original Message- From: qemu-discuss-bounces+yehuday=marvell@nongnu.org [mailto:qemu-discuss-bounces+yehuday=marvell@nongnu.org] On Behalf Of Jakob Bohm Sent: Wednesday, May 27, 2015 19:31 To: qemu-discuss@nongnu.org Subject: Re: [Qemu-discuss] Cross Building QEMU for Aarch64 On 27/05/2015 17:21, Yehuda Yitschak wrote: Hello i am trying to cross build qemu v2.3.0 to Aarch64 on a i386 host i am using "gcc-linaro-aarch64-linux-gnu-4.9-2014.05_linux" toolchain the qemu configure scripts complains: ERROR: zlib check failed Make sure to have the zlib libs and headers installed. i believe this is a problems with the toolchain. Does anyone know a way around this ? Make sure there is an aarch64 compiled zlib *and its headers* installed in the cross toolchain's aarch64 include and lib directories. It's missing. I tried downloading the aarch64 compiler from Ubuntu 13.04 but that toolchain is also missing zlib Look at the failing lines in the configure scripts and try to figure out where it looks for zlib.h and ( zlib.so.* or zlib.a.*), or if it simply does a test compile/link of a dummy program that referenceszlib. It does the latter. It passes -lz as linker flags and it fails. I ran with --disable-zlib-test and managed to get a bit further but then I failed on "ERROR: glib-2.12 gthread-2.0 is required to compile QEMU" I found this very old bug where it suggested to use open-embedded but I'm looking for something simpler. https://bugs.launchpad.net/linaro-oe/+bug/1097561 Download the source tarballs for zlib, glib, gthread etc. and cross compile those before cross compiling qemu. If those libraries are already on your target system (or its repository), just copy them from there instead. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Any way to recover after qemu-img snapshot -a? desparate!
On 29/05/2015 11:55, Aaron Williams wrote: First some background, I managed to screw up a VM running an older version of Ubuntu so it wouldn't boot and was going to attempt to fix it by restoring the boot partition from an earlier snapshot. Unfortunately I didn't fully read the documentation saying that it would blow away anything newer than the snapshot. In any event, at this point it looks like all data is lost. What happened is that my snapshot list looks something like this according to qemu-img snapshot -l: Snapshot list: IDTAG VM SIZEDATE VM CLOCK 0 1969-12-31 16:00:00 00:00:00.000 0 1969-12-31 16:00:00 00:00:00.000 0 1969-12-31 16:00:00 00:00:00.000 1 broken-boot 0 2015-05-29 00:56:26 00:00:00.000 I had made some earlier snapshots, but those were created through virt-manager and it didn't insert the proper labels in the qcow2 image. I later learned that virsh shows the following: Name Creation Time State Broken 2015-05-29 01:06:03 -0700 shutoff post-con 20142014-11-22 02:44:45 -0800 running snapshot12015-04-06 12:23:14 -0700 running Working 2014-11-07 00:32:45 -0800 running In any event, I issues the following command which seems to have totally nuked everything. I backed up the qcow2 file, unfortunately it was not before I did this: qemu-img snapshot -a "" vm1.qcow2 Is there any way I can recover either my "snapshot1" or "Broken" image from the qcow2 file or is all the data forever lost? There is a lot of critical data here. I should have read the man pages and backed up the file before I started this but one doesn't always think straight when suffering from the flu. Step 1: Save everything, back ups at every step from here forward. Step 2: Recover from the flu, the rest needs a very clear head. Step 3: In COPIES of the qcow2 file, try restoring to each of the nameless snapshots, then check if any of those contain your data Step 4: Read the qcow2 file format specification. Step 5: Use a hex editor or other low level tools to examine a COPY of your backed up qcow2 files. Step 6: Look for traces of: Old (now unused) sectors listing mappings from virtual disk sectors to file offsets for older data. Step 7: Look for file offsets/sectors no longer referenced by the currently active (incomplete) image. Step 8: Try to reassemble the unreferenced file offsets/sectors into your missing data, either as a disk image or as files. Step 9: This becomes a lot harder if Ubuntu was encrypting its (virtual) disks, as you are then looking for encrypted sectors that all look alike. Hope for the best... Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Connect physical discs to a virtual machine
On 19/06/2015 11:10, Wolfgang Mader wrote: Dear list, please CC me on reply, since I am off-list. I have a linux server on which a linux quest is running in a qemu virtual machine. To the server, I attached an external hd enclosure (usb) of 8 disks. 4 of these disks form a btrfs raid which is already formated and full of data. I am now searching for a way to connect the disks to the virtual machine. I already found out, that I can mount the btrfs file system on the server and export the mount point folders to the virtual machine. This approach, however, seem to imply nasty problems with data permission. The files on the btrfs file system do belong to different people, and I want to keep it that way. Since qemu is running as a user process, I do not see how is is possible to get r/w access to the data. What I rather would like to archive is to wire the bare metal disks up to the virtual machine and manage the btrfs filesystem on file-system level from withing the virtual machine. How can this be done? Thanks for your help! Simple really: Instruct qemu to provide the physical disks as "raw disk images", by simply using their device names on the host (e.g. /dev/sdk and /dev/sdm) as the filenames of the disk images on the qemu command line. Then the guest OS will see those two physical disks as physical disks with different names, such as /dev/vdb and /dev/vdc or /dev/sdb and /dev/sdc depending on the type of virtual hardware you specified for them on the qemu command line. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Assigning public IPs to Guests
On 21/06/2015 00:21, Peter Reid wrote: Hi, I'm not necessarily sure if this is the correct mailing list; but I may as well give it a shot. Here we go: So I want to be able to assign public IP's to the guests that I am running; I have a 46.105.121.xx address that I want to be my 'host address', and then I 8 IP's in the 5.135.xx.xx range that I want to assign to the guests that I am running. I am running Debian 8 (Jessie), and my existing /etc/network/interfaces config is as follows: http://pastebin.com/gHmWtGt9 I've read about virtual bridges, but I haven't seen an explanation that's made me understand how to set them up yet... I'd be looking for info on how to configure it so that my VM's can 'use' those IP's and any config i need to do for each VM individually. I can provide further info upon request Note: My answers are based on Debian7, I have not checked the differences in Debian8. A bridge that will include only virtual machines is set up as follows in /etc/network/interfaces(I think, all mine include physical interfaces too): # The 10.xx.xx.1 address is just to avoid wasting a public IPv4. auto br0 iface br0 inet static address 10.xx.xx.1 netmask 255.255.255.0 network 10.xx.xx.0 broadcast 10.xx.xx.255 post-up ip link set br0 address y2:yy:yy:yy:yy:yy post-up ip route add broadcast 5.135.xx.x8/29 dev br0 table local proto kernel scope link src 10.xx.xx.1 bridge_stp on bridge_maxwait 0 bridge_fd 0 iface br0 inet6 static pre-up modprobe ipv6 address :::::1 netmask 64 privext 0 Enable forwarding on the host. You will need to choose you own locally unique MAC addresses for the virtual machines and interfaces. On the qemu command lines use something like -netdev bridge,id=hostnet0,br=br0 -device=,netdev=hostnet0,id=net0,mac=z2:zz:zz:zz:zz:zz,... Inside the VMs configure the virtual network interface with static IPv4 and static IPv6 addresses as usual, but with 10.xx.xx.1 as IPv4 gateway. The IPv6 gateway will of cause by :::::1. You may or may not need to add post-up ip route add broadcast 10.xx.xx.1/24 dev eth0 table local proto kernel scope link src 5.135.xx.xx With this setup, the host becomes a router between its physical network and the completely virtual network "br0". Because addressing will be static, there is no need to set up dhcpd or radvd on this network. IPv4 routing on the br0 virtual network is special because the VMs have public IPs, but the router has a private IP. So there are extra "on-link" routes for the other IP address space. ++ Because the virtual network above exists only within a single host machine, there is no redundancy against failure of that machine. If you want the ability to move VMs between different hosts for redundancy, you will need a different setup: Set up a VLAN on your physical network to carry 5.135.xx.xx traffic and the matching IPv6 network let's call this VLAN 1357 (any number from 2 to 4094 should do). Make sure your physical network switches support VLANs and STP, otherwise there will be big problems. The only managed switches I have encountered with such problems was a popular HP Procurve model. A D-link managed switch did much better. On the host machines add something like this to /etc/network/interfaces: auto br1357 iface br1357 inet6 static # pre-up modprobe ipv6 bridge_ports eth0.1357 hw-mac-address y2:yy:yy:yy:yy:yy bridge_stp on bridge_maxwait 0 bridge_fd 0 address :::::: netmask 64 privext 0 Don't enable forwarding on the hosts, they are now switches, not routers. Note that the hosts have no IPv4 addresses on the public IP vlan. Management of the hosts would be done over a different VLAN not reachable from the big bad Internet. On your actual router, set up routing of your 5.135.xx.x8/29 and its IPv6 equivalent on VLAN 1357. As explained in the single-host setup, you may be able to savea public IPv4 by giving the router a private IPv4 and setting upspecial on-link routing rules on both the router and the VMs. You will need to choose you own locally unique MAC addresses for the virtual machines and interfaces. On the qemu command lines use something like -netdev bridge,id=hostnet1357,br=br1357 -device=,netdev=hostnet1357,id=net1357,mac=z2:zz:zz:zz:zz:zz,... Inside the VMs configure the virtual network interface with static IPv4 and static IPv6 addresses as usual. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Error starting domain: vfio-pci
On 23-06-2015 03:20, Gabriel Laupre wrote: ... I noticed that the pba_offset was wrong and it is a problem in my NIC. NIC = Network Interface Card (not driver). ... Maybe it is better that qemu catches bad drivers before they harm real hardware. Of cause if a real NIC does something useful when given that out of range value (such as changing a flag or doing other effects while transferring no data), then it would be best for qemu to emulate that feature when emulating that real hardware. If the driver in question is the standard driver in a major OS (such as upstream Linux), and has been for some time, with no adverse effect on real hardware, then it might make sense for qemu to accept the quirky driver as a service to the many users who want to boot stock OS images. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] help using qemu to emulate raspberry pi2
On 09/08/2015 15:34, James wrote: This is what the console on the host says (it doesn't loop): Uncompressing Linux... done, booting the kernel. Raspbian GNU/Linux 7 (none) ttyAMA0 (none) login: pi Password: Linux (none) 3.2.0 #1 Sun Jan 29 03:14:15 CST 2012 armv6l The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. NOTICE: the software on this Raspberry Pi has not been fully configured. Please run 'sudo raspi-config' Raspbian GNU/Linux 7 (none) ttyAMA0 (none) login: The following assumes Raspian 7 uses a traditional sysv init process, just like Debian 7. From the emulated ttyAMA0, try looking at the contents of /etc/inittab and try to see which line looks most like it could have "Id 1" . That is the process that keeps failing to run so fast that /sbin/init refuses to relaunch it until a 5 minute cool off period. Then look in /var/log/syslog, /var/log/messages etc. for reasons why that process refuses to run. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Can't remove libvirt USB controller via QEMU, but can via virsh
On 11/08/2015 00:33, james harvey wrote: Using libvirt 1.2.18 (-1 Arch) and QEMU git-master (2.4.0.r40384.2d69736). Installed using Q35 chipset. Using managedsave/start is unusable due to a non-stop repeating uhci_hcd error. (Fills tty0 screen every 0.003 kernel seconds with errors.) QEMU won't let me remove the USB controller -- remove is grayed out. virsh edit allows me to remove them, which lets me use managedsave/start. QEMU only lets me remove the USB redirection entries, which didn't help. Last time I checked (has been some time though) qemu did not have a GUI that can gray out anything. There are 2 ways in which libvirt (including virsh) can add/remove devices from qemu: A) By changing the (long) list of devices and other options on the qemu command line when starting qemu, this is reflected in the output of the command virsh domxml-to-native qemu-argv /path/to/guestconfig.xml B) By using the monitor and qml sockets to "hotplug" devices on a running guest. I am quite confident that the usb controller can be removed by simply omitting it (and all usb devices needing it) from the qemu command line (method A), but I don't know if it can be done with method B, (except that it must be possible if libvirt/virsh does it without rebooting the guest). Note that older versions of libvirt refused to remove the USB tablet it uses for mouse movements, and thus the USB controller, which was quite annoying. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] wake on lan support
On 17/08/2015 10:52, Lior Nahmanson wrote: Hi, does QEMU support Wake On Lan feature while using the standard libvirt (assuming I implement a NIC that support this feature)? Thanks, Lior Nahmanson When a QEMU virtual machine is "powered off", the qemu program is not running and cannot do anything. However if another program on the host computer (maybe libvirtd, maybe something extra) watches the network for "wake-on-lan" packets matching the MAC in the virtual machines configuration, then that program could start the computer when the wake-on-lan packet arrives). I don't remember the standard way for a physical wake-on-lan card to tell the booting OS that the computer was powered on due to a wake-on-lan packet arriving, but this can probably be emulated in some trivial way too. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Guest fails to reconnect after host network goes down
On 27/08/2015 18:26, Vlad Yasevich wrote: On 08/27/2015 12:13 PM, Davide Baldini wrote: ... ... I just looked and Debian7 is very old and has 3.2 kernel. Any chance to try with something a bit more modern? FYI: Debian has very long release cycles (years between releases), and Debian 8 was released only sometime during 2015 (and was obviously premature, for example DNS resolver config is broken out of the box on any network with both IPv4 and IPv6, unlike in previous releases). Debian 6 is still supported with LTS updates, while Debian 7 receives even more updates. It isn't bleeding edge, because it aims for stability and safety. Now I don't know if Mr. Baldini is in a position to update to Debian 8 at this time, but just wanted to clear up the mistaken belief that "Debian 7 is very old". Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Help resurrecting the dead
On 30/08/2015 22:22, Dennis Murphy wrote: Hi folks, I’m new to QEMU and I’m trying to use it for a special project I’m working on. What I’m trying to do is resurrect a dead operating system - Banyan VINES 8.5. Given how closed and dead VINES is, I’m really limited as to what drivers and devices I can use to virtualize it. QEMU is super exciting because it can emulate LSI 53c810 and NE2000 Ethernet devices, both of which VINES knows how to talk to natively. My issue is disk-related. What I need to do is mount a hard disk image to install VINES to, as well as the VINES CD-ROM install media. If I use the standard QEMU IDE emulation, I can’t get VINES to install correctly; I get an “srmount: Cannot mount root” panic from VINES when I install Compaq’s EIDE/ATAPI driver (Which is necessary if I’m using IDE to emulate the CD-ROM device.) I’ve tried that 6 ways from Sunday but something in QEMU or SeaBios’s CHS translation is giving VINES fits. Maybe you can change the CHS translation via qemu command line options, I seem to recall there are options to tell qemu the desired CHS mapping for each disk. So, Plan B, let’s try the SCSI route…. If I try to use the LSI 53c810 device and an option ROM, VINES can see the hard disk and CD image, but when VINES tries to read the disk image, QEMU throws an “lsi_scsi: Unhanded writeb” error and goes into a 100% CPU loop. I’m still new to QEMU, so is there any way someone can help me generate some debug or I/O logs to see what’s hanging the lsi_scsi driver? Unfortunately, again, this is such a closed OS that any changes have to be made on the QEMU side; I can’t change out the drivers or anything in VINES. Thanks folks — I’m *this* close to having VINES up and running (I can actually get the server booted and running in VMware, but given that they only emulate a PCNet32 Ethernet device, I can’t get anything to talk to VINES since it doesn’t have a PCNet32 driver!) …. so frustrating!! —Dennis Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Issues with QEMU on Tegra K1 _HOST_.
On 01/09/2015 15:39, jscottkas...@yahoo.com wrote: Hi guys, I'm hoping maybe some of you have some information on a problem that I am seeing. I have tried using QEMU on an ARM host, specifically a Tegra K1, and have found it almost completely unusable. Usually the guest OS hangs and never frees up. Sometimes the guest starts reporting errors or crashes in quest applications that doesn't happen using the same guest on a non-ARM host. I've looked through the mailing list and found one reference last year that seems to be the same problem: http://lists.nongnu.org/archive/html/qemu-discuss/2014-05/msg00012.html I have tried many things to narrow this down, but nothing has helped. I've tried building QEMU versions 2.1.2, 2.2.1, 2.3.1, and 2.4.0. I've tried compiling QEMU on the Debian ARMHF arch (hard float), and the plain (soft float) ARM arch. I've tried versions of GCC from 4.1, 4.5, 4.8, and 4.9. I've tried guests from Debian, NetBSD, OpenBSD, MIPS64, x86_64, and SPARC64. NONE of this has worked to produce even a single usable VM image. I've tried "formulas" found in various places on the web for versions of QEMU, guest OS, arch, and emulated device selection. All to no avail. Yet I can take one of these formulas to an x86 host and it will work strait away. The gist is that _something_ is horribly broken when qemu is used on an ARM host. I'm open to any suggestions that anyone might have. One important thing you may have overlooked: When using qemu to emulate a completely different kind of CPU (x86 on ARM or ARM on x86), things will be a lot slower because each foreign instruction is translated to a sequence of (often multiple) native instructions, and every jump or call triggers a complex mechanism which tries to find the cached translation of the jump/call destination code, and create it if not found. On the other hand when using qemu to emulate an identical CPU (Armv7a w/VFP on Armv7a w/VFP or x86-intel-core-i5-4210 on x86-intel-core-i5-4210), most of the code can be run directly, with qemu (and its kvm helper) just doing the sandboxing (and device I/O emulation). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Wake On Lan support for Windows VM
On 07/09/2015 16:32, Lior Nahmanson wrote: Hi, I have Windows 2012R2 VM with a NIC device that I create. I’m trying to add Wake On Lan support to this device. I turn on all the relevant flags in the PCI configuration space. the’ Wake on magic packet’ feature is enable (under device properties -> advanced) but the feature ‘allow this device to wake the computer’ under Power Management is gray and I can’t check this box. what could be the problem? Do I need to use different bios then the default bios (pc-bios) that QEMU use? Consider the possibility that the virtual "motherboard" hardware emulated by qemu might not have any "wake-on-PCI" feature. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Qemu-system-sparc: What would I need to change to add functionality to save the nvram content between boots ?
On 14/09/2015 11:33, Peter Maydell wrote: On 14 September 2015 at 00:25, Programmingkid wrote: On Sep 13, 2015, at 7:20 PM, Bodo Eggert wrote: On Sun, 13 Sep 2015, Programmingkid wrote: On Sep 13, 2015, at 5:50 PM, Bodo Eggert wrote: On Sun, 13 Sep 2015, Mark Cave-Ayland wrote: On 11/09/15 19:01, Programmingkid wrote: Mark, do you think it is possible for a QEMU command line option to actually load the saved OpenBIOS settings from a file and restore them into memory? I'm thinking something along the lines of -prom-memory . From memory the reason this hasn't been done is because the NVRAM interface is used across multiple architectures and there hasn't been a solution devised that would work well enough for all of them. Then again as the BIOSs involved have continued to develop, it may be now that some of the issues can now be solved so it never hurts to ask on the -devel list. I'm wondering, why wouldn't it be enough to memmap a nvram file instead of mallocing (or whatever) the memory area? What advantage does mmap() have over malloc()? I think malloc() is a lot more familiar to use than mmap(). The advantage is that it's file-backed (optionally, but intended here). Everything you write into that memory area will be automatically saved and restored just like a nvram should be. At least, that's what the documentation promises, I never used it myself, therefore I ask. You make mmap() sound good. Maybe we should use it. Rather than reinventing the wheel I suggest looking at how QEMU already supports file-backed ROMs for other platforms... Note that this is about *writable* persistent memory, such as the non-time battery backed registers in a PC clock or the thing that stores EFI BIOS variables such as those holding Linux crash dumps. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Qemu-system-sparc: What would I need to change to add functionality to save the nvram content between boots ?
On 14/09/2015 16:49, Peter Maydell wrote: On 14 September 2015 at 15:44, Jakob Bohm wrote: On 14/09/2015 11:33, Peter Maydell wrote: Rather than reinventing the wheel I suggest looking at how QEMU already supports file-backed ROMs for other platforms... Note that this is about *writable* persistent memory, such as the non-time battery backed registers in a PC clock or the thing that stores EFI BIOS variables such as those holding Linux crash dumps. Yep. We already support flash backed by a block device for x86 and ARM. Then using that on SPARC would be much closer to solving the OPs problem than trying to create new code based on mmap(), ROM loading code etc. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to detect old arch?
On 15/09/2015 13:47, Mohsen Pahlevanzadeh wrote: Dear All, I have an ISO fromat via dd from an old scsi unix: // dd if=/dev/hdc of=./all_files.iso // I have the following information from my iso: /// parted ./all_of_disk.iso print Error: /home/mohsen/all_of_disk.iso: unrecognised disk label Model: (file) Disk /home/mohsen/all_of_disk.iso: 577MB Sector size (logical/physical): 512B/512B Partition Table: unknown Disk Flags: And : / fdisk -l ./all_of_disk.iso Disk ./all_of_disk.iso: 550.5 MiB, 577241088 bytes, 1127424 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes /// I think it's sysv unix. How can I detedct its' arch via qemu? --Best regards Mohsen The qemu toolset has no functions that could do this. Trying to treat it like a harddisk (parted, fdisk etc.) is not useful either. Try to unpack the iso with e.g. 7-zip (or just loop mount it). Then use the file command on various program files inside. Another technique would be to use ISO specific tools to examine the metadata of the ISO image, there may be useful info there. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to detect old arch?
On 16/09/2015 05:30, Dale R. Worley wrote: Peter Maydell writes: I have an ISO fromat via dd from an old scsi unix: // dd if=/dev/hdc of=./all_files.iso I think it's sysv unix. How can I detedct its' arch via qemu? I would start off by running strings on the iso image, to see if there's any helpful human-readable text in it that would more usefully narrow down: * what the disk actually has on it * what OS it is * what CPU architecture it's for Then you can probably make an educated guess at the image format. What you have is an ISO file system. That is, it contains a bunch of files. You can mount it on your system and read the files. Of course, the important question is "What are the files for?" Probably there is an operating system on it, but is there a reason for you to care? The data files are the important part, and you can already read them. Determining the arch of the system on which the binaries originally ran is interesting only if you want to run those particular binaries again. But it's more effective if you rebuild the binaries on your current system from the sources that should be in the ISO image. Dale Just out of curiosity, how many SysV CD-ROMs included the source code (this required a more expensive license)? Anyway, qemu is often used for running outdated operating systems just for the fun of it, no useful reason is needed. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to Use QEMU
You downloaded the source code of the emulator. Windows builds are available from http://qemu.weilnetz.de/ and http://www.omledom.com/ (Those are different builds by different people). Documentation is available at http://wiki.qemu.org/Manual On 29/09/2015 10:25, Paolo G. Santiago wrote: Good day, Hi I was browsing the net for OS emulators and I found you website. I plan to install QEMU in my Windows PC to emulate SPARC hardware then I will install on the emulated SPARC hardware Solaris 11.2. But I don't know how to use QEMU. I downloaded this file /qemu-2.4.0.1.tar.bz2/ but I don't know what to do with it. I can't find an .exe file to install. Sorry but I'm a total noob at this. Hope someone can give me a walkthrough on how to use it Looking forward to your urgent reply. Paolo Santiago Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] softmmu
On 07/10/2015 09:56, MAMEESH Rania wrote: Does all guest memory accesses require address translation, i.e., they all pass by the softmmu ? Thanks for the help. RM They all require address translation (even in the case where the addresses happen to end up the same). However the address translation may be done by either of 3 virtual MMUs (listed from slowest to fastest). - The softmmu in the qemu "translating" interpreter (kvm accelleration not used) - The page table virtualization done by kvm (it reconfigures the hardware MMU to map each guest virtual address directly to the host physical address that would result from combining what the guest OS asks the virtual MMU to do in terms of guest virtual address to guest physical address mapping with what the host OS wants the physical MMU to do in terms of mapping guest physical addresses to host physical addresses). This only does things when the guest OS and/or the host OS changes a mapping. - The double page table hardware in some CPUs, configured by kvm so the guest OS controls a hardware virtualized MMU, which the hardware then combines with the hardware real MMU mapping set by the host OS. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Qemu tcp server option not working.
On 06/11/2015 15:17, Supriti Singh wrote: Hello, I installed qemu on windows 8.1 from this link http://qemu.weilnetz.de/w64/ and the package is "qemu-20121204-w64.exe". That file is almost 3 years old, try something current. I used this command: qemu-system-x86-64 -serial tcp:localhost:20711,server -pflash="path to the image". I expect qemu process to wait for a listening process before doing anything because of "server" option. But it does not wait. The monitor is started right away. Also, if I try to do telnet on 20711, I expected some serial output. But I don't see any thing. I have tried the same setup on my other windows 8.1 with same version of qemu and it worked there. But I don't know why it does not work here. Really, I wouldn't expect a December 2012 version to work flawlessly on Windows 8.1 (NT 6.03) from 2014. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to send a message from guest machine to host
On 11/11/2015 12:12, Jimmy Situ wrote: Dear All, I looking for a method to send a message from my guest machine to the host side when my program on guest machine is finished. I have a monitor on the QEMU QMP side, it is possible to send a message by using QMP on the guest machine? Or there is a better way? Thanks Jimmy The common methods from easiest to most complicated: 1. If you want to shut down the virtual machine at that point anyway, just do so, when the shutdown tells the VM to power off, qemu will exit and/or signal QMP. 2. Have your program create a file on a shared file location, then your code on the guest (or another host) can detect its appearance and proceed. 3. Have your program send a "network" message (such as sending a packet to a port on the host) which will signal your code on the host (or another guest). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Guest Self-Suspension
I think you would have more luck using the APM or ACPI BIOS calls to cause or handle suspend. On 20/11/2015 13:34, Guy David wrote: I am running a FreeDOS guest (used as a gaming platform) on a Windows host. I wish a specific .bat script to run as soon as I resume the VM. My current attempts aim to somehow utilize the monitor from the guest itself and request it to stop the emulation. The only thing which comes to mind is a communicating by a COM port, but I can not seem to grasp how to do so. Any thoughts? Thank you, Guy Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Experimenting with x86 real mode code in qemu
On 07/12/2015 16:17, Jay Aurabind wrote:
Hello Everyone,
I am doing an experiment to emulate x86 real mode in qemu so as to
study assembly programming in GNU assembler. My current status is that
qemu exits with the error:
qemu: fatal: Trying to execute code outside RAM or ROM at 0x000a
Please help me get my expt working. Coming to the details, my simple
source files:
startup.S:
==
.globl start
.text
.code16gcc
start:
jmp main
==
main.S
==
.globl main
.text
.code16gcc
main: mov $1, %ax
mov $1, %di
mov $1, %si
mov $1, %dx
cli
hlt
==
Linker script: As x86 has the hard coded value 0x0 for the address
of first execution, I am putting my startup script which calls the
main function at 0x0:
==
OUTPUT(binary);
ENTRY(start);
SECTIONS
{
.text : { *(.text) }
.data : { *(.data) }
.bss : { *(.bss COMMON) }
. = 0x0;
.startup . : { startup.o (.text)}
}
===
My make output:
===
cc -nostdinc -nostdlib -ffreestanding -nostartfiles -nodefaultlibs
-mno-red-zone -c startup.S -o startup.o
cc -nostdinc -nostdlib -ffreestanding -nostartfiles -nodefaultlibs
-mno-red-zone -c main.S -o main.o
ld -T link.ld startup.o main.o -o bin
===
Later creating flash image:
dd if=/dev/zero of=flash.bin bs=4096 count=1024
dd if=bin of=flash.bin bs=4096 conv=notrunc
And finally starting qemu:
qemu-system-i386 -pflash flash.bin -nographic
=
That fails with the error mentioned I already mentioned :
=
WARNING: Image format was not specified for 'flash.bin' and probing guessed raw.
Automatically detecting the format is dangerous for raw
images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
qemu: fatal: Trying to execute code outside RAM or ROM at 0x000a
EAX= EBX= ECX= EDX=0663
ESI= EDI= EBP= ESP=
EIP=000afff2 EFL=0046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES = 9300
CS =f000 9b00
SS = 9300
DS = 9300
FS = 9300
GS = 9300
LDT= 8200
TR = 8b00
GDT=
IDT=
CR0=6010 CR2= CR3= CR4=
DR0= DR1= DR2= DR3=
DR6=0ff0 DR7=0400
CCS= CCD= CCO=ADDB
EFER=
FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80
FPR0= FPR1=
FPR2= FPR3=
FPR4= FPR5=
FPR6= FPR7=
XMM00= XMM01=
XMM02= XMM03=
XMM04= XMM05=
XMM06= XMM07=
===
Please let me know what I missed.
Thanking in Advance!
Aurabindo
Hard to tell from your data, but what exactly are you
putting at 0x0 == : == F000: ?
And where are you trying to place your code?
And how do you expect the use of arm/mips style flash
options to apply to an emulator which tries to emulate
the PC hardware architecture, where (amongst many other
things):
Addresses from 0xF to 0xF (and possibly more)
are reserved for the "BIOS" bootstrap ROM (such as
SeaBIOS)
Addresses from 0xC8000 to 0xE are reserved for
additional memory mappad hardware, such as older
network cards
Addresses from 0xC to 0xC7FFF (and possibly more)
are reserved for the "plug-in BIOS" on the graphics
card
Addresses from 0xA to 0xB are reserved for
memory mapping part/all of the memory on the graphics
card.
Addresses from (variable) to 0x9 is RAM but reserved
for supplemental BIOS variables.
Addresses from 0x00600 to (variable) is RAM available to
the OS (such as freeDOS or your own bare metal program)
Addresses from 0x00500 to 0x005FF is RAM but may be
reserved
Addresses from 0x00400 to 0x004FF is RAM and holds public
BIOS variables, including one saying where the
available RAM ends.
Addresses from 0x0 to 0x003FF is RAM and holds the
interrupt/exception handler table.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S
Re: [Qemu-discuss] Windows Update Virtio Drivers
On 08/12/2015 22:21, James Watson wrote: Hi, Has anyone noticed that Windows Update recently added a signed virtio driver package under the optional updates. From what I can tell: -It is the current release from the "Latest" branch (110). -Only available from Server 2012 R2 or newer. I have done testing with it and been unable to find a problem as of yet. Does anyone have information on why the stable branch was not used? Were older windows versions not included because it wasn't found to not be stable with the "latest" drivers? https://fedoraproject.org/wiki/Windows_Virtio_Drivers states the following for the "latest" release: " This provides the latest driver builds. They may be development quality, or bug free, or complete broken. Caveat emptor :)" I don't have concrete information, but it appears from other incidents that Microsoft now auto-publishes drivers submitted to them for signing, causing all kinds of havoc (such as official vendor tools trying to downgrade to the stable driver version and Windows Update forcing it back to the latest driver version in an endless loop.) Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Modern Network Interface Card emulation
On 17/12/2015 05:06, Wink Saville wrote: Yea, I saw that, but I couldn't find any motherboards with an 8254x chip (maybe that's the wrong chip), would you happen to of know of one? On Wed, Dec 16, 2015 at 7:48 PM B Cran <mailto:bruce.c...@gmail.com>> wrote: On Wed, Dec 16, 2015 at 5:54 PM, Wink Saville mailto:w...@saville.com>> wrote: > I'm wondering if there is a modern NIC emulated with qemu. By modern, I mean > a NIC that it exists on a motherboard and that I could by today on Amazon or > Newegg. Qemu can emulate an Intel e1000, which while a bit old is still being built into systems. If it is still being built into systems, it is probably as part of a larger chip/chipset such as a "northbridge", where it shares the silicon with stuff like the SATA controllers and the USB2/3 host controller. I have not specific ideas as to which Ethernet programming interface is in which of the modern chipsets. One of the best ways to find out would probably be to look in the e1000 driver files in Linux, those usually contain a list of PCI IDs of compatible adapters, which you can then lookup online to see the names of the chipsets (if not already mentioned in that list). On a similar note, I wonder if the "PC" model in qemu is still stuck emulating a now historic base chipset that originally didn't support PCIe. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to use qemu for non-interactive testing
There is a monitor and/or qmp command to simulate a "soft" press on the power button, which would trigger any OS provided clean shut down logic via ACPI/APM. There is a different monitor and/or qmp command to simulate a hard power off while still keeping the virtualization aspect of e.g. qcow2 files consistent. Finally, there is the option to simply kill the qemu process. On 19/12/2015 00:39, Wink Saville wrote: Peter, I ended up using the first technique for VersatilePB and works just fine. Now I want to be able "power off" a qemu-system-i386 and I was wondering what you might suggest? I'm hoping there might be something "easy". -- Wink On Mon, Oct 19, 2015 at 1:49 PM Wink Saville <mailto:w...@saville.com>> wrote: THANKS, I'll give those things a try! On Mon, Oct 19, 2015, 1:46 PM Peter Maydell mailto:peter.mayd...@linaro.org>> wrote: On 19 October 2015 at 20:30, Wink Saville mailto:w...@saville.com>> wrote: > I would like to use qemu in a test environment where I give a "kernel" image > to qmeu have it execute it and then when complete have qemu exit. Currently > when > executing: > > $ qemu-system-arm -M versatilepb -m 128M -nographic -kernel test.bin > ... > Hi > $ > > After test.bin prints "Hi" via the "Uart in VersatilePB" I have to > interactively press ctrl-a then press 'x' to have it return to the command > prompt. Is there a way for test.bin to cause qemu to exit without having to > type commands in the terminal. That depends on the machine you're using (in this case versatilepb). Basically if the guest binary tells the emulated hardware "please power down" via whatever that hardware's mechanism is, then QEMU will exit. For versatilepb there isn't any way to do that, but you can use QEMU's "-no-reboot" option (which turns resets into power downs) and then tell the emulated hardware to do a reset. For info on how to do a reset on the versatilepb board see the h/w docs: http://infocenter.arm.com/help/topic/com.arm.doc.dui0224i/Caccifgi.html You need to write to SYS_LOCKCTL to unlock the register and then to SYS_RESETCTL to actually request the reset. Other options you can use: * an ARM-specific option is to use the -semihosting command line option, and then you can use the semihosting ABI to exit. You need to do an angel_SWIreason_ReportException: http://infocenter.arm.com/help/topic/com.arm.doc.dui0471l/pge1358787050566.html (which is to say, set r0 to 0x18, r1 to 0x20026 and do a swi 0x123456 in ARM mode or swi 0xab (Thumb mode) or bkpt 0xab (M profile). Semihosting also lets you conveniently output to the terminal, read files or look at the command line, so it's handy for writing test cases. * use the 'expect' utility to script up the "make QEMU exit when it's printed the result" handling. It's generally nicer to avoid this, but as a last resort it's nice to have: a handful of lines of expect scripting are sufficient to do basic "start this program, then exit when some string appears in its output" control. PS: you might prefer '-display none' over '-nographic'; that gives you the serial output to standard output, but doesn't do the 'monitor on alternate screen, ctrl-a ' behaviour. You can just ctrl-c to exit qemu then. Basically -nographic is a magic combination of a bunch of options and sometimes you only want one of them. thanks -- PMM Enjoy and Merry Christmas Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to use qemu for non-interactive testing
I know that libvirt (which is huge) uses those. I am not up to date on what the specific monitor and qmp commands are or where to find out. So basically, I don't know either. Hard killing the qemu process you launched yourself should be pretty trivial using whatever tool/language you used to launch it. On 21/12/2015 15:53, Wink Saville wrote: Can you give me some pointers to implementations that use these techniques. Txs. On Mon, Dec 21, 2015, 4:02 AM Jakob Bohm mailto:jb-gnumli...@wisemo.com>> wrote: There is a monitor and/or qmp command to simulate a "soft" press on the power button, which would trigger any OS provided clean shut down logic via ACPI/APM. There is a different monitor and/or qmp command to simulate a hard power off while still keeping the virtualization aspect of e.g. qcow2 files consistent. Finally, there is the option to simply kill the qemu process. On 19/12/2015 00:39, Wink Saville wrote: > Peter, > > I ended up using the first technique for VersatilePB and works just fine. > > Now I want to be able "power off" a qemu-system-i386 and I was > wondering what you might suggest? I'm hoping there might be something > "easy". > > -- Wink > > On Mon, Oct 19, 2015 at 1:49 PM Wink Saville mailto:w...@saville.com> > <mailto:w...@saville.com <mailto:w...@saville.com>>> wrote: > > THANKS, I'll give those things a try! > > > On Mon, Oct 19, 2015, 1:46 PM Peter Maydell > mailto:peter.mayd...@linaro.org> <mailto:peter.mayd...@linaro.org <mailto:peter.mayd...@linaro.org>>> wrote: > > On 19 October 2015 at 20:30, Wink Saville mailto:w...@saville.com> > <mailto:w...@saville.com <mailto:w...@saville.com>>> wrote: > > I would like to use qemu in a test environment where I give > a "kernel" image > > to qmeu have it execute it and then when complete have qemu > exit. Currently > > when > > executing: > > > > $ qemu-system-arm -M versatilepb -m 128M -nographic > -kernel test.bin > > ... > > Hi > > $ > > > > After test.bin prints "Hi" via the "Uart in VersatilePB" I > have to > > interactively press ctrl-a then press 'x' to have it return > to the command > > prompt. Is there a way for test.bin to cause qemu to exit > without having to > > type commands in the terminal. > > That depends on the machine you're using (in this case > versatilepb). > Basically if the guest binary tells the emulated hardware "please > power down" via whatever that hardware's mechanism is, then QEMU > will exit. For versatilepb there isn't any way to do that, but you > can use QEMU's "-no-reboot" option (which turns resets into power > downs) and then tell the emulated hardware to do a reset. For info > on how to do a reset on the versatilepb board see the h/w docs: > http://infocenter.arm.com/help/topic/com.arm.doc.dui0224i/Caccifgi.html > You need to write to SYS_LOCKCTL to unlock the register and then > to SYS_RESETCTL to actually request the reset. > > Other options you can use: > * an ARM-specific option is to use the -semihosting command line > option, and then you can use the semihosting ABI to exit. > You need to do an angel_SWIreason_ReportException: > http://infocenter.arm.com/help/topic/com.arm.doc.dui0471l/pge1358787050566.html > (which is to say, set r0 to 0x18, r1 to 0x20026 and do a > swi 0x123456 in ARM mode or swi 0xab (Thumb mode) or > bkpt 0xab (M profile). > Semihosting also lets you conveniently output to the terminal, > read files or look at the command line, so it's handy for > writing test cases. > > * use the 'expect' utility to script up the "make QEMU exit > when it's printed the result" handling. It's generally nicer > to avoid this, but as a last resort it's nice to have: a > handful of lines of exp
Re: [Qemu-discuss] How to use qemu for non-interactive testing
By "manually" I meant that your outer test-automating program or script
would do it by simply sending the appropriate "kill" message/signal,
not that you as a human would have to sit at the console ready to hit
ctrl+C.
For qemu-system-i386 (and qemu-system-x86_64), you can use the same
PC BIOS request used by the Linux kernel when someone runs the "halt"
command line program. You should be able to find that somewhere in
the Linux kernel source code.
For qemu-i386 (and any of the other non-system qemu interpreters)
simply doing a normal program exit should do the trick.
On 21/12/2015 20:13, Wink Saville wrote:
Killing it "manually" is trivial usually just "ctrl-a", "x" but I want
to qemu to exit when my application completes. Following the lead from
Peter you start qemu with -no-reboot option and then "reset the cpu". So
for ARM the poweroff is:
void ac_poweroff(void) {
volatile ac_u32* pUnlockResetReg = (ac_u32*)0x1020;
volatile ac_u32* pResetReg = (ac_u32*)0x1040;
*pUnlockResetReg = 0xA05F;
*pResetReg = 0x106;
}
So I'm hoping for x86 there would be something similar.
On Mon, Dec 21, 2015 at 9:33 AM Jakob Bohm mailto:jb-gnumli...@wisemo.com>> wrote:
I know that libvirt (which is huge) uses those. I am not up to date on
what the specific monitor and qmp commands are or where to find out.
So basically, I don't know either.
Hard killing the qemu process you launched yourself should be pretty
trivial using whatever tool/language you used to launch it.
On 21/12/2015 15:53, Wink Saville wrote:
> Can you give me some pointers to implementations that use these
> techniques. Txs.
>
>
> On Mon, Dec 21, 2015, 4:02 AM Jakob Bohm mailto:jb-gnumli...@wisemo.com>
> <mailto:jb-gnumli...@wisemo.com
<mailto:jb-gnumli...@wisemo.com>>> wrote:
>
> There is a monitor and/or qmp command to simulate a "soft"
> press on the power button, which would trigger any OS
> provided clean shut down logic via ACPI/APM.
>
> There is a different monitor and/or qmp command to simulate
> a hard power off while still keeping the virtualization
> aspect of e.g. qcow2 files consistent.
>
> Finally, there is the option to simply kill the qemu
> process.
>
> On 19/12/2015 00:39, Wink Saville wrote:
> > Peter,
> >
> > I ended up using the first technique for VersatilePB and works
> just fine.
> >
> > Now I want to be able "power off" a qemu-system-i386 and I was
> > wondering what you might suggest? I'm hoping there might
be something
> > "easy".
> >
> > -- Wink
> >
> > On Mon, Oct 19, 2015 at 1:49 PM Wink Saville
mailto:w...@saville.com>
> <mailto:w...@saville.com <mailto:w...@saville.com>>
> > <mailto:w...@saville.com <mailto:w...@saville.com>
<mailto:w...@saville.com <mailto:w...@saville.com>>>> wrote:
> >
> > THANKS, I'll give those things a try!
> >
> >
> > On Mon, Oct 19, 2015, 1:46 PM Peter Maydell
> > mailto:peter.mayd...@linaro.org> <mailto:peter.mayd...@linaro.org
<mailto:peter.mayd...@linaro.org>>
> <mailto:peter.mayd...@linaro.org
<mailto:peter.mayd...@linaro.org> <mailto:peter.mayd...@linaro.org
<mailto:peter.mayd...@linaro.org>>>>
> wrote:
> >
> > On 19 October 2015 at 20:30, Wink Saville
> mailto:w...@saville.com>
<mailto:w...@saville.com <mailto:w...@saville.com>>
> > <mailto:w...@saville.com <mailto:w...@saville.com>
<mailto:w...@saville.com <mailto:w...@saville.com>>>> wrote:
> > > I would like to use qemu in a test environment
where I give
> > a "kernel" image
> > > to qmeu have it execute it and then when
complete have qemu
> > exit. Currently
> > > when
> > > executing:
> > >
> > > $ qemu-system-arm -M versatilepb -m 128M
-nographic
> > -kernel test.bin
> > > ...
> > > Hi
> > > $
> > >
>
Re: [Qemu-discuss] Getting qemu-system-i386 to use more than one core on Cortex A7 host
On 04/01/2016 13:21, Peter Maydell wrote:
On 3 January 2016 at 20:57, David Durham wrote:
Any suggestions or comments on how to do this are very welcome
... I built qemu with --target-list i386-softmmu and when I run
qemu, top only shows one qemu-system-i386 using 100% of one core
This is expected. Our current emulation is single threaded
even when emulating multiple target CPUs, so we'll only
use one host core. (We do have some helper threads for a
few IO tasks etc but those are not cpu-bound.)
There is some development work in progress to try to
make better use of multi-core hosts but it's not very
far advanced yet. (Also emulating x86 guests on arm hosts
with multiple cpus might not ever be supported because
the x86 memory model would require barriers everywhere
and it's not clear it would overall improve performance.
ARM-on-x86 is the primary initial usecase.)
thanks
-- PMM
For your information, the x86 memory model only requires
barriers in the following cases (this is somewhat
implemented on modern machines with multiple actual x86
CPU sockets, as opposed to multicore chips, it may also
be observed when using any kind of DMA/bus-master
hardware such as GPUs):
1. Instructions with the explicit "LOCK" prefix, these
require a memory barrier, then a locked read-modify-write
on a single address, then another memory barrier.
2. Explicit memory barrier instructions (there have been
a few over the years).
3. Some of the XCHG-family instructions implicitly behave
as though there was a LOCK in front.
4. On modern CPUs, the floating point ("ESC") instructions
are treated as normal instructions, the related historic
"WAIT" opcode is now a NOP (optionally throwing an
"FPU disabled" exception), (on 386 and older, floating
point instructions might postpone their memory writes
to any point up to and including the next same-CPU WAIT,
but this was never a multi-CPU barrier, just
synchronization between the CPU and FPU chips within
each two-chip CPU).
5. Some specific operations (see the architecture manuals)
typically associated with cache management, system calls
and/or thread switching also act as barriers.
6. Only a minority of instructions flush the instruction
decode (and hence TCG translation) buffers, though for
highest consistency any actual write to a memory page
with code should cause the translation of that code to
be discarded from cache.
7. If doing cycle-accurate bug-for-bug emulation of
specific CPU models, it might be necessary to exactly
model the implicit size limitations of their various
caches, such as how many page table entries are cached
by the on-CPU TLB or how many bytes ahead the
instruction decoder may look. But I don't think that
is a qemu feature anyway.
This still leaves the majority of code not doing memory barriers.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Qemu reports slightly less memory on PXE boot
On 04/01/2016 15:21, Prem Anand Haridoss wrote: On booting an iso (from the cdrom) with 8G memory, after bootup, I see 8179816 kB as the total memory available. However on PXE booting the same vCPE livecd iso over a nic, with the same 8G memory, we see only 7857512 kB as the total memory available Is this a bug in qemu? I am using version 2.5.0 (Debian 1:2.5+dfsg-1) Regards Prem Logs: = On booting iso using cdrom -- pharidos@uks2:~/work/tplaf/☸qemu-system-x86_64 -name Test -nographic -enable-kvm -cpu Haswell -smp 2 -m 8G -cdrom $iso -boot d ISOLINUX 4.05 20121210 Copyright (C) 1994-2011 H. Peter Anvin et al Press control and F then 1 for help, or ENTER to boot: live-serial init=/bin/bash Loading, please wait... bash: no job control in this shell root@(none):/# root@(none):/# cat /proc/meminfo | head -n3 MemTotal: 8179816 kB MemFree: 8138960 kB MemAvailable: 8131960 kB On pxe booting the iso over a nic - pharidos@uks2:~/☸qemu-system-x86_64 -name Test -nographic -enable-kvm -cpu Haswell -smp 2 -m 8G -netdev user,id=net0,tftp=$HOME/TPLAF//TB2/rtr2/tftp,bootfile=/gpxelinux.0 -device rtl8139,netdev=net0 -boot once=n PXELINUX 6.03 PXE 6.03-pre20 Copyright (C) 1994-2014 H. Peter Anvin et al boot: Build109.11 Loading memdisk... ok Loading ...ok ISOLINUX 4.05 20121210 Copyright (C) 1994-2011 H. Peter Anvin et al Press control and F then 1 for help, or ENTER to boot: live-serial init=/bin/bash Loading, please wait... bash: no job control in this shell root@(none):/# root@(none):/# cat /proc/meminfo | head -n3 MemTotal: 7857512 kB MemFree: 7816764 kB MemAvailable: 7809988 kB Presumably, the PXE BIOS needs some RAM for the network stack and CD-ROM emulation. Perhaps it simply loads the entire ISO image into RAM before booting it, so the boot process can play with the network adapter without loosing access to its "CD". Check if the difference is close to the size of the ISO file. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Qemu reports slightly less memory on PXE boot
On 04/01/2016 17:23, Prem Anand Haridoss wrote: From: qemu-discuss-bounces+h.prem.anand=gmail@nongnu.org [mailto:qemu-discuss-bounces+h.prem.anand=gmail@nongnu.org] On Behalf Of Jakob Bohm Sent: 04 January 2016 15:56 To: qemu-discuss@nongnu.org Subject: Re: [Qemu-discuss] Qemu reports slightly less memory on PXE boot On 04/01/2016 15:21, Prem Anand Haridoss wrote: On booting an iso (from the cdrom) with 8G memory, after bootup, I see 8179816 kB as the total memory available. However on PXE booting the same vCPE livecd iso over a nic, with the same 8G memory, we see only 7857512 kB as the total memory available Is this a bug in qemu? I am using version 2.5.0 (Debian 1:2.5+dfsg-1) Regards Prem Logs: = On booting iso using cdrom -- pharidos@uks2:~/work/tplaf/☸ qemu-system-x86_64 -name Test -nographic -enable-kvm -cpu Haswell -smp 2 -m 8G -cdrom $iso -boot d ISOLINUX 4.05 20121210 Copyright (C) 1994-2011 H. Peter Anvin et al Press control and F then 1 for help, or ENTER to boot: live-serial init=/bin/bash Loading, please wait... bash: no job control in this shell root@(none):/# root@(none):/# cat /proc/meminfo | head -n3 MemTotal:8179816 kB MemFree: 8138960 kB MemAvailable:8131960 kB On pxe booting the iso over a nic - pharidos@uks2:~/☸ qemu-system-x86_64 -name Test -nographic -enable-kvm -cpu Haswell -smp 2 -m 8G -netdev user,id=net0,tftp=$HOME/TPLAF//TB2/rtr2/tftp,bootfile=/gpxelinux.0 -device rtl8139,netdev=net0 -boot once=n >> PXELINUX 6.03 PXE 6.03-pre20 Copyright (C) 1994-2014 H. Peter Anvin et al boot: Build109.11 Loading memdisk... ok Loading ...ok ISOLINUX 4.05 20121210 Copyright (C) 1994-2011 H. Peter Anvin et al Press control and F then 1 for help, or ENTER to boot: live-serial init=/bin/bash Loading, please wait... bash: no job control in this shell root@(none):/# root@(none):/# cat /proc/meminfo | head -n3 MemTotal:7857512 kB MemFree: 7816764 kB MemAvailable:7809988 kB Presumably, the PXE BIOS needs some RAM for the network stack and CD-ROM emulation. Perhaps it simply loads the entire ISO image into RAM before booting it, so the boot process can play with the network adapter without loosing access to its "CD". Check if the difference is close to the size of the ISO file. Thanks Jakob You are right! The difference in memory matches the iso file size. So it seems that qemu uses certain portion of ram for storing the iso But shouldn't qemu allocate extra memory needed to store the iso instead of using the ram specified to the guest OS. Or is this how a real device would behave when we PXE boot? I believe this is all happening in the PXE ROM running inside the virtual machine and would be the same if the same version/brand of PXE code was used on a real machine. I don't know if all PXE ROMs on real machines do this. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Getting qemu-system-i386 to use more than one core on Cortex A7 host
On 04/01/2016 22:29, Peter Maydell wrote: On 4 January 2016 at 13:24, Jakob Bohm wrote: For your information, the x86 memory model only requires barriers in the following cases (this is somewhat implemented on modern machines with multiple actual x86 CPU sockets, as opposed to multicore chips, it may also be observed when using any kind of DMA/bus-master hardware such as GPUs): [list elided] This still leaves the majority of code not doing memory barriers. This implies that x86 has no stricter restrictions on reordering of plain loads and stores than ARM does. That surprises me -- I thought x86's memory model imposed stricter constraints on the implementation. (For instance https://en.wikipedia.org/wiki/Memory_ordering#In_symmetric_multiprocessing_.28SMP.29_microprocessor_systems lists several cases like load-after-load that ARM might reorder but x86 forbids reordering for.) But I haven't looked into the details beyond mentally tagging the situation as "here be dragons" for if/when I ever need to review any code dealing with it. Looking briefly at that table, I am unsure which items are covered by those first 3 lines they say are not permitted on x86, but are permitted on ARMv7. The key difference I am aware of is that most traditional x86 atomic operations are defined to also act as full memory barriers, meaning that any memory operations that logically occur before/after the operation also do so as seen by other CPUs (the lines in the table marked "atomic reordered with..."). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Getting qemu-system-i386 to use more than one core on Cortex A7 host
On 05/01/2016 18:35, Peter Maydell wrote: On 4 January 2016 at 22:00, Jakob Bohm wrote: On 04/01/2016 22:29, Peter Maydell wrote: On 4 January 2016 at 13:24, Jakob Bohm wrote: https://en.wikipedia.org/wiki/Memory_ordering#In_symmetric_multiprocessing_.28SMP.29_microprocessor_systems lists several cases like load-after-load that ARM might reorder but x86 forbids reordering for.) But I haven't looked into the details beyond mentally tagging the situation as "here be dragons" for if/when I ever need to review any code dealing with it. Looking briefly at that table, I am unsure which items are covered by those first 3 lines they say are not permitted on x86, but are permitted on ARMv7. For instance, x86 forbids reordering of writes with other writes (excluding a few special cases like the temporal move instructions), and in an MP system requires that writes by one processor are observed in the same order by other processors. ARM doesn't require this. So for this sequence of operations: (initial state: both locations X and Y contain 0) P1: store 1 to address X store 1 to address Y P2: load register R1 from address Y load register R2 from address X on ARM it is possible for P2 to finish with R1 == 1 and R2 == 0 (ie for P2 to observe P1's store to Y before it observes P1's store to X). On x86 this is not permitted. (Compare the ARMv8 ARM ARM rev A.h appendix K10.6 section K10.6.1, and the Intel architecture reference volume 3 section 8.2.2 "Memory ordering in P6 and more recent processor families"; in particular this is the code sequence "weakly ordered message passing problem" in the ARM ARM and the example 8-1 "stores not reordered with other stores" in the x86 documentation.) This means that if we want to emulate the x86 architecture's memory ordering guarantees on an ARM host, we need to add extra barriers after emulated loads and stores to enforce that P2 does not see the stores P1 makes2 in an order that the x86 architecture doesn't permit. (It would also be possible to use the v8 ARM load-acquire and store-release instructions rather than full on barriers, but on v7 I think barriers are the only answer.) The Load acquire/store if no conflict instruction pair was introduced halfway through the Armv6 architecture, though it may be missing on some non-A Armv7 cores, since it is not required for that processor class. Additionally, I think some ARM MMUs have page or region level memory ordering flags, including some flag combinations that break normal Arm synchronization instructions. But anyway, it might be worth allowing the P5 reordering rules on x86 if that improves the situation. It might also be worth doing some "is the host CPU too aggressively reordering" conditionals both compile time and runtime, switching between different TCG multi-core strategies depending on the exact host CPU. Another tactic could be to not let more than one virtual core have actual access to the same page if at least one of them has write access. So the minority of code that actually does do multi-core data updates to the same virtualized memory page and might thus be affected by ordering rules would cause the emulator to constantly switch the shared page back and forth, while most other code will just run along nicely using shared read or exclusive write page accesses. But in the end if x86 really makes these guarantees even in multi- socket setups (more than one physical x86 CPU in a suitable motherboard), despite the normal effects of caching, while ARM doesn't, that kind of sucks. Though we shouldn't forget that those are not the only 2 architectures involved. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Getting qemu-system-i386 to use more than one core on Cortex A7 host
On 06/01/2016 00:53, Peter Maydell wrote: On 5 January 2016 at 23:10, Jakob Bohm wrote: On 05/01/2016 18:35, Peter Maydell wrote: (It would also be possible to use the v8 ARM load-acquire and store-release instructions rather than full on barriers, but on v7 I think barriers are the only answer.) The Load acquire/store if no conflict instruction pair was introduced halfway through the Armv6 architecture, though it may be missing on some non-A Armv7 cores, since it is not required for that processor class. I think you are thinking of the load-exclusive/store-exclusive instructions, which did indeed appear in ARMv6 and provide "only store if no conflict" semantics for implementing atomic operations. Load-acquire/store-release are different and are new in ARMv8 -- they are a bit like a normal load/store with a built-in one-sided barrier: if you do a load-acquire then some normal loads/stores, other CPUs must see your load-acquire before the other operations (but loads/stores that happened before the load-acquire might still be ordered after it). Similarly if you do some loads and stores followed by a store-release then other processors must see your store-release last. (I've simplified rather here, see the architecture manual for the exact semantics.) Ahh, sorry, I am not completely up to date on aarch64 assembly yet. Additionally, I think some ARM MMUs have page or region level memory ordering flags, including some flag combinations that break normal Arm synchronization instructions. This is true but not really important for considering QEMU running on an ARM host -- all the RAM we get from the host OS will be Normal memory, not Device or Strongly-ordered. I was thinking of maybe getting kernel help, e.g. via an extension of madvise() or similar. But anyway, it might be worth allowing the P5 reordering rules on x86 if that improves the situation. It might also be worth doing some "is the host CPU too aggressively reordering" conditionals both compile time and runtime, switching between different TCG multi-core strategies depending on the exact host CPU. I'm not sure how you would test at runtime whether the CPU might decide to reorder accesses -- I think you have to assume the worst case imposed by the architecture. Basically checking for known safe CPU core models, like if e.g. "cortex A8" is safe but "cortex A9" is not, we could test for that (with appropriate OS calls because someone decided to make ARM CPUID a privileged instruction). Another tactic could be to not let more than one virtual core have actual access to the same page if at least one of them has write access. So the minority of code that actually does do multi-core data updates to the same virtualized memory page and might thus be affected by ordering rules would cause the emulator to constantly switch the shared page back and forth, while most other code will just run along nicely using shared read or exclusive write page accesses. This is an interesting idea; I guess it would need to be implemented and benchmarked to see if the overhead on typical workloads was low enough to make it make sense. But in the end if x86 really makes these guarantees even in multi- socket setups (more than one physical x86 CPU in a suitable motherboard), despite the normal effects of caching, while ARM doesn't, that kind of sucks. Though we shouldn't forget that those are not the only 2 architectures involved. It's just an unfortunate architectural philosophy mismatch, and ARM and x86 are my usual examples of the two possibilities. I think most non-x86 architectures go for a weaker memory model than x86 did, so MIPS and PPC are on the ARM end of the spectrum. I suspect it might also be about backward compatibility with millions of programs that were tested only with single CPU machines where such ordering would be a natural side effect of shared caches. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] how to setup a watchdog?
On 16/01/2016 11:43, lejeczek wrote: hi everybody, I tried to get a help on libvirt mailing list but not luck, I hope maybe here? I'm trying Qemu's watchdog. My understanding was that hardware (here qemu's watchdog) would take action, eg. cold reboot) the system if there is no ping from the OS watchdog, so I thought stopping watchdog service in VM should be a quick test. I have this in the guest: function='0x0'/> and I see /dev/watchdog in my guest. Yet nothing happens, guest runs uninterrupted. I must be missing something, an expert said it's config problem? many thanks In Linux there is a setting determining if closing the /dev/watchdog file handle (e.g. by stopping the watchdog daemon) stops the watchdog timer, or lets the time keep running until it times out and takes the configured action. You may want to check what that setting has been set to in your guest VM. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] which guest sent io request?!
On 25/01/2016 19:38, Ata Fatahi baarzi wrote: hi all is there any way to find which guest(vm) sent a typicall io request in qemu source code? and does qemu care about which guest or vm sent io request or just process that io request? Because there is (at least) one qemu process per guest and each qemu process handles only one guest, all of the qemu source code knows which guest it is handling, because it is only handling one guest. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] which guest sent io request?!
Every single variable and other state in the process pertains only to this one Guest. Some of the things that code could look at: The Process ID of the qemu process, this is unique even if two copies of the same guest is started at the same time from the exact same virtual hard drive etc. etc. But it will change if a guest is "live migrated" to another copy of qemu, usually on another physical computer. The "name" of the guest, as specified on the qemu command line. This is probably stored in some place the code can read from, but the user of qemu is not required to specify such a name at all. The path and file name of the virtual hard drive (if any). Because this is usually mounted read/write, only one guest can safely use the same file at any one time. However once again, live migrating can change this (to a path and file name on the other physical computer) and it is possible to run a guest virtual machine that has no writable hard drive, and thus no file name storing such a hard drive. However in most cases the proper solution is to look at parameters specific to whatever the code needs to use that identifier for. For example code that emulates a network card would look at the virtual hardware MAC address it is providing and the parameters that connect that virtual network card to something outside the virtual machine. Similarly, code that emulates a hard drive will look at parameters that control its access to the virtual hard drive. Code that emulates some kind of machine serial number hardware feature would read that virtual serial number from a command line parameter or from a file specified in a command line parameter. Remember that virtual machines are not real, everything is a simulation and can be set to whatever the user wants to simulate. On 26/01/2016 04:51, Ata Fatahi baarzi wrote: thank you dear, so is there an identifier, something like guest_id in source code to find it each time which guest is getting service? On Tue, Jan 26, 2016 at 1:11 AM, Jakob Bohm <mailto:jb-gnumli...@wisemo.com>> wrote: On 25/01/2016 19:38, Ata Fatahi baarzi wrote: hi all is there any way to find which guest(vm) sent a typicall io request in qemu source code? and does qemu care about which guest or vm sent io request or just process that io request? Because there is (at least) one qemu process per guest and each qemu process handles only one guest, all of the qemu source code knows which guest it is handling, because it is only handling one guest. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Problem converting from vmdk to raw format
qemu-img used the "sparse file" feature of your file system to not actually allocate disk sectors for many of the all-zero blocks in the raw file. Your iso creation tool did not. On 01/02/2016 05:42, Alexandre Schenberg wrote: thanks, was a permission problem, to change it solved the issue I have another doubt, converting it to raw generate a 126.9GB file. However df shows that the space used in the partition is only 8.x GB. And now I am converting the raw file to an iso file and the iso is really occupying space. So why the space occupied by the raw file seems to do not exists and the one from Iso does? And why the Iso is becaming so big? Thanks for the aid. 2016-02-01 2:40 GMT-02:00, Fam Zheng: On Mon, 02/01 01:53, Alexandre Schenberg wrote: Hello, I tried to convert a .vmdk file to a raw file with this command: "qemu-img convert -f vmdk /media/windows/WinXP-disk1.vmdk winxp.raw" The result was: "qemu-img: Could not open '/media/windows/WinXP-disk1.vmdk" I am sure that the vmdk file, WinXP-disk1.vmdk it is in the dir /media/windows So what I am doing wrongly? Which qemu-img version are you using? Or which commit in the case of git tree? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Problem converting from vmdk to raw format
qemu-img used the "sparse file"
feature of your file system to not
actually allocate disk sectors for many of the all-zero blocks
in
the raw file. Your iso creation tool did not.
On 01/02/2016 05:42, Alexandre Schenberg wrote:
thanks, was a permission problem, to change it solved the issue
I have another doubt, converting it to raw generate a 126.9GB file.
However df shows that the space used in the partition is only 8.x GB.
And now I am converting the raw file to an iso file and the iso is
really occupying space.
So why the space occupied by the raw file seems to do not exists and
the one from Iso does?
And why the Iso is becaming so big?
Thanks for the aid.
2016-02-01 2:40 GMT-02:00, Fam Zheng :
On Mon, 02/01 01:53, Alexandre Schenberg wrote:
Hello, I tried to convert a .vmdk file to a raw file with this
command: "qemu-img convert -f vmdk /media/windows/WinXP-disk1.vmdk
winxp.raw"
The result was: "qemu-img: Could not open
'/media/windows/WinXP-disk1.vmdk"
I am sure that the vmdk file, WinXP-disk1.vmdk it is in the dir
/media/windows
So what I am doing wrongly?
Which qemu-img version are you using? Or which commit in the case of git
tree?
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] qemu network question/problem
On 15/02/2016 16:25, Bruce R JONES wrote: I am running several qemu/kvm guests on my server. Both server and guests are running Linux. The guests images all need to be able to communicate with each other, the server and other machines on the network. Because of that I chose to setup a bridged network using tun/tap interfaces. On the server I created a bridge interface. The server interface eth0 is placed in that bridge. I also created tunnel endpoints with the tunctl command. Those tunnel endpoints also go into the bridge. The args to qemu include the following: -netdev type=tap,id=eth0,ifname=tun0,script=no,downscript=no –device virtio-net-pci,netdev=eth0 I assign IP addresses to the various interfaces. The network runs fine and I have the necessary connectivity everywhere. The problem is when there is a large amount of traffic on the server network. We have had several broadcast storms, or other heavy blasts of traffic. This seems to bring the qemu guests to their knees. I suspect it is because the qemu guests receive all of that traffic and have a hard time handling it. Since the server eth0 is part of a bridge it is in promiscuous mode, so instead of filtering traffic in the NIC by MAC address all traffic is received and sent to the bridge. The bridge has to handle it all and flood it to each of the qemu guests. Each of those guests in turn have to handle all that traffic in software. Just wondering if there is a better way to design the network, or some recommendations on how to prevent or reduce the impact of such traffic. I don't know, what you are doing is actually the usually "best" setup. Some possibilities: 1. Check if there is a bug in the qemu virtio-net-pci emulation which fails to emulate the "not promiscuous" setting as set by the Guest OS. In other words, does the qemu code on host fail to drop incoming packets that are not supposed to be delivered to the Guest (because the MAC destination is wrong and the Guest has not set the virtual NIC in promiscuous mode)? 2. Check if there is a bug in the Host linux kernel bridge code which fails to do the packet limitation work that a real Ethernet switch would do. 3. Check if the traffic blasts are actually broadcasts/multicasts that are (from an Ethernet perspective) supposed to reach the Guests. 4. If 1 or 2 is the problem, set up ebtables on the Host with rules that simply emulate the missing MAC filtering. For example packets to 12:34:56:78:9A:BC should only be delivered to the VM tap interface whose corresponding Guest NIC has that MAC address, 5. Check if there is some kind of add-on for the Linux Bridge subsystem (on the Host) to do the missing MAC filtering (similar to how the ARP subsystem can be supplemented with an ARP daemon). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] golang on linux-user
g information, either between users (qemu versus guest process) or between different virtual signals carried by the same host signal. The inspiration for all this is of cause the ways that qemu-user shares other resources, such as the process address space. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] golang on linux-user
On 01/03/2016 14:08, Peter Maydell wrote: On 1 March 2016 at 13:00, Jakob Bohm wrote: As an alternative, could it be useful to look beyond the current glibc code and see if there is a way for qemu-user to provide the full set of Linux syscall provided facilities (including signals and calls), without having to reserve some for itself. The classic test is of cause to nest qemu-user under itself to infinite depth using the same pair of qemu-user binaries repeatedly without them knowing their own nesting depth. This is possible in theory, but you basically have to ditch the idea of linking against glibc, which would be enormously painful. This is how valgrind does things, which gives it much more control. But since QEMU isn't a special purpose user-mode emulation codebase it would be really awkward -- much of our system emulation assumes and makes heavy use of not just glibc but also glib and other libraries. No need to drop glibc. Working around glibc (e.g. by issuing syscalls and accessing glibc internals) would probably be enough. Another possibility could be to make glibc issue some syscalls through wrapper functions in the qemu code. There are lots of possibilities once you don't treat glibc as a black box, For example, qemu-user could have a SIGSETXID handler which both caters to the internal needs of qemu and its libraries but also forwards the signal to any guest handlers (installed or default) for that same signal. QEMU can't register a SIGSETXID handler for its own use, because glibc refuses to let you do that. There's a comment in the code which notes that it might be possible to multiplex some guest signals onto one host signal, which would let the guest use the currently inaccessible SIGRTMAX. thanks -- PMM Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Black screen when emulating Raspberry Pi with qemu 2.5
On 08/03/2016 13:01, Per Olofsson wrote: Hello, after some minor tweaks to pixman I got qemu 2.5.0 to compile and run on OS X 10.11. I'm trying to set it up to emulate a Raspberry Pi for development, but I can't get it to boot from the raspbian linux image. I'm using: 2016-02-26-raspbian-jessie.img downloaded from https://www.raspberrypi.org/downloads/raspbian/ kernel-qemu-4.1.13-jessie downloaded from https://github.com/dhruvvyas90/qemu-rpi-kernel If I start with: /usr/local/qemu/bin/qemu-system-arm -M versatilepb -m 512 -cpu arm1176 -no-reboot -kernel kernel-qemu-4.1.13-jessie -hda 2016-02-26-raspbian-jessie.img -serial stdio Are you sure -hda filename.img causes filename.img to be used as a virtual *SDCard* (which is what Raspberry Pi has)? it prints "Uncompressing Linux... done, booting the kernel." and then the kernel hangs since it can't find a root fs: https://www.dropbox.com/s/ogmhcyser53ko32/Sk%C3%A4rmklipp%202016-03-08%2012.53.21.png The problem is that if I try to add any kernel boot options with -append, even if it's just "panic=1", all I get is a black screen for a few seconds and then it exits: /usr/local/qemu/bin/qemu-system-arm -M versatilepb -m 512 -cpu arm1176 -no-reboot -kernel kernel-qemu-4.1.13-jessie -hda 2016-02-26-raspbian-jessie.img -serial stdio -append "panic=1" https://www.dropbox.com/s/e11mzvdjcpcfylf/Sk%C3%A4rmklipp%202016-03-08%2012.57.22.png Don't know about that. If i use "root=/dev/sda" I still only get a black screen, but it doesn't exit after a few seconds so maybe it's booting? On the Raspberry Pi, root is not the whole SDcard, but a partition on it. root may even be a subitem within the first partition on the SDcard, I don't remember right now, it depends on the instructions for putting that .img file on a physical SDcard for booting a real Raspberry. I also tried compiling the latest git version, and the results are the same. I tried the experimental raspi2 machine too, but that just gives me a debugger prompt. I'm completely new to qemu so any pointers are welcome. In general, a real Raspberry Pi 1 boots like this: 1. ROM (not EPROM/Flash) code inside the Broadcom chip connects to the SDcard. 2. ROM looks on the first (FAT) partition for a magical file with a proprietary boot blob and copies the beginning of that to RAM. 3. The proprietary boot blob bootstraps the on-chip proprietary GPU and other global CPU features, then chains to the Linux kernel, using a config file on the FAT partition to specify part of the Linux command line). 4. The Linux kernel loads on the pre-initialized CPU and accesses the pre-initialized GPU for display output. 5. At some point in the boot process (maybe the kernel phase, maybe a script on an initrd) mounts the "ext2, ext3 or ext4" partition or file as Linux root, then proceeds with a mostly ordinary Debian boot process. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] User Behavior Tracking defenses in VMs
On 14/03/2016 16:54, ban...@openmailbox.org wrote: Intended for qemu-discuss /cc/ libvir-list, whonix-devel, tor-dev *** Hello. I work on WhonixOS an anonymity distro based on Tor. This feature request is related to the topics of privacy and anonymity. Its a complex topic and probably not in your area of focus but I think it has important implications because security and privacy are very much related in today's hostile computing environment. Virtualization is useful in presenting an identical environment and set of "hardware" for each user which goes a long way in creating an anonymity set of systems. That way a system attacker, advertisers and online trackers would not be able to fingerprint a user or their hardware. The problem: Tracking techniques have become more sophisticated with time. They advanced from simple cookies to browser/device fingerprinting (which Tor Browser focuses on defeating) to user behavior fingerprinting. The latter is about profiling how a user types on a keyboard or uses a mouse [2]. Keystroke dynamics is a super creepy way to track users based on how long they press keys (dwell time) and the time between key presses (gap time). This is extremely accurate at identifying individuals because of how unique these measurements are. Advertising networks (Google, Facebook...) that fingeprprint users on both the clearnet and Tor can deanonymize users. This technique is already actively used in the wild [6][7]. Potential Solutions: Since input devices are all emulated its a great opportunity to stop this profiling technique. * A security researcher designed a proof of concept plugin for Chrome browser that mitigates this. Implementing something like the PoC addon in [1] known as KeyBoardPrivacy. Some random delay in milliseconds in a 50 millisecond range for dwell and gap times for the emulated keyboards is enough to skew the values to render this attack useless while not affecting performance. * The changes made to Tor Borwser to make JS timers more coarse grained but constant (250ms for keyboard events) were not enough to stop keystroke dynamics fingerprinting because a malicious script can evict the cache and allow extrapolation of true timing events within 1-5ms accuracy .[3][5] Their goal is to instead add jitter to the timers [4]. A similar solution proposed in [4] can be implemented in all QEMU-KVM timers to mitigate both attacks. [1] https://paul.reviews/behavioral-profiling-the-password-you-cant-change/ [2] http://jcarlosnorte.com/security/2016/03/06/advanced-tor-browser-fingerprinting.html [3] https://www.lightbluetouchpaper.org/2015/07/30/double-bill-password-hashing-competition-keyboardprivacy/#comment-1288166 [4] https://trac.torproject.org/projects/tor/ticket/16110 [5] https://trac.torproject.org/projects/tor/ticket/1517 [6] http://scraping.pro/no-captcha-recaptcha-challenge/ [7] https://nakedsecurity.sophos.com/2013/11/01/facebook-to-silent-track-users-cursor-movements-to-see-which-ads-we-like-best/ Another option, when operating at a low enough level (OS, qemu or anywhere else outside the browser sandbox) is to rearrange keystroke events as follows: Regular keys (letters, digits, characters, F-keys etc.) are presented as if they are released right after each keypress (0 or 1ms delay) regardless of actual dwell time. The 4 lock keys (caps, num, scroll and unshifted insert) are similarly filtered as instantaneous presses. Arrow keys etc. are filtered so short presses are reduced to a single (0 or 1ms) short press while long (autorepeating) presses are not (because good browsers adjust the scroll rate to use the actual scroll speed rather than the autorepeat time of the keyboard). Shift keys (shift, ctrl, alt, some uses of the logo keys) are filtered to appear as if pressed 1ms before the affected key and released 1ms after. Shift key+mouse button combinations also need to be handled. All key times are rounded to multiples of some fixed granularity clock (e.g. multiples of 50ms), except for the simulated 1ms and 0ms delays above (which would then be relative to the rounded times). Additional considerations are needed for east Asian keyboard layouts (China/Japan/Korea) due to their complex extra shift keys. Some non-spying applications (browser and otherwise) may need to be exempted because their user-oriented purpose is to do unusual keyboard interpretations. I happen to make one such application. Qemu itself is another. Games that use letters as navigation keys (WASD etc.) or require precise timing of key presses is a third group. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] response to BIOS int 0x13 calls
On 14/03/2016 21:09, Dale R. Worley wrote: bilsch writes: How can an OS partition and format a drive if it doesn't know the number of sectors? I have a vague memory that in the earliest days of hard disks on PC's that you had to enter that number from documentation that came with the drive. Only to configure the BIOS itself (because there was no hardware level request the BIOS could use). On the INT 13h level, there is a much older call (other than AH=48h), which returns the size in C/H/S form, this is what OS-es use on machines without the modern LBA calls. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Difference between qemu-kvm, qemu-system-x86_64, qemu-x86_64
On 05/04/2016 05:40, Zhang Qiang wrote: For new features, I recently updated qemu-kvm 1.5.3 to qemu 2.5.0, but I saw several qemu binaries, `/usr/local/bin/qemu-x86_64` and `/usr/local/bin/qemu-system-x86_64`. It seems `qemu-system-x86_64` is the emulator program, since libvirt won't recognize `qemu-x86_64`. Then what's qemu-x86_64 for? qemu-cpuname is for running cpuname Linux programs under Linux without starting an actual cpuname virtual machine. Instead the program is run with the system configuration of the host Linux. And according to this qemu doc: http://wiki.qemu.org/KVM: > qemu-kvm fork for x86 (deprecated, use upstream QEMU now) It seems qemu is replacing qemu-kvm. But if qemu not qemu-kvm is used, is the guest CPU still provided by KVM? If not so, will the performance be worse? qemu-system-x86_64 now takes an option to specify if the virtual machine should be run using the KVM kernel module or not. In the past that option was only available in the special qemu-kvm program. See man qemu-system-x86_64 for the exact names options. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
