date:20130309

[Qemu-devel] [PATCH V14 0/5] VMXNET3 paravirtual NIC device implementation

2013-03-09 Thread Dmitry Fleytman

This set of patches implements VMWare VMXNET3 paravirtual NIC device.
The device supports of all the device features including offload capabilties,
VLANs and etc.
The device is tested on different OSes:
Fedora 15
Ubuntu 10.4
Centos 6.2
Windows 2008R2
Windows 2008 64bit
Windows 2008 32bit
Windows 2003 64bit
Windows 2003 32bit

Changes in V14:
   QOM-related fixes

Changes in V13:
   Licensing of vmxnet3.h changed to match original
   VMWare's header file license

Changes in V12:
   Reported-by: Andreas Farber 
   QOM-related fixes 

Changes in V11:
   Rebased to master HEAD to fix multiqueue compilation issues

Changes in V10:
   Reported-by: Stefan Hajnoczi 
   Issues reported by Stefan Hajnoczi fixed.

Changes in V9:
   Reported-by: Stefan Hajnoczi 
   Issues reported by Stefan Hajnoczi fixed.

Changes in V8:
   Reported-by: Stefan Hajnoczi 
   Issues reported by Stefan Hajnoczi reviewed and mostly fixed:

> +}
> +curr_src_off += src[i].iov_len;
> +}
> +return j;
> +}

The existing iov_copy() function provides equivalent functionality.  I
don't think iov_rebuild() is needed.

[DF] Done. Thanks, missed it.


> +size -= len;
> +}
> +iovec_off += iov[i].iov_len;
> +}
> +return res;
> +}

Rename this net_checksum_add_iov() and place it in net/checksum.c,
then the new dependency on net from block can be dropped.

[DF] Done. 

> +vmw_shmem_read(hwaddr addr, void *buf, int len)
>  {
>  VMW_SHPRN("SHMEM r: %" PRIx64 ", len: %d to %p", addr, len, buf);
>  cpu_physical_memory_read(addr, buf, len);
>  }

All changes to this file should be squashed with the previous patch.

[DF] Done

> +#ifdef VMXNET_DEBUG_SHMEM_ACCESS
> +#define VMW_SHPRN(fmt, ...)  
>  \
> +do { 
>  \
> +printf("[%s][SH][%s]: " fmt "\n", VMXNET_DEVICE_NAME, __func__,  
>  \
> +## __VA_ARGS__); 
>  \
> +} while (0)
> +#else
> +#define VMW_SHPRN(fmt, ...) do {} while (0)
> +#endif

Please use QEMU tracing.  It eliminates all this boilerplate and
conditional compilation.  Tracing can be enabled/disabled at runtime
and works with SystemTap/DTrace.  See docs/tracing.txt.

[DF] We'd like to stick with compile time logic in this case becase of 2 
reasons:
[DF] 1. These printouts are intended for reverse engineering/development only 
and there is
[DF]no need to enable them at run time
[DF] 2. There is a big number of printouts, all driver-device communication is 
traced,
[DF]they hit performance even on strongest x86 in case of run-time logic


> +struct eth_header {
> +uint8_t  h_dest[ETH_ALEN];   /* destination eth addr */
> +uint8_t  h_source[ETH_ALEN]; /* source ether addr*/
> +uint16_t h_proto;/* packet type ID field */
> +};

Looks like it's copy-pasted stuff from /usr/include/linux/if_*.h,
/usr/include/netinet/*.h, and friends.  If the system-wide headers are
included names will collide for some of the macros at least.

Did you check if the slirp/ definitions can be reused?

[DF] Yes, you are right. This is copy-pasted from different places.
[DF] Slips definishing do not fully cover our needs.


I'd rather we import network header definitions once in a generic
place into the source tree.  That way vmxnet and other components
don't need to redefine these structs.

[DF] Exaclty! Our intention is to create generic header with network 
definitions and make everyone use it.
[DF] We can move our header to some shared place if you want, however I'd do it 
in parallel with cleanup
[DF] of similar definitions in existing code and this is a big change that os 
out of scope of these patches.

> + 
> *===*/

Is this huge comment box a sign that the code should be split into a
foo_tx.c and an foo_rx.c file?

[DF] As for me this file is not that big to be splitted (<800 lines), however 
I'll do this if you insist :)

> +size_t vmxnet_tx_pkt_send(VmxnetTxPktH pkt, NetClientState *vc)

'vc' is an old name that was used for VLANClientState.  The struct has
since been renamed to NetClientState and the rest of QEMU uses 'nc'
instead of 'vc'.

[DF] Fixed. Thanks.

> +/* tx module context handle */
> +typedef void *VmxnetTxPktH;

Forward-declaring the struct is nicer:

typedef struct VmxnetTxPkt VmxnetTxPkt;

The definition of VmxnetTxPkt is still hidden from the caller but you
avoid the void* and casting.  In vmxnet_pkt.c define using:

struct VmxnetTxPkt {
...
};

[DF] Agree, fixed. Thanks.


> diff --git a/hw/vmxnet_utils.h b/hw/vmxnet_utils.h
> index 7fd9a01..fac3b7b 100644
> --- a/hw/vmxnet_utils.h
> +++ b/hw/vmxnet_utils.h

Please squash these fixes into the previous patch.


[DF] Oops, my bad. Fixed.


> +uint8_t msix_used;
> +/* Whether MSI supp

[Qemu-devel] [PATCH V14 1/5] Checksum-related utility functions

2013-03-09 Thread Dmitry Fleytman

net_checksum_add_cont()
checksum calculation for scattered data with odd chunk sizes

net_raw_checksum()
checksum calculation for a buffer

Signed-off-by: Dmitry Fleytman 
Signed-off-by: Yan Vugenfirer 
---
 include/net/checksum.h | 14 +-
 net/checksum.c | 13 +++--
 2 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/include/net/checksum.h b/include/net/checksum.h
index 1f05298..3e7b93d 100644
--- a/include/net/checksum.h
+++ b/include/net/checksum.h
@@ -20,10 +20,22 @@
 
 #include 
 
-uint32_t net_checksum_add(int len, uint8_t *buf);
+uint32_t net_checksum_add_cont(int len, uint8_t *buf, int seq);
 uint16_t net_checksum_finish(uint32_t sum);
 uint16_t net_checksum_tcpudp(uint16_t length, uint16_t proto,
  uint8_t *addrs, uint8_t *buf);
 void net_checksum_calculate(uint8_t *data, int length);
 
+static inline uint32_t
+net_checksum_add(int len, uint8_t *buf)
+{
+return net_checksum_add_cont(len, buf, 0);
+}
+
+static inline uint16_t
+net_raw_checksum(uint8_t *data, int length)
+{
+return net_checksum_finish(net_checksum_add(length, data));
+}
+
 #endif /* QEMU_NET_CHECKSUM_H */
diff --git a/net/checksum.c b/net/checksum.c
index 9919b2e..4fa5563 100644
--- a/net/checksum.c
+++ b/net/checksum.c
@@ -20,16 +20,17 @@
 #define PROTO_TCP  6
 #define PROTO_UDP 17
 
-uint32_t net_checksum_add(int len, uint8_t *buf)
+uint32_t net_checksum_add_cont(int len, uint8_t *buf, int seq)
 {
 uint32_t sum = 0;
 int i;
 
-for (i = 0; i < len; i++) {
-   if (i & 1)
-   sum += (uint32_t)buf[i];
-   else
-   sum += (uint32_t)buf[i] << 8;
+for (i = seq; i < seq + len; i++) {
+if (i & 1) {
+sum += (uint32_t)buf[i - seq];
+} else {
+sum += (uint32_t)buf[i - seq] << 8;
+}
 }
 return sum;
 }
-- 
1.8.1.2

[Qemu-devel] [PATCH V14 3/5] Common definitions for VMWARE devices

2013-03-09 Thread Dmitry Fleytman

Signed-off-by: Dmitry Fleytman 
Signed-off-by: Yan Vugenfirer 
---
 hw/vmware_utils.h | 143 ++
 hw/vmxnet_debug.h | 115 ++
 include/net/eth.h | 347 ++
 net/Makefile.objs |   1 +
 net/eth.c | 217 ++
 5 files changed, 823 insertions(+)
 create mode 100644 hw/vmware_utils.h
 create mode 100644 hw/vmxnet_debug.h
 create mode 100644 include/net/eth.h
 create mode 100644 net/eth.c

diff --git a/hw/vmware_utils.h b/hw/vmware_utils.h
new file mode 100644
index 000..5307e2c
--- /dev/null
+++ b/hw/vmware_utils.h
@@ -0,0 +1,143 @@
+/*
+ * QEMU VMWARE paravirtual devices - auxiliary code
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman 
+ * Yan Vugenfirer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef VMWARE_UTILS_H
+#define VMWARE_UTILS_H
+
+#include "qemu/range.h"
+
+#ifndef VMW_SHPRN
+#define VMW_SHPRN(fmt, ...) do {} while (0)
+#endif
+
+/*
+ * Shared memory access functions with byte swap support
+ * Each function contains printout for reverse-engineering needs
+ *
+ */
+static inline void
+vmw_shmem_read(hwaddr addr, void *buf, int len)
+{
+VMW_SHPRN("SHMEM r: %" PRIx64 ", len: %d to %p", addr, len, buf);
+cpu_physical_memory_read(addr, buf, len);
+}
+
+static inline void
+vmw_shmem_write(hwaddr addr, void *buf, int len)
+{
+VMW_SHPRN("SHMEM w: %" PRIx64 ", len: %d to %p", addr, len, buf);
+cpu_physical_memory_write(addr, buf, len);
+}
+
+static inline void
+vmw_shmem_rw(hwaddr addr, void *buf, int len, int is_write)
+{
+VMW_SHPRN("SHMEM r/w: %" PRIx64 ", len: %d (to %p), is write: %d",
+  addr, len, buf, is_write);
+
+cpu_physical_memory_rw(addr, buf, len, is_write);
+}
+
+static inline void
+vmw_shmem_set(hwaddr addr, uint8 val, int len)
+{
+int i;
+VMW_SHPRN("SHMEM set: %" PRIx64 ", len: %d (value 0x%X)", addr, len, val);
+
+for (i = 0; i < len; i++) {
+cpu_physical_memory_write(addr + i, &val, 1);
+}
+}
+
+static inline uint32_t
+vmw_shmem_ld8(hwaddr addr)
+{
+uint8_t res = ldub_phys(addr);
+VMW_SHPRN("SHMEM load8: %" PRIx64 " (value 0x%X)", addr, res);
+return res;
+}
+
+static inline void
+vmw_shmem_st8(hwaddr addr, uint8_t value)
+{
+VMW_SHPRN("SHMEM store8: %" PRIx64 " (value 0x%X)", addr, value);
+stb_phys(addr, value);
+}
+
+static inline uint32_t
+vmw_shmem_ld16(hwaddr addr)
+{
+uint16_t res = lduw_le_phys(addr);
+VMW_SHPRN("SHMEM load16: %" PRIx64 " (value 0x%X)", addr, res);
+return res;
+}
+
+static inline void
+vmw_shmem_st16(hwaddr addr, uint16_t value)
+{
+VMW_SHPRN("SHMEM store16: %" PRIx64 " (value 0x%X)", addr, value);
+stw_le_phys(addr, value);
+}
+
+static inline uint32_t
+vmw_shmem_ld32(hwaddr addr)
+{
+uint32_t res = ldl_le_phys(addr);
+VMW_SHPRN("SHMEM load32: %" PRIx64 " (value 0x%X)", addr, res);
+return res;
+}
+
+static inline void
+vmw_shmem_st32(hwaddr addr, uint32_t value)
+{
+VMW_SHPRN("SHMEM store32: %" PRIx64 " (value 0x%X)", addr, value);
+stl_le_phys(addr, value);
+}
+
+static inline uint64_t
+vmw_shmem_ld64(hwaddr addr)
+{
+uint64_t res = ldq_le_phys(addr);
+VMW_SHPRN("SHMEM load64: %" PRIx64 " (value %" PRIx64 ")", addr, res);
+return res;
+}
+
+static inline void
+vmw_shmem_st64(hwaddr addr, uint64_t value)
+{
+VMW_SHPRN("SHMEM store64: %" PRIx64 " (value %" PRIx64 ")", addr, value);
+stq_le_phys(addr, value);
+}
+
+/* Macros for simplification of operations on array-style registers */
+
+/*
+ * Whether  lies inside of array-style register defined by ,
+ * number of elements () and element size ()
+ *
+*/
+#define VMW_IS_MULTIREG_ADDR(addr, base, cnt, regsize) \
+range_covers_byte(base, cnt * regsize, addr)
+
+/*
+ * Returns index of given register () in array-style register defined by
+ *  and element size ()
+ *
+*/
+#define VMW_MULTIREG_IDX_BY_ADDR(addr, base, regsize)  \
+(((addr) - (base)) / (regsize))
+
+#endif
diff --git a/hw/vmxnet_debug.h b/hw/vmxnet_debug.h
new file mode 100644
index 000..96dae0f
--- /dev/null
+++ b/hw/vmxnet_debug.h
@@ -0,0 +1,115 @@
+/*
+ * QEMU VMWARE VMXNET* paravirtual NICs - debugging facilities
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman 
+ * Tamir Shomer 
+ * Yan Vugenfirer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef _QEMU_VMXNET_DEBUG_H
+#define _QEMU_VMXNET_DEBUG_H
+
+#define VMXNET_DEVICE_NAME "vmxnet3"
+
+/* #define VMXNET_DEBUG_CB */
+#define VMXNET

[Qemu-devel] [PATCH V14 4/5] Packet abstraction for VMWARE network devices

2013-03-09 Thread Dmitry Fleytman

Signed-off-by: Dmitry Fleytman 
Signed-off-by: Yan Vugenfirer 
---
 hw/Makefile.objs   |   1 +
 hw/vmxnet_rx_pkt.c | 187 ++
 hw/vmxnet_rx_pkt.h | 174 
 hw/vmxnet_tx_pkt.c | 567 +
 hw/vmxnet_tx_pkt.h | 148 ++
 5 files changed, 1077 insertions(+)
 create mode 100644 hw/vmxnet_rx_pkt.c
 create mode 100644 hw/vmxnet_rx_pkt.h
 create mode 100644 hw/vmxnet_tx_pkt.c
 create mode 100644 hw/vmxnet_tx_pkt.h

diff --git a/hw/Makefile.objs b/hw/Makefile.objs
index 40ebe46..14922cb 100644
--- a/hw/Makefile.objs
+++ b/hw/Makefile.objs
@@ -119,6 +119,7 @@ common-obj-$(CONFIG_PCNET_PCI) += pcnet-pci.o
 common-obj-$(CONFIG_PCNET_COMMON) += pcnet.o
 common-obj-$(CONFIG_E1000_PCI) += e1000.o
 common-obj-$(CONFIG_RTL8139_PCI) += rtl8139.o
+common-obj-$(CONFIG_VMXNET3_PCI) += vmxnet_tx_pkt.o vmxnet_rx_pkt.o
 
 common-obj-$(CONFIG_SMC91C111) += smc91c111.o
 common-obj-$(CONFIG_LAN9118) += lan9118.o
diff --git a/hw/vmxnet_rx_pkt.c b/hw/vmxnet_rx_pkt.c
new file mode 100644
index 000..a40e346
--- /dev/null
+++ b/hw/vmxnet_rx_pkt.c
@@ -0,0 +1,187 @@
+/*
+ * QEMU VMWARE VMXNET* paravirtual NICs - RX packets abstractions
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman 
+ * Tamir Shomer 
+ * Yan Vugenfirer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "vmxnet_rx_pkt.h"
+#include "net/eth.h"
+#include "qemu-common.h"
+#include "qemu/iov.h"
+#include "net/checksum.h"
+#include "net/tap.h"
+
+/*
+ * RX packet may contain up to 2 fragments - rebuilt eth header
+ * in case of VLAN tag stripping
+ * and payload received from QEMU - in any case
+ */
+#define VMXNET_MAX_RX_PACKET_FRAGMENTS (2)
+
+struct VmxnetRxPkt {
+struct virtio_net_hdr virt_hdr;
+uint8_t ehdr_buf[ETH_MAX_L2_HDR_LEN];
+struct iovec vec[VMXNET_MAX_RX_PACKET_FRAGMENTS];
+uint16_t vec_len;
+uint32_t tot_len;
+uint16_t tci;
+bool vlan_stripped;
+bool has_virt_hdr;
+eth_pkt_types_e packet_type;
+
+/* Analysis results */
+bool isip4;
+bool isip6;
+bool isudp;
+bool istcp;
+};
+
+void vmxnet_rx_pkt_init(struct VmxnetRxPkt **pkt, bool has_virt_hdr)
+{
+struct VmxnetRxPkt *p = g_malloc0(sizeof *p);
+p->has_virt_hdr = has_virt_hdr;
+*pkt = p;
+}
+
+void vmxnet_rx_pkt_uninit(struct VmxnetRxPkt *pkt)
+{
+g_free(pkt);
+}
+
+struct virtio_net_hdr *vmxnet_rx_pkt_get_vhdr(struct VmxnetRxPkt *pkt)
+{
+assert(pkt);
+return &pkt->virt_hdr;
+}
+
+void vmxnet_rx_pkt_attach_data(struct VmxnetRxPkt *pkt, const void *data,
+   size_t len, bool strip_vlan)
+{
+uint16_t tci = 0;
+uint16_t ploff;
+assert(pkt);
+pkt->vlan_stripped = false;
+
+if (strip_vlan) {
+pkt->vlan_stripped = eth_strip_vlan(data, pkt->ehdr_buf, &ploff, &tci);
+}
+
+if (pkt->vlan_stripped) {
+pkt->vec[0].iov_base = pkt->ehdr_buf;
+pkt->vec[0].iov_len = ploff - sizeof(struct vlan_header);
+pkt->vec[1].iov_base = (uint8_t *) data + ploff;
+pkt->vec[1].iov_len = len - ploff;
+pkt->vec_len = 2;
+pkt->tot_len = len - ploff + sizeof(struct eth_header);
+} else {
+pkt->vec[0].iov_base = (void *)data;
+pkt->vec[0].iov_len = len;
+pkt->vec_len = 1;
+pkt->tot_len = len;
+}
+
+pkt->tci = tci;
+
+eth_get_protocols(data, len, &pkt->isip4, &pkt->isip6,
+&pkt->isudp, &pkt->istcp);
+}
+
+void vmxnet_rx_pkt_dump(struct VmxnetRxPkt *pkt)
+{
+#ifdef VMXNET_RX_PKT_DEBUG
+VmxnetRxPkt *pkt = (VmxnetRxPkt *)pkt;
+assert(pkt);
+
+printf("RX PKT: tot_len: %d, vlan_stripped: %d, vlan_tag: %d\n",
+  pkt->tot_len, pkt->vlan_stripped, pkt->tci);
+#endif
+}
+
+void vmxnet_rx_pkt_set_packet_type(struct VmxnetRxPkt *pkt,
+eth_pkt_types_e packet_type)
+{
+assert(pkt);
+
+pkt->packet_type = packet_type;
+
+}
+
+eth_pkt_types_e vmxnet_rx_pkt_get_packet_type(struct VmxnetRxPkt *pkt)
+{
+assert(pkt);
+
+return pkt->packet_type;
+}
+
+size_t vmxnet_rx_pkt_get_total_len(struct VmxnetRxPkt *pkt)
+{
+assert(pkt);
+
+return pkt->tot_len;
+}
+
+void vmxnet_rx_pkt_get_protocols(struct VmxnetRxPkt *pkt,
+ bool *isip4, bool *isip6,
+ bool *isudp, bool *istcp)
+{
+assert(pkt);
+
+*isip4 = pkt->isip4;
+*isip6 = pkt->isip6;
+*isudp = pkt->isudp;
+*istcp = pkt->istcp;
+}
+
+struct iovec *vmxnet_rx_pkt_get_iovec(struct VmxnetRxPkt *pkt)
+{
+assert(pkt);
+
+return pkt->vec;
+}
+
+void vmxnet_rx_pkt_set_vhdr(struct VmxnetRxPkt *pkt,
+struct virtio_net_hdr *vhdr)
+{
+assert(pkt);
+
+memcpy(&pkt->virt_hdr, vhdr, sizeof pkt->virt

[Qemu-devel] [PATCH V14 2/5] net: iovec checksum calculator

2013-03-09 Thread Dmitry Fleytman

Signed-off-by: Dmitry Fleytman 
Signed-off-by: Yan Vugenfirer 
---
 include/net/checksum.h | 12 
 net/checksum.c | 29 +
 2 files changed, 41 insertions(+)

diff --git a/include/net/checksum.h b/include/net/checksum.h
index 3e7b93d..80203fb 100644
--- a/include/net/checksum.h
+++ b/include/net/checksum.h
@@ -38,4 +38,16 @@ net_raw_checksum(uint8_t *data, int length)
 return net_checksum_finish(net_checksum_add(length, data));
 }
 
+/**
+ * net_checksum_add_iov: scatter-gather vector checksumming
+ *
+ * @iov: input scatter-gather array
+ * @iov_cnt: number of array elements
+ * @iov_off: starting iov offset for checksumming
+ * @size: length of data to be checksummed
+ */
+uint32_t net_checksum_add_iov(const struct iovec *iov,
+  const unsigned int iov_cnt,
+  uint32_t iov_off, uint32_t size);
+
 #endif /* QEMU_NET_CHECKSUM_H */
diff --git a/net/checksum.c b/net/checksum.c
index 4fa5563..14c0855 100644
--- a/net/checksum.c
+++ b/net/checksum.c
@@ -15,6 +15,7 @@
  *  along with this program; if not, see .
  */
 
+#include "qemu-common.h"
 #include "net/checksum.h"
 
 #define PROTO_TCP  6
@@ -84,3 +85,31 @@ void net_checksum_calculate(uint8_t *data, int length)
 data[14+hlen+csum_offset]   = csum >> 8;
 data[14+hlen+csum_offset+1] = csum & 0xff;
 }
+
+uint32_t
+net_checksum_add_iov(const struct iovec *iov, const unsigned int iov_cnt,
+ uint32_t iov_off, uint32_t size)
+{
+size_t iovec_off, buf_off;
+unsigned int i;
+uint32_t res = 0;
+uint32_t seq = 0;
+
+iovec_off = 0;
+buf_off = 0;
+for (i = 0; i < iov_cnt && size; i++) {
+if (iov_off < (iovec_off + iov[i].iov_len)) {
+size_t len = MIN((iovec_off + iov[i].iov_len) - iov_off , size);
+void *chunk_buf = iov[i].iov_base + (iov_off - iovec_off);
+
+res += net_checksum_add_cont(len, chunk_buf, seq);
+seq += len;
+
+buf_off += len;
+iov_off += len;
+size -= len;
+}
+iovec_off += iov[i].iov_len;
+}
+return res;
+}
-- 
1.8.1.2

Re: [Qemu-devel] [PATCH 1/7] block: only force IO completion in .bdrv_truncate if we are shrinking

2013-03-09 Thread Peter Lieven

Am 08.03.2013 12:56, schrieb Kevin Wolf:
> Am 08.03.2013 um 12:46 hat Peter Lieven geschrieben:
>>
>> Am 08.03.2013 um 10:35 schrieb Kevin Wolf :
>>
>>> Am 08.03.2013 um 10:23 hat Paolo Bonzini geschrieben:
 Il 08/03/2013 08:53, Peter Lieven ha scritto:
>>
>> I think the fix is to only call it for the monitor command.  Optionally,
>> when shrinking, assert that there are no requests in flight.
>
> Okay.
>
> What is the plan? just fix this or fix the whole thing (which seems to
> be some
> work).
>
> The suggested patch from Jeff will break iscsi_truncate as bs->growable
> is 1 currently.

 I guess it's up to Kevin and Jeff.
>>>
>>> Someone needs to send a fix for the qcow1 (and probably vmdk)
>>> regression, otherwise I'll have to revert the patch.
>>
>> What about Paolos suggestion to call bdrv_drain_all() from the block_resize
>> command and not in bdrv_truncate? It is not a the complete solution, but it
>> will fix the regression. However, what happens if someone resizes a qcow2
>> device?
> 
> I suppose you mean qcow1? It doesn't support resizing images, but even
> if it did, this shouldn't be a problem: The problematic case is the call
> of bdrv_drain_all() during a read/write function, which would have to
> wait for itself to complete. As soon as you restrict the
> bdrv_drain_all() to the monitor, waiting for in-flight I/Os should just
> work.

Ok, then please ignore / revert the patch that added bdrv_drain_all() in 
bdrv_truncate() and
I will sent a new one that adds bdrv_drain_all() to qmp_block_resize().

Peter


> 
> Kevin
>

Re: [Qemu-devel] [RFC PATCH v1 2/4] bitops: Add UInt32StateInfo and helper functions

2013-03-09 Thread Blue Swirl

On Thu, Mar 7, 2013 at 2:26 AM, Peter Maydell  wrote:
> On 7 March 2013 10:00, Peter Crosthwaite  wrote:
>> Not 100% accurate. My goal here it to control (or wrap) only guest
>> accesses, in the first instance via the Memory API, but other forms of
>> guest access are perfectly valid as well, and PCI config space, would
>> be a good example. If we are going to share code however, we will need
>> to make changes PCI side, as that uint8_t with compulsory accessors is
>> too verbose to be used in devices.
>
> We should just use the memory API. If other bits of QEMU like PCI
> don't care to update their interfaces to be memory API then they
> don't get to use your new features. I don't see any point in defining
> a new interface at a level below the memory API for their benefit.

I'm also leaning towards memory API approach, it should be better in
the long term since all devices should use that sooner or later.

>
> -- PMM

Re: [Qemu-devel] [PATCHv3 2/2] bridge helper: support conf dirs

2013-03-09 Thread Blue Swirl

On Thu, Mar 7, 2013 at 6:32 AM, Doug Goldstein  wrote:
> Allow the bridge helper to take a config directory rather than having to
> specify every file in the directory manually via an include statement.
>
> Signed-off-by: Doug Goldstein 
> CC: Anthony Liguori 
> CC: Richa Marwaha 
> CC: Corey Bryant 
> TO: qemu-devel@nongnu.org
> ---
>  qemu-bridge-helper.c | 55 
> 
>  1 file changed, 55 insertions(+)
>
> diff --git a/qemu-bridge-helper.c b/qemu-bridge-helper.c
> index 95486e7..cebfcf8 100644
> --- a/qemu-bridge-helper.c
> +++ b/qemu-bridge-helper.c
> @@ -16,6 +16,7 @@
>  #include "config-host.h"
>
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -70,12 +71,28 @@ static void usage(void)
>  "Usage: qemu-bridge-helper [--use-vnet] --br=bridge 
> --fd=unixfd\n");
>  }
>
> +static int filter_bridge_conf_dir(const struct dirent *entry)
> +{
> +ssize_t len = strlen(entry->d_name);
> +
> +/* We only want files ending in .conf */
> +if (len > 5 &&
> +strcmp(".conf", &entry->d_name[len-5]) == 0) {
> +return 1;
> +}
> +
> +return 0;
> +}
> +
>  static int parse_acl_file(const char *filename, ACLList *acl_list)
>  {
>  FILE *f;
>  char line[4096];
>  int ret = -EINVAL;
>  ACLRule *acl_rule;
> +struct dirent **include_list = NULL;
> +int i, include_count = 0;
> +char *conf_file;
>
>  f = fopen(filename, "r");
>  if (f == NULL) {
> @@ -137,6 +154,37 @@ static int parse_acl_file(const char *filename, ACLList 
> *acl_list)
>  snprintf(acl_rule->iface, IFNAMSIZ, "%s", arg);
>  }
>  QSIMPLEQ_INSERT_TAIL(acl_list, acl_rule, entry);
> +} else if (strcmp(cmd, "includedir") == 0) {
> +include_count = scandir(arg, &include_list,
> +filter_bridge_conf_dir, alphasort);
> +if (include_count < 0) {
> +ret = -errno;
> +fprintf(stderr, "Unable to retrieve conf files from '%s': 
> %s\n",
> +arg, strerror(errno));
> +goto failure;
> +}
> +
> +for (i = 0; i < include_count; i++) {
> +if (asprintf(&conf_file, "%s/%s", arg,

Please use g_strdup_printf() and g_free() instead. This will make the
check go away too since it will not fail.

> + include_list[i]->d_name) < 0) {
> +fprintf(stderr, "Failed to allocate memory for "
> +"file path: %s/%s\n",
> +arg, include_list[i]->d_name);
> +ret = -ENOMEM;
> +goto failure;
> +}
> +
> +/* ignore errors like 'include' cmd */
> +parse_acl_file(conf_file, acl_list);
> +
> +free(conf_file);
> +free(include_list[i]);
> +include_list[i] = NULL;
> +}
> +free(include_list);
> +include_list = NULL;
> +include_count = 0;
> +
>  } else if (strcmp(cmd, "include") == 0) {
>  /* ignore errors */
>  parse_acl_file(arg, acl_list);
> @@ -152,6 +200,13 @@ static int parse_acl_file(const char *filename, ACLList 
> *acl_list)
>  failure:
>  fclose(f);
>
> +if (include_list) {

This check is somewhat redundant since the for loop below will also do
nothing for cases where include_count is either zero (freed) or < 0
(failure).

> +for (i = 0; i < include_count; i++) {
> +free(include_list[i]);
> +}
> +free(include_list);
> +}
> +
>  return ret;
>  }
>
> --
> 1.7.12.4
>
>

[Qemu-devel] [PATCH 1/2] baum: fix build

2013-03-09 Thread Blue Swirl

08744c98115cfa144ed3493556024e400b2e2573 removed hw/baum.h
but did not adjust hw/baum.c, breaking build. Fix.

Signed-off-by: Blue Swirl 
---
 hw/baum.c |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/hw/baum.c b/hw/baum.c
index d75b150..d8919d5 100644
--- a/hw/baum.c
+++ b/hw/baum.c
@@ -25,7 +25,6 @@
 #include "char/char.h"
 #include "qemu/timer.h"
 #include "usb.h"
-#include "baum.h"
 #include 
 #include 
 #include 
-- 
1.7.2.5

[Qemu-devel] [PATCH 2/2] qemu-char: fix win32 build

2013-03-09 Thread Blue Swirl

96c6384776d631839a9c8fe02bf135f9ba22586c did not adjust
Win32 #ifdeffery properly, breaking build in later commits. Fix.

Signed-off-by: Blue Swirl 
---
 qemu-char.c |6 --
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/qemu-char.c b/qemu-char.c
index b82d643..04aa589 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -535,8 +535,6 @@ int send_all(int fd, const void *_buf, int len1)
 }
 #endif /* !_WIN32 */
 
-#ifndef _WIN32
-
 typedef struct IOWatchPoll
 {
 GSource *src;
@@ -634,6 +632,7 @@ static guint io_add_watch_poll(GIOChannel *channel,
 return tag;
 }
 
+#ifndef _WIN32
 static GIOChannel *io_channel_from_fd(int fd)
 {
 GIOChannel *chan;
@@ -649,6 +648,7 @@ static GIOChannel *io_channel_from_fd(int fd)
 
 return chan;
 }
+#endif
 
 static GIOChannel *io_channel_from_socket(int fd)
 {
@@ -699,6 +699,8 @@ static int io_channel_send_all(GIOChannel *fd, const void 
*_buf, int len1)
 return len1 - len;
 }
 
+#ifndef _WIN32
+
 typedef struct FDCharDriver {
 CharDriverState *chr;
 GIOChannel *fd_in, *fd_out;
-- 
1.7.2.5

[Qemu-devel] [PATCH] hw/baum.c: fix build error about baum.h

2013-03-09 Thread Hu Tao

commit 08744c98115cfa144ed3493556024e400b2e2573 removes baum.h, but baum.c
still includes it. Remove it from baum.c.

Signed-off-by: Hu Tao 
---
 hw/baum.c |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/hw/baum.c b/hw/baum.c
index d75b150..d8919d5 100644
--- a/hw/baum.c
+++ b/hw/baum.c
@@ -25,7 +25,6 @@
 #include "char/char.h"
 #include "qemu/timer.h"
 #include "usb.h"
-#include "baum.h"
 #include 
 #include 
 #include 
-- 
1.7.2.5

[Qemu-devel] [with today's upstream git] Invoking QMP via telnet fails to start VNC

2013-03-09 Thread Kashyap Chamarthy

Heya,

After building qemu, I tried to invoke QMP by via telnet:

Build:
==
 #./configure --target-list=x86_64-softmmu --disable-werror
--enable-debug ; make -j5
==

Invoke QMP:
==
kashyap@qemu$ ./x86_64-softmmu/qemu-system-x86_64 --enable-kvm -smp 2
-m 2048 /export/images/el6box1.qcow2 -qmp tcp:localhost:,server
--monitor stdio
QEMU waiting for connection on: tcp:127.0.0.1:,server
Failed to start VNC server on `localhost:0,to=99': error parsing to= argument
kashyap@qemu$
==

The moment I tried to connect via telnet from a different terminal, it
fails this way:
==
kashyap@~$ telnet localhost 
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.
kashyap@~$
==

Any hints ?  I'm sure this worked for few weeks ago.

Re: [Qemu-devel] [PULL 00/18] hw/ reorganization, part 1

2013-03-09 Thread Paolo Bonzini

Il 05/03/2013 18:17, Paolo Bonzini ha scritto:
> This includes the mechanical changes, as well as moving stuff
> out of hw/.  Boards and devices referencing CPU are moved to
> per-arch subdirectories.  This part was uncontroversial, and
> I have already posted it separately in this exact shape.
> 
> The following changes since commit 288f1e3f87ec24abeac38399f175fe74243f7bc5:
> 
>   cadence_gem: Add debug msgs for rx desc movement (2013-02-28 18:49:24 +)
> 
> are available in the git repository at:
> 
>   git://github.com/bonzini/qemu.git hw-dirs
> 
> for you to fetch changes up to 7a2771d1541ec9a0c585e9b853e5f4dc036919ad:
> 
>   sh: move files referencing CPU to hw/sh4/ (2013-03-01 15:01:20 +0100)

Rebased and repushed to the same location, new top commit is
15e87f983a73f512ac73ae5fc6759157f0a9e0e1.

Changes that conflicted included:

- the ARM KVM merge

- the removal of baum.h and msmouse.h

- the conversion of dataplane to AioContext

Paolo

> 
> Paolo Bonzini (18):
>   hw: move char backends to backends/
>   hw: move fifo.[ch] to libqemuutil
>   hw: move qdev-monitor.o to toplevel directory
>   hw: move device-hotplug.o to toplevel, compile it once
>   virtio-9p: use CONFIG_VIRTFS, not CONFIG_LINUX
>   virtio-9p: remove PCI dependencies from hw/9pfs/
>   vt82c686: vt82c686 is not a PCI host bridge
>   ppc: do not use ../ in include files
>   hw: include hw header files with full paths
>   build: always link device_tree.o into emulators if libfdt available
>   ppc: express FDT dependency of pSeries and e500 boards via 
> default-configs/
>   hw: move boards and other isolated files to hw/ARCH
>   arm: move files referencing CPU to hw/arm/
>   i386: move files referencing CPU to hw/i386/
>   m68k: move files referencing CPU to hw/m68k/
>   ppc: move files referencing CPU to hw/ppc/
>   ppc: move more files to hw/ppc
>   sh: move files referencing CPU to hw/sh4/
> 
>  Makefile.objs |  1 +
>  Makefile.target   |  1 +
>  backends/Makefile.objs|  4 ++
>  {hw => backends}/baum.c   |  4 +-
>  {hw => backends}/msmouse.c|  2 +-
>  configure | 10 +---
>  default-configs/ppc-softmmu.mak   |  1 +
>  default-configs/ppc64-softmmu.mak |  2 +
>  default-configs/ppcemb-softmmu.mak|  1 +
>  hw/device-hotplug.c => device-hotplug.c   | 13 +---
>  hw/9pfs/virtio-9p-device.c| 53 +
>  hw/{9p.h => 9pfs/virtio-9p-device.h}  |  4 +-
>  hw/9pfs/virtio-9p-proxy.c |  1 +
>  hw/9pfs/virtio-9p.c   |  3 +-
>  hw/9pfs/virtio-9p.h   |  1 -
>  hw/Makefile.objs  |  8 +--
>  hw/a15mpcore.c|  2 +-
>  hw/a9mpcore.c |  2 +-
>  hw/a9scu.c|  2 +-
>  hw/ac97.c |  6 +-
>  hw/acpi.c |  6 +-
>  hw/acpi_ich9.c| 10 ++--
>  hw/acpi_ich9.h|  2 +-
>  hw/acpi_piix4.c   | 14 ++---
>  hw/adb.c  |  4 +-
>  hw/adb.h  |  2 +-
>  hw/adlib.c|  8 +--
>  hw/ads7846.c  |  2 +-
>  hw/alpha/Makefile.objs|  4 +-
>  hw/{alpha_dp264.c => alpha/dp264.c}   | 16 ++---
>  hw/{alpha_pci.c => alpha/pci.c}   |  2 +-
>  hw/alpha_sys.h| 10 ++--
>  hw/alpha_typhoon.c|  6 +-
>  hw/apb_pci.c  | 12 ++--
>  hw/apic.c | 12 ++--
>  hw/apic_common.c  |  4 +-
>  hw/apic_internal.h|  2 +-
>  hw/apm.c  |  6 +-
>  hw/apm.h  |  2 +-
>  hw/applesmc.c |  4 +-
>  hw/arm/Makefile.objs  | 43 +++---
>  hw/{ => arm}/armv7m.c |  6 +-
>  hw/{arm_boot.c => arm/boot.c} |  8 +--
>  hw/{ => arm}/collie.c | 14 ++---
>  hw/{ => arm}/exynos4210.c | 12 ++--
>  hw/{ => arm}/exynos4_boards.c |  8 +--
>  hw/{ => arm}/gumstix.c| 10 ++--
>  hw/{ => arm}/highbank.c

Re: [Qemu-devel] [PATCH 0/7] linux-user updates

2013-03-09 Thread Peter Maydell

On 7 March 2013 18:21,   wrote:
> From: Riku Voipio 
> I did a dig through the archive for linux-user patches
> that have slipped through during my inactivity. The ones
> in this patchset appear good and pass smoketest.

These are the linux-user patches I've submitted that are
still waiting for review/commit:

linux-user: Fix layout of usage table to account for option text
  http://patchwork.ozlabs.org/patch/220481/
linux-user: make bogus negative iovec lengths fail EINVAL
  http://patchwork.ozlabs.org/patch/219240/
linux-user: Implement sendfile and sendfile64
  http://patchwork.ozlabs.org/patch/219229/
linux-user: Implement accept4
  http://patchwork.ozlabs.org/patch/219160/
linux-user/syscall.c: Don't warn about unimplemented get_robust_list
  http://patchwork.ozlabs.org/patch/219159/

thanks
-- PMM

Re: [Qemu-devel] [PULL 00/66] ppc patch queue 2013-03-08

2013-03-09 Thread Blue Swirl

On Fri, Mar 8, 2013 at 8:06 PM, Alexander Graf  wrote:
> Hi Blue / Aurelien,
>
> This is my current patch queue for ppc.  Please pull.

Thanks, pulled.

>
> Alex
>
>
> The following changes since commit 0bc472a9d6b80567c212023c5eae413f4dfb53ad:
>   Kuo-Jung Su (1):
> hw/nand.c: correct the sense of the BUSY/READY status bit
>
> are available in the git repository at:
>
>   git://github.com/agraf/qemu.git ppc-for-upstream
>
> Amadeusz Sławiński (2):
>   PPC: Fix dma interrupt
>   PPC: xnu kernel expects FLUSH to be cleared on STOP
>
> Andreas Färber (58):
>   target-ppc: Fix CPU_POWERPC_MPC8547E
>   target-ppc: Fix "G2leGP3" PVR
>   target-ppc: Update error handling in ppc_cpu_realize()
>   target-ppc: Drop nested TARGET_PPC64 guard for POWER7
>   target-ppc: Inline comma into POWERPC_DEF_SVR() macro
>   target-ppc: Extract aliases from definitions list
>   target-ppc: Make -cpu "ppc" an alias to "ppc32"
>   target-ppc: Extract MPC5xx aliases
>   target-ppc: Extract MGT823/MPC8xx as aliases
>   target-ppc: Extract 40x aliases
>   target-ppc: Extract 440 aliases
>   target-ppc: Turn "ppc32" and "ppc64" CPUs into aliases
>   target-ppc: Extract 74x7[A] aliases
>   target-ppc: Extract 74x5 as aliases
>   target-ppc: Extract 74x1 aliases
>   target-ppc: Extract 7450 alias
>   target-ppc: Extract 7448 alias
>   target-ppc: Extract 7410 alias
>   target-ppc: Extract 7400 alias
>   target-ppc: Extract 7x5 aliases
>   target-ppc: Extract 750 aliases
>   target-ppc: Extract 740/750 aliases
>   target-ppc: Extract 603e alias
>   target-ppc: Extract 603r alias
>   target-ppc: Extract 601/601v aliases
>   target-ppc: Extract 604e alias
>   target-ppc: Extract MPC85xx aliases
>   target-ppc: Extract e500v1/e500v2 aliases
>   target-ppc: Extract MPC83xx aliases
>   target-ppc: Extract e300 alias
>   target-ppc: Extract e200 alias
>   target-ppc: Extract MPC82xx alias
>   target-ppc: Extract MPC8247/MPC8248/MPC8270-80 aliases
>   target-ppc: Extract MPC82xx aliases to *_HiP4
>   target-ppc: Extract MPC82xx_HiP{3, 4} aliases
>   target-ppc: Extract MPC52xx alias
>   target-ppc: Extract MPC5200/MPC5200B aliases
>   target-ppc: Extract MPC8240 alias
>   target-ppc: Extract 405GPe alias
>   target-ppc: Extract 970 aliases
>   target-ppc: Extract POWER7 alias
>   target-ppc: Get model name from type name
>   target-ppc: Convert CPU definitions
>   target-ppc: Introduce abstract CPU family types
>   target-ppc: Set instruction flags on CPU family classes
>   target-ppc: Register all types for TARGET_PPCEMB
>   target-ppc: Set remaining fields on CPU family classes
>   target-ppc: Turn descriptive CPU family comments into device 
> descriptions
>   target-ppc: Turn descriptive CPU model comments into device descriptions
>   target-ppc: Update Coding Style for CPU models
>   target-ppc: Split model definitions out of translate_init.c
>   target-ppc: Fix remaining microcontroller typos among models
>   target-ppc: Change "POWER7" CPU alias
>   target-ppc: Fix PPC_DUMP_SPR_ACCESS build
>   target-ppc: Make host CPU a subclass of the host's CPU model
>   target-ppc: List alias names alongside CPU models
>   target-ppc: Report CPU aliases for QMP
>   target-ppc: Move CPU aliases out of translate_init.c
>
> David Gibson (4):
>   pseries: Add cleanup hook for PAPR virtual LAN device
>   target-ppc: Add mechanism for synchronizing SPRs with KVM
>   target-ppc: Synchronize FPU state with KVM
>   pseries: Add compatible property to root of device tree
>
> Erlon Cruz (1):
>   pseries: Implement h_read hcall
>
> Fabien Chouteau (1):
>   Save memory allocation in the elf loader
>
>  hw/elf_ops.h|   19 +-
>  hw/loader.c |   75 +-
>  hw/loader.h |2 +
>  hw/mac_dbdma.c  |4 +
>  hw/ppc/mac_newworld.c   |2 +-
>  hw/spapr.c  |1 +
>  hw/spapr_hcall.c|   31 +
>  hw/spapr_llan.c |8 +
>  target-ppc/Makefile.objs|1 +
>  target-ppc/cpu-models.c | 1419 
>  target-ppc/cpu-models.h |  741 ++
>  target-ppc/cpu-qom.h|   17 +-
>  target-ppc/cpu.h|   26 +-
>  target-ppc/kvm.c|  311 +++-
>  target-ppc/translate_init.c | 5275 
> +--
>  15 files changed, 4117 insertions(+), 3815 deletions(-)
>  create mode 100644 target-ppc/cpu-models.c
>  create mode 100644 target-ppc/cpu-models.h

Re: [Qemu-devel] [PULL 0/5] s390 patch queue 2013-03-08

2013-03-09 Thread Blue Swirl

On Fri, Mar 8, 2013 at 8:17 PM, Alexander Graf  wrote:
> Hi Blue / Aurelien,
>
> This is my current patch queue for s390.  Please pull.

Thanks, pulled.

>
> Alex
>
>
> The following changes since commit 0bc472a9d6b80567c212023c5eae413f4dfb53ad:
>   Kuo-Jung Su (1):
> hw/nand.c: correct the sense of the BUSY/READY status bit
>
> are available in the git repository at:
>
>   git://github.com/agraf/qemu.git s390-for-upstream
>
> Christian Borntraeger (4):
>   s390: simplify kvm cpu init
>   Allow virtio-net features for legacy s390 virtio bus
>   s390/css: Fix subchannel detection
>   s390/virtio-ccw: remove redundant call to blockdev_mark_auto_del
>
> Cornelia Huck (1):
>   s390: virtio-ccw maintainer
>
>  MAINTAINERS|   14 ++
>  hw/s390x/css.c |   11 +++
>  hw/s390x/s390-virtio-bus.c |1 +
>  hw/s390x/virtio-ccw.c  |1 -
>  target-s390x/cpu.h |2 +-
>  target-s390x/ioinst.c  |2 +-
>  target-s390x/kvm.c |9 ++---
>  7 files changed, 26 insertions(+), 14 deletions(-)

Re: [Qemu-devel] [PATCH 0/3] make_device_config.sh fixes

2013-03-09 Thread Blue Swirl

Thanks, applied all.

On Thu, Feb 21, 2013 at 3:26 PM, Andreas Färber  wrote:
> Hello,
>
> A long-standing patch of mine, applied for 1.4, attempted to fix dependency
> issues observed when adding MegaSAS emulation to pci.mak.
>
> As it turns out that patch was wrong. This mini-series reverts it and adds a
> cleanup to avoid that thinko as well as the real dependency fix.
>
> I don't see any conflict with Gerd's pending make_device_config.sh changes.
>
> Regards,
> Andreas
>
> Cc: Peter Maydell 
> Cc: Blue Swirl 
> Cc: Paolo Bonzini 
> Cc: Anthony Liguori 
> Cc: Gerd Hoffmann 
>
> Andreas Färber (3):
>   Revert "make_device_config.sh: Fix target path in generated
> dependency file"
>   make_device_config.sh: Emit dependency file to directory where
> included
>   Makefile: Add subdir dependency on config-devices-all.mak
>
>  Makefile  |5 -
>  scripts/make_device_config.sh |4 ++--
>  2 Dateien geändert, 6 Zeilen hinzugefügt(+), 3 Zeilen entfernt(-)
>
> --
> 1.7.10.4
>

Re: [Qemu-devel] [PATCH V8 01/20] build: add block/snapshot.c

2013-03-09 Thread Eric Blake

On 03/08/2013 09:00 PM, Wenchao Xia wrote:
> 
>> On 03/06/2013 11:07 PM, Wenchao Xia wrote:
>>>This file will have internal snapshot related functions.
>>>
>>> Signed-off-by: Wenchao Xia 
>>> ---
>>
>>> --- /dev/null
>>> +++ b/include/block/snapshot.h
>>> @@ -0,0 +1,4 @@
>>> +#ifndef SNAPSHOT_H
>>
>> Introducing a new file without a copyright notice is a no-no.
>>
> 
>   Copy right is notice in snapshot.c, other header files seems
> no copy right declaration, what more should be added?

The copyright of the .c file should be copied to the .h file then.
EVERY file needs a copyright, not just one of a pair of files.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v9 0/4] Moxie CPU port

2013-03-09 Thread Anthony Green

This version of the patch implements 3 changes...

1. Blue's suggestion of lazy CC calculation, dramatically simplifying
   the branching code.
2. Blue's suggestion of turning brk instruction into an exception.
3. Richard's div helper cpu_restore_state suggestion.

Please consider merging this version of the patch.

Thank you,

AG

Anthony Green (4):
  Add moxie target code
  Add moxie disassembler
  Add sample moxie system
  Add top level changes for moxie

 MAINTAINERS   |   5 +
 arch_init.c   |   2 +
 configure |   9 +-
 cpu-exec.c|   2 +
 default-configs/moxie-softmmu.mak |   2 +
 disas.c   |   6 +
 disas/Makefile.objs   |   1 +
 disas/moxie.c | 360 +++
 hw/moxie/Makefile.objs|   6 +
 hw/moxie/moxiesim.c   | 174 
 include/disas/bfd.h   |   2 +
 include/sysemu/arch_init.h|   1 +
 qapi-schema.json  |   6 +-
 target-moxie/Makefile.objs|   2 +
 target-moxie/cpu.c| 172 +++
 target-moxie/cpu.h| 169 +++
 target-moxie/helper.c | 171 +++
 target-moxie/helper.h |  10 +
 target-moxie/machine.c|  28 ++
 target-moxie/machine.h|   2 +
 target-moxie/mmu.c|  36 ++
 target-moxie/mmu.h|  19 +
 target-moxie/translate.c  | 918 ++
 23 files changed, 2099 insertions(+), 4 deletions(-)
 create mode 100644 default-configs/moxie-softmmu.mak
 create mode 100644 disas/moxie.c
 create mode 100644 hw/moxie/Makefile.objs
 create mode 100644 hw/moxie/moxiesim.c
 create mode 100644 target-moxie/Makefile.objs
 create mode 100644 target-moxie/cpu.c
 create mode 100644 target-moxie/cpu.h
 create mode 100644 target-moxie/helper.c
 create mode 100644 target-moxie/helper.h
 create mode 100644 target-moxie/machine.c
 create mode 100644 target-moxie/machine.h
 create mode 100644 target-moxie/mmu.c
 create mode 100644 target-moxie/mmu.h
 create mode 100644 target-moxie/translate.c

-- 
1.8.1.4

[Qemu-devel] [PATCH v9 2/4] Add moxie disassembler

2013-03-09 Thread Anthony Green


Signed-off-by: Anthony Green 
---
 disas.c |   6 +
 disas/Makefile.objs |   1 +
 disas/moxie.c   | 360 
 include/disas/bfd.h |   2 +
 4 files changed, 369 insertions(+)
 create mode 100644 disas/moxie.c

diff --git a/disas.c b/disas.c
index a46faee..74d3ba0 100644
--- a/disas.c
+++ b/disas.c
@@ -256,6 +256,9 @@ void target_disas(FILE *out, CPUArchState *env, 
target_ulong code,
 #elif defined(TARGET_MICROBLAZE)
 s.info.mach = bfd_arch_microblaze;
 print_insn = print_insn_microblaze;
+#elif defined(TARGET_MOXIE)
+s.info.mach = bfd_arch_moxie;
+print_insn = print_insn_moxie;
 #elif defined(TARGET_LM32)
 s.info.mach = bfd_mach_lm32;
 print_insn = print_insn_lm32;
@@ -462,6 +465,9 @@ void monitor_disas(Monitor *mon, CPUArchState *env,
 #elif defined(TARGET_S390X)
 s.info.mach = bfd_mach_s390_64;
 print_insn = print_insn_s390;
+#elif defined(TARGET_MOXIE)
+s.info.mach = bfd_arch_moxie;
+print_insn = print_insn_moxie;
 #elif defined(TARGET_LM32)
 s.info.mach = bfd_mach_lm32;
 print_insn = print_insn_lm32;
diff --git a/disas/Makefile.objs b/disas/Makefile.objs
index ed75f9a..3b1e77a 100644
--- a/disas/Makefile.objs
+++ b/disas/Makefile.objs
@@ -7,6 +7,7 @@ common-obj-$(CONFIG_IA64_DIS) += ia64.o
 common-obj-$(CONFIG_M68K_DIS) += m68k.o
 common-obj-$(CONFIG_MICROBLAZE_DIS) += microblaze.o
 common-obj-$(CONFIG_MIPS_DIS) += mips.o
+common-obj-$(CONFIG_MOXIE_DIS) += moxie.o
 common-obj-$(CONFIG_PPC_DIS) += ppc.o
 common-obj-$(CONFIG_S390_DIS) += s390.o
 common-obj-$(CONFIG_SH4_DIS) += sh4.o
diff --git a/disas/moxie.c b/disas/moxie.c
new file mode 100644
index 000..4c5f180
--- /dev/null
+++ b/disas/moxie.c
@@ -0,0 +1,360 @@
+/* Disassemble moxie instructions.
+   Copyright (c) 2009  Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, see . */
+
+#include 
+#define STATIC_TABLE
+#define DEFINE_TABLE
+
+#include "disas/bfd.h"
+
+static void *stream;
+
+/* Form 1 instructions come in different flavors:
+
+   Some have no arguments  (MOXIE_F1_NARG)
+   Some only use the A operand (MOXIE_F1_A)
+   Some use A and B registers  (MOXIE_F1_AB)
+   Some use A and consume a 4 byte immediate value (MOXIE_F1_A4)
+   Some use just a 4 byte immediate value  (MOXIE_F1_4)
+   Some use just a 4 byte memory address   (MOXIE_F1_M)
+   Some use B and an indirect A(MOXIE_F1_AiB)
+   Some use A and an indirect B(MOXIE_F1_ABi)
+   Some consume a 4 byte immediate value and use X (MOXIE_F1_4A)
+   Some use B and an indirect A plus 4 bytes   (MOXIE_F1_AiB4)
+   Some use A and an indirect B plus 4 bytes   (MOXIE_F1_ABi4)
+
+   Form 2 instructions also come in different flavors:
+
+   Some have no arguments  (MOXIE_F2_NARG)
+   Some use the A register and an 8-bit value  (MOXIE_F2_A8V)
+
+   Form 3 instructions also come in different flavors:
+
+   Some have no arguments  (MOXIE_F3_NARG)
+   Some have a 10-bit PC relative operand  (MOXIE_F3_PCREL).  */
+
+#define MOXIE_F1_NARG 0x100
+#define MOXIE_F1_A0x101
+#define MOXIE_F1_AB   0x102
+/* #define MOXIE_F1_ABC  0x103 */
+#define MOXIE_F1_A4   0x104
+#define MOXIE_F1_40x105
+#define MOXIE_F1_AiB  0x106
+#define MOXIE_F1_ABi  0x107
+#define MOXIE_F1_4A   0x108
+#define MOXIE_F1_AiB4 0x109
+#define MOXIE_F1_ABi4 0x10a
+#define MOXIE_F1_M0x10b
+
+#define MOXIE_F2_NARG 0x200
+#define MOXIE_F2_A8V  0x201
+
+#define MOXIE_F3_NARG  0x300
+#define MOXIE_F3_PCREL 0x301
+
+typedef struct moxie_opc_info_t {
+short opcode;
+unsigned  itype;
+const char *  name;
+} moxie_opc_info_t;
+
+extern const moxie_opc_info_t moxie_form1_opc_info[64];
+extern const moxie_opc_info_t moxie_form2_opc_info[4];
+extern const moxie_opc_info_t moxie_form3_opc_info[16];
+
+/* The moxie processor's 16-bit instructions come in two forms:
+
+   FORM 1 instructions start with a 0 bit...
+
+   0ooo
+   0  F
+
+   ooo - form 1 opcode number
+   - operand A
+   - operand B
+
+   FORM 2 instructions start with bits "10"...
+
+   10oo
+   0  F
+
+   oo   - form 2 opcode number
+    - operand A
+

[Qemu-devel] [PATCH v9 1/4] Add moxie target code

2013-03-09 Thread Anthony Green


Signed-off-by: Anthony Green 
---
 target-moxie/Makefile.objs |   2 +
 target-moxie/cpu.c | 172 +
 target-moxie/cpu.h | 169 +
 target-moxie/helper.c  | 171 +
 target-moxie/helper.h  |  10 +
 target-moxie/machine.c |  28 ++
 target-moxie/machine.h |   2 +
 target-moxie/mmu.c |  36 ++
 target-moxie/mmu.h |  19 +
 target-moxie/translate.c   | 918 +
 10 files changed, 1527 insertions(+)
 create mode 100644 target-moxie/Makefile.objs
 create mode 100644 target-moxie/cpu.c
 create mode 100644 target-moxie/cpu.h
 create mode 100644 target-moxie/helper.c
 create mode 100644 target-moxie/helper.h
 create mode 100644 target-moxie/machine.c
 create mode 100644 target-moxie/machine.h
 create mode 100644 target-moxie/mmu.c
 create mode 100644 target-moxie/mmu.h
 create mode 100644 target-moxie/translate.c

diff --git a/target-moxie/Makefile.objs b/target-moxie/Makefile.objs
new file mode 100644
index 000..6381d4d
--- /dev/null
+++ b/target-moxie/Makefile.objs
@@ -0,0 +1,2 @@
+obj-y += translate.o helper.o machine.o cpu.o machine.o
+obj-$(CONFIG_SOFTMMU) += mmu.o
diff --git a/target-moxie/cpu.c b/target-moxie/cpu.c
new file mode 100644
index 000..c17d3f0
--- /dev/null
+++ b/target-moxie/cpu.c
@@ -0,0 +1,172 @@
+/*
+ * QEMU Moxie CPU
+ *
+ * Copyright (c) 2013 Anthony Green
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#include "cpu.h"
+#include "qemu-common.h"
+#include "migration/vmstate.h"
+#include "machine.h"
+
+static void moxie_cpu_reset(CPUState *s)
+{
+MoxieCPU *cpu = MOXIE_CPU(s);
+MoxieCPUClass *mcc = MOXIE_CPU_GET_CLASS(cpu);
+CPUMoxieState *env = &cpu->env;
+
+if (qemu_loglevel_mask(CPU_LOG_RESET)) {
+qemu_log("CPU Reset (CPU %d)\n", s->cpu_index);
+log_cpu_state(env, 0);
+}
+
+mcc->parent_reset(s);
+
+memset(env, 0, offsetof(CPUMoxieState, breakpoints));
+env->pc = 0x1000;
+
+tlb_flush(env, 1);
+}
+
+static void moxie_cpu_realizefn(DeviceState *dev, Error **errp)
+{
+MoxieCPU *cpu = MOXIE_CPU(dev);
+MoxieCPUClass *occ = MOXIE_CPU_GET_CLASS(dev);
+
+qemu_init_vcpu(&cpu->env);
+cpu_reset(CPU(cpu));
+
+occ->parent_realize(dev, errp);
+}
+
+static void moxie_cpu_initfn(Object *obj)
+{
+CPUState *cs = CPU(obj);
+MoxieCPU *cpu = MOXIE_CPU(obj);
+static int inited;
+
+cs->env_ptr = &cpu->env;
+cpu_exec_init(&cpu->env);
+
+if (tcg_enabled() && !inited) {
+inited = 1;
+moxie_translate_init();
+}
+}
+
+static ObjectClass *moxie_cpu_class_by_name(const char *cpu_model)
+{
+ObjectClass *oc;
+
+if (cpu_model == NULL) {
+return NULL;
+}
+
+oc = object_class_by_name(cpu_model);
+if (oc != NULL && (!object_class_dynamic_cast(oc, TYPE_MOXIE_CPU) ||
+   object_class_is_abstract(oc))) {
+return NULL;
+}
+return oc;
+}
+
+static void moxie_cpu_class_init(ObjectClass *oc, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(oc);
+CPUClass *cc = CPU_CLASS(oc);
+MoxieCPUClass *mcc = MOXIE_CPU_CLASS(oc);
+
+mcc->parent_realize = dc->realize;
+dc->realize = moxie_cpu_realizefn;
+
+mcc->parent_reset = cc->reset;
+cc->reset = moxie_cpu_reset;
+
+cc->class_by_name = moxie_cpu_class_by_name;
+
+dc->vmsd = &vmstate_moxie_cpu;
+}
+
+static void moxielite_initfn(Object *obj)
+{
+/* Set cpu feature flags */
+}
+
+static void moxie_any_initfn(Object *obj)
+{
+/* Set cpu feature flags */
+}
+
+typedef struct MoxieCPUInfo {
+const char *name;
+void (*initfn)(Object *obj);
+} MoxieCPUInfo;
+
+static const MoxieCPUInfo moxie_cpus[] = {
+{ .name = "MoxieLite",  .initfn = moxielite_initfn },
+{ .name = "any",.initfn = moxie_any_initfn },
+};
+
+MoxieCPU *cpu_moxie_init(const char *cpu_model)
+{
+MoxieCPU *cpu;
+ObjectClass *oc;
+
+oc = moxie_cpu_class_by_name(cpu_model);
+if (oc == NULL) {
+return NULL;
+}
+cpu = MOXIE_CPU(object_new(object_class_get_name(oc)));
+cpu->env.cpu_model_str = cpu_model;
+
+object_property_set_bool(OBJECT(cpu), true, "realized", NULL);
+
+return cpu;
+}
+
+static void cpu_register(const MoxieCPUInfo *info)
+{
+TypeInfo type_info = {
+.parent = TYPE_MOXIE_CPU,
+

[Qemu-devel] [PATCH v9 3/4] Add sample moxie system

2013-03-09 Thread Anthony Green


Signed-off-by: Anthony Green 
---
 hw/moxie/Makefile.objs |   6 ++
 hw/moxie/moxiesim.c| 174 +
 include/sysemu/arch_init.h |   1 +
 3 files changed, 181 insertions(+)
 create mode 100644 hw/moxie/Makefile.objs
 create mode 100644 hw/moxie/moxiesim.c

diff --git a/hw/moxie/Makefile.objs b/hw/moxie/Makefile.objs
new file mode 100644
index 000..d0772d1
--- /dev/null
+++ b/hw/moxie/Makefile.objs
@@ -0,0 +1,6 @@
+# moxie boards
+obj-y = serial.o mc146818rtc.o vga.o
+obj-$(CONFIG_FDT) += device_tree.o
+
+obj-y := $(addprefix ../,$(obj-y))
+obj-y += moxiesim.o
diff --git a/hw/moxie/moxiesim.c b/hw/moxie/moxiesim.c
new file mode 100644
index 000..e1e88a9
--- /dev/null
+++ b/hw/moxie/moxiesim.c
@@ -0,0 +1,174 @@
+/*
+ * QEMU/moxiesim emulation
+ *
+ * Emulates a very simple machine model similiar to the one use by the
+ * GDB moxie simulator.
+ *
+ * Copyright (c) 2008, 2009, 2010, 2013 Anthony Green
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#include "hw/sysbus.h"
+#include "hw/hw.h"
+#include "hw/pc.h"
+#include "hw/isa.h"
+#include "net/net.h"
+#include "sysemu/sysemu.h"
+#include "hw/boards.h"
+#include "hw/loader.h"
+#include "hw/serial.h"
+#include "exec/address-spaces.h"
+
+#define PHYS_MEM_BASE 0x8000
+
+typedef struct {
+uint64_t ram_size;
+const char *kernel_filename;
+const char *kernel_cmdline;
+const char *initrd_filename;
+} LoaderParams;
+
+static void load_kernel(MoxieCPU *cpu, LoaderParams *loader_params)
+{
+uint64_t entry, kernel_low, kernel_high;
+long kernel_size;
+long initrd_size;
+ram_addr_t initrd_offset;
+
+kernel_size = load_elf(loader_params->kernel_filename,  NULL, NULL,
+   &entry, &kernel_low, &kernel_high, 1,
+   ELF_MACHINE, 0);
+
+if (!kernel_size) {
+fprintf(stderr, "qemu: could not load kernel '%s'\n",
+loader_params->kernel_filename);
+exit(1);
+}
+
+/* load initrd */
+initrd_size = 0;
+initrd_offset = 0;
+if (loader_params->initrd_filename) {
+initrd_size = get_image_size(loader_params->initrd_filename);
+if (initrd_size > 0) {
+initrd_offset = (kernel_high + ~TARGET_PAGE_MASK)
+  & TARGET_PAGE_MASK;
+if (initrd_offset + initrd_size > loader_params->ram_size) {
+fprintf(stderr,
+"qemu: memory too small for initial ram disk '%s'\n",
+loader_params->initrd_filename);
+exit(1);
+}
+initrd_size = load_image_targphys(loader_params->initrd_filename,
+  initrd_offset,
+  ram_size);
+}
+if (initrd_size == (target_ulong)-1) {
+fprintf(stderr, "qemu: could not load initial ram disk '%s'\n",
+loader_params->initrd_filename);
+exit(1);
+}
+}
+}
+
+static void main_cpu_reset(void *opaque)
+{
+MoxieCPU *cpu = opaque;
+
+cpu_reset(CPU(cpu));
+}
+
+static inline DeviceState *
+moxie_intc_create(hwaddr base, qemu_irq irq, int kind_of_intr)
+{
+DeviceState *dev;
+
+dev = qdev_create(NULL, "moxie,intc");
+qdev_prop_set_uint32(dev, "kind-of-intr", kind_of_intr);
+qdev_init_nofail(dev);
+sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, base);
+sysbus_connect_irq(SYS_BUS_DEVICE(dev), 0, irq);
+return dev;
+}
+
+static void moxiesim_init(QEMUMachineInitArgs *args)
+{
+MoxieCPU *cpu = NULL;
+ram_addr_t ram_size = args->ram_size;
+const char *cpu_model = args->cpu_model;
+const char *kernel_filename = args->kernel_filename;
+const char *kernel_cmdline = args->kernel_cmdline;
+const char *initrd_filename = args->initrd_filename;
+CPUMoxieState *env;
+MemoryRegion *address_space_mem = get_syste

[Qemu-devel] [PATCH v9 4/4] Add top level changes for moxie

2013-03-09 Thread Anthony Green


Signed-off-by: Anthony Green 
---
 MAINTAINERS   | 5 +
 arch_init.c   | 2 ++
 configure | 9 -
 cpu-exec.c| 2 ++
 default-configs/moxie-softmmu.mak | 2 ++
 qapi-schema.json  | 6 +++---
 6 files changed, 22 insertions(+), 4 deletions(-)
 create mode 100644 default-configs/moxie-softmmu.mak

diff --git a/MAINTAINERS b/MAINTAINERS
index 2439614..c8036c0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -91,6 +91,11 @@ M: Aurelien Jarno 
 S: Odd Fixes
 F: target-mips/
 
+Moxie
+M: Anthony Green 
+S: Maintained
+F: target-moxie/
+
 PowerPC
 M: Alexander Graf 
 L: qemu-...@nongnu.org
diff --git a/arch_init.c b/arch_init.c
index 8daeafa..ddae1b7 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -85,6 +85,8 @@ int graphic_depth = 15;
 #define QEMU_ARCH QEMU_ARCH_MICROBLAZE
 #elif defined(TARGET_MIPS)
 #define QEMU_ARCH QEMU_ARCH_MIPS
+#elif defined(TARGET_MOXIE)
+#define QEMU_ARCH QEMU_ARCH_MOXIE
 #elif defined(TARGET_OPENRISC)
 #define QEMU_ARCH QEMU_ARCH_OPENRISC
 #elif defined(TARGET_PPC)
diff --git a/configure b/configure
index 2f98c5a..da7e407 100755
--- a/configure
+++ b/configure
@@ -958,6 +958,7 @@ mips-softmmu \
 mipsel-softmmu \
 mips64-softmmu \
 mips64el-softmmu \
+moxie-softmmu \
 or32-softmmu \
 ppc-softmmu \
 ppcemb-softmmu \
@@ -3917,7 +3918,7 @@ target_arch2=`echo $target | cut -d '-' -f 1`
 target_bigendian="no"
 
 case "$target_arch2" in
-  
armeb|lm32|m68k|microblaze|mips|mipsn32|mips64|or32|ppc|ppcemb|ppc64|ppc64abi32|s390x|sh4eb|sparc|sparc64|sparc32plus|xtensaeb)
+  
armeb|lm32|m68k|microblaze|mips|mipsn32|mips64|moxie|or32|ppc|ppcemb|ppc64|ppc64abi32|s390x|sh4eb|sparc|sparc64|sparc32plus|xtensaeb)
   target_bigendian=yes
   ;;
 esac
@@ -4023,6 +4024,8 @@ case "$target_arch2" in
 echo "TARGET_ABI_MIPSN64=y" >> $config_target_mak
 target_long_alignment=8
   ;;
+  moxie)
+  ;;
   or32)
 TARGET_ARCH=openrisc
 TARGET_BASE_ARCH=openrisc
@@ -4267,6 +4270,10 @@ for i in $ARCH $TARGET_BASE_ARCH ; do
 echo "CONFIG_MIPS_DIS=y"  >> $config_target_mak
 echo "CONFIG_MIPS_DIS=y"  >> config-all-disas.mak
   ;;
+  moxie*)
+echo "CONFIG_MOXIE_DIS=y"  >> $config_target_mak
+echo "CONFIG_MOXIE_DIS=y"  >> config-all-disas.mak
+  ;;
   or32)
 echo "CONFIG_OPENRISC_DIS=y"  >> $config_target_mak
 echo "CONFIG_OPENRISC_DIS=y"  >> config-all-disas.mak
diff --git a/cpu-exec.c b/cpu-exec.c
index 9092145..12060f4 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -236,6 +236,7 @@ int cpu_exec(CPUArchState *env)
 #elif defined(TARGET_LM32)
 #elif defined(TARGET_MICROBLAZE)
 #elif defined(TARGET_MIPS)
+#elif defined(TARGET_MOXIE)
 #elif defined(TARGET_OPENRISC)
 #elif defined(TARGET_SH4)
 #elif defined(TARGET_CRIS)
@@ -686,6 +687,7 @@ int cpu_exec(CPUArchState *env)
   | env->cc_dest | (env->cc_x << 4);
 #elif defined(TARGET_MICROBLAZE)
 #elif defined(TARGET_MIPS)
+#elif defined(TARGET_MOXIE)
 #elif defined(TARGET_OPENRISC)
 #elif defined(TARGET_SH4)
 #elif defined(TARGET_ALPHA)
diff --git a/default-configs/moxie-softmmu.mak 
b/default-configs/moxie-softmmu.mak
new file mode 100644
index 000..d378363
--- /dev/null
+++ b/default-configs/moxie-softmmu.mak
@@ -0,0 +1,2 @@
+# Default configuration for moxie-softmmu
+
diff --git a/qapi-schema.json b/qapi-schema.json
index 28b070f..233ea1b 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2994,9 +2994,9 @@
 ##
 { 'enum': 'TargetType',
   'data': [ 'alpha', 'arm', 'cris', 'i386', 'lm32', 'm68k', 'microblazeel',
-'microblaze', 'mips64el', 'mips64', 'mipsel', 'mips', 'or32',
-'ppc64', 'ppcemb', 'ppc', 's390x', 'sh4eb', 'sh4', 'sparc64',
-'sparc', 'unicore32', 'x86_64', 'xtensaeb', 'xtensa' ] }
+'microblaze', 'mips64el', 'mips64', 'mipsel', 'mips', 'moxie',
+'or32', 'ppc64', 'ppcemb', 'ppc', 's390x', 'sh4eb', 'sh4',
+'sparc64', 'sparc', 'unicore32', 'x86_64', 'xtensaeb', 'xtensa' ] }
 
 ##
 # @TargetInfo:
-- 
1.8.1.4

Re: [Qemu-devel] [PATCH v4 3/6] add backup related monitor commands

2013-03-09 Thread Dietmar Maurer

> >> For these reasons, I'm against putting backup archive code into QEMU.
> >
> > That is OK for me - I already maintain the code outside of qemu.
> 
> Does this mean you will keep this patch series out-of-tree?

You are 'against putting backup archive code into QEMU', so I need to
maintain it out-of-tree.

Re: [Qemu-devel] [PATCH] tcg: Don't make exitreq flag a local temporary

2013-03-09 Thread Aurelien Jarno

On Tue, Mar 05, 2013 at 08:54:41AM -0800, Richard Henderson wrote:
> The value is not actually live across basic blocks, so there's no
> need for the local property.  This eliminates storing the temporary
> to its home location at the branch.
> 
> Cc: Peter Maydell 
> Signed-off-by: Richard Henderson 
> ---
>  include/exec/gen-icount.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/exec/gen-icount.h b/include/exec/gen-icount.h
> index 4e3b17b..4fc7b29 100644
> --- a/include/exec/gen-icount.h
> +++ b/include/exec/gen-icount.h
> @@ -15,7 +15,7 @@ static inline void gen_tb_start(void)
>  TCGv_i32 flag;
>  
>  exitreq_label = gen_new_label();
> -flag = tcg_temp_local_new_i32();
> +flag = tcg_temp_new_i32();
>  tcg_gen_ld_i32(flag, cpu_env,
> offsetof(CPUState, tcg_exit_req) - ENV_OFFSET);
>  tcg_gen_brcondi_i32(TCG_COND_NE, flag, 0, exitreq_label);

Thanks, applied.

-- 
Aurelien Jarno  GPG: 1024D/F1BCDB73
aurel...@aurel32.net http://www.aurel32.net

Re: [Qemu-devel] [PATCH v9 1/4] Add moxie target code

2013-03-09 Thread Blue Swirl

On Sat, Mar 9, 2013 at 1:34 PM, Anthony Green  wrote:
>
> Signed-off-by: Anthony Green 
> ---
>  target-moxie/Makefile.objs |   2 +
>  target-moxie/cpu.c | 172 +
>  target-moxie/cpu.h | 169 +
>  target-moxie/helper.c  | 171 +
>  target-moxie/helper.h  |  10 +
>  target-moxie/machine.c |  28 ++
>  target-moxie/machine.h |   2 +
>  target-moxie/mmu.c |  36 ++
>  target-moxie/mmu.h |  19 +
>  target-moxie/translate.c   | 918 
> +
>  10 files changed, 1527 insertions(+)
>  create mode 100644 target-moxie/Makefile.objs
>  create mode 100644 target-moxie/cpu.c
>  create mode 100644 target-moxie/cpu.h
>  create mode 100644 target-moxie/helper.c
>  create mode 100644 target-moxie/helper.h
>  create mode 100644 target-moxie/machine.c
>  create mode 100644 target-moxie/machine.h
>  create mode 100644 target-moxie/mmu.c
>  create mode 100644 target-moxie/mmu.h
>  create mode 100644 target-moxie/translate.c
>
> diff --git a/target-moxie/Makefile.objs b/target-moxie/Makefile.objs
> new file mode 100644
> index 000..6381d4d
> --- /dev/null
> +++ b/target-moxie/Makefile.objs
> @@ -0,0 +1,2 @@
> +obj-y += translate.o helper.o machine.o cpu.o machine.o
> +obj-$(CONFIG_SOFTMMU) += mmu.o
> diff --git a/target-moxie/cpu.c b/target-moxie/cpu.c
> new file mode 100644
> index 000..c17d3f0
> --- /dev/null
> +++ b/target-moxie/cpu.c
> @@ -0,0 +1,172 @@
> +/*
> + * QEMU Moxie CPU
> + *
> + * Copyright (c) 2013 Anthony Green
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see .
> + */
> +
> +#include "cpu.h"
> +#include "qemu-common.h"
> +#include "migration/vmstate.h"
> +#include "machine.h"
> +
> +static void moxie_cpu_reset(CPUState *s)
> +{
> +MoxieCPU *cpu = MOXIE_CPU(s);
> +MoxieCPUClass *mcc = MOXIE_CPU_GET_CLASS(cpu);
> +CPUMoxieState *env = &cpu->env;
> +
> +if (qemu_loglevel_mask(CPU_LOG_RESET)) {
> +qemu_log("CPU Reset (CPU %d)\n", s->cpu_index);
> +log_cpu_state(env, 0);
> +}
> +
> +mcc->parent_reset(s);
> +
> +memset(env, 0, offsetof(CPUMoxieState, breakpoints));
> +env->pc = 0x1000;
> +
> +tlb_flush(env, 1);
> +}
> +
> +static void moxie_cpu_realizefn(DeviceState *dev, Error **errp)
> +{
> +MoxieCPU *cpu = MOXIE_CPU(dev);
> +MoxieCPUClass *occ = MOXIE_CPU_GET_CLASS(dev);
> +
> +qemu_init_vcpu(&cpu->env);
> +cpu_reset(CPU(cpu));
> +
> +occ->parent_realize(dev, errp);
> +}
> +
> +static void moxie_cpu_initfn(Object *obj)
> +{
> +CPUState *cs = CPU(obj);
> +MoxieCPU *cpu = MOXIE_CPU(obj);
> +static int inited;
> +
> +cs->env_ptr = &cpu->env;
> +cpu_exec_init(&cpu->env);
> +
> +if (tcg_enabled() && !inited) {
> +inited = 1;
> +moxie_translate_init();
> +}
> +}
> +
> +static ObjectClass *moxie_cpu_class_by_name(const char *cpu_model)
> +{
> +ObjectClass *oc;
> +
> +if (cpu_model == NULL) {
> +return NULL;
> +}
> +
> +oc = object_class_by_name(cpu_model);
> +if (oc != NULL && (!object_class_dynamic_cast(oc, TYPE_MOXIE_CPU) ||
> +   object_class_is_abstract(oc))) {
> +return NULL;
> +}
> +return oc;
> +}
> +
> +static void moxie_cpu_class_init(ObjectClass *oc, void *data)
> +{
> +DeviceClass *dc = DEVICE_CLASS(oc);
> +CPUClass *cc = CPU_CLASS(oc);
> +MoxieCPUClass *mcc = MOXIE_CPU_CLASS(oc);
> +
> +mcc->parent_realize = dc->realize;
> +dc->realize = moxie_cpu_realizefn;
> +
> +mcc->parent_reset = cc->reset;
> +cc->reset = moxie_cpu_reset;
> +
> +cc->class_by_name = moxie_cpu_class_by_name;
> +
> +dc->vmsd = &vmstate_moxie_cpu;
> +}
> +
> +static void moxielite_initfn(Object *obj)
> +{
> +/* Set cpu feature flags */
> +}
> +
> +static void moxie_any_initfn(Object *obj)
> +{
> +/* Set cpu feature flags */
> +}
> +
> +typedef struct MoxieCPUInfo {
> +const char *name;
> +void (*initfn)(Object *obj);
> +} MoxieCPUInfo;
> +
> +static const MoxieCPUInfo moxie_cpus[] = {
> +{ .name = "MoxieLite",  .initfn = moxielite_initfn },
> +{ .name = "any",.initfn = moxie_any_initfn },
> +};
> +
> +MoxieCPU *cpu_moxie_init(const char *cpu_model)
> +{
> +MoxieCPU *cpu;
> +ObjectClass *oc;
> +
> +oc = moxie_cpu_class_by_nam

[Qemu-devel] [RFC 1/8] block: add virtual_size to query-block QMP output

2013-03-09 Thread Stefan Hajnoczi

There is currently no way to query the size of a drive.  Add a
'virtual_size' field to the 'query-block' QMP output.

Signed-off-by: Stefan Hajnoczi 
---
 block.c  | 1 +
 qapi-schema.json | 5 -
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/block.c b/block.c
index 124a9eb..0128e27 100644
--- a/block.c
+++ b/block.c
@@ -2908,6 +2908,7 @@ BlockInfo *bdrv_query_info(BlockDriverState *bs)
 info->has_inserted = true;
 info->inserted = g_malloc0(sizeof(*info->inserted));
 info->inserted->file = g_strdup(bs->filename);
+info->inserted->virtual_size = bdrv_getlength(bs);
 info->inserted->ro = bs->read_only;
 info->inserted->drv = g_strdup(bs->drv->format_name);
 info->inserted->encrypted = bs->encrypted;
diff --git a/qapi-schema.json b/qapi-schema.json
index 28b070f..6b64aec 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -751,6 +751,8 @@
 #
 # @iops_wr: write I/O operations per second is specified
 #
+# @virtual_size: size of block device, in bytes
+#
 # Since: 0.14.0
 #
 # Notes: This interface is only found in @BlockInfo.
@@ -760,7 +762,8 @@
 '*backing_file': 'str', 'backing_file_depth': 'int',
 'encrypted': 'bool', 'encryption_key_missing': 'bool',
 'bps': 'int', 'bps_rd': 'int', 'bps_wr': 'int',
-'iops': 'int', 'iops_rd': 'int', 'iops_wr': 'int'} }
+'iops': 'int', 'iops_rd': 'int', 'iops_wr': 'int',
+'virtual_size': 'int' } }
 
 ##
 # @BlockDeviceIoStatus:
-- 
1.8.1.4

[Qemu-devel] [RFC 0/8] block: Live backup prototype

2013-03-09 Thread Stefan Hajnoczi

Dietmar Maurer's recent "Efficient VM backup for qemu" patch series has spawned
a lot of discussion.  I'm afraid we are close to an impasse so I decided to
prototype the approach that I'm advocating - just make sure it actually works
:).

This series implements vmstate and disk backup for running guests.  It's just a
quick hack, especially the Python code is missing error handling.  My goal is
to show by example how the 'block-backup' block job should work and how a VMA
backup archive writer can be implemented as an external program.

The code is available in my public repo:

  git://github.com/stefanha/qemu.git backup-block-job

The key feature is the new 'block-backup' QMP command.  It takes a
point-in-time copy of a block device and writes it to a target device.  This is
related to block-stream and drive-mirror but the snapshot is copied out while
the guest continues writing to the block device.

Later patches are Python scripts that use the new 'block-backup' QMP command to
implement live backup.  The scripts implement Dietmar's VMA backup archive
format in order to prove that VMA can be done with 'block-backup'.

  $ python backup.py /tmp/monitor.sock backup-20130301.vma
  Running migration...
  Running block-backup...
  Finished device "virtio-drive0"
  Backup complete, terminating writer process

Behind the scenes backup.py spawns a script called vma-writer.py.  The
vma-writer.py process receives migration vmstate and 'block-backup' data.  The
'block-backup' block jobs send data to vma-writer.py using the NBD protocol.

The difference between this approach and Dietmar's series is that the backup
archive format is implemented outside QEMU and runs as a separate program.

This way, management tools like proxmox, oVirt, OpenStack, and others can
provide their preferred backup archive formats without modifying QEMU.  This
has many advantages:

 * 'block-backup' composes with 'migration' and other commands, unlike the
   monolithic 'backup' command designed just for writing backup archives

 * Backup code can be updated or added outside the QEMU release cycle

 * Choice of language, coding style, and license for backup code

 * Less QEMU code to test and maintain

The objection to this approach has been performance.  Exporting vmstate and
disk data over UNIX domain sockets to an external process incurs IPC overhead.
This prototype shows that even Python code hacked up in a day achieves decent
performance.

I'm leaving benchmarking as an exercise for the reader.  I tested a single
scenario with a 16 GB raw disk and 1 GB RAM, backup time was 28% longer (+30
seconds) than Dietmar's series.  Below are a few starting points:

 * I moved the buffer_is_zero() check from the VMA writer into the block job.
   We now skip writing zero clusters and the file contains no extents for them.

 * Migration data uses no fixed block size, there are many small writes that
   could be optimized by batching into a buffer.

 * The block job issues 64 KB writes, it should be possible to use a larger
   buffer size like the 512 KB buffer in block/stream.c.

 * Another trick in block/mirror.c is to use asynchronous I/O so that there can
   be multiple requests pending.

Dietmar Maurer (1):
  add basic backup support to block driver

Stefan Hajnoczi (7):
  block: add virtual_size to query-block QMP output
  backup: write to BlockDriverState instead of BackupDumpFunc
  block: add block_backup QMP command
  Add nbd server Python module
  Add VMA backup archive writer Python module
  Add vma-writer.py tool
  Add backup.py tool

 Makefile.objs |   1 +
 backup.c  | 321 ++
 backup.py |  84 
 block.c   |  72 ++-
 blockdev.c|  92 +
 include/block/block.h |   2 +
 include/block/block_int.h |  16 +++
 include/block/blockjob.h  |  10 ++
 nbd.py| 124 ++
 qapi-schema.json  |  33 -
 qmp-commands.hx   |   6 +
 vma-writer.py | 126 ++
 vma.py| 236 ++
 13 files changed, 1116 insertions(+), 7 deletions(-)
 create mode 100644 backup.c
 create mode 100644 backup.py
 create mode 100644 nbd.py
 create mode 100644 vma-writer.py
 create mode 100644 vma.py

-- 
1.8.1.4

[Qemu-devel] [RFC 7/8] Add vma-writer.py tool

2013-03-09 Thread Stefan Hajnoczi

The VMA writer bundles vmstate and disk backups into a backup archive
file.  It is normally invoked by a management tool like this:

  vma-writer.py --incoming /tmp/migrate.sock \
--nbd /tmp/nbd.sock \
--drive name=drive0,size=10737418240 \
--drive name=drive1,size=34359738368 \
--output backup-20130301.vma

The basic flow is:

1. Set up UNIX domain listen sockets.
2. Print 'Ready' so parent process knows sockets are listening.
3. Stream migration data into VMA.
4. Stream NBD disk data into VMA.

The VMA writer runs in its own thread.  Other threads can send commands
via a thread-safe Queue.  This way multiple NBD export threads can queue
data in parallel, the data will be serialized and written out by the VMA
writer thread.

Signed-off-by: Stefan Hajnoczi 
---
 vma-writer.py | 126 ++
 1 file changed, 126 insertions(+)
 create mode 100644 vma-writer.py

diff --git a/vma-writer.py b/vma-writer.py
new file mode 100644
index 000..dc7f828
--- /dev/null
+++ b/vma-writer.py
@@ -0,0 +1,126 @@
+#!/usr/bin/env python
+#
+# VMA backup archive writer
+#
+# Copyright 2013 Red Hat, Inc. and/or its affiliates
+#
+# Authors:
+#   Stefan Hajnoczi 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+import argparse
+import threading
+import Queue
+import socket
+import sys
+import os
+import nbd
+import vma
+
+WRITER_OP_WRITE = 0
+WRITER_OP_STOP = 1
+
+def setup_listen_sockets(paths):
+'''Return list of listening UNIX domain sockets'''
+socks = []
+for path in paths:
+s = socket.socket(socket.AF_UNIX)
+try:
+os.unlink(path)
+except OSError:
+pass
+s.bind(path)
+s.listen(0)
+socks.append(s)
+return socks
+
+def vma_writer_thread(writer, queue):
+while True:
+cmd = queue.get()
+if cmd[0] == WRITER_OP_STOP:
+break
+assert cmd[0] == WRITER_OP_WRITE
+_, stream_id, offset, data = cmd
+writer.write(stream_id, offset, data)
+writer.close()
+
+def setup_vma_writer(filename, drives):
+vma_file = open(filename, 'wb')
+writer = vma.Writer(vma_file)
+
+vmstate_id = writer.add_stream('vmstate', 1)
+for drive in drives:
+drive['stream_id'] = writer.add_stream(drive['name'], 
int(drive['size']))
+
+queue = Queue.Queue()
+t = threading.Thread(target=vma_writer_thread, args=(writer, queue))
+t.start()
+return queue, vmstate_id
+
+def consume_migration(sock, queue, stream_id):
+'''Write vmstate data into archive'''
+conn, _ = sock.accept()
+sock.close()
+
+offset = 0
+while True:
+buf = conn.recv(256 * 1024)
+if len(buf) == 0:
+break
+queue.put((WRITER_OP_WRITE, stream_id, offset, buf))
+offset += len(buf)
+
+conn.close()
+
+def parse_option_list(s):
+return dict(kv.split('=') for kv in s.split(','))
+
+class NBDHandler(nbd.ExportHandler):
+def __init__(self, size, queue, stream_id):
+self._size = size
+self.queue = queue
+self.stream_id = stream_id
+
+def write(self, offset, data):
+self.queue.put((WRITER_OP_WRITE, self.stream_id, offset, data))
+
+def size(self):
+return self._size
+
+def consume_nbd(sock, drives):
+server = nbd.Server(sock)
+for drive in drives:
+server.add_export(drive['name'],
+NBDHandler(int(drive['size']), queue, drive['stream_id']))
+server.run()
+
+parser = argparse.ArgumentParser(description='VMA backup archive writer')
+parser.add_argument('--incoming',
+help='UNIX domain socket for incoming migration',
+required=True)
+parser.add_argument('--nbd',
+help='UNIX domain socket for NBD server',
+required=True)
+parser.add_argument('--drive',
+help='Device name of drive to back up',
+action='append',
+default=[])
+parser.add_argument('--output',
+help='Backup archive filename',
+required=True)
+
+args = parser.parse_args()
+drives = [parse_option_list(opts) for opts in args.drive]
+queue, vmstate_id = setup_vma_writer(args.output, drives)
+socks = setup_listen_sockets((args.incoming, args.nbd))
+
+# Let parent process know the sockets are listening
+sys.stdout.write('Ready\n')
+sys.stdout.flush()
+
+consume_migration(socks[0], queue, vmstate_id)
+consume_nbd(socks[1], drives)
+
+queue.put((WRITER_OP_STOP,))
-- 
1.8.1.4

[Qemu-devel] [RFC 3/8] backup: write to BlockDriverState instead of BackupDumpFunc

2013-03-09 Thread Stefan Hajnoczi

Remove the BackupDumpFunc function pointer and write directly to a
BlockDriverState.  This allows the 'backup' command to copy out into a
fresh qcow2 or raw image.  If no built-in image format is suitable, use
NBD to export the data to an external process.

A few other things in this commit:
 * Detect zero clusters with buffer_is_zero()
 * Skip zero clusters, don't write them to the target
 * Use 0 delay instead of 1us, like other block jobs
 * Delete the backup.h header file, it is no longer necessary
 * Unify creation/start functions into backup_start()
 * Simplify cleanup, free bitmap in backup_run() instead of cb function
 * Use bdrv_getlength() instead of accessing ->total_sectors directly

Signed-off-by: Stefan Hajnoczi 
---
 backup.c  | 110 --
 backup.h  |  30 -
 include/block/block_int.h |  16 +++
 3 files changed, 54 insertions(+), 102 deletions(-)
 delete mode 100644 backup.h

diff --git a/backup.c b/backup.c
index 8955e1a..8ee3450 100644
--- a/backup.c
+++ b/backup.c
@@ -19,7 +19,6 @@
 #include "block/block_int.h"
 #include "block/blockjob.h"
 #include "qemu/ratelimit.h"
-#include "backup.h"
 
 #define DEBUG_BACKUP 0
 
@@ -29,19 +28,20 @@
 do { if (DEBUG_BACKUP) { printf("backup: " fmt, ## __VA_ARGS__); } } \
 while (0)
 
+#define BACKUP_CLUSTER_BITS 16
+#define BACKUP_CLUSTER_SIZE (1backup_dump_cb(job->opaque, bs, start,
-  zero ? NULL : bounce_buffer);
-if (ret < 0) {
-DPRINTF("brdv_co_backup_cow dump_cluster_cb C%" PRId64 " failed\n",
-start);
-goto out;
+if (!zero) {
+ret = bdrv_co_writev(job->target, start * 
BACKUP_BLOCKS_PER_CLUSTER,
+ BACKUP_BLOCKS_PER_CLUSTER,
+ &bounce_qiov);
+if (ret < 0) {
+DPRINTF("brdv_co_backup_cow dump_cluster_cb C%" PRId64
+" failed\n", start);
+goto out;
+}
 }
 
 DPRINTF("brdv_co_backup_cow done C%" PRId64 "\n", start);
@@ -217,8 +222,8 @@ static void coroutine_fn backup_run(void *opaque)
 int64_t start, end;
 
 start = 0;
-end = (bs->total_sectors + BACKUP_BLOCKS_PER_CLUSTER - 1) /
-BACKUP_BLOCKS_PER_CLUSTER;
+end = (bdrv_getlength(bs) / BDRV_SECTOR_SIZE +
+   BACKUP_BLOCKS_PER_CLUSTER - 1) / BACKUP_BLOCKS_PER_CLUSTER;
 
 DPRINTF("backup_run start %s %" PRId64 " %" PRId64 "\n",
 bdrv_get_device_name(bs), start, end);
@@ -233,7 +238,6 @@ static void coroutine_fn backup_run(void *opaque)
 
 /* we need to yield so that qemu_aio_flush() returns.
  * (without, VM does not reboot)
- * Note: use 1000 instead of 0 (0 prioritize this task too much)
  */
 if (job->common.speed) {
 uint64_t delay_ns = ratelimit_calculate_delay(
@@ -241,7 +245,7 @@ static void coroutine_fn backup_run(void *opaque)
 job->sectors_read = 0;
 block_job_sleep_ns(&job->common, rt_clock, delay_ns);
 } else {
-block_job_sleep_ns(&job->common, rt_clock, 1000);
+block_job_sleep_ns(&job->common, rt_clock, 0);
 }
 
 if (block_job_is_cancelled(&job->common)) {
@@ -272,84 +276,46 @@ static void coroutine_fn backup_run(void *opaque)
 qemu_co_rwlock_wrlock(&job->rwlock);
 qemu_co_rwlock_unlock(&job->rwlock);
 
-DPRINTF("backup_run complete %d\n", ret);
-block_job_completed(&job->common, ret);
-}
-
-static void backup_job_cleanup_cb(void *opaque, int ret)
-{
-BlockDriverState *bs = opaque;
-assert(bs);
-BackupBlockJob *job = (BackupBlockJob *)bs->job;
-assert(job);
-
-DPRINTF("backup_job_cleanup_cb start %d\n", ret);
-
-job->backup_complete_cb(job->opaque, ret);
-
-DPRINTF("backup_job_cleanup_cb end\n");
-
 g_free(job->bitmap);
-}
 
-void
-backup_job_start(BlockDriverState *bs, bool cancel)
-{
-assert(bs);
-assert(bs->job);
-assert(bs->job->co == NULL);
+bdrv_delete(job->target);
 
-if (cancel) {
-block_job_cancel(bs->job); /* set cancel flag */
-}
-
-bs->job->co = qemu_coroutine_create(backup_run);
-qemu_coroutine_enter(bs->job->co, bs->job);
+DPRINTF("backup_run complete %d\n", ret);
+block_job_completed(&job->common, ret);
 }
 
-int
-backup_job_create(BlockDriverState *bs, BackupDumpFunc *backup_dump_cb,
-  BlockDriverCompletionFunc *backup_complete_cb,
-  void *opaque, int64_t speed)
+void backup_start(BlockDriverState *bs, BlockDriverState *target,
+  int64_t speed,
+  BlockDriverCompletionFunc *cb, void *opaque,
+  Error **errp)
 {
-assert(bs);
-assert(backup_dump_cb);
-assert(backup_complete_cb);
-
-if (bs->job) {
-

[Qemu-devel] [RFC 2/8] add basic backup support to block driver

2013-03-09 Thread Stefan Hajnoczi

From: Dietmar Maurer 

Function backup_job_create() creates a block job to backup a block device.
The coroutine is started with backup_job_start().

We call backup_do_cow() for each write during backup. That function
reads the original data and pass it to backup_dump_cb().

The tracked_request infrastructure is used to serialize access.

Currently backup cluster size is hardcoded to 65536 bytes.

Signed-off-by: Dietmar Maurer 
Signed-off-by: Stefan Hajnoczi 
---
 Makefile.objs|   1 +
 backup.c | 355 +++
 backup.h |  30 
 block.c  |  71 +-
 include/block/block.h|   2 +
 include/block/blockjob.h |  10 ++
 6 files changed, 463 insertions(+), 6 deletions(-)
 create mode 100644 backup.c
 create mode 100644 backup.h

diff --git a/Makefile.objs b/Makefile.objs
index a68cdac..df64f70 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -13,6 +13,7 @@ block-obj-$(CONFIG_POSIX) += aio-posix.o
 block-obj-$(CONFIG_WIN32) += aio-win32.o
 block-obj-y += block/
 block-obj-y += qapi-types.o qapi-visit.o
+block-obj-y += backup.o
 
 block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
 block-obj-y += qemu-coroutine-sleep.o
diff --git a/backup.c b/backup.c
new file mode 100644
index 000..8955e1a
--- /dev/null
+++ b/backup.c
@@ -0,0 +1,355 @@
+/*
+ * QEMU backup
+ *
+ * Copyright (C) 2013 Proxmox Server Solutions
+ *
+ * Authors:
+ *  Dietmar Maurer (diet...@proxmox.com)
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include 
+#include 
+#include 
+
+#include "block/block.h"
+#include "block/block_int.h"
+#include "block/blockjob.h"
+#include "qemu/ratelimit.h"
+#include "backup.h"
+
+#define DEBUG_BACKUP 0
+
+#define USE_ALLOCATION_CHECK 0
+
+#define DPRINTF(fmt, ...) \
+do { if (DEBUG_BACKUP) { printf("backup: " fmt, ## __VA_ARGS__); } } \
+while (0)
+
+
+#define SLICE_TIME 1ULL /* ns */
+
+typedef struct BackupBlockJob {
+BlockJob common;
+RateLimit limit;
+CoRwlock rwlock;
+uint64_t sectors_read;
+unsigned long *bitmap;
+int bitmap_size;
+BackupDumpFunc *backup_dump_cb;
+BlockDriverCompletionFunc *backup_complete_cb;
+void *opaque;
+} BackupBlockJob;
+
+static bool backup_get_bitmap(BackupBlockJob *job, int64_t cluster_num)
+{
+assert(job);
+assert(job->bitmap);
+
+unsigned long val, idx, bit;
+
+idx = cluster_num / BITS_PER_LONG;
+
+assert(job->bitmap_size > idx);
+
+bit = cluster_num % BITS_PER_LONG;
+val = job->bitmap[idx];
+
+return !!(val & (1UL << bit));
+}
+
+static void backup_set_bitmap(BackupBlockJob *job, int64_t cluster_num,
+  bool dirty)
+{
+assert(job);
+assert(job->bitmap);
+
+unsigned long val, idx, bit;
+
+idx = cluster_num / BITS_PER_LONG;
+
+assert(job->bitmap_size > idx);
+
+bit = cluster_num % BITS_PER_LONG;
+val = job->bitmap[idx];
+if (dirty) {
+val |= 1UL << bit;
+} else {
+val &= ~(1UL << bit);
+}
+job->bitmap[idx] = val;
+}
+
+static int coroutine_fn backup_do_cow(BlockDriverState *bs,
+  int64_t sector_num, int nb_sectors)
+{
+assert(bs);
+BackupBlockJob *job = (BackupBlockJob *)bs->job;
+assert(job);
+
+BlockDriver *drv = bs->drv;
+struct iovec iov;
+QEMUIOVector bounce_qiov;
+void *bounce_buffer = NULL;
+int ret = 0;
+
+qemu_co_rwlock_rdlock(&job->rwlock);
+
+int64_t start, end;
+
+start = sector_num / BACKUP_BLOCKS_PER_CLUSTER;
+end = (sector_num + nb_sectors + BACKUP_BLOCKS_PER_CLUSTER - 1) /
+BACKUP_BLOCKS_PER_CLUSTER;
+
+DPRINTF("brdv_co_backup_cow enter %s C%" PRId64 " %" PRId64 " %d\n",
+bdrv_get_device_name(bs), start, sector_num, nb_sectors);
+
+for (; start < end; start++) {
+bool zero = 0;
+
+if (backup_get_bitmap(job, start)) {
+DPRINTF("brdv_co_backup_cow skip C%" PRId64 "\n", start);
+continue; /* already copied */
+}
+
+/* immediately set bitmap (avoid coroutine race) */
+backup_set_bitmap(job, start, 1);
+
+DPRINTF("brdv_co_backup_cow C%" PRId64 "\n", start);
+
+if (!bounce_buffer) {
+iov.iov_len = BACKUP_CLUSTER_SIZE;
+iov.iov_base = bounce_buffer = qemu_blockalign(bs, iov.iov_len);
+qemu_iovec_init_external(&bounce_qiov, &iov, 1);
+}
+
+#if USE_ALLOCATION_CHECK
+int n = 0;
+ret = bdrv_co_is_allocated_above(bs, NULL,
+ start * BACKUP_BLOCKS_PER_CLUSTER,
+ BACKUP_BLOCKS_PER_CLUSTER, &n);
+if (ret < 0) {
+DPRINTF("brdv_co_backup_cow is_allocated C%" PRId64 " failed\n",
+start);
+goto out;
+

[Qemu-devel] [RFC 4/8] block: add block_backup QMP command

2013-03-09 Thread Stefan Hajnoczi

@block-backup

Start a point-in-time copy of a block device to a new destination.

@device:  the name of the device whose writes should be mirrored.

@target: the target of the new image. If the file exists, or if it
 is a device, the existing file/device will be used as the new
 destination.  If it does not exist, a new file will be created.

@format: #optional the format of the new destination, default is to
 probe if @mode is 'existing', else the format of the source

@mode: #optional whether and how QEMU should create a new image, default is
   'absolute-paths'.

@speed:  #optional the maximum speed, in bytes per second

Returns: nothing on success
 If @device is not a valid block device, DeviceNotFound

Signed-off-by: Stefan Hajnoczi 
---
 blockdev.c   | 92 
 qapi-schema.json | 28 +
 qmp-commands.hx  |  6 
 3 files changed, 126 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index 0e67d06..95a72de 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1208,6 +1208,98 @@ void qmp_block_commit(const char *device,
 drive_get_ref(drive_get_by_blockdev(bs));
 }
 
+void qmp_block_backup(const char *device, const char *target,
+  bool has_format, const char *format,
+  bool has_mode, enum NewImageMode mode,
+  bool has_speed, int64_t speed,
+  Error **errp)
+{
+BlockDriverState *bs;
+BlockDriverState *target_bs;
+BlockDriver *proto_drv;
+BlockDriver *drv = NULL;
+Error *local_err = NULL;
+int flags;
+uint64_t size;
+int ret;
+
+if (!has_speed) {
+speed = 0;
+}
+if (!has_mode) {
+mode = NEW_IMAGE_MODE_ABSOLUTE_PATHS;
+}
+
+bs = bdrv_find(device);
+if (!bs) {
+error_set(errp, QERR_DEVICE_NOT_FOUND, device);
+return;
+}
+
+if (!bdrv_is_inserted(bs)) {
+error_set(errp, QERR_DEVICE_HAS_NO_MEDIUM, device);
+return;
+}
+
+if (!has_format) {
+format = mode == NEW_IMAGE_MODE_EXISTING ? NULL : bs->drv->format_name;
+}
+if (format) {
+drv = bdrv_find_format(format);
+if (!drv) {
+error_set(errp, QERR_INVALID_BLOCK_FORMAT, format);
+return;
+}
+}
+
+if (bdrv_in_use(bs)) {
+error_set(errp, QERR_DEVICE_IN_USE, device);
+return;
+}
+
+flags = bs->open_flags | BDRV_O_RDWR;
+
+proto_drv = bdrv_find_protocol(target);
+if (!proto_drv) {
+error_set(errp, QERR_INVALID_BLOCK_FORMAT, format);
+return;
+}
+
+bdrv_get_geometry(bs, &size);
+size *= 512;
+if (mode != NEW_IMAGE_MODE_EXISTING) {
+assert(format && drv);
+bdrv_img_create(target, format,
+NULL, NULL, NULL, size, flags, &local_err, false);
+}
+
+if (error_is_set(&local_err)) {
+error_propagate(errp, local_err);
+return;
+}
+
+target_bs = bdrv_new("");
+ret = bdrv_open(target_bs, target, flags, drv);
+
+if (ret < 0) {
+bdrv_delete(target_bs);
+error_set(errp, QERR_OPEN_FILE_FAILED, target);
+return;
+}
+
+backup_start(bs, target_bs, speed, block_job_cb, bs, &local_err);
+if (local_err != NULL) {
+bdrv_delete(target_bs);
+error_propagate(errp, local_err);
+return;
+}
+
+/* Grab a reference so hotplug does not delete the BlockDriverState from
+ * underneath us.
+ */
+drive_get_ref(drive_get_by_blockdev(bs));
+}
+
 #define DEFAULT_MIRROR_BUF_SIZE   (10 << 20)
 
 void qmp_drive_mirror(const char *device, const char *target,
diff --git a/qapi-schema.json b/qapi-schema.json
index 6b64aec..1dbf7b5 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1716,6 +1716,34 @@
 '*speed': 'int' } }
 
 ##
+# @block-backup
+#
+# Start a point-in-time copy of a block device to a new destination.
+#
+# @device:  the name of the device whose writes should be mirrored.
+#
+# @target: the target of the new image. If the file exists, or if it
+#  is a device, the existing file/device will be used as the new
+#  destination.  If it does not exist, a new file will be created.
+#
+# @format: #optional the format of the new destination, default is to
+#  probe if @mode is 'existing', else the format of the source
+#
+# @mode: #optional whether and how QEMU should create a new image, default is
+#'absolute-paths'.
+#
+# @speed:  #optional the maximum speed, in bytes per second
+#
+# Returns: nothing on success
+#  If @device is not a valid block device, DeviceNotFound
+#
+# Since 1.5
+##
+{ 'command': 'block-backup',
+  'data': { 'device': 'str', 'target': 'str', '*format': 'str',
+'*mode': 'NewImageMode', '*speed': 'int' } }
+
+##
 # @drive-mirror
 #
 # Start mirroring a block device's writes to a new destination.
diff --git a/qmp-command

[Qemu-devel] [RFC 5/8] Add nbd server Python module

2013-03-09 Thread Stefan Hajnoczi

The nbd module works like this:

  server = nbd.Server(sock)
  server.add_export('drive0', handler0)
  server.add_export('drive1', handler1)
  server.run()

The user must provide a handler object which defines the behavior of an
export:

  class MyNBDHandler(nbd.ExportHandler):
  def write(self, offset, data):
  pass # do something

  def size(self):
  return 10 * 1024 * 1024 * 1024

Note that the handler is invoked from a thread.

Signed-off-by: Stefan Hajnoczi 
---
 nbd.py | 124 +
 1 file changed, 124 insertions(+)
 create mode 100644 nbd.py

diff --git a/nbd.py b/nbd.py
new file mode 100644
index 000..2528531
--- /dev/null
+++ b/nbd.py
@@ -0,0 +1,124 @@
+# NBD server module
+#
+# Copyright 2013 Red Hat, Inc. and/or its affiliates
+#
+# Authors:
+#   Stefan Hajnoczi 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+import struct
+import collections
+import threading
+
+__all__ = ['ExportHandler', 'Server']
+
+NBD_CMD_WRITE = 1
+NBD_CMD_DISC = 2
+NBD_REQUEST_MAGIC = 0x25609513
+NBD_REPLY_MAGIC = 0x67446698
+NBD_PASSWD = 0x4e42444d41474943
+NBD_OPTS_MAGIC = 0x49484156454F5054
+NBD_OPT_EXPORT_NAME = 1 << 0
+
+neg1_struct = struct.Struct('>QQH')
+export_tuple = collections.namedtuple('Export', 'reserved magic opt len')
+export_struct = struct.Struct('>IQII')
+neg2_struct = struct.Struct('>QH124x')
+request_tuple = collections.namedtuple('Request', 'magic type handle from_ 
len')
+request_struct = struct.Struct('>IIQQI')
+reply_struct = struct.Struct('>IIQ')
+
+def recvall(sock, bufsize):
+received = 0
+chunks = []
+while received < bufsize:
+chunk = sock.recv(bufsize - received)
+if len(chunk) == 0:
+raise Exception('unexpected disconnect')
+chunks.append(chunk)
+received += len(chunk)
+return ''.join(chunks)
+
+class ExportHandler(object):
+def write(self, offset, data):
+pass
+
+def size(self):
+return 0
+
+def negotiate(conn, exports):
+'''Negotiate export with client'''
+# Send negotiation part 1
+buf = neg1_struct.pack(NBD_PASSWD, NBD_OPTS_MAGIC, 0)
+conn.sendall(buf)
+
+# Receive export option
+buf = recvall(conn, export_struct.size)
+export = export_tuple._make(export_struct.unpack(buf))
+assert export.magic == NBD_OPTS_MAGIC
+assert export.opt == NBD_OPT_EXPORT_NAME
+name = recvall(conn, export.len)
+
+if name not in exports:
+print 'name "%s" not in exports' % name
+return None
+handler = exports[name]
+
+# Send negotiation part 2
+buf = neg2_struct.pack(handler.size(), 0)
+conn.sendall(buf)
+return handler
+
+def read_request(conn):
+'''Parse NBD request from client'''
+buf = recvall(conn, request_struct.size)
+req = request_tuple._make(request_struct.unpack(buf))
+assert req.magic == NBD_REQUEST_MAGIC
+return req
+
+def write_reply(conn, error, handle):
+buf = reply_struct.pack(NBD_REPLY_MAGIC, error, handle)
+conn.sendall(buf)
+
+def server_connection_thread(conn, exports):
+handler = negotiate(conn, exports)
+if handler is None:
+conn.close()
+return
+
+while True:
+req = read_request(conn)
+if req.type == NBD_CMD_WRITE:
+# Reply immediately, don't propagate internal errors to client
+write_reply(conn, 0, req.handle)
+
+data = recvall(conn, req.len)
+handler.write(req.from_, data)
+elif req.type == NBD_CMD_DISC:
+break
+else:
+print 'unrecognized command type %#02x' % req.type
+break
+conn.close()
+
+class Server(object):
+def __init__(self, sock):
+self.sock = sock
+self.exports = {}
+
+def add_export(self, name, handler):
+self.exports[name] = handler
+
+def run(self):
+threads = []
+for i in range(len(self.exports)):
+conn, _ = self.sock.accept()
+t = threading.Thread(target=server_connection_thread,
+ args=(conn, self.exports))
+t.daemon = True
+t.start()
+threads.append(t)
+for t in threads:
+t.join()
-- 
1.8.1.4

[Qemu-devel] [RFC 6/8] Add VMA backup archive writer Python module

2013-03-09 Thread Stefan Hajnoczi

The vma module provides an interface for writing VMA backup archives:

  writer = vma.Writer(open('test.vma', 'wb))
  writer.add_config('guest.xml', '')
  stream_id = writer.add_stream('foo',  # name
65536)  # size
  writer.write(stream_id, 0, '\0' * 32768)
  writer.write(stream_id, 32768, '\1' * 32768)
  writer.close()

The Writer handles sequential writes that are not cluster-aligned.  This
is typically only the vmstate.  Disk writes are 64 KB aligned in
practice.

VMA supports zero regions within a 64 KB cluster.  The vma module does
not implement this, the full cluster is written.

Signed-off-by: Stefan Hajnoczi 
---
 vma.py | 236 +
 1 file changed, 236 insertions(+)
 create mode 100644 vma.py

diff --git a/vma.py b/vma.py
new file mode 100644
index 000..236ba14
--- /dev/null
+++ b/vma.py
@@ -0,0 +1,236 @@
+# VMA writer module
+#
+# Copyright 2013 Red Hat, Inc. and/or its affiliates
+#
+# Authors:
+#   Stefan Hajnoczi 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+import array
+import struct
+import hashlib
+import uuid
+import time
+
+__all__ = ['Writer']
+
+VMA_MAGIC = 0x564d4100
+VMA_VERSION = 1
+VMA_MAX_CONFIGS = 256
+VMA_CLUSTER_SIZE = 65536
+VMA_BLOCKS_PER_EXTENT = 59
+VMA_EXTENT_MAGIC = 0x564d4145
+
+header_struct = struct.Struct('>II16cQ16cIII')
+dev_info_struct = struct.Struct('>IIQQQ')
+extent_struct = struct.Struct('>I2xH16c16c')
+le16_struct = struct.Struct('I')
+be64_struct = struct.Struct('>Q')
+
+class Writer(object):
+def __init__(self, fobj):
+self.fobj = fobj
+self.uuid = uuid.uuid4().bytes
+self.streams = []
+self.blobs = ['\0']
+self.blob_offset = 1
+self.config_names = []
+self.config_data = []
+self.header_written = False
+self.align_bufs = {}
+self.extent = []
+
+def alloc_blob(self, blob):
+'''Return allocated blob buffer offset'''
+offset = self.blob_offset
+self.blobs.append(le16_struct.pack(len(blob)))
+self.blobs.append(blob)
+self.blob_offset += le16_struct.size + len(blob)
+return offset
+
+def alloc_blob_str(self, s):
+'''Return allocated blob buffer offset for string'''
+return self.alloc_blob(s + '\0')
+
+def build_dev_info(self):
+'''Return a buffer with device infos'''
+bufs = ['\0' * dev_info_struct.size]
+for name, size in self.streams:
+name_ptr = self.alloc_blob_str(name)
+buf = dev_info_struct.pack(name_ptr, 0, size, 0, 0)
+bufs.append(buf)
+padding = (255 - len(self.streams)) * dev_info_struct.size
+bufs.append('\0' * padding)
+return ''.join(bufs)
+
+def build_blob_buffer(self):
+'''Return a buffer with blob data'''
+return ''.join(self.blobs)
+
+def build_config(self):
+'''Return a buffer with config names and data'''
+bufs = []
+
+for ptr in self.config_names:
+bufs.append(be32_struct.pack(ptr))
+padding = (VMA_MAX_CONFIGS - len(self.config_names)) * be32_struct.size
+bufs.append('\0' * padding)
+
+for ptr in self.config_data:
+bufs.append(be32_struct.pack(ptr))
+padding = (VMA_MAX_CONFIGS - len(self.config_data)) * be32_struct.size
+bufs.append('\0' * padding)
+
+return ''.join(bufs)
+
+def write_header(self):
+# Build header pieces
+config = self.build_config()
+dev_info = self.build_dev_info()
+blob_buffer = self.build_blob_buffer()
+
+# Size the header
+blob_buffer_offset = header_struct.size + 1984 + \
+ len(config) + 4 + len(dev_info)
+header_size = blob_buffer_offset + len(blob_buffer)
+
+# Build header without checksum
+fields = (VMA_MAGIC,
+  VMA_VERSION) + \
+ tuple(self.uuid) + \
+ (int(time.mktime(time.gmtime())),) + \
+ tuple('\0' * 16) + \
+ (blob_buffer_offset,
+  len(blob_buffer),
+  header_size)
+header = header_struct.pack(*fields)
+
+# Checksum header
+buf = ''.join([header,
+   '\0' * 1984,
+   config,
+   '\0' * 4, # VMAHeader.dev_info is unaligned (vma.h bug)
+   dev_info,
+   blob_buffer])
+digest = hashlib.md5(buf).digest()
+buf = array.array('c', buf) # string does not support assignment
+buf[32:32 + 16] = array.array('c', digest)
+
+self.fobj.write(buf)
+
+def add_config(self, name, data):
+name_ptr = self.alloc_blob_str(name)
+data_ptr = self.alloc_blob(data)
+self.config_names.append(name_ptr)
+

[Qemu-devel] [RFC 8/8] Add backup.py tool

2013-03-09 Thread Stefan Hajnoczi

The backup.py tool connects to a running guest's QMP monitor and
orchestrates a live backup.

To invoke a backup:

  qemu -qmp unix:/tmp/monitor.sock,server,nowait ...
  backup.py /tmp/monitor.sock backup-20130301.vma

The script uses vma-writer.py to bundle vmstate and disk backups into a
single backup archive file.  vma-writer.py is launched as a separate
process.

First a 'migrate' command is used to send the vmstate to the writer
process and pause the guest.

Then the 'block-backup' command is used to send a point-in-time copy of
all disks to the writer process before resuming the guest.

The guest continues running while disks are being backed up over NBD.

Finally, the backup completes when the 'block-backup' block jobs finish.

Signed-off-by: Stefan Hajnoczi 
---
 backup.py | 84 +++
 1 file changed, 84 insertions(+)
 create mode 100644 backup.py

diff --git a/backup.py b/backup.py
new file mode 100644
index 000..2e0b179
--- /dev/null
+++ b/backup.py
@@ -0,0 +1,84 @@
+#!/usr/bin/env python
+#
+# Live backup tool for QEMU
+#
+# Copyright 2013 Red Hat, Inc. and/or its affiliates
+#
+# Authors:
+#   Stefan Hajnoczi 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+import sys; sys.path.append('QMP')
+import qmp
+import subprocess
+
+MIGRATE_PATH = '/tmp/migrate.sock'
+NBD_PATH = '/tmp/nbd.sock'
+
+def find_backup_drives(mon):
+'''Return list of devices that should be backed up'''
+result = []
+for info in mon.command('query-block'):
+# Skip empty drives
+if 'inserted' not in info:
+continue
+
+# Skip read-only drives
+if info['inserted']['ro']:
+continue
+
+result.append((info['device'], info['inserted']['virtual_size']))
+return result
+
+def spawn_writer_process(filename, drives):
+'''Return Popen instance for vma-writer.py process'''
+args = ['python', 'vma-writer.py',
+'--output', filename,
+'--incoming', MIGRATE_PATH,
+'--nbd', NBD_PATH]
+for name, size in drives:
+args.extend(('--drive', 'name=%s,size=%s' % (name, size)))
+writer = subprocess.Popen(args, stdout=subprocess.PIPE)
+writer.stdout.readline() # Wait for "Ready"
+return writer
+
+def main(args):
+mon = qmp.QEMUMonitorProtocol(args[1])
+mon.connect()
+
+drives = find_backup_drives(mon)
+writer = spawn_writer_process(args[2], drives)
+
+sys.stderr.write('Running migration...\n')
+mon.command('migrate', uri='unix:' + MIGRATE_PATH)
+while True:
+evt = mon.pull_event(wait=True)
+if evt['event'] == 'STOP':
+break
+
+sys.stderr.write('Running block-backup...\n')
+for name, _ in drives:
+mon.command('block-backup',
+device=name,
+target='nbd+unix:///%s?socket=%s' % (name, NBD_PATH),
+format='raw',
+mode='existing')
+mon.command('cont')
+pending_drives = set(name for name, _ in drives)
+while pending_drives:
+evt = mon.pull_event(wait=True)
+if evt['event'] == 'BLOCK_JOB_COMPLETED':
+name = evt['data']['device']
+sys.stderr.write('Finished device "%s"\n' % name)
+pending_drives.discard(name)
+sys.stderr.write('Backup complete, terminating writer process\n')
+
+# Wait for writer process to terminate and print its output
+sys.stdout.write(writer.communicate()[0])
+
+mon.close()
+
+if __name__ == '__main__':
+main(sys.argv)
-- 
1.8.1.4

[Qemu-devel] [Bug 583296] Re: I/O errors with qemu-nbd/qcow2

2013-03-09 Thread Shehbaz Jaffer

Hi,

I tried this with qemu-img version 1.0, Ubuntu 12.04 host and gentoo guest, 
with qcow2 disk image. 
This seems to work fine. Perhaps it was fixed in the new qemu-img versions.

shehbaz@BLAckArrOw:~$sudo dd if=/dev/nbd0 of=/dev/null bs=1M
8192+0 records in
8192+0 records out
8589934592 bytes (8.6 GB) copied, 185.003 s, 46.4 MB/s
shehbaz@BLAckArrOw:~$

Did you try working with some other disk image? Did this issue get
resolved already? Perhaps more information will be required to debug
this now.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/583296

Title:
  I/O errors with qemu-nbd/qcow2

Status in QEMU:
  New

Bug description:
  I tried to open a qcow2 file with qemu-nbd and backup the files in it.
  After some coping I get lot of I/O errors in dmesg and the system
  hangs. One time I got even a kernel panic (Of course on a productive
  Server ;-) )

  How to reproduce:
  1. Connect nbd to a qcow2 file, a virtual machine mustn't use this file:
  qemu-nbd --connect=/dev/nbd0 /mnt/qcow/andromeda.qcow2 
  2. Read some data from /dev/nbd0p1
  dd if=/dev/nbd0p1 of=/dev/null bs=1M

  After a few Seconds till Minutes somethings crash's

  Attached some dmesg logs

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/583296/+subscriptions

Re: [Qemu-devel] [PATCH v4 3/6] add backup related monitor commands

2013-03-09 Thread Stefan Hajnoczi

On Fri, Mar 8, 2013 at 6:44 PM, Dietmar Maurer  wrote:
>> >> This is a strong indicator that the backup archive code should
>> >>live outside QEMU.  I doesn't make sense for proxmox, oVirt,
>> >>OpenStack, and others to each maintain their backup archive code
>> >>inside qemu.git, tied to QEMU's C codebase, release cycle, and
>> >>license.
>> >> 2. QEMU already has interfaces to export the vmstate and block device
>> >>snapshots: migration/savevm and BlockDriverState (NBD for IPC or
>> >>raw/qcow2/vmdk for file).  It is not necessary to add a
>> >>special-purpose interface just for backup.
>> >>
>> >> 3. The backup block job can be composed together with other QMP
>> commands
>> >>to achieve scenarios besides just VMA backup.  It's more flexible to
>> >>add simple primitives that can be combined instead of adding a
>> >>monolithic backup command.
>> >>
>> >> For these reasons, I'm against putting backup archive code into QEMU.
>> >
>> > That is OK for me - I already maintain the code outside of qemu.
>>
>> Does this mean you will keep this patch series out-of-tree?
>>
>> What I am looking for is a stripped down patch series with just a backup 
>> block
>> job (no backup archive writer or migration code).  That would be easily 
>> merged
>> and saves you front rebasing this series as QEMU changes.
>
> That is Patch 2/6?

Yes.  I sent an RFC series that shows this approach.  It is just a
prototype but it demonstrates that the NBD approach works and performs
reasonably well for a quick Python hack.

The stripped down patch series needs:
1. (Optional) query-block virtual_size field if management tool needs
to query drive size
2. Patch 2/6 with review comments addressed
3. 'block-backup' QMP command (and optionally HMP command)

That's it!

Stefan

37 matches

Mail list logo