Monday, November 17, 2014, 5:34:16 PM, you wrote:

> On Fri, Nov 14, 2014 at 11:09:58PM +0100, Sander Eikelenboom wrote:
>> 
>> Friday, November 14, 2014, 9:25:13 PM, you wrote:
>> 
>> > On Fri, Nov 14, 2014 at 05:59:23PM +0100, Sander Eikelenboom wrote:
>> >> 
>> >> Friday, November 14, 2014, 4:43:58 PM, you wrote:
>> >> 
>> >> >>>> On 14.11.14 at 16:20, <li...@eikelenboom.it> wrote:
>> >> >> If it still helps i could try Andrews suggestion and try out with only 
>> >> >> commit aeeea485 ..
>> >> 
>> >> > Yes, even if it's pretty certain it's the second of the commits, 
>> >> > verifying
>> >> > this would be helpful (or if the assumption is wrong, the pattern it's
>> >> > dying with would change and hence perhaps provide further clues).
>> >> 
>> >> > Jan
>> >> 
>> >> 
>> >> Ok with a revert of f6dd295 .. it survived cooking and eating a nice bowl 
>> >> of 
>> >> pasta without a panic. So it would probably be indeed that specific 
>> >> commit.
>> 
>> > Could you try running with these two patches while you enjoy an beer in 
>> > the evening?
>> 
>> Hmm i didn't expect it not to panic and reboot anymore :-)

> I should have also asked for your to run with 'iommu=verbose,debug', but
> that can be done later..

I was running with iommu=on,verbose,amd-iommu-debug ..


> The guest d16 looks to have two PCI passthrough devices:
> XEN) [2014-11-14 21:31:26.569] io.c:550: d16: bind: m_gsi=37 g_gsi=36 
> dev=00.00.5 intx=0
> XEN) [2014-11-14 21:31:28.095] io.c:550: d16: bind: m_gsi=47 g_gsi=40 
> dev=00.00.6 intx=0

> And one of them uses just the GSI while the other uses four MSI-X, is
> that about right?

Yes guest 16 has 1 USB controller(guest side 00:05.0) which has MSI-X enabled, 
and 1 conexant video-grabber 
(guest side 00:06.0) which should be MSI capable, but is is not enabled 
(probably by the driver) so 
using legacy interrupts.

> I tried to reproduce that on my AMD box with two NICs:

> # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.2 USB Controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton 
> II] (rev 01)
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
> 00:02.0 VGA compatible controller: Technical Corp. Device 1111
> 00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
> 00:04.0 Ethernet controller: Intel Corporation 82576 Gigabit Network 
> Connection (rev 01)
> 00:05.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet 
> Controller (rev 05)

> # cat /proc/interrupts |grep eth
>  36:     384183          0  xen-pirq-ioapic-level  eth0
>  63:          1          0  xen-pirq-msi-x     eth1
>  64:         24     661961  xen-pirq-msi-x     eth1-rx-0
>  65:        205          0  xen-pirq-msi-x     eth1-rx-1
>  66:        162          0  xen-pirq-msi-x     eth1-tx-0
>  67:        190          0  xen-pirq-msi-x     eth1-tx-1
> Is that a similar distribution of IRQ/MSIx you end up having?

These are when they are still active and assigned to dom0 (and not owned by 
pci-back) or in the guest ?

I attached my /proc/interrupts for both dom0 as guest 16 with all guests 
running 
(on a Xen from before the dpci changes). 
With the devices passed through I only see one line with the IRQ of a 
PCI soundcard passed through to a PV guest:
  22:      38959          0          0          0          0          0  
xen-pirq-ioapic-level  xen-pciback[0000:03:06.0]

All the other devices passed through (to HVM guests) are not visible in 
/proc/interrupts of dom0.

In the guest i do get these:
 23:         35          0          0          0  xen-pirq-ioapic-level  
uhci_hcd:usb3
 40:   13440077          0          0          0  xen-pirq-ioapic-level  
cx25821[1], cx25821[1]
 84:    2956369          0          0          0  xen-pirq-msi-x     xhci_hcd
 85:          0          0          0          0  xen-pirq-msi-x     xhci_hcd
 86:          0          0          0          0  xen-pirq-msi-x     xhci_hcd
 87:          0          0          0          0  xen-pirq-msi-x     xhci_hcd
 88:          0          0          0          0  xen-pirq-msi-x     xhci_hcd

>> 
>> However xl dmesg (complete one attached) showed it would have:
>> 
>> (XEN) [2014-11-14 21:35:50.646] --MARK--
>> (XEN) [2014-11-14 21:35:56.861] grant_table.c:305:d0v0 Increased maptrack 
>> size to 9 frames
>> (XEN) [2014-11-14 21:36:00.647] --MARK--
>> (XEN) [2014-11-14 21:36:10.410] grant_table.c:1299:d16v1 Expanding dom (16) 
>> grant table from (5) to (6) frames.
>> (XEN) [2014-11-14 21:36:10.820] --MARK--
>> (XEN) [2014-11-14 21:36:20.820] --MARK--
>> (XEN) [2014-11-14 21:36:30.820] --MARK--
>> (XEN) [2014-11-14 21:36:40.821] --MARK--
>> (XEN) [2014-11-14 21:36:50.821] --MARK--
>> (XEN) [2014-11-14 21:37:00.388] CPU00:
>> (XEN) [2014-11-14 21:37:00.399] CPU01:
>> (XEN) [2014-11-14 21:37:00.410] d16 OK-softirq 20msec ago, state:1, 41220 
>> count, [prev:ffff83054ef5e3e0, next:ffff83054ef5e3e0]  PIRQ:0
>> (XEN) [2014-11-14 21:37:00.445] d16 OK-raise   46msec ago, state:1, 41223 
>> count, [prev:0000000000200200, next:0000000000100100]  PIRQ:0
>> (XEN) [2014-11-14 21:37:00.481] d16 ERR-poison 92msec ago, state:0, 1 count, 
>> [prev:0000000000200200, next:0000000000100100]  PIRQ:0
>> (XEN) [2014-11-14 21:37:00.515] d16 Z-softirq  28853msec ago, state:2, 1 
>> count, [prev:0000000000200200, next:0000000000100100]  PIRQ:0

> The PIRQ:0 would imply that this is the legacy interrupt - which would be you 
> 0a:00.0 device 
> (Conexant Systems, Inc. Device 8210).

Correct.

> And it is pounding on this CPU - and the issue is that the 
> 'test_and_clear_bit' ends
> up returning 0 - which means it was not able to set STATE_SCHED:
> (!?)
>     if ( test_and_clear_bit(STATE_SCHED, &pirq_dpci->state) )               
>         {                                                                     
>   
>             hvm_dirq_assist(d, pirq_dpci);                                    
>   
>             put_domain(d);                                                    
>   
>         }                                                                     
>   
>         else                                                                  
>   
>         {                                                                     
>   
>             _record(&debug->zombie_softirq, pirq_dpci);        

> which causes us to record it [Z-softirq],  which says we we are in state 2
> (1<<STATE_RUN).

>             reset = 1;                                                        
>   
>         }                                        

> .. eons ago (28853msec).

> Hmm. There is something fishy there but the only theory I have is that
> we end up doing 'list_del' twice on different CPUs on the same structure.

The pounding would be correct .. since it's a videograbber ... wouldn't be 
fun not stretching the limits ;-) (however it's running fine for about 2 or 3 
years) 


> That should not be possible, but then this check - 'test_and_clear_bit' 
> returned
> 0 which means that somebody had cleared it (or it failed to clear it?)

> But the only other path for 'clearing' it is via the error paths and you are
> not hitting any of them.

> In the mean-time, could you try this patch. It adds a bit more debug to help
> me figure this out.

Ok will do this evening, thx !

> diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
> index 23e5ed1..443975c 100644
> --- a/xen/drivers/passthrough/io.c
> +++ b/xen/drivers/passthrough/io.c
> @@ -126,17 +126,17 @@ static void dump_record(struct _debug_f *d, unsigned 
> int type)
>          BUG();
>  
>      now = NOW();
> -    printk("d%d %s %lumsec ago, state:%x, %ld count, [prev:%p, next:%p] ",
> +    printk("d%d %s %lumsec ago, state:%x, %ld count, [prev:%p, next:%p] %p",
>             d->domid, names[type],
>             (unsigned long)((now - d->last) / MILLISECS(1)),
-            d->>state, d->count, d->list.prev, d->list.next);
+            d->>state, d->count, d->list.prev, d->list.next, d->dpci);
>  
>      if ( d->dpci )
>      {
>          struct hvm_pirq_dpci *pirq_dpci = d->dpci;
>  
>          for ( i = 0; i <= _HVM_IRQ_DPCI_GUEST_MSI_SHIFT; i++ )
> -            if ( pirq_dpci->flags & 1 << _HVM_IRQ_DPCI_TRANSLATE_SHIFT )
> +            if ( pirq_dpci->flags & (1 << i) )
>                  printk("%s ", names_flag[i]);
>  
>          printk(" PIRQ:%d", pirq_dpci->pirq);
            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       
   1:          2          0          0          0          0          0  
xen-pirq-ioapic-edge  i8042
   8:          0          0          0          0          0          0  
xen-pirq-ioapic-edge  rtc0
  12:          4          0          0          0          0          0  
xen-pirq-ioapic-edge  i8042
  16:        826          0          0          0          0          0  
xen-pirq-ioapic-level  snd_hda_intel
  17:          4          0          0          0          0          0  
xen-pirq-ioapic-level  ehci_hcd:usb1, ehci_hcd:usb2, ehci_hcd:usb3
  18:     694231          0          0          0          0          0  
xen-pirq-ioapic-level  ohci_hcd:usb4, ohci_hcd:usb5, ohci_hcd:usb6, 
ohci_hcd:usb7
  22:      38959          0          0          0          0          0  
xen-pirq-ioapic-level  xen-pciback[0000:03:06.0]
  56:   12177441          0          0          0          0          0  
xen-percpu-virq      timer0
  57:        267          0          0          0          0          0  
xen-percpu-ipi       spinlock0
  58:    4654275          0          0          0          0          0  
xen-percpu-ipi       resched0
  59:      23808          0          0          0          0          0  
xen-percpu-ipi       callfunc0
  60:          0          0          0          0          0          0  
xen-percpu-virq      debug0
  61:      82341          0          0          0          0          0  
xen-percpu-ipi       callfuncsingle0
  62:          9          0          0          0          0          0  
xen-percpu-ipi       irqwork0
  63:          0   14154255          0          0          0          0  
xen-percpu-virq      timer1
  64:          0        131          0          0          0          0  
xen-percpu-ipi       spinlock1
  65:          0   15021132          0          0          0          0  
xen-percpu-ipi       resched1
  66:          0      26512          0          0          0          0  
xen-percpu-ipi       callfunc1
  67:          0          0          0          0          0          0  
xen-percpu-virq      debug1
  68:          0      89412          0          0          0          0  
xen-percpu-ipi       callfuncsingle1
  69:          0          0          0          0          0          0  
xen-percpu-ipi       irqwork1
  70:          0          0   20187695          0          0          0  
xen-percpu-virq      timer2
  71:          0          0        137          0          0          0  
xen-percpu-ipi       spinlock2
  72:          0          0   13424711          0          0          0  
xen-percpu-ipi       resched2
  73:          0          0      25186          0          0          0  
xen-percpu-ipi       callfunc2
  74:          0          0          0          0          0          0  
xen-percpu-virq      debug2
  75:          0          0      95426          0          0          0  
xen-percpu-ipi       callfuncsingle2
  76:          0          0          4          0          0          0  
xen-percpu-ipi       irqwork2
  77:          0          0          0   22029851          0          0  
xen-percpu-virq      timer3
  78:          0          0          0        134          0          0  
xen-percpu-ipi       spinlock3
  79:          0          0          0   12698006          0          0  
xen-percpu-ipi       resched3
  80:          0          0          0      28372          0          0  
xen-percpu-ipi       callfunc3
  81:          0          0          0          0          0          0  
xen-percpu-virq      debug3
  82:          0          0          0     100185          0          0  
xen-percpu-ipi       callfuncsingle3
  83:          0          0          0          5          0          0  
xen-percpu-ipi       irqwork3
  84:          0          0          0          0   22473368          0  
xen-percpu-virq      timer4
  85:          0          0          0          0        135          0  
xen-percpu-ipi       spinlock4
  86:          0          0          0          0   12204793          0  
xen-percpu-ipi       resched4
  87:          0          0          0          0      28057          0  
xen-percpu-ipi       callfunc4
  88:          0          0          0          0          0          0  
xen-percpu-virq      debug4
  89:          0          0          0          0     100463          0  
xen-percpu-ipi       callfuncsingle4
  90:          0          0          0          0          3          0  
xen-percpu-ipi       irqwork4
  91:          0          0          0          0          0   22712944  
xen-percpu-virq      timer5
  92:          0          0          0          0          0        120  
xen-percpu-ipi       spinlock5
  93:          0          0          0          0          0   11590989  
xen-percpu-ipi       resched5
  94:          0          0          0          0          0      28061  
xen-percpu-ipi       callfunc5
  95:          0          0          0          0          0          0  
xen-percpu-virq      debug5
  96:          0          0          0          0          0      98564  
xen-percpu-ipi       callfuncsingle5
  97:          0          0          0          0          0          1  
xen-percpu-ipi       irqwork5
  98:      12552          0          0          0          0          0   
xen-dyn-event     xenbus
  99:          0          0          0          0          0          0  
xen-percpu-virq      xen-pcpu
 114:    2941803          0          0          0          0          0  
xen-pirq-msi       ahci0
 115:          0          0          0          0          0          0  
xen-pirq-msi       ahci1
 116:     203026          0          0          0          0          0  
xen-pirq-msi       ahci2
 117:          0          0          0          0          0          0  
xen-pirq-msi       ahci3
 118:          0          0          0          0          0          0  
xen-pirq-msi       ahci4
 119:          0          0          0          0          0          0  
xen-pirq-msi       ahci5
 122:   21186947          0          0          0          0          0  
xen-pirq-msi       eth0
 123:    5413652          0          0          0          0          0  
xen-pirq-msi       eth1
 124:         28          0          0          0          0          0  
xen-pirq-msi       snd_hda_intel
 125:       8678          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 126:          0          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 127:        328          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 128:        491          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 129:      46709          0          0          0          0          0   
xen-dyn-event     blkif-backend
 130:     697773          0          0          0          0          0   
xen-dyn-event     blkif-backend
 131:     605273          0          0          0          0          0   
xen-dyn-event     vif1.0-q0-tx
 132:          6          0          0          0          0          0   
xen-dyn-event     vif1.0-q0-rx
 133:     475856          0          0          0          0          0   
xen-dyn-event     vif1.0-q1-tx
 134:          2          0          0          0          0          0   
xen-dyn-event     vif1.0-q1-rx
 135:        214          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 136:        476          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 137:     105977          0          0          0          0          0   
xen-dyn-event     blkif-backend
 138:   12523359          0          0          0          0          0   
xen-dyn-event     vif2.0-q0-tx
 139:     106301          0          0          0          0          0   
xen-dyn-event     vif2.0-q0-rx
 140:        243          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 141:        485          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 142:      30770          0          0          0          0          0   
xen-dyn-event     blkif-backend
 143:    1238851          0          0          0          0          0   
xen-dyn-event     vif3.0-q0-tx
 144:       2632          0          0          0          0          0   
xen-dyn-event     vif3.0-q0-rx
 145:     140302          0          0          0          0          0   
xen-dyn-event     vif3.0-q1-tx
 146:         42          0          0          0          0          0   
xen-dyn-event     vif3.0-q1-rx
 147:        214          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 148:        502          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 149:      53178          0          0          0          0          0   
xen-dyn-event     blkif-backend
 150:    1075289          0          0          0          0          0   
xen-dyn-event     vif4.0-q0-tx
 151:       1032          0          0          0          0          0   
xen-dyn-event     vif4.0-q0-rx
 152:        214          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 153:        530          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 154:      51115          0          0          0          0          0   
xen-dyn-event     blkif-backend
 155:    1205378          0          0          0          0          0   
xen-dyn-event     vif5.0-q0-tx
 156:       5090          0          0          0          0          0   
xen-dyn-event     vif5.0-q0-rx
 157:        219          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 158:        489          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 159:      10642          0          0          0          0          0   
xen-dyn-event     blkif-backend
 160:      19035          0          0          0          0          0   
xen-dyn-event     vif6.0-q0-tx
 161:          1          0          0          0          0          0   
xen-dyn-event     vif6.0-q0-rx
 162:        218          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 163:        538          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 164:      11157          0          0          0          0          0   
xen-dyn-event     blkif-backend
 165:      31852          0          0          0          0          0   
xen-dyn-event     vif7.0-q0-tx
 166:          1          0          0          0          0          0   
xen-dyn-event     vif7.0-q0-rx
 167:        219          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 168:        490          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 169:      15096          0          0          0          0          0   
xen-dyn-event     blkif-backend
 170:       3918          0          0          0          0          0   
xen-dyn-event     vif8.0-q0-tx
 171:          1          0          0          0          0          0   
xen-dyn-event     vif8.0-q0-rx
 172:        243          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 173:        540          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 174:      24094          0          0          0          0          0   
xen-dyn-event     blkif-backend
 175:    6784871          0          0          0          0          0   
xen-dyn-event     vif9.0-q0-tx
 176:      11328          0          0          0          0          0   
xen-dyn-event     vif9.0-q0-rx
 177:    6284170          0          0          0          0          0   
xen-dyn-event     vif9.0-q1-tx
 178:       1513          0          0          0          0          0   
xen-dyn-event     vif9.0-q1-rx
 179:        216          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 180:        555          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 181:      13586          0          0          0          0          0   
xen-dyn-event     blkif-backend
 182:       1495          0          0          0          0          0   
xen-dyn-event     vif10.0-q0-tx
 183:          2          0          0          0          0          0   
xen-dyn-event     vif10.0-q0-rx
 184:        240          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 185:        550          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 186:      17149          0          0          0          0          0   
xen-dyn-event     blkif-backend
 187:    1214716          0          0          0          0          0   
xen-dyn-event     vif11.0-q0-tx
 188:        175          0          0          0          0          0   
xen-dyn-event     vif11.0-q0-rx
 189:    1115309          0          0          0          0          0   
xen-dyn-event     vif11.0-q1-tx
 190:          7          0          0          0          0          0   
xen-dyn-event     vif11.0-q1-rx
 191:        210          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 192:        529          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 193:      10422          0          0          0          0          0   
xen-dyn-event     blkif-backend
 194:      57918          0          0          0          0          0   
xen-dyn-event     vif12.0-q0-tx
 195:          2          0          0          0          0          0   
xen-dyn-event     vif12.0-q0-rx
 196:        271          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 197:        641          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 198:        375          0          0          0          0          0   
xen-dyn-event     xen-pciback
 199:      10690          0          0          0          0          0   
xen-dyn-event     blkif-backend
 200:      72627          0          0          0          0          0   
xen-dyn-event     vif13.0-q0-tx
 201:         83          0          0          0          0          0   
xen-dyn-event     vif13.0-q0-rx
 202:        218          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 203:        597          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 204:      19220          0          0          0          0          0   
xen-dyn-event     blkif-backend
 205:     822807          0          0          0          0          0   
xen-dyn-event     vif14.0-q0-tx
 206:          2          0          0          0          0          0   
xen-dyn-event     vif14.0-q0-rx
 207:        215          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 208:        537          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 209:      19872          0          0          0          0          0   
xen-dyn-event     blkif-backend
 210:        516          0          0          0          0          0   
xen-dyn-event     vif15.0-q0-tx
 211:          1          0          0          0          0          0   
xen-dyn-event     vif15.0-q0-rx
 212:        537          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 213:          5          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 214:     330738          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 215:    2234503          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 216:      23800          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 217:      23211          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 218:        123          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 219:        379          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 220:          1          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 221:     409341          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 222:       3774          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 223:       1136          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 224:         32          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 225:        422          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 226:          9          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 227:     360518          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 228:       2314          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 229:        898          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 230:       1620          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 231:         97          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 232:      94589          0          0          0          0          0   
xen-dyn-event     blkif-backend
 233:      36804          0          0          0          0          0   
xen-dyn-event     blkif-backend
 234:     188388          0          0          0          0          0   
xen-dyn-event     vif16.0-q0-tx
 235:          1          0          0          0          0          0   
xen-dyn-event     vif16.0-q0-rx
 236:      53810          0          0          0          0          0   
xen-dyn-event     vif16.0-q1-tx
 237:          1          0          0          0          0          0   
xen-dyn-event     vif16.0-q1-rx
 238:      48083          0          0          0          0          0   
xen-dyn-event     vif16.0-q2-tx
 239:          1          0          0          0          0          0   
xen-dyn-event     vif16.0-q2-rx
 240:      95695          0          0          0          0          0   
xen-dyn-event     vif16.0-q3-tx
 241:          2          0          0          0          0          0   
xen-dyn-event     vif16.0-q3-rx
 242:      12508          0          0          0          0          0   
xen-dyn-event     blkif-backend
 243:      56174          0          0          0          0          0   
xen-dyn-event     vif17.0-q0
 244:      15437          0          0          0          0          0   
xen-dyn-event     blkif-backend
 245:         46          0          0          0          0          0   
xen-dyn-event     vif18.0-q0-tx
 246:          1          0          0          0          0          0   
xen-dyn-event     vif18.0-q0-rx
 247:         27          0          0          0          0          0   
xen-dyn-event     vif18.0-q1-tx
 248:          1          0          0          0          0          0   
xen-dyn-event     vif18.0-q1-rx
 249:          8          0          0          0          0          0   
xen-dyn-event     vif18.0-q2-tx
 250:          1          0          0          0          0          0   
xen-dyn-event     vif18.0-q2-rx
 251:          8          0          0          0          0          0   
xen-dyn-event     vif18.0-q3-tx
 252:          1          0          0          0          0          0   
xen-dyn-event     vif18.0-q3-rx
 253:        352          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 254:          1          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 255:     159042          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 256:      36101          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 257:        209          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 258:        355          0          0          0          0          0   
xen-dyn-event     evtchn:oxenstored
 259:          6          0          0          0          0          0   
xen-dyn-event     evtchn:xenconsoled
 260:     211185          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 261:        132          0          0          0          0          0   
xen-dyn-event     evtchn:qemu-system-i38
 262:      45784          0          0          0          0          0   
xen-dyn-event     blkif-backend
 263:          5          0          0          0          0          0   
xen-dyn-event     vif19.0-q0-tx
 264:          1          0          0          0          0          0   
xen-dyn-event     vif19.0-q0-rx
 265:      25140          0          0          0          0          0   
xen-dyn-event     blkif-backend
 266:     100595          0          0          0          0          0   
xen-dyn-event     vif20.0-q0
 NMI:          0          0          0          0          0          0   
Non-maskable interrupts
 LOC:          0          0          0          0          0          0   Local 
timer interrupts
 SPU:          0          0          0          0          0          0   
Spurious interrupts
 PMI:          0          0          0          0          0          0   
Performance monitoring interrupts
 IWI:          9          0          4          5          3          1   IRQ 
work interrupts
 RTR:          0          0          0          0          0          0   APIC 
ICR read retries
 RES:    4654275   15021132   13424711   12698006   12204793   11590989   
Rescheduling interrupts
 CAL:     106149     115924     120612     128557     128520     126625   
Function call interrupts
 TLB:          0          0          0          0          0          0   TLB 
shootdowns
 TRM:          0          0          0          0          0          0   
Thermal event interrupts
 THR:          0          0          0          0          0          0   
Threshold APIC interrupts
 MCE:          0          0          0          0          0          0   
Machine check exceptions
 MCP:        310        310        310        310        310        310   
Machine check polls
 THR:   84498403   28932303   33373015   34517510   34480476   34113277   
Hypervisor callback interrupts
 ERR:          0
 MIS:          0
           CPU0       CPU1       CPU2       CPU3       
  0:        136          0          0          0   IO-APIC-edge      timer
  1:          9          0          0          0  xen-pirq-ioapic-edge  i8042
  4:        538          0          0          0  xen-pirq-ioapic-edge  serial
  8:          2          0          0          0  xen-pirq-ioapic-edge  rtc0
  9:          0          0          0          0   IO-APIC-fasteoi   acpi
 12:        125          0          0          0  xen-pirq-ioapic-edge  i8042
 23:         35          0          0          0  xen-pirq-ioapic-level  
uhci_hcd:usb3
 40:   13440077          0          0          0  xen-pirq-ioapic-level  
cx25821[1], cx25821[1]
 48:   13973033          0          0          0  xen-percpu-virq      timer0
 49:     782827          0          0          0  xen-percpu-ipi       resched0
 50:    1825582          0          0          0  xen-percpu-ipi       callfunc0
 51:          0          0          0          0  xen-percpu-virq      debug0
 52:     695548          0          0          0  xen-percpu-ipi       
callfuncsingle0
 53:       6032          0          0          0  xen-percpu-ipi       spinlock0
 54:          0    9252391          0          0  xen-percpu-virq      timer1
 55:          0    3665035          0          0  xen-percpu-ipi       resched1
 56:          0    1842364          0          0  xen-percpu-ipi       callfunc1
 57:          0          0          0          0  xen-percpu-virq      debug1
 58:          0     631299          0          0  xen-percpu-ipi       
callfuncsingle1
 59:          0       8587          0          0  xen-percpu-ipi       spinlock1
 60:          0          0    8239096          0  xen-percpu-virq      timer2
 61:          0          0    2537679          0  xen-percpu-ipi       resched2
 62:          0          0    1877050          0  xen-percpu-ipi       callfunc2
 63:          0          0          0          0  xen-percpu-virq      debug2
 64:          0          0     545407          0  xen-percpu-ipi       
callfuncsingle2
 65:          0          0       7974          0  xen-percpu-ipi       spinlock2
 66:          0          0          0    8083500  xen-percpu-virq      timer3
 67:          0          0          0    2004257  xen-percpu-ipi       resched3
 68:          0          0          0    1917596  xen-percpu-ipi       callfunc3
 69:          0          0          0          0  xen-percpu-virq      debug3
 70:          0          0          0     578174  xen-percpu-ipi       
callfuncsingle3
 71:          0          0          0       7463  xen-percpu-ipi       spinlock3
 72:        511          0          0          0   xen-dyn-event     xenbus
 73:          2          0          0          0   xen-dyn-event     hvc_console
 74:     101869          0          0          0   xen-dyn-event     blkif
 75:      62003          0          0          0   xen-dyn-event     blkif
 76:     191416          0          0          0   xen-dyn-event     eth0-q0-tx
 77:     138239          0          0          0   xen-dyn-event     eth0-q0-rx
 78:      58361          0          0          0   xen-dyn-event     eth0-q1-tx
 79:     131127          0          0          0   xen-dyn-event     eth0-q1-rx
 80:      50933          0          0          0   xen-dyn-event     eth0-q2-tx
 81:      43942          0          0          0   xen-dyn-event     eth0-q2-rx
 82:      94197          0          0          0   xen-dyn-event     eth0-q3-tx
 83:     101692          0          0          0   xen-dyn-event     eth0-q3-rx
 84:    2956369          0          0          0  xen-pirq-msi-x     xhci_hcd
 85:          0          0          0          0  xen-pirq-msi-x     xhci_hcd
 86:          0          0          0          0  xen-pirq-msi-x     xhci_hcd
 87:          0          0          0          0  xen-pirq-msi-x     xhci_hcd
 88:          0          0          0          0  xen-pirq-msi-x     xhci_hcd
 89:          0          0          0          0   xen-dyn-event     vkbd
NMI:          0          0          0          0   Non-maskable interrupts
LOC:          0          0          0          0   Local timer interrupts
SPU:          0          0          0          0   Spurious interrupts
PMI:          0          0          0          0   Performance monitoring 
interrupts
IWI:          0          0          0          0   IRQ work interrupts
RTR:          0          0          0          0   APIC ICR read retries
RES:     782827    3665035    2537679    2004257   Rescheduling interrupts
CAL: 4294856524 4294911444 4294962534 4294953352   Function call interrupts
TLB:    2631902    2529515    2427219    2509714   TLB shootdowns
TRM:          0          0          0          0   Thermal event interrupts
THR:          0          0          0          0   Threshold APIC interrupts
MCE:          0          0          0          0   Machine check exceptions
MCP:        308        308        308        308   Machine check polls
THR:   33978071   15296475   13131641   12526119   Hypervisor callback 
interrupts
ERR:          0
MIS:          0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Reply via email to