[Xen-devel] dom0, guests, pci hardware and interrupts...

[Mike Carney <mc-al34luc@xxxxxxxxxxxxx>]

Greetings,

I configured my dom0 to hand a 3com NIC through to a guest:

# Unbind a PCI network card from its network driver
echo -n 0000:04:0d.0 > /sys/bus/pci/drivers/3c59x/unbind
# and now tell PCI Backend to watch the slot.
modprobe pciback
sleep 5
echo -n 0000:04:0d.0 > /sys/bus/pci/drivers/pciback/new_slot
# Now that the backend is watching for the slot, bind to it
echo -n 0000:04:0d.0 > /sys/bus/pci/drivers/pciback/bind

This works fine; the guest sees the NIC after the guest's py file is updated
appropriately:
pci =  [ '04:0d.0' ]

What happens:

This setup works for awhile.

The interrupt for the NIC is shared with another device used by dom0. In the
best case, dom0 and the guest are running on separate CPUS, and the
interrupt driver in dom0 is called whenever a packet comes in the guest,
which is unnecessary work, imho. In the worst case, when dom0 and the guest
are sharing a CPU, and the dom0 device sharing the interrupt is something
like the disk subsystem, dom0 and the guest appear to steal interrupts from
one another, and dom0 and all the guests grind to a halt (no disk I/O in
this example).

I've tried various techniques (kernel parameters in dom0) to ensure that the
interrupt assigned to the guest's NIC is *not* shared in order to avoid
this problem, to no avail.

Any suggestions for how to influence the interrupts that are assigned to
devices in dom0? It seems that there are a number of physical interrupts
that are not used, and some are shared:
           CPU0
  1:          2  Phys-irq-level     i8042
  6:          5  Phys-irq-level     floppy
  8:          1  Phys-irq-level     rtc
  9:          0  Phys-irq-level     acpi
 14:      89901  Phys-irq-level     libata
 15:       9347  Phys-irq-level     libata
 16:          0  Phys-irq-level     uhci_hcd:usb1
 17:     682628  Phys-irq-level     uhci_hcd:usb2, peth0
 18:      96245  Phys-irq-level     aic7xxx
 19:        356  Phys-irq-level     Intel 82801BA-ICH2, eth1
256:     263502  Dynamic-irq-level     timer0
257:          0  Dynamic-irq-level     resched0
258:          0  Dynamic-irq-level     callfunc0
259:       3823  Dynamic-irq-level     xenbus
260:        180  Dynamic-irq-level     console
261:      18936  Dynamic-irq-level     blkif-backend
262:       9431  Dynamic-irq-level     blkif-backend
263:      20035  Dynamic-irq-level     blkif-backend
264:      44080  Dynamic-irq-level     vif1.0
265:      12878  Dynamic-irq-level     blkif-backend
266:       1071  Dynamic-irq-level     vif4.0
267:       3352  Dynamic-irq-level     vif2.0
268:      17074  Dynamic-irq-level     blkif-backend
269:       5598  Dynamic-irq-level     vif5.0
270:     118463  Dynamic-irq-level     vif3.0

One would think that if the physical interrupts are available, and the
device can be configured to use them, that no sharing would occur until the
system allocated all the physical interrupts.. 

TIA,
Mike







_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel