Re: [Xen-devel] MSI and VT-d interrupt remapping

[Espen Skoglund <espen.skoglund@xxxxxxxxxxxxx>]

[Keir Fraser]
> On 6/3/08 21:07, "Espen Skoglund" <espen.skoglund@xxxxxxxxxxxxx> wrote:
>> If an MSI capable device was able to make use of the above feature,
>> the device could be set up to generate different interrupts
>> depending on where the incoming interrupt was to be handled.  For
>> example, incoming data for a particular guest could trigger an
>> interrupt on the processor where that guest is running.  Obviously,
>> a dom0-like backend driver would not be involved in the actual
>> event delivery in these situations.  The event would be delivered
>> directly to the frontend.
>> 
>> The necessary changes would enable a device driver for an MSI capable
>> device to allocate a range of pirqs and bind these to different
>> frontends.

> The only tricky bit here is deciding what the interface should be to
> the hypervisor to specify these allocation constraints.

> Another thought though: there's no good reqson for Xen to scatter
> its irq-vector allocations across the vector space. That's a
> holdover from classic-Pentium-era systems, which could lose
> interrupts if too many got 'queued up' at any single priority
> level. So we could actually allocate our vectors contiguously,
> making it much more likely that you could successfully allocate a
> contiguous range even without remapping.

> However, I guess you want to be able to specify different target
> APICs for different vectors too, so again it comes back to: what
> should the guest interface to irq-remapping hardware be?

Right.  The reason for bringing up this suggestion now rather than
later is because MSI support has not yet found its way into mainline.
Whoever decides on the interface used for registering MSI and MSI-X
interrupts might want to take multi-message MSIs into account as well.

I do not think explicitly specifying destination APIC upon allocation
is the best idea.  Setting the affinity upon binding the interrupt
like it's done today seems like a better approach.  This leaves us
with dealing with the vectors.

My initial thought was to make use of the new msix_entries[] field in
the xen_pci_op structure.  This field is already used as an in/out
parameter for allocating MSI-X interrupts.  The pciback_enable_msi()
function can then attempt to allocate multiple interrupts instead of a
single one, and return the allocated vectors.

The current MSI patchset also lacks a set_affinity() function for
changing the APIC destination similar to what is done for, e.g.,
IOAPICs.  Also similar to IOAPICs, the MSI support should have
something like the io_apic_write_remap_rte() for rewriting the
interrupt remapping table when enabled.

A special case must exist when setting the interrupt affinity for
multiple-message enabled MSI devices.  There probably should exist
some magic in the set_affinity() function for handling this properly.
That is, setting affinity for the whole group of MSI interrupts does
not make all that much sense.  It makes more sense when one can set
the per-interrupt affinity through the interrupt remapping table.

It should be evident by now that my suggestions for deciding upon an
interface and implementation of it is rather fluffy; borderlining
non-existent.  This is partly because I'm talking about not yet
existing MSI support, but mainly because I'm still new to Xen
internals.  Nonetheless, I believe it would be good for people working
on MSI support to take multi-message and interrupt remapping into
account as well.

        eSk

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel