RE: [Xen-devel] [PATCH 0/5] VT-d support for PV guests

[Espen Skoglund <espen.skoglund@xxxxxxxxxxxxx>]

[Ian Pratt]
>> The patchset does, as you guessed, enable isolation for PV guests
>> with direct hardware access.  If you assign a PCI device to a guest
>> you are guaranteed that the assigned device can't access the memory
>> of other guests or Xen itself.  The patchseet allows the device to
>> access all its own memory which it has write access to, and memory
>> which is granted to it.

> Not that I particularly think it matters, but does the patch
> configure the IOMMU to distinguish between read-only and read-write
> access to a guest's own memory or granted memory? If not, we should
> at least clearly document that we're being a little more permissive.

The idea was indeed to distinguish between this properly.  However,
the current VT-d code only handles read-write or no-access.  For PV
guests I've made it so that page tables and such are mapped with no
access in the IOMMU.  This is a bit more restrictive than necessary,
but it shouldn't really matter for the common usage scenarios.

Anyhow, read-only access can indeed be supported for VT-d.  I just
wanted to get basic PV guest support in there first.  Also, I'm not
familiar with AMD's IOMMU, but I would guess that it also supports
read-only access.



> Have you got any with-and-without performance results with a decent
> high-throughput device (e.g. a HBA or 10Gb/s NIC)?

I don't have a 10GbE NIC in my VT-d enabled machine right now, so I
can't test it.  We have however tried with a 10GbE NIC running in dom0
with VT-d enabled, and there was as far as I rememeber no performance
hit.  Of course, any performance degradation will largely depend on
the networking memory footprint and the size of the IOTLB.



> It would be good if you could provide a bit more detail on when the
> patch populates IOMMU entries, and how it keeps them in sync. For
> example, does the IOMMU map all the guest's memory, or just that
> which will soon be the subject of a DMA? How synchronous is the
> patch in removing mappings, e.g. due to page type changes (pagetable
> pages, balloon driver) or due to unmapping grants?

All writable memory is initially mapped in the IOMMU.  Page type
changes are also reflected there.  In general all maps and unmaps to a
domain are synced with the IOMMU.  According to the feedback I got I
apparently missed some places, though.  Will look into this and fix
it.

It's clear that performance will pretty much suck if you do frequent
updates in grant tables, but the whole idea of having passthrough
access for NICs is to avoid this netfront/netback data plane scheme
altogether.  This leaves you with grant table updates for block device
access.  I don't know what the expected update frequency is for that
one.

It must be noted that reflecting grant table updates in the IOMMU is
required for correctness.  The alternative --- which is indeed
possible --- is to catch DMA faults to such memory regions and somehow
notify the driver to, e.g., drop packets or retry the DMA transaction
once the IOMMU mapping has been established.



> There's been a lot of discussion at various xen summits about
> different IOMMU optimizations (e.g. for IBM Summit, Power etc) and
> I'd like to understand exactly what tradeoffs your implementation
> makes. Anyhow, good stuff, thanks!

I can't say I know much about those other IOMMUs, but as far as I know
they are quite limited in that they only support a fixed number of
mappings and can not differentiate between different DMA sources
(i.e., PCI devices).  Someone please correct me if I'm wrong here.  In
short, "my" implementation don't actually do many tradeoffs.  It's
simply based on the VT-d implementation by the Intel folks.  It
assumes a more fully fledged IOMMU that can have different mappings
for different devices.

        eSk



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel