RE: [Xen-devel] Dom0 hypercall for adding and removing PCI devices

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

Isn't enough to first switch VT-d page table, and then flush IOTLB?
As long as RMRRs are kept same in two VT-d tables, and in any
time a valid entry (either in IOTLB or by walking) can be found, 
above sequence seems complete.

Thanks,
Kevin

>From: Keir Fraser
>Sent: 2008年7月24日 16:38
>
>Can the pagetable switch not be made atomic (i.e., so that the 
>RMRR regions
>appear continuously available throughout)? I'd have thought 
>that would just
>naturally happen.
>
>If creating/destroying RMRR mappings is part of the switch 
>operation, you'd
>have to destroy RMRR mappings in the dom0 table after the 
>switch, and create
>RMRR mappings in the fallback table before the switch. Or just 
>have RMRR
>mappings always mapped into all tables.
>
> -- Keir
>
>On 24/7/08 09:32, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>
>> We found USB (has RMRR) is firstly assigned to dom0, but 
>pci_bus_probe()
>> failed, then it was removed from dom0. The removing needs 
>switch to RMRR
>> VT-d page table. At the same time, BIOS was using its RMRR.
>> 
>> Randy (Weidong)
>> 
>> Keir Fraser wrote:
>>> If a device is assigned to a domain (in this case dom0) then that
>>> domain's VT-d pagetables will contain the RMRR mappings for that
>>> device. Hence BIOS can perform DMA in those RMRR-indicated regions
>>> without swapping to and fro between domain tables and fallback RMRR
>>> tables. The new fallback RMRR table would be just that -- a fallback
>>> table used for any device not currently assigned to any domain (and
>>> hence those devices should only have DMAs initiated by the BIOS, if
>>> at all). 
>>> 
>>>  -- Keir
>>> 
>>> On 24/7/08 09:20, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>> 
>>>> I have another concern, when BIOS is initiating DMA operation using
>>>> RMRR, can we use RMRR VT-d page table to replace dom0 VT-d page
>>>> table? Does it result in some DMA failures?
>>>> 
>>>> Randy (weidong)
>>>> 
>>>> Han, Weidong wrote:
>>>>> Espen Skoglund wrote:
>>>>>> [Keir Fraser]
>>>>>>> On 23/7/08 10:26, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>>>>>>>> So this would be one extra VT-d pagetable, for the 
>whole system,
>>>>>>>>> which would be the fallback location for RMRR mappings for
>>>>>>>>> devices which are currently not assigned to any domain? Thus
>>>>>>>>> allowing firmware to successfully initiate DMA operations on
>>>>>>>>> those devices? Sounds sensible.
>>>>>> 
>>>>>> Well, the VT-d page table for RMRR devices need not contain the
>>>>>> whole system memory---only those regions specified in the DMAR
>>>>>> tables. 
>>>>>> 
>>>>>>>> Is it possible that idle_domain owns the RMRR VT-d page table?
>>>>>> 
>>>>>>> If that's a convenient place to stash it then why not? 
>Either way,
>>>>>>> seems you're going to have it special-cased in the code as
>>>>>>> fallback owner for unassigned devices. It's possible that having
>>>>>>> it stashed in the idle domain will simply make the code more
>>>>>>> confusing. I'm not sure though.
>>>>>> 
>>>>>> Right.  I don't see any particular good reason to 
>associate it with
>>>>>> the idle domain.  As noted above, the page table need not even
>>>>>> cover the whole memory, and it will never change after being set
>>>>>> up at boot time.  If special case code is needed anyway, then one
>>>>>> might as well install a custom VT-d page table.
>>>>> 
>>>>> What does "custom VT-d page table" mean?
>>>>> 
>>>>> Due to domain id is not the same with DID field in context, we can
>>>>> reverse a DID for RMRR VT-d page table, it can avoid to associate
>>>>> with idle domain.
>>>>> 
>>>>> Currently we reassign the device from dom0 to target domain when
>>>>> assign a device, and return the device to dom0 when target domain
>>>>> tears down. It's not correct due to some devices may be 
>not assigned
>>>>> to any domain. Current device_assigned() also needs to be changed.
>>>>> Maybe it needs more changes on VT-d.
>>>>> 
>>>>> I have some concerns, maybe I missed something. Why did 
>you use dom0
>>>>> hypercall approach to replace original method? What's the main
>>>>> benefit? I also wonder it's suitable to wrap pci_bus_probe()
>>>>> function. 
>>>>> 
>>>>> Randy (Weidong)
>>>>> 
>>>>>> 
>>>>>> If supported by hardware, the extra page tables can even 
>be skipped
>>>>>> altogether and the device marked as having passthrough access.
>>>>>> That would give the RMRR device complete access to system memory,
>>>>>> though. 
>>>>>> 
>>>>>> eSk
>> 
>
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@xxxxxxxxxxxxxxxxxxx
>http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel