RE: [Xen-devel] Dom0 hypercall for adding and removing PCI devices

["Han, Weidong" <weidong.han@xxxxxxxxx>]

VT-d spec says: Software must not modify fields other than the Present (P) 
field of currently present root-entries or context-entries. If modifications to 
other fields are required, software must first make these entries not-present 
(P=0), which requires serial invalidation of context-cache and IOTLB, and then 
transition them to present (P=1) state along with the modifications.

So your suggestion is not feasible. 

Randy (Weidong)

Keir Fraser wrote:
> Exactly my thought.
> 
>  K.
> 
> On 24/7/08 09:43, "Tian, Kevin" <kevin.tian@xxxxxxxxx> wrote:
> 
>> Isn't enough to first switch VT-d page table, and then flush IOTLB?
>> As long as RMRRs are kept same in two VT-d tables, and in any
>> time a valid entry (either in IOTLB or by walking) can be found,
>> above sequence seems complete.
>> 
>> Thanks,
>> Kevin
>> 
>>> From: Keir Fraser
>>> Sent: 2008年7月24日 16:38
>>> 
>>> Can the pagetable switch not be made atomic (i.e., so that the RMRR
>>> regions appear continuously available throughout)? I'd have thought
>>> that would just naturally happen.
>>> 
>>> If creating/destroying RMRR mappings is part of the switch
>>> operation, you'd have to destroy RMRR mappings in the dom0 table
>>> after the switch, and create RMRR mappings in the fallback table
>>> before the switch. Or just have RMRR mappings always mapped into
>>> all tables. 
>>> 
>>> -- Keir
>>> 
>>> On 24/7/08 09:32, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>> 
>>>> We found USB (has RMRR) is firstly assigned to dom0, but
>>>> pci_bus_probe() failed, then it was removed from dom0. The
>>>> removing needs switch to RMRR VT-d page table. At the same time,
>>>> BIOS was using its RMRR. 
>>>> 
>>>> Randy (Weidong)
>>>> 
>>>> Keir Fraser wrote:
>>>>> If a device is assigned to a domain (in this case dom0) then that
>>>>> domain's VT-d pagetables will contain the RMRR mappings for that
>>>>> device. Hence BIOS can perform DMA in those RMRR-indicated regions
>>>>> without swapping to and fro between domain tables and fallback
>>>>> RMRR tables. The new fallback RMRR table would be just that -- a
>>>>> fallback table used for any device not currently assigned to any
>>>>> domain (and hence those devices should only have DMAs initiated
>>>>> by the BIOS, if at all). 
>>>>> 
>>>>>  -- Keir
>>>>> 
>>>>> On 24/7/08 09:20, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>>>> 
>>>>>> I have another concern, when BIOS is initiating DMA operation
>>>>>> using RMRR, can we use RMRR VT-d page table to replace dom0 VT-d
>>>>>> page table? Does it result in some DMA failures?
>>>>>> 
>>>>>> Randy (weidong)
>>>>>> 
>>>>>> Han, Weidong wrote:
>>>>>>> Espen Skoglund wrote:
>>>>>>>> [Keir Fraser]
>>>>>>>>> On 23/7/08 10:26, "Han, Weidong" <weidong.han@xxxxxxxxx>
>>>>>>>>> wrote: 
>>>>>>>>>>> So this would be one extra VT-d pagetable, for the whole
>>>>>>>>>>> system, which would be the fallback location for RMRR
>>>>>>>>>>> mappings for devices which are currently not assigned to
>>>>>>>>>>> any domain? Thus allowing firmware to successfully initiate
>>>>>>>>>>> DMA operations on those devices? Sounds sensible.
>>>>>>>> 
>>>>>>>> Well, the VT-d page table for RMRR devices need not contain the
>>>>>>>> whole system memory---only those regions specified in the DMAR
>>>>>>>> tables. 
>>>>>>>> 
>>>>>>>>>> Is it possible that idle_domain owns the RMRR VT-d page
>>>>>>>>>> table? 
>>>>>>>> 
>>>>>>>>> If that's a convenient place to stash it then why not? Either
>>>>>>>>> way, seems you're going to have it special-cased in the code
>>>>>>>>> as fallback owner for unassigned devices. It's possible that
>>>>>>>>> having it stashed in the idle domain will simply make the
>>>>>>>>> code more confusing. I'm not sure though.
>>>>>>>> 
>>>>>>>> Right.  I don't see any particular good reason to associate it
>>>>>>>> with the idle domain.  As noted above, the page table need not
>>>>>>>> even cover the whole memory, and it will never change after
>>>>>>>> being set up at boot time.  If special case code is needed
>>>>>>>> anyway, then one might as well install a custom VT-d page
>>>>>>>> table. 
>>>>>>> 
>>>>>>> What does "custom VT-d page table" mean?
>>>>>>> 
>>>>>>> Due to domain id is not the same with DID field in context, we
>>>>>>> can reverse a DID for RMRR VT-d page table, it can avoid to
>>>>>>> associate with idle domain. 
>>>>>>> 
>>>>>>> Currently we reassign the device from dom0 to target domain when
>>>>>>> assign a device, and return the device to dom0 when target
>>>>>>> domain tears down. It's not correct due to some devices may be
>>>>>>> not assigned to any domain. Current device_assigned() also
>>>>>>> needs to be changed. Maybe it needs more changes on VT-d.
>>>>>>> 
>>>>>>> I have some concerns, maybe I missed something. Why did you use
>>>>>>> dom0 hypercall approach to replace original method? What's the
>>>>>>> main benefit? I also wonder it's suitable to wrap
>>>>>>> pci_bus_probe() function. 
>>>>>>> 
>>>>>>> Randy (Weidong)
>>>>>>> 
>>>>>>>> 
>>>>>>>> If supported by hardware, the extra page tables can even be
>>>>>>>> skipped altogether and the device marked as having passthrough
>>>>>>>> access. That would give the RMRR device complete access to
>>>>>>>> system memory, though. 
>>>>>>>> 
>>>>>>>> eSk
>>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel