Re: [Xen-devel] FW: VT-d async invalidation for Device-TLB.

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 16.06.15 at 05:07, <yang.z.zhang@xxxxxxxxx> wrote:
> Jan Beulich wrote on 2015-06-15:
>>>>> On 13.06.15 at 16:44, <quan.xu@xxxxxxxxx> wrote:
>>>> On 12.06.15 at 14:47, <JBeulich@xxxxxxxx> wrote:
>>>>>>> On 12.06.15 at 04:40, <quan.xu@xxxxxxxxx> wrote:
>>>>> I tested it by a Myri-10G Dual-Protocol NIC, which is an ATS device.
>>>>> for an invalidation:
>>>>>  By sync way, it takes about 1.4 ms.
>>>>>  By async way, it takes about 4.3 ms.
> 
> It is a typo for the time: should be 1.4 us and 4.3 us.
> 
>>>> 
>>>> What's the theory on why this is? After all, it shouldn't matter
>>>> how the completion of the invalidation gets signaled.
>>>> 
>>> 
>>> Let me introduce more about how I get these test data.
>>> 
>>> For sync way, I get the time by NOW() macro, when I update Queue
>>> Tail Register  (qinval_update_qtail()).
>>> Then get time by NOW() macro in current spinning when it has wrote
>>> the value in the Status Data field to the address specified in the Status 
>>> Address.
>>> The difference of these 2 value is the time of an sync invalidation.
>>> 
>>> 
>>> For async way, as the previous email, I have introduced the IF bit
>>> of Invalidation Wait Descriptor.
>>> (IF: Indicate the invalidation wait descriptor completion by
>>> generating an invalidation completion event per the programing of
>>> the Invalidation Completion Event Registers.) I have implemented an
>>> interrupt for invalidation completion event.
>>> Also I get the time by NOW() macro, when I update Queue Tail
>>> Register (by qinval_update_qtail()).
>>> Then get time by NOW() macro in invalidation completion event handler.
>>> The difference of these 2 value is the time of an async invalidation.
>> 
>> Okay, thanks for the explanation. As this includes the time it takes
>> to deliver and (partly) handle the interrupt, the difference is of
>> course within what one would expect (and also of what would seem 
> acceptable).
> 
> The time doesn't include the cost of handling of interrupt. We just record 
> it at the entry of interrupt handler. So the cost should bigger than 4.3 us 
> if taking the handing cost into consideration. And the costs will much bigger 
> if there are more pass-through VMs runs. We can start from ATS case firstly. 
> And apply it to non-ATS case later if the async approach doesn't hurt the 
> performance.

In which case we're back to the question I raised originally: How do
you explain the time difference if the interrupt delivery overhead
isn't included?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel