Re: [Xen-devel] Xen/arm: Virtual ITS command queue handling

[Vijay Kilari <vijay.kilari@xxxxxxxxx>]

On Tue, May 5, 2015 at 7:39 PM, Julien Grall <julien.grall@xxxxxxxxxx> wrote:
> On 05/05/15 13:14, Vijay Kilari wrote:
>> Hi,
>>
>
> Hi Vijay,
>
>>    As discussed, here is the design doc/txt.
>
> I will comment on the proposal 2 as it seems to be the preferred one
> assuming you are able to find why it's slow.
>
>> Proposal 2:
>> ----------------
>> Here when guest writes command to vITS queue and updates CWRITER registers,
>> it is trapped in XEN and below steps are followed to process ITS command
>>
>> - Dom0 creates a ITS completion device with device id (00:00.1) and reserves
>>   n number (256 or so) irqs (LPIs) for this device.
>> - One irq/LPI (called completion_irq) of this completion device is
>> allocated per domain
>> - With this irq/LPI descriptor we can identify the domain/vITS.
>> - Info of all the ongoing ITS requests(put in pITS Queue) of this domain is
>>   stored in ITS command status array (called its_requests). This is
>> managed per vITS.
>>
>> 1) Trap of CWRITER write by guest
>> 2) Take vITS lock
>> 3) Read all the commands written by guest, translate it
>>     - If one of the guest command is INT command
>
> Why do you need a specific handling for the guest INT command?

  If guest driver is using interrupt mechanism instead of polling
then INT command is passed by guest. To make sure that CREADER is updated
before INT command raises interrupt to guest, Xen has to insert completion
interrupt and update CREADER

>>        a) Append INT command with completion_irq and write this batch as
>>           seperate request and goto (3) to process next commands
>>     - If more than 'n' commands are sent by guest, start a timer to process
>>       remaining commands
>
> Hmmm... How are you sure the time for the timer would be enough?
>
   Not thought of how much time. May be the number of pending
   commands in physical queue might give some hueristic on timer value.

>> 4) Append INT command with completion_irq of current domain
>> 5) Release vITS lock
>> 6) Take physical ITS (pITS) lock
>> 7) Write translated cmds to physical ITS
>> 8) Add entry in its_requests
>
> You don't explain what is its_requests.
>
>> 9) Release pITS lock
>> 10) return from trap
>>
>> One receiving completion interrupt:
>>
>> 1) Take the first pending request from its_requests.
>
> I'm assuming that you have some kind of array/list to store the pending
> request? I think this would be more difficult to manage than only
> supporting one batch per domain at any time.

  Yes, If only one batch per domain is processed at a time,
then the array could store only one entry. I will tune it when I implement

>> 2) Update vITS CREADER of the guest indicating completion of command to guest
>>
>> Cons:
>>    - Has overhead of processing completion interrupt.
>>    - Need to reserve a fake device to generate completion interrupt and
>>      reserve one LPI per-domain
>>
>> Pros:
>>    - VCPU does not poll in Xen for completion of commands.
>>    - Handles guest flooding command queue with commands. But needs timer
>>
>> Handling Command queue state:
>>  - Physical Queue cannot be full as it 64KB there by it can accomodate
>> 1K ITS commands.
>
> I don't understand this sentence. Why do you think the physical queue
> cannot be full?

  I mean that it is unlikely that physical ITS command Q would be full
because of 64KB size. If at all if it full then below action is taken

>
>>    In case it is full, VCPU has to poll with timeout till physical
>> Queue is empty before it post
>>    next command
>>  - If vITS Queue condition should be managed by guest ITS driver.
>
> Same here.

    vITS Queue is under guest control. If Xen is processing commands slowly
and if guest sees its queue is full then guest driver will handle it.

>
>> Behaviour of Polling and completion interrupt based guest driver:
>>  - If completion interrupt (INT) is used by guest driver, then insert
>> Xen completion
>>    INT command so that CREADER is updated before guest's INT command is 
>> injected
>>  - If polling mode is used, trap on CREADER checks for completion of command
>>
>
> Regards,
>
> --
> Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel