Re: [Xen-devel] [Draft F] Xen on ARM vITS Handling

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Fri, 2015-06-12 at 09:14 -0400, Julien Grall wrote:
> 
> On 12/06/2015 08:55, Ian Campbell wrote:
> > On Thu, 2015-06-11 at 10:40 +0100, Ian Campbell wrote:
> >> ## Command Queue Virtualisation
> >>
> >> The command translation/emulation in this design has been arranged to
> >> be as cheap as possible (e.g. in many cases the actions are NOPs),
> >> avoiding previous concerns about the length of time which an emulated
> >> write to a `CWRITER` register may block the vcpu.
> >>
> >> The vits will simply track its reader and writer pointers. On write
> >> to `CWRITER` it will immediately and synchronously process all
> >> commands in the queue and update its state accordingly.
> >>
> >> It might be possible to implement a rudimentary form of preemption by
> >> periodically (as determined by `hypercall_preempt_check()`) returning
> >> to the guest without incrementing PC but with updated internal
> >> `CREADR` state, meaning it will reexecute the write to `CWRITER` and
> >> we can pickup where we left off for another iteration. This at least
> >> lets us schedule other vcpus etc and prevents a monopoly.
> >
> > In the presence of multiple VCPUs writing to GITS_CWRITER preemption
> > actually gets pretty subtle. I suggest leaving it out for now.
> 
> Would it be possible to do it with re-doing the write to the GITS_CWRITER?

Not easily.

For example one subtle case is:

      * VCPUA writes W1 to GITS_CWRITER, processes a proportion and is
        preempted.
      * VCPUB writes W2 (>W1) GITS_CWRITER W2 and processes a proportion
        such that W1 < GITS_CREADR < W2, is preempted.
      * VCPUA resumes, sees GITS_CREADR != W1, processes up to W2, and
        then keeps going processing junk until it wraps around to W1
        again.

The != is necessary because the command queue is a circular buffer, so
GITS_CREADR < GITS_CWRITER is not a reliable way to determine if there
is anything on the ring.

Another case I haven't decided if I'm concerned about yet is an OS which
issues two writes to GITS_CWRITER in what it things is the proper order
(maybe it's using barriers or atmomic variables or something to try and
ensure this) but the trap for the first takes longer to get to the
handler than the second (maybe it takes an interrupt on the way in, or
more cache misses etc). This may or may not be an issue, I've not
decided yet (mainly because I haven't decided if that OS behaviour is
valid or not).

Unlike with proper hypercall preemption we do not have the ability to
record progress in the hypercall arguments, nor to indicate that a
particular call is a continuation rather than a fresh call, both of
which make this harder to deal with. It's possible something could be
constructed with some per-vcpu data, I'm still thinking it through.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel