[Xen-devel] [XenSummit 2017] Notes from the PVH performance session

[Juergen Gross <jgross@xxxxxxxx>]

Hey,

I took a few notes at the PVH performance session at the summit.
I hope there isn't any major stuff missing...

Participants (at least naming the active ones): Andrew Cooper,
Jan Beulich, Paul Durrant, Roger Pau Monné and myself (the list is
just from my memory).

Following performance problems with PVH, especially when being used
as Dom0 or in driver domains, have been named to expected:

- Domain creation will be slower compared to PV Dom0, as especially
  hypercalls are much more expensive in PVH. Most calls into the
  hypervisor will result from hypercall continuations. Measurements
  with a PVH Dom0 based on BSD by Roger showed a slowdown of about
  factor 3-4 for domain creation.

- Live migration will have the same issues as domain creation,
  additionally mapping/unmapping the guest's memory will add more
  overhead.

- Backends for PV devices will suffer from worse hypercall performance
  as well, especially event channel operations, maps and unmaps have
  been named.

The following tuning options have been suggested:

- For live migration add a "mem copy" option similar to "grant copy".
  This avoids one hypercall compared to "map, copy, unmap" done today.

- For domain creation a possible solution could be a service domain
  doing the major amount of hypercalls (this service domain would be
  PV again, so Wei's idea of PV inside of a PVH container is no
  option then). Other ideas are asynchronous hypercalls (via a
  hypercall ring), but this would require some kind of service-vcpu
  in the hypervisor.
  This topic has to be discussed further.

- Backend performance could be enhanced by using "grant copy" instead
  of "map, use, unmap". OTOH this adds the need for bounce buffers.
  Depending on the backend type this might be a good idea, though.

- A general way to speed up some hypercalls might be the handling of
  hypercall parameters: for some hypercalls parameters could be passed
  in registers instead of guest memory. This would remove the need for
  walking the guest's page tables when retrieving those parameters.
  Hypercalls requiring memory parameters can be sped up by registering
  the memory buffers and just referencing those buffers when doing the
  hypercalls. The buffers could be kept mapped in the hypervisor so
  again there would be no need to walk the guest's pagetables on a hot
  path. Another possibility would be to use guest physical addresses
  as hypercall parameters.
  This should be sorted out and implemented in 4.10 IMO.

I hope I didn't forget anything.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel