Re: [Xen-devel] [Minios-devel] [PATCH v4 0/<VARIOUS>] Begin to disentangle libxenctrl and provide some stable libraries

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

(Trimming CCs a bit)

On Wed, 2015-10-21 at 16:22 +0100, Ian Campbell wrote:
> 
[...]
> Still to come would be libraries for specific out of tree purposes
> (device model, kexec), which would be adding new library at the same
> level as libxc I think, rather than underneath, i.e. also using the
> libraries split out here, but hopefully not libxenctrl itself.

Next steps for qemu-xen, with reference to:

http://xenbits.xen.org/people/ianc/libxenctrl-split/v4.html#by-functional-area

First the two simple cases:

The PV backend support now only (I think) uses got from these new libraries.

The PV domain builder is now configured off by default, I don't intend to
make this
use a stable API so when enabling this qemu will then be linked against the
unstable
libxen{guest,ctrl} libraries.

Now the more complex cases.

The actual DM functionality looks in reasonably good shape, from the
pandoc
source to the above (the "S" columns represent whether the column to the
left is a stable interface or not):

Interface                          S Underlying Interface           S Other 
users
---------------------------------- - ------------------------------ - 
---------------
`xenevtchn_bind_interdomain`       Y
`xenevtchn_close`                  Y
`xenevtchn_fd`                     Y
`xenevtchn_notify`                 Y
`xenevtchn_open`                   Y
`xenevtchn_pending`                Y
`xenevtchn_unmask`                 Y
`xenforeignmemory_map`             Y
`xenforeignmemory_open`            Y
`xenforeignmemory_unmap`           Y
`xc_domain_add_to_physmap`         N `XENMEM_add_to_physmap`        Y libxc 
(dombuilder)
`xc_domain_populate_physmap_exact` N `XENMEM_populate_physmap`      Y libxc 
(several)
`xc_domain_pin_memory_cacheattr`   N `XEN_DOMCTL_pin_mem_cacheattr` N None
`xc_domain_shutdown`               N `SCHEDOP_remote_shutdown`      Y libxl
`xc_set_hvm_param`                 N `HVM_PARAM_ACPI_S_STATE`       Y None
`xc_hvm_inject_msi`                N `HVMOP_inject_msi`             Y None
`xc_hvm_modified_memory`           N `HVMOP_modified_memory`        Y None
`xc_hvm_set_isa_irq_level`         N `HVMOP_set_isa_irq_level`      Y None
`xc_hvm_set_mem_type`              N `HVMOP_set_mem_type`           Y None
`xc_hvm_set_pci_intx_level`        N `HVMOP_set_pci_intx_level`     Y None
`xc_hvm_set_pci_link_route`        N `HVMOP_set_pci_link_route`     Y None
`xc_hvm_track_dirty_vram`          N `HVMOP_track_dirty_vram`       Y None
`xc_hvm_unmap_io_range_from_ir...` N `HVMOP_IO_RANGE_(PORT|MEM...)` Y None
`xc_hvm_unmap_pcidev_from_iore...` N `HVMOP_unmap_io_range_from...` Y None

I think it would be reasonable to add a new library (say, libxendevicemodel, 
for arguments sake) with a stable ABI and to move all of the xc_hvm_* above 
into it. They are not used for anything else and are based on HVMOP which is a 
stable interface (AFAIK).

Within qemu xc_domain_add_to_physmap and xc_domain_pin_memory_cacheattr are
used in tandem solely to populate VRAM with WB memory and to remove again.

xc_domain_populate_physmap_exact is used only to populate RAM and
xc_domain_shutdown is used on shutdown.

I think having three or four functions in libxendevicemodel which offer
these exact facilities while retaining the underlying (possibly more
flexible) functionality in libxenctrl for other users (dombuilder, other in
tree tools) would be tolerable.

Three or four functions depends on whether the uses of
xc_domain_add_to_physmap+xc_domain_pin_memory_cacheattr and
xc_domain_populate_physmap_exact can be satisfied with a single API or not.
I don't know that yet.

The main question would then be whether libxendevicemodel should wrap
libxenctrl in a stable layer or whether it should actually duplicate the
functionality in the thin wrappers.

Wrapping libxenctrl would mean that libxendevicemodel would need to be
built by all releases with a fixed ABI such that the corresponding one
(linking against the correct libxenctrl) can be flipped into place on boot.
I think that is going to be too tricky for distros and users a like and
that the small amount of duplication is therefore more tolerable.

The only blocker for that is that xc_domain_pin_memory_cacheattr is
XEN_DOMCTL_pin_mem_cacheattr (i.e. not stable), but this route would either
require it to move to a stable hypercall interface or for the library to
DTRT for all versions. I think moving to a stable h/call interface is more
feasible.

In terms of deprivileging QEMU (one of the end goals of this work) I think
all the xen* libraries could very easily gain a xen???_set_target_domain
which would make an appropriate ioctl to lock the underlying fd into
operating only on that domain (another argument in finding a way to do
pin_mem_cacheattr as a stable API). This would require support from all the
underlying drivers, but could be added right away with errno=ENOTTY +
return -1 and then plumbed in later. Obviously qemu would need to check the
return values.

The PCI passthrough case is less clear:

Interface                          S Underlying Interface           S Other 
users
---------------------------------- - ------------------------------ - 
---------------
`xc_domain_bind_pt_pci_irq`        N `XEN_DOMCTL_bind_pt_irq`       N None
`xc_domain_ioport_mapping`         N `XEN_DOMCTL_ioport_mapping`    N None
`xc_domain_memory_mapping`         N `XEN_DOMCTL_memory_mapping`    N libxl
`xc_domain_unbind_msi_irq`         N `XEN_DOMCTL_unbind_pt_irq`     N None
`xc_domain_unbind_pt_irq`          N `XEN_DOMCTL_unbind_pt_irq`     N None
`xc_domain_update_msi_irq`         N `XEN_DOMCTL_bind_pt_irq`       N None
`xc_physdev_map_pirq`              N `PHYSDEVOP_map_pirq`           Y libxl
`xc_physdev_map_pirq_msi`          N `PHYSDEVOP_map_pirq`           Y None
`xc_physdev_unmap_pirq`            N `PHYSDEVOP_unmap_pirq`         Y libxl

NB: More might be used by libxl in the future e.g. for ARM or PVH
passthrough?

It seems like much of this would be candidates for adding to
libxendevicemodel, but the underlying unstable interfaces pose a problem
there. I'm going to leave this for another day.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel