Re: [Xen-devel] [Minios-devel] [PATCH v4 0/<VARIOUS>] Begin to disentangle libxenctrl and provide some stable libraries

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Wed, 21 Oct 2015, Ian Campbell wrote:
> (Trimming CCs a bit)
> 
> On Wed, 2015-10-21 at 16:22 +0100, Ian Campbell wrote:
> > 
> [...]
> > Still to come would be libraries for specific out of tree purposes
> > (device model, kexec), which would be adding new library at the same
> > level as libxc I think, rather than underneath, i.e. also using the
> > libraries split out here, but hopefully not libxenctrl itself.
> 
> Next steps for qemu-xen, with reference to:
> 
> http://xenbits.xen.org/people/ianc/libxenctrl-split/v4.html#by-functional-area
> 
> First the two simple cases:
> 
> The PV backend support now only (I think) uses got from these new libraries.
> 
> The PV domain builder is now configured off by default, I don't intend to
> make this
> use a stable API so when enabling this qemu will then be linked against the
> unstable
> libxen{guest,ctrl} libraries.
> 
> Now the more complex cases.
> 
> The actual DM functionality looks in reasonably good shape, from the
> pandoc
> source to the above (the "S" columns represent whether the column to the
> left is a stable interface or not):
> 
> Interface                          S Underlying Interface           S Other 
> users
> ---------------------------------- - ------------------------------ - 
> ---------------
> `xenevtchn_bind_interdomain`       Y
> `xenevtchn_close`                  Y
> `xenevtchn_fd`                     Y
> `xenevtchn_notify`                 Y
> `xenevtchn_open`                   Y
> `xenevtchn_pending`                Y
> `xenevtchn_unmask`                 Y
> `xenforeignmemory_map`             Y
> `xenforeignmemory_open`            Y
> `xenforeignmemory_unmap`           Y
> `xc_domain_add_to_physmap`         N `XENMEM_add_to_physmap`        Y libxc 
> (dombuilder)
> `xc_domain_populate_physmap_exact` N `XENMEM_populate_physmap`      Y libxc 
> (several)
> `xc_domain_pin_memory_cacheattr`   N `XEN_DOMCTL_pin_mem_cacheattr` N None
> `xc_domain_shutdown`               N `SCHEDOP_remote_shutdown`      Y libxl
> `xc_set_hvm_param`                 N `HVM_PARAM_ACPI_S_STATE`       Y None
> `xc_hvm_inject_msi`                N `HVMOP_inject_msi`             Y None
> `xc_hvm_modified_memory`           N `HVMOP_modified_memory`        Y None
> `xc_hvm_set_isa_irq_level`         N `HVMOP_set_isa_irq_level`      Y None
> `xc_hvm_set_mem_type`              N `HVMOP_set_mem_type`           Y None
> `xc_hvm_set_pci_intx_level`        N `HVMOP_set_pci_intx_level`     Y None
> `xc_hvm_set_pci_link_route`        N `HVMOP_set_pci_link_route`     Y None
> `xc_hvm_track_dirty_vram`          N `HVMOP_track_dirty_vram`       Y None
> `xc_hvm_unmap_io_range_from_ir...` N `HVMOP_IO_RANGE_(PORT|MEM...)` Y None
> `xc_hvm_unmap_pcidev_from_iore...` N `HVMOP_unmap_io_range_from...` Y None
> 
> I think it would be reasonable to add a new library (say, libxendevicemodel, 
> for arguments sake) with a stable ABI and to move all of the xc_hvm_* above 
> into it. They are not used for anything else and are based on HVMOP which is 
> a stable interface (AFAIK).
> 
> Within qemu xc_domain_add_to_physmap and xc_domain_pin_memory_cacheattr are
> used in tandem solely to populate VRAM with WB memory and to remove again.
> 
> xc_domain_populate_physmap_exact is used only to populate RAM and
> xc_domain_shutdown is used on shutdown.
> 
> I think having three or four functions in libxendevicemodel which offer
> these exact facilities while retaining the underlying (possibly more
> flexible) functionality in libxenctrl for other users (dombuilder, other in
> tree tools) would be tolerable.
> 
> Three or four functions depends on whether the uses of
> xc_domain_add_to_physmap+xc_domain_pin_memory_cacheattr and
> xc_domain_populate_physmap_exact can be satisfied with a single API or not.
> I don't know that yet.
> 
> The main question would then be whether libxendevicemodel should wrap
> libxenctrl in a stable layer or whether it should actually duplicate the
> functionality in the thin wrappers.

Couldn't we rename the existing libxc calls and make them stable?  Keep
in mind that we already have stable wrappers in QEMU in
include/hw/xen/xen_common.h. I don't think we need two layers of
wrappers :-)


> Wrapping libxenctrl would mean that libxendevicemodel would need to be
> built by all releases with a fixed ABI such that the corresponding one
> (linking against the correct libxenctrl) can be flipped into place on boot.
> I think that is going to be too tricky for distros and users a like and
> that the small amount of duplication is therefore more tolerable.

I am not in favor of wrapping, but I don't think that what you describe
is a problem, unless I am misunderstanding.  libxendevicemodel and
libxenctrl are built together from the same place.  We don't really
expect users to mix and match them.  Even if libxendevicemodel was
wrapping libxenctrl, it wouldn't really be a problem, given that they
would always be updated together.
We do expect users to mix and match QEMU and Xen, and that is the
problem that libxendevicemodel would solve.


> The only blocker for that is that xc_domain_pin_memory_cacheattr is
> XEN_DOMCTL_pin_mem_cacheattr (i.e. not stable), but this route would either
> require it to move to a stable hypercall interface or for the library to
> DTRT for all versions. I think moving to a stable h/call interface is more
> feasible.
> 
> In terms of deprivileging QEMU (one of the end goals of this work) I think
> all the xen* libraries could very easily gain a xen???_set_target_domain
> which would make an appropriate ioctl to lock the underlying fd into
> operating only on that domain (another argument in finding a way to do
> pin_mem_cacheattr as a stable API). This would require support from all the
> underlying drivers, but could be added right away with errno=ENOTTY +
> return -1 and then plumbed in later. Obviously qemu would need to check the
> return values.

+1


> The PCI passthrough case is less clear:
> 
> Interface                          S Underlying Interface           S Other 
> users
> ---------------------------------- - ------------------------------ - 
> ---------------
> `xc_domain_bind_pt_pci_irq`        N `XEN_DOMCTL_bind_pt_irq`       N None
> `xc_domain_ioport_mapping`         N `XEN_DOMCTL_ioport_mapping`    N None
> `xc_domain_memory_mapping`         N `XEN_DOMCTL_memory_mapping`    N libxl
> `xc_domain_unbind_msi_irq`         N `XEN_DOMCTL_unbind_pt_irq`     N None
> `xc_domain_unbind_pt_irq`          N `XEN_DOMCTL_unbind_pt_irq`     N None
> `xc_domain_update_msi_irq`         N `XEN_DOMCTL_bind_pt_irq`       N None
> `xc_physdev_map_pirq`              N `PHYSDEVOP_map_pirq`           Y libxl
> `xc_physdev_map_pirq_msi`          N `PHYSDEVOP_map_pirq`           Y None
> `xc_physdev_unmap_pirq`            N `PHYSDEVOP_unmap_pirq`         Y libxl
> 
> NB: More might be used by libxl in the future e.g. for ARM or PVH
> passthrough?
> 
> It seems like much of this would be candidates for adding to
> libxendevicemodel, but the underlying unstable interfaces pose a problem
> there. I'm going to leave this for another day.

Does the Xen<->libxc interface need to be stable for libxendevicemodel
to have a stable ABI?
I think that libxendevicemodel could be stable, no matter what hypercall
is eventually called underneath. Am I missing something?

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel