[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v4 02/11] vpci: cancel pending map/unmap on vpci removal


  • To: Oleksandr Andrushchenko <andr2000@xxxxxxxxx>, roger.pau@xxxxxxxxxx
  • From: Jan Beulich <jbeulich@xxxxxxxx>
  • Date: Wed, 17 Nov 2021 09:28:34 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NZqmx/f9SxWx6HKq+SKbz6qRkXxJuCzX6FjMoUHU+TI=; b=gLyiLwGbbQ5pvBo0kvP5jdbKM2Si7NBi3oskhaRMlgqu+5Kttp1pNteCkRHQ47b/UXWwA7is2S2/9e1yniBRl6ZlQgHIBVPVGAURwLLUHmfnPX+RK5xXchYxI/o2Jua7RWvlit8kw2PKJZ6EHjOUSs9oNhTpY2tFA8EYR1QIfky6XU3oeU3TTj2XNRd+7a1DWDmB01nmKEmFdYVFQR6S9vIdsMNSkDEhLUIGeGJE4XnLJ5bPTbRLO5T9U60i4X62j4sl1GM7hMc9Sw++39CM8WcSIY33TiR+8a1s6kvad5JmkcTXXkoqR1OixKwBLRtmKuI781aadi6Ou2JKsE9f2A==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LT6tPDzLpEvX1a1N9Q6UFRxdqTNoppdDeFVOsHMc6wyNhtpiky7XijKy3Fwka/tCDXkVxLdKh8x9fea05s5GveANO+HkQvzMUwRKEUjnRJhmAAjy6E7P4CQaf1/AX4DfsOMueoo9ST8QJiv1V6Fgbl0E8OpeUvpvawNDb9OV2l3QLz4W5Tlp+pDF4WmlJx4MNRcMo6q1yjhiZE77XJCSDf9gB6N8L0udQa8r83GLetpCF7ttc5lf7CRfx6UzFYjx670Q+9bHJR89qTT9h81YhV2uTv22odE39bvZIW55+nNP0bKFIuwDFhSU/QA5IAp8is0UHYADygFjUltSMuT43g==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com;
  • Cc: julien@xxxxxxx, sstabellini@xxxxxxxxxx, oleksandr_tyshchenko@xxxxxxxx, volodymyr_babchuk@xxxxxxxx, Artem_Mygaiev@xxxxxxxx, andrew.cooper3@xxxxxxxxxx, george.dunlap@xxxxxxxxxx, paul@xxxxxxx, bertrand.marquis@xxxxxxx, rahul.singh@xxxxxxx, Oleksandr Andrushchenko <oleksandr_andrushchenko@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxxx
  • Delivery-date: Wed, 17 Nov 2021 08:28:57 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 05.11.2021 07:56, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@xxxxxxxx>
> 
> When a vPCI is removed for a PCI device it is possible that we have
> scheduled a delayed work for map/unmap operations for that device.
> For example, the following scenario can illustrate the problem:
> 
> pci_physdev_op
>    pci_add_device
>        init_bars -> modify_bars -> defer_map -> 
> raise_softirq(SCHEDULE_SOFTIRQ)
>    iommu_add_device <- FAILS
>    vpci_remove_device -> xfree(pdev->vpci)
> 
> leave_hypervisor_to_guest
>    vpci_process_pending: v->vpci.mem != NULL; v->vpci.pdev->vpci == NULL
> 
> For the hardware domain we continue execution as the worse that
> could happen is that MMIO mappings are left in place when the
> device has been deassigned
> 
> For unprivileged domains that get a failure in the middle of a vPCI
> {un}map operation we need to destroy them, as we don't know in which
> state the p2m is. This can only happen in vpci_process_pending for
> DomUs as they won't be allowed to call pci_add_device.
> 
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@xxxxxxxx>

Thinking about it some more, I'm not convinced any of this is really
needed in the presented form. Removal of a vPCI device is the analogue
of hot-unplug on baremetal. That's not a "behind the backs of
everything" operation. Instead the host admin has to prepare the
device for removal, which will result in it being quiescent (which in
particular means no BAR adjustments anymore). The act of removing the
device from the system has as its virtual counterpart "xl pci-detach".
I think it ought to be in this context when pending requests get
drained, and an indicator be set that no further changes to that
device are permitted. This would mean invoking from
vpci_deassign_device() as added by patch 4, not from
vpci_remove_device(). This would yield removal of a device from the
host being independent of removal of a device from a guest.

The need for vpci_remove_device() seems questionable in the first
place: Even for hot-unplug on the host it may be better to require a
pci-detach from (PVH) Dom0 before the actual device removal. This
would involve an adjustment to the de-assignment logic for the case
of no quarantining: We'd need to make sure explicit de-assignment
from Dom0 actually removes the device from there; right now
de-assignment assumes "from DomU" and "to Dom0 or DomIO" (depending
on quarantining mode).

Thoughts?

Jan




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.