[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH RFC] vPCI: account for hidden devices in modify_bars()

  • To: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • From: Jan Beulich <jbeulich@xxxxxxxx>
  • Date: Tue, 14 Sep 2021 14:05:01 +0200
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=vjJDOYTtIahsSojHg6DggSb0G+WVqbSXp5whCbCigC0=; b=Fn3T7OLuF2J+XTKbXSN/9vOEeA19AklD3R/9aMxbwt9Avbpk65ZlIXZgAb7sO/Pi3GyBpJ02rHj0oxl+EwFGFFgqtRdy7zrHWZqX0FqHIhKhV4bKDdS/iu6xbHGcQY+009WsMOUKtstBM+x0y7CKnPR0eR5mN7rWkPoG4W/uMYM7sjrVDJnWepOe7Pdh5rIVL3dT6nhbhs0Pa+TDSp4yDMC6TedN2vcNORVHaoOYkDEcB+M6QyyuwX7NSPfBMG18iC2q2CAVCoysQmtcj/BCTb8SPE7qkoK+b12aZGxQIiQrr7yULuoWUZrlThVQruVb+XFoDa6qfbyPD8jkBdV2Jg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qxc9L2XgCI0K+84fi/NnCVJVu2SxW7Zb8r7TH+HEXozed/0RZOYz13U4/KVgg83ahQR3Bo0Kqm5td8ctVdFf+XaZ3dEJRzuR9JabfQGKZrb+dsIOUiuJqbz8J+u18uuu0fJLop2BKCWuDfnoE4TAAf1qO8UD9ZNBIRVKT42SnlP6v9CiwB5fI6dcVejEe7qM4J/+zztqNRxW3RK6/+w1p3lAk340xr2luLwUBPGgVvM37S4Ac7fMu3PyXuSDYqBvYhpa8YRT36/g1MDfCBoSRc5JrFowM6Z8lHISYX5uOI5iz2fEFEddmOPrrQVOj7uQ8y4rlK62ehPBQGE9WyPDJg==
  • Authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=suse.com;
  • Cc: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Tue, 14 Sep 2021 12:05:14 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 14.09.2021 13:21, Roger Pau Monné wrote:
> On Tue, Sep 14, 2021 at 12:12:12PM +0200, Jan Beulich wrote:
>> On 14.09.2021 12:00, Roger Pau Monné wrote:
>>> On Mon, Aug 30, 2021 at 03:04:55PM +0200, Jan Beulich wrote:
>>>> Hidden devices (e.g. an add-in PCI serial card used for Xen's serial
>>>> console) are associated with DomXEN, not Dom0. This means that while
>>>> looking for overlapping BARs such devices cannot be found on Dom0's
>>>> list of devices; DomXEN's list also needs to be scanned.
>>> Thanks for looking into this, I certainly didn't take hidden devices
>>> into account for vPCI dom0.
>>>> Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
>>>> ---
>>>> RFC: Patch intentionally mis-formatted, as the necessary re-indentation
>>>>      would make the diff difficult to read. At this point I'd merely
>>>>      like to gather input towards possible better approaches to solve
>>>>      the issue (not the least because quite possibly there are further
>>>>      places needing changing).
>>> I have a couple of questions, AFAICT we currently hide the serial
>>> console and/or the VGA adapter if it's in use by Xen.
>>> I wonder whether we need to add vPCI handlers for those:
>>> setup_one_hwdom_device will attempt to add vPCI handlers to the hidden
>>> device because of the temporary override of pdev->domain done in
>>> _setup_hwdom_pci_devices, but I think that for hidden devices we
>>> shouldn't add vPCI handlers. We should instead block PCI config space
>>> access from dom0 to the device so that it doesn't mess with Xen usage.
>> The answer to this follows (I think) from the one below.
>>> It's also not clear why does Xen want to have those hidden devices
>>> partly controlled by dom0, as it would seem to me that dom0 interfering
>>> with hidden devices in use by Xen can only lead to trouble.
>> Dom0 can't interfere as long as it can only read from the device.
>> Restricting accesses to reads is one of the purposes of "hiding"
>> (the other is to make it impossible to assign these to a DomU). Not
>> allowing Dom0 to read from such devices would lead to wrong PCI
>> device discovery - some devices would be missing (which may or may
>> not be merely a cosmetic issue). If the missing device is a multi-
>> function one at function 0, other devices in the same slot may also
>> not be found by Dom0 (depending on whether it looks at non-zero
>> function numbers in this case).
> Hm, indeed seems possible that missing function 0 the whole device is
> skipped.
> Maybe we need a special vPCI handling for those devices that just
> allows reads but not writes, and that doesn't maps the BARs into dom0
> p2m?

Not sure about mapping. They could be mapped r/o. And they may
actually need mapping for multi-function devices, but I guess for
such devices to actually function properly then there would be
more work required.

> Or could the BARs potentially be shared between multiple
> functions on the same device?

I don't think that's allowed (or would even work) - if anything
(like in general) small BARs may share a page, but without overlaps.

> This also makes me wonder why they have to be added to the IOMMU, is
> that related to device groups and the fact that Xen is not clever
> enough to figure whether devices belong to the same IOMMU group and
> thus must be assigned together?

I'm actually not sure whether they need to be added to the IOMMU.
With multi-function devices in mind, back at the time I though it
would make more sense that way. But I may have been wrong, and
hence we may want to revisit this. Problem with such changes is
that they may cause rare issues on very specific systems, which we
may have no way noticing even during a full release cycle.




Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.