[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [RFC PATCH 1/3] xen: Introduce "gpaddr_bits" field to XEN_SYSCTL_physinfo

  • To: Oleksandr Tyshchenko <olekstysh@xxxxxxxxx>, <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • Date: Tue, 7 Sep 2021 18:35:47 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=GsjaAJfgrCNlIL34Yd1ZnN8XhjsnG6kdA+YSA8g09jA=; b=X51eO0bU1D5ZcbYB2zRrRkfg6zzGrgHoM9AyyCVWlNpFbRT9qtBIqyDk1JhfZuDH8nMrqpjornwS4KwjMwofGrBAELdvC1ywcwhwjFTptq7MCyb8U/afX7DnuHFiY8YzNQCrctBTzzWMsvzet85RngdlOW2wMG1PMCL2xEZJGyYFXbXQhDxbNS7dNqi9JA9oOs/yAhx9KQK6j1fhJ/OC/KU12EXqenZ4ZZ90qR3hUYQiUZf9f59B7RH5API2ck8PI1IuesWxH6O0XAm6m7hFKtGJ5y0HdqwbOymzehiPo0zPm7AUzxh6tl+ZoGsgKb484cJP/3mMkwKiboxyjznT1Q==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h2kVk7IR4CL6JHvSHeBqgj7uXWQIFar8/3EJ4/kHNDSFAePmw3Oj4bI/4MH/ufqBRRFrsZWxWJBksQuRau3ZappG/N9KkG8XnSkJORDiCE2ldilCu+JD9VHyaq+HY8t4EEERDVyVYDwdmjbSE/wFh/b54vJKMvydBrCQP0hXl8+eut2d+FERIrWnukC5OJd8lZsXL/0911FS640w34VYFFX3ExSQzoJRK1kJlD6LSXk1k+218E3Jth2WYKf0wirGR8JQiTR6KKehLwjGxWliZTBY4lJJ4Cpe8uKsLP0JzB81c9jyATCWbzqwOBnK+xlrYkL+W/Lwk1EdE8Z5trOZDw==
  • Authentication-results: esa4.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
  • Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>, Ian Jackson <iwj@xxxxxxxxxxxxxx>, Wei Liu <wl@xxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Juergen Gross <jgross@xxxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, Roger Pau Monné <roger.pau@xxxxxxxxxx>, Bertrand Marquis <bertrand.marquis@xxxxxxx>, Wei Chen <Wei.Chen@xxxxxxx>, Henry Wang <Henry.Wang@xxxxxxx>
  • Delivery-date: Tue, 07 Sep 2021 17:36:17 +0000
  • Ironport-hdrordr: A9a23:IhhBFaoFt/emnMtespdSRMIaV5u8L9V00zEX/kB9WHVpm5Oj+P xGzc526farslsssREb+OxpOMG7MBbhHO1OkPYs1NCZLXXbUQqTXfxfBO7ZrQEIdBeOjtK1uZ 0QFJSWTeeAd2SS7vyKkDVQcexQueVvmZrA7Yy1rwYPPHJXguNbnmNE426gYzxLrWJ9dPwE/f Snl6h6TnabCA8qhpPRPAh6YwGPnayGqLvWJTo9QzI34giHij2lrJb8Dhijxx8bFxdC260r/2 TpmxHwovzLiYD09jbsk0voq7hGktrozdVOQOSKl8guMz3pziKlfp5oVbGutC085Muv9FEput /RpApIBbU911rhOkWO5Tf90Qjp1zgjr1fk1F+jmHPm5ff0QTorYvAxzr5xQ1/80Q4Nrdt82K VE0yayrJxMFy7Nmyz7+pzhSwxqvlDcmwtgrccjy1hkFacOYr5YqoISuGlPFo0bIS784Ic7VM FzEcDn4upMe1/yVQGYgoBW+q3oYp0PJGbDfqBb0fbllAS+3UoJjnfw/fZv3Evpr/kGOt95D+ etCNUhqFgBdL5OUUrRbN1xNvdfMVa9NC4kBljiaGgPJJt3SU4llKSHlIndxNvaMqDgn6FC1a gobjtjxBgPkgTVeJWz4KE=
  • Ironport-sdr: zB0VZmyGTwWt9Gp2YY0ZTcUaN4I7s/9xxEWc7AgJRZaxRmpIWhQMJuigtR8tFbt7NGDlCQix9V Yl9sg4DS7rpdvBt2YyErcykB2PvKaQO5Ip5x4TwO05QXEwkcwam6akdwwmhqUMa0Bszih1rhJn T3e0IbaYBEu13hOeI/RzW1IC0aBLed6LM7MXX9Bt/tQYVORXhTmjMmMdK1P+AacTL/eASdr6vA 0QzFo9cBMqu1B7kuLXXNm5gIAharup2oiII8tEDWzCrW4j392qA7GZ4pfrtM00LDy04Ge0z0vs 3SVpeL6365z04GXdqiR9lzP+
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 07/09/2021 18:09, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>
> We need to pass info about maximum supported guest address
> space size to the toolstack on Arm in order to properly
> calculate the base and size of the extended region (safe range)
> for the guest. The extended region is unused address space which
> could be safely used by domain for foreign/grant mappings on Arm.
> The extended region itself will be handled by the subsequents
> patch.
> Use p2m_ipa_bits variable on Arm, the x86 equivalent is
> hap_paddr_bits.
> As we change the size of structure bump the interface version.
> Suggested-by: Julien Grall <jgrall@xxxxxxxxxx>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>

So while I think this is a suitable way forward, you're painting
yourself into a corner WRT migration.

On x86, the correct value is d->arch.cpuid->extd.maxphysaddr and this
value is under toolstack control, not Xen control.  In particular, it
needs to be min(hostA, hostB) to make migration safe, and yes - this is
currently a hole in x86's migration logic that will cause large VMs to

The same will be true on ARM as/when you gain migration support.

I think this would be better as a domctl.  On ARM, it can reference
p2m_ipa_bits for now along with a /* TODO - make per-domain for
migration support */, while on x86 it can take the appropriate value
(which will soon actually be safe in migration scenarios).

However, that does change the ordering requirements in the toolstack -
this hypercall would need to be made after the domain is created, and
has been levelled, and before its main memory layout is decided.

Alternatively, the abstraction could be hidden in libxl itself in arch
specific code, with x86 referring to the local cpu policy (as libxl has
the copy it is/has worked on), and ARM making a hypercall.  This does
make the ordering more obvious.

(As a note on the x86 specifics of this patch, hap_paddr_bits is
actually unused in practice.  It was a proposal from AMD for the
hypervisor to fill in those details, which wasn't implemented by anyone,
not even Xen, because the important field to modify is maxphysaddr if
you don't want to rewrite every kernel to behave differently when




Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.