[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

xen_platform_pci and 4k-blocksize issue on DomU



Hello.

We run several Linux virtual machines with latest Linux kernels.

Most of machines (group A) have xen_platform_pci enabled and single disk sdrive emulated by physical LVM residing on SSD. SSD has logical and physical block size of 512 bytes. Machines have virtual disk /dev/xvda, gdisk on that /dev/xvda reports logical and physical block of 512 bytes.

Few more machines (group B) are running on the same servers but have xen_platform_pci disabled. Their LVM resides on either SSD or NVMe, those NVMe has 4096-byte blocks unlike SSDs. All the machines of this group regardless of the physical storage under LVM report virtual disk as /dev/sda and gdisk reports block size of 512 bytes there.

Last group (group C) has two virtual disks connected to them, one is rootfs residing on SSD and another one is data partition that physically resides on NVMe.

Machines of this last group C have xen_platform_pci enabled, so disks are visible as /dev/xvda and /dev/xvdb. Before upgrade to Linux-6.9.3 these machines have had /dev/xvda reporting pysical/logical block size of 512 bytes and /dev/xvdb with pysical/logical blocks of 4096-byte size.

Problem 1. Machines of group A can't boot in case of underlying storage migration to LVM that physically resides on NVMe - as their partitions starts to report physical/logical blocks of 4096-bytes (instead of 512 logical/4096 physical that could be expected), so it was impossible to migrate VMs running with xen_platform_pci enabled. This problem doesn't affect machines of group B (having xen_platform_pci disabled and always using 512-byte blocks)

Problem 2. On upgrading Linux from kernel 6.8.12 to kernel version 6.9.3 machines of group C started to report block size of both partitions - the first one that physically runs over SSD and second one over NVMe - of the same value, so blocks of all the devices became 512-byte (and amount of blocks is 8x times bigger on data partitions that used to be reporting 4k-blocks with older linux kernel). Kernel configuration almost didn't change, just new options introduced in 6.9.3 appeared, but none of them is disk or xen-related. The issue persists on all the 6.9 branch, the latest one, and never happen on 6.6 or 6.8 kernels, on reboot to 6.6.30 machine still reports one partition as having 512-byte block, another one as 4kbyte-blocks, so matches underlying storage block size.

The problem seems to be related to some change of Xen PVHVM driver in Linux kernel, but it was not mentioned anywhere explicitly.

In case of either disabling xen_platform_pci or using "sdb" instead of "hdb" in disk configuration (so guest kernel detects SCSI drive), block has (and has always had) 512-byte size on any drive regardless of underlying storage block size.

The question is whether to consider this as a bug (as block size is not aligned in guest and physical storage anymore while it used to be) or fix as now all the drivers are not depending from physical storage parameters? If this is a fix and desired behavior, then how big the performance impact will be because of block size inside the VM being 8 times smaller than the physical one?..

--
Regards,
Eugene K.




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.