[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: MirageOS on OpenStack problem
Hello, Was able to reproduce it and now have a better idea of what's going on (with a temporary fix included). The issue happens when using qemu with a boot disk image exposed as a virtio-blk device. The problem is that solo5 tries to configure a virtio device that was already initialized/configured at VM early boot time. Qemu gets "confused" with the second configuration attempt, throws the "virtio: zero sized buffers are not allowed" error and gets stuck (somewhere that I couldn't determine). Just in case, these are all the tests I tried (using qemu v3 as v5 has an issue https://github.com/Solo5/solo5/issues/463). To make things simpler I've been using the solo5 tests/test-net unikernel (c code, not mirage). You can create disk-net.img in solo5 using: tests (master) [83]> ../scripts/virtio-mkimage/solo5-virtio-mkimage.sh -f raw disk-net.img test_net/test_net.virtio These work: + boot disk image as an IDE device: qemu-3 -drive file=disk-net.img,if=ide,format=raw -cpu Westmere -m 128 -nodefaults -no-acpi -display none \ -serial stdio -device virtio- net,netdev=n0 -netdev tap,id=n0,ifname=tap100,script=no,downscript=no -device isa-debug-exit +boot disk image as a virtio-scsi device (same as google cloud):
qemu-3 -device virtio-scsi-pci,id=scsi \
-device scsi-hd,drive=hd \
-drive if=none,id=hd,file=disk-net.img,format=raw \
-cpu Westmere -m 128 -nodefaults -no-acpi -display none -serial stdio \
-device virtio-net,netdev=n0 -netdev tap,id=n0,ifname=tap100,script=no,downscript=no -device isa-debug-exit
These do NOT work:
+ boot disk image as a virtio-blk device:
qemu-3 -drive file=disk-net.img,if=virtio,format=raw -cpu Westmere -m 128 -nodefaults -no-acpi -display none \
-serial stdio -device virtio-net,netdev=n0 -netdev tap,id=n0,ifname=tap100,script=no,downscript=no -device isa-debug-exit
The right fix would be to check in solo5 init code that the virtio device is already configured, and just skip it. For now, this temporary patch (disable virtio-blk completely) works:
==============================================================================================
--- a/bindings/virtio/pci.c
+++ b/bindings/virtio/pci.c
@@ -54,7 +54,6 @@
static uint32_t net_devices_found;
-static uint32_t blk_devices_found;
#define PCI_CONF_SUBSYS_NET 1
#define PCI_CONF_SUBSYS_BLK 2
@@ -72,15 +71,6 @@ static void virtio_config(struct pci_config_info *pci)
log(WARN, "Solo5: PCI:%02x:%02x: not configured\n", pci->bus,
pci->dev);
break;
- case PCI_CONF_SUBSYS_BLK:
- log(INFO, "Solo5: PCI:%02x:%02x: virtio-block device, base=0x%x, irq=%u\n",
- pci->bus, pci->dev, pci->base, pci->irq);
- if (!blk_devices_found++)
- virtio_config_block(pci);
- else
- log(WARN, "Solo5: PCI:%02x:%02x: not configured\n", pci->bus,
- pci->dev);
- break;
default:
log(WARN, "Solo5: PCI:%02x:%02x: unknown virtio device (0x%x)\n",
pci->bus, pci->dev, pci->subsys_id);
==============================================================================================
Thanks,
Ricardo
Hello,
Will reproduce it and update with some findings, hopefully by the end of this week.
Ricardo
Hi,
$ qemu-system-x86_64 --version
QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.32)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
--
Hans Ole
(Re-sending with a current email for Ricardo)
On Monday, 26.10.2020 at 11:37, Martin Lucina wrote:
> Hi,
>
> On Sunday, 25.10.2020 at 13:37, Hans Ole Rafaelsen wrote:
> > Hi,
> >
> > I have been investigating some more, and I seem to be a 'virtio-block
> > device' problem. On the OpenStack cloud this device is reported when Solo5
> > boots, but not on my local installation.
> >
> > I changed from IDE (default) to virtio as a drive on my local installation,
> > and that gives the same behaviour as on the cloud. So the behaviour can be
> > shown with these two different examples.
> >
> > Using a IDE drive for the image:
> > $ qemu-system-x86_64 -cpu Westmere -m 128 -nodefaults -no-acpi -display
> > none -serial stdio -device virtio-net,netdev=n0 -netdev
> > tap,id=n0,ifname=tap100,script=no,downscript=no -device isa-debug-exit
> > -device lsi,id=scsi0,bus=pci.0,addr=0x9 \
> > -drive
> > file=/home/hans/src/5g/mirage/repository.qcow2,format=qcow2,if=none,id=drive-ide0-0-0
> > -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
> >
> > SYSLINUX 6.03 20171017 Copyright (C) 1994-2014 H. Peter Anvin et al
> > Loading unikernel.bin... ok
> > | ___|
> > __| _ \ | _ \ __ \
> > \__ \ ( | | ( | ) |
> > ____/\___/ _|\___/____/
> > Solo5: Bindings version v0.6.7
> > Solo5: Memory map: 128 MB addressable:
> > Solo5: reserved @ (0x0 - 0xfffff)
> > Solo5: text @ (0x100000 - 0x481fff)
> > Solo5: rodata @ (0x482000 - 0x51dfff)
> > Solo5: data @ (0x51e000 - 0x74efff)
> > Solo5: heap >= 0x74f000 < stack < 0x8000000
> > Solo5: Clock source: TSC, frequency estimate is 2808856460 Hz
> > Solo5: PCI:00:02: virtio-net device, base=0xc100, irq=10
> > Solo5: PCI:00:02: configured, mac=52:54:00:12:34:56, features=0x79bfffe7
> > Solo5: Application acquired 'service' as network device
> > 2020-10-25 12:16:34 -00:00: INF [netif] Plugging into service with mac
> > 52:54:00:12:34:56 mtu 1500
> > 2020-10-25 12:16:34 -00:00: INF [ethernet] Connected Ethernet interface
> > 52:54:00:12:34:56
> >
> > Using virtio drive for the image:
> > $ qemu-system-x86_64 -cpu Westmere -m 128 -nodefaults -no-acpi -display
> > none -serial stdio -device virtio-net,netdev=n0 -netdev
> > tap,id=n0,ifname=tap100,script=no,downscript=no -device isa-debug-exit \
> > -drive
> > file=/home/hans/src/hermod-5g/mirage/repository.qcow2,format=qcow2,if=none,id=drive-virtio-disk0
> > -device
> > virtio-blk-pci,scsi=off,bus=pci.0,addr=0xa,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> >
> >
> > SYSLINUX 6.03 20171017 Copyright (C) 1994-2014 H. Peter Anvin et al
> > Loading unikernel.bin... ok
> > | ___|
> > __| _ \ | _ \ __ \
> > \__ \ ( | | ( | ) |
> > ____/\___/ _|\___/____/
> > Solo5: Bindings version v0.6.7
> > Solo5: Memory map: 127 MB addressable:
> > Solo5: reserved @ (0x0 - 0xfffff)
> > Solo5: text @ (0x100000 - 0x481fff)
> > Solo5: rodata @ (0x482000 - 0x51dfff)
> > Solo5: data @ (0x51e000 - 0x74efff)
> > Solo5: heap >= 0x74f000 < stack < 0x7ffe000
> > Solo5: Clock source: TSC, frequency estimate is 2809165540 Hz
> > Solo5: PCI:00:02: virtio-net device, base=0xc040, irq=10
> > Solo5: PCI:00:02: configured, mac=52:54:00:12:34:56, features=0x79bfffe7
> > Solo5: PCI:00:0a: virtio-block device, base=0xc000, irq=10
> > Solo5: PCI:00:0a: configured, capacity=2097152 sectors, features=0x79000e54
> > qemu-system-x86_64: virtio: zero sized buffers are not allowed
>
> Looks like some bug with recent QEMU and the Solo5 virtio-block
> implementation.
>
> What QEMU version is that?
>
> As I wrote, the virtio target does not get much testing :-(
>
> > Is the problem that I need to set up handlers etc. to handle a virtio block
> > device in the MirageOS application? I can't find anything about this in the
> > documentation. Or might it be a bug in Solo5/MirageOS?
>
> No, it's a bug.
>
> Cc:ing Ricardo who may be able to look into it.
>
> Martin
>
|
