Xen project Mailing List

Hi Bertrand,

Thanks for your support!

As suggested, I enabled BLK_DEV_LOOP in the kernel & now I can see loop devices in the "/dev" directory.

I could see xen-guest-image-minimal loading but at the end it's throwing error INIT: cannot execute "/sbin/getty-wrapper"

I found start_getty in the image but getty-wrapper is not there. while exploring source could see getty-wrapper in meta-virtualization sysvinit,

but start_getty is in poky sysvinit.

Any suggestions on this issue? Adding full logs below for reference:

root@raspberrypi4-64:~# xl create -c guest1.cfg
Parsing config from guest1.cfg
Booting Linux on physical CPU 0x0000000000 [0x410fd083]
Linux version 5.15.54-yocto-standard (oe-user@oe-host) (aarch64-poky-linux-gcc (GCC) 11.3.0, GNU ld (GNU Binutils) 2.38.20220623) #1 SMP PREEMPT Thu Jul 14 18:52:13 UTC 2022
random: crng init done
Machine model: XENVM-4.16
Xen 4.16 support found
efi: UEFI not found.
Zone ranges:
DMA [mem 0x0000000040000000-0x000000007fffffff]
DMA32 empty
Normal empty
Movable zone start for each node
Early memory node ranges
node 0: [mem 0x0000000040000000-0x000000007fffffff]
Initmem setup node 0 [mem 0x0000000040000000-0x000000007fffffff]
cma: Reserved 16 MiB at 0x000000007dc00000
psci: probing for conduit method from DT.
psci: PSCIv1.1 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: Trusted OS migration not required
psci: SMC Calling Convention v1.1
percpu: Embedded 27 pages/cpu s70360 r8192 d32040 u110592
Detected PIPT I-cache on CPU0
CPU features: detected: Spectre-v2
CPU features: detected: Spectre-v4
CPU features: detected: Spectre-BHB
CPU features: detected: ARM errata 1165522, 1319367, or 1530923
Built 1 zonelists, mobility grouping on. Total pages: 258048
Kernel command line: console=hvc0 earlyprintk=xen sync_console root=/dev/xvda
Unknown kernel command line parameters "sync_console earlyprintk=xen", will be passed to user space.
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 985220K/1048576K available (13376K kernel code, 2060K rwdata, 4372K rodata, 4160K init, 714K bss, 46972K reserved, 16384K cma-reserved)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
ftrace: allocating 41866 entries in 164 pages
ftrace: allocated 164 pages with 3 groups
trace event string verifier disabled
rcu: Preemptible hierarchical RCU implementation.
rcu: RCU event tracing is enabled.
rcu: RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.
Trampoline variant of Tasks RCU enabled.
Rude variant of Tasks RCU enabled.
Tracing variant of Tasks RCU enabled.
rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
Root IRQ handler: gic_handle_irq
kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)
arch_timer: cp15 timer(s) running at 54.00MHz (virt).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
Console: colour dummy device 80x25
printk: console [hvc0] enabled
Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
pid_max: default: 32768 minimum: 301
LSM: Security Framework initializing
landlock: Up and running.
Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
xen:grant_table: Grant tables using version 1 layout
Grant table initialized
xen:events: Using FIFO-based ABI
Xen: initializing cpu0
rcu: Hierarchical SRCU implementation.
EFI services will not be available.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated.
CPU features: detected: 32-bit EL0 Support
CPU features: detected: CRC32 instructions
CPU: All CPU(s) started at EL1
alternatives: patching kernel code
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
futex hash table entries: 256 (order: 2, 16384 bytes, linear)
pinctrl core: initialized pinctrl subsystem
DMI not present or invalid.
NET: Registered PF_NETLINK/PF_ROUTE protocol family
DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
thermal_sys: Registered thermal governor 'step_wise'
hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
ASID allocator initialised with 65536 entries
Serial: AMBA PL011 UART driver
raid6: neonx8 gen() 1585 MB/s
raid6: neonx8 xor() 1177 MB/s
raid6: neonx4 gen() 1701 MB/s
raid6: neonx4 xor() 1279 MB/s
raid6: neonx2 gen() 1473 MB/s
raid6: neonx2 xor() 1161 MB/s
raid6: neonx1 gen() 1131 MB/s
raid6: neonx1 xor() 929 MB/s
raid6: int64x8 gen() 946 MB/s
raid6: int64x8 xor() 555 MB/s
raid6: int64x4 gen() 985 MB/s
raid6: int64x4 xor() 570 MB/s
raid6: int64x2 gen() 914 MB/s
raid6: int64x2 xor() 506 MB/s
raid6: int64x1 gen() 714 MB/s
raid6: int64x1 xor() 377 MB/s
raid6: using algorithm neonx4 gen() 1701 MB/s
raid6: .... xor() 1279 MB/s, rmw enabled
raid6: using neon recovery algorithm
xen:balloon: Initialising balloon driver
iommu: Default domain type: Translated
iommu: DMA domain TLB invalidation policy: strict mode
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@xxxxxxxx>
PTP clock support registered
Bluetooth: Core ver 2.22
NET: Registered PF_BLUETOOTH protocol family
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP socket layer initialized
Bluetooth: SCO socket layer initialized
clocksource: Switched to clocksource arch_sys_counter
NET: Registered PF_INET protocol family
IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes, linear)
Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
TCP established hash table entries: 8192 (order: 4, 65536 bytes, linear)
TCP bind hash table entries: 8192 (order: 5, 131072 bytes, linear)
TCP: Hash tables configured (established 8192 bind 8192)
UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
NET: Registered PF_UNIX/PF_LOCAL protocol family
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: CLS 0 bytes, default 64
Initialise system trusted keyrings
workingset: timestamp_bits=46 max_order=18 bucket_order=0
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
Key type cifs.idmap registered
xor: measuring software checksum speed
8regs : 2522 MB/sec
32regs : 2865 MB/sec
arm64_neon : 2319 MB/sec
xor: using function: 32regs (2865 MB/sec)
async_tx: api initialized (async)
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
io scheduler mq-deadline registered
io scheduler kyber registered
xen:xen_evtchn: Event-channel device installed
Serial: 8250/16550 driver, 1 ports, IRQ sharing enabled
cacheinfo: Unable to detect cache hierarchy for CPU 0
brd: module loaded
loop: module loaded
Invalid max_queues (4), will use default max: 1.
xen_netfront: Initialising Xen virtual ethernet driver
usbcore: registered new interface driver lan78xx
usbcore: registered new interface driver asix
usbcore: registered new interface driver ax88179_178a
usbcore: registered new interface driver cdc_ether
usbcore: registered new interface driver smsc95xx
usbcore: registered new interface driver net1080
usbcore: registered new interface driver cdc_subset
usbcore: registered new interface driver zaurus
usbcore: registered new interface driver cdc_ncm
usbcore: registered new interface driver aqc111
usbcore: registered new interface driver usb-storage
device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@xxxxxxxxxx
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pltfm: SDHCI platform and OF driver helper
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
u32 classifier
input device check on
Actions configured
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered PF_PACKET protocol family
Bridge firewalling registered
lib80211: common routines for IEEE802.11 drivers
Key type dns_resolver registered
Loading compiled-in X.509 certificates
Key type ._fscrypt registered
Key type .fscrypt registered
Key type fscrypt-provisioning registered
Btrfs loaded, crc32c=crc32c-generic, zoned=no, fsverity=no
Key type encrypted registered
xen_netfront: backend supports XDP headroom
blkfront: xvda: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled; bounce buffer: enabled
printk: console [netcon0] enabled
netconsole: network logging started
md: Waiting for all devices to be available before autodetect
md: If you don't use raid, use raid=noautodetect
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
EXT4-fs (xvda): mounting ext3 file system using the ext4 subsystem
EXT4-fs (xvda): INFO: recovery required on readonly filesystem
EXT4-fs (xvda): write access will be enabled during recovery
EXT4-fs (xvda): recovery complete
EXT4-fs (xvda): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
VFS: Mounted root (ext3 filesystem) readonly on device 202:0.
devtmpfs: mounted
Freeing unused kernel memory: 4160K
Run /sbin/init as init process
INIT: version 3.01 booting
Starting udev
udevd[128]: starting version 3.2.10
udevd[129]: starting eudev-3.2.10
EXT4-fs (xvda): re-mounted. Opts: (null). Quota mode: disabled.
INIT: Entering runlevel: 5
Configuring network interfaces... udhcpc: started, v1.35.0
udhcpc: broadcasting discover
IPv6: eth0: IPv6 duplicate address fe80::e65f:1ff:fecd:7bdb used by e4:5f:01:cd:7b:db detected!
udhcpc: broadcasting discover
udhcpc: broadcasting discover
udhcpc: no lease, failing
ifup: failed to bring up eth0
Starting syslogd/klogd: done
Starting domain watchdog daemon: [ OK ]
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: cannot execute "/sbin/getty-wrapper"
INIT: Id "X0" respawning too fast: disabled for 5 minutes

Regards,

Vipul Kumar

On Thu, Aug 25, 2022 at 2:58 PM Bertrand Marquis <Bertrand.Marquis@xxxxxxx> wrote:

Hi Vipul,

> On 25 Aug 2022, at 09:56, Vipul Suneja <vsuneja63@xxxxxxxxx> wrote:
>
> Hi Bertrand,
>
> Thanks!
>
> No, I couldn't see /dev/loop0. Can you please guide me to create it?

First thing to try is “modprobe loop"

It that does not work (ie module not found) you should check in your linux config if BLK_DEV_LOOP is enabled.

>
> I didn't change dom0 configurations, it's default generated by yocto.
>
> I will append this "IMAGE_FSTYPES:append = " wic.gz”" in local.conf & will update you.
>

Cheers
Bertrand

> Regards,
> Vipul Kumar
>
> On Thu, Aug 25, 2022 at 1:25 PM Bertrand Marquis <Bertrand.Marquis@xxxxxxx> wrote:
> Hi Vipul,
>
> > On 25 Aug 2022, at 08:31, Vipul Suneja <vsuneja63@xxxxxxxxx> wrote:
> >
> > Hi Stefano,
> >
> > Thanks!
> >
> > As suggested, I changed the guest1.cfg file. Below are the contents of config file
> >
> > kernel = "/home/root/Image"
> > cmdline = "console=hvc0 earlyprintk=xen sync_console root=/dev/xvda"
> > memory = "1024"
> > name = "guest1"
> > vcpus = 1
> > serial="pty"
> > disk = [ 'file:/home/root/xen-guest-image-minimal-raspberrypi4-64.ext3,xvda,w' ]
> > vif=[ 'mac=00:11:22:66:88:22,bridge=xenbr0,type=netfront', ]
> >
> > Its failing with below logs:
> >
> > root@raspberrypi4-64:~# xl create -c guest1.cfg
> > Parsing config from guest1.cfg
> > Invalid parameter `type'.
> > libxl: error: libxl_exec.c:117:libxl_report_child_exitstatus: /etc/xen/scripts/block add [742] exited with error status 1
> > libxl: error: libxl_device.c:1265:device_hotplug_child_death_cb: script: losetup /dev/loop0 /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3 failed
> > libxl: error: libxl_create.c:1643:domcreate_launch_dm: Domain 1:unable to add disk devices
> > libxl: error: libxl_exec.c:117:libxl_report_child_exitstatus: /etc/xen/scripts/block remove [793] exited with error status 1
> > libxl: error: libxl_device.c:1265:device_hotplug_child_death_cb: script: /etc/xen/scripts/block failed; error detected.
> > libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain 1:Non-existant domain
> > libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 1:Unable to destroy guest
> > libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 1:Destruction of domain failed
>
> I think you have a loop issue.
>
> Could you check if /dev/loop0 exists ?
>
> Did you change something on the dom0 linux configuration generated by Yocto ?
>
> We are using Yocto on RPI4 here without any issue like that, only difference with
> your setup is that we generate a wic image to have a real disk image instead of
> using the ext3/ext4 one.
>
> Should be possible to do the same on your side by adding the following in local.conf:
> IMAGE_FSTYPES:append = " wic.gz”
>
> >
> > Even after removing 'type=netfront' from vif it's failing.
>
> This option is only for hvm on x86, so you can remove it from your configuration.
>
> > One more doubt here, could this mac address be a dummy or actual here?
>
> This is a dummy one you set for the guest network interface and this is the Mac
> address other devices on your network will see so it must be fully valid (and
> not conflicting with other devices on your network).
>
> Cheers
> Bertrand
>
> >
> > Regards,
> > Vipul Kumar
> >
> > On Thu, Aug 25, 2022 at 2:36 AM Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> > On Wed, 24 Aug 2022, Vipul Suneja wrote:
> > > Hi Bertrand,
> > > Thanks for your response!
> > >
> > > I builded the guest image on yocto kirkstone source which has FSTYPE ext3. Guest image generated is
> > > xen-guest-image-minimal-raspberrypi4-64.ext3.
> > > Below is the content of guest.cfg file
> > >
> > > kernel = "/home/root/Image"
> > > cmdline = "console=hvc0 earlyprintk=xen sync_console root=/dev/xvda"
> > > memory = "256"
> > > name = "guest1"
> > > vcpus = 1
> > > serial="pty"
> > > disk = [ 'phy:/dev/loop0,xvda,w' ]
> > > vif=[ 'mac=00:11:22:66:88:22,bridge=xenbr0,type=netfront', ]
> > >
> > > I am trying to mount xen-guest-image-minimal-raspberrypi4-64.ext3 to a virtual device & then will run the guest VM by command "xl create -c
> > > guest.cfg". But facing issue while trying to mount.
> >
> > You don't actually need to mount
> > xen-guest-image-minimal-raspberrypi4-64.ext3 anywhere to use it to run
> > your guest VM with "xl create".
> >
> > It is enough to do this instead, as Bertrand suggested:
> >
> > disk=["file:/path/to/file/xen-guest-image-minimal-raspberrypi4-64.ext3,xvda,w"]
> >
> > No need to call losetup or mount. Just xl create -c.
> >
> > More answers below.
> >
> >
> > > Regards,
> > > Vipul Kumar
> > >
> > > On Wed, Aug 24, 2022 at 8:06 PM Bertrand Marquis <Bertrand.Marquis@xxxxxxx> wrote:
> > > Hi Vipul,
> > >
> > > > On 24 Aug 2022, at 15:16, Vipul Suneja <vsuneja63@xxxxxxxxx> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I am porting xen hypervisor on rpi4 with yocto kirkstone sources. Followed the basic steps to build xen-image-minimal &
> > > xen-guest-image-minimal. I could flash sd card with xen minimal image & could see dom0 up. I copied "Image",
> > > "xen-guest-image-minimal" .ext3 file & guest.cfg to "/home/root". After that created a bridge with below step:
> > > >
> > > > killall -SIGUSR2 udhcpc
> > > > brctl addbr xenbr0
> > > > brctl addif xenbr0 eth0
> > > > killall udhcpc
> > > > udhcpc -R -b -p /var/run/udhcpc.xenbr0.pid -i xenbr0
> > > >
> > > > Could see the xenbr0 interface up.
> > > > After that while mounting the guest file system it shows no such file or directory but the file is already there.
> > > >
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# ls -l
> > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root 24652288 Mar 9 12:36 Image
> > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root 247 Mar 9 12:37 guest1.cfg
> > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root 868220928 Mar 9 12:39 xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# chmod 0777 xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# ls -l
> > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root 24652288 Mar 9 12:36 Image
> > > > [23:40:15] <Guest9046> -rw-r--r-- 1 root root 247 Mar 9 12:37 guest1.cfg
> > > > [23:40:15] <Guest9046> -rwxrwxrwx 1 root root 868220928 Mar 9 12:39 xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# losetup /dev/loop0 xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > [23:40:15] <Guest9046> losetup: xen-guest-image-minimal-raspberrypi4-64.ext3: No such file or directory
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# losetup /dev/loop0 /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > [23:40:15] <Guest9046> losetup: /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3: No such file or directory
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~#
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~#
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~#
> > > > [23:40:15] <Guest9046> root@raspberrypi4-64:~# losetup /dev/loop0 /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3
> > > > [23:40:15] <Guest9046> losetup: /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3: No such file or directory
> >
> > It looks like either
> > /home/root/xen-guest-image-minimal-raspberrypi4-64.ext3 doesn't exist or
> > /dev/loop0 doesn't exist
> >
> >
> > > Why do you want to mount the file system ?
> > >
> > > Anyway this is not related to Xen, I guess you could start without xen and still not manage to mount the file like that (linux
> > > configuration issue ?)
> > >
> > > What is the content of you guest.cfg
> > > How do you want to pass the guest root file system ?
> > >
> > > Yocto should actually generate an img file and you could use it by having something like this in your guest.cfg:
> > > disk=["file:/home/root/guest1.img,xvda,w”]
> > >
> > > Cheers
> > > Bertrand
> > >
> > > >
> > > > Any input on this issue will be really helpful, expecting your response.
> > > >
> > > > Thanks & Regards,
> > > > Vipul Kumar
> > >
> > >
> > >
>

Re: Porting xen on rpi4