Re: Question about xen and Rasp 4B

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 3 Feb 2021, Jukka Kaartinen wrote:
> On 3.2.2021 2.18, Stefano Stabellini wrote:
> > On Tue, 2 Feb 2021, Jukka Kaartinen wrote:
> > > > > Good catch.
> > > > > GPU works now and I can start X! Thanks! I was also able to create
> > > > > domU
> > > > > that runs Raspian OS.
> > > > 
> > > > This is very interesting that you were able to achieve that - congrats!
> > > > 
> > > > Now, sorry to be a bit dense -- but since this thread went into all
> > > > sorts of interesting
> > > > directions all at once -- I just have a very particular question: what
> > > > is
> > > > exact
> > > > combination of versions of Xen, Linux kernel and a set of patches that
> > > > went
> > > > on top that allowed you to do that? I'd love to obviously see it
> > > > productized in Xen
> > > > upstream, but for now -- I'd love to make available to Project EVE/Xen
> > > > community
> > > > since there seems to be a few folks interested in EVE/Xen combo being
> > > > able
> > > > to
> > > > do that.
> > > 
> > > I have tried Xen Release 4.14.0, 4.14.1 and master (from week 4, 2021).
> > > 
> > > Kernel rpi-5.9.y and rpi-5.10.y branches from
> > > https://github.com/raspberrypi/linux
> > > 
> > > and
> > > 
> > > U-boot (master).
> > > 
> > > For the GPU to work it was enough to disable swiotlb from the kernel(s) as
> > > suggested in this thread.
> > 
> > How are you configuring and installing the kernel?
> > 
> > make bcm2711_defconfig
> > make Image.gz
> > make modules_install
> > 
> > ?
> > 
> > The device tree is the one from the rpi-5.9.y build? How are you loading
> > the kernel and device tree with uboot? Do you have any interesting
> > changes to config.txt?
> > 
> > I am asking because I cannot get to the point of reproducing what you
> > are seeing: I can boot my rpi-5.9.y kernel on recent Xen but I cannot
> > get any graphics output on my screen. (The serial works.) I am using the
> > default Ubuntu Desktop rpi-install target as rootfs and uboot master.
> > 
> 
> This is what I do:
> 
> make bcm2711_defconfig
> cat "xen_additions" >> .config
> make Image  modules dtbs
> 
> make INSTALL_MOD_PATH=rootfs modules_install
> depmod -a
> 
> cp arch/arm64/boot/dts/broadcom/bcm2711-rpi-4-b.dtb boot/
> cp arch/arm64/boot/dts/overlays/*.dtbo boot/overlays/

Thanks for the detailed instructions. This helps a lot. I saw below in
boot2.source that you are using ${fdt_addr} as DTB source (instead of
loading one), which means you are using the DTB as provided by U-Boot at
runtime, instead of loading your own file.

With these two copies, I take you meant to update the first partition on
the SD card, the one where config.txt lives, right? So that Xen is
getting the DTB and overlays from the rpi-5.9.y kernel tree but passed
down by the RPi loader and U-Boot?

I think the DTB must be the issue as I wasn't applying any overlays
before. I ran a test to use the DTB and overlay from U-Boot but maybe I
haven't updated them properly because I still don't see any output.


> config.txt:
> 
> [pi4]
> max_framebuffers=2
> enable_uart=1
> arm_freq=1500
> force_turbo=1
> 
> [all]
> arm_64bit=1
> kernel=u-boot.bin
> 
> start_file=start4.elf
> fixup_file=fixup4.dat
> 
> # Enable the audio output, I2C and SPI interfaces on the GPIO header
> dtparam=audio=on
> dtparam=i2c_arm=on
> dtparam=spi=on
> 
> # Enable the FKMS ("Fake" KMS) graphics overlay, enable the camera firmware
> # and allocate 128Mb to the GPU memory
> dtoverlay=vc4-fkms-v3d,cma-64
> gpu_mem=128
> 
> # Comment out the following line if the edges of the desktop appear outside
> # the edges of your display
> disable_overscan=1
> 
> 
> boot.source:
> setenv serverip 10.42.0.1
> setenv ipaddr 10.42.0.231
> tftpb 0xC00000 boot2.scr
> source 0xC00000
> 
> boot2.source:
> tftpb 0xE00000 xen
> tftpb 0x1000000 Image
> setenv lin_size $filesize
> 
> fdt addr ${fdt_addr}
> fdt resize 1024
> 
> fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0 sync_console
> dom0_mem=1024M dom0_max_vcpus=1 bootscrub=0 vwfi=native sched=credit2"
> 
> fdt mknod /chosen dom0
> 
> # These will break the default framebuffer@3e2fe000 that
> # is the same chosen -node.
> #fdt set /chosen/dom0 \#address-cells <0x2>
> #fdt set /chosen/dom0 \#size-cells <0x2>
> 
> fdt set /chosen/dom0 compatible "xen,linux-zimage" "xen,multiboot-module"
> fdt set /chosen/dom0 reg <0x1000000 0x${lin_size}>
> 
> fdt set /chosen xen,dom0-bootargs "dwc_otg.lpm_enable=0 console=hvc0
> earlycon=xen earlyprintk=xen root=/dev/sda4 elevator=deadline rootwait fixrtc
> quiet splash"
> 
> setenv fdt_high 0xffffffffffffffff
> 
> fdt print /chosen
> 
> #xen
> booti 0xE00000 - ${fdt_addr}
>