Re: [RFC PATCH v2 00/16] Initial support for machine creation via QMP

[Alistair Francis <alistair23@xxxxxxxxx>]

On Thu, Sep 23, 2021 at 2:22 AM Damien Hedde <damien.hedde@xxxxxxxxxxxxx> wrote:
>
> Hi,
>
> The goal of this work is to bring dynamic machine creation to QEMU:
> we want to setup a machine without compiling a specific machine C
> code. It would ease supporting highly configurable platforms (for
> example resulting from an automated design flow). The requirements
> for such configuration include begin able to specify the number of
> cores, available peripherals, emmory mapping, IRQ mapping, etc.
>
> This series focuses on the first step: populating a machine with
> devices during its creation. We propose patches to support this
> using QMP commands. This is a working set of patches and improves
> over the earlier rfc (posted in May):
> https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg03706.html
>
> Although it is working and could be merged, it is tag as an RFC:
> we probably need to discuss the conditions for allowing a device to
> be created at an early stage. Patches 6, 10 and 13, 15 and 16 depend
> on such conditions and are subject to change. Other patches are
> unrelated to this point.
>
> We address several issues in this series. They are detailed below.
>
> ## 1. Stoping QEMU to populate the machine with devices
>
> QEMU goes through several steps (called _machine phases_) when
> creating the machine: 'no-machine', 'machine-created',
> 'accel-created', 'initialized', and finally 'ready'. At 'ready'
> phase, QEMU is ready to start (see Paolo's page
> https://wiki.qemu.org/User:Paolo_Bonzini/Machine_init_sequence for
> more details).
>
> Using the -preconfig CLI option, QEMU can be stopped today during
> the 'accel-created' phase. Then the 'x-exit-preconfig' QMP command
> triggers QEMU moving forwards to the completion of the machine
> creation ('ready' phase).
>
> The devices are created during the 'initialized' phase.
> In this phase the machine init() method has been executed and thus
> machine properties have been handled. Although the sysbus exists and
> the machine may have been populated by the init(),
> _machine_init_done_ notifiers have not been called yet. At this point
> we can add more devices to a machine.
>
> We propose to add 2 QMP commands:
> + The 'query-machine-phase' command would return the current machine
>   phase.
> + The 'x-machine-init' command would advance the machine phase to
>   'initialized'. 'x-exit-preconfig' could then still be used to
>   advance to the last phase.
>
> ## 2. Adding devices
>
> Right now, the user can create devices in 2 ways: using '-device' CLI
> option or 'device_add' QMP command. Both are executed after the
> machine is ready: such devices are hot-plugged. We propose to allow
> 'device_add' QMP command to be used during the 'initialized' phase.
>
> In this series, we keep the constraint that the device must be
> 'user-creatable' (this is a device class flag). We do not see any
> reason why a device the user can hot-plug could not be created at an
> earlier stage.
>
> This part is still RFC because, as Peter mentioned it (in this thread
> https://lists.gnu.org/archive/html/qemu-devel/2021-08/msg01933.html),
> we may want additional or distinct conditions for:
> + device we can hot-plug
> + device we can add in '-preconfig' (cold-plug)
> We are open to suggestions. We could for example add a
> 'preconfig-creatable' or 'init-creatable' flag to device class, which
> can identify a set of devices we can create this way.
>
> The main addition is how we handle the case of sysbus devices. Sysbus
> devices are particular because unlike, for example, pci devices, you
> have to manually handle the memory mapping and interrupts wiring. So
> right now, a sysbus device is dynamically creatable (using -device
> CLI option or device_add QMP command) only if:
> + it is 'user_creatable' (like any other device),
> + and it is in the current machine sysbus device allow list.
>
> In this series, we propose to relax the second constraint during the
> earlier phases of machine creation so that when using -preconfig we
> can create any 'user-creatable' sysbus device. When the machine
> progresses to the 'ready' phase, sysbus devices creation will come
> back to the legacy behavior: it will be possible only based on the
> per-machine authorization basis.
>
> For sysbus devices, wiring interrupts is not a problem as we can use
> the 'qom-set' QMP command, but memory mapping is.
>
> ## 3. Memory mapping
>
> There is no point allowing the creation sysbus devices if we cannot
> map them onto the memory bus (the 'sysbus').
>
> As far as we know, right now, there is no way to add memory mapping
> for sysbus device using QMP commands. We propose a 'x-sysbus-mmio-map'
> command to do this. This command would only be allowed during the
> 'initialized' phase when using -preconfig.
>
> ## 4. Working example
>
> The last patches of the series add and modify devices in order to
> build a working machine starting from the 'none' machine.
>
> We add a new sysbus device modeling a simple memory (ram or rom). We
> also set 'user-creatable' flag of some sysbus devices. These are
> trivial patches, but they depends on the conditions we choose to allow
> creating devices with -preconfig. Therefore, there is really no need
> to review them until we settled on the device conditions first.
>
> With these devices (memory, ibex_uart, ibex_plic) we can dynamically
> configure a part (we did not add the timer, but we could) the
> opentitan machine very easily and run firmwares which demonstrates
> interrupts and memory-mapping are working.
>
> We use the existing qmp-shell script to issue machine devices
> from a qmp commands script file which contains qmp commands listed in
> a file.
>
> The following qmp commands add some memories, an interrupt controller
> and an uart with an interrupt.
>
> cat > opentitan.qmp <<EOF
> x-machine-init
>
> # ROM 0x00008000
> device_add        driver=sysbus-memory id=rom size=0x4000 readonly=true
> x-sysbus-mmio-map device=rom addr=32768
>
> # FLASH 0x20000000
> device_add        driver=sysbus-memory id=flash size=0x80000 readonly=true
> x-sysbus-mmio-map device=flash addr=536870912
>
> # RAM 0x10000000
> device_add        driver=sysbus-memory id=ram size=0x10000
> x-sysbus-mmio-map device=ram addr=268435456
>
> # PLIC 0x41010000
> device_add        driver=ibex-plic id=plic
> x-sysbus-mmio-map device=plic addr=1090584576
>
> # UART 0x40000000
> device_add        driver=ibex-uart id=uart chardev=serial0
> x-sysbus-mmio-map device=uart addr=1073741824
> qom-set path=uart property=sysbus-irq[1] value=plic/unnamed-gpio-in[2]
>
> x-exit-preconfig
> EOF
>
> We've put the opentitan.qmp and a firmware opentitan-echo.elf here
> (among some other qmp machine files we are working on):
> https://github.com/GreenSocs/qemu-qmp-machines

I am unable to access this repo, maybe it's not public?

Alistair