Re: [PATCH v3 6/7] x86/dmop: Add XEN_DMOP_enable_ext_dest_id DM op

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Tue, Mar 17, 2026 at 10:22:20AM +0000, Julian Vetter wrote:
> On 3/9/26 14:29, Roger Pau Monné wrote:
> > On Mon, Mar 09, 2026 at 12:31:03PM +0000, Julian Vetter wrote:
> >> Xen cannot simply advertise XEN_HVM_CPUID_EXT_DEST_ID to the guest
> >> without knowing that the device model will handle extended destination
> >> IDs correctly for passthrough MSIs. A device model that still uses
> >> XEN_DOMCTL_bind_pt_irq would pass only the low 8 bits of the destination
> >> ID, misrouting interrupts to vCPUs with APIC IDs greater than 255. So,
> >> add a DM op XEN_DMOP_enable_ext_dest_id that the device model can call
> >> during domain setup (before vCPUs are started) to signal that it will
> >> use XEN_DMOP_bind_pt_msi_irq for all passthrough MSI bindings. When
> >> called, Xen sets ext_dest_id_enabled in struct hvm_domain, so it's
> >> visible to the guest via CPUID.
> > 
> > Have you considered whether you could re-use the padding in
> > XEN_DMOP_create_ioreq_server to signal whether the device model
> > supports Extended ID parsing?
> > 
> > Also, you might want some negotiation between multiple ioreq servers
> > on the same domain.  IOW: is multiple ioreq servers are registered
> > ahead of the domain having finished creation you could level whether
> > extended ID should be announced.  For ioreqs that are registered after
> > the domain have started you need to enforce the currently set Extended
> > ID support.  If the domain is running, and Extended ID is advertised
> > you must prevent registering any new ioreq that doesn't support
> > Extended ID.
> > 
> 
> Thank you Roger for your feedback! It's very appreciated! This was a 
> good idea. I have implemented this now. I have used one of the reserved 
> bytes and use it as a flag field.
> 
> But I have a remaining question/concern, which maybe you can clarify.
> If server A (e.g., a secondary emulator) registers WITH ext_dest_id 
> before the domain starts, hvm_ext_dest_id_enabled() returns true and the 
> guest will be advertised XEN_HVM_CPUID_EXT_DEST_ID. If QEMU's primary 
> ioreq server then registers WITHOUT the flag

But QEMU would also register ahead of domain build having finished,
and hence you need to take the minimum of all supported emulators.  In
this case, if there's a single emulator registered ahead of the domain
build having finished that doesn't support the Extended Destination ID
format you cannot advertise it.

> and goes on to use 
> XEN_DOMCTL_bind_pt_irq, pass-through MSIs will be misrouted (or rejected 
> if I then refuse calls to XEN_DOMCTL_bind_pt_irq for that domain). My 
> implementation allows this combination before 'd->creation_finished'.

Yes, the combination is fine as the feature is leveled across all
registered emulators before domain creation finished.  Once domain
creation has finished newly registered emulators must have Extended
Destination support if the feature has been advertised by Xen in
CPUID.

> I have added a check in ioreq_server_dm_op() for the 
> XEN_DMOP_create_ioreq_server case for the "runtime" case as you 
> suggested. If one of the existing ioreq servers announces it, and a new 
> server wants to join, that doesn't have this flag set, it will be 
> rejected with -EINVAL:

That seems fine, as long as the feature is levelled across all
registered ioreq server prior to the domain being started.  IOW: you
shouldn't take the value of the first registered ioreq server, it has
to be levelled across all registered ioreqs ahead of domain start.

> if ( d->creation_finished && hvm_ext_dest_id_enabled(d) &&
>       !(data->flags & XEN_DMOP_IOREQ_SERVER_EXT_DEST_ID) )
>      break;
> 
> But what happens to the first case I described, how can I reasonably 
> negotiate between different ioreq servers or determine who is the "main" 
> server? 

Why do you care how is the "main" ioreq server?  All ioreq servers
must support the feature for it to be enabled, it doesn't matter
whether it's a primary or a secondary server.

> In practice, usually if QEMU ("the only" server that matters for 
> pass-through) opted in, the feature is enabled.

Hm, not really, On XenServer we use a separate ioreq server (not QEMU)
just for passthrough of certain devices.  So I can tell you there are
other users of ioreq outside of QEMU that do passthrough.

> If it didn't, no other 
> server would have opted in either, no? The only "sort of" issue would be 
> if someone wrote a secondary emulator that sets ext_dest_id to basically 
> "lock out" a QEMU that doesn't support the new XEN_DMs?

Ahead of the domain being created you need to figure out whether it's
safe to expose the feature across all registered ioreq server at that
point.  After that, if Extended Destination ID is exposed, only server
supporting it can be added for the lifetime of the domain.

You will also need to migrate this setting AFAICT, so the guest
doesn't see it disappearing when it resumes on a different host.

Hope this makes sense, let me know if you think some of the above is
wrong or undoable.

Thanks, Roger.