Re: [PATCH v3 19/22] x86/pv: Guest exception handling in FRED mode

[Jan Beulich <jbeulich@xxxxxxxx>]

On 04.10.2025 00:53, Andrew Cooper wrote:
> Under FRED, entry_from_pv() handles everything.  To start with, implement
> exception handling in the same manner as entry_from_xen(), although we can
> unconditionally enable interrupts after the async/fatal events.
> 
> After entry_from_pv() returns, test_all_events() needs to run to perform
> exception and interrupt injection.  Split entry_FRED_R3() into two and
> introduce eretu_exit_to_guest() as the latter half, coming unilaterally from
> restore_all_guest().
> 
> For all of this, there is a slightly complicated relationship with CONFIG_PV.
> entry_FRED_R3() must exist irrespective of CONFIG_PV, because it's the
> entrypoint registered with hardware.  For simplicity, entry_from_pv() is
> always called, but it collapses into fatal_trap() in the !PV case.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>

Reviewed-by: Jan Beulich <jbeulich@xxxxxxxx>

Nevertheless ...

> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -2266,9 +2266,82 @@ void asmlinkage check_ist_exit(const struct 
> cpu_user_regs *regs, bool ist_exit)
>  
>  void asmlinkage entry_from_pv(struct cpu_user_regs *regs)
>  {
> +    struct fred_info *fi = cpu_regs_fred_info(regs);
> +    uint8_t type = regs->fred_ss.type;
> +    uint8_t vec = regs->fred_ss.vector;
> +
>      /* Copy fred_ss.vector into entry_vector as IDT delivery would have 
> done. */
> -    regs->entry_vector = regs->fred_ss.vector;
> +    regs->entry_vector = vec;
> +
> +    if ( !IS_ENABLED(CONFIG_PV) )
> +        goto fatal;
> +
> +    /*
> +     * First, handle the asynchronous or fatal events.  These are either
> +     * unrelated to the interrupted context, or may not have valid context
> +     * recorded, and all have special rules on how/whether to re-enable IRQs.
> +     */
> +    switch ( type )
> +    {
> +    case X86_ET_EXT_INTR:
> +        return do_IRQ(regs);
> +
> +    case X86_ET_NMI:
> +        return do_nmi(regs);
> +
> +    case X86_ET_HW_EXC:
> +        switch ( vec )
> +        {
> +        case X86_EXC_DF: return do_double_fault(regs);
> +        case X86_EXC_MC: return do_machine_check(regs);
> +        }
> +        break;
> +    }
> +
> +    /*
> +     * With the asynchronous events handled, what remains are the synchronous
> +     * ones.  PV guest context always had interrupts enabled.
> +     */
> +    local_irq_enable();
> +
> +    switch ( type )
> +    {
> +    case X86_ET_HW_EXC:
> +    case X86_ET_PRIV_SW_EXC:
> +    case X86_ET_SW_EXC:
> +        switch ( vec )
> +        {
> +        case X86_EXC_PF:  handle_PF(regs, fi->edata); break;
> +        case X86_EXC_GP:  do_general_protection(regs); break;
> +        case X86_EXC_UD:  do_invalid_op(regs); break;
> +        case X86_EXC_NM:  do_device_not_available(regs); break;
> +        case X86_EXC_BP:  do_int3(regs); break;
> +        case X86_EXC_DB:  handle_DB(regs, fi->edata); break;
> +        case X86_EXC_CP:  do_entry_CP(regs); break;
> +
> +        case X86_EXC_DE:
> +        case X86_EXC_OF:
> +        case X86_EXC_BR:
> +        case X86_EXC_NP:
> +        case X86_EXC_SS:
> +        case X86_EXC_MF:
> +        case X86_EXC_AC:
> +        case X86_EXC_XM:
> +            do_trap(regs);
> +            break;
>  
> +        default:
> +            goto fatal;
> +        }
> +        break;
> +
> +    default:
> +        goto fatal;
> +    }
> +
> +    return;
> +
> + fatal:
>      fatal_trap(regs, false);
>  }

... I'm still somewhat bothered by this almost entirely duplicating the
other entry function, i.e. I continue to wonder if we wouldn't be better
off by eliminating that duplication (say by way of an always_inline
helper with a suitable extra parameter).

> --- a/xen/arch/x86/x86_64/entry.S
> +++ b/xen/arch/x86/x86_64/entry.S
> @@ -63,7 +63,7 @@ UNLIKELY_END(syscall_no_callback)
>          /* Conditionally clear DF */
>          and   %esi, UREGS_eflags(%rsp)
>  /* %rbx: struct vcpu */
> -test_all_events:
> +LABEL(test_all_events, 0)
>          ASSERT_NOT_IN_ATOMIC
>          cli                             # tests must not race interrupts
>  /*test_softirqs:*/
> @@ -152,6 +152,8 @@ END(switch_to_kernel)
>  FUNC_LOCAL(restore_all_guest)
>          ASSERT_INTERRUPTS_DISABLED
>  
> +        ALTERNATIVE "", "jmp eretu_exit_to_guest", X86_FEATURE_XEN_FRED
> +
>          /* Stash guest SPEC_CTRL value while we can read struct vcpu. */
>          mov VCPU_arch_msrs(%rbx), %rdx

I also continue to wonder if we wouldn't do a tiny bit better by using

        ALTERNATIVE "mov VCPU_arch_msrs(%rbx), %rdx", \
                    "jmp eretu_exit_to_guest", \
                    X86_FEATURE_XEN_FRED

Or by converting the few jumps to restore_all_guest to alternatives
(duplicating the ASSERT_INTERRUPTS_DISABLED there).

Jan