Xen vs Trenchboot: TXT AP bringup

[Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>]

Hello,

Krystian is working on the next bit of the Trenchboot project's plans,
specifically getting Xen able to operate with TXT directly (i.e. without
tboot in the middle).

The way TXT releases APs back to the BSP is causing some fun.  All APs
release simultaneously, at a point of the BSP's choosing.  (SDM Vol2 6.2)

While the APs do release in 32bit flat unpaged mode (so much nicer than
16bit mode), it is still inherently a parallel boot, and the APIC_ID is
the only sensible piece of uniquely identifying information.


The simple bodge is probably to have a variable similar to smpboot's
booting_cpu, where all APs spin until they find their own APIC_ID.  This
serialises the APs around the existing boot sequence.


However, we're also very close to supporting parallel boot.  The
serialising point we currently have is __high_start loading %rsp from
stack_start, because that's a single pointer adjusted by do_boot_cpu(). 
Everything else, even the processor's idea of Xen's CPU id is derived
from the stack; specifically in the cpu_info block on the top of the
stack, which is prepared by the BSP.

All we need to do full parallel boot is an array mapping APIC_ID -> Xen
CPU index.  The AP boot path doesn't use the trampoline stack at all. 
Given some way of transforming APIC_ID into a Xen CPU index, the APs
could pull their stack pointer straight out of the stack_bases[] array. 
This would allow us in the general case to boot APs using an
all-but-self INIT-SIPI-SIPI.  (This is how firmware does AP bringup anyway.)


Thoughts?  In particular, is there an obvious option I've missed[1]?

~Andrew

[1] There is another way of doing this with multiple copies of the AP
trampoline at different positions, where trampoline_realmode_entry() can
derive the Xen CPU ID from %cs, but this is already horrible and isn't a
sensible approach to consider.