Re: [Xen-devel] [PATCH v1 4/4] xen/arm: Reuse R-Car Gen2 platform code for Stout board

[Oleksandr Tyshchenko <olekstysh@xxxxxxxxx>]

On Fri, Aug 10, 2018 at 3:50 PM, Julien Grall <julien.grall@xxxxxxx> wrote:
> Hi Oleksandr,
Hi Julien.

>
> On 08/10/2018 12:47 PM, Oleksandr Tyshchenko wrote:
>>
>> On Fri, Aug 10, 2018 at 12:44 PM, Julien Grall <julien.grall@xxxxxxx>
>> wrote:
>>>
>>> On 08/09/2018 07:18 PM, Oleksandr Tyshchenko wrote:
>>>>
>>>> On Thu, Aug 9, 2018 at 7:20 PM, Julien Grall <julien.grall@xxxxxxx>
>>>> wrote:
>>>>>
>>>>> On 08/09/2018 05:18 PM, Oleksandr Tyshchenko wrote:
>>>
>>> I am quite confused here. If a CPU will be in hypervisor mode for Xen,
>>> how
>>> come the same CPU will boot in monitor mode for Linux? Surely there
>>> should
>>> be no difference in term of boot here. Are you using different firmware
>>> here?
>>
>>
>> Yes, in order to run modified U-Boot is used. You can see here what
>> exactly was modified.
>> This instruction [1] contains link to patches [2] (see "Apply
>> additional patches" step)
>> which should be applied to bare U-Boot.
>>
>
> I don't think it is reasonable to request a specific U-boot just for Xen.
> What if the user decide to run Linux on baremetal? Will he have to switch
> back and forth between firmware?

Unfortunately, yes. At least for now.

>
>> [1]
>> https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Lager
>> [2]
>> https://lists.xenproject.org/archives/html/xen-devel/2015-01/msg02475.html
>>
>>>
>>>>
>>>> U-Boot brings non-boot cores up (using ported from Linux's
>>>> platsmp-apmu code functions). Then U-Boot switches all cores to HYP
>>>> mode.
>>>
>>>
>>>
>>> Who did the port? Is it upstream?
>>
>>
>> AFAIR Iurii Konovalenko did this port, he is also an author of
>> xen/arch/arm/platforms/rcar2.c.
>> I don't think U-Boot changes are upsteamed. But anybody who wants to
>> run Xen on Lager board is able (I think) to apply
>> published patches [2] to U-Boot and build it.
>> The same way anybody who will want to run Xen on Stout board will be able
>> to apply U-Boot patches I will attach to the future Wiki page for Stout
>> board.
>
>
> I am a bit concerned that patch for lager has never been upstreamed. Is it
> at least part of the Renesas BSP?

U-Boot patches for Lager board for bringing not-boot cores up and
switching into HYP mode have never been up-streamed.
No, Renesas BSP doesn't contain it either.

>
> What's the plan for Stout support?
>
>
>>>
>>>> While jump into Xen on a boot core, all non-boot cores are left in
>>>> U-Boot to wait to be "woken up by Xen" (actually rcar2_smp_init() is
>>>> used for).
>>>> Platform R-Car2 code in Xen doesn't power on cores, it just "brings
>>>> them to Xen".
>>>
>>>
>>>
>>> What does the address 0xe63c0000 corresponds too? From the DT it seems it
>>> is
>>> part of the ICRAM. But it is not clear why you write in there.
>>
>> Yes, it is a part of the ICRAM.
>>
>> Unfortunately, I am not an expert in such code to definitively say how
>> it is supposed to work.
>> But it seems that 0xE63C0FFC is an address which is being polled by
>> non-boot core in U-Boot. It was set to zero value
>> beforehand. So, writing an actual value to this address, Xen triggers
>> an action for bringing non-boot core up to Xen.
>>
>> Waiting loop in U-Boot:
>>
>> asm(".arm \n"
>> ".align 2 \n"
>> ".global smp_waitloop \n"
>> "smp_waitloop: \n"
>> "1: wfe \n"
>> "ldr r0, =0xE63C0FFC \n"
>> "ldr r0, [r0] \n"
>> "teq r0, #0x0 \n"
>> "beq 1b \n"
>>    "b _do_nonsec_entry \n"
>> ".type smp_waitloop, %function \n"
>> ".size smp_waitloop, .-smp_waitloop \n");
>>
>> Trigger code in Xen:
>>
>> writel(__pa(init_secondary), 0xE63C0FFC);
>>
>>>
>>> Could you point to any relevant documentation?
>>
>> I am afraid, I can't point to documentation.
>
>
> While the code was added for Lager and you just piggy-back on it,
> if you already hack U-boot to bring-up CPUs in hyp mode, then you better add
> PSCI support.
>
> To be honest, the lager support was likely accepted based on the code would
> make the way to Upstream. It is now been 3 years without any change. If
> there are no plan to address that, then I will send a patch to drop the
> support for Lager in Xen.

Definitely we would like to address that. I don't now exactly what
direction we will stick to,
either adding PSCI support for Gen2 boards in U-Boot (which will cover
Stout board) or
using some firmware in SW for providing PSCI services (ARM TF for ARM
V7 or even OP-TEE), etc

I think, I will come up with something more concrete after some time...
For this period current patch series is postponed.

>
> Cheers,
>
> --
> Julien Gral

-- 
Regards,

Oleksandr Tyshchenko

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel