Re: [Xen-devel] [RFC 0/5] xen/arm: support big.little SoC

[Peng Fan <van.freenix@xxxxxxxxx>]

Hello Julien,

On Mon, Sep 19, 2016 at 10:09:06AM +0200, Julien Grall wrote:
>Hello Peng,
>
>On 19/09/2016 04:08, van.freenix@xxxxxxxxx wrote:
>>From: Peng Fan <peng.fan@xxxxxxx>
>>
>>This patchset is to support XEN run on big.little SoC.
>>The idea of the patch is from
>>"https://lists.xenproject.org/archives/html/xen-devel/2016-05/msg00465.html";
>>
>>There are some changes to cpupool and add x86 stub functions to avoid build
>>break. Sending The RFC patchset out is to request for comments to see whether
>>this implementation is acceptable or not. Patchset have been tested based on
>>xen-4.8 unstable on NXP i.MX8.
>>
>>I use Big/Little CPU and cpupool to explain the idea.
>>A pool contains Big CPUs is called Big Pool.
>>A pool contains Little CPUs is called Little Pool.
>>If a pool does not contains any physical cpus, Little CPUs or Big CPUs
>>can be added to the cpupool. But the cpupool can not contain both Little
>>and Big CPUs. The CPUs in a cpupool must have the same cpu type(midr value 
>>for ARM).
>>CPUs can not be added to the cpupool which contains cpus that have different 
>>cpu type.
>>Little CPUs can not be moved to Big Pool if there are Big CPUs in Big Pool,
>>and versa. Domain in Big Pool can not be migrated to Little Pool, and versa.
>>When XEN tries to bringup all the CPUs, only add CPUs with the same cpu 
>>type(same midr value)
>>into cpupool0.
>
>As mentioned in the mail you pointed above, this series is not enough to make
>big.LITTLE working on then. Xen is always using the boot CPU to detect the
>list of features. With big.LITTLE features may not be the same.
>
>And I would prefer to see Xen supporting big.LITTLE correctly before
>beginning to think to expose big.LITTLE to the userspace (via cpupool)
>automatically.

Do you mean vcpus be scheduled between big and little cpus freely?

This patchset is to use cpupool to block the vcpu be scheduled between big and
little cpus.

>
>See for instance v->arch.actlr = READ_SYSREG32(ACTLR_EL1).

Thanks for this. I only expose cpuid to guest, missed actlr. I'll check
the A53 and A72 TRM about AArch64 implementationd defined registers.
This actlr can be added to the cpupool_arch_info as midr.

Reading "vcpu_initialise", seems only MIDR and ACTLR needs to be handled.
Please advise if I missed anything else.

>
>>
>>Thinking an SoC with 4 A53(cpu[0-3]) + 2 A72(cpu[4-5]), cpu0 is the first one
>>that boots up. When XEN tries to bringup secondary CPUs, add cpu[0-3] to
>>cpupool0 and leave cpu[4-5] not in any cpupool. Then when Dom0 boots up,
>>`xl cpupool-list -c` will show cpu[0-3] in Pool-0.
>>
>>Then use the following script to create a new cpupool and add cpu[4-5] to
>>the cpupool.
>> #xl cpupool-create name=\"Pool-A72\" sched=\"credit2\"
>> #xl cpupool-cpu-add Pool-A72 4
>> #xl cpupool-cpu-add Pool-A72 5
>> #xl create -d /root/xen/domu-test pool=\"Pool-A72\"
>
>I am a bit confused with these runes. It means that only the first kind of
>CPUs have pool assigned. Why don't you directly create all the pools at boot
>time?

If need to create all the pools, need to decided how many pools need to be 
created.
I thought about this, but I do not come out a good idea.

The cpupool0 is defined in xen/common/cpupool.c, if need to create many pools,
need to alloc cpupools dynamically when booting. I would not like to change a
lot to common code.

The implementation in this patchset I think is an easy way to let Big and Little
CPUs all run.

>
>Also, in which pool a domain will be created if none is specified?
>
>>Now `xl cpupool-list -c` shows:
>>Name            CPU list
>>Pool-0          0,1,2,3
>>Pool-A72        4,5
>>
>>`xl cpupool-list` shows:
>>Name               CPUs   Sched     Active   Domain count
>>Pool-0               4    credit       y          1
>>Pool-A72             2   credit2       y          1
>>
>>`xl cpupool-cpu-remove Pool-A72 4`, then `xl cpupool-cpu-add Pool-0 4`
>>not success, because Pool-0 contains A53 CPUs, but CPU4 is an A72 CPU.
>>
>>`xl cpupool-migrate DomU Pool-0` will also fail, because DomU is created
>>in Pool-A72 with A72 vcpu, while Pool-0 have A53 physical cpus.
>>
>>Patch 1/5:
>>use "cpumask_weight(cpupool0->cpu_valid);" to replace "num_online_cpus()",
>>because num_online_cpus() counts all the online CPUs, but now we only
>>need Big or Little CPUs.
>
>So if I understand correctly, if the boot CPU is a little CPU, DOM0 will
>always be able to only use little ones. Is that right?

Yeah. Dom0 only use the little ones.

Thanks,
Peng.
>
>Regards,
>
>-- 
>Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel