|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] cpufreq implementation for OMAP under xen hypervisor.
The issue with that limitation is that it doesn't scale well on large systems. You really wouldn't want dom0 to have 18 vcpus on a Xeon E5 because it would badly affect performances. Even on a 8 cores SoC it would be best to assign less than 8 vcpus to dom0. On Wed, 10 Sep 2014, Vitaly Chernooky wrote: > I've intensively discussed my suggestions here and now it is transparent to > me that we should not try to use Cpufreq on ARM > SoCs without direct 1:1 pcpu:vcpu mapping in dom0. So if someone want to > break 1:1 mapping he should forget Cpufreq. > With best regards, > > > On Wed, Sep 10, 2014 at 12:58 AM, Stefano Stabellini > <stefano.stabellini@xxxxxxxxxxxxx> wrote: > On Tue, 9 Sep 2014, Vitaly Chernooky wrote: > > Hi All! > > > > On Fri, Sep 5, 2014 at 12:56 AM, Stefano Stabellini > <stefano.stabellini@xxxxxxxxxxxxx> wrote: > >   ÂOn Thu, 4 Sep 2014, Oleksandr Dmytryshyn wrote: > >   Â> Hi to all. > >   Â> > >   Â> I want to implement cpufreq driver in the next way: > >   Â> 1. Cpufreq governor will be implemented in the Xen > >   Â> 2. dom0 will only change cpu frequency and voltage of the > physical cpus > >   Â> > >   Â> But there are some nuances: > >   Â> 1. dom0 driver should read an information about operation > points > >   Â> (frequencies and voltages) and cpu supply source from the > device tree for each > >   Â> physical cpu. In the omap processor case this driver suspects > that > >   Â> those settings > >   Â> located in the /cpus/cpu@0/ node. But hypervisor creates an > cpu node > >   Â> for each vcpu > >   Â> for kernel dom0 in the device tree and those information is > lost in the dom0. > >   Â> 2. What about this case if we will have some physical cpus > with different > >   Â> operation points (for example 2 cpus) and we give only one > cpu for dom0? > >   Â> > >   Â> How should I transfer all information from the original > cpu@xxxxxx@n nodes > >   Â> about all physical cpus to the kernel dom0 driver? Maybe an > additional > >   Â> nodes should be created by the hypervisor in the device tree > for dom0 > >   Â> and named as pcpu@xxxxxxx@n? > > > >   ÂIf we do that, wouldn't we require changes to the core OMAP > drivers or > >   Âcpu initialization code in Linux (to parse "pcpu" instead of > "cpu" > >   Ânodes)? I don't expect they would be easy to upstream or > maintain going > >   Âforward. > > > >   ÂI am trying to think of an alternative, such as passing the > real cpu > >   Ânodes to dom0 but then adding status = "disabled", but I am not > sure > >   Âwhether Linux checks the status for cpu nodes. In addition this > scheme > >   Âwouldn't support the case where dom0 has more vcpus than pcpus > on the > >   Âsystem. Granted it is not very common and might even be > detrimental for > >   Âperformances, but we should be able to support it. > > > > > > In case where dom0 has more vcpus than pcpus on the > > system, the dom0 kernel is the most bug-prone place for pcpu cpufreq > governor. So I still believe that > separate driver > > domain with direct 1:1 vcpu:pcpu mapping is the best place for > cpufreq governor. But it also reasonable to run > cpufreq > > governor as userspace daemon in dom0. > > > > Also what do you think about PM QoS support? On bare metal cpufreq is > tightly integrated with PM QoS and > intensively > > cooperate in frequency scaling. > > Device PM needs to be done in Dom0. > CPU an Platform level PM architecturally belongs to Xen, but I do > understand that to do that in Xen we would need to add lots of code to > the hypervisor. There is no silver bullet here. > > A driver domain with 1:1 vcpu:pcpu mapping could work, but what kernel > are you going to use for that? Linux? Wouldn't Linux be too big for a > cpufreq driver domain, especially in embedded deployments? I think it > would need at least 32MB to run. > > > > With best regards, > >  > >   ÂIan, what do you think about this? > > > > > > > >   Â> Oleksandr Dmytryshyn | Product Engineering and Development > >   Â> GlobalLogic > >   Â> M +38.067.382.2525 > >   Â> www.globallogic.com > >   Â> > >   Â> http://www.globallogic.com/email_disclaimer.txt > >   Â> > >   Â> > >   Â> On Tue, Sep 2, 2014 at 9:46 PM, Andrii Tseglytskyi > >   Â> <andrii.tseglytskyi@xxxxxxxxxxxxxxx> wrote: > >   Â> > > >   Â> > Hi Stefano, > >   Â> > > >   Â> > Thank you for explanation. > >   Â> > I think this requires more and deeper investigation, but for sure > >dom0 > >   Â> > must be able to do this. > >   Â> > Let us investigate this. > >   Â> > > >   Â> > Thank you, > >   Â> > > >   Â> > Regards, > >   Â> > Andrii > >   Â> > > >   Â> > On Tue, Sep 2, 2014 at 9:39 PM, Stefano Stabellini > >   Â> > <stefano.stabellini@xxxxxxxxxxxxx> wrote: > >   Â> > > On Tue, 2 Sep 2014, Andrii Tseglytskyi wrote: > >   Â> > >> On Tue, Sep 2, 2014 at 4:00 AM, Stefano Stabellini > >   Â> > >> <stefano.stabellini@xxxxxxxxxxxxx> wrote: > >   Â> > >> > On Fri, 29 Aug 2014, Andrii Tseglytskyi wrote: > >   Â> > >> >> Hi, > >   Â> > >> >> > >   Â> > >> >> Stefano, Ian, > >   Â> > >> >> > >   Â> > >> >> Could you please clarify the following point: > >   Â> > >> >> > >   Â> > >> >> I agree that decision about frequency change should be > >taken by Xen > >   Â> > >> >> hypervisor. But what about hardware frequency changing? > >   Â> > >> >> In general when frequency changed to bigger value (for > >example from 1 > >   Â> > >> >> GHz to 1.5 GHz) for ARM kernels sequence looks like the > >following: > >   Â> > >> >> > >   Â> > >> >> 1) cpufreq governor decides that frequency should be > >changed. This > >   Â> > >> >> decision is taken after analysing of CPU performance data > >taking in > >   Â> > >> >> account governor policy. > >   Â> > >> >> 2) cpufreq governor asks cpufreq driver about new frequency. > >   Â> > >> >> 3) cpufreq driver compares current and target frequencies > >and asks > >   Â> > >> >> cpufreq regulator about voltage change. > >   Â> > >> >> 4) cpufreq regulator send i2c command to standalone > >microchip, which > >   Â> > >> >> is responsible for voltage changing. > >   Â> > >> >> 5) cpufreq driver asks clock framework about new frequency > >for CPU clock > >   Â> > >> >> 6) clock framework performs frequency sanity checks, taking > >in account > >   Â> > >> >> clock parents and clock divider settings, and call platform > >specific > >   Â> > >> >> "set_frequency" callback. > >   Â> > >> >> 7) platform specific callback performs proper HW registers > >   Â> > >> >> configuration for newly selected frequency > >   Â> > >> >> > >   Â> > >> >> Also there are some special cases - for example for OMAP5+ > >when > >   Â> > >> >> frequency is changed to 1.5 GHz+, two additional HW IPs > >should be > >   Â> > >> >> triggered (ABB and DCC, if someone is familiar with OMAP5+ ) > >   Â> > >> >> > >   Â> > >> >> So, for generic ARM kernel we have 3 entities to change > >frequency: > >   Â> > >> >> > >   Â> > >> >> - cpufreq governor > >   Â> > >> >> - cpufreq driver > >   Â> > >> >> - cpufreq regulator > >   Â> > >> >> > >   Â> > >> >> + 2 additional IP for OMAP5+ > >   Â> > >> >> - ABB > >   Â> > >> >> - DCC > >   Â> > >> >> > >   Â> > >> >> Taking in account all above, it looks like it would be > >better to > >   Â> > >> >> implement only Xen cpufreq governor. Xen will take a > >decision about > >   Â> > >> >> new frequency, and kernel dom0 will perform other steps. > >Dom0 contains > >   Â> > >> >> all generic and platform specific frameworks, needed for > >frequency > >   Â> > >> >> changing. > >   Â> > >> >> > >   Â> > >> >> What do you think ? > >   Â> > >> > > >   Â> > >> > Keep in mind that the architecture must be able to handle > >the case where > >   Â> > >> > dom0 has only 1 or 2 vcpus on a 4 or 8 cores system with > >multiple > >   Â> > >> > physical cpus. > >   Â> > >> > Could dom0 change the frequency of a physical core or a > >physical cpu is > >   Â> > >> > not even running on? If that is not a problem, because cpus > >and > >   Â> > >> > frequency changing are decoupled enough in Linux to allow > >it, then I am > >   Â> > >> > OK with it. But I suspect they are not. > >   Â> > >> > > >   Â> > >> > >   Â> > >> Not sure that I got your point correctly - dom0 will change > >frequency > >   Â> > >> on physical CPU. > >   Â> > >> And in case of OMAP - this changing affects on both ARM > >physical cpus > >   Â> > >> - changing is coupled. > >   Â> > >> In case of other ARM platforms - changing may be not coupled > >(I've > >   Â> > >> heard that Snapdragon can change cpu freqs independently on > >each > >   Â> > >> physical cpu) > >   Â> > > > >   Â> > > Let me explain with a concrete example. > >   Â> > > > >   Â> > > Let's suppose that the platform has 2 physical cpus, each cpu > >has 4 > >   Â> > > cores. Let's also supposed that dom0 has only 2 vcpus, > >currently > >   Â> > > running on core0 and core1 of cpu0. > >   Â> > > > >   Â> > > In this case would dom0 be able to change the frequency of > >core3 of > >   Â> > > cpu1, given that is not even running on it? > >   Â> > > If it can be done without any hacks, then we can go ahead with > >this > >   Â> > > approach. > >   Â> > > >   Â> > > >   Â> > > >   Â> > -- > >   Â> > > >   Â> > Andrii Tseglytskyi | Embedded Dev > >   Â> > GlobalLogic > >   Â> > www.globallogic.com > >   Â> > > > >   Â_______________________________________________ > >   ÂXen-devel mailing list > >   ÂXen-devel@xxxxxxxxxxxxx > >   Âhttp://lists.xen.org/xen-devel > > > > > > > > > > -- > > Vitaly Chernooky |ÂSenior Developer - Product Engineering and Development > > GlobalLogic > > P +380.44.4929695 ext.1136 M +380.98.7920568 S cvv_2k > > www.globallogic.com > > > > http://www.globallogic.com/email_disclaimer.txt > > > > > > > > > -- > Vitaly Chernooky |ÂSenior Developer - Product Engineering and Development > GlobalLogic > P +380.44.4929695 ext.1136 M +380.98.7920568 S cvv_2k > www.globallogic.com > > http://www.globallogic.com/email_disclaimer.txt > > _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |