Re: [Xen-devel] cpufreq implementation for OMAP under xen hypervisor.

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Tue, 9 Sep 2014, Ian Campbell wrote:
> On Thu, 2014-09-04 at 22:56 +0100, Stefano Stabellini wrote:
> > I am trying to think of an alternative, such as passing the real cpu
> > nodes to dom0 but then adding status = "disabled", but I am not sure
> > whether Linux checks the status for cpu nodes.
> 
> status = "disabled" is defined to have a specific (i.e. non-default)
> meaning for cpu nodes, Julien mentioned this when I tried to add a
> similar patch to Xen to ignore them. I think it basically means "present
> but not running, you should start them!".
> 
> >  In addition this scheme
> > wouldn't support the case where dom0 has more vcpus than pcpus on the
> > system. Granted it is not very common and might even be detrimental for
> > performances, but we should be able to support it.
> 
> It's a bit of an edge case, for sure. I guess it wouldn't be totally
> unreasonable to say that if you use this sort of configuration you may
> not get cpufreq support.
> 
> > Ian, what do you think about this?
> 
> All the options suck in one way or another AFAICT. I think we are going
> to be looking for the least bad solution not necessarily a good one.
> 
> Fundamentally are we trying to avoid having to have a i2c subsystem etc
> in the hypervisor to be be able to change the voltages before/after
> changing the frequency?
>
> We can't just say "that's part of the cpufreq driver" since different
> boards using the same SoC might use different voltage regulators, over
> i2c or some other bus etc, so we end up with a matrix.
> 
> It's arguable that we should be letting dom0 poke at that regulator
> functionality anyway, at least not all of it. Taking that ability away
> would necessarily imply more platform specific functionality in the
> hypervisor.

Right.
I am afraid that in order to avoid more code in Xen, we end up with an
unmaintainable interface and unupstreamable hacks in dom0.


> How does this stuff work on x86?

As far as I can tell, after an initial version based on Dom0 doing the
work, the functionality has been moved into the hypervisor.
Of course doing that is easier on x86 where differences across
platforms are more limited.


> Sorry, more questions than answers here.
> 
> Ian.
> 
> > > Oleksandr Dmytryshyn | Product Engineering and Development
> > > GlobalLogic
> > > M +38.067.382.2525
> > > www.globallogic.com
> > > 
> > > http://www.globallogic.com/email_disclaimer.txt
> > > 
> > > 
> > > On Tue, Sep 2, 2014 at 9:46 PM, Andrii Tseglytskyi
> > > <andrii.tseglytskyi@xxxxxxxxxxxxxxx> wrote:
> > > >
> > > > Hi Stefano,
> > > >
> > > > Thank you for explanation.
> > > > I think this requires more and deeper investigation, but for sure dom0
> > > > must be able to do this.
> > > > Let us investigate this.
> > > >
> > > > Thank you,
> > > >
> > > > Regards,
> > > > Andrii
> > > >
> > > > On Tue, Sep 2, 2014 at 9:39 PM, Stefano Stabellini
> > > > <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> > > > > On Tue, 2 Sep 2014, Andrii Tseglytskyi wrote:
> > > > >> On Tue, Sep 2, 2014 at 4:00 AM, Stefano Stabellini
> > > > >> <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> > > > >> > On Fri, 29 Aug 2014, Andrii Tseglytskyi wrote:
> > > > >> >> Hi,
> > > > >> >>
> > > > >> >> Stefano, Ian,
> > > > >> >>
> > > > >> >> Could you please clarify the following point:
> > > > >> >>
> > > > >> >> I agree that decision about frequency change should be taken by 
> > > > >> >> Xen
> > > > >> >> hypervisor. But what about hardware frequency changing?
> > > > >> >> In general when frequency changed to bigger value (for example 
> > > > >> >> from 1
> > > > >> >> GHz to 1.5 GHz) for ARM kernels sequence looks like the following:
> > > > >> >>
> > > > >> >> 1) cpufreq governor decides that frequency should be changed. This
> > > > >> >> decision is taken after analysing of CPU performance data taking 
> > > > >> >> in
> > > > >> >> account governor policy.
> > > > >> >> 2) cpufreq governor asks cpufreq driver about new frequency.
> > > > >> >> 3) cpufreq driver compares current and target frequencies and asks
> > > > >> >> cpufreq regulator about voltage change.
> > > > >> >> 4) cpufreq regulator send i2c command to standalone microchip, 
> > > > >> >> which
> > > > >> >> is responsible for voltage changing.
> > > > >> >> 5) cpufreq driver asks clock framework about new frequency for 
> > > > >> >> CPU clock
> > > > >> >> 6) clock framework performs frequency sanity checks, taking in 
> > > > >> >> account
> > > > >> >> clock parents and clock divider settings, and call platform 
> > > > >> >> specific
> > > > >> >> "set_frequency" callback.
> > > > >> >> 7) platform specific callback performs proper HW registers
> > > > >> >> configuration for newly selected frequency
> > > > >> >>
> > > > >> >> Also there are some special cases - for example for OMAP5+ when
> > > > >> >> frequency is changed to 1.5 GHz+, two additional HW IPs should be
> > > > >> >> triggered (ABB and DCC, if someone is familiar with OMAP5+ )
> > > > >> >>
> > > > >> >> So, for generic ARM kernel we have 3 entities to change frequency:
> > > > >> >>
> > > > >> >> - cpufreq governor
> > > > >> >> - cpufreq driver
> > > > >> >> - cpufreq regulator
> > > > >> >>
> > > > >> >> + 2 additional IP for OMAP5+
> > > > >> >> - ABB
> > > > >> >> - DCC
> > > > >> >>
> > > > >> >> Taking in account all above, it looks like it would be better to
> > > > >> >> implement only Xen cpufreq governor. Xen will take a decision 
> > > > >> >> about
> > > > >> >> new frequency, and kernel dom0 will perform other steps. Dom0 
> > > > >> >> contains
> > > > >> >> all generic and platform specific frameworks, needed for frequency
> > > > >> >> changing.
> > > > >> >>
> > > > >> >> What do you think ?
> > > > >> >
> > > > >> > Keep in mind that the architecture must be able to handle the case 
> > > > >> > where
> > > > >> > dom0 has only 1 or 2 vcpus on a 4 or 8 cores system with multiple
> > > > >> > physical cpus.
> > > > >> > Could dom0 change the frequency of a physical core or a physical 
> > > > >> > cpu is
> > > > >> > not even running on? If that is not a problem, because cpus and
> > > > >> > frequency changing are decoupled enough in Linux to allow it, then 
> > > > >> > I am
> > > > >> > OK with it. But I suspect they are not.
> > > > >> >
> > > > >>
> > > > >> Not sure that I got your point correctly - dom0 will change frequency
> > > > >> on physical CPU.
> > > > >> And in case of OMAP - this changing affects on both ARM physical cpus
> > > > >> - changing is coupled.
> > > > >> In case of other ARM platforms - changing may be not coupled (I've
> > > > >> heard that Snapdragon can change cpu freqs independently on each
> > > > >> physical cpu)
> > > > >
> > > > > Let me explain with a concrete example.
> > > > >
> > > > > Let's suppose that the platform has 2 physical cpus, each cpu has 4
> > > > > cores.  Let's also supposed that dom0 has only 2 vcpus, currently
> > > > > running on core0 and core1 of cpu0.
> > > > >
> > > > > In this case would dom0 be able to change the frequency of core3 of
> > > > > cpu1, given that is not even running on it?
> > > > > If it can be done without any hacks, then we can go ahead with this
> > > > > approach.
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Andrii Tseglytskyi | Embedded Dev
> > > > GlobalLogic
> > > > www.globallogic.com
> > > 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel