Re: [Xen-devel] [RFC V2] xen: interface: introduce pvclk interface

[Peng Fan <van.freenix@xxxxxxxxx>]

Hi Ian and Stefano,

On Thu, Jan 21, 2016 at 04:11:45PM +0000, Stefano Stabellini wrote:
>On Thu, 21 Jan 2016, Ian Campbell wrote:
>> On Thu, 2016-01-21 at 12:55 +0000, Stefano Stabellini wrote:
>> > On Thu, 21 Jan 2016, Peng Fan wrote:
>> > > Hi Ian,
>> > > 
>> > > On Thu, Jan 21, 2016 at 12:26:04PM +0000, Ian Campbell wrote:
>> > > > On Thu, 2016-01-21 at 19:55 +0800, Peng Fan wrote:
>> > > > > Hi Ian,
>> > > > > 
>> > > > > On Thu, Jan 21, 2016 at 10:19:32AM +0000, Ian Campbell wrote:
>> > > > > > On Thu, 2016-01-21 at 16:59 +0800, Peng Fan wrote:
>> > > > > > >  
>> > > > > > > To platform device of ARM, hypervisor is responsible for the
>> > > > > > > mapping
>> > > > > > > between machine address and guest physical address, also
>> > > > > > > responsible
>> > > > > > > for the irq mapping.
>> > > > > > > 
>> > > > > > > But to embedded ARM SoC, the hardware clk IP is handled in
>> > > > > > > Dom0.
>> > > > > > 
>> > > > > > Arguably Xen ought to be the one to do this, but we have
>> > > > > > determined
>> > > > > > (rightly, I think) that doing so for the entirely clk tree of an
>> > > > > > SoC
>> > > > > > would
>> > > > > > involve importing an unmanageable amount of code into Xen[0].
>> > > > > > 
>> > > > > > In the meantime we defer this to Dom0.
>> > > > > > 
>> > > > > > I wonder though if it would be possible to manage the clocks for
>> > > > > > a
>> > > > > > passthrough device from the toolstack, i.e. is there a sysfs node
>> > > > > > where
>> > > > > > one
>> > > > > > can say "keep the clock for this device enabled (at xMhz) even
>> > > > > > though
>> > > > > > you
>> > > > > > think the device is unused"?
>> > > > > 
>> > > > > I am afraid not. The linux device driver without xen can work well,
>> > > > > because
>> > > > > there is clk tree and "clk_get,clk_prepare,clk_set_rate" work well
>> > > > > in the
>> > > > > driver.
>> > > > > I do not want to remove the clk apis usage from the device driver
>> > > > > for
>> > > > > xen, because the driver
>> > > > > also serves when no xen support. The pvclk interface is mainly to
>> > > > > let the
>> > > > > clk api can work as no xen.
>> > > > 
>> > > > Would adding a dummy fixed-clock[0] (or several of them) to the guest
>> > > > passthrough DT satisfy the driver's requirements? They would be
>> > > > hardcoded
>> > > > to whatever rate dom0 and/or the tools has decided upon (and had set
>> > > > in the
>> > > > real h/w).
>> > > 
>> > > If using this way, we have the assumption that DomU device driver would
>> > > not
>> > > change the rate of the clock driving the device. I am not sure whether
>> > > this is
>> > > ok for so many platform devices based ARM core.
>> > > 
>> > > In /sys/kernel/debug/clk/...., there are clk tree info, but
>> > > clk api are not exposed to userspace as far as I know, so
>> > > if using sysfs interface to set a known fixed rate or enable/disable
>> > > the clock,
>> > > we need to expose the clk info to userspace.
>> > 
>> > Keeping in mind that we might now want to let DomU change the actual
>> 
>> s/now/not/?
>
>Sorry, I meant "we might not want to let DomU change" ...

Why not want to let DomU change? I can not get this.

>
>
>> > clock frequency anyway, exposing a dummy clock to keep the DomU driver
>> > happy might be a good option.
>> > 
>> > However whether we set the frequency from the Dom0 kernel or from the
>> > toolstack, how do we find out the right clock rate? From a grep in the
>> > kernel sources, it looks like some drivers still have hardcoded values.
>> > 
>> > Asking the user to provide the clock rate seems a bit too much to me.
>> 
>> Remember that for this use case they already need to provide the host DT
>> path to the device, the list of iomem resources, the list of irqs and a
>> suitable device tree snippet.
>> 
>> platform device passthrough is very much an "expert" option (intended for
>> folks building embedded device, not really for normal end users), which
>> already needs a fair bit of familiarity with the system in question, I
>> don't think providing some info about the clocks goes too much further than
>> this.
>> 
>> Seamless & easy passthrough is something we are aiming for for PCI, but not
>> for the general case of platform passthrough.
>> 
>> Making platform device passthrough as easy as PCI passthrough is much
>> harder and far reaching than avoiding the need to specify some clock
>> frequencies (remember we had very long discussions about the design and
>> couldn't come up with anything especially feasible).
>
>I agree that passthrough of non-PCI devices is a feature for "experts"
>and we don't have many good solutions to this problem.
>
>But one thing is asking for a device tree snippet, another is asking
>for the clock frequency: as an expert user, finding out the device tree
>node can be done by reading the device tree, the docs or googling for
>it, but how is a user supposed to find out the clock frequency?  I don't
>think that asking expert users to grep for values in the Linux source tree 
>should be an option. Is the frequency at least supposed to be documented
>in the hardware manual?

I just asked our module owners about the clock rate. The rate for different
device is fixed most of the time, except runs into suspend state.
And the value for the rate is recommended by IC owner.

I think I can try the fixed clock way, if this is the better one, and we can 
ignore the runtime device clock change.

1. expose the kernel clk api to userspace, dir /sys/kernel/debug/clk/xxx
2. Introduce a entry in xl configuration file, like this 
clks=["name=uart_root_clk, rate=24000000", "name=gpu_root_clk,rate=400000000"].
2. Let libxl parse the clks entry and construct the clock node for DomU dts, 
and marked as fixed clock.
3. In DomU kernel, wrote a xen clk driver to serve the clk api usage in 
platform device drivers.

I am not sure whether vfio can be used here or not.

Any comments on this?

Thanks,
Peng.

>Should we have a list of known devices and frequencies in libxl?  Maybe
>not, but we could consider giving users the option to allow the DomU to
>set the frequency. Maybe we could introduce something like:
>"xen,passthrough,allow-clock-change".
>
>I am not arguing for one specific solution, I am just brainstorming.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel