Re: [Xen-devel] Question about running Xen on NVIDIA Jetson-TK1

[Meng Xu <xumengpanda@xxxxxxxxx>]

On Mon, May 16, 2016 at 7:27 PM, Kyle Temkin <ktemkin@xxxxxxxxxxxxxx> wrote:
> Hi, Meng:
>

Hi Kyle,

> Julien is correct-- a coworker and I are working on support for Tegra
> SoCs, and we've made pretty good progress; there's work yet to be
> done, but we have dom0 and guests booting on the Jetson TK1, Jetson
> TX1, and the Google Pixel C. We hope to get a patch set out soon--
> unfortunately, our employer has to take some time to verify that
> everything's okay to be open-sourced, so I can't send out our
> work-in-progress just yet. We'll have an RFC patchset out soon, I
> hope!

Looking forward to your RFC patchset... Could you please cc. me when
you send out your RFC patchset. I really love to have a look at (maybe
review) it.

>
> There are two main hardware differences that cause Tegra SoCs to have
> trouble with Xen:
>
> - The primary interrupt controller for those systems isn't a single
> GIC, as Xen expects. Instead, there's an NVIDIA Legacy Interrupt
> Controller (LIC, or ICTLR) that gates all peripheral interrupts before
> passing them to a standard GICv2. This interrupt controller has to be
> programmed to ensure Xen can receive interrupts from the hardware
> (e.g. serial), programmed to ensure that interrupts for pass-through
> devices are correctly unmasked, and virtualized so dom0 can program
> the "sections" related to interrupts not being routed to Xen or to a
> domain for hardware passthrough.
>
> - The serial controller on the Tegra SoCs doesn't behave in the same
> was as most NS16550-compatibles; it actually adheres to the NS16550
> spec a little more rigidly than most compatible controllers. A
> coworker (Chris Patterson, cc'd) figured out what was going on; from
> what I understand, most 16550s generate the "transmit ready" interrupt
> once, when the device first can accept new FIFO entries. Both the
> original 16550 and the Tegra implementation generate the "transmit
> ready" interrupt /continuously/ when there's space available in the
> FIFO, slewing the CPU with a stream of constant interrupts.

I see. Thank you very much for explaining this so clearly! :-)

>
> What you're seeing is likely a symptom of the first difference. In
> your logs, you see messages that indicate Xen is having trouble
> correctly routing IRQ that are parented by the legacy interrupt
> controller:
>
>> irq 0 not connected to primary controller.Connected to 
>> /interrupt-controller@60004000

Right. I see the root issue now.  Thank you so much for pointing it out!

>
> The issue here is that Xen is currently explicitly opting not to route
> legacy-interrupt-controller interrupts, as they don't belong to the
> primary GIC. As a result, these interrupts never make it to dom0. The
> logic that needs to be tweaked is here:
>
> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/domain_build.c;h=00dc07af637b67153d33408f34331700dff84f93;hb=HEAD#l1137
>
> We re-write this logic in our forthcoming patch-set to be more
> general. As an interim workaround, you might opt to rewrite that logic
> so LIC interrupts (which have an interrupt-parent compatible with
> "tegra124-ictlr", in your case) can be routed by Xen, as well. Off the
> top of my head, a workaround might look like:
>
> /*
> * Don't map IRQ that have no physical meaning
> * ie: IRQ whose controller is not the GIC
> */
> - if ( rirq.controller != dt_interrupt_controller )
> +if ( (rirq.controller != dt_interrupt_controller) &&
> (!dt_device_is_compatible(rirq.controller, "tegra124-ictlr") )

It should have "nvidia" before "tegra124-ictlr". ;-)
After change it to
!dt_device_is_compatible(rirq.controller, "nvidia , tegra124-ictlr")

dom0 boots up~~~ :-D

>
> Of course, that's off-the-cuff code I haven't tried, but hopefully it
> should help to get you started.

Sure! It does work and get me started! I really appreciate your help
and explanation!

Looking forward to your RFC patch set. :-)

Thank you again for your help and time in this issue! It helps a lot!

Best Regards,

Meng
-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel