Re: [Xen-devel] Xen ARM - Exposing a PL011 to the guest

[Bhupinder Thakur <bhupinder.thakur@xxxxxxxxxx>]

Hi,

Currently, xenconsoled supports only PV console. To support one
additional pl011 console, some of the data structures in struct domain
and certain functions such as buffer_append, ring_free_bytes, etc.
will have to be be scaled to support now two types of consoles.

Some changes are required in xenconsole client (client/maiun.c) and
tools/libxl/xl_cmdimpl.c to support one additional pl011 console.

I have done the above-mentioned changes and can now connect to both PV
and pl011 consoles.

Command for connecting to pl011 console:

xl console -t vpl011 <dom_name> connects to the pl011 console.

Currently, the console type definitions such as LIBXL_CONSOLE_TYPE_PV
are defined in /usr/include/_libxl_types.h, which is not part of the
xen tree. How should I add a new console type in this header file?

Thanks.

Regards,
Bhupinder

On 16 February 2017 at 02:51, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> On Wed, 15 Feb 2017, Bhupinder Thakur wrote:
>> On 15 February 2017 at 13:45, Bhupinder Thakur
>> <bhupinder.thakur@xxxxxxxxxx> wrote:
>> > Hi Stefano,
>> >
>> > On 14 February 2017 at 03:14, Stefano Stabellini <sstabellini@xxxxxxxxxx> 
>> > wrote:
>> >> On Mon, 13 Feb 2017, Bhupinder Thakur wrote:
>> >>> Hi Stefano,
>> >>>
>> >>> On 9 February 2017 at 05:40, Stefano Stabellini <sstabellini@xxxxxxxxxx> 
>> >>> wrote:
>> >>> > On Wed, 8 Feb 2017, Bhupinder Thakur wrote:
>> >>> >> Hi Julien,
>> >>> >>
>> >>> >> On 3 February 2017 at 19:38, Julien Grall <julien.grall@xxxxxxx> 
>> >>> >> wrote:
>> >>> >> > So if I understand correctly, you don't receive anymore output. 
>> >>> >> > Correct?
>> >>> >> > Have you tried to see whether the pl011 driver is receiving 
>> >>> >> > interrupt or
>> >>> >> > even Linux calling it.
>> >>> >>
>> >>> >> The pl011 driver is not receiving the interrupts (I put a
>> >>> >> HYPERVISOR_console_io() call inside the interrupt handler and I do not
>> >>> >> see the print on the xen console) . When the console is in this state,
>> >>> >> if I keep typing the characters on the console then Xen keeps raising
>> >>> >> the interrupts (for each character typed) but all those interrupts are
>> >>> >> marked as inflight. So it seems that interrupt is not getting cleared
>> >>> >> in GIC. Which GIC function can I look at to further debug what is the
>> >>> >> issue?
>> >>> >>
>> >>> >> Earlier, when I used to manually write characters to /dev/ttyAMA0 then
>> >>> >> I used to receive the interrupts all fine.
>> >>> >
>> >>> > DomU doesn't get emulated PL011 interrupts from Xen? Make sure you 
>> >>> > don't
>> >>> > mark the interrupt as a real hardware interrupt by mistake
>> >>> > (GICH_V2_LR_HW). At the same time, check that the virtual PL011
>> >>> > interrupt has been added correctly to the LRs
>> >>> > (xen/arch/arm/gic-v2.c:gicv2_update_lr).
>> >>>
>> >>> I verified that the PL011 interrupt is getting marked pending by Xen
>> >>> but guest does not receive it. I am not sure if the local interrupt
>> >>> delivery is disabled on the guest because of which it may not be
>> >>> receiving the interrupts. Is there a way to verify that interrupts are
>> >>> indeed disabled at that point. Since the interrupt delivery works fine
>> >>> when I manually read/write characters to /dev/ttyAMA0, I think there
>> >>> is no issue from the Xen side in terms of raising an interrupt.
>> >>>
>> >>> One observation is that pl011_request_port () calls request_mem_region
>> >>> () for the MMIO space allocated to pl011. The function returns -EBUSY
>> >>> indicating that the memory region is already in use. But when I do cat
>> >>> /proc/iomem, I do not see any overlaps. I am using 0x22000000 as the
>> >>> start of the pl011 MMIO space. I see the same error while booting the
>> >>> dom0 kernel also.
>> >>>
>> >>> I have attached the guest boot logs when console=ttyAMA0. I have added
>> >>> debug prints marked with [DEBUG]. I added debug prints in amba-pl011.c
>> >>> and serial_core.c. The output from the console seems to suddenly stop
>> >>> once the pl011 port is started. I have not added getty for ttyAMA0 in
>> >>> the /etc/inittab file. I believe auto-serial-console should be able to
>> >>> attach a getty session to ttyAMA0 automatically (i did try with adding
>> >>> a getty command in the inittab file but that did not solve the issue).
>> >>>
>> >>> Towards the end of the log, we can see that every time a character is
>> >>> typed, Xen raises an interrupt and the characters keep getting
>> >>> collected in the IN RING buffer (in_prod keeps incrementing). Later
>> >>> interrupts remain in inflight as the earlier interrupt is not cleared.
>> >>
>> >> Let me get this straight: who is printing all those messages we see on
>> >> the guest console (xl console guest) up until "Freeing unused kernel
>> >> memory"?  Is it the early pl011 driver? Or is it the Xen PV console
>> >> driver?
>> >>
>> >> Are you using a DEBUG build of Xen? Only DEBUG builds allow DomUs to
>> >> issue HYPERVISOR_console_io hypercalls. You might want to check that
>> >> do_console_io in Xen is letting DomU calls through.
>> >>
>> >> Otherwise pl011_int might actually be called but the printk might not
>> >> show up on the Xen console.
>> >>
>> >> The interrupt is enabled from Xen point of view, otherwise it would not
>> >> be added to the LR. However Linux might still refuse to call the
>> >> interrupt handler for some reason. You might want to add a printk in the
>> >> Linux generic interrupt handler, probably
>> >> kernel/irq/chip.c:handle_fasteoi_irq in your case.
>> >
>> > The pl011 console is working now. I did not fix any thing specific
>> > except I cleaned up the debug code which I had added. I am not sure if
>> > some of my debug code was causing the issue.
>> >
>> > After cleaning up the debug code and building the code afresh, I
>> > started receiving the interrupts and input/output on the console is
>> > working fine. Now I am able to execute commands on the console.
>> >
>> I believe since I was using printk in the pl011 console driver code
>> for debugging, that might be causing some infinite loop. Once I
>> replaced all printks with HYPERVISOR_console_io(), it started working.
>> Later I compiled out the debug code, and it was working fine.
>
> Yep, I tripped over that problem myself at least once or twice :-)

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel