Re: [Xen-devel] Xen optimization

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>]

On Thu, 11 Oct 2018, Milan Boberic wrote:
> On Wed, Oct 10, 2018 at 6:41 PM Meng Xu <xumengpanda@xxxxxxxxx> wrote:
> >
> > The jitter may come from Xen or the OS in dom0.
> > It will be useful to know what is the jitter if you run the test on 
> > PetaLinux.
> > (It's understandable the jitter is gone without OS. It is also common
> > that OS introduces various interferences.)
> 
> Hi Meng,
> well... I'm using bare-metal application and I need it exclusively to
> be ran on one CPU as domU (guest) without OS (and I'm not sure how
> would I make the same app to be ran on PetaLinux dom0 :D haha).
> Is there a chance that PetaLinux as dom0 is creating this jitter and
> how? Is there a way of decreasing it?
> 
> Yes, there are no prints.
> 
> I'm not sure about this timer interrupt passthrough because I didn't
> find any example of it, in attachment I included xen-overlay.dtsi file
> which I edited to add passthrough, in earlier replies there are
> bare-metal configuration file. It would be helpful to know if those
> setting are correct. If they are not correct it would explain the
> jitter.
> 
> Thanks in advance, Milan Boberic!

Hi Milan,

Sorry for taking so long to go back to this thread. But I am here now :)

First, let me ask a couple of questions to understand the scenario
better: is there any interference from other virtual machines while you
measure the jitter? Or is the baremetal app the only thing actively
running on the board?

Second, it would be worth double-checking that Dario's patch to fix
sched=null is not having unexpected side effects. I don't think so, it
would be worth testing with it and without it to be sure.

I gave a look at your VM configuration. The configuration looks correct.
There is no dtdev settings, but given that none of the devices you are
assigning to the guest does any DMA, it should be OK. You want to make
sure that Dom0 is not trying to use those same devices -- make sure to
add "xen,passthrough;" to each corresponding node on the host device
tree.

The error messages "No valid vCPU found" are due to the baremetal
applications trying to configure as target cpu for the interrupt cpu1
(the second cpu in the system), while actually only 1 vcpu is assigned
to the VM. Hence, only cpu0 is allowed. I don't think it should cause
any jitter issues, because the request is simply ignored. Just to be
safe, you might want to double check that the physical interrupt is
delivered to the right physical cpu, which would be cpu1 in your
configuration, the one running the only vcpu of the baremetal app. You
can do that by adding a printk to xen/arch/arm/vgic.c:vgic_inject_irq,
for example:

diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 5a4f082..208fde7 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -591,6 +591,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, 
unsigned int virq,
 out:
     spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
 
+    if (v != current) printk("DEBUG irq slow path!\n");
     /* we have a new higher priority irq, inject it into the guest */
     vcpu_kick(v);
 
You don't want "DEBUG irq slow path!" to get printed.

Finally, I would try to set the timer to generate events less frequently
than every 1us and see what happens, maybe every 5-10us. In my tests,
the IRQ latency overhead caused by Xen is around 1us, so injecting 1
interrupt every 1us, plus 1us of latency caused by Xen, cannot lead to
good results.

I hope this helps, please keep us updated with your results, they are
very interesting!

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel