Re: [Embedded-pv-devel] Driver domain under Xen

[Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>]

Hi Julien,

On Fri, Jan 23, 2015 at 5:12 PM, Julien Grall <julien.grall@xxxxxxxxxx> wrote:
> On 23/01/15 15:03, Andrii Tseglytskyi wrote:
>> On Fri, Jan 23, 2015 at 4:48 PM, Julien Grall <julien.grall@xxxxxxxxxx> 
>> wrote:
>>> On 23/01/15 14:19, Andrii Tseglytskyi wrote:
>>>> Hi,
>>>
>>> Hi Andrii,
>>>
>>>> Xen related development in GlobalLogic is going with good progress, and
>>>> latest what we've done on our platform is *driver domain*.
>>>> Right now on our platform dom0 doesn't access any hardware (except SATA
>>>> and block device). Driver domain manages all hardware and run all
>>>> backend drivers.
>>>> This required several changes in hypervisor, toolstack, kernels,
>>>> filesystems, device trees etc. I would like to write an article abut our
>>>> experience of driver domain + backends bringup.
>>>>
>>>> Will this be interesting?
>>>
>>> It's a good blog post.
>>>
>>> OOI, what did you modify in the hypervisor and toolstack? I was able to
>>> create drivers domain without any modification aside my non-pci
>>> passthrough series.
>>>
>>
>> Briefly - I added possibility to map driver domain memory 1 to 1,
>> added possibility to allocate 128/256/512 Mb of RAM in one chunk, new
>> XSM label, and reworked IRQ routing a bit. It's great that this is
>
> If it's possible, I would be interested to have more detail about the
> IRQ routing changes now.
>
> I have few patches (part of non-pci passthrough series) which take care
> of IRQ routing. What is missing in thoses patches? If you are not using
> them, what did you modify?

OK. I'm sending you my local changes - 3 pieces of code.

Author: Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>
Date:   Tue Dec 30 18:38:55 2014 +0200

    xen/arm: always use count number to release hardware irq

    As soons as hardware interrupts are always mapped 1 to 1
    count number plus 32 is always equal to interrupt number which
    should be released. In current implementation if we don't receive
    any irq which number should be released existing p->irq number will
    be equal to 0, and interrupt will be not released properly.
    So, use lopp counter + 32 here.

    Change-Id: If89c928ff3ac5b83319d8b2b91439a7d598ed66d
    Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>

diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index e50e35d..aee2521 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -133,7 +133,7 @@ void domain_vgic_free(struct domain *d)

         if ( p->desc )
         {
-            ret = release_guest_irq(d, p->irq);
+            ret = release_guest_irq(d, i + 32);
             if ( ret )
                 dprintk(XENLOG_G_WARNING, "d%u: Failed to release
virq %u ret = %d\n",
                         d->domain_id, p->irq, ret);

commit 52db4eacef9c3ab050bba2548196bcaff657bc00
Author: Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>
Date:   Fri Aug 29 14:39:31 2014 +0300

    xen/arm: allow reassigning of hw interrupts to guest domain

    Patch allows reassigning of hardware interrupts from dom0 to
    other guest domain.

    Change-Id: Ie85485830d87b07393cc7fe7e64de5fd5f65ebb8
    Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>

diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index ba33571..018a2f6 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -455,12 +455,24 @@ int route_irq_to_guest(struct domain *d,
unsigned int virq,
             goto out;

         if ( test_bit(_IRQ_GUEST, &desc->status) )
-            dprintk(XENLOG_G_ERR, "IRQ %u is already used by domain %u\n",
-                    irq, ad->domain_id);
+        {
+            dprintk(XENLOG_G_DEBUG, "IRQ %u is reassigned from domain
%u to domain %u\n",
+                    irq, ad->domain_id, d->domain_id);
+
+            clear_bit(_IRQ_DISABLED, &desc->status);
+            retval = gic_remove_irq_from_guest(ad, irq, desc);
+            if ( retval )
+                dprintk(XENLOG_G_ERR, "failed to remove IRQ %u from
domain %u (%d)\n",
+                        irq, ad->domain_id, retval);
+            xfree(desc->action);
+            desc->action = NULL;
+        }
         else
+        {
             dprintk(XENLOG_G_ERR, "IRQ %u is already used by Xen\n", irq);
-        retval = -EBUSY;
-        goto out;
+            retval = -EBUSY;
+            goto out;
+        }
     }

     retval = __setup_irq(desc, 0, action);

commit 09f50d7864b249bc4d53cb68c5c409ce7e8560f3
Author: Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>
Date:   Tue Nov 25 19:27:44 2014 +0200

    xen/arm: map pv domain interrupts numbers one to one

    Currently driver domain uses hardware interrupts. They
    are mapped as SPIs to guest domains. Patch allows to use
    one to one mappings between hardware interrupts numbers
    and SPIs numbers.

    Change-Id: Ie5704c88979724b489cc33431bdf6a78ff03fe8c
    Signed-off-by: Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>

diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 653da31..e50e35d 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -66,7 +66,7 @@ int domain_vgic_init(struct domain *d)

     d->arch.vgic.ctlr = 0;

-    if ( is_hardware_domain(d) )
+    if ( is_hardware_domain(d) || is_pv_domain(d) )
         d->arch.vgic.nr_spis = gic_number_lines() - 32;
     else
         d->arch.vgic.nr_spis = 0; /* We don't need SPIs for the guest */
@@ -470,7 +470,7 @@ int vgic_allocate_virq(struct domain *d, unsigned int irq)
     int virq = -1;

     /* Hardware domain has IRQ mapped 1:1 */
-    if ( is_hardware_domain(d) )
+    if ( is_hardware_domain(d) || is_pv_domain(d) )
         return irq;

     spin_lock(&d->arch.vgic.lock);


>
> Regards,
>
> --
> Julien Grall



-- 

Andrii Tseglytskyi | Lead engineer
GlobalLogic
www.globallogic.com

_______________________________________________
Embedded-pv-devel mailing list
Embedded-pv-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/embedded-pv-devel