Re: [Xen-devel] [PATCH v1 0/2] xen/arm: maintenance_interrupt SMP fix

[Julien Grall <julien.grall@xxxxxxxxxx>]

Hi,

It's weird, physical IRQ should not be injected twice ...
Were you able to print the IRQ number?

In any case, you are using the old version of the interrupt patch series.
Your new error may come of race condition in this code.

Can you try to use a newest version?

On 29 Jan 2014 18:40, "Oleksandr Tyshchenko" <oleksandr.tyshchenko@xxxxxxxxxxxxxxx> wrote:

> Right, that's why changing it to cpumask_of(0) shouldn't make any
> difference for xen-unstable (it should make things clearer, if nothing
> else) but it should fix things for Oleksandr.

Unfortunately, it is not enough for stable work.

I was tried to use cpumask_of(smp_processor_id()) instead of cpumask_of(0) in
gic_route_irq_to_guest(). And as result, I don't see our situation
which cause to deadlock in on_selected_cpus function (expected).
But, hypervisor sometimes hangs somewhere else (I have not identified
yet where this is happening) or I sometimes see traps, like that:
("WARN_ON(p->desc != NULL)" in maintenance_interrupt() leads to them)

(XEN) CPU1: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4-unstable Âarm32 Âdebug=y ÂNot tainted ]----
(XEN) CPU: Â Â1
(XEN) PC: Â Â 00242c1c __warn+0x20/0x28
(XEN) CPSR: Â 200001da MODE:Hypervisor
(XEN) Â Â ÂR0: 0026770c R1: 00000001 R2: 3fd2fd00 R3: 00000fff
(XEN) Â Â ÂR4: 00406100 R5: 40020ee0 R6: 00000000 R7: 4bfdf000
(XEN) Â Â ÂR8: 00000001 R9: 4bfd7ed0 R10:00000001 R11:4bfd7ebc R12:00000002
(XEN) HYP: SP: 4bfd7eb4 LR: 00242c1c
(XEN)
(XEN) Â VTCR_EL2: 80002558
(XEN) ÂVTTBR_EL2: 00020000dec6a000
(XEN)
(XEN) ÂSCTLR_EL2: 30cd187f
(XEN) Â ÂHCR_EL2: 00000000000028b5
(XEN) ÂTTBR0_EL2: 00000000d2014000
(XEN)
(XEN) Â ÂESR_EL2: 00000000
(XEN) ÂHPFAR_EL2: 0000000000482110
(XEN) Â Â ÂHDFAR: fa211190
(XEN) Â Â ÂHIFAR: 00000000
(XEN)
(XEN) Xen stack trace from sp=4bfd7eb4:
(XEN) Â Â0026431c 4bfd7efc 00247a54 00000024 002e6608 002e6608 00000097 00000001
(XEN) Â Â00000000 4bfd7f54 40017000 40005f60 40017014 4bfd7f58 00000019 00000000
(XEN) Â Â40005f60 4bfd7f24 00248e60 00000009 00000019 00404000 4bfd7f58 00000000
(XEN) Â Â00405000 000045f0 002e7694 4bfd7f4c 00248978 c0079a90 00000097 00000097
(XEN) Â Â00000000 fa212000 ea80c900 00000001 c05b8a60 4bfd7f54 0024f4b8 4bfd7f58
(XEN) Â Â00251830 ea80c950 00000000 00000001 c0079a90 00000097 00000097 00000000
(XEN) Â Âfa212000 ea80c900 00000001 c05b8a60 00000000 e9879e3c ffffffff b6efbca3
(XEN) Â Âc03b29fc 60000193 9fffffe7 b6c0bbf0 c0607500 c03b3140 e9879eb8 c007680c
(XEN) Â Âc060750c c03b32c0 c0607518 c03b3360 00000000 00000000 00000000 00000000
(XEN) Â Â00000000 00000000 3ff6bebf a0000113 800b0193 800b0093 40000193 00000000
(XEN) Â Âffeffbfe fedeefff fffd5ffe
(XEN) Xen call trace:
(XEN) Â Â[<00242c1c>] __warn+0x20/0x28 (PC)
(XEN) Â Â[<00242c1c>] __warn+0x20/0x28 (LR)
(XEN) Â Â[<00247a54>] maintenance_interrupt+0xfc/0x2f4
(XEN) Â Â[<00248e60>] do_IRQ+0x138/0x198
(XEN) Â Â[<00248978>] gic_interrupt+0x58/0xc0
(XEN) Â Â[<0024f4b8>] do_trap_irq+0x10/0x14
(XEN) Â Â[<00251830>] return_from_trap+0/0x4
(XEN)

Also I am posting maintenance_interrupt() from my tree:

static void maintenance_interrupt(int irq, void *dev_id, struct
cpu_user_regs *regs)
{
  int i = 0, virq, pirq;
  uint32_t lr;
  struct vcpu *v = current;
  uint64_t eisr = GICH[GICH_EISR0] | (((uint64_t) GICH[GICH_EISR1]) << 32);

  while ((i = find_next_bit((const long unsigned int *) &eisr,
               64, i)) < 64) {
    struct pending_irq *p, *n;
    int cpu, eoi;

    cpu = -1;
    eoi = 0;

    spin_lock_irq(&gic.lock);
    lr = GICH[GICH_LR + i];
    virq = lr & GICH_LR_VIRTUAL_MASK;

    p = irq_to_pending(v, virq);
    if ( p->desc != NULL ) {
      p->desc->status &= ~IRQ_INPROGRESS;
      /* Assume only one pcpu needs to EOI the irq */
      cpu = p->desc->arch.eoi_cpu;
      eoi = 1;
      pirq = p->desc->irq;
    }
    if ( !atomic_dec_and_test(&p->inflight_cnt) )
    {
      /* Physical IRQ can't be reinject */
      WARN_ON(p->desc != NULL);
      gic_set_lr(i, p->irq, GICH_LR_PENDING, p->priority);
      spin_unlock_irq(&gic.lock);
      i++;
      continue;
    }

    GICH[GICH_LR + i] = 0;
    clear_bit(i, &this_cpu(lr_mask));

    if ( !list_empty(&v->arch.vgic.lr_pending) ) {
      n = list_entry(v->arch.vgic.lr_pending.next, typeof(*n), lr_queue);
      gic_set_lr(i, n->irq, GICH_LR_PENDING, n->priority);
      list_del_init(&n->lr_queue);
      set_bit(i, &this_cpu(lr_mask));
    } else {
      gic_inject_irq_stop();
    }
    spin_unlock_irq(&gic.lock);

    spin_lock_irq(&v->arch.vgic.lock);
    list_del_init(&p->inflight);
    spin_unlock_irq(&v->arch.vgic.lock);

    if ( eoi ) {
      /* this is not racy because we can't receive another irq of the
      Â* same type until we EOI it. Â*/
      if ( cpu == smp_processor_id() )
        gic_irq_eoi((void*)(uintptr_t)pirq);
      else
        on_selected_cpus(cpumask_of(cpu),
                Âgic_irq_eoi, (void*)(uintptr_t)pirq, 0);
    }

    i++;
  }
}


Oleksandr Tyshchenko | Embedded Developer
GlobalLogic

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel