Re: [Xen-devel] [PATCH] x86: correct socket_cpumask allocation for AP

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On Wed, 2015-07-08 at 13:38 +0100, Jan Beulich wrote:
> >>> On 08.07.15 at 11:36, <chao.p.peng@xxxxxxxxxxxxxxx> wrote:
> > @@ -84,11 +85,21 @@ void *stack_base[NR_CPUS];
> >  static void smp_store_cpu_info(int id)
> >  {
> >      struct cpuinfo_x86 *c = cpu_data + id;
> > +    unsigned int socket;
> >  
> >      *c = boot_cpu_data;
> >      if ( id != 0 )
> > +    {
> >          identify_cpu(c);
> >  
> > +        socket = cpu_to_socket(id);
> > +        if ( !socket_cpumask[socket] )
> > +        {
> > +            socket_cpumask[socket] = secondary_socket_cpumask;
> > +            secondary_socket_cpumask = NULL;
> 
> I don't think this will build with small enough NR_CPUS.
>
And it *does* *not* fix the issue on my box.

In fact, even with this applied, I see the following splat, on CPU #0,
as you can see:

...
(XEN) I/O virtualisation enabled
(XEN) Interrupt remapping enabled
(XEN) CPU0: Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz stepping 02
(XEN) ----[ Xen-4.6-unstable  x86_64  debug=y  Tainted:    C ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d08018a52a>] set_cpu_sibling_map+0x65/0x396
(XEN) RFLAGS: 0000000000010287   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: ffff82d080332240   rcx: 0000000000000000
(XEN) rdx: 0000000000000001   rsi: 000000000000000a   rdi: ffff82d08029b768
(XEN) rbp: ffff82d0802efe28   rsp: ffff82d0802efdd8   r8:  ffff82d0802d5c40
(XEN) r9:  0000000000000001   r10: 0000000000000002   r11: 0000000000000001
(XEN) r12: 0000000000000000   r13: ffff82d0803321e0   r14: ffff82d0803321e0
(XEN) r15: 000000000000000f   cr0: 000000008005003b   cr4: 00000000000006e0
(XEN) cr3: 00000000dba9a000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen stack trace from rsp=ffff82d0802efdd8:
(XEN)    ffff82d0802efe28 0000000000000020 ffff8303206c3610 ffff82d000000000
(XEN)    ffff82d0803321e0 ffff82d080332240 ffff8303206c3610 ffff82d0803321e0
(XEN)    ffff82d0803321e0 000000000000000f ffff82d0802efe48 ffff82d0802c14b3
(XEN)    ffff82d0802e8000 ffff82d0802e8000 ffff82d0802eff08 ffff82d0802c0d5a
(XEN)    0000000000000000 ffff82d0802dd900 0000000001478000 0000000000324000
(XEN)    0080016300000000 0000000001478001 0000000000000000 0000000000000015
(XEN)    ffff830000079c80 ffff830000079ee0 0000000100000002 ffff830000079fb0
(XEN)    0000000800000000 000000010000006e 0000000000000003 00000000000002f8
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff82d080100073
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 ffff8300dbdf4000 0000000000000000
(XEN)    0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82d08018a52a>] set_cpu_sibling_map+0x65/0x396
(XEN)    [<ffff82d0802c14b3>] smp_prepare_cpus+0x12e/0x24f
(XEN)    [<ffff82d0802c0d5a>] __start_xen+0x1ff3/0x2536
(XEN)    [<ffff82d080100073>] __high_start+0x53/0x55
(XEN) 
(XEN) Pagetable walk from 0000000000000000:
(XEN)  L4[0x000] = 0000000320734063 ffffffffffffffff
(XEN)  L3[0x000] = 0000000320733063 ffffffffffffffff
(XEN)  L2[0x000] = 0000000320732063 ffffffffffffffff 
(XEN)  L1[0x000] = 0000000000000000 ffffffffffffffff
(XEN) 
Jan
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel
> 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0002]
(XEN) Faulting linear address: 0000000000000000
(XEN) ****************************************
...

The configuration of the testbox in question is as follows:

root@Zhaman:~# xl info -n
...
machine                : x86_64
nr_cpus                : 16
max_cpu_id             : 63
nr_nodes               : 2
cores_per_socket       : 4
threads_per_core       : 2
...
cpu_topology           :
cpu:    core    socket     node
  0:       0        1        0
  1:       0        1        0
  2:       1        1        0
  3:       1        1        0
  4:       9        1        0
  5:       9        1        0
  6:      10        1        0
  7:      10        1        0
  8:       0        0        1
  9:       0        0        1
 10:       1        0        1
 11:       1        0        1
 12:       9        0        1
 13:       9        0        1
 14:      10        0        1
 15:      10        0        1

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel