[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: static-mem preventing dom0 from booting


  • To: Stefano Stabellini <sstabellini@xxxxxxxxxx>
  • From: Penny Zheng <Penny.Zheng@xxxxxxx>
  • Date: Fri, 5 Nov 2021 03:23:17 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+pzeIrlEfGm/kn9oItGNutQTWoFYDoC1gT21Ki9oJd0=; b=D5+QI/fMJktWhob4tDizm/jM1L0IvJQpLs4pj0jldcJWwMWvqF4YHdezvesTb9EMAkg+Q0BFwlBVPaxcRR9F2BagSoPtA3gSRdAicV6Oss9pKkhfaBj2GQ8EkoTMxq69ZpAoMcc7aO6lKUmWgsAdCnEGF4yzlXdccdHyezaWjsxGEiFYt3FE4E7mWTIgNebzFYLAuuO0on3uG9n/pTrszKO/x/VUBszS5wxTK5udCc8LWjGldQGaniP5VLY1/h4HnJsfYEFIiSgvwh5UKeu31C7eSRlQdA10uKdmdYZzqKm56zJ7V3/CKf9Yx9Zeo6HPN7G+R3F4OuRVZC3dKWIhtQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LCbpePdUiN3gTeoShFPKIfMnWzQJq5qxKKq5ysAAoQbx6h/cDsvEFPjW0xUzF2kXW7sKsfz1DrOgYHjBEHEcsl7UTdLpEAF/71VILqmBwNQOSI3CkoTs0TFzbiTz9KDzcHao2rzIhEnVSAhgS67esYbp64rf7uudjZdm4xktAVTgzTx+/80kQr412DAWHnA4rKl0oOJHCyPRjZLOip5wJW8ZksdR6HFz3WUhQPX2N50r2O0c8j03RAieZWeWDxujLfTWXlvPKzKfjdHDbwbRLyFzpMR+NXO4nGtNnJp1ASQL8WvPpfv1cS/MLDG1ExXiS67j3ZxqrgMdaKXv72B03w==
  • Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Cc: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Wei Chen <Wei.Chen@xxxxxxx>
  • Delivery-date: Fri, 05 Nov 2021 03:23:53 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Thread-index: AQHX0eWC7qpRq+VAt0Cu1/k92WugAqv0ODjw
  • Thread-topic: static-mem preventing dom0 from booting

Hi Stefano

> -----Original Message-----
> From: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> Sent: Friday, November 5, 2021 9:36 AM
> To: Penny Zheng <Penny.Zheng@xxxxxxx>
> Cc: sstabellini@xxxxxxxxxx; xen-devel@xxxxxxxxxxxxxxxxxxxx
> Subject: static-mem preventing dom0 from booting
> 
> Hi Penny,
> 
> I am trying to test the static-mem feature in Xen 4.16 but I am having some
> issues. I can boot a dom0less domU with static-mem configured correctly, but
> when I do so, dom0 doesn't boot any longer.
> 

Hmm, In our first intention, dom0less is a mandatory option for static 
allocation.
domU on static allocation shall boot when dom0 doesn't boot there.

We think that, nevertheless, dom0 memory range is allocated by Xen 
automatically,
and it leads to the unpredictability. Static allocation through device tree 
configuration
prefers the total static environment to avoid unpredictability.

> In the same configuration, if I remove the static-mem related lines from the
> domU node on device tree, both dom0 and domU boot successfully.
> 
> I am sure the dom0 memory range, allocated by Xen automatically, and the
> domU memory range, hand-picked by me, do not clash as you can see from the
> boot logs (appended).
> 
> Am I missing anything in the configuration? I am using a ZCU102 board, the
> memory node on the host device tree is:
> 

Are you suggesting one scenario where dom0 with static-mem domU?

Hmmm, one quick thought, it may not be working with vpl011 emulation.
When dom0 exists, vpl011 will take dom0 as backend, which requests event
channel, xen store, etc, involved to communicate. but static-mem domU
without CONFIG_XEN shall not handle it.

>       memory {
>               device_type = "memory";
>               reg = <0x00 0x00 0x00 0x7ff00000 0x08 0x00 0x00 0x80000000>;
>       };
> 
> I am attaching my u-boot boot.scr script with the device tree modification to
> add static memory.
> 
> Many thanks for your help!
> 
> Cheers,
> 
> Stefano
> 
> 
> (XEN) Xen version 4.16-rc (sstabellini@) (gcc (Ubuntu 9.3.0-17ubuntu1~20.04)
> 9.3.0) debug=y Thu Nov  4 18:28:30 PDT 2021
> (XEN) Latest ChangeSet: Mon Nov 1 12:36:26 2021 +0000 git:9f8434d778-dirty
> (XEN) Processor: 00000000410fd034: "ARM Limited", variant: 0x0, part
> 0xd03,rev 0x4
> (XEN) 64-bit Execution:
> (XEN)   Processor Features: 1100000000002222 0000000000000000
> (XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
> (XEN)     Extensions: FloatingPoint AdvancedSIMD
> (XEN)   Debug Features: 0000000010305106 0000000000000000
> (XEN)   Auxiliary Features: 0000000000000000 0000000000000000
> (XEN)   Memory Model Features: 0000000000001122 0000000000000000
> (XEN)   ISA Features:  0000000000011120 0000000000000000
> (XEN) 32-bit Execution:
> (XEN)   Processor Features: 0000000000001231:0000000000011011
> (XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 ThumbEE Jazelle
> (XEN)     Extensions: GenericTimer Security
> (XEN)   Debug Features: 0000000003010066
> (XEN)   Auxiliary Features: 0000000000000000
> (XEN)   Memory Model Features: 0000000010101105 0000000040000000
> (XEN)                          0000000001260000 0000000002102211
> (XEN)   ISA Features: 0000000002101110 0000000013112111
> 0000000021232042
> (XEN)                 0000000001112131 0000000000011142 0000000000011121
> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 65000 KHz
> (XEN) GICv2 initialization:
> (XEN)         gic_dist_addr=00000000f9010000
> (XEN)         gic_cpu_addr=00000000f9020000
> (XEN)         gic_hyp_addr=00000000f9040000
> (XEN)         gic_vcpu_addr=00000000f9060000
> (XEN)         gic_maintenance_irq=25
> (XEN) GICv2: Adjusting CPU interface base to 0xf902f000
> (XEN) GICv2: 192 lines, 4 cpus, secure (IID 00000000).
> (XEN) XSM Framework v1.0.1 initialized
> (XEN) Initialising XSM SILO mode
> (XEN) Using scheduler: null Scheduler (null)
> (XEN) Initializing null scheduler
> (XEN) WARNING: This is experimental software in development.
> (XEN) Use at your own risk.
> (XEN) Allocated console ring of 16 KiB.
> (XEN) Bringing up CPU1
> (XEN) Bringing up CPU2
> (XEN) Bringing up CPU3
> (XEN) Brought up 4 CPUs
> (XEN) I/O virtualisation enabled
> (XEN)  - Dom0 mode: Relaxed
> (XEN) P2M: 40-bit IPA with 40-bit PA and 8-bit VMID
> (XEN) P2M: 3 levels with order-1 root, VTCR 0x0000000080023558
> (XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Loading d0 kernel from boot module @ 0000000000e00000
> (XEN) Loading ramdisk from boot module @ 0000000002200000
> (XEN) Allocating 1:1 mappings totalling 1024MB for dom0:
> (XEN) BANK[0] 0x00000020000000-0x00000060000000 (1024MB)
> (XEN) Grant table range: 0x00000005600000-0x00000005640000
> (XEN) Allocating PPI 16 for event channel interrupt
> (XEN) Extended region 0: 0x80000000->0xc0000000
> (XEN) Extended region 1: 0xc8000000->0xe0000000
> (XEN) Extended region 2: 0xf0000000->0xf9000000
> (XEN) Extended region 3: 0xffc00000->0x600000000
> (XEN) Extended region 4: 0x880000000->0x8000000000
> (XEN) Extended region 5: 0x8001000000->0x10000000000
> (XEN) Loading zImage from 0000000000e00000 to 0000000020000000-
> 0000000021367a00
> (XEN) Loading d0 initrd from 0000000002200000 to 0x0000000028200000-
> 0x0000000029d60ac0
> (XEN) Loading d0 DTB to 0x0000000028000000-0x00000000280096d4
> (XEN) *** LOADING DOMU cpus=1 memory=80000KB ***
> (XEN) Loading d1 kernel from boot module @ 0000000003e00000
> (XEN) Loading ramdisk from boot module @ 0000000005200000
> (XEN) Loading zImage from 0000000003e00000 to 0000000040000000-
> 0000000041367a00
> (XEN) Loading d1 initrd from 0000000005200000 to 0x0000000048200000-
> 0x0000000048383400
> (XEN) Loading d1 DTB to 0x0000000048000000-0x00000000480007f3
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: All
> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
> (XEN) DEBUG start_xen 1054 id=0
> (XEN) null.c:353: 0 <-- d0v0
> (XEN) DEBUG start_xen 1054 id=1
> (XEN) null.c:353: 1 <-- d1v0
> (XEN) Freed 348kB init memory.
> (XEN) traps.c:2013:d0v0 HSR=0x00000092000045 pc=0xffffffc0105300ec
> gva=0xfffffffdfda3b000 gpa=0x0000081ffff000
> (XEN) d1v0 Unhandled SMC/HVC: 0x84000050
> (XEN) d1v0 Unhandled SMC/HVC: 0x8600ff01
> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER4
> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER8
> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER12
> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER16
> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER20
> (XEN) d1v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER0
> (XEN) DOM1: [    0.000000] Booting Linux on physical CPU 0x0000000000
> [0x410fd034]
> (XEN) DOM1: [    0.000000] Linux version 5.14.0-00002-g9c4bc20097e8
> (sstabellini@ubuntu-linux-20-04-desktop) (gcc (Ubuntu 9.3.0-17ubuntu1~
> (XEN) DOM1: 20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #9 SMP Wed
> Nov 3 16:50:00 PDT 2021
> (XEN) DOM1: [    0.000000] efi: UEFI not found.
> (XEN) DOM1: [    0.000000] Zone ranges:
> (XEN) DOM1: [    0.000000]   DMA32    [mem 0x0000000040000000-
> 0x000000005fffffff]
> (XEN) DOM1: [    0.000000]   Normal   empty
> (XEN) DOM1: [    0.000000] Movable zone start for each node
> (XEN) DOM1: [    0.000000] Early memory node ranges
> (XEN) DOM1: [    0.000000]   node   0: [mem 0x0000000040000000-
> 0x000000005fffffff]
> (XEN) DOM1: [    0.000000] Initmem setup node 0 [mem 0x0000000040000000-
> 0x000000005fffffff]
> (XEN) DOM1: [    0.000000] cma: Reserved 256 MiB at 0x000000004f400000
> (XEN) DOM1: [    0.000000] psci: probing for conduit method from DT.
> (XEN) DOM1: [    0.000000] psci: PSCIv1.1 detected in firmware.
> (XEN) DOM1: [    0.000000] psci: Using standard PSCI v0.2 function IDs
> (XEN) DOM1: [    0.000000] psci: Trusted OS migration not required
> (XEN) DOM1: [    0.000000] psci: SMC Calling Convention v1.1
> (XEN) DOM1: [    0.000000] percpu: Embedded 22 pages/cpu s51672 r8192
> d30248 u90112
> (XEN) DOM1: [    0.000000] Detected VIPT I-cache on CPU0
> (XEN) DOM1: [    0.000000] CPU features: detected: ARM erratum 843419
> (XEN) DOM1: [    0.000000] CPU features: detected: ARM erratum 845719
> (XEN) DOM1: [    0.000000] Built 1 zonelists, mobility grouping on.  Total 
> pages:
> 129280
> (XEN) DOM1: [    0.000000] Kernel command line: console=ttyAMA0
> (XEN) DOM1: [    0.000000] Dentry cache hash table entries: 65536 (order: 7,
> 524288 bytes, linear)
> (XEN) DOM1: [    0.000000] Inode-cache hash table entries: 32768 (order: 6,
> 262144 bytes, linear)
> (XEN) DOM1: [    0.000000] mem auto-init: stack:off, heap alloc:off, heap
> free:off
> (XEN) DOM1: [    0.000000] Memory: 230280K/524288K available (13184K
> kernel code, 862K rwdata, 3636K rodata, 2048K init, 315K bss, 31864K
> (XEN) DOM1: reserved, 262144K cma-reserved)
> (XEN) DOM1: [    0.000000] rcu: Hierarchical RCU implementation.
> (XEN) DOM1: [    0.000000] rcu:         RCU event tracing is enabled.
> (XEN) DOM1: [    0.000000] rcu:         RCU restricting CPUs from NR_CPUS=4 to
> nr_cpu_ids=1.
> (XEN) DOM1: [    0.000000] rcu: RCU calculated value of scheduler-enlistment
> delay is 25 jiffies.
> (XEN) DOM1: [    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16,
> nr_cpu_ids=1
> (XEN) DOM1: [    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
> (XEN) DOM1: [    0.000000] Root IRQ handler: gic_handle_irq
> (XEN) DOM1: [    0.000000] random: get_random_bytes called from
> start_kernel+0x4a8/0x728 with crng_init=0
> (XEN) DOM1: [    0.000000] arch_timer: cp15 timer(s) running at 65.00MHz 
> (virt).
> (XEN) DOM1: [    0.000000] clocksource: arch_sys_counter: mask: 
> 0xffffffffffffff
> max_cycles: 0xefdb196da, max_idle_ns: 440795204367 ns
> (XEN) DOM1: [    0.000055] sched_clock: 56 bits at 65MHz, resolution 15ns,
> wraps every 2199023255550ns
> (XEN) DOM1: [    0.015323] Console: colour dummy device 80x25
> (XEN) DOM1: [    0.017224] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 130.00 BogoMIPS (lpj=260000)
> (XEN) DOM1: [    0.017335] pid_max: default: 32768 minimum: 301
> (XEN) DOM1: [    0.019932] Mount-cache hash table entries: 1024 (order: 1,
> 8192 bytes, linear)
> (XEN) DOM1: [    0.020022] Mountpoint-cache hash table entries: 1024 (order:
> 1, 8192 bytes, linear)
> (XEN) DOM1: [    0.070463] rcu: Hierarchical SRCU implementation.
> (XEN) DOM1: [    0.073611] EFI services will not be available.
> (XEN) DOM1: [    0.074889] smp: Bringing up secondary CPUs ...
> (XEN) DOM1: [    0.075178] smp: Brought up 1 node, 1 CPU
> (XEN) DOM1: [    0.075205] SMP: Total of 1 processors activated.
> (XEN) DOM1: [    0.075253] CPU features: detected: 32-bit EL0 Support
> (XEN) DOM1: [    0.075304] CPU features: detected: CRC32 instructions
> (XEN) DOM1: [    0.076596] CPU: All CPU(s) started at EL1
> (XEN) DOM1: [    0.076717] alternatives: patching kernel code
> (XEN) DOM1: [    0.092884] devtmpfs: initialized
> (XEN) DOM1: [    0.109553] clocksource: jiffies: mask: 0xffffffff max_cycles:
> 0xffffffff, max_idle_ns: 7645041785100000 ns
> (XEN) DOM1: [    0.109716] futex hash table entries: 256 (order: 2, 16384 
> bytes,
> linear)
> (XEN) DOM1: [    0.124058] pinctrl core: initialized pinctrl subsystem
> (XEN) DOM1: [    0.143628] NET: Registered PF_NETLINK/PF_ROUTE protocol
> family
> (XEN) DOM1: [    0.157764] DMA: preallocated 128 KiB GFP_KERNEL pool for
> atomic allocations
> (XEN) DOM1: [    0.158089] DMA: preallocated 128 KiB
> GFP_KERNEL|GFP_DMA32 pool for atomic allocations
> (XEN) DOM1: [    0.158426] audit: initializing netlink subsys (disabled)
> (XEN) DOM1: [    0.166578] cpuidle: using governor menu
> (XEN) DOM1: [    0.167213] hw-breakpoint: found 6 breakpoint and 4
> watchpoint registers.
> (XEN) DOM1: [    0.168930] audit: type=2000 audit(0.140:1): state=initialized
> audit_enabled=0 res=1
> (XEN) DOM1: [    0.169475] ASID allocator initialised with 65536 entries
> (XEN) DOM1: [    0.171250] Serial: AMBA PL011 UART driver
> (XEN) DOM1: [    0.181920] 22000000.sbsa-uart: ttyAMA0 at MMIO 0x22000000
> (irq = 13, base_baud = 0) is a SBSA
> (XEN) DOM1: [    0.533061] printk: console [ttyAMA0] enabled
> 

Cheers,

Penny  Zheng
> [... cutting here, DOM1 boots until the end, no signs of dom0]



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.