|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] questions of vm save/restore on arm64
On Fri, 27 May 2016, Chenxiao Zhao wrote:
> Hi,
>
> My board is Hikey on which have octa-core of arm cortex-a53. I have applied
> patches [1] to try vm save/restore on arm.
> These patches originally do not working on arm64. I have made some changes
> based on patch set [2].
Hello Chenxiao,
thanks for your interest in Xen on ARM save/restore.
> What I have got so far is
>
> 1. if I run 'xl save -p guest memState' to leave guest in suspend state, then
> run 'xl unpause guest'.
> the guest can resume successfully. so I suppose the guest works find on
> suspend/resume.
>
> 2. if I run 'xl restore -p memState' to restore guest and use xenctx to dump
> all vcpu's registers.
> all the registers are identical to the state on save. After I run 'xl
> unpause guest', I got no error but can not connect to console.
> After restore the guest's PC is at a function called
> user_disable_single_step(), which is called by single_step_handler().
>
> My question is
>
> 1. How could I debug guest on restore progress? are there any tools available?
Nothing special. You can use ctrl-AAA on the console to switch to the
hypervisor console and see the state of the guest. You can also add some
debug printks; if the console doesn't work you can use
dom0_write_console in Linux to get messages out of your guest (you need
to compile Xen with debug=y for that to work).
> 2. From my understanding, the restore not working is because some status is
> missing when saving.
> e.g. on cpu_save, it know the domain is 64bit, but on cpu_load, it always
> think it is a 32bit domain. so I have hard coded the domain type to
> DOMAIN_64BIT.
> Am I correct?
If Xen thinks the domain is 32-bit at restore, it must be a bug.
> 3. How could I dump all VM's status? I only found xenctx can dump vcpu's
> registers.
You can use the hypervisor console via the ctrl-aaa menu.
> I have attached my patch and log below.
>
> Looking forward for your feedback.
> Thanks
>
> xl list
> Name ID Mem VCPUs State
> Time(s)
> Domain-0 0 1024 8 r-----
> 11.7
> root@linaro-alip:~# xl create guest.cfg
> Parsing config from guest.cfg
> [ 39.238723] xen-blkback: ring-ref 8, event-channel 3, protocol 1 (arm-abi)
> persistent grants
>
> root@linaro-alip:~# xl save -p guest memState
> Saving to memState new xl format (info 0x3/0x0/931)
> xc: info: Saving domain 1, type ARM
> (XEN) HVM1 save: VCPU
> (XEN) HVM1 save: A15_TIMER
> (XEN) HVM1 save: GICV2_GICD
> (XEN) HVM1 save: GICV2_GICC
> (XEN) HVM1 save: GICV3
> root@linaro-alip:~# /usr/lib/xen/bin/xenctx -a 1
> PC: ffffffc0000ab028
> LR: ffffffc00050458c
> ELR_EL1: ffffffc000086b34
> CPSR: 200001c5
> SPSR_EL1: 60000145
> SP_EL0: 0000007ff6f2a850
> SP_EL1: ffffffc0140a7ca0
>
> x0: 0000000000000001 x1: 00000000deadbeef x2: 0000000000000002
> x3: 0000000000000002 x4: 0000000000000004 x5: 0000000000000000
> x6: 000000000000001b x7: 0000000000000001 x8: 000000618e589e00
> x9: 0000000000000000 x10: 0000000000000000 x11: 0000000000000000
> x12: 00000000000001a3 x13: 000000001911a7d9 x14: 0000000000002ee0
> x15: 0000000000000005 x16: 00000000deadbeef x17: 0000000000000001
> x18: 0000000000000007 x19: 0000000000000000 x20: ffffffc014163d58
> x21: ffffffc014163cd8 x22: 0000000000000001 x23: 0000000000000140
> x24: ffffffc000d5bb18 x25: ffffffc014163cd8 x26: 0000000000000000
> x27: 0000000000000000 x28: 0000000000000000 x29: ffffffc0140a7ca0
>
> SCTLR: 34d5d91d
> TTBCR: 00000032b5193519
> TTBR0: 002d000054876000
> TTBR1: 0000000040dcf000
> root@linaro-alip:~# xl destroy guest
> (XEN) mm.c:1265:d0v1 gnttab_mark_dirty not implemented yet
> root@linaro-alip:~# xl restore -p memState
> Loading new save file memState (new xl fmt info 0x3/0x0/931)
> Savefile contains xl domain config in JSON format
> Parsing config from <saved>
> xc: info: (XEN) HVM2 restore: VCPU 0
> Found ARM domain from Xen 4.7
> xc: info: Restoring domain
> (XEN) HVM2 restore: A15_TIMER 0
> (XEN) HVM2 restore: A15_TIMER 0
> (XEN) HVM2 restore: GICV2_GICD 0
> (XEN) HVM2 restore: GICV2_GICC 0
> (XEN) GICH_LRs (vcpu 0) mask=0
> (XEN) VCPU_LR[0]=0
> (XEN) VCPU_LR[1]=0
> (XEN) VCPU_LR[2]=0
> (XEN) VCPU_LR[3]=0
> xc: info: Restore successful
> xc: info: XenStore: mfn 0x39001, dom 0, evt 1
> xc: info: Console: mfn 0x39000, dom 0, evt 2
> root@linaro-alip:~# /usr/lib/xen/bin/xenctx -a 2
> PC: ffffffc0000ab028
> LR: ffffffc00050458c
> ELR_EL1: ffffffc000086b34
> CPSR: 200001c5
> SPSR_EL1: 60000145
> SP_EL0: 0000007ff6f2a850
> SP_EL1: ffffffc0140a7ca0
>
> x0: 0000000000000000 x1: 00000000deadbeef x2: 0000000000000002
> x3: 0000000000000002 x4: 0000000000000004 x5: 0000000000000000
> x6: 000000000000001b x7: 0000000000000001 x8: 000000618e589e00
> x9: 0000000000000000 x10: 0000000000000000 x11: 0000000000000000
> x12: 00000000000001a3 x13: 000000001911a7d9 x14: 0000000000002ee0
> x15: 0000000000000005 x16: 00000000deadbeef x17: 0000000000000001
> x18: 0000000000000007 x19: 0000000000000000 x20: ffffffc014163d58
> x21: ffffffc014163cd8 x22: 0000000000000001 x23: 0000000000000140
> x24: ffffffc000d5bb18 x25: ffffffc014163cd8 x26: 0000000000000000
> x27: 0000000000000000 x28: 0000000000000000 x29: ffffffc0140a7ca0
>
> SCTLR: 34d5d91d
> TTBCR: 00000000b5193519
> TTBR0: 002d000054876000
> TTBR1: 0000000040dcf000
> root@linaro-alip:~# xl unpause guest
> root@linaro-alip:~# xl list
> Name ID Mem VCPUs State
> Time(s)
> Domain-0 0 1024 8 r-----
> 22.2
> guest 2 0 1 r-----
> 4.8
> root@linaro-alip:~# /usr/lib/xen/bin/xenctx -a 2
> PC: ffffffc000084a00
> LR: ffffffc00050458c
> ELR_EL1: ffffffc000084a00
> CPSR: 000003c5
> SPSR_EL1: 000003c5
> SP_EL0: 0000007ff6f2a850
> SP_EL1: ffffffc0140a7ca0
>
> x0: 0000000000000000 x1: 00000000deadbeef x2: 0000000000000002
> x3: 0000000000000002 x4: 0000000000000004 x5: 0000000000000000
> x6: 000000000000001b x7: 0000000000000001 x8: 000000618e589e00
> x9: 0000000000000000 x10: 0000000000000000 x11: 0000000000000000
> x12: 00000000000001a3 x13: 000000001911a7d9 x14: 0000000000002ee0
> x15: 0000000000000005 x16: 00000000deadbeef x17: 0000000000000001
> x18: 0000000000000007 x19: 0000000000000000 x20: ffffffc014163d58
> x21: ffffffc014163cd8 x22: 0000000000000001 x23: 0000000000000140
> x24: ffffffc000d5bb18 x25: ffffffc014163cd8 x26: 0000000000000000
> x27: 0000000000000000 x28: 0000000000000000 x29: ffffffc0140a7ca0
>
> SCTLR: 34d5d91d
> TTBCR: 00000000b5193519
> TTBR0: 002d000054876000
> TTBR1: 0000000040dcf000
> root@linaro-alip:~# xl console guest
> xenconsole: Could not read tty from store: Success
> root@linaro-alip:~#
>
>
> diff --git a/xen/arch/arm/hvm.c b/xen/arch/arm/hvm.c
> index aee3353..411bab4 100644
> --- a/xen/arch/arm/hvm.c
> +++ b/xen/arch/arm/hvm.c
> @@ -120,7 +120,8 @@ static int cpu_save(struct domain *d,
> hvm_domain_context_t *h)
> ctxt.dfar = v->arch.dfar;
> ctxt.dfsr = v->arch.dfsr;
> #else
> - /* XXX 64-bit */
> + ctxt.far = v->arch.far;
> + ctxt.esr = v->arch.esr;
> #endif
>
> #ifdef CONFIG_ARM_32
> @@ -187,6 +188,9 @@ static int cpu_load(struct domain *d,
> hvm_domain_context_t *h)
> if ( hvm_load_entry(VCPU, h, &ctxt) != 0 )
> return -EINVAL;
>
> +#ifdef CONFIG_ARM64
> + v->arch.type = DOMAIN_64BIT;
> +#endif
> v->arch.sctlr = ctxt.sctlr;
> v->arch.ttbr0 = ctxt.ttbr0;
> v->arch.ttbr1 = ctxt.ttbr1;
> @@ -199,7 +203,8 @@ static int cpu_load(struct domain *d,
> hvm_domain_context_t *h)
> v->arch.dfar = ctxt.dfar;
> v->arch.dfsr = ctxt.dfsr;
> #else
> - /* XXX 64-bit */
> + v->arch.far = ctxt.far;
> + v->arch.esr = ctxt.esr;
> #endif
>
> #ifdef CONFIG_ARM_32
> diff --git a/xen/include/public/arch-arm/hvm/save.h
> b/xen/include/public/arch-arm/hvm/save.h
> index db916b1..89e6e89 100644
> --- a/xen/include/public/arch-arm/hvm/save.h
> +++ b/xen/include/public/arch-arm/hvm/save.h
> @@ -46,8 +46,12 @@ DECLARE_HVM_SAVE_TYPE(HEADER, 1, struct hvm_save_header);
>
> struct hvm_hw_cpu
> {
> +#ifdef CONFIG_ARM_32
> uint64_t vfp[34]; /* Vector floating pointer */
> /* VFP v3 state is 34x64 bit, VFP v4 is not yet supported */
> +#else
> + uint64_t vfp[66];
> +#endif
>
> /* Guest core registers */
> struct vcpu_guest_core_regs core_regs;
> @@ -60,6 +64,9 @@ struct hvm_hw_cpu
> uint32_t dacr;
> uint64_t par;
>
> + uint64_t far;
> + uint64_t esr;
> +
> uint64_t mair0, mair1;
> uint64_t tpidr_el0;
> uint64_t tpidr_el1;
>
>
>
> [1] http://lists.xen.org/archives/html/xen-devel/2015-12/msg01053.html
> [2] http://lists.xen.org/archives/html/xen-devel/2014-04/msg01544.html
>
> _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |