Re: [Xen-devel] [Xen-users] DomU not starting under pvhv2

[George Dunlap <dunlapg@xxxxxxxxx>]

Moving to xen-devel, and cc'ing Roger and Boris (who developed PVH)

On Mon, Jan 8, 2018 at 5:24 AM, Peter <xen@xxxxxxxxxxxxxxxxxx> wrote:
> Hi.
>
> Running Xen 4.10.0

What version of Linux are you using?

>
> A VM is not starting with type = 'pvh'.  The VM starts, but exits prior to
> any data being read off the domU disk image.
>
> The same VMs works if the type is changed to 'pv'.
>
> Other VMs on other hosts (with the exact same setup) work with pvh fine.
>
> The main difference between hosts where type=pvh does not work and ones
> where it does is that the pvh does not work on older CPUs:
>  Intel(R) Core(TM)2 Duo CPU     E4600  @ 2.40GHz
>  Intel(R) Xeon(R) CPU           E3110  @ 3.00GHz
>  Intel(R) Xeon(R) CPU           E5310  @ 1.60GHz
>  Intel(R) Xeon(R) CPU           E5320  @ 1.86GHz
>  Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
>  Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz
>
>
> Newer CPUs seem to be fine (start VMs with type = pvh):
>  Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
>  Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
>  Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
>  Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
>  Intel(R) Xeon(R) CPU E3-1240 v5 @ 3.50GHz
>  Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
>  Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
>  Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
>  Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
>  Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
>  Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
>  Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
>  Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
>  Intel(R) Xeon(R) CPU           X3440  @ 2.53GHz
>
> Dom0 command line is: dom0_max_vcpus=2 dom0_vcpus_pin loglvl=info
> vcpu_migration_delay=1000
>
> I see (all?) the older CPUs don't have VT-d.  However some of the CPUs where
> pvh is working do not report hvm_directio in xl info virt_caps (e.g. the
> E5506 servers) though they do have VT-d available per ark.intel.com.
>
> And ideas on what may be stopping type=pvh VMs starting on those hosts?  And
> how we could get it working?
>
> Else we'll likely be dumping some hardware....
>
> I did a diff of xl info; xl dmesg and xl create startup messages.  Between
> one of the non-pvh-able hosts (the - line) and a host where pvh works (the +
> line):
> diff host of xl -v create and xlinfo:
> -hw_caps                :
> bfebfbff:0004e3bd:20100800:00000001:00000000:00000000:00000000:00000000
> -virt_caps              : hvm
> +hw_caps                :
> bfebfbff:009ee3fd:2c100800:00000001:00000000:00000000:00000000:00000100
> +virt_caps              : hvm hvm_directio
>
> +libxl: detail: libxl_dom.c:204:numa_place_domain: NUMA placement candidate
> with 1 nodes, 8 cpus and 16819 KB free selected
>
> - (XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
> + (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
>
> - (XEN) I/O virtualisation disabled
> + (XEN) Intel VT-d Snoop Control enabled.
> + (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> + (XEN) Intel VT-d Queued Invalidation enabled.
> + (XEN) Intel VT-d Interrupt Remapping enabled.
> + (XEN) Intel VT-d Posted Interrupt not enabled.
> + (XEN) Intel VT-d Shared EPT tables not enabled.
> + (XEN) I/O virtualisation enabled
>
> -(XEN)  -> Using new ACK method
> +(XEN)  -> Using old ACK method
>
>   (XEN) VMX: Supported advanced features:
> + (XEN)  - APIC MMIO access virtualisation
> + (XEN)  - Extended Page Tables (EPT)
> + (XEN)  - Virtual-Processor Identifiers (VPID)
> + (XEN)  - Virtual NMI
> + (XEN)  - Unrestricted Guest
>
>
> -(XEN) HVM: Hardware Assisted Paging (HAP) not detected
> +(XEN) HVM: Hardware Assisted Paging (HAP) detected
>
> +(XEN) Found masked UR signaling on 0000:00:00.0
> +(XEN) Masked UR signaling on 0000:00:01.0
> +(XEN) Masked UR signaling on 0000:00:03.0
> +(XEN) Masked UR signaling on 0000:00:05.0
> +(XEN) Masked UR signaling on 0000:00:07.0
> +(XEN) Masked VT-d error signaling on 0000:00:14.0
>
>
> Regards, Peter
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxxx
> https://lists.xenproject.org/mailman/listinfo/xen-users

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel