Re: [Xen-devel] PVH cpuid feature flags

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Tue, Jan 21, 2014 at 08:28:14PM +0100, Roger Pau Monné wrote:
> Hello,
> 
> While doing some benchmarks on PV/PVH/PVHVM, I've realized that the
> cpuid feature flags exposed to PVH guests are kind of strange, this is
> the output of the feature flags as seen by an HVM domain:
> 

What about a PV guest? I presume if you ran an NetBSD PV guest it would
give a format similar to this?

> Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
>  Features2=0x81b82201<SSE3,SSSE3,CX16,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,HV>
> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
> AMD Features2=0x1<LAHF>
> 
> And this is what a PVH domain sees when running on the same hardware:
> 
> Features=0x1fc98b75<FPU,DE,TSC,MSR,PAE,CX8,APIC,SEP,CMOV,PAT,CLFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT>
> Features2=0x80982201<SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,HV>
> AMD Features=0x20100800<SYSCALL,NX,LM>
> AMD Features2=0x1<LAHF>
> 
> I would expect the feature flags to be quite similar between an HVM
> domain and a PV domain (since they both run inside of a HVM container).
> AFAIK, there's no reason to disable PSE, PGE, PSE36 and RDTSCP for PVH
> guests. Also, is there any reason why PVH guests have the ACPI, SS and
> CLFLUSH feature flags but not HVM?

S5?

ACPI is enabled for PV I think, but Linux PV guests disable them
as there is no ACPI tables in PV mode:

 429         if (!xen_initial_domain())                                         
     
 430                 cpuid_leaf1_edx_mask &=                                    
     
 431                         ~((1 << X86_FEATURE_ACPI));  /* disable ACPI */    
     
 432                                                                            
     
 434                                                                      

CLFLUSH - no idea why it would be disabled.


The rdsctp should be enabled. In the past I think it was related to
'timer=' option. We would either trap it, or emulate it with a constant
value, or passthrough. It should be passing it through but maybe there
is a bug?

PSE, PGE, PSE36, PG1GB, etc, should all be exposed. Actually the PG1TB
is not exposed because of another bug:
http://www.gossamer-threads.com/lists/xen/devel/313596

> 
> Most (if not all) of this probably comes from the fact that we are
> reporting the same feature flags as pure PV guests, but I see no reason
> to do that for PVH guests, we should decide what's supported on PVH and
> set the feature flags accordingly.

Right and also have a nice policy. The problem is that we set/unset
the cpuid flags in the toolstack (and in two places - depending on the
architecture) and also in the hypervisor.

Anyhow, these I know we disable:

 425         cpuid_leaf1_edx_mask =                                             
     
 426                 ~((1 << X86_FEATURE_MTRR) |  /* disable MTRR */            
     
 427                   (1 << X86_FEATURE_ACC));   /* thermal monitoring */      
     

And I  think your patch to the Xen hypervisor does it too - it clears
the MTRR by default now. The ACC is (if my memory is correct) for
the Pentium 4 and such - we can disable it as well.

 428                                                                            
     
 433         cpuid_leaf1_ecx_mask &= ~(1 << (X86_FEATURE_X2APIC % 32));         
     

And this we definitly need to disable. The x2APIC should not
be exposed as we want to use the Xen's version of APIC ops. And
if the x2APIC bit is enabled in Linux, there is some other code
(NMI handler) that will use that without using the APIC ops.
And blow up :-(


Then there is the MWAIT. Actually we can put that on the side.
I know it is important for dom0, but since we don't have those
patches yet in, we can ignore that. However, the hypervisor
(pv_cpuid) disables it. 


There is also the XSAVE business:

save_mask =                                                            
 440                 (1 << (X86_FEATURE_XSAVE % 32)) |                          
     
 441                 (1 << (X86_FEATURE_OSXSAVE % 32));                         
     
 442                                                                            
     
 443         /* Xen will set CR4.OSXSAVE if supported and not disabled by force 
*/   
 444         if ((cx & xsave_mask) != xsave_mask)                               
     
 445                 cpuid_leaf1_ecx_mask &= ~xsave_mask; /* disable XSAVE & 
OSXSAVE */

Which I am not clear about.


This looks like to make a uniform 'cpuid' look in the hypervisor
we need somehow to glue hvm_cpuid and pv_cpuid with some
form of tables/lookups.

And make sure that the same logic is reflected in
xc_cpuid_x86.c (toolstack).


> 
> Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel