Re: [Xen-devel] Windows SMP

[Steve Ofsthun <sofsthun@xxxxxxxxxxxxxxx>]

Venefax wrote:
> I use only Windows 2003 R2 SP2 fully patched. Therefore that patch would not 
> apply since it is for Vista (Novell shim (Vista/2008 guests). But I will ask 
> Novell.
> I would love to try that second patch that would "force all guest callback 
> interrupts to VCPU 0". My setup is to use one core for xen, only, and the 
> rest 15 cores for virtual machines, as instructed by Novell support. I am not 
> over-committing CPU's. Even if I run one single virtual machine with 8 CPU's, 
> the performance is awful. It takes  10 mins to load an application that loads 
> in 1.5-2.0 mins with one single CPU. I switched all my 16 virtual machines to 
> Standard PC Hal and that is the only way that the application works. However, 
> it is an administrative nightmare. Instead of managing 16 independent windows 
> machines via VNC, I should be able to manage two, maybe one.
> Can you send me information regarding both patches mentioned?

I'll have to check into the state of the PAUSE exit patch.  We are doing the 
work on an Intel Tylersburg machine.  I don't think these are generally 
available yet.

I have attached the referenced callback IRQ routing patch.

Whose Windows PV-on-HVM drivers are you running?

Steve

> Federico
> 
> 
> -----Original Message-----
> From: Steve Ofsthun [mailto:sofsthun@xxxxxxxxxxxxxxx] 
> Sent: Monday, January 05, 2009 1:59 PM
> To: Venefax
> Cc: 'Keir Fraser'; 'Andrew Lyon'; 'James Harper'; 'Dirk Utterback'; 
> xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] Windows SMP
> 
> Venefax wrote:
>> Dear Gentlemen
> 
>> I have a machine with Intel 7350, 4 quad core. I can test since I have two
>> virtual machines with 8 vcpus, showing a gross overhead that makes them
>> unsuited for business. I use SLES 10 SP2, and I don't know how to apply the
>> patch, but if somebody can log in and apply it, we can see the results
>> immediately. The issue is affecting me directly. My two VM's have a VOIP
>> application, very intensive in network and CPU usage.
> 
> If your problem isn't the previously discussed TPR issues ...
> 
> What Windows version are you running in your guests?
> 
> Windows 2000 SMP has serious problems due to the CPU wasting idle loop.  We 
> avoid this with a special idler daemon running in the guest.
> 
> 
> Are you over-committing VCPUS at all (more than one active VCPU per CPU)?
> 
> We have noticed significant performance degradation with SMP windows once you 
> over-commit VCPUS on the host.  This seems to be due to excessive guest 
> spinlock overhead caused by the spinning VCPUS wasting their entire quantum 
> looping for the guest lock at the same time they are preventing the guest 
> lock holding VCPU from running.  We have been experimenting with new CPU 
> features for exiting on PAUSE instructions.  This can possibly be used to 
> detect the CPU wasting spinners.
> 
> 
> Have you tried using the Novell shim (Vista/2008 guests)?
> 
> I'm not sure the shim provides locking enhancements, but we have seen 
> benefits for certain workloads.
> 
> 
> Are you running PV on HVM drivers?
> 
> One last area of concern may be distributing callback interrupts to all 
> VCPUS.  We still run Xen (SLES10 SP2 based) with a patch to force all guest 
> callback interrupts to VCPU 0.  This has consistently improved our SMP guest 
> I/O performance while running PV-on-HVM drivers.
> 
> 
> Steve
> 
>> Federico
>> -----Original Message-----
>> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx] 
>> Sent: Wednesday, December 31, 2008 5:10 AM
>> To: Andrew Lyon
>> Cc: Venefax; James Harper; Dirk Utterback; xen-devel@xxxxxxxxxxxxxxxxxxx
>> Subject: Re: [Xen-devel] Windows SMP
>>
>> On 31/12/2008 10:08, "Andrew Lyon" <andrew.lyon@xxxxxxxxx> wrote:
>>
>>> Thanks Keir, I added a couple of printk's as you suggested and I can
>>> now see if the feature is supported:
>>>
>>> Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz = (XEN) APIC Access
>>> virtualized
>>> Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz = (XEN) APIC Access
>> emulated
>>> I didnt expect my Xeon system to support it, I've read some pdf's
>>> about Intel virtualizaton features and they seemed to suggest it was a
>>> new feature on 7xxx Xeons.
>>>
>>> The results fit the performance I've seen, a windows xp 32 bit hvm
>>> with 2 cpu's runs a lot faster on the Xeon 2.5 than on the Core 2.4.
>> I think it is probably worth printing out. I'll add a patch to xen-unstable.
>>
>>  Thanks,
>>  Keir
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
> 

diff -r 60570c3d7ea7 xen/arch/x86/hvm/vioapic.c
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -341,21 +341,18 @@ void vioapic_deliver(struct hvm_hw_vioap
         return;
     }
 
+#ifdef IRQ0_SPECIAL_ROUTING
+    /* Force round-robin to pick VCPU 0 */
+    if ( ((irq == hvm_isa_irq_to_gsi(0)) && pit_channel0_enabled()) ||
+        is_hvm_callback_irq(vioapic, irq) )
+       deliver_bitmask = (uint32_t)1;
+#endif
+
     switch ( delivery_mode )
     {
     case dest_LowestPrio:
     {
-#ifdef IRQ0_SPECIAL_ROUTING
-        /* Force round-robin to pick VCPU 0 */
-        if ( ((irq == hvm_isa_irq_to_gsi(0)) && pit_channel0_enabled()) ||
-                    is_hvm_callback_irq(vioapic, irq) )
-        {
-            v = vioapic_domain(vioapic)->vcpu[0];
-            target = v ? vcpu_vlapic(v) : NULL;
-        }
-        else
-#endif
-            target = apic_round_robin(vioapic_domain(vioapic),
+       target = apic_round_robin(vioapic_domain(vioapic),
                                       vector, deliver_bitmask);
         if ( target != NULL )
         {

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel