RE: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU kernel with PCI passthrough

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

In case it is related:
http://lists.xensource.com/archives/html/xen-devel/2010-07/msg01247.html 

Although I never went further on this investigation, it
appeared to me that pvclock_clocksource_read was getting
called at least an order-of-magnitude more frequently than
expected in some circumstances for some kernels.  And IIRC
it was scaled by the number of vcpus.

> -----Original Message-----
> From: Dante Cinco [mailto:dantecinco@xxxxxxxxx]
> Sent: Thursday, November 18, 2010 12:36 PM
> To: Konrad Rzeszutek Wilk
> Cc: Jeremy Fitzhardinge; Xen-devel; mathieu.desnoyers@xxxxxxxxxx;
> Andrew Thomas; keir.fraser@xxxxxxxxxxxxx; Chris Mason
> Subject: Re: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2
> pvops domU kernel with PCI passthrough
> 
> I mentioned earlier in an previous post to this thread that I'm able
> to apply Dulloor's xenoprofile patch to the dom0 kernel but not the
> domU kernel. So I can't do active-domain profiling but I'm able to do
> passive-domain profiling but I don't know how reliable the results are
> since it shows pvclock_clocksource_read as the top consumer of CPU
> cycles at 28%.
> 
> CPU: Intel Architectural Perfmon, speed 2665.98 MHz (estimated)
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a
> unit mask of 0x00 (No unit mask) count 100000
> samples  %        image name               app name
> symbol name
> 918089   27.9310
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           pvclock_clocksource_read
> 217811    6.6265  domain1-modules          domain1-modules
> /domain1-modules
> 188327    5.7295  vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> mutex_spin_on_owner
> 186684    5.6795
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __xen_spin_lock
> 149514    4.5487
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __write_lock_failed
> 123278    3.7505
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __kernel_text_address
> 122906    3.7392
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           xen_spin_unlock
> 90903     2.7655
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __spin_time_accum
> 85880     2.6127
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __module_address
> 75223     2.2885
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           print_context_stack
> 66778     2.0316
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __module_text_address
> 57389     1.7459
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           is_module_text_address
> 47282     1.4385  xen-syms-4.1-unstable    domain1-xen
> syscall_enter
> 47219     1.4365
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           prio_tree_insert
> 46495     1.4145  vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> pvclock_clocksource_read
> 44501     1.3539
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           prio_tree_left
> 32482     0.9882
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           native_read_tsc
> 
> I ran oprofile (0.9.5 with xenoprofile patch) for 20 seconds while the
> I/Os were running. Here's the command I used:
> 
> opcontrol --start --xen=/boot/xen-syms-4.1-unstable
> --vmlinux=/boot/vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> --passive-domains=1
> --passive-images=/boot/vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-
> 5.11.dcinco-debug
> 
> I had to remove dom0_max_vcpus=1 (but kept dom0_vcpus_pin=true) in the
> Xen command line. Otherwise, oprofile only gives the samples from
> CPU0.
> 
> I'm going to try perf next.
> 
> - Dante
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel