Re: [Xen-users] network CPU consumption question

[Tim Freeman <tfreeman@xxxxxxxxxxx>]

On Fri, 2 Jun 2006 10:41:31 -0500
Tim Freeman <tfreeman@xxxxxxxxxxx> wrote:

> On Fri, 2 Jun 2006 08:51:14 -0400
> "Tim Wood" <twwood@xxxxxxxxx> wrote:
> 
> > Sorry I don't have full concrete answers, but you might be able to get some
> > ideas from either of these papers if you haven't seen them:
> > 
> > XenMon - http://www.hpl.hp.com/techreports/2005/HPL-2005-187.html
> > Measuring CPU Overhead for IO -
> > http://www.hpl.hp.com/personal/Lucy_Cherkasova/projects/papers/final-perf-study-usenix.pdf
> > 
> > They show how the scheduling parameters can have a pretty large effect on
> > performance mainly due to wasted context switches between domains.
> 
> Thanks, I am aware of both the papers.  Since I have the XenMon output
> I looked at the ex/s and did not see anything to explain the leap with
> context switching.  The two I/O bound guests together are 'charged'
> roughly 19% of the CPU usage to process the same amt of packets as the
> one I/O guest could process with roughly 11.5%. 
> 
> It's my assumption that context switching between any two domains
> results in CPU usage being increased for both of those domains.  Isn't
> that true?  (I remember being roughly convinced of this by data at one
> point, but it would take me some time to find that). 
> 
> If true, in the situation I reported I think it would mean that the two
> I/O guest domains (dom1/dom2) in scenario 2 would together have to have
> had a relative increase in ex/s over dom0 to account for the extra CPU
> usage seen for dom1/dom2 but not dom0.  (i.e., maybe dom1 and dom2 are
> being switched between themselves far more often and that is causing
> extra load for just these two domains). 
> 
> But in scenario one, dom0 and dom1 read ~1300 each, in scenario two
> dom0, dom1 and dom2 each read ~380.  So it's far less in fact...
> (totals including cpu domain: scenario 1 = ~4000 ex/s, scenario 2 =
> ~1900 ex/s). 

And this is intuititive to me, the more domains that are running (and
assuming somewhat fair scheduling parameters) the less frequently they
have a chance for the CPU and the more virtual interrupts are stored
up, i.e., more work to do = domain doesn't block as quickly (when it's
run out of work).  Less blocking means less context switching given the
same workloads.  In the data, the overall blocked % does indeed
plummet in scenario 2. 

Tim 

> 
> Thanks,
> Tim
> 
> 
> >
> > On 5/31/06, Tim Freeman <tfreeman@xxxxxxxxxxx> wrote:
> > >
> > > Running into a mysterious situation here...
> > >
> > > I have a Xen3 setup running SEDF measuring bandwidth offnode on a
> > > non-VMM machine. Running one guest domain bridged through domain
> > > 0, transfers from memory (/dev/zero) to the non-VMM machine.  Another
> > > domain runs pure CPU bound activity, as much as it can (this avoids the
> > > non-interrupt-batching context-switching-heavy situation that exists
> > > otherwise).
> > >
> > > On these computers, getting roughly 12.3 MB/s with this distribution
> > > of CPU consumption (measured with XenMon):
> > >
> > > dom0: 17.79%, dom1 (net): 11.41%, dom2 (cpu): 69.48%
> > >
> > > (these are means from many runs)
> > >
> > > I get an extremely similar bandwidth reading in another scenario with
> > > two net transferring domains (their bandwidths are halved, talking
> > > about the *sum* of the two).  However, the CPU consumption did not
> > > scale very well:
> > >
> > > dom0: 19.38%, dom1 (net): 9.61%, dom2 (net): 9.55%, dom3 (cpu): 60.18%
> > >
> > > In the previous scenario dom1 used around 11.5 percent of the CPU to
> > > accomplish what here takes roughly 19 percent.
> > >
> > > Any ideas what this extra CPU work is?  There is extra context
> > > switching in the second situation (which could account for the dom0
> > > increase), but the network bound guest disparity seems a little extreme
> > > to only be that?
> > >
> > > Thanks,
> > > Tim
> > >
> > >
> > > _______________________________________________
> > > Xen-users mailing list
> > > Xen-users@xxxxxxxxxxxxxxxxxxx
> > > http://lists.xensource.com/xen-users
> > >
> > 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
> 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users