Re: [Xen-devel] [PATCH RFC v2 4/7] libxl/vNUMA: vNUMA libxl functionality.

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On mar, 2013-09-17 at 17:56 +0100, George Dunlap wrote:
> On 09/17/2013 05:38 PM, Ian Campbell wrote:
> > On Tue, 2013-09-17 at 17:36 +0100, George Dunlap wrote:
>
> >> Why would someone want to make a VM with two vnodes and then put them
> >> on the same pnode?  Apart from testing, of course, but our defaults
> >> should be for the common case of real users.
> >
> > If your pool of machines included 1- and 2-node systems might you want
> > to do this so that when you migrate the vm to a two node system it can
> > make use of it?
> 
> Yes, but from the description (and from a brief glance at the code) it 
> looks like if it can happen to fit all the vnodes on a single pnode, it 
> will do so -- even if the node affinity for the domain spans several nodes.
> 
That is something we discussed with Elena (I think you were copied to
the thread, but anyway), and it's not an easy thing to wrap the mind
around!

Actually, how to come up with a decent interface, sensible default
values, etc., is really quite a piece of work, and I think we should
have some discussion about it (perhaps in a new thread).

So, initially, I rose exactly the same objection: 2 vnodes needs means
trying to put and map the guest to 2 pnodes. However, if the purpose of
the whole thing (i.e., the combined action of automatic NUMA placement
and vNUMA) is to maximize the performance, well, no matter what the
vNUMA topology is, having the guest on just one physical node (if it
fits there, of course) is always the best solution... So why shouldn't
we do that?

> I think if I was a user, and made a VM with 2 vnodes, and then set its 
> node affinity to two pnodes, I would expect the tools to place each 
> vnode on one pnode.
> 
That is exactly the question: is the fact that the user asked for 2
vnodes enough for disallowing allocatin the VM on just one pnode?

As it has been pointed out by Jan already, we really want another
parameter, or in general something that will allow the user to specify
the exact pnode-to-vnode mapping, and in case such parameter is
provided, all bets are off. But what if it is not... What the default
behaviour should be?

And that is a genuine question, i.e., I really can't decide which
solution is better, as I see both advantages and disadvantages on both
of them.

> At this point it may be bike-shedding, though... at some point we'll 
> have the ability to specify node affinity for individual vcpus, at which 
> time we can specify a pnode affinity for individual vnodes.  As long as 
> we have something not barking mad here, we can revisit it before the 
> release (as long as someone writes down to do it).
> 
I have per-vcpu NUMA node-affinity almost ready, and I really plan to
submit either today or tomorrow. However, I'm not sure I understood how
you think that is going to help... Being able to specify a node-affinity
for each single vcpu of a domain (which surely means being able to
specify it consistently for all the vcpus that forms a vnode, which I
think is what you call 'specify a pnode affinity for individual vnodes')
does not forbid to map two or more vnodes on the same pnode (and, most
likely, specify the same node-affinity for all their vcpus, although
that is not mandatory).

So, it looks to me that, even at that point, it will still be our call
to decide what to do and what the most sensible default behaviour is,
won't it?

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel