Re: [Xen-devel] Xen Platform QoS design discussion

["Xu, Dongxiao" <dongxiao.xu@xxxxxxxxx>]

> -----Original Message-----
> From: Xu, Dongxiao
> Sent: Friday, May 09, 2014 10:41 AM
> To: Andrew Cooper
> Cc: George Dunlap; Ian Campbell; Jan Beulich; xen-devel@xxxxxxxxxxxxx
> Subject: RE: [Xen-devel] Xen Platform QoS design discussion
> 
> > -----Original Message-----
> > From: Andrew Cooper [mailto:andrew.cooper3@xxxxxxxxxx]
> > Sent: Thursday, May 08, 2014 7:26 PM
> > To: Xu, Dongxiao
> > Cc: George Dunlap; Ian Campbell; Jan Beulich; xen-devel@xxxxxxxxxxxxx
> > Subject: Re: [Xen-devel] Xen Platform QoS design discussion
> >
> > On 08/05/14 06:21, Xu, Dongxiao wrote:
> >
> > <massive snip>
> >
> > >>
> > >>> We have two different hypercalls right now for getting "dominfo": a
> > >>> domctl and a sysctl.  You use the domctl if you want information about
> > >>> a single domain, you use sysctl if you want information about all
> > >>> domains.  The sysctl implementation calls the domctl implementation
> > >>> internally.
> > >> It is not a fair comparison, given the completely different nature of
> > >> the domctls in question.  XEN_DOMCTL_getdomaininfo is doing very little
> > >> more than reading specific bits of data out the appropriate struct
> > >> domain and its struct vcpu's which can trivially be done by the cpu
> > >> handling the hypercall.
> > >>
> > >>> Is there a problem with doing the same thing here?  Or, with starting
> > >>> with a domctl, and then creating a sysctl if iterating over all
> > >>> domains (and calling the domctl internally) if we measure the domctl
> > >>> to be too slow for many callers?
> > >>>
> > >>>  -George
> > >> My problem is not with the domctl per-se.
> > >>
> > >> My problem is that this is not a QoS design discussion;  this is an
> > >> email thread about a specific QoS implementation which is not answering
> > >> the concerns raised against it to the satisfaction of people raising the
> > >> concerns.
> > >>
> > >> The core argument here is that a statement of "OpenStack want to get a
> > >> piece of QoS data back from libvirt/xenapi when querying a specific
> > >> domain" is being used to justify implementing the hypercall in an
> > >> identical fashion.
> > >>
> > >> This is not a libxl design; this is a single user story forming part of
> > >> the requirement "I as a cloud service provider would like QoS
> > >> information for each VM to be available to my
> > >> $CHOSEN_ORCHESTRATION_SOFTWARE so I can {differentially charge
> > >> customers, balance my load more evenly, etc}".
> > >>
> > >> The only valid justification for implementing a brand new hypercall in a
> > >> certain way is "Because $THIS_CERTAIN_WAY is the $MOST_SENSIBLE way to
> > >> perform the actions I need to perform", for appropriately
> > >> substitutions.  Not "because it is the same way I want to hand this
> > >> information off at the higher level".
> > >>
> > >> As part of this design discussion. I have raised a concern saying "I
> > >> believe the usecase of having a stats gathering daemon in dom0 has not
> > >> been appropriately considered", qualified with "If you were to use the
> > >> domctl as currently designed from a stats gathering daemon, you will
> > >> cripple Xen with the overhead".
> > >>
> > >> Going back to the original use, xenapi has a stats daemon for these
> > >> things.  It has an rpc interface so a query given a specific domain can
> > >> return some or all data for that domain, but it very definitely does not
> > >> translate each request into a hypercall for the requested information.
> > >> I have no real experience with libvirt, so can't comment on stats
> > >> gathering in that context.
> > >>
> > >> I have proposed an alternative Xen->libxc interface designed with a
> > >> stats daemon in mind, explaining why I believe it has lower overheads to
> > >> Xen and why is more in line with what I expect ${VENDOR}Stack to
> > >> actually want.
> > >>
> > >> I am now waiting for a reasoned rebuttal which has more content than
> > >> "because there are a set of patches which already implement it in this 
> > >> way".
> > > No, I don't have the patch for domctl implementation.
> > >
> > > In the past half year, all previous v1-v10 patches are implemented in 
> > > sysctl
> way,
> > however based on that, people raised a lot of comments (large size of 
> > memory,
> > runtime non-0 order of memory allocation, page sharing with user space, CPU
> > online/offline special logic, etc.), and these make the platform QoS
> > implementation more and more complex in Xen. That's why I am proposing the
> > domctl method that can make things easier.
> > >
> > > I don't have more things to argue or rebuttal, and if you prefer sysctl, 
> > > I can
> > continue to work out a v11, v12 or more, to present the big 2-dimension 
> > array
> to
> > end user and let them withdraw their real required data, still includes the 
> > extra
> > CPU online/offline logics to handle the QoS resource runtime allocation.
> > >
> > > Thanks,
> > > Dongxiao
> >
> > I am sorry - I was not trying to make an argument for one of the
> > proposed mechanisms over the other.  The point I was trying to make
> > (which on further consideration isn't as clear as I was hoping) is that
> > you cannot possibly design the hypercall interface before knowing the
> > library usecases, and there is a clear lack of understanding (or at
> > least communication) in this regard.
> >
> >
> > So, starting from the top. OpenStack want QoS information, and want to
> > get it from libvirt/XenAPI.  I think libvirt/XenAPI is the correct level
> > to do this at, and think exactly the same would apply to CloudStack as
> > well.  The relevant part of this is the question "how does
> > libvirt/XenAPI collect stats".
> >
> > XenAPI collects stats with the RRD Daemon, running in dom0.  It has an
> > internal database of statistics, and hands data from this database out
> > upon RPC requests.  It also has threads whose purpose is to periodically
> > refresh the data in the database.  This provides a disconnect between
> > ${FOO}Stack requesting stats for a domain and the logic to obtain stats
> > for that domain.
> >
> > I am however unfamiliar with libvirt in this regard.  Could you please
> > explain how the libvirt daemon deals with stats?
> 
> I am not the libvirt expert either.
> Consult from other guys who work in libvirt that, libvirt doesn't maintain the
> domain status itself, but just expose the APIs for upper cloud/openstack to 
> query,
> and these APIs accept the domain id as input parameter.

Hi Andrew,

Do you have more thought considering this libvirt usage?

Thanks,
Dongxiao

> 
> Thanks,
> Dongxiao
> 
> >
> > ~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel