Re: [Xen-devel] [RFC] Extending numbers of event channels

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Mon, 2012-12-03 at 17:52 +0000, Wei Liu wrote:
> On Mon, 2012-12-03 at 17:35 +0000, Jan Beulich wrote:
> > >>> On 03.12.12 at 17:29, Wei Liu <Wei.Liu2@xxxxxxxxxx> wrote:
> > > Regarding Jan's comment in [0], I don't think allowing user to specify
> > > arbitrary number of levels a good idea. Because only the last level
> > > should be shared among vcpus, other level should be in percpu struct to
> > > allow for quicker lookup. The idea to let user specify levels will be
> > > too complicated in implementation and blow up percpu section (since the
> > > size grows exponentially). Three levels should be quite enough. See
> > > maths below.
> > 
> > I didn't ask to implement more than three levels, I just asked for
> > the interface to establish the number of levels a guest wants to
> > use to allow for higher numbers (passing of which would result in
> > -EINVAL in your implementation).
> > 
> 
> Ah, I understand now. How about something like this:
> 
> struct EVTCHNOP_reg_nlevel {
>     int levels;
>     void *level_specified_reg_struct;
> }
> 
> > > Number of event channels:
> > >  * 32bit: 1024 * sizeof(unsigned long long) * BITS_PER_BYTE = 64k
> > >  * 64bit: 4096 * sizeof(unsigned long long) * BITS_PER_BYTE = 512k
> > > Basically the third level is a new ABI, so I choose to use unsigned long
> > > long here to get more event channels.
> > 
> > Please don't: This would make things less consistent to handle
> > at least in the guest side code. And I don't see why you would
> > have a need to do so anyway (or else your argument above
> > against further levels would become questionable).
> > 
> 
> It was suggested by Ian to use unsigned long long. Ian, why do you
> prefer unsigned long long to unsigned long?

I thought having 32 and 64 bit be the same might simplify some things,
but if not then that's fine.

Is 32k event channels going to be enough in the long run? I suppose any
system capable of running such a number of guests ought to be using 64
bit == 512k which should at least last a bit longer.

> > > Pages occupied by the third level (if PAGE_SIZE=4k):
> > >  * 32bit: 64k  / 8 / 4k = 2
> > >  * 64bit: 512k / 8 / 4k = 16
> > > 
> > > Making second level percpu will incur overhead. In fact we move the
> > > array in shared info into percpu struct:
> > >  * 32bit: sizeof(unsigned long) * 8 * sizeof(unsigned long) = 128 byte
> > >  * 64bit: sizeof(unsigned long) * 8 * sizeof(unsigned long) = 512 byte
> > > 
> > > What concerns me is that the struct evtchn buckets are allocated all at
> > > once during initialization phrase. To save memory inside Xen, the
> > > internal allocation/free scheme for evtchn needs to be modified. Ian
> > > suggested we do small number of buckets at start of day then dynamically
> > > fault in more as required.
> > > 
> > > To sum up:
> > >      1. Guest should allocate pages for third level evtchn.
> > >      2. Guest should register third level pages via a new hypercall op.
> > 
> > Doesn't the guest also need to set up space for the 2nd level?
> > 
> 
> Yes. That will be embedded in percpu struct vcpu_info, which will be
> also register via the same hypercall op.

NB that there is already a vcpu info placement hypercall. I have no
problem making this be a prerequisite for this work.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel