Re: [Xen-devel] [PATCH 15/16] Infrastructure for manipulating 3-level event channel pages

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 04.02.13 at 12:20, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
> On Mon, 2013-02-04 at 09:23 +0000, Jan Beulich wrote:
>> >>> On 31.01.13 at 15:43, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:
>> > +static long __map_l3_arrays(struct domain *d, xen_pfn_t *pending,
>> > +                            xen_pfn_t *mask, int nr_pages)
>> > +{
>> > +    int rc;
>> > +    void *mapping;
>> > +    struct page_info *pginfo;
>> > +    unsigned long gfn;
>> > +    int pending_count = 0, mask_count = 0;
>> > +
>> > +#define __MAP(src, dst, cnt)                                    \
>> > +    for ( (cnt) = 0; (cnt) < nr_pages; (cnt)++ )                \
>> > +    {                                                           \
>> > +        rc = -EINVAL;                                           \
>> > +        gfn = (src)[(cnt)];                                     \
>> > +        pginfo = get_page_from_gfn(d, gfn, NULL, P2M_ALLOC);    \
>> > +        if ( !pginfo )                                          \
>> > +            goto err;                                           \
>> > +        if ( !get_page_type(pginfo, PGT_writable_page) )        \
>> > +        {                                                       \
>> > +            put_page(pginfo);                                   \
>> > +            goto err;                                           \
>> > +        }                                                       \
>> > +        mapping = __map_domain_page_global(pginfo);             \
>> > +        if ( !mapping )                                         \
>> > +        {                                                       \
>> > +            put_page_and_type(pginfo);                          \
>> > +            rc = -ENOMEM;                                       \
>> > +            goto err;                                           \
>> > +        }                                                       \
>> > +        (dst)[(cnt)] = mapping;                                 \
>> > +    }
>> > +
>> > +    __MAP(pending, d->evtchn_pending, pending_count)
>> > +    __MAP(mask, d->evtchn_mask, mask_count)
>> > +#undef __MAP
>> > +
>> > +    rc = 0;
>> > +
>> > + err:
>> > +    return rc;
>> > +}
>> 
>> So this alone already is up to 16 pages per guest, and hence a
>> theoretical maximum of 512k pages, i.e. 2G mapped space.
> 
> That's given a theoretical 32k guests? Ouch. It also ignores the need
> for other global mappings.
> 
> on the flip side only a minority of domains are likely to be using the
> extended scheme, and I expect even those which are would not be using
> all 16 pages, so maybe we can fault them in on demand as we bind/unbind
> evtchns.
> 
> Where does 16 come from? How many pages to we end up with at each level
> in the new scheme?

Patch 11 defines EVTCHN_MAX_L3_PAGES to be 8, and we've
got two of them (pending and mask bits).

> Some levels of the trie are per-VCPU, did you account for that already
> in the 2GB?

No, I didn't, as it would only increase the number, and make
the math less clear.

>>  The
>> global page mapping area, however, is only 1Gb in size on x86-64
>> (didn't check ARM at all)...
> 
> There isn't currently a global page mapping area on 32-bit ARM (I
> suppose we have avoided them somehow...) but obviously 2G would be a
> problem in a 4GB address space.
> 
> On ARM we currently have 2G for domheap mappings which I suppose we
> would split if we needed a global page map
> 
> These need to be global so we can deliver evtchns to VCPUs which aren't
> running, right? I suppose mapping on demand (other than for a running
> VCPU) would be prohibitively expensive.

Likely, especially for high rate ones.

> Could we make this space per-VCPU (or per-domain) by saying that a
> domain maps its own evtchn pages plus the required pages from other
> domains with which an evtchn is bound? Might be tricky to arrange
> though, especially with the per-VCPU pages and affinity changes?

Even without that trickiness it wouldn't work I'm afraid: In various
cases we need to be able to raise the events out of context (timer,
IRQs from passed through devices).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel