Re: [Xen-devel] [RFC Design Doc] Add vNVDIMM support for Xen

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 08.04.16 at 07:02, <haozhong.zhang@xxxxxxxxx> wrote:
> On 03/29/16 04:49, Jan Beulich wrote:
>> >>> On 29.03.16 at 12:10, <haozhong.zhang@xxxxxxxxx> wrote:
>> > On 03/29/16 03:11, Jan Beulich wrote:
>> >> >>> On 29.03.16 at 10:47, <haozhong.zhang@xxxxxxxxx> wrote:
> [..]
>> >> > I still cannot find a neat approach to manage guest permissions for
>> >> > nvdimm pages. A possible one is to use a per-domain bitmap to track
>> >> > permissions: each bit corresponding to an nvdimm page. The bitmap can
>> >> > save lots of spaces and even be stored in the normal ram, but
>> >> > operating it for a large nvdimm range, especially for a contiguous
>> >> > one, is slower than rangeset.
>> >> 
>> >> I don't follow: What would a single bit in that bitmap mean? Any
>> >> guest may access the page? That surely wouldn't be what we
>> >> need.
>> >>
>> > 
>> > For a host having a N pages of nvdimm, each domain will have a N bits
>> > bitmap. If the m'th bit of a domain's bitmap is set, then that domain
>> > has the permission to access the m'th host nvdimm page.
>> 
>> Which will be more overhead as soon as there are enough such
>> domains in a system.
>>
> 
> Sorry for the late reply.
> 
> I think we can make some optimization to reduce the space consumed by
> the bitmap.
> 
> A per-domain bitmap covering the entire host NVDIMM address range is
> wasteful especially if the actual used ranges are congregated. We may
> take following ways to reduce its space.
> 
> 1) Split the per-domain bitmap into multiple sub-bitmap and each
>    sub-bitmap covers a smaller and contiguous sub host NVDIMM address
>    range. In the beginning, no sub-bitmap is allocated for the
>    domain. If the access permission to a host NVDIMM page in a sub
>    host address range is added to a domain, only the sub-bitmap for
>    that address range is allocated for the domain. If access
>    permissions to all host NVDIMM pages in a sub range are removed
>    from a domain, the corresponding sub-bitmap can be freed.
> 
> 2) If a domain has access permissions to all host NVDIMM pages in a
>    sub range, the corresponding sub-bitmap will be replaced by a range
>    struct. If range structs are used to track adjacent ranges, they
>    will be merged into one range struct. If access permissions to some
>    pages in that sub range are removed from a domain, the range struct
>    should be converted back to bitmap segment(s).
> 
> 3) Because there might be lots of above bitmap segments and range
>    structs per-domain, we can organize them in a balanced interval
>    tree to quickly search/add/remove an individual structure.
> 
> In the worst case that each sub range has non-contiguous pages
> assigned to a domain, above solution will use all sub-bitmaps and
> consume more space than a single bitmap because of the extra space for
> organization. I assume that the sysadmin should be responsible to
> ensure the host nvdimm ranges assigned to each domain as contiguous
> and congregated as possible in order to avoid the worst case. However,
> if the worst case does happen, xen hypervisor should refuse to assign
> nvdimm to guest when it runs out of memory.

To be honest, this all sounds pretty unconvincing wrt not using
existing code paths - a lot of special treatment, and hence a lot
of things that can go (slightly) wrong.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel