Re: [Xen-devel] question about memory allocation for driver domain

[Oleksandr Tyshchenko <oleksandr.tyshchenko@xxxxxxxxxxxxxxx>]

Hi, Ian

On Thu, Feb 5, 2015 at 3:12 PM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
> On Wed, 2015-02-04 at 18:47 +0200, Oleksandr Tyshchenko wrote:
>> Hi, all.
>>
>> We have begun to use the driver domain on OMAP5 platform.
>> To make driver domain running on OMAP5 platform we need to have it
>> memory 1 to 1 mapped because of lacking SMMU support on this platform.
>> To satisfy this requirement we have done temporally solution which
>> works but not entirely completed and looks not good from Xen
>> perspective. The main question in memory allocator in Xen.
>>
>> We did next steps:
>> 1. toolstack:
>> - allow to allocate 128/256/512 Mb memory chunks
>> - add ability to set rambase_pfn via cfg file
>>
>> 2. hypervisor:
>> - alloc driver domain memory 1 to 1
>>   - mark domain with id 1 as privileged
>>   - update memory allocation logic for such domain
>>   - allocate memory in the same way as for domain 0
>>
>> But we have encountered with one thing related to memory allocation 
>> algorithm.
>> Let me describe what I mean.
>> Our requirements is to allocate one specified chunk if it is present in 
>> domheap.
>> We have this "spec_mfn" which are a "rambase_pfn" for driver domain we
>> want to map 1 to 1. We can get it from extent_list. So, we need to
>> alloc chunk which would be correspond to "spec_mfn".
>> In other world we need to allocate known chunk of memory. But if I
>> understood correctly that the existing allocator doesn't allow us to
>> do that directly.
>>
>> There are some thoughts how to do that. But, at first, we need to
>> choose right direction:
>> 1. Add the separate memory allocator which allow us to alloc specified
>> chunk if it is present. And use it in case when we need to allocate
>> memory for driver domain 1:1 only. We can pass bool variable via cfg
>> file to show what we want (1:1 or not).
>> 2. Don't add separate allocator. Modify existing allocator to add
>> ability to alloc specified chunk.
>> 3. Don't add and modify anything. Let the allocator to work as usual.
>> Return mfn of allocating chunk and correct the default rambase_pfn in
>> toolstack.
>> What we actually have in the moment but do that manually. We see what
>> the mfn we got and corrected "rambase_pfn" property in cfg file.
>>
>> Could someone explain me what is a right way?
>
> The approach we've taken with dom0 is not to require specific addresses,
> but rather to tailor the address map to the contiguous region the
> allocator gives us.
>
> Since I presume you are using a 1:1 layout in the driver domain too I
> expect that approach should work there too (I think this is your #3?).
yes
>
> Your #2 might be possible, but would probably involve a reasonably
> expensive scan of the various free pools in the hopes of finding the
> block you want, since it isn't designed to be looked up in this way.
>
> I suppose #1 would be something like Linux's CMA allocator -- i.e.
> carving out 1 or more regions on boot and keeping them away from the
> main allocator (but still in e.g. the frametable etc) and a simple way
> to allocate one of the chunks.
This is interesting. Can I use /memreserve/ logic or something similar to
keeping them away from the main allocator?
But at the stages where we "initialize" domheap we know nothing about
guest domain (domd) and how we need to allocate memory for it (1:1 or not).

>
> So I think any of the approaches you suggest could work, I'd probably
> say #3, #1, #2 in decreasing order of preference.
I got it.

>
> Now, if you were asking for ideas on how to make this stuff
> upstreamable, well that's a harder question. I'm not really sure :-/
Ideally, I would like to make this stuff upstreamable.

>
> Ian.
>



-- 

Oleksandr Tyshchenko | Embedded Dev
GlobalLogic
www.globallogic.com

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel