Re: [Xen-devel] [PATCH 2/2] xen: spread page scrubbing across all idle CPU

[Bob Liu <bob.liu@xxxxxxxxxx>]

On 06/11/2014 06:13 PM, George Dunlap wrote:
> On Wed, Jun 11, 2014 at 9:13 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>>> On 11.06.14 at 04:51, <bob.liu@xxxxxxxxxx> wrote:
>>> On 06/10/2014 10:12 PM, Jan Beulich wrote:
>>>>>>> On 10.06.14 at 14:18, <lliubbo@xxxxxxxxx> wrote:
>>>>> +    if( is_tasklet )
>>>>> +        tasklet_schedule_on_cpu(&global_scrub_tasklet, cpu);
>>>>
>>>> So you re-schedule this tasklet immediately - while this may be
>>>> acceptable inside the hypervisor, did you consider the effect this
>>>> will have on the guest (normally Dom0)? Its respective vCPU won't
>>>> get run _at all_ until you're done scrubbing.
>>>>
>>>
>>> Yes, that's a problem. I don't have any better idea right now.
>>>
>>> What I'm trying is doing the scrubbing on current CPU as well as on all
>>> idle vcpus in parallel.
>>> I also considered your suggestion about doing the scrubbing in the
>>> background as well as on the allocation path. But I think it's more
>>> unacceptable for users to get blocked randomly for a uncertain time when
>>> allocating a large mount of memory.
>>> That's why I still chose the sync way that once 'xl destroy' return all
>>> memory are scrubbed.
>>
>> But I hope you realize that in the current shape, with the shortcomings
>> pointed out un-addressed, there's no way for this to go in.
> 
> Would it make more sense to do something like the following:
> * Have a "clean" freelist and a "dirty" freelist
> * When destroying a domain, simply move pages to the dirty freelist
> * Have idle vcpus scrub the dirty freelist before going to sleep
>  - ...and wake up idle vcpus to do some scrubbing when adding pages to
> the dirty freelist
> * In alloc_domheap_pages():
>  - If there are pages on the "clean" freelist, allocate them
>  - If there are no pages on the "clean" freelist but there are on the
> "dirty" freelist, scrub pages from the "dirty" freelist synchronously.
> 

Thank you very much for your suggestion, it's similar as Jan suggested.

My concern of this approach is in some bad situation the allocation path
may be blocked for a long time waiting for scrubbing "dirty" freelist.

What the users see is it's much faster to destroy a domain but may
slower to create an new one(or slow down other routines need to alloc
large memory).

If you and Jan think it's fine I can make a patch towards this way.

-- 
Regards,
-Bob

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel