[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NULL scheduler DoS


  • To: "julien@xxxxxxx" <julien@xxxxxxx>, "ahmeddan@xxxxxxxxxxxx" <ahmeddan@xxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Dario Faggioli <dfaggioli@xxxxxxxx>
  • Date: Wed, 11 Aug 2021 09:59:48 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V1hOXW7Zw+mPvRHvhj/Z+RVRBDnwLK7Hro0eooYltls=; b=TRbQIcm8WKNXN9WT4i131eOql1LtbgoXy7aMl5FWhznr2rnvWX8/9V2tKzOEkbHW7jNmZZgU37BfFz+2RRCpCZPmbV2tpnqKiUF5MA/LK6VLqhJG0goW8Exdjy2EHItjDGdAKGLCBCPw5ky5f8j1Kg+M8Dk7O6qejzwOAQurpu8VAahLqCR+LLpz0Y4FoONVUCdY5oy3AmHKOXLgurPA4PxYRjJ2yzQCW9IKruCrDOh8k0KC+u6mAHMakOXASInlVtSZ4hBcBUIP9eD899hWz7OAtF1VpXsl8VpzpfJsklnc+eNNIY1raZ359/SPvWJLf/hdgu9TKawMzTrPCmgODQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NckpYN7/zwJPv3x/GWrbGczaQHJ6Gj6GmEfoFeCagCFW61LfZuno2FD8rEtcsAXlmA9VgRj0tvEjydW4zQSTeub+V77wKsC1tcZ3LKLNqGxh+X8nJE+kz6VBBTMwu1PB+g9+kNSzbs/ZvSEf7H9WOrGOlTyaetH1cXQRnq/Ap7KgWAXg2+rijgbxhZvPwCkCsaZcTzxPlsBwo84eXajuPuKrvg3p74Ry5GX9Lfbjj5s0FA0Y0BB3u1NvS8IpbImLBxIJTvRGSuCBxYS3ZnU25WnVf3dDMcSwG5ZbHWWwGl/k7pJBIcs5n2ABRZ5WxBy/nQq+J2UMzSAWucQRgYUsDw==
  • Authentication-results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=suse.com;
  • Cc: "sstabellini@xxxxxxxxxx" <sstabellini@xxxxxxxxxx>, "mpohlack@xxxxxxxxx" <mpohlack@xxxxxxxxx>, "jgrall@xxxxxxxxxxxx" <jgrall@xxxxxxxxxxxx>, "doebel@xxxxxxxxx" <doebel@xxxxxxxxx>
  • Delivery-date: Wed, 11 Aug 2021 09:59:59 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHXjTpOV/XWdF2YLUCIZdYgL03Tzatrb1sAgAKlbQA=
  • Thread-topic: NULL scheduler DoS

On Mon, 2021-08-09 at 18:35 +0100, Julien Grall wrote:
> On 09/08/2021 17:19, Ahmed, Daniele wrote:
> > Hi all,
> 
> Hi Daniele,
> 
Hello everyone from me as well,

> Thank you for the report!
> 
Indeed. :-)

> 
> The ASSERT() is triggered because the pCPU was already assigned to
> one 
> of the dom0 vCPU. This problem is happening regardless whether there
> is 
> free pCPU.
> 
Right. Can we raise the appropriate log level, so that we can see these
messages:

dprintk(XENLOG_G_INFO, "%d <-- %pdv%d\n", cpu, unit->domain, unit->unit_id);

(and then see a full `xl dmesg`, or even better, a serial console dump,
since we crash! :-P)

> I have added some debugging in sched_set_res():
> 
> diff --git a/xen/common/sched/private.h b/xen/common/sched/private.h
> index a870320146ef..2355f531dc13 100644
> --- a/xen/common/sched/private.h
> +++ b/xen/common/sched/private.h
> @@ -150,6 +150,10 @@ static inline void sched_set_res(struct
> sched_unit 
> *unit,
>       unsigned int cpu = cpumask_first(res->cpus);
>       struct vcpu *v;
> 
> +    printk("%s: res->master_cpu %u unit %p %pd %pv\n", __func__,
> +           res->master_cpu, unit, unit->domain, unit->vcpu_list);
> +    WARN();
> +
>       for_each_sched_unit_vcpu ( unit, v )
>       {
>           ASSERT(cpu < nr_cpu_ids);
> 
> This traced the problem to null_unit_migrate():
> 
> (XEN) sched_set_res: res->master_cpu 0 unit ffff830200887f00 d1 d1v0
> (XEN) Xen WARN at private.h:155
> (XEN) ----[ Xen-4.16-unstable  x86_64  debug=y  Tainted:   C   ]----
> (XEN) CPU:    1
> (XEN) RIP:    e008:[<ffff82d04023fd9f>]
> core.c#sched_set_res+0x5b/0xc6
> [...]
> (XEN) Xen call trace:
> (XEN)    [<ffff82d04023fd9f>] R core.c#sched_set_res+0x5b/0xc6
> (XEN)    [<ffff82d040241614>] F sched_init_vcpu+0x3dc/0x5d7
> (XEN)    [<ffff82d04020527d>] F vcpu_create+0xfb/0x37a
> (XEN)    [<ffff82d040238dd9>] F do_domctl+0xac0/0x184a
> (XEN)    [<ffff82d04030d8bc>] F pv_hypercall+0x10d/0x2b8
> (XEN)    [<ffff82d04038829d>] F lstar_enter+0x12d/0x140
> (XEN)
> 
So, it's entirely possible that I'm missing something obvious here, but
what it is that is making you think that we're in null_unit_migrate() ?

Does that come from a different instance of this WARN() ?

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

Attachment: signature.asc
Description: This is a digitally signed message part


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.