From mirageos-devel-bounces@lists.xenproject.org Mon May 25 16:21:26 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Mon, 25 May 2020 16:21:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdFqd-00038x-67; Mon, 25 May 2020 16:21:07 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdFqc-00038R-4U for mirageos-devel@lists.xenproject.org; Mon, 25 May 2020 16:21:06 +0000 X-Inumbo-ID: ba31e596-9ea3-11ea-ae69-bc764e2007e4 Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id ba31e596-9ea3-11ea-ae69-bc764e2007e4; Mon, 25 May 2020 16:21:03 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id E07E2122804; Mon, 25 May 2020 18:04:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590422641; bh=BNe2R8tifla9iLYf7B+jPurPruxQbGfizIkZY85wFFQ=; h=Date:From:To:Cc:Subject:From; b=nJDNlPUTcplLpcXGA1OC2kJC3ykorw2KC32zRXopyDAAongcoYRPSOE3Bzd80aUG9 +wCuKbPx+NljUZPxapYylXrc/h6neHckG2N7y1bnncIOHRvaNVzc1nLitX2NXvmfrI dy7DiYnSMM4j2em88jDei3j8xQ+80YE1A1Np8sL4kbWN+rwVh5zs57cZxvY07GAJoU CVMuzrROpBSyr9rukyFJX81KFdLq0hvAPpy3CAZawPDKsFCFFkFuaEvcf+Ud3uzAIF jq3rCgwUyDSM2C+x7EkJunx32dYsbYOZFgAhrCX9vBAu38J4+esTMG/Pwb/XJRDYYQ tOntArvN7qIyA== Received: by nodbug.lucina.net (Postfix, from userid 1000) id B515E268436E; Mon, 25 May 2020 18:04:01 +0200 (CEST) Date: Mon, 25 May 2020 18:04:01 +0200 From: Martin Lucina To: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org Subject: Xen PVH domU start-of-day VCPU state Message-ID: <20200525160401.GA3091@nodbug.lucina.net> Mail-Followup-To: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, anil@recoil.org, dave@recoil.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: dave@recoil.org, anil@recoil.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" Hi, I'm trying to bootstrap a new PVH-only Xen domU OS "from scratch", to replace our existing use of Mini-OS for the early boot/low-level support layer in MirageOS. I've done this by creating new Xen bindings for Solo5 [1], basing them on our existing virtio code [2]. Unfortunately, I can't seem to get past the first few instructions on VCPU boot. Here's what I have at the moment (abridged): .section .note.solo5.xen .align 4 .long 4 .long 4 .long XEN_ELFNOTE_PHYS32_ENTRY .ascii "Xen\0" .long _start32 /* ... */ .code32 ENTRY(_start32) cld lgdt (gdt64_ptr) ljmp $0x10, $1f 1: movl $0x18, %eax movl %eax, %ds movl %eax, %es movl %eax, %ss xorl %eax, %eax movl %eax, %fs movl %eax, %gs I have verified, via xl -v create -c ..., that the domain builder appears to be doing the right thing, and is interpreting the ELF NOTE correctly. However, for some reason I cannot fathom, I get a triple fault on the ljmp following the lgdt instruction above: (XEN) d31v0 Triple fault - invoking HVM shutdown action 1 (XEN) *** Dumping Dom31 vcpu#0 state: *** (XEN) ----[ Xen-4.11.4-pre x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: 0000:[<0000000000100028>] (XEN) RFLAGS: 0000000000010002 CONTEXT: hvm guest (d31v0) (XEN) rax: 0000000000000000 rbx: 0000000000116000 rcx: 0000000000000000 (XEN) rdx: 0000000000000000 rsi: 0000000000000000 rdi: 0000000000000000 (XEN) rbp: 0000000000000000 rsp: 0000000000000000 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000000000011 cr4: 0000000000000000 (XEN) cr3: 0000000000000000 cr2: 0000000000000000 (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: 0000 Cross-checking 0x100028 via gdb: Dump of assembler code for function _start32: 0x00100020 <+0>: cld 0x00100021 <+1>: lgdtl 0x108040 0x00100028 <+8>: ljmp $0x10,$0x10002f 0x0010002f <+15>: mov $0x18,%eax I've spent a couple of days trying various things and cross-checking both with the Mini-OS PVH/HVM startup [3] and the Intel SDM, but no joy. I've also re-checked the GDT selector values used by the original virtio code which this is based on, and they appear to be fine. This is not helped by the fact that the Xen domU PVH start-of-day VCPU state does not seem to be documented anywhere, with the exception of "struct hvm_start_info is passed in %ebx" as stated in arch-x86/hvm/start_info.h. In case it's relevant, I'm testing with Xen 4.11.4 as shipped with Debian 10, on an Intel Broadwell CPU. Any ideas? Any help much appreciated. Thanks, -mato [1] https://github.com/mato/solo5/tree/xen/bindings/xen / https://github.com/mato/solo5/commit/f2539d588883a2e8854998c75bdea9b10f113ed6 [2] https://github.com/mato/solo5/tree/xen/bindings/virtio [3] https://xenbits.xen.org/gitweb/?p=mini-os.git;a=blob;f=arch/x86/x86_hvm.S;h=6e8ad983a16adbe97b343f7dbc17e281ee0c389f;hb=HEAD From mirageos-devel-bounces@lists.xenproject.org Mon May 25 16:43:08 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Mon, 25 May 2020 16:43:08 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdGBr-00058Z-SR; Mon, 25 May 2020 16:43:03 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdGBq-00058U-TR for mirageos-devel@lists.xenproject.org; Mon, 25 May 2020 16:43:02 +0000 X-Inumbo-ID: c66a1128-9ea6-11ea-aef8-12813bfff9fa Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id c66a1128-9ea6-11ea-aef8-12813bfff9fa; Mon, 25 May 2020 16:42:52 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id D7BD3AC51; Mon, 25 May 2020 16:42:53 +0000 (UTC) Subject: Re: Xen PVH domU start-of-day VCPU state To: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, anil@recoil.org, dave@recoil.org References: <20200525160401.GA3091@nodbug.lucina.net> From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= Message-ID: Date: Mon, 25 May 2020 18:42:50 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <20200525160401.GA3091@nodbug.lucina.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On 25.05.20 18:04, Martin Lucina wrote: > Hi, > > I'm trying to bootstrap a new PVH-only Xen domU OS "from scratch", to > replace our existing use of Mini-OS for the early boot/low-level support > layer in MirageOS. I've done this by creating new Xen bindings for Solo5 > [1], basing them on our existing virtio code [2]. > > Unfortunately, I can't seem to get past the first few instructions on VCPU > boot. Here's what I have at the moment (abridged): > > .section .note.solo5.xen > > .align 4 > .long 4 > .long 4 > .long XEN_ELFNOTE_PHYS32_ENTRY > .ascii "Xen\0" > .long _start32 > > /* ... */ > > .code32 > > ENTRY(_start32) > cld > > lgdt (gdt64_ptr) > ljmp $0x10, $1f You need to setup virtual addressing and enable 64 bit mode before using 64-bit GDT. See Mini-OS source arch/x86/x86_hvm.S Juergen From mirageos-devel-bounces@lists.xenproject.org Mon May 25 16:59:46 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Mon, 25 May 2020 16:59:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdGRx-0006JK-MI; Mon, 25 May 2020 16:59:41 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdGRx-0006JF-4f for mirageos-devel@lists.xenproject.org; Mon, 25 May 2020 16:59:41 +0000 X-Inumbo-ID: 1f3be284-9ea9-11ea-aef9-12813bfff9fa Received: from ppsw-31.csi.cam.ac.uk (unknown [131.111.8.131]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1f3be284-9ea9-11ea-aef9-12813bfff9fa; Mon, 25 May 2020 16:59:40 +0000 (UTC) X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://help.uis.cam.ac.uk/email-scanner-virus Received: from 88-109-182-220.dynamic.dsl.as9105.com ([88.109.182.220]:42506 helo=[192.168.1.219]) by ppsw-31.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:465) with esmtpsa (PLAIN:amc96) (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) id 1jdGRu-0000n7-M1 (Exim 4.92.3) (return-path ); Mon, 25 May 2020 17:59:38 +0100 Subject: Re: Xen PVH domU start-of-day VCPU state To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, anil@recoil.org, dave@recoil.org References: <20200525160401.GA3091@nodbug.lucina.net> From: Andrew Cooper Message-ID: <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> Date: Mon, 25 May 2020 17:59:38 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-GB X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On 25/05/2020 17:42, Jürgen Groß wrote: > On 25.05.20 18:04, Martin Lucina wrote: >> Hi, >> >> I'm trying to bootstrap a new PVH-only Xen domU OS "from scratch", to >> replace our existing use of Mini-OS for the early boot/low-level support >> layer in MirageOS. I've done this by creating new Xen bindings for Solo5 >> [1], basing them on our existing virtio code [2]. >> >> Unfortunately, I can't seem to get past the first few instructions on >> VCPU >> boot. Here's what I have at the moment (abridged): >> >>      .section .note.solo5.xen >> >>              .align  4 >>              .long   4 >>              .long   4 >>              .long   XEN_ELFNOTE_PHYS32_ENTRY >>              .ascii "Xen\0" >>              .long   _start32 >> >>      /* ... */ >> >>      .code32 >> >>      ENTRY(_start32) >>              cld >> >>              lgdt (gdt64_ptr) >>              ljmp $0x10, $1f > > You need to setup virtual addressing and enable 64 bit mode before using > 64-bit GDT. > > See Mini-OS source arch/x86/x86_hvm.S Or https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD But yes - Juergen is correct.  Until you have enabled long mode, lgdt will only load the bottom 32 bits of GDTR.base. Is there a less abridged version to look at? ~Andrew From mirageos-devel-bounces@lists.xenproject.org Mon May 25 17:41:30 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Mon, 25 May 2020 17:41:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdH6H-0002Dl-Gy; Mon, 25 May 2020 17:41:21 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdH6G-0002De-B3 for mirageos-devel@lists.xenproject.org; Mon, 25 May 2020 17:41:20 +0000 X-Inumbo-ID: ed4dd754-9eae-11ea-b9cf-bc764e2007e4 Received: from ppsw-31.csi.cam.ac.uk (unknown [131.111.8.131]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id ed4dd754-9eae-11ea-b9cf-bc764e2007e4; Mon, 25 May 2020 17:41:13 +0000 (UTC) X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://help.uis.cam.ac.uk/email-scanner-virus Received: from 88-109-182-220.dynamic.dsl.as9105.com ([88.109.182.220]:43642 helo=[192.168.1.219]) by ppsw-31.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:465) with esmtpsa (PLAIN:amc96) (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) id 1jdH68-000Mfb-Jg (Exim 4.92.3) (return-path ); Mon, 25 May 2020 18:41:12 +0100 Subject: Re: Xen PVH domU start-of-day VCPU state To: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, anil@recoil.org, dave@recoil.org, martin@lucina.net References: <20200525160401.GA3091@nodbug.lucina.net> From: Andrew Cooper Message-ID: <6fadfd84-0fc4-d462-a917-1c88ec0822b8@citrix.com> Date: Mon, 25 May 2020 18:41:11 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200525160401.GA3091@nodbug.lucina.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-GB X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On 25/05/2020 17:04, Martin Lucina wrote: > Hi, > > I'm trying to bootstrap a new PVH-only Xen domU OS "from scratch", to > replace our existing use of Mini-OS for the early boot/low-level support > layer in MirageOS. I've done this by creating new Xen bindings for Solo5 > [1], basing them on our existing virtio code [2]. > > Unfortunately, I can't seem to get past the first few instructions on VCPU > boot. Here's what I have at the moment (abridged): > > .section .note.solo5.xen > > .align 4 > .long 4 > .long 4 > .long XEN_ELFNOTE_PHYS32_ENTRY > .ascii "Xen\0" > .long _start32 > > /* ... */ > > .code32 > > ENTRY(_start32) > cld > > lgdt (gdt64_ptr) > ljmp $0x10, $1f > > 1: movl $0x18, %eax > movl %eax, %ds > movl %eax, %es > movl %eax, %ss > > xorl %eax, %eax > movl %eax, %fs > movl %eax, %gs > > I have verified, via xl -v create -c ..., that the domain builder appears > to be doing the right thing, and is interpreting the ELF NOTE correctly. > However, for some reason I cannot fathom, I get a triple fault on the ljmp > following the lgdt instruction above: > > (XEN) d31v0 Triple fault - invoking HVM shutdown action 1 > (XEN) *** Dumping Dom31 vcpu#0 state: *** > (XEN) ----[ Xen-4.11.4-pre x86_64 debug=n Not tainted ]---- > (XEN) CPU: 0 > (XEN) RIP: 0000:[<0000000000100028>] > (XEN) RFLAGS: 0000000000010002 CONTEXT: hvm guest (d31v0) > (XEN) rax: 0000000000000000 rbx: 0000000000116000 rcx: 0000000000000000 > (XEN) rdx: 0000000000000000 rsi: 0000000000000000 rdi: 0000000000000000 > (XEN) rbp: 0000000000000000 rsp: 0000000000000000 r8: 0000000000000000 > (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 > (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 > (XEN) r15: 0000000000000000 cr0: 0000000000000011 cr4: 0000000000000000 > (XEN) cr3: 0000000000000000 cr2: 0000000000000000 > (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000 > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: 0000 For extra help debugging this, you can dump the vmcs here: andrewcoop@andrewcoop:/local/xen.git/xen$ git d diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 74c9f84462..8ae23545ae 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -1687,6 +1687,7 @@ void hvm_triple_fault(void)              "Triple fault - invoking HVM shutdown action %d\n",              reason);      vcpu_show_execution_state(v); +    vmcs_dump_vcpu(v);      domain_shutdown(d, reason);  }   which will include the segment cache, including the just loaded GDT details. > Cross-checking 0x100028 via gdb: > > Dump of assembler code for function _start32: > 0x00100020 <+0>: cld > 0x00100021 <+1>: lgdtl 0x108040 > 0x00100028 <+8>: ljmp $0x10,$0x10002f > 0x0010002f <+15>: mov $0x18,%eax > > I've spent a couple of days trying various things and cross-checking both > with the Mini-OS PVH/HVM startup [3] and the Intel SDM, but no joy. I've > also re-checked the GDT selector values used by the original virtio code > which this is based on, and they appear to be fine. > > This is not helped by the fact that the Xen domU PVH start-of-day VCPU > state does not seem to be documented anywhere, with the exception of > "struct hvm_start_info is passed in %ebx" as stated in > arch-x86/hvm/start_info.h. https://xenbits.xen.org/docs/unstable/misc/pvh.html The starting state is described there.  It is 32bit flat mode, very similar to multiboot's entry. > In case it's relevant, I'm testing with Xen 4.11.4 as shipped with Debian > 10, on an Intel Broadwell CPU. > > Any ideas? Sadly no. From https://github.com/mato/solo5/commit/f2539d588883a2e8854998c75bdea9b10f113ed6 all data looks to be linked below the 4G boundary, so the 32/64bitness of lgdt shouldn't matter in this case. Reordering the logic as per MiniOS/XTF will avoid the need for a 32bit CS selector - it is safe to run on the ABI-provided %cs until you switch into 64bit mode. It might also be interesting to see exactly what value is in gdt64_ptr, just to check that the base an limit are set sensibly. ~Andrew From mirageos-devel-bounces@lists.xenproject.org Tue May 26 08:27:23 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 08:27:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdUvT-00066q-1N; Tue, 26 May 2020 08:27:07 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdUvS-00066l-Jb for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 08:27:06 +0000 X-Inumbo-ID: adfd328a-9f2a-11ea-9947-bc764e2007e4 Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id adfd328a-9f2a-11ea-9947-bc764e2007e4; Tue, 26 May 2020 08:27:05 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 2B303122804; Tue, 26 May 2020 10:27:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590481624; bh=nMbw4GKCmitbAyTsPCbgeJZDMgcv0pZGQeT3lxD0qLU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=O58q11Z6PMCr4KpKLZFxlfh72IAdiaHSUro7vrskbvfQB/G5jvVkuiXFfcvnHRbGe usat0Jdc6C2ZbqD+CQD7P3xTrQDmcHCcjupohbNSQaRK3lJ7nEHfD90zKK9xIFf7Yz OCT+Y1VsUkrf01ysO9rkAi2gt2KCBZIiVFClD1SLNt0PNXu/pyYfF+z+YtjyIk1i6O 8FwTb23AFsnPBoP6FZYURvZksDbDrTLJkYRK4IMImQbyKlUsqCC7otXFPKC3uaC3a9 NsE4Fmw5F6yWGtiJdB5e9xCA81Oo8H+TcTH9KGld8bg+pw+lIi0Lx0NmIZTvQsTY+s y2Qh/go8Nwzcw== Received: by nodbug.lucina.net (Postfix, from userid 1000) id EB756268436E; Tue, 26 May 2020 10:27:03 +0200 (CEST) Date: Tue, 26 May 2020 10:27:03 +0200 From: Martin Lucina To: Andrew Cooper Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526082703.GA5942@nodbug.lucina.net> Mail-Followup-To: Andrew Cooper , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, anil@recoil.org, dave@recoil.org References: <20200525160401.GA3091@nodbug.lucina.net> <6fadfd84-0fc4-d462-a917-1c88ec0822b8@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6fadfd84-0fc4-d462-a917-1c88ec0822b8@citrix.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: xen-devel@lists.xenproject.org, dave@recoil.org, mirageos-devel@lists.xenproject.org, anil@recoil.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Monday, 25.05.2020 at 18:41, Andrew Cooper wrote: > On 25/05/2020 17:04, Martin Lucina wrote: > > Hi, > > > > I'm trying to bootstrap a new PVH-only Xen domU OS "from scratch", to > > replace our existing use of Mini-OS for the early boot/low-level support > > layer in MirageOS. I've done this by creating new Xen bindings for Solo5 > > [1], basing them on our existing virtio code [2]. > > > > Unfortunately, I can't seem to get past the first few instructions on VCPU > > boot. Here's what I have at the moment (abridged): > > > > .section .note.solo5.xen > > > > .align 4 > > .long 4 > > .long 4 > > .long XEN_ELFNOTE_PHYS32_ENTRY > > .ascii "Xen\0" > > .long _start32 > > > > /* ... */ > > > > .code32 > > > > ENTRY(_start32) > > cld > > > > lgdt (gdt64_ptr) > > ljmp $0x10, $1f > > > > 1: movl $0x18, %eax > > movl %eax, %ds > > movl %eax, %es > > movl %eax, %ss > > > > xorl %eax, %eax > > movl %eax, %fs > > movl %eax, %gs > > > > I have verified, via xl -v create -c ..., that the domain builder appears > > to be doing the right thing, and is interpreting the ELF NOTE correctly. > > However, for some reason I cannot fathom, I get a triple fault on the ljmp > > following the lgdt instruction above: > > > > (XEN) d31v0 Triple fault - invoking HVM shutdown action 1 > > (XEN) *** Dumping Dom31 vcpu#0 state: *** > > (XEN) ----[ Xen-4.11.4-pre x86_64 debug=n Not tainted ]---- > > (XEN) CPU: 0 > > (XEN) RIP: 0000:[<0000000000100028>] > > (XEN) RFLAGS: 0000000000010002 CONTEXT: hvm guest (d31v0) > > (XEN) rax: 0000000000000000 rbx: 0000000000116000 rcx: 0000000000000000 > > (XEN) rdx: 0000000000000000 rsi: 0000000000000000 rdi: 0000000000000000 > > (XEN) rbp: 0000000000000000 rsp: 0000000000000000 r8: 0000000000000000 > > (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 > > (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 > > (XEN) r15: 0000000000000000 cr0: 0000000000000011 cr4: 0000000000000000 > > (XEN) cr3: 0000000000000000 cr2: 0000000000000000 > > (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000 > > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: 0000 > > For extra help debugging this, you can dump the vmcs here: > > andrewcoop@andrewcoop:/local/xen.git/xen$ git d > diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c > index 74c9f84462..8ae23545ae 100644 > --- a/xen/arch/x86/hvm/hvm.c > +++ b/xen/arch/x86/hvm/hvm.c > @@ -1687,6 +1687,7 @@ void hvm_triple_fault(void) >              "Triple fault - invoking HVM shutdown action %d\n", >              reason); >      vcpu_show_execution_state(v); > +    vmcs_dump_vcpu(v); >      domain_shutdown(d, reason); >  } >   > > which will include the segment cache, including the just loaded GDT details. Thanks, I'll try that and report back. > > > Cross-checking 0x100028 via gdb: > > > > Dump of assembler code for function _start32: > > 0x00100020 <+0>: cld > > 0x00100021 <+1>: lgdtl 0x108040 > > 0x00100028 <+8>: ljmp $0x10,$0x10002f > > 0x0010002f <+15>: mov $0x18,%eax > > > > I've spent a couple of days trying various things and cross-checking both > > with the Mini-OS PVH/HVM startup [3] and the Intel SDM, but no joy. I've > > also re-checked the GDT selector values used by the original virtio code > > which this is based on, and they appear to be fine. > > > > This is not helped by the fact that the Xen domU PVH start-of-day VCPU > > state does not seem to be documented anywhere, with the exception of > > "struct hvm_start_info is passed in %ebx" as stated in > > arch-x86/hvm/start_info.h. > > https://xenbits.xen.org/docs/unstable/misc/pvh.html > > The starting state is described there.  It is 32bit flat mode, very > similar to multiboot's entry. > > > In case it's relevant, I'm testing with Xen 4.11.4 as shipped with Debian > > 10, on an Intel Broadwell CPU. > > > > Any ideas? > > Sadly no. > > From > https://github.com/mato/solo5/commit/f2539d588883a2e8854998c75bdea9b10f113ed6 > > all data looks to be linked below the 4G boundary, so the 32/64bitness > of lgdt shouldn't matter in this case. That's correct, the virtio code this is based on doesn't use anything above 1GB. > Reordering the logic as per MiniOS/XTF will avoid the need for a 32bit > CS selector - it is safe to run on the ABI-provided %cs until you switch > into 64bit mode. I can try poking at the order some more, but was aiming for a minimal diff against virtio to start with. > It might also be interesting to see exactly what value is in gdt64_ptr, > just to check that the base an limit are set sensibly. Seems fine: (gdb) info address gdt64_ptr Symbol "gdt64_ptr" is at 0x108040 in a file compiled without debugging. (gdb) x /1xg 0x108040 0x108040: 0x000000108000002f (gdb) p/x (struct gdtptr)gdt64_ptr $3 = {limit = 0x2f, base = 0x108000} -mato From mirageos-devel-bounces@lists.xenproject.org Tue May 26 08:52:34 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 08:52:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdVK2-0000Qg-TA; Tue, 26 May 2020 08:52:30 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdVK1-0000QN-OV for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 08:52:29 +0000 X-Inumbo-ID: 3760aee6-9f2e-11ea-9947-bc764e2007e4 Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 3760aee6-9f2e-11ea-9947-bc764e2007e4; Tue, 26 May 2020 08:52:24 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 182F3122804; Tue, 26 May 2020 10:52:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590483142; bh=XwDVAhYIHN59igx5z/ZwfLAUYZdwFiO8r9vROke/t0s=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=gcGJMuBsYGMtvtKYzIbnugjaX8mt0TgWIGKfN0MYQYf6YfB0rnbpeBn8oxJ+mVlW1 BHkwh9+oXWVjWW5caGEBIcWaExTGk9g+xxUR4vQXfl+WmTHmTWRqd7WTv+0Ven7LmH yVfVx45jtlVfna12QO7Tb+EfsjbQNGhIGU38rdqz+m2lJ9rzZQUpmiUT1KXH7dc9X6 jpPccijphOyl/4WDZhD51i/VXvfYzO44H99+UdxZcCdbCbnFBMcYYLlRNxpmOCK//s Q4dEdfQEib+QJWvJCkEyH7B8FJAapNh0KCBlVn0ZLvHDnqywCzOeFzDUn7To6Su0zz iZajJDdb/9pjg== Received: by nodbug.lucina.net (Postfix, from userid 1000) id EB3AB268436E; Tue, 26 May 2020 10:52:21 +0200 (CEST) Date: Tue, 26 May 2020 10:52:21 +0200 From: Martin Lucina To: Andrew Cooper Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526085221.GB5942@nodbug.lucina.net> Mail-Followup-To: Andrew Cooper , =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, anil@recoil.org, dave@recoil.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , xen-devel@lists.xenproject.org, dave@recoil.org, mirageos-devel@lists.xenproject.org, anil@recoil.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > On 25/05/2020 17:42, Jürgen Groß wrote: > > You need to setup virtual addressing and enable 64 bit mode before using > > 64-bit GDT. > > > > See Mini-OS source arch/x86/x86_hvm.S > > Or > https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > > But yes - Juergen is correct.  Until you have enabled long mode, lgdt > will only load the bottom 32 bits of GDTR.base. Ah, I missed Jurgen's and your reply here. LGDT loading only the bottom 32 bits of GDTR.base shouldn't matter. Examining gdt_ptr some more: (gdb) set architecture i386 The target architecture is assumed to be i386 (gdb) x /xh 0x108040 0x108040: 0x002f (gdb) x /xw 0x108042 0x108042: 0x00108000 (gdb) x /6xb 0x108040 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 (gdb) x /8xb 0x108040 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 0x00 0x00 > Is there a less abridged version to look at? https://github.com/mato/solo5/blob/xen/bindings/xen/boot.S As I wrote in another reply, this boot.S is based off the virtio (multiboot) boot.S, which has worked fine for years and should have the same environment (32-bit flat protected mode). -mato From mirageos-devel-bounces@lists.xenproject.org Tue May 26 09:35:05 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 09:35:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdVz0-0004Zp-QW; Tue, 26 May 2020 09:34:50 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdVz0-0004Zk-6m for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 09:34:50 +0000 X-Inumbo-ID: 24249198-9f34-11ea-9947-bc764e2007e4 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 24249198-9f34-11ea-9947-bc764e2007e4; Tue, 26 May 2020 09:34:48 +0000 (UTC) Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: WuLYWAQddVlidClve6sgpt1aMbR1w/JuXyEnOdxAtjCsL/iK1mzpoExUyUmnlpcZK8yY+xdhcr Fx2u0I2BEK/kPiAGjG/7RNvJUyMor2aGJQHKwGRK1kluQi5ZoengjiHaO+vniDa4m3buLacnvC uaUIVwO5Q+eN7m8KORkzoyr4DvieRelbBHDriH5preC+GwmWMPDMXvJgEq9ZbB3eb9yat4YbYw EysZQmH7b1ZplhSs7BuELbE/SwuPQLl0QgueZPeTqxpHKdYjFtc9D3D/KdedZSZo0MMFIuLa2t l44= X-SBRS: None X-MesageID: 18390190 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-URL-LookUp-ScanningError: 1 Date: Tue, 26 May 2020 11:34:21 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Martin Lucina Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526093421.GA38408@Air-de-Roger> References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526085221.GB5942@nodbug.lucina.net> X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tue, May 26, 2020 at 10:52:21AM +0200, Martin Lucina wrote: > On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > > On 25/05/2020 17:42, Jürgen Groß wrote: > > > You need to setup virtual addressing and enable 64 bit mode before using > > > 64-bit GDT. > > > > > > See Mini-OS source arch/x86/x86_hvm.S > > > > Or > > https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > > > > But yes - Juergen is correct.  Until you have enabled long mode, lgdt > > will only load the bottom 32 bits of GDTR.base. > > Ah, I missed Jurgen's and your reply here. > > LGDT loading only the bottom 32 bits of GDTR.base shouldn't matter. > Examining gdt_ptr some more: > > (gdb) set architecture i386 > The target architecture is assumed to be i386 > (gdb) x /xh 0x108040 > 0x108040: 0x002f > (gdb) x /xw 0x108042 > 0x108042: 0x00108000 > (gdb) x /6xb 0x108040 > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 > (gdb) x /8xb 0x108040 > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 0x00 0x00 Could you also print the GDT entry at 0x10 (ie: 0x108000 + 0x10), just to make sure it contains the right descriptor? Roger. From mirageos-devel-bounces@lists.xenproject.org Tue May 26 10:04:13 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 10:04:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWRN-0007x1-6x; Tue, 26 May 2020 10:04:09 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWRM-0007w4-CD for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 10:04:08 +0000 X-Inumbo-ID: 3c1add44-9f38-11ea-a5fb-12813bfff9fa Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 3c1add44-9f38-11ea-a5fb-12813bfff9fa; Tue, 26 May 2020 10:04:07 +0000 (UTC) Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: sEdFN5Lh6elRsteVMPSO85RlLCzyksheTxVv0xvTAvzuSXt7L8hhnqt72/SDbi20OfRac0z5Tr GwE/4Do88nreLCpzqNT3Y0IxHINxhOYoRlhh/tZlHU+MjMvzfMZSebve4g06MR/JhJ9L9x0My4 WTUxLdh7JUkDzhwDScjKRyw6S5t+pDXytc8Y13XJfkTGoLlKCC5VSQEX1TffzaT5GnPhxRDtnW dAboLqFhhaBzW2eOs76qQ3FCh//kdAwewGfoSeTsuLYNv2qm20H+8H2fjOBDIbcJIuq5HvxPfn EGg= X-SBRS: None X-MesageID: 18394027 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-URL-LookUp-ScanningError: 1 Date: Tue, 26 May 2020 12:03:37 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Martin Lucina Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526100337.GB38408@Air-de-Roger> References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526093421.GA38408@Air-de-Roger> X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tue, May 26, 2020 at 11:34:21AM +0200, Roger Pau Monné wrote: > On Tue, May 26, 2020 at 10:52:21AM +0200, Martin Lucina wrote: > > On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > > > On 25/05/2020 17:42, Jürgen Groß wrote: > > > > You need to setup virtual addressing and enable 64 bit mode before using > > > > 64-bit GDT. > > > > > > > > See Mini-OS source arch/x86/x86_hvm.S > > > > > > Or > > > https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > > > > > > But yes - Juergen is correct.  Until you have enabled long mode, lgdt > > > will only load the bottom 32 bits of GDTR.base. > > > > Ah, I missed Jurgen's and your reply here. > > > > LGDT loading only the bottom 32 bits of GDTR.base shouldn't matter. > > Examining gdt_ptr some more: > > > > (gdb) set architecture i386 > > The target architecture is assumed to be i386 > > (gdb) x /xh 0x108040 > > 0x108040: 0x002f > > (gdb) x /xw 0x108042 > > 0x108042: 0x00108000 > > (gdb) x /6xb 0x108040 > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 > > (gdb) x /8xb 0x108040 > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 0x00 0x00 > > Could you also print the GDT entry at 0x10 (ie: 0x108000 + 0x10), just > to make sure it contains the right descriptor? Forgot to ask, but can you also add the output of readelf -lW ? Roger. From mirageos-devel-bounces@lists.xenproject.org Tue May 26 10:08:17 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 10:08:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWVL-0008Hn-7A; Tue, 26 May 2020 10:08:15 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWVJ-0008Hd-G9 for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 10:08:13 +0000 X-Inumbo-ID: cb620d74-9f38-11ea-a5fe-12813bfff9fa Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id cb620d74-9f38-11ea-a5fe-12813bfff9fa; Tue, 26 May 2020 10:08:07 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 725C6122804; Tue, 26 May 2020 12:08:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590487686; bh=hqRXSznMCr4tJH1FgF5vK1/580jutbWPg/BvlK0PEEU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=ns3/sJIXPIqV52TPuBJPwBfQJCtih9cK/sXO9G3+En8yA2D8lU/iLyddoJvZZM6U0 +Ohcw1P6oSEaKtIznsl7bhChiJsoFMV3r2ysjRZzN0ICK2zkChTzPOesoW7FpI4fqf Hlneod7iPocEvVM0Lga6+xc7/d3gHFa1gSZYNajLfaVSyBnb800PWRCdTOfxKVmhmT EJzSRC55gqS6qW+TQXVrJv440cAWASK9qw7SZDflurmmJ8DDbgyGEIhgMmngs6HZqh 5Ft6fSC9YvoYWJtAQlP3TzD7BpIARfxnxLGHdS2D0UxCcklMFzTY+MMav5JcGmK1g+ rqbK3dB5M45Aw== Received: by nodbug.lucina.net (Postfix, from userid 1000) id 14173268436E; Tue, 26 May 2020 12:08:06 +0200 (CEST) Date: Tue, 26 May 2020 12:08:06 +0200 From: Martin Lucina To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526100806.GD5942@nodbug.lucina.net> Mail-Followup-To: Roger Pau =?iso-8859-1?Q?Monn=E9?= , =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526093421.GA38408@Air-de-Roger> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tuesday, 26.05.2020 at 11:34, Roger Pau Monné wrote: > On Tue, May 26, 2020 at 10:52:21AM +0200, Martin Lucina wrote: > > On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > > > On 25/05/2020 17:42, Jürgen Groß wrote: > > > > You need to setup virtual addressing and enable 64 bit mode before using > > > > 64-bit GDT. > > > > > > > > See Mini-OS source arch/x86/x86_hvm.S > > > > > > Or > > > https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > > > > > > But yes - Juergen is correct.  Until you have enabled long mode, lgdt > > > will only load the bottom 32 bits of GDTR.base. > > > > Ah, I missed Jurgen's and your reply here. > > > > LGDT loading only the bottom 32 bits of GDTR.base shouldn't matter. > > Examining gdt_ptr some more: > > > > (gdb) set architecture i386 > > The target architecture is assumed to be i386 > > (gdb) x /xh 0x108040 > > 0x108040: 0x002f > > (gdb) x /xw 0x108042 > > 0x108042: 0x00108000 > > (gdb) x /6xb 0x108040 > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 > > (gdb) x /8xb 0x108040 > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 0x00 0x00 > > Could you also print the GDT entry at 0x10 (ie: 0x108000 + 0x10), just > to make sure it contains the right descriptor? I triple-checked that on Friday, but here you go: (gdb) x /xg 0x108010 0x108010: 0x00cf9b000000ffff (gdb) x /tg 0x108010 0x108010: 0000000011001111100110110000000000000000000000001111111111111111 Translates to: base_31_24 = { 0 } g = 1 (4 kB) b = 1 (32-bit) l = 0 (32-bit) avl = 0 limit_19_16 = { 1 } p = 1 dpl = 0 s = 1 type = 1011 (code, exec/read, accessed) base23_16 = { 0 } base15_0 = { 0 } limit_15_0 = { 1 } type should technically not include accessed, but that shouldn't matter. In any case, changing it to 1010 does not help. Looks like I'll have to build a patched Xen as per Andrew's suggestion, and dump the VMCS on the triple fault. -mato From mirageos-devel-bounces@lists.xenproject.org Tue May 26 10:12:12 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 10:12:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWZ8-0000jk-RA; Tue, 26 May 2020 10:12:10 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWZ8-0000jf-Il for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 10:12:10 +0000 X-Inumbo-ID: 589ff96c-9f39-11ea-a5fe-12813bfff9fa Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 589ff96c-9f39-11ea-a5fe-12813bfff9fa; Tue, 26 May 2020 10:12:04 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 6AC7D122804; Tue, 26 May 2020 12:12:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590487923; bh=ktMxHkN8RbubbKp+2Me4K8XKCkvYrm8mNijcddKeBhQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=es4vTGLQqFdUhQ+yBIXQPO6grScTZUzrVVbgxaAR0elX/6pAF6McJX4Ko4AF9lYvI h2Eb3E9Motw5p4CKmhEGMZ+cq8ejEqdGQxihe0sy9DAgPsRdnGfVFN8+BRK5Ke9h/U pEY0ByiS9jDd+KtXGCS/mYD80ATOPXXkaIi2pM2FjMTwqvap/wfogDJmm3TmP4PDru OnjDBN8ofpW8zx8NAzev1ePPegKcuOpg4SCgZ+R7c8Wu4OVhKx7nC+v18UrzonAGpH 37zxy7tk/iRxRZKgx3ws/kQRcBz12i8kE3Eg6OVT/bb4AbaohKqAhJDRUft9Io7Pu6 cA/YgIOrEOXAA== Received: by nodbug.lucina.net (Postfix, from userid 1000) id 4542B268436E; Tue, 26 May 2020 12:12:03 +0200 (CEST) Date: Tue, 26 May 2020 12:12:03 +0200 From: Martin Lucina To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526101203.GE5942@nodbug.lucina.net> Mail-Followup-To: Roger Pau =?iso-8859-1?Q?Monn=E9?= , =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526100337.GB38408@Air-de-Roger> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tuesday, 26.05.2020 at 12:03, Roger Pau Monné wrote: > On Tue, May 26, 2020 at 11:34:21AM +0200, Roger Pau Monné wrote: > > On Tue, May 26, 2020 at 10:52:21AM +0200, Martin Lucina wrote: > > > On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > > > > On 25/05/2020 17:42, Jürgen Groß wrote: > > > > > You need to setup virtual addressing and enable 64 bit mode before using > > > > > 64-bit GDT. > > > > > > > > > > See Mini-OS source arch/x86/x86_hvm.S > > > > > > > > Or > > > > https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > > > > > > > > But yes - Juergen is correct.  Until you have enabled long mode, lgdt > > > > will only load the bottom 32 bits of GDTR.base. > > > > > > Ah, I missed Jurgen's and your reply here. > > > > > > LGDT loading only the bottom 32 bits of GDTR.base shouldn't matter. > > > Examining gdt_ptr some more: > > > > > > (gdb) set architecture i386 > > > The target architecture is assumed to be i386 > > > (gdb) x /xh 0x108040 > > > 0x108040: 0x002f > > > (gdb) x /xw 0x108042 > > > 0x108042: 0x00108000 > > > (gdb) x /6xb 0x108040 > > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 > > > (gdb) x /8xb 0x108040 > > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 0x00 0x00 > > > > Could you also print the GDT entry at 0x10 (ie: 0x108000 + 0x10), just > > to make sure it contains the right descriptor? > > Forgot to ask, but can you also add the output of readelf -lW > ? Elf file type is EXEC (Executable file) Entry point 0x1001e0 There are 7 program headers, starting at offset 64 Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 [Requesting program interpreter: /nonexistent/solo5/] LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00626c 0x00626c R E 0x1000 LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed48 RW 0x1000 NOTE 0x0080ac 0x00000000001070ac 0x00000000001070ac 0x000018 0x000018 R 0x4 NOTE 0x00f120 0x00000000001070c4 0x00000000001070c4 0x000014 0x000000 R 0x4 NOTE 0x008088 0x0000000000107088 0x0000000000107088 0x000024 0x000024 R 0x4 NOTE 0x008000 0x0000000000107000 0x0000000000107000 0x000088 0x000088 R 0x4 Section to Segment mapping: Segment Sections... 00 .interp 01 .interp .text .rodata .eh_frame 02 .note.solo5.manifest .note.solo5.abi .note.solo5.not-openbsd .data .bss 03 .note.solo5.not-openbsd 04 .note.solo5.xen 05 .note.solo5.abi 06 .note.solo5.manifest The PT_INTERP and multiple PT_NOTE headers are that way due to specifics of how Solo5 ABIs work, but I've verified that the domain builder is interpreting XEN_ELFNOTE_PHYS32_ENTRY correctly. -mato From mirageos-devel-bounces@lists.xenproject.org Tue May 26 10:32:53 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 10:32:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWt5-0002fG-Fz; Tue, 26 May 2020 10:32:47 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdWt4-0002fA-V1 for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 10:32:46 +0000 X-Inumbo-ID: 3c54960c-9f3c-11ea-a601-12813bfff9fa Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 3c54960c-9f3c-11ea-a601-12813bfff9fa; Tue, 26 May 2020 10:32:45 +0000 (UTC) Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: xxe8AOw4QDajVG4fl/Tl4hCojrrpsVEXkYh8SHc39ewwj/UFxLeOjKDlbL0LNICLFez0GsYRou ekEKJWG3u1nPA1+F61Kw4PWIiaM30pg88uDP1t2qyohyBoljWPOnWIfPD4CgK5kE6OGvqNAhxG pJ74kWBcbq4MmdCiklRivyLo4pt8vCViaDYcrYGXUOjgdP9wGQxtmfiUyeponFjQsBpeRl7PSe 43VitVyCZ9rTtVWRWXfRz8m2swzx8YoMc0iWhTQ6916qx9KC27/8Mb/HqQgE58ifC2/qBz3g1+ PKU= X-SBRS: None X-MesageID: 18425278 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-URL-LookUp-ScanningError: 1 Date: Tue, 26 May 2020 12:32:18 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Martin Lucina Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526102707.GC38408@Air-de-Roger> References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> <20200526101203.GE5942@nodbug.lucina.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526101203.GE5942@nodbug.lucina.net> X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" BTW, not sure why but my MUA (Mutt 11.0.3) seems to add everyone on Cc to the To: field on reply, and drops your email address from the list. I don't see a 'Reply-to:' on the headers, so I'm not sure why it does that, but you might want to check your config. I have to manually fix the headers to properly set the To: field to your address and the Cc to everyone else. On Tue, May 26, 2020 at 12:12:03PM +0200, Martin Lucina wrote: > On Tuesday, 26.05.2020 at 12:03, Roger Pau Monné wrote: > > On Tue, May 26, 2020 at 11:34:21AM +0200, Roger Pau Monné wrote: > > > On Tue, May 26, 2020 at 10:52:21AM +0200, Martin Lucina wrote: > > > > On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > > > > > On 25/05/2020 17:42, Jürgen Groß wrote: > > > > > > You need to setup virtual addressing and enable 64 bit mode before using > > > > > > 64-bit GDT. > > > > > > > > > > > > See Mini-OS source arch/x86/x86_hvm.S > > > > > > > > > > Or > > > > > https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > > > > > > > > > > But yes - Juergen is correct.  Until you have enabled long mode, lgdt > > > > > will only load the bottom 32 bits of GDTR.base. > > > > > > > > Ah, I missed Jurgen's and your reply here. > > > > > > > > LGDT loading only the bottom 32 bits of GDTR.base shouldn't matter. > > > > Examining gdt_ptr some more: > > > > > > > > (gdb) set architecture i386 > > > > The target architecture is assumed to be i386 > > > > (gdb) x /xh 0x108040 > > > > 0x108040: 0x002f > > > > (gdb) x /xw 0x108042 > > > > 0x108042: 0x00108000 > > > > (gdb) x /6xb 0x108040 > > > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 > > > > (gdb) x /8xb 0x108040 > > > > 0x108040: 0x2f 0x00 0x00 0x80 0x10 0x00 0x00 0x00 > > > > > > Could you also print the GDT entry at 0x10 (ie: 0x108000 + 0x10), just > > > to make sure it contains the right descriptor? > > > > Forgot to ask, but can you also add the output of readelf -lW > > ? > > Elf file type is EXEC (Executable file) > Entry point 0x1001e0 > There are 7 program headers, starting at offset 64 > > Program Headers: > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > [Requesting program interpreter: /nonexistent/solo5/] > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00626c 0x00626c R E 0x1000 > LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed48 RW 0x1000 > NOTE 0x0080ac 0x00000000001070ac 0x00000000001070ac 0x000018 0x000018 R 0x4 > NOTE 0x00f120 0x00000000001070c4 0x00000000001070c4 0x000014 0x000000 R 0x4 > NOTE 0x008088 0x0000000000107088 0x0000000000107088 0x000024 0x000024 R 0x4 > NOTE 0x008000 0x0000000000107000 0x0000000000107000 0x000088 0x000088 R 0x4 > > Section to Segment mapping: > Segment Sections... > 00 .interp > 01 .interp .text .rodata .eh_frame > 02 .note.solo5.manifest .note.solo5.abi .note.solo5.not-openbsd .data .bss > 03 .note.solo5.not-openbsd > 04 .note.solo5.xen > 05 .note.solo5.abi > 06 .note.solo5.manifest > > The PT_INTERP and multiple PT_NOTE headers are that way due to specifics of > how Solo5 ABIs work, but I've verified that the domain builder is > interpreting XEN_ELFNOTE_PHYS32_ENTRY correctly. Right, just wanted to double check that virtaddr == physaddr since you didn't use any offset to get the physical address of symbols, but I guess that if this wasn't correct you won't be even able to execute the first instruction anyway. Thanks, Roger. From mirageos-devel-bounces@lists.xenproject.org Tue May 26 10:58:52 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 10:58:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdXIB-0004ly-CS; Tue, 26 May 2020 10:58:43 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdXIA-0004ls-Oe for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 10:58:42 +0000 X-Inumbo-ID: db778dae-9f3f-11ea-9947-bc764e2007e4 Received: from ppsw-31.csi.cam.ac.uk (unknown [131.111.8.131]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id db778dae-9f3f-11ea-9947-bc764e2007e4; Tue, 26 May 2020 10:58:40 +0000 (UTC) X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://help.uis.cam.ac.uk/email-scanner-virus Received: from 88-109-182-220.dynamic.dsl.as9105.com ([88.109.182.220]:43890 helo=[192.168.1.219]) by ppsw-31.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:465) with esmtpsa (PLAIN:amc96) (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) id 1jdXI7-000qyu-KT (Exim 4.92.3) (return-path ); Tue, 26 May 2020 11:58:39 +0100 Subject: Re: Xen PVH domU start-of-day VCPU state To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, anil@recoil.org, dave@recoil.org, martin@lucina.net References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> From: Andrew Cooper Message-ID: <36363b39-c8c1-82bf-af37-f0d917844bb4@citrix.com> Date: Tue, 26 May 2020 11:58:38 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200526085221.GB5942@nodbug.lucina.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-GB X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On 26/05/2020 09:52, Martin Lucina wrote: > On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: >> On 25/05/2020 17:42, Jürgen Groß wrote: >>> You need to setup virtual addressing and enable 64 bit mode before using >>> 64-bit GDT. >>> >>> See Mini-OS source arch/x86/x86_hvm.S >> Or >> https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD >> >> But yes - Juergen is correct.  Until you have enabled long mode, lgdt >> will only load the bottom 32 bits of GDTR.base. > Ah, I missed Jurgen's and your reply here. So the mailing list is doing something evil and setting: Mail-Followup-To: Andrew Cooper ,     =?iso-8859-1?Q?J=FCrgen_Gro=DF?= ,     xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org,     anil@recoil.org, dave@recoil.org which causes normal replies to cut you out. ~Andrew From mirageos-devel-bounces@lists.xenproject.org Tue May 26 11:55:02 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 11:55:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYAZ-0001hq-IE; Tue, 26 May 2020 11:54:55 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYAX-0001hj-MI for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 11:54:53 +0000 X-Inumbo-ID: b27c9a4a-9f47-11ea-9947-bc764e2007e4 Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id b27c9a4a-9f47-11ea-9947-bc764e2007e4; Tue, 26 May 2020 11:54:48 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 0D2D2122804; Tue, 26 May 2020 13:54:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590494087; bh=QFPqT+wMGFf0hsPuefeEdPUGLpxgodt6U7aKLQIqJtQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=jLgJZgWPv9JNi73tAsS754tnBDH0rRz6wRinU0Se2qyr7pgjUxvs8OLsHRS95HevT hppI8NMQngUX7awqmCeuZu7eIS9OIgiMMh0EAy+svRk/JJYUc7XqKewSuJS87XzXXZ KjMhT+GIrF9yQG9gIGyiNHs5jJLqcwMR/nuWKxZcN56HqUIgCJNimNVYLHsFp9WTi4 d1D+pmC/FD0kDs4zNY5VApzb72hGEImzYhUD8eD1ihgh2f2/53nFfCoyPEl8ebJ7tv XUtacmkeS9hPhjLIc5Cym0+j8IP1ZzRVKm0qvf/Z1edMa+rSw8BVe7NJTINUZ/blV8 w7IBUPKcFkvIg== Received: by nodbug.lucina.net (Postfix, from userid 1000) id DF22D268436E; Tue, 26 May 2020 13:54:46 +0200 (CEST) Date: Tue, 26 May 2020 13:54:46 +0200 From: Martin Lucina To: Andrew Cooper Subject: Mail-Followup-To (was Re: Xen PVH domU start-of-day VCPU state) Message-ID: <20200526115446.GA24386@nodbug.lucina.net> Mail-Followup-To: Andrew Cooper , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <36363b39-c8c1-82bf-af37-f0d917844bb4@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <36363b39-c8c1-82bf-af37-f0d917844bb4@citrix.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tuesday, 26.05.2020 at 11:58, Andrew Cooper wrote: > On 26/05/2020 09:52, Martin Lucina wrote: > > On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > >> On 25/05/2020 17:42, Jürgen Groß wrote: > >>> You need to setup virtual addressing and enable 64 bit mode before using > >>> 64-bit GDT. > >>> > >>> See Mini-OS source arch/x86/x86_hvm.S > >> Or > >> https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > >> > >> But yes - Juergen is correct.  Until you have enabled long mode, lgdt > >> will only load the bottom 32 bits of GDTR.base. > > Ah, I missed Jurgen's and your reply here. > > So the mailing list is doing something evil and setting: > > Mail-Followup-To: Andrew Cooper , >     =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , >     xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, >     anil@recoil.org, dave@recoil.org > > which causes normal replies to cut you out. I _think_ I've fixed this, it was due to ancient Mutt configuration (using xensource.com / xen.org !?) for xen-devel at my end. Dropped the other direct Ccs to lessen the noise, but I have no real way of testing without replying, so here goes. -mato From mirageos-devel-bounces@lists.xenproject.org Tue May 26 11:59:03 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 11:59:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYEU-0001si-VO; Tue, 26 May 2020 11:58:58 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYEU-0001sd-HR for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 11:58:58 +0000 X-Inumbo-ID: 4719d3a2-9f48-11ea-81bc-bc764e2007e4 Received: from ppsw-31.csi.cam.ac.uk (unknown [131.111.8.131]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 4719d3a2-9f48-11ea-81bc-bc764e2007e4; Tue, 26 May 2020 11:58:57 +0000 (UTC) X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://help.uis.cam.ac.uk/email-scanner-virus Received: from 88-109-182-220.dynamic.dsl.as9105.com ([88.109.182.220]:45642 helo=[192.168.1.219]) by ppsw-31.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:465) with esmtpsa (PLAIN:amc96) (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) id 1jdYES-000V8O-L6 (Exim 4.92.3) (return-path ); Tue, 26 May 2020 12:58:56 +0100 Subject: Re: Mail-Followup-To (was Re: Xen PVH domU start-of-day VCPU state) To: martin@lucina.net References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <36363b39-c8c1-82bf-af37-f0d917844bb4@citrix.com> <20200526115446.GA24386@nodbug.lucina.net> From: Andrew Cooper Message-ID: <14a27bd6-7013-2cbb-e202-05f0b32caf9a@citrix.com> Date: Tue, 26 May 2020 12:58:55 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200526115446.GA24386@nodbug.lucina.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-GB X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On 26/05/2020 12:54, Martin Lucina wrote: > On Tuesday, 26.05.2020 at 11:58, Andrew Cooper wrote: >> On 26/05/2020 09:52, Martin Lucina wrote: >>> On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: >>>> On 25/05/2020 17:42, Jürgen Groß wrote: >>>>> You need to setup virtual addressing and enable 64 bit mode before using >>>>> 64-bit GDT. >>>>> >>>>> See Mini-OS source arch/x86/x86_hvm.S >>>> Or >>>> https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD >>>> >>>> But yes - Juergen is correct.  Until you have enabled long mode, lgdt >>>> will only load the bottom 32 bits of GDTR.base. >>> Ah, I missed Jurgen's and your reply here. >> So the mailing list is doing something evil and setting: >> >> Mail-Followup-To: Andrew Cooper , >>     =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , >>     xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, >>     anil@recoil.org, dave@recoil.org >> >> which causes normal replies to cut you out. > I _think_ I've fixed this, it was due to ancient Mutt configuration (using > xensource.com / xen.org !?) for xen-devel at my end. > > Dropped the other direct Ccs to lessen the noise, but I have no real way of > testing without replying, so here goes. Sorry - still no luck.  Had to add you back in manually. ~Andrew From mirageos-devel-bounces@lists.xenproject.org Tue May 26 12:41:33 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 12:41:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYtZ-0006Ii-UB; Tue, 26 May 2020 12:41:25 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYtZ-0006Id-PN for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 12:41:25 +0000 X-Inumbo-ID: 355e72e8-9f4e-11ea-9dbe-bc764e2007e4 Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 355e72e8-9f4e-11ea-9dbe-bc764e2007e4; Tue, 26 May 2020 12:41:24 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 93C19122804; Tue, 26 May 2020 14:41:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590496883; bh=mxUxU5t9xa4Ak/X6Fq1E38zK1T3h4xJHYO9Uxz717QI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=N/ttzeYNqHDj/uLecT8xdoCkI9UMRPxwZHYz66WxZcWhHiY7NXIPaTQAXr/eJRFSV AoSSP3IQLc5TnXHgXgF2Gzgc1UU7Js0PAgoxfptStuy/LMz1ABWXiY+vbtlswr0Adw F577C6xq+ylyGDBmNJuTrKg/J0gJh4Xx6lHdcDNV4vsKgFxYmTp8rqPx5tlWwijepS KOqj0tW84V1ElpBIFhIoZ591GNNArX7IOa6l3to7Z0zS6mArfLBZGMScjzoohQxaaN 0gFr6hUdsZ6RAFjLRxzDC0lSTelvrFuTm9sSVRBfEFBLjFRK4Ke6zftaeR1MmMcZnC i3nm4WMLwux1w== Received: by nodbug.lucina.net (Postfix, from userid 1000) id 7A7D6268436E; Tue, 26 May 2020 14:41:23 +0200 (CEST) Date: Tue, 26 May 2020 14:41:23 +0200 From: Martin Lucina To: Andrew Cooper Subject: Re: Mail-Followup-To (was Re: Xen PVH domU start-of-day VCPU state) Message-ID: <20200526124123.GA25283@nodbug.lucina.net> Mail-Followup-To: Martin Lucina , Andrew Cooper , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <36363b39-c8c1-82bf-af37-f0d917844bb4@citrix.com> <20200526115446.GA24386@nodbug.lucina.net> <14a27bd6-7013-2cbb-e202-05f0b32caf9a@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <14a27bd6-7013-2cbb-e202-05f0b32caf9a@citrix.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tuesday, 26.05.2020 at 12:58, Andrew Cooper wrote: > On 26/05/2020 12:54, Martin Lucina wrote: > > On Tuesday, 26.05.2020 at 11:58, Andrew Cooper wrote: > >> On 26/05/2020 09:52, Martin Lucina wrote: > >>> On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > >>>> On 25/05/2020 17:42, Jürgen Groß wrote: > >>>>> You need to setup virtual addressing and enable 64 bit mode before using > >>>>> 64-bit GDT. > >>>>> > >>>>> See Mini-OS source arch/x86/x86_hvm.S > >>>> Or > >>>> https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > >>>> > >>>> But yes - Juergen is correct.  Until you have enabled long mode, lgdt > >>>> will only load the bottom 32 bits of GDTR.base. > >>> Ah, I missed Jurgen's and your reply here. > >> So the mailing list is doing something evil and setting: > >> > >> Mail-Followup-To: Andrew Cooper , > >>     =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , > >>     xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, > >>     anil@recoil.org, dave@recoil.org > >> > >> which causes normal replies to cut you out. > > I _think_ I've fixed this, it was due to ancient Mutt configuration (using > > xensource.com / xen.org !?) for xen-devel at my end. > > > > Dropped the other direct Ccs to lessen the noise, but I have no real way of > > testing without replying, so here goes. > > Sorry - still no luck.  Had to add you back in manually. How about now? -mato From mirageos-devel-bounces@lists.xenproject.org Tue May 26 12:42:27 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 12:42:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYuZ-0006Of-JH; Tue, 26 May 2020 12:42:27 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYuY-0006OU-BT for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 12:42:26 +0000 X-Inumbo-ID: 55ca768a-9f4e-11ea-a636-12813bfff9fa Received: from ppsw-31.csi.cam.ac.uk (unknown [131.111.8.131]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 55ca768a-9f4e-11ea-a636-12813bfff9fa; Tue, 26 May 2020 12:42:18 +0000 (UTC) X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://help.uis.cam.ac.uk/email-scanner-virus Received: from 88-109-182-220.dynamic.dsl.as9105.com ([88.109.182.220]:47048 helo=[192.168.1.219]) by ppsw-31.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.157]:465) with esmtpsa (PLAIN:amc96) (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) id 1jdYuQ-000yfY-Jg (Exim 4.92.3) (return-path ); Tue, 26 May 2020 13:42:18 +0100 Subject: Re: Mail-Followup-To (was Re: Xen PVH domU start-of-day VCPU state) To: Martin Lucina , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <36363b39-c8c1-82bf-af37-f0d917844bb4@citrix.com> <20200526115446.GA24386@nodbug.lucina.net> <14a27bd6-7013-2cbb-e202-05f0b32caf9a@citrix.com> <20200526124123.GA25283@nodbug.lucina.net> From: Andrew Cooper Message-ID: Date: Tue, 26 May 2020 13:42:17 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200526124123.GA25283@nodbug.lucina.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-GB X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On 26/05/2020 13:41, Martin Lucina wrote: > On Tuesday, 26.05.2020 at 12:58, Andrew Cooper wrote: >> On 26/05/2020 12:54, Martin Lucina wrote: >>> On Tuesday, 26.05.2020 at 11:58, Andrew Cooper wrote: >>>> On 26/05/2020 09:52, Martin Lucina wrote: >>>>> On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: >>>>>> On 25/05/2020 17:42, Jürgen Groß wrote: >>>>>>> You need to setup virtual addressing and enable 64 bit mode before using >>>>>>> 64-bit GDT. >>>>>>> >>>>>>> See Mini-OS source arch/x86/x86_hvm.S >>>>>> Or >>>>>> https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD >>>>>> >>>>>> But yes - Juergen is correct.  Until you have enabled long mode, lgdt >>>>>> will only load the bottom 32 bits of GDTR.base. >>>>> Ah, I missed Jurgen's and your reply here. >>>> So the mailing list is doing something evil and setting: >>>> >>>> Mail-Followup-To: Andrew Cooper , >>>>     =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , >>>>     xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, >>>>     anil@recoil.org, dave@recoil.org >>>> >>>> which causes normal replies to cut you out. >>> I _think_ I've fixed this, it was due to ancient Mutt configuration (using >>> xensource.com / xen.org !?) for xen-devel at my end. >>> >>> Dropped the other direct Ccs to lessen the noise, but I have no real way of >>> testing without replying, so here goes. >> Sorry - still no luck.  Had to add you back in manually. > How about now? That works. ~Andrew From mirageos-devel-bounces@lists.xenproject.org Tue May 26 12:44:28 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 12:44:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYwV-0006cm-2e; Tue, 26 May 2020 12:44:27 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdYwU-0006cY-1Q for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 12:44:26 +0000 X-Inumbo-ID: 9dacff18-9f4e-11ea-a636-12813bfff9fa Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 9dacff18-9f4e-11ea-a636-12813bfff9fa; Tue, 26 May 2020 12:44:19 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id B40E5122804; Tue, 26 May 2020 14:44:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590497058; bh=DCk1HFmCWgmAdqJZvcRKU/bEbj5+p//dr2VZiaADZ60=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=N1gUJLExhUISbdI+NDunp+bQwwnT9Cq+5rDSIAmeSQVlGRi3crYWxNPl8fUJA10qZ gTAxHtBti9DrgLOzYQ0lXII72lJV+us9sxX10Mzr3D+ywFax69ZH//XpF2PP4J37J6 7OnPy7Nvu5dJKbTqJ74QeEgg3/BKbweMSGGR/fNocOE710B20Jkmv5031JQf6fczFF M1nhuEB2IPCaEWuf1VwDdt4S+w5RSwoksOTzPV477EFDXAKcDxKzS1t0D8AOWNiUu6 lkUt9MysCtp+Go/vdTExZ8+k3PBNbjweY20uNCyjfO3PYBXgFQ2PZ893Ru2s7wKgXy YmsDo7J8NgbMA== Received: by nodbug.lucina.net (Postfix, from userid 1000) id 984BC268436E; Tue, 26 May 2020 14:44:18 +0200 (CEST) Date: Tue, 26 May 2020 14:44:18 +0200 From: Martin Lucina To: Andrew Cooper Subject: Re: Mail-Followup-To (was Re: Xen PVH domU start-of-day VCPU state) Message-ID: <20200526124418.GB25283@nodbug.lucina.net> Mail-Followup-To: Martin Lucina , Andrew Cooper , xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <36363b39-c8c1-82bf-af37-f0d917844bb4@citrix.com> <20200526115446.GA24386@nodbug.lucina.net> <14a27bd6-7013-2cbb-e202-05f0b32caf9a@citrix.com> <20200526124123.GA25283@nodbug.lucina.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tuesday, 26.05.2020 at 13:42, Andrew Cooper wrote: > On 26/05/2020 13:41, Martin Lucina wrote: > > On Tuesday, 26.05.2020 at 12:58, Andrew Cooper wrote: > >> On 26/05/2020 12:54, Martin Lucina wrote: > >>> On Tuesday, 26.05.2020 at 11:58, Andrew Cooper wrote: > >>>> On 26/05/2020 09:52, Martin Lucina wrote: > >>>>> On Monday, 25.05.2020 at 17:59, Andrew Cooper wrote: > >>>>>> On 25/05/2020 17:42, Jürgen Groß wrote: > >>>>>>> You need to setup virtual addressing and enable 64 bit mode before using > >>>>>>> 64-bit GDT. > >>>>>>> > >>>>>>> See Mini-OS source arch/x86/x86_hvm.S > >>>>>> Or > >>>>>> https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen-test-framework.git;a=blob;f=arch/x86/hvm/head.S;h=f7dc72b58ab9ec68538f0087969ab6f72d181d80;hb=HEAD > >>>>>> > >>>>>> But yes - Juergen is correct.  Until you have enabled long mode, lgdt > >>>>>> will only load the bottom 32 bits of GDTR.base. > >>>>> Ah, I missed Jurgen's and your reply here. > >>>> So the mailing list is doing something evil and setting: > >>>> > >>>> Mail-Followup-To: Andrew Cooper , > >>>>     =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , > >>>>     xen-devel@lists.xenproject.org, mirageos-devel@lists.xenproject.org, > >>>>     anil@recoil.org, dave@recoil.org > >>>> > >>>> which causes normal replies to cut you out. > >>> I _think_ I've fixed this, it was due to ancient Mutt configuration (using > >>> xensource.com / xen.org !?) for xen-devel at my end. > >>> > >>> Dropped the other direct Ccs to lessen the noise, but I have no real way of > >>> testing without replying, so here goes. > >> Sorry - still no luck.  Had to add you back in manually. > > How about now? > > That works. Ok, TIL that I've been doing "subscribe" vs. "lists" in Mutt wrong. Here's a good explanation of how it's intended to work: https://lists.debian.org/debian-user/2003/05/msg05016.html TL;DR if you want a Mail-Followup-To, use "lists", otherwise use "subscribe". -mato From mirageos-devel-bounces@lists.xenproject.org Tue May 26 15:42:43 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 15:42:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdbil-00070H-W1; Tue, 26 May 2020 15:42:27 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdbik-00070C-Bj for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 15:42:26 +0000 X-Inumbo-ID: 7f183838-9f67-11ea-a664-12813bfff9fa Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 7f183838-9f67-11ea-a664-12813bfff9fa; Tue, 26 May 2020 15:42:25 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id C74C1122804; Tue, 26 May 2020 17:42:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590507744; bh=6B1mwmaG5KlpcOX1ZvW8LOn+uZ4wZogEf3CjfrNwJ3M=; h=Date:From:To:Subject:References:In-Reply-To:From; b=i7drQIPpJjc9AbVLB4iBbP2qe0AvwzA2hleRl0LUhof8taNt21yw+TfJljfrxfhr4 R0NzttbfyipltZIkPQCGSHiGq+zSGCsSHEcJ9rAQq4YAZIt5ZyLzJi0n0GdtCUlp+n AbAAx5R+BhiPqzpfe0QgX5epJx2CKatbmpnnxCqFRaLeIusGZzWCEFPUhtH/k56EVW LKAJvH9CzOF5tkCCDuV488q3Rjd2mNfs2GU2g9QUz7Px5QNLDBTsfyI4f8C/u5ccS9 eHVJIE7rcrQncNRrHdwCIHxI/pZBNoOFTRaIyA0EBBykaP50ozFfvVyB0CbX3OCsc7 cTA4JHcJF61Cg== Received: by nodbug.lucina.net (Postfix, from userid 1000) id 7DC03268436E; Tue, 26 May 2020 17:42:24 +0200 (CEST) Date: Tue, 26 May 2020 17:42:24 +0200 From: Martin Lucina To: Roger Pau =?iso-8859-1?Q?Monn=E9?= , =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526154224.GC25283@nodbug.lucina.net> Mail-Followup-To: Martin Lucina , Roger Pau =?iso-8859-1?Q?Monn=E9?= , =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> <20200526101203.GE5942@nodbug.lucina.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526101203.GE5942@nodbug.lucina.net> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" Oh! I think I've found a solution, even though I don't entirely understand the problem/root cause: On Tuesday, 26.05.2020 at 12:12, Martin Lucina wrote: > > On Tue, May 26, 2020 at 11:34:21AM +0200, Roger Pau Monné wrote: > > Forgot to ask, but can you also add the output of readelf -lW > > ? > > Elf file type is EXEC (Executable file) > Entry point 0x1001e0 > There are 7 program headers, starting at offset 64 > > Program Headers: > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > [Requesting program interpreter: /nonexistent/solo5/] > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00626c 0x00626c R E 0x1000 > LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed48 RW 0x1000 > NOTE 0x0080ac 0x00000000001070ac 0x00000000001070ac 0x000018 0x000018 R 0x4 > NOTE 0x00f120 0x00000000001070c4 0x00000000001070c4 0x000014 0x000000 R 0x4 ^^^^^^^^ I should have picked up on the above, but thought it wasn't relevant. > NOTE 0x008088 0x0000000000107088 0x0000000000107088 0x000024 0x000024 R 0x4 > NOTE 0x008000 0x0000000000107000 0x0000000000107000 0x000088 0x000088 R 0x4 > > Section to Segment mapping: > Segment Sections... > 00 .interp > 01 .interp .text .rodata .eh_frame > 02 .note.solo5.manifest .note.solo5.abi .note.solo5.not-openbsd .data .bss And also the missing .note.solo5.xen above. > 03 .note.solo5.not-openbsd > 04 .note.solo5.xen > 05 .note.solo5.abi > 06 .note.solo5.manifest Turns out that the .note.solo5.xen section as defined in boot.S was not marked allocatable, and that was doing that was confusing our linker script[1] (?). If I make this simple change: --- a/bindings/xen/boot.S +++ b/bindings/xen/boot.S @@ -32,7 +32,7 @@ #define ENTRY(x) .text; .globl x; .type x,%function; x: #define END(x) .size x, . - x -.section .note.solo5.xen +.section .note.solo5.xen, "a", @note .align 4 .long 4 then I get the expected output from readelf -lW, and I can get as far as the C _start() with no issues! FWIW, here's the diff of readelf -lW before/after: --- before 2020-05-26 17:36:46.117885855 +0200 +++ after 2020-05-26 17:38:07.090508322 +0200 @@ -8,9 +8,9 @@ INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 [Requesting program interpreter: /nonexistent/solo5/] LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00615c 0x00615c R E 0x1000 - LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed28 RW 0x1000 + LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x006120 0x00dd28 RW 0x1000 NOTE 0x0080ac 0x00000000001070ac 0x00000000001070ac 0x000018 0x000018 R 0x4 - NOTE 0x00f120 0x00000000001070c4 0x00000000001070c4 0x000014 0x000000 R 0x4 + NOTE 0x0080c4 0x00000000001070c4 0x00000000001070c4 0x000014 0x000014 R 0x4 NOTE 0x008088 0x0000000000107088 0x0000000000107088 0x000024 0x000024 R 0x4 NOTE 0x008000 0x0000000000107000 0x0000000000107000 0x000088 0x000088 R 0x4 @@ -18,7 +18,7 @@ Segment Sections... 00 .interp 01 .interp .text .rodata .eh_frame - 02 .note.solo5.manifest .note.solo5.abi .note.solo5.not-openbsd .data .bss + 02 .note.solo5.manifest .note.solo5.abi .note.solo5.not-openbsd .note.solo5.xen .data .bss 03 .note.solo5.not-openbsd 04 .note.solo5.xen 05 .note.solo5.abi -mato [1] https://github.com/mato/solo5/blob/xen/bindings/xen/solo5_xen.lds From mirageos-devel-bounces@lists.xenproject.org Tue May 26 16:30:41 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Tue, 26 May 2020 16:30:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdcTM-0003aC-F2; Tue, 26 May 2020 16:30:36 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdcTL-0003Zv-44 for mirageos-devel@lists.xenproject.org; Tue, 26 May 2020 16:30:35 +0000 X-Inumbo-ID: 3596f44a-9f6e-11ea-a676-12813bfff9fa Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 3596f44a-9f6e-11ea-a676-12813bfff9fa; Tue, 26 May 2020 16:30:28 +0000 (UTC) Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: C66bPiiyt6YKoDqn5r/JZkaVknDvTkdz8iCAQJLH4uHbQspyC92LK43ixaC7HZSDdvalXk9Kij 1QdMNsQHuHF7pXkAYXUZ2pw/ts9V7Tt1sQxU5Afm6l2DzNsepCIotmfFznhD+rlo79bVI7gtF7 cbxKBQQFF2S7Bm/4QWLc+6fQFfpvIgKv1IKkLyQwEMMZbuz1bxcCI2Dqbh6F0vMsLB6xTpB4Wz jE3hf3im7nQQFP3ZCpUfGx7BzTywSkFnS0+Jf4mBzBjVWztu4PnYZzchro9xMScqESKb15tNmO qkk= X-SBRS: 2.7 X-MesageID: 19210299 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.73,437,1583211600"; d="scan'208";a="19210299" Date: Tue, 26 May 2020 18:30:21 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Martin Lucina Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200526163021.GE38408@Air-de-Roger> References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> <20200526101203.GE5942@nodbug.lucina.net> <20200526154224.GC25283@nodbug.lucina.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526154224.GC25283@nodbug.lucina.net> X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tue, May 26, 2020 at 05:42:24PM +0200, Martin Lucina wrote: > Oh! I think I've found a solution, even though I don't entirely understand the > problem/root cause: > > On Tuesday, 26.05.2020 at 12:12, Martin Lucina wrote: > > > On Tue, May 26, 2020 at 11:34:21AM +0200, Roger Pau Monné wrote: > > > Forgot to ask, but can you also add the output of readelf -lW > > > ? > > > > Elf file type is EXEC (Executable file) > > Entry point 0x1001e0 > > There are 7 program headers, starting at offset 64 > > > > Program Headers: > > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align > > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > > [Requesting program interpreter: /nonexistent/solo5/] > > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00626c 0x00626c R E 0x1000 > > LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed48 RW 0x1000 > > NOTE 0x0080ac 0x00000000001070ac 0x00000000001070ac 0x000018 0x000018 R 0x4 > > NOTE 0x00f120 0x00000000001070c4 0x00000000001070c4 0x000014 0x000000 R 0x4 > ^^^^^^^^ > > I should have picked up on the above, but thought it wasn't relevant. > > > NOTE 0x008088 0x0000000000107088 0x0000000000107088 0x000024 0x000024 R 0x4 > > NOTE 0x008000 0x0000000000107000 0x0000000000107000 0x000088 0x000088 R 0x4 > > > > Section to Segment mapping: > > Segment Sections... > > 00 .interp > > 01 .interp .text .rodata .eh_frame > > 02 .note.solo5.manifest .note.solo5.abi .note.solo5.not-openbsd .data .bss > > And also the missing .note.solo5.xen above. > > > 03 .note.solo5.not-openbsd > > 04 .note.solo5.xen > > 05 .note.solo5.abi > > 06 .note.solo5.manifest > > Turns out that the .note.solo5.xen section as defined in boot.S was not > marked allocatable, and that was doing that was confusing our > linker script[1] (?). Hm, I would have said there was no need to load notes into memory, and hence using a MemSize of 0 would be fine. Maybe libelf loader was somehow getting confused and not loading the image properly? Can you paste the output of `xl -vvv create ...` when using the broken image? > > If I make this simple change: > > --- a/bindings/xen/boot.S > +++ b/bindings/xen/boot.S > @@ -32,7 +32,7 @@ > #define ENTRY(x) .text; .globl x; .type x,%function; x: > #define END(x) .size x, . - x > > -.section .note.solo5.xen > +.section .note.solo5.xen, "a", @note > > .align 4 > .long 4 > > then I get the expected output from readelf -lW, and I can get as far as > the C _start() with no issues! > > FWIW, here's the diff of readelf -lW before/after: > > --- before 2020-05-26 17:36:46.117885855 +0200 > +++ after 2020-05-26 17:38:07.090508322 +0200 > @@ -8,9 +8,9 @@ > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > [Requesting program interpreter: /nonexistent/solo5/] > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00615c 0x00615c R E 0x1000 > - LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed28 RW 0x1000 > + LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x006120 0x00dd28 RW 0x1000 This seems suspicious, there's a change of the size of the LOAD section, but your change to the note type should not affect the LOAD section? Hm, maybe it does because the .note.solo5.xen was considered writable by default? Roger. From mirageos-devel-bounces@lists.xenproject.org Wed May 27 08:00:30 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 27 May 2020 08:00:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdqz4-0001bb-NM; Wed, 27 May 2020 08:00:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdqz3-0001WT-Ea for mirageos-devel@lists.xenproject.org; Wed, 27 May 2020 08:00:17 +0000 X-Inumbo-ID: 1573d188-9ff0-11ea-a70a-12813bfff9fa Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 1573d188-9ff0-11ea-a70a-12813bfff9fa; Wed, 27 May 2020 08:00:09 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 95158122804; Wed, 27 May 2020 10:00:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590566408; bh=6rQ6u7P+sESM3XhmLHwdlYYRer/3126WoGV8Y9T1yC0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=aPvdAznh47EwVX3q6zIkaFezmhg2LF7L49VQRqpfaZRFhVd62/GF0m49g7ijEso9W BFo1ITbwLFx/53S/kuT9n+art4id2s4u7SdwfbySqQx4zOfdypQH5d+ExbNvCyyZxS Ti2mNZk7DVSAbPqwBGSgv6onmBKpsY1E5J8sfbPVjwq9bpcS2B7WTU4FQLEkG8fS5K aGew6Ork10UXDzI5jC8JL3MXVYN+0//rHT6h+lEnZ/HMcljjhrkBCSBc0eouOoKSIb /fyuEpLSMKYl3YcM1NMS/nr2gqlmZXrRN69EP1kn8nx+fJKl4vHZmfWHnLMRMPQbrJ ey7USF7Sy+buQ== Received: by nodbug.lucina.net (Postfix, from userid 1000) id 6F852265E722; Wed, 27 May 2020 10:00:08 +0200 (CEST) Date: Wed, 27 May 2020 10:00:08 +0200 From: Martin Lucina To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200527080008.GC4788@nodbug.lucina.net> Mail-Followup-To: Martin Lucina , Roger Pau =?iso-8859-1?Q?Monn=E9?= , =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> <20200526101203.GE5942@nodbug.lucina.net> <20200526154224.GC25283@nodbug.lucina.net> <20200526163021.GE38408@Air-de-Roger> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200526163021.GE38408@Air-de-Roger> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Tuesday, 26.05.2020 at 18:30, Roger Pau Monné wrote: > > Turns out that the .note.solo5.xen section as defined in boot.S was not > > marked allocatable, and that was doing that was confusing our > > linker script[1] (?). > > Hm, I would have said there was no need to load notes into memory, and > hence using a MemSize of 0 would be fine. > > Maybe libelf loader was somehow getting confused and not loading the > image properly? > > Can you paste the output of `xl -vvv create ...` when using the broken > image? Here you go: Parsing config from ./test_hello.xl libxl: debug: libxl_create.c:1671:do_domain_create: Domain 0:ao 0x5593c42e7e30: create: how=(nil) callback=(nil) poller=0x5593c42e7670 libxl: debug: libxl_create.c:1007:initiate_domain_create: Domain 2:running bootloader libxl: debug: libxl_bootloader.c:335:libxl__bootloader_run: Domain 2:no bootloader configured, using user supplied kernel libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x5593c42e9590: deregister unregistered libxl: debug: libxl_sched.c:82:libxl__set_vcpuaffinity: Domain 2:New soft affinity for vcpu 0 has unreachable cpus domainbuilder: detail: xc_dom_allocate: cmdline="", features="" domainbuilder: detail: xc_dom_kernel_file: filename="test_hello.xen" domainbuilder: detail: xc_dom_malloc_filemap : 191 kB domainbuilder: detail: xc_dom_boot_xen_init: ver 4.11, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying HVM-generic loader ... domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... domainbuilder: detail: xc_dom_probe_bzimage_kernel: kernel is not a bzImage domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK xc: detail: ELF: phdr: paddr=0x100000 memsz=0x6264 xc: detail: ELF: phdr: paddr=0x107000 memsz=0xed48 xc: detail: ELF: memory: 0x100000 -> 0x115d48 xc: detail: ELF: note: PHYS32_ENTRY = 0x100020 xc: detail: ELF: Found PVH image xc: detail: ELF: VIRT_BASE unset, using 0 xc: detail: ELF_PADDR_OFFSET unset, using 0 xc: detail: ELF: addresses: xc: detail: virt_base = 0x0 xc: detail: elf_paddr_offset = 0x0 xc: detail: virt_offset = 0x0 xc: detail: virt_kstart = 0x100000 xc: detail: virt_kend = 0x115d48 xc: detail: virt_entry = 0x1001e0 xc: detail: p2m_base = 0xffffffffffffffff domainbuilder: detail: xc_dom_parse_elf_kernel: hvm-3.0-x86_32: 0x100000 -> 0x115d48 domainbuilder: detail: xc_dom_mem_init: mem 256 MB, pages 0x10000 pages, 4k each domainbuilder: detail: xc_dom_mem_init: 0x10000 pages domainbuilder: detail: xc_dom_boot_mem_init: called domainbuilder: detail: range: start=0x0 end=0x10000400 domainbuilder: detail: xc_dom_malloc : 512 kB xc: detail: PHYSICAL MEMORY ALLOCATION: xc: detail: 4KB PAGES: 0x0000000000000c00 xc: detail: 2MB PAGES: 0x000000000000007a xc: detail: 1GB PAGES: 0x0000000000000000 domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x100+0x16 at 0x7f5609445000 domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x100000 -> 0x116000 (pfn 0x100 + 0x16 pages) xc: detail: ELF: phdr 1 at 0x7f5609445000 -> 0x7f560944b264 xc: detail: ELF: phdr 2 at 0x7f560944c000 -> 0x7f5609453120 domainbuilder: detail: xc_dom_load_acpi: 64 bytes at address fc008000 domainbuilder: detail: xc_dom_load_acpi: 4096 bytes at address fc000000 domainbuilder: detail: xc_dom_load_acpi: 28672 bytes at address fc001000 domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x116+0x1 at 0x7f5609ace000 domainbuilder: detail: xc_dom_alloc_segment: HVM start info : 0x116000 -> 0x117000 (pfn 0x116 + 0x1 pages) domainbuilder: detail: alloc_pgtables_hvm: doing nothing domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0x117000 domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x0 domainbuilder: detail: xc_dom_boot_image: called domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32 <= matches domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64 domainbuilder: detail: domain builder memory footprint domainbuilder: detail: allocated domainbuilder: detail: malloc : 515 kB domainbuilder: detail: anon mmap : 0 bytes domainbuilder: detail: mapped domainbuilder: detail: file mmap : 191 kB domainbuilder: detail: domU mmap : 92 kB domainbuilder: detail: vcpu_hvm: called domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0xff000 domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0xff001 domainbuilder: detail: xc_dom_release: called libxl: debug: libxl_event.c:2194:libxl__ao_progress_report: ao 0x5593c42e7e30: progress report: callback queued aop=0x5593c42fea10 libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x5593c42e7e30: complete, rc=0 libxl: debug: libxl_event.c:1404:egc_run_callbacks: ao 0x5593c42e7e30: progress report: callback aop=0x5593c42fea10 libxl: debug: libxl_create.c:1708:do_domain_create: Domain 0:ao 0x5593c42e7e30: inprogress: poller=0x5593c42e7670, flags=ic libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x5593c42e7e30: destroy xencall:buffer: debug: total allocations:233 total releases:233 xencall:buffer: debug: current allocations:0 maximum allocations:3 xencall:buffer: debug: cache current size:3 xencall:buffer: debug: cache hits:215 misses:3 toobig:15 xencall:buffer: debug: total allocations:0 total releases:0 xencall:buffer: debug: current allocations:0 maximum allocations:0 xencall:buffer: debug: cache current size:0 xencall:buffer: debug: cache hits:0 misses:0 toobig:0 > > > > > If I make this simple change: > > > > --- a/bindings/xen/boot.S > > +++ b/bindings/xen/boot.S > > @@ -32,7 +32,7 @@ > > #define ENTRY(x) .text; .globl x; .type x,%function; x: > > #define END(x) .size x, . - x > > > > -.section .note.solo5.xen > > +.section .note.solo5.xen, "a", @note > > > > .align 4 > > .long 4 > > > > then I get the expected output from readelf -lW, and I can get as far as > > the C _start() with no issues! > > > > FWIW, here's the diff of readelf -lW before/after: > > > > --- before 2020-05-26 17:36:46.117885855 +0200 > > +++ after 2020-05-26 17:38:07.090508322 +0200 > > @@ -8,9 +8,9 @@ > > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > > [Requesting program interpreter: /nonexistent/solo5/] > > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00615c 0x00615c R E 0x1000 > > - LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed28 RW 0x1000 > > + LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x006120 0x00dd28 RW 0x1000 > > This seems suspicious, there's a change of the size of the LOAD > section, but your change to the note type should not affect the LOAD > section? Indeed. > > Hm, maybe it does because the .note.solo5.xen was considered writable > by default? I don't think so. From the broken image: [ 8] .note.solo5.xen NOTE 00000000001070c4 0000f120 0000000000000014 0000000000000000 0 0 4 >From the good image: [ 8] .note.solo5.xen NOTE 00000000001070c4 000080c4 0000000000000014 0000000000000000 A 0 0 4 -mato From mirageos-devel-bounces@lists.xenproject.org Wed May 27 14:57:22 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 27 May 2020 14:57:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdxUV-0004wA-JE; Wed, 27 May 2020 14:57:11 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdxUU-0004w3-HX for mirageos-devel@lists.xenproject.org; Wed, 27 May 2020 14:57:10 +0000 X-Inumbo-ID: 535092fe-a02a-11ea-a75b-12813bfff9fa Received: from smtp.lucina.net (unknown [62.176.169.44]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 535092fe-a02a-11ea-a75b-12813bfff9fa; Wed, 27 May 2020 14:57:04 +0000 (UTC) Received: from nodbug.lucina.net (78-141-76-187.dynamic.orange.sk [78.141.76.187]) by smtp.lucina.net (Postfix) with ESMTPSA id 27A01122804; Wed, 27 May 2020 16:57:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lucina.net; s=dkim-201811; t=1590591423; bh=gCASpuwhr1jiW6oKsT1tI6Zwmv/ACpqwRxqwguDYUJ0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=qiW8NyhCrP/mIBzkTgusiSWV8Ui4+vu8fNB/+7vnRonbmWNdzmFd583CzU/dgmM3L qrXqDJFvUKn/U6fLaN71Xhg5TtiLst5ck5UZyhNDAx/2wUhYNStdyDtV8Wq2tWPgSr WnAQU8xn0ae7RXXIYiPGNBMqBfQ0uYu5bmQCKeIBobVQyYMqr8PC6ue5Wsc+W3rpTe EKNSvK8b0auZMXKO8KnI/t6JT1IgsjN1J8RN1c4ZS97BSKLvNxNG/UONrObTIwQVVm UpPVoZSTsK4G+ElChvnFYhqaHto/X1FYhts0XIZOXDRwKyQJEVb298+xKhkjnq26Sm B5H3zmLQ9lEaA== Received: by nodbug.lucina.net (Postfix, from userid 1000) id 05BB4265E722; Wed, 27 May 2020 16:57:03 +0200 (CEST) Date: Wed, 27 May 2020 16:57:02 +0200 From: Martin Lucina To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200527145702.GE4788@nodbug.lucina.net> Mail-Followup-To: Martin Lucina , Roger Pau =?iso-8859-1?Q?Monn=E9?= , =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org References: <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> <20200526101203.GE5942@nodbug.lucina.net> <20200526154224.GC25283@nodbug.lucina.net> <20200526163021.GE38408@Air-de-Roger> <20200527080008.GC4788@nodbug.lucina.net> <20200527143644.GA1195@Air-de-Roger> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200527143644.GA1195@Air-de-Roger> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?iso-8859-1?Q?J=FCrgen_Gro=DF?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Wednesday, 27.05.2020 at 16:41, Roger Pau Monné wrote: > > > > If I make this simple change: > > > > > > > > --- a/bindings/xen/boot.S > > > > +++ b/bindings/xen/boot.S > > > > @@ -32,7 +32,7 @@ > > > > #define ENTRY(x) .text; .globl x; .type x,%function; x: > > > > #define END(x) .size x, . - x > > > > > > > > -.section .note.solo5.xen > > > > +.section .note.solo5.xen, "a", @note > > > > > > > > .align 4 > > > > .long 4 > > > > > > > > then I get the expected output from readelf -lW, and I can get as far as > > > > the C _start() with no issues! > > > > > > > > FWIW, here's the diff of readelf -lW before/after: > > > > > > > > --- before 2020-05-26 17:36:46.117885855 +0200 > > > > +++ after 2020-05-26 17:38:07.090508322 +0200 > > > > @@ -8,9 +8,9 @@ > > > > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > > > > [Requesting program interpreter: /nonexistent/solo5/] > > > > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00615c 0x00615c R E 0x1000 > > > > - LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed28 RW 0x1000 > > > > + LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x006120 0x00dd28 RW 0x1000 > > > > > > This seems suspicious, there's a change of the size of the LOAD > > > section, but your change to the note type should not affect the LOAD > > > section? > > > > Indeed. > > You could try to disassemble the text sections with objdump -d (or -D > for all sections) and see if there's a difference between both > versions, but having solved the issue maybe you just want to move > on. I have moved on, making good progress: domainbuilder: detail: xc_dom_release: called Hello, world! Solo5: Xen hvm_start_info @0x0000000000119000 Solo5: magic=0x336ec578 version=1 Solo5: cmdline_paddr=0x0 Solo5: memmap_paddr=0x119878 entries=5 Solo5: memmap[0] = { 0x0, 0x10000400, 1 } Solo5: mem_size=0x10000000 | ___| __| _ \ | _ \ __ \ \__ \ ( | | ( | ) | ____/\___/ _|\___/____/ Solo5: Bindings version v0.6.5-4-g57724f8-dirty Solo5: Memory map: 256 MB addressable: Solo5: reserved @ (0x0 - 0xfffff) Solo5: text @ (0x100000 - 0x104fff) Solo5: rodata @ (0x105000 - 0x106fff) Solo5: data @ (0x107000 - 0x118fff) Solo5: heap >= 0x119000 < stack < 0x10000000 Solo5: Clock source: KVM paravirtualized clock Solo5: trap: type=#GP ec=0x0 rip=0x103a96 rsp=0xfffff90 rflags=0x2 Solo5: ABORT: cpu_x86_64.c:181: Fatal trap Solo5: Halted (The #GP is due to the timekeeping code not yet ported to Xen). Random question: With memory="256" in the xl config, why is the size of the first XEN_HVM_MEMMAP_TYPE_RAM memmap entry not a multiple of PAGE_SIZE? I had to align it down, since we put the stack at the top of RAM. 0x10000400 seems... odd. Thanks all for your help so far, I'm sure I'll run into some more details that will need clarifying. Enough for today, now going for a walk in the woods :-) -mato From mirageos-devel-bounces@lists.xenproject.org Wed May 27 15:37:20 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 27 May 2020 15:37:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdy7H-0000ba-FY; Wed, 27 May 2020 15:37:15 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdxFL-0003kV-GF for mirageos-devel@lists.xenproject.org; Wed, 27 May 2020 14:41:31 +0000 X-Inumbo-ID: 26d14dce-a028-11ea-9947-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 26d14dce-a028-11ea-9947-bc764e2007e4; Wed, 27 May 2020 14:41:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=In-Reply-To:Content-Transfer-Encoding:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=62mMkHYb9wXQ+M+kXG8GvEUvOczD2Ydx/xkpqyMXgIE=; b=t9bDoPdJpvG7EfgeWOOjc4YDuD TBgIUh1I5IbS15gaDzI2RXn0xmXxhozo9ym2ZA6IkwT0GQBjsBGAm1uN6Xqvf71x4iNzeepQ4JYJy OQGbZZ0SByjGlTKnGtBCpH+gm++7SYvHkIKMBinx3+NvPjanqVHOw5RGVOPGA92abX4s=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdxFE-0005N6-Qq; Wed, 27 May 2020 14:41:24 +0000 Received: from [93.176.191.173] (helo=localhost) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jdxFE-00010X-G7; Wed, 27 May 2020 14:41:24 +0000 Date: Wed, 27 May 2020 16:41:10 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Martin Lucina Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200527143644.GA1195@Air-de-Roger> References: <20200525160401.GA3091@nodbug.lucina.net> <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> <20200526101203.GE5942@nodbug.lucina.net> <20200526154224.GC25283@nodbug.lucina.net> <20200526163021.GE38408@Air-de-Roger> <20200527080008.GC4788@nodbug.lucina.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200527080008.GC4788@nodbug.lucina.net> X-Mailman-Approved-At: Wed, 27 May 2020 15:37:14 +0000 X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Wed, May 27, 2020 at 10:00:08AM +0200, Martin Lucina wrote: > On Tuesday, 26.05.2020 at 18:30, Roger Pau Monné wrote: > > > Turns out that the .note.solo5.xen section as defined in boot.S was not > > > marked allocatable, and that was doing that was confusing our > > > linker script[1] (?). > > > > Hm, I would have said there was no need to load notes into memory, and > > hence using a MemSize of 0 would be fine. > > > > Maybe libelf loader was somehow getting confused and not loading the > > image properly? > > > > Can you paste the output of `xl -vvv create ...` when using the broken > > image? > > Here you go: > > Parsing config from ./test_hello.xl > libxl: debug: libxl_create.c:1671:do_domain_create: Domain 0:ao 0x5593c42e7e30: create: how=(nil) callback=(nil) poller=0x5593c42e7670 > libxl: debug: libxl_create.c:1007:initiate_domain_create: Domain 2:running bootloader > libxl: debug: libxl_bootloader.c:335:libxl__bootloader_run: Domain 2:no bootloader configured, using user supplied kernel > libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x5593c42e9590: deregister unregistered > libxl: debug: libxl_sched.c:82:libxl__set_vcpuaffinity: Domain 2:New soft affinity for vcpu 0 has unreachable cpus > domainbuilder: detail: xc_dom_allocate: cmdline="", features="" > domainbuilder: detail: xc_dom_kernel_file: filename="test_hello.xen" > domainbuilder: detail: xc_dom_malloc_filemap : 191 kB > domainbuilder: detail: xc_dom_boot_xen_init: ver 4.11, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 > domainbuilder: detail: xc_dom_parse_image: called > domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... > domainbuilder: detail: loader probe failed > domainbuilder: detail: xc_dom_find_loader: trying HVM-generic loader ... > domainbuilder: detail: loader probe failed > domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... > domainbuilder: detail: xc_dom_probe_bzimage_kernel: kernel is not a bzImage > domainbuilder: detail: loader probe failed > domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... > domainbuilder: detail: loader probe OK > xc: detail: ELF: phdr: paddr=0x100000 memsz=0x6264 > xc: detail: ELF: phdr: paddr=0x107000 memsz=0xed48 > xc: detail: ELF: memory: 0x100000 -> 0x115d48 > xc: detail: ELF: note: PHYS32_ENTRY = 0x100020 > xc: detail: ELF: Found PVH image > xc: detail: ELF: VIRT_BASE unset, using 0 > xc: detail: ELF_PADDR_OFFSET unset, using 0 > xc: detail: ELF: addresses: > xc: detail: virt_base = 0x0 > xc: detail: elf_paddr_offset = 0x0 > xc: detail: virt_offset = 0x0 > xc: detail: virt_kstart = 0x100000 > xc: detail: virt_kend = 0x115d48 > xc: detail: virt_entry = 0x1001e0 > xc: detail: p2m_base = 0xffffffffffffffff > domainbuilder: detail: xc_dom_parse_elf_kernel: hvm-3.0-x86_32: 0x100000 -> 0x115d48 > domainbuilder: detail: xc_dom_mem_init: mem 256 MB, pages 0x10000 pages, 4k each > domainbuilder: detail: xc_dom_mem_init: 0x10000 pages > domainbuilder: detail: xc_dom_boot_mem_init: called > domainbuilder: detail: range: start=0x0 end=0x10000400 > domainbuilder: detail: xc_dom_malloc : 512 kB > xc: detail: PHYSICAL MEMORY ALLOCATION: > xc: detail: 4KB PAGES: 0x0000000000000c00 > xc: detail: 2MB PAGES: 0x000000000000007a > xc: detail: 1GB PAGES: 0x0000000000000000 > domainbuilder: detail: xc_dom_build_image: called > domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x100+0x16 at 0x7f5609445000 > domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x100000 -> 0x116000 (pfn 0x100 + 0x16 pages) > xc: detail: ELF: phdr 1 at 0x7f5609445000 -> 0x7f560944b264 > xc: detail: ELF: phdr 2 at 0x7f560944c000 -> 0x7f5609453120 > domainbuilder: detail: xc_dom_load_acpi: 64 bytes at address fc008000 > domainbuilder: detail: xc_dom_load_acpi: 4096 bytes at address fc000000 > domainbuilder: detail: xc_dom_load_acpi: 28672 bytes at address fc001000 > domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x116+0x1 at 0x7f5609ace000 > domainbuilder: detail: xc_dom_alloc_segment: HVM start info : 0x116000 -> 0x117000 (pfn 0x116 + 0x1 pages) > domainbuilder: detail: alloc_pgtables_hvm: doing nothing > domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0x117000 > domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x0 > domainbuilder: detail: xc_dom_boot_image: called > domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 > domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p > domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32 <= matches > domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p > domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64 > domainbuilder: detail: domain builder memory footprint > domainbuilder: detail: allocated > domainbuilder: detail: malloc : 515 kB > domainbuilder: detail: anon mmap : 0 bytes > domainbuilder: detail: mapped > domainbuilder: detail: file mmap : 191 kB > domainbuilder: detail: domU mmap : 92 kB > domainbuilder: detail: vcpu_hvm: called > domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0xff000 > domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0xff001 > domainbuilder: detail: xc_dom_release: called > libxl: debug: libxl_event.c:2194:libxl__ao_progress_report: ao 0x5593c42e7e30: progress report: callback queued aop=0x5593c42fea10 > libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x5593c42e7e30: complete, rc=0 > libxl: debug: libxl_event.c:1404:egc_run_callbacks: ao 0x5593c42e7e30: progress report: callback aop=0x5593c42fea10 > libxl: debug: libxl_create.c:1708:do_domain_create: Domain 0:ao 0x5593c42e7e30: inprogress: poller=0x5593c42e7670, flags=ic > libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x5593c42e7e30: destroy > xencall:buffer: debug: total allocations:233 total releases:233 > xencall:buffer: debug: current allocations:0 maximum allocations:3 > xencall:buffer: debug: cache current size:3 > xencall:buffer: debug: cache hits:215 misses:3 toobig:15 > xencall:buffer: debug: total allocations:0 total releases:0 > xencall:buffer: debug: current allocations:0 maximum allocations:0 > xencall:buffer: debug: cache current size:0 > xencall:buffer: debug: cache hits:0 misses:0 toobig:0 That looks fine AFAICT. The program headers seems to be correctly identified and sized. > > > > > > > > If I make this simple change: > > > > > > --- a/bindings/xen/boot.S > > > +++ b/bindings/xen/boot.S > > > @@ -32,7 +32,7 @@ > > > #define ENTRY(x) .text; .globl x; .type x,%function; x: > > > #define END(x) .size x, . - x > > > > > > -.section .note.solo5.xen > > > +.section .note.solo5.xen, "a", @note > > > > > > .align 4 > > > .long 4 > > > > > > then I get the expected output from readelf -lW, and I can get as far as > > > the C _start() with no issues! > > > > > > FWIW, here's the diff of readelf -lW before/after: > > > > > > --- before 2020-05-26 17:36:46.117885855 +0200 > > > +++ after 2020-05-26 17:38:07.090508322 +0200 > > > @@ -8,9 +8,9 @@ > > > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > > > [Requesting program interpreter: /nonexistent/solo5/] > > > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00615c 0x00615c R E 0x1000 > > > - LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed28 RW 0x1000 > > > + LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x006120 0x00dd28 RW 0x1000 > > > > This seems suspicious, there's a change of the size of the LOAD > > section, but your change to the note type should not affect the LOAD > > section? > > Indeed. You could try to disassemble the text sections with objdump -d (or -D for all sections) and see if there's a difference between both versions, but having solved the issue maybe you just want to move on. Roger. From mirageos-devel-bounces@lists.xenproject.org Wed May 27 15:37:20 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Wed, 27 May 2020 15:37:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdy7H-0000bf-I5; Wed, 27 May 2020 15:37:15 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdxd0-00062m-VO for mirageos-devel@lists.xenproject.org; Wed, 27 May 2020 15:05:58 +0000 X-Inumbo-ID: 8ee513b6-a02b-11ea-9947-bc764e2007e4 Received: from mail.xenproject.org (unknown [104.130.215.37]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 8ee513b6-a02b-11ea-9947-bc764e2007e4; Wed, 27 May 2020 15:05:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=In-Reply-To:Content-Transfer-Encoding:Content-Type: MIME-Version:References:Message-ID:Subject:To:From:Date:Sender:Reply-To:Cc: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=6c3/y3ONIrPCVAnbVGJRw/qLH1x2DrpPzwDDCf4/n1U=; b=jYroH60y9cMXFRr0Fe/uU9l0Xb M5d+GjmDIyQfj5wZrl/vLaj4iiiQlz6OnnmkMy0AKwVFaaT+8vm7JzKX+ARhpjSyslyF/RiU3VFoh Gu/Ag2sngTaM5bkxKrLn/lorePV+XXewpWIMuu5x0n9mpEsuCA7Q73xT/0Yo+i1n4RmE=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jdxcr-0005vl-Qv; Wed, 27 May 2020 15:05:49 +0000 Received: from [93.176.191.173] (helo=localhost) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jdxcr-0002MU-GF; Wed, 27 May 2020 15:05:49 +0000 Date: Wed, 27 May 2020 17:05:39 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Martin Lucina , =?utf-8?B?SsO8cmdlbiBHcm/Dnw==?= , anil@recoil.org, Andrew Cooper , mirageos-devel@lists.xenproject.org, dave@recoil.org, xen-devel@lists.xenproject.org Subject: Re: Xen PVH domU start-of-day VCPU state Message-ID: <20200527150539.GB1195@Air-de-Roger> References: <6a22e477-c9e7-f0d7-6cb1-615137a778be@citrix.com> <20200526085221.GB5942@nodbug.lucina.net> <20200526093421.GA38408@Air-de-Roger> <20200526100337.GB38408@Air-de-Roger> <20200526101203.GE5942@nodbug.lucina.net> <20200526154224.GC25283@nodbug.lucina.net> <20200526163021.GE38408@Air-de-Roger> <20200527080008.GC4788@nodbug.lucina.net> <20200527143644.GA1195@Air-de-Roger> <20200527145702.GE4788@nodbug.lucina.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200527145702.GE4788@nodbug.lucina.net> X-Mailman-Approved-At: Wed, 27 May 2020 15:37:14 +0000 X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" On Wed, May 27, 2020 at 04:57:02PM +0200, Martin Lucina wrote: > On Wednesday, 27.05.2020 at 16:41, Roger Pau Monné wrote: > > > > > If I make this simple change: > > > > > > > > > > --- a/bindings/xen/boot.S > > > > > +++ b/bindings/xen/boot.S > > > > > @@ -32,7 +32,7 @@ > > > > > #define ENTRY(x) .text; .globl x; .type x,%function; x: > > > > > #define END(x) .size x, . - x > > > > > > > > > > -.section .note.solo5.xen > > > > > +.section .note.solo5.xen, "a", @note > > > > > > > > > > .align 4 > > > > > .long 4 > > > > > > > > > > then I get the expected output from readelf -lW, and I can get as far as > > > > > the C _start() with no issues! > > > > > > > > > > FWIW, here's the diff of readelf -lW before/after: > > > > > > > > > > --- before 2020-05-26 17:36:46.117885855 +0200 > > > > > +++ after 2020-05-26 17:38:07.090508322 +0200 > > > > > @@ -8,9 +8,9 @@ > > > > > INTERP 0x001000 0x0000000000100000 0x0000000000100000 0x000018 0x000018 R 0x8 > > > > > [Requesting program interpreter: /nonexistent/solo5/] > > > > > LOAD 0x001000 0x0000000000100000 0x0000000000100000 0x00615c 0x00615c R E 0x1000 > > > > > - LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x007120 0x00ed28 RW 0x1000 > > > > > + LOAD 0x008000 0x0000000000107000 0x0000000000107000 0x006120 0x00dd28 RW 0x1000 > > > > > > > > This seems suspicious, there's a change of the size of the LOAD > > > > section, but your change to the note type should not affect the LOAD > > > > section? > > > > > > Indeed. > > > > You could try to disassemble the text sections with objdump -d (or -D > > for all sections) and see if there's a difference between both > > versions, but having solved the issue maybe you just want to move > > on. > > I have moved on, making good progress: > > domainbuilder: detail: xc_dom_release: called > Hello, world! > Solo5: Xen hvm_start_info @0x0000000000119000 > Solo5: magic=0x336ec578 version=1 > Solo5: cmdline_paddr=0x0 > Solo5: memmap_paddr=0x119878 entries=5 > Solo5: memmap[0] = { 0x0, 0x10000400, 1 } > Solo5: mem_size=0x10000000 > | ___| > __| _ \ | _ \ __ \ > \__ \ ( | | ( | ) | > ____/\___/ _|\___/____/ > Solo5: Bindings version v0.6.5-4-g57724f8-dirty > Solo5: Memory map: 256 MB addressable: > Solo5: reserved @ (0x0 - 0xfffff) > Solo5: text @ (0x100000 - 0x104fff) > Solo5: rodata @ (0x105000 - 0x106fff) > Solo5: data @ (0x107000 - 0x118fff) > Solo5: heap >= 0x119000 < stack < 0x10000000 > Solo5: Clock source: KVM paravirtualized clock > Solo5: trap: type=#GP ec=0x0 rip=0x103a96 rsp=0xfffff90 rflags=0x2 > Solo5: ABORT: cpu_x86_64.c:181: Fatal trap > Solo5: Halted > > (The #GP is due to the timekeeping code not yet ported to Xen). > > Random question: With memory="256" in the xl config, why is the size of the > first XEN_HVM_MEMMAP_TYPE_RAM memmap entry not a multiple of PAGE_SIZE? I > had to align it down, since we put the stack at the top of RAM. 0x10000400 > seems... odd. IIRC some memory is stolen by the domain builder to place stuff (like the ACPI tables), and the end is not aligned to a page size boundary because it's not mandatory. Roger. From mirageos-devel-bounces@lists.xenproject.org Sat May 30 14:18:45 2020 Return-path: Envelope-to: archives@lists.xenproject.org Delivery-date: Sat, 30 May 2020 14:18:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jf2Jg-0007wY-Ey; Sat, 30 May 2020 14:18:28 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jf2Je-0007wT-9i for mirageos-devel@lists.xenproject.org; Sat, 30 May 2020 14:18:26 +0000 X-Inumbo-ID: 6bb29f42-a280-11ea-a9a2-12813bfff9fa Received: from orbitalfox.eu (unknown [95.172.232.202]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 6bb29f42-a280-11ea-a9a2-12813bfff9fa; Sat, 30 May 2020 14:18:24 +0000 (UTC) Received: from [192.168.88.5] (unknown [192.168.88.5]) by orbitalfox.eu (Postfix) with ESMTPSA id 530B3AA0019; Sat, 30 May 2020 15:18:22 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=orbitalfox.eu; s=orbitalfox; t=1590848302; bh=I4odS4RhcKhKL11f4L1rVW67tIukOXcKyGh91eqDYiw=; h=To:From:Subject:Cc:Date; b=l7GJM5JuQnviqreJISKb0D0xO4ioz/4rOOYePnyDEUczwwq2p0785/RC/aU+x52Kt mvGKQQcHPTYLVwtguMRygVtQKb64xuafpOKPtEwyTFPhgLIfG9PasKJasEJI/ANkHi yhtAext9gOsLJXOGwpPb+qphf7hPNtgtbxYWZayfMu2o3XYGZ/kn7l/RnOF55ZozGI zEjB1QD4fsIUnOiZgfTfjz87uTkDu0DXgTzeJf3ps60scW9IiCf9/B8L5C/eqE8DJc h8H1pgW9dVtfii6jtJJgGM9Wq3bERRzRUEa96K3siyRt6SmGX2/WCVD/HF4zGO/ZHm 6C04UJloJwVsQ== To: Etienne Millon From: =?UTF-8?B?b3JiaWZ4IPCfpoo=?= Subject: Hacl ARM compilation issue Message-ID: <16034b74-790d-bbee-da24-4450e89adeae@orbitalfox.eu> Date: Sat, 30 May 2020 15:18:21 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:68.0) Gecko/20100101 Thunderbird/68.8.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: mirageos-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Developer list for MirageOS List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: mirageos-devel@lists.xenproject.org Errors-To: mirageos-devel-bounces@lists.xenproject.org Sender: "MirageOS-devel" Hello! I encountered a warning, which causes Hacl compilation failure on armv7l. I'm guessing it can't do a 64bit shift since it's 32bit? Copying mirageos list as this appears to be a subproject. ``` [...] kremlib.h:498:42: warning: right shift count >= width of type [-Wshift-count-overflow] 498 | ~(FStar_UInt64_eq_mask(x >> 64, y >> 64))) | | ^~ kremlib.h:499:31: warning: right shift count >= width of type [-Wshift-count-overflow] 499 | (FStar_UInt64_eq_mask(x >> 64, y >> 64) & FStar_UInt64_gte_mask(x, y)); | ^~ kremlib.h:499:40: warning: right shift count >= width of type [-Wshift-count-overflow] 499 | (FStar_UInt64_eq_mask(x >> 64, y >> 64) & FStar_UInt64_gte_mask(x, y)); | ^~ kremlib.h:500:28: warning: left shift count >= width of type [-Wshift-count-overflow] 500 | return ((uint128_t)mask) << 64 | mask; | ^~ ```