[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Minios-devel] [UNIKRAFT PATCHv4 21/43] plat/kvm: Add Arm64 basic entry code





On 13/07/18 11:11, Wei Chen wrote:
Hi Julien,

Hi Wei,

-----Original Message-----
From: Julien Grall <julien.grall@xxxxxxx>
Sent: 2018年7月12日 18:06
To: Wei Chen <Wei.Chen@xxxxxxx>; minios-devel@xxxxxxxxxxxxxxxxxxxx;
simon.kuenzer@xxxxxxxxx
Cc: Kaly Xin <Kaly.Xin@xxxxxxx>; nd <nd@xxxxxxx>
Subject: Re: [Minios-devel] [UNIKRAFT PATCHv4 21/43] plat/kvm: Add Arm64 basic
entry code



On 12/07/18 10:43, Wei Chen wrote:
Hi Julien,

Hi Wei,

-----Original Message-----
From: Julien Grall <julien.grall@xxxxxxx>
Sent: 2018年7月11日 23:52
To: Wei Chen <Wei.Chen@xxxxxxx>; minios-devel@xxxxxxxxxxxxxxxxxxxx;
simon.kuenzer@xxxxxxxxx
Cc: Kaly Xin <Kaly.Xin@xxxxxxx>; nd <nd@xxxxxxx>
Subject: Re: [Minios-devel] [UNIKRAFT PATCHv4 21/43] plat/kvm: Add Arm64
basic
entry code



On 11/07/18 10:50, Wei Chen wrote:
Hi Julien,

Hi Wei,

-----Original Message-----
From: Julien Grall <julien.grall@xxxxxxx>
Sent: 2018年7月8日 6:24
To: Wei Chen <Wei.Chen@xxxxxxx>; minios-devel@xxxxxxxxxxxxxxxxxxxx;
simon.kuenzer@xxxxxxxxx
Cc: Kaly Xin <Kaly.Xin@xxxxxxx>; nd <nd@xxxxxxx>
Subject: Re: [Minios-devel] [UNIKRAFT PATCHv4 21/43] plat/kvm: Add Arm64
basic
entry code

Hi,

On 07/06/2018 10:03 AM, Wei Chen wrote:
QEMU/KVM can boot an Arm64 elf image without multiboot. In this case,
we can plage _libkvmplat_entry to entry64.S directly as the vCPU
reset entry. In this basic entry code, we just initialize the boot
stack and prepare jumping to _libkvmplat_start.
Can you clarify why you are using the ELF format and not Image? My main
concern is the former does not seem to have a clear description of the
state of the VM at boot.


It's little hard for me to answer your question. This is why I reply this
Comment at the last. Actually, when I was selecting the elf image I didn’t
think so much. And most Unikernel projects that I have involved (ukvm,
mini-
os)
are using the elf image, both for arm and x86.

Mini-OS ARM is using the zImage format, not ELF. For UKVM, IIRC, you

Yes, arm32 is zImage, but x86_64 is using OUTPUT_FORMAT("elf64-x86-64").
And about the ukvm, Yes, I wrote it by myself, because the ukvm requires elf
format, it only support elf loader. We want to make our Unikernel
application
like a normal elf application that can run on virtual machine directly. We
don't
want to be compatible with the Linux image boot protocol, it's too complex
for us.
And we don't have some many information need to pass.

Why is it too complex? The Image format is basically a couple fields to
slap at the top of your binary and a few guidelines for the tools how to
deal with the VM state. Most of the boot code will be the same.

Also, most likely if you are going to support kvmtools, you would need
to use the Image format. For Xen, support for ELF will require a lot of
reworks and to be honest it is not worth the effort (I tried it before).


I have read the Xen code and Shijie's Arm64 mini-OS patches. Shijie was
Using elf format as image.From his code, I didn't see lots of reworks
That I need to do If I will enable elf format for Xen?
Arm64 Mini-OS is not using ELF. Xen tools does not have ELF support for Arm, and you can look at the patch [1] to see the image been created using objcopy.



Kvmtools doesn't support Arm elf loader. But it supports other architectures.
And as your said, if we using the kernel image format? How about ukvm,
it requires elf format? Relative to kvmtools, I am more like to support ukvm.
Because in a way, qemu and kvmtools are overlapping. Actually, I don't mind to
provide different image format for different platform.

May I ask why did you decide to require using ELF for UKVM? What was the rationale? Is that just because x86 was doing it?

But then why using that for QEMU? You need to compile your ELF assuming VA == PA, which is rather fragile. You also yet haven't told me what is the expect ABI (e.g state of the machine...) for ELF.



wrote it yourself. So I guess you based your understanding of the state
of the VM from somewhere?


Each SoC, include QEMU virtual machine, they will give most system registers
a default value. I am using cortex-a53 VCPU, the QEMU gives it a reset value
to cpu->reset_sctlr = 0x00c50838; MMU, I/D cache are disabled.
But yes, you're right, I ignore the MMU has been disabled already, and
disable
It again in entry code.

This looks very fragile to rely on for a specific processor. What if we
decide to use another Cortex-A*? Or even another Arm CPU?

We need a clear definition of the VM state. For instance, you say the
cache is disabled. Has the kernel image/DT been cleaned to PoC by the
tools? You will also have to be careful when writing the page-table as
"Cache disabled" does not rely mean "disabled". It is more a by-pass of
the cache.


I can read them from sctlr if I want. I am curious about how does other
Bootloader handle such case? Don't think my code is an entry code for
OS. Let's treat it as a bootloader.

I don't understand what you mean. Whether your code is a bootloader or an OS, you have to know exactly what is the state of your platform when the processor jumps to your entry code.

Your Image has been loaded by QEMU in the memory. If you modify the page-table with cache disabled and not clean the DT. Then you may end up using stall information when re-enabling the cache.

If you disable the MMU, then this will still have an impact on your cache as well. This is because KVM has alias to that region you will modify and therefore you may have stall data in your cache when turning on the MMU.

Overall, I think you should treat Unikraft as an OS as usually the format will give you more guaranty than the platform itself. This will also allow you to re-use that code accross multiple environment rather than tailoring for QEMU.

Cheers,

[1] https://lists.xenproject.org/archives/html/minios-devel/2018-04/msg00139.html

--
Julien Grall

_______________________________________________
Minios-devel mailing list
Minios-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/minios-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.