[Xen-devel] Early x86 boot code build system questions

[Daniel Kiper <daniel.kiper@xxxxxxxxxx>]

Hi all,

During work on multiboot2 protocol support for Xen others and I spotted some
issues and/or deficiencies in early x86 boot code build system.

Some people reported that after applying v2 of my patches they cannot build
xen.gz due to following error:

objdump -h reloc.o | sed -n '/[0-9]/{s,00*,0,g;p;}' |\
        while read idx name sz rest; do \
                case "$name" in \
                .data|.data.*|.rodata|.rodata.*|.bss|.bss.*) \
                        test $sz != 0 || continue; \
                        echo "Error: non-empty $name: 0x$sz" >&2; \
                        exit $(expr $idx + 1);; \
                esac; \
        done
Error: non-empty .rodata: 0x01c
build32.mk:22: recipe for target 'reloc.o' failed

Surprisingly second run usually builds xen.gz without any issue. The problem is
in xen/arch/x86/boot/build32.mk %.o: %.c recipe and I will post relevant fix
for this in v3.

Anyway, issue was introduced by "[PATCH v2 08/23] x86: add multiboot2 protocol
support" patch 
(http://lists.xen.org/archives/html/xen-devel/2015-07/msg03828.html)
and "[PATCH v2 21/23] x86/boot: implement early command line parser in C" patch
(http://lists.xen.org/archives/html/xen-devel/2015-07/msg03838.html). The 
problem
is that optimization engine in newer GCC versions try to optimize switch/case
construct by putting some data in .rodata section (try GCC with -S option and
look for labels starting with "CSWTCH"). As you can see this is not allowed by
our build rules.

Potentially we can avoid switch/case construct and use ifs. Not nice. After some
search I found that problematic GCC optimization feature could be disabled by
-fno-tree-switch-conversion option. However, I am not happy with that because if
newer GCC version will once again create not allowed sections then we will be 
hit
by the same problem.

So, I asked myself why we do not allow .rodata sections (and others) in 
intermediate
*.o files? I did some tests with relevant checks disabled and xen.gz works 
without
any issue. Of course final image is larger due to some alignments but I do not 
believe
that this is real explanation for this restriction. So, I would like to ask why 
these
constraints were put here and in other places? Could we safely remove them? If 
yes then how?

During later discussion about patches Jan suggested another change in Xen build 
system
(check http://lists.xen.org/archives/html/xen-devel/2015-09/msg02877.html for 
more details):

**********************************************************************

[...]

> Ditto. So, what is your decision? Include or define? If include then
> we should think how to generate relevant dependencies automatically.

I think the question should rather be whether we can't make cmdline.c
build the normal way, not the reloc.c one.

[...]

**********************************************************************

Ha! The simplest way looks like direct linking with other object files. No way!
Early x86 boot code object files are 32-bit and others are 64-bit. I have not
found anything which allows us such weired things. However, I am sure that guys
in other OS related projects were hit by the same problem. I am right. Linux
kernel looks like good example (yep, I know that there are others in the wild
but I have relevant sources under my feet and I am lazy). As I saw x86-64 Linux
kernel contains 32-bit early boot code and 64-bit main code. However, both of
them are not linked together by binutils linker. They are linked to separate ELF
executables at first. Later special tool reads them and creates proper vmlinuz 
file.

Potentially we can go Linux way and avoid all above mentioned issues. However,
this requires huge changes in our build system. So, I think that right now we
can use workaround with -fno-tree-switch-conversion GCC option. Later, during
work on new PE file (this thing requires also substantial changes in build 
system)
which will support three boot protocols we could revisit the issue and resolve
it by building 32-bit and 64-bit code separately. Then linking both of them
with our special tool. What do you think about that?

Daniel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel