Re: [Xen-devel] Re: [PATCH 0 of 3] Patches for PCI passthrough with modified E820 (v3) - resent.

[Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>]

On Tue, 2011-05-10 at 16:51 +0100, Konrad Rzeszutek Wilk wrote:
> > > > > > > > > This has been tested with 2.6.18 (RHEL5), 2.6.27(SLES11), 
> > > > > > > > > 2.6.36, 2.6.37, 2.6.38,
> > > > > > > > > and 2.6.39 kernels. Also tested with PV NetBSD 5.1.
> > > > > > 
> > > > > > They all saw the full amount of RAM? Since the domain builder does 
> > > > > > not
> > > > > 
> > > > > Correct. So if the guest had 6GB passed in, roughly 3GB ended below 
> > > > > the
> > > > > 4GB mark, and then rest - 3GB got stuck behind the 4GB.
> > > > 
> > > > Do you happen to know what caused that to work? I wasn't aware of any
> > > > code which moved the p2m around based on e820 in older guests (and I
> > > > can't find any right now) and I didn't think the builder itself paid any
> > > > attention to the e820 either.
> > > 
> > > Correct. The build does not pay any attention to it. But the Xen setup 
> > > code
> > > in the older kernels (or perhaps it is the balloon code itself), does 
> > > inflate
> > > back to what was deflated.
> > 
> > I'm not concerned with the ballooned case, I'm thinking of the normal
> > "boot with all memory" case. If the builder has populated 0..4G with RAM
> > but the E820 says 0..3G,4G..5G (i.e. an IO hole at 3G..4G) then somebody
> > must move the mfns from pfn-space 3G..4G to 4G..5G, mustn't they? Or at
> > least deallocate the ones in 3G..4G so that ballooning will subsequently
> > take over.
> 
> Right - and in the case of RHEL5, SLES11 ..
> > 
> > >  The PVOPS did not do that (argh, I didn't say
> > > that in the earlier email - sorry about misleading you about that).
> > > 
> > > But just to make sure I am not feeding false information let me re-run 
> > > this
> > > with an RHEL5 guest and double-check.
> > 
> 
> .. the kernels do all the magic stuff. I provided 4G to the guest, with a PCI
> device (and hence enabled the E820 patches), and while the bootup at the start
> initially finds:
> 
> Linux version 2.6.18-194.el5xen (mockbuild@xxxxxxxxxxxxxxxxxxxx) (gcc version 
> 4.
> 1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Fri Apr 2 15:34:40 EDT 2010
> BIOS-provided physical RAM map:
>  Xen: 0000000000000000 - 00000000bffb0000 (usable)
>  Xen: 00000000bffb0000 - 00000000bffbe000 (ACPI data)
>  Xen: 00000000bffbe000 - 00000000bffe0000 (ACPI NVS)
>  Xen: 00000000bffe0000 - 00000000c0000000 (reserved)
>  Xen: 00000000fec00000 - 00000000fec01000 (reserved)
>  Xen: 00000000fee00000 - 00000000fee01000 (reserved)
>  Xen: 00000000fff00000 - 0000000100000000 (reserved)
>  Xen: 0000000100000000 - 0000000140850000 (usable)
> ..
> Memory: 3026548k/5251392k available (2512k kernel code, 118176k reserved, 
> 1396k 
> ..
> when it finished booting, I found that 4GBs are available:
> [root@g-rhel5-amd64 ~]# cat /proc/meminfo  | grep MemTotal
> MemTotal:      4186112 kB

Can you actually allocate and use (nearly) all of that 4G? (modulo
kernel allocations/overheads etc). The above doesn't really show that
the p2m above 4G is actually sane (I admit I'd be surprised if we got
away with it enough to boot if it wasn't...).

linux-2.6.18-xen.hg has a decrease_reservation in in setup_arch which
will give back pages between max_pfn and xen_start_info->nr_pages but I
don't think that will trigger for this case, will it? I don't see any
other plausible looking decrease_reservation calls...

Anyway, I guess if it works then it works and although I'm curious as to
how/why I'm not curious enough to start digging too deeply...

> With the PVOPS one, it does almost all of the same thing except
> it does not balloon back to the initial size (4GB), so the user
> has to manually inflate. But that seems like a Linux kernel feature
> is missing: I get the same thing when my 8GB machine boots the PVOPS kernel.
> I end up with 6GB (no dom0_mem thing). If I try to 'inflate' past the 6GB it
> ignores me - while if I boot with the older Xen-O-Linux I get 8GB to Dom0.
> 
> NetBSD seems to do the same thing as RHEL5, SLES11 (note, I could not actually
> give it 4096MB, b/c the NetBSD unstable kernel would crash - irregardless if
> a PCI device was passed in. But passing in 3900MB made it work). With 3900MB
> and a PCI device (so with a modified E820), the guest told me I've
> 3891 MB free (the rest seem to be used by the kernel).
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel