Re: [Xen-users] domU only see 512MB RAM when enabling PCI passthrough

[Casey DeLorme <cdelorme@xxxxxxxxx>]

As mentioned, that patch is only for Dom0 with EFI and has nothing to do with DomU's.

I have very little PV experience myself, but I can suggest a couple of ideas.

You might try dropping the RAM to maybe 3G (3072) and seeing if anything changes. ÂPerhaps try increments of 1G and see where it breaks. ÂIf reducing the supplied RAM suddenly provides more memory then maybe the bug I referenced earlier wasn't fixed after all. ÂIt would give you something more to work with anyways.

Another possibility is manually setting `e820_host`, which according to the man pages is specifically for supporting PCI Passthrough to PV, however the same description claims that it should automatically be on when devices are passed. ÂCan't hurt to give it a try.

Also, just a reminder to use "Reply All" to keep the list in the loop.

Out of curiosity, is there any reason why you would choose PV over PVHVM? ÂMy understanding is that PVHVM performs nearly the same while remaining mostly encapsulated without all the confusion I see regarding PV configuration and bugs. ÂIf you don't have a specific reason it might be worth investigating.

In any event, I hope these help with troubleshooting.

Sincerely,

~Casey

PS - Sorry for top posting, just wanted the original message to remain in context when I included xen-users again.

Also, another reference link:

http://wiki.xen.org/wiki/Virtualization_Spectrum

On Mon, Mar 11, 2013 at 10:47 PM, Baojun Wang <wangbj@xxxxxxxxx> wrote:

Yep, I tried comment the code by shadowofgray, but it still yields the
same result.

Below is domU configuration:

kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/pvgrub.lst"

memory = '8192'

vcpus = 4

disk = [ '/dev/sda1,raw,xvda1,w', '/dev/vg00/gentoo,raw,xvda2,w',
'/dev/vg00/exports,raw,xvda3,w', Â'/dev/vg01/home,raw,xvda4,w',
'/dev/vg01/swap,raw,xvda5,w' ]

name = 'gentoo'

serial = 'pty'
#
gfx_passthru=1

pci = [ '00:02.0', '00:1a.0', '00:1b.0', '00:1d.0', '03:00.0', '04:00.0' ]

vif = [ 'mac=fe:00:36:79:00:01, bridge=br0, virtio' ]


boot option for xen.gz already includes pci devices above. and pci
devices passed through are video/audio/usb:
Below is lspci output:

00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor
Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core
Processor Family PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation
Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200
Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset
Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset
Family PCI Express Root Port 1 (rev b5)
00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset
Family PCI Express Root Port 2 (rev b5)
00:1c.2 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset
Family PCI Express Root Port 3 (rev b5)
00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset
Family PCI Express Root Port 4 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset
Family PCI Express Root Port 5 (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation H67 Express Chipset Family LPC
Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series
Chipset Family SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family
SMBus Controller (rev 05)
03:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB
Host Controller
04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB
Host Controller
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168 PCI Express Gigabit Ethernet controller (rev 06)



Thanks
baojun

On Mon, Mar 11, 2013 at 5:15 PM, Casey DeLorme <cdelorme@xxxxxxxxx> wrote:
> That patch was for the EFI Dom0 memory recognition, and was patched sometime
> before 4.2 released (by reversing the if statement). ÂYour xl dmesg shows
> e820 (and approx. 16G) so I am quite certain that is not the cause of your
> DomU memory bug.
>
> Can you provide your PV DomU configuration?
>
>
> I know I've heard about the bug before, usually in the context of over 3GB
> to a PV domain, but from what I recall it was fixed:
> http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1673
>
>
>
> On Mon, Mar 11, 2013 at 6:42 PM, ShadesOfGrey <shades_of_grey@xxxxxxxxxxxxx>
> wrote:
>>
>> From Casey Delorme's "Comprehensive Xen Debian Wheezy PCI Passthrough
>> Tutorial", this may be the potential cause of your issue.
>>
>> EFI Source Modification
>>
>> Given that I encountered no make errors, I was shocked to find that they
>> still hadn't addressed the grub efi compatibility bug where it fails to
>> recognize available system memory.
>>
>> The fix for this is a source modification, though they have supposedly
>> added a build process for xen.efi, which would replace the debian.efi
>> generated by grub. I have yet to find adequate instructions to make that
>> work, and even if I did may break adding xen-pciback.hide to the grub
>> configuration, which we need if we are planning to do the easy method of PCI
>> Passthrough.
>>
>> So, let's open up xen/arch/x86/setup.c with our favorite text editor, and
>> find the line containing ( e820_raw_nr != 0 ), and make that area look like
>> this:
>>
>> #if 0
>> Âelse if ( e820_raw_nr != 0 )
>> Â{
>> Âmemmap_type = "Xen-e820";
>> Â}
>> Âelse if ( bootsym(lowmem_kb) )
>> Â{
>> Âmemmap_type = "Xen-e801";
>> Âe820_raw[0].addr = 0;
>> Âe820_raw[0].size = bootsym(lowmem_kb) << 10;
>> Âe820_raw[0].type = E820_RAM;
>> Âe820_raw[1].addr = 0x100000;
>> Âe820_raw[1].size = bootsym(highmem_kb) << 10;
>> Âe820_raw[1].type = E820_RAM;
>> Âe820_raw_nr = 2;
>> Â}
>> Â#endif
>> Âelse if ( mbi->flags & MBI_MEMMAP )
>>
>>
>> Notice we are commenting out a section of code using "if 0", which always
>> evaluates to false, to eliminate the section of e801 mapping which breaks
>> our ram recognition.
>>
>> I have no idea what kind of adverse affects this has on other systems, but
>> it has worked for me since Xen 4.1.2.
>>
>> End EFI Source Modification
>>
>>
>>
>> On 03/08/2013 07:44 PM, Baojun Wang wrote:
>>
>> Hi list,
>>
>> I'm trying to run pvlinux (via pvgrub) using pci passthrough (and vga
>> pass through), xen version is 4.2.1, dom0 & domU kernel are both
>> 3.8.2, I checked the kernel log (DMAR) and the motherboard (H67)
>> supports vt-d, processor is i7-2600 and it supports vt-d too.
>>
>> When pci passthrough is not enabled, everything is fine;
>> but after I enabled pci pass through, domU can only see 512MB RAM (USB
>> passed from pci-passthrough works, but video/intel didn't work, likely
>> due to i915 load error from dmesg, everything else is also good).
>>
>> I googled and found a similar issue is reported at:
>> http://lists.xen.org/archives/html/xen-users/2012-12/msg00179.html
>>
>> But seems there is no follow up how the problem is fixed, how can I
>> fix the issue?
>>
>>
>> Below are some links from pastebin.com:
>>
>> 1) native dmesg without loading xen.gz: http://pastebin.com/BTVNTi8D
>> 2) (kernel) dmesg of dom0: http://pastebin.com/DEZ7gfmG
>> 3) (xl) dmesg of dom0: http://pastebin.com/46bR2ngv
>> 4) domU dmesg without pci passthrough: http://pastebin.com/DGZR4NLP
>> 5) domU dmesg with pci passthrough: http://pastebin.com/aujxwys2
>>
>> I tried xen version 4.2.0 too but still no luck.
>> My kernel have all xen feature enabled; and I can provide more info if
>> necessary.
>>
>> Thanks
>> baojun
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxx
>> http://lists.xen.org/xen-users
>>
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxx
>> http://lists.xen.org/xen-users
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxx
> http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users