[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Ethernet PCI passthrough problem


  • To: Jan Beulich <JBeulich@xxxxxxxx>
  • From: Frédéric Pierret <frederic.pierret@xxxxxxxxxxxx>
  • Date: Sat, 6 Jul 2019 17:11:33 +0200
  • Arc-authentication-results: i=1; mx.zoho.com; dkim=pass header.i=qubes-os.org; spf=pass smtp.mailfrom=frederic.pierret@xxxxxxxxxxxx; dmarc=pass header.from=<frederic.pierret@xxxxxxxxxxxx> header.from=<frederic.pierret@xxxxxxxxxxxx>
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1562425897; h=Content-Type:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To:ARC-Authentication-Results; bh=yUbDP3q/txd/A/4NN41sFAcw4qGUMzVEw7CbVsUeA/8=; b=C4Lw841GDhZmi3AlWknelWcZ7qsPARaLloqCixwfIJ/vVDpSgSPjGhfVJeXJAyZtAa+06yDN+MOILiEEXvvyyWpl8QI2f46I6VvDu/21gy18zJ/70yRGrnXsA82FSzkdg4ykRofL1p2zKXesSFDgNr0JYl7Qu2aP209qilV9dWg=
  • Arc-seal: i=1; a=rsa-sha256; t=1562425897; cv=none; d=zoho.com; s=zohoarc; b=A4y/RrPnH6I5gLZzbP28xvoYm8KNpolC7EcB0vBhwId8kHU7iTOUBjEzrVz8dIN2UuwEdKTcSGuXo420R4PcQcyN6AW+H1ybp67b0/m4z8rtHurM6s0d+Ew0Lk8RyjAUKOJvwNaAjREu/9QE22iEkptxASvQhZAcmAECvmrPgTU=
  • Cc: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Sat, 06 Jul 2019 15:12:01 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 7/5/19 4:24 PM, Jan Beulich wrote:
> On 05.07.2019 15:46, Frédéric Pierret  wrote:
>> I'm experiencing problem to perform PCI passthrough of Ethernet card
>> with 4 ports (HP Ethernet 1Gb 4-port 331FLR Adapter) on an HP DL360 Gen8.
>>
>> I have two server like this one where the first is under CentOS and the
>> other one, under Qubes. Under CentOS, the nics are not attached to any
>> other domain and classical dmesg shows no errors (see attached
>> 'centos_kvm.png'). It's working very well since long time.
> The name of the image suggests this is under KVM, not Xen. The device
> being at bus 3 rather than bus 0 also suggests this isn't inside a
> Xen HVM DomU.
Yes it's under KVM and not attached to any VM. It was to confirm that
the NICs are working with the driver, not necessary under Xen.
>> I'm trying to
>> switch these servers to Qubes and I'm facing trouble. In Qubes, we
>> attach all the nics into a domain, usually called 'sys-net' in HVM mode.
>>
>> The nics are attached with 'rdm_policy=relaxed' to 'sys-net' but are not
>> loaded in the domain due to errors (see attached 'HVM_dom0.png' and
>> 'HVM_sys_net.png').
> The former of these shows a fundamental problem: Two of the RMRRs
> overlap the BIOS area inside the guest. I'm afraid I don't see how
> to deal with this (short of shuffling the BIOS elsewhere, which
> imo is not really an option). I wonder how this gets dealt with in
> the CentOS case, where you say things work (I take it that you've
> verified that the RMRRs on both systems are at exactly the same
> addresses).
I attempt to follow
https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229,
in order to exclude RMRR region but as I cannot determine the slot of
the extension card, I had no success at this point. It's not a PCI-E
card it's an embedded module.
>
> And then I'm puzzled by there being further messages about 03:00.2,
> suggesting that domain construction (or device assignment)
> continues. Yet then the same messages don't appear for the other
> two devices (you did say there are four of them, and other logs
> also support this).
Me too as sometimes it is 03:00.3 or .1 instead of .2 .
>> I tried in PV mode, I got it working but I was not
>> happy with that for security reason. I decided to update my bios to the
>> most recent one, and even in PV, it does not work anymore (see attached
>> 'PV_dom0.png' and 'PV_sys_net.png').
> That'll require figuring out what exactly the driver isn't liking.
> At the first glance I'm inclined to think the BIOS update broke
> things.
I will try to dig into 'sys-net' to debug the driver itself.
>> All have been tried under Qubes 4.0.1 (xen-4.8) and Qubes 4.1 under
>> development (xen-4.12). Current attached log images are with xen-4.12.
> You saying "log images" already points at a problem: Actual (and
> complete as well as sufficiently verbose) log files would be more
> helpful when diagnosing issues like this one.
Sorry about that. As I'm using remote console management (ILO) and no
network for debugging Qubes, it was easier to provide screenshot.
>
> Jan

Thank you for your answer. Any tips for debugging more, even the driver
are welcome. Thank you again.

Frédéric


Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.