Re: [Xen-devel] Ethernet PCI passthrough problem

[Frédéric Pierret <frederic.pierret@xxxxxxxxxxxx>]

On 7/5/19 4:24 PM, Jan Beulich wrote:
> On 05.07.2019 15:46, Frédéric Pierret  wrote:
>> I'm experiencing problem to perform PCI passthrough of Ethernet card
>> with 4 ports (HP Ethernet 1Gb 4-port 331FLR Adapter) on an HP DL360 Gen8.
>>
>> I have two server like this one where the first is under CentOS and the
>> other one, under Qubes. Under CentOS, the nics are not attached to any
>> other domain and classical dmesg shows no errors (see attached
>> 'centos_kvm.png'). It's working very well since long time.
> The name of the image suggests this is under KVM, not Xen. The device
> being at bus 3 rather than bus 0 also suggests this isn't inside a
> Xen HVM DomU.
Yes it's under KVM and not attached to any VM. It was to confirm that
the NICs are working with the driver, not necessary under Xen.
>> I'm trying to
>> switch these servers to Qubes and I'm facing trouble. In Qubes, we
>> attach all the nics into a domain, usually called 'sys-net' in HVM mode.
>>
>> The nics are attached with 'rdm_policy=relaxed' to 'sys-net' but are not
>> loaded in the domain due to errors (see attached 'HVM_dom0.png' and
>> 'HVM_sys_net.png').
> The former of these shows a fundamental problem: Two of the RMRRs
> overlap the BIOS area inside the guest. I'm afraid I don't see how
> to deal with this (short of shuffling the BIOS elsewhere, which
> imo is not really an option). I wonder how this gets dealt with in
> the CentOS case, where you say things work (I take it that you've
> verified that the RMRRs on both systems are at exactly the same
> addresses).
I attempt to follow
https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229,
in order to exclude RMRR region but as I cannot determine the slot of
the extension card, I had no success at this point. It's not a PCI-E
card it's an embedded module.
>
> And then I'm puzzled by there being further messages about 03:00.2,
> suggesting that domain construction (or device assignment)
> continues. Yet then the same messages don't appear for the other
> two devices (you did say there are four of them, and other logs
> also support this).
Me too as sometimes it is 03:00.3 or .1 instead of .2 .
>> I tried in PV mode, I got it working but I was not
>> happy with that for security reason. I decided to update my bios to the
>> most recent one, and even in PV, it does not work anymore (see attached
>> 'PV_dom0.png' and 'PV_sys_net.png').
> That'll require figuring out what exactly the driver isn't liking.
> At the first glance I'm inclined to think the BIOS update broke
> things.
I will try to dig into 'sys-net' to debug the driver itself.
>> All have been tried under Qubes 4.0.1 (xen-4.8) and Qubes 4.1 under
>> development (xen-4.12). Current attached log images are with xen-4.12.
> You saying "log images" already points at a problem: Actual (and
> complete as well as sufficiently verbose) log files would be more
> helpful when diagnosing issues like this one.
Sorry about that. As I'm using remote console management (ILO) and no
network for debugging Qubes, it was easier to provide screenshot.
>
> Jan

Thank you for your answer. Any tips for debugging more, even the driver
are welcome. Thank you again.

Frédéric

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel