Re: [PATCH v5 07/11] xen/domctl: Introduce XEN_DOMCTL_CDF_vpci flag

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 13 Oct 2021, Jan Beulich wrote:
> On 13.10.2021 14:11, Bertrand Marquis wrote:
> >> On 13 Oct 2021, at 11:56, Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> >> If vPCI for Arm on 4.16 is not going to be functional, why so much
> >> pressure in pushing those patches so fast? I understand the need to
> >> remove stuff from the queue, but I don't think it's worth the cost of
> >> introducing a broken interface deliberately on a release.
> 
> +1
> 
> > We did not push that fast, those have been on the mailing list for a while 
> > (this is the 5th version of the serie).
> > PCI passthrough is a big change requiring lots of patches and we decided to 
> > push it piece by piece to make
> > the review easier.
> 
> 5 versions for a series like this one was to be expected. Imo it has
> been wrong in the past to rush in new features (even if experimental
> ones) very close to the freeze, and it has mislead people to think
> they can delay work until very (too?) late a point in time.


Hi Roger, Jan,

Let me take a few minutes to clarify and provide context for this work.


I don't think anyone "pushed hard" any of the ARM series close to the
release. I sent "one" email in public as a reminder of things that need
reviewing:
https://marc.info/?l=xen-devel&m=163373776611154

I did send a couple of private emails to Jan but they were to synchronize
ourselves rather than push; Jan, I hope you didn't take them the wrong
way.


At the same time it is certainly true that all the people involved here
worked very hard to get these series ready for 4.16. Oct 15 is the Xen
Project feature freeze. It is also the deadline for many of us working
with upstream Xen Project to upstream features and report back to our
management. I know it doesn't affect you guys directly, but you can
imagine that as employees of our respective companies we feel pressure
to complete our objectives in the given timeframe. Of course we need to
make sure to do that without compromising quality and without
overextending upstream review capacity.


The ARM PCI series are public since Dec 2020, pushed to a gitlab branch
for testing: 
https://gitlab.com/xen-project/fusa/xen-integration/-/tree/integration/pci-passthrough
I helped creating, managing and testing the branch.

So, I can see why Bertrand feels like they have been around for a while:
we have been dealing with these patches for almost a year, even if from
a xen-devel perspective we are "only" at version 5.


In the past there have been contributors dropping a large half-broken
series at the last minute before the code freeze, asking for it to be
committed, promising it will be improved later, and then disappearing
right after the release, leaving the code half-broken forever.

It is easy to draw comparisons between that kind of behavior and what is
happening in this thread. We have all been burned by it at some point.

However, that is really *not* what is happening this week.

PCI Passthrough for ARM is a difficult feature to implement. We have
been discussing it for 5+ years, close to 10 years. I know it sounds
unbelievable but it is true. For one reason or the other it never
materialized. Until now.

Even with all the outstanding series committed, PCI Passthrough for ARM
is not fully functional, but it is a lot closer to the objective.
Rahul, Oleksandr and their teams have been already "sticking around" for
almost a year making improvements and without their help going forward
it is not going to get finished anyway. We still don't have MSI support.

Given that Rahul, Oleksandr and the others are required to finish the
work anyway, and they are not at risk of disappearing, I think we can
deal with one or two TODO left by Oct 15 [1].

My suggestion is to accept the TODO for patch #8 [1] but I agree with
Roger that we shouldn't introduce bad interfaces, especially as they
impact x86 which is not "tech preview". So I think we should revert
patch #7 (this patch.) I'll reply with more details in separate shorter
email.

[1] https://marc.info/?l=xen-devel&m=163412120531248


> In fact I'm getting somewhat angry seeing how much effort is put into
> getting this work in (including myself doing reviews there), when at
> the same time far older series of mine aren't given any chance to
> make any progress. Not your fault, sure, but an observation.


Jan,

Let me start by pointing out that without your help we would never have
managed to get so many ARM patches committed for 4.16. Not just PCI
related, but also EFI too. You should really get credit for that.

I can imagine it is frustrating helping so much somebody else but not
getting help for your own series. In a way it happened to me too as the
only Xilinx series outstanding didn't make any progress:
https://marc.info/?l=xen-devel&m=163056545414965

The only way to solve this problem is by growing the review capacity of
the community. You'll be happy to know that (without naming names) I
have heard from multiple people, new members of the community, that they
are going to commit time to do reviews during the next months. We are
growing the community again and some of the new contributors will become
good reviewers and maintainers over time. I am positive that the next 12
months will be a lot better than the last 12 months from that
perspective.


Thanks Jan, Roger, the two Oleksandr, Rahul, Michal, Bertrand, Luca and
everyone else that stayed up late these last few weeks to push Xen
forward with new groundbreaking features and pushing the project forward
in new directions. I do think 4.16 is going to be a great release and
the both CI-loops are still fully working with staging very close to
master, further proof of the great work of this community.

Cheers,

Stefano