Re: [Xen-devel] Criteria / validation proposal: drop Xen

[Adam Williamson <awilliam@xxxxxxxxxx>]

On Mon, 2019-05-13 at 16:29 -0600, Lars Kurth wrote:
> Hi all,
> 
> I am going to step in here with my hat as Xen Project community
> manager. We had a discussion about this issue as part of last week's
> community call. I CC'ed a number of stake-holders, which probably
> should have been on the thread such as ITL (which builds QubesOS
> on top of Fedora) and Michael A Young (the Xen package manager).
> 
> First of all apologies that this issue has lingered so long. Key
> members of the community were not aware of the issues raised in
> this thread, otherwise we would have acted earlier. With this in
> mind, please in future raise issues with me, on xen-devel@,
> committers@ or the Xen-Fedora package manager. The Xen Community
> would like to see Fedora running as guest: in fact it would be
> somewhat odd if there was a Xen-Dom0 package and guest support
> didn't work. And there are some downstreams such as QubesOS,
> which depend on this support.

Thanks for stepping in. Of course we always want as much stuff as
possible to work, but that does not mean we block the release on it. We
certainly want Fedora to work as a guest on VMWare, VirtualBox and
Parallels too; we don't block the release on any of those either...

> > Well, I mean, every few *days* a compose gets nominated for validation
> > testing, and a mail is sent to test-announce. Just check your test-
> > announce archives for mails with "nominated for testing" in their
> > subject lines, and you'll see dozens. Is this not sufficient
> > notification?
> 
> We discussed this at the community call and came to the conclusion that
> we can run regular tests of Fedora RC's as part of our OSSTEST
> infrastructure. Ian Jackson volunteered to implement this, but there
> are some questions on
> a) The installer (which we can handle ourselves)
> b) When we would trigger a test - aka is there some trigger other than the
> c) How would results best be reported back to Fedora
> 
> Apologies, I am not very familiar with how the Fedora Test group works.
> Is there some documentation which would help integrate what you to with
> the test system of another open source project?

b) you can use fedmsg / fedora-messaging:
https://github.com/fedora-infra/fedmsg
https://github.com/fedora-infra/fedora-messaging
A message is emitted every time a compose attempt finishes (on the fedmsg
topic 'org.fedoraproject.prod.pungi.compose.status.change': see
https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.pungi.compose.status.change
for a log of past messages). What you will want to do is listen for
completed Branched and Rawhide composes and run tests whenever one
completes successfully. This is already exactly what we do to schedule
openQA tests; you can crib from the openQA test scheduler:
https://pagure.io/fedora-qa/fedora_openqa
particularly the fedmsg consumer:
https://pagure.io/fedora-qa/fedora_openqa/blob/master/f/fedora_openqa/consumer.py

c) ideally it would be good to report to both resultsdb and to the
wiki. Again, we already do this for openQA, and you can crib from the
code there:
https://pagure.io/fedora-qa/fedora_openqa/blob/master/f/fedora_openqa/report.py
reporting to ResultsDB might be tricky due to authentication issues,
I'm not sure if we ever put the openID auth stuff into production. For
wiki reporting you will either have to auth manually every so often or
ask Fedora infra for a special token that doesn't expire (this is what
we do for the openQA results).
Reporting to ResultsDB you do through resultsdb_api -
https://pagure.io/taskotron/resultsdb_api - and optionally you can use
my resultsdb_conventions -
https://pagure.io/taskotron/resultsdb_conventions - which makes it
somewhat easier (IMO anyway) and will make your results consistent with
those from openQA and Autocloud. Reporting to the wiki you can do
through my crazy python-wikitcms library -
https://pagure.io/fedora-qa/python-wikitcms . Again, fedora_openqa does
all this for openQA results, so you can crib from that. Let me know if
you have trouble with this.

> > > > from Oracle. On that basis, I'm proposing we remove this Final
> > > > criterion:
> > > 
> > > s/Oracle/Xen Project/ I believe?
> > 
> > Perhaps, it's just that it always seemed to be you doing the testing,
> > so they got a bit conflated :)
> 
> Can we come to some arrangement, by which such issues get communicated
> to the Xen Project earlier? Aka me, xen-devel@ or committers@

It would be nice if you could ensure someone from Xen is actually
watching the Fedora lists, if working in Fedora is a target for Xen. We
*could* try and CC stuff all the time, but imagine if we tried to do
that for everybody. But yes, for future conversations of this nature
I'll try and remember to include those lists.

> > > > "The release must boot successfully as Xen DomU with releases providing
> > > > a functional, supported Xen Dom0 and widely used cloud providers
> > > > utilizing Xen."
> > > > 
> > > > and change the 'milestone' for the test case -
> > > > https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_Xen_Para_Virt -
> > > > from Final to Optional.
> > > > 
> > > > Thoughts? Comments? Thanks!
> > > 
> > > I would prefer for it to remain as it is.
> > 
> > This is only practical if it's going to be tested, and tested regularly
> > - not *only* on the final release candidate, right before we sign off
> > on the release. It needs to be tested regularly throughout the release
> > cycle, on the composes that are "nominated for testing".
> 
> Would the proposal above work for you? I think it satisfies what you are
> looking for. We would also have someone who monitors these test results
> pro-actively.

In theory, yeah, but given the history here I'm somewhat sceptical. I'd
also say we still haven't really got a convincing case for why we
should continue to block the release (at least in theory) on Fedora
working in Xen when we don't block on any other virt stack apart from
our 'official' one, and we don't block on all sorts of other stuff we'd
"like to have working" either. Regardless of the testing issues, I'd
like to see that too if we're going to keep blocking on Xen...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel