[Xen-devel] Xen 4.5 development update

[konrad.wilk@xxxxxxxxxx]

Below is a summary of the projects / features being worked on for the 4.5
time frame.  The tentative feature freeze is scheduled for September 10th,
which is months away.  With that in mind, I think it's time to take
stock of the development, so we know whether to ask for more help or divert
resources.

The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
Instead of values (0->100) it is now five states:

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.

Not exactly sure what that is in terms of libvirt.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


= Open =

== ARM ==

*  ARM  - IOMMU support (good)
  -  Julien Grall

*  ARM - Device assigment on ARM (good)
   Xen parts done. Linux at risk
  -  Julien Grall

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
  -  Arianna Avanzini

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  ARM Xen UEFI booting on ARM (none)
   Based on Daniel's multiboot2-EFI work
  -  Roy Franz

*  ARM PSCI v0.2 (good)
  -  Parth Dixit

*  ARM GICv3 support (good)
  -  Vijay Kilari

*  ARM GICv2m support (none)
  -  Linaro (unknown)

*  ARM VM save/restore/live migration (fair)
  -  Wei Huang

== x86 ==

*  New migration. (good)
  -  Andrew Cooper & David Vrabel

*  PVH - AMD hardware support. (fair)
   Issues with FSBASE MSRs
  -  Mukesh Rathor

*  Xen multiboot2-EFI support (fair)
   Needed for SecureBoot
  -  Daniel Kiper

*  Re-write of vHPET (fair)
   Xen 4.5 blocker
  -  Don Slutz

*  HT enabled, virtualization overhead is high (Xen 4.4) (none)
   kernbench demonstrated it
  -  Dario Faggioli

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  dirty vram / IOMMU bug (fair)
  -  Zhang, Yang Z

*  Convert tasklet to per-cpu tasklets (fair)
  -  Konrad Rzeszutek Wilk

*  VMware backdoor (hypercall) (ok)
   Needs to be split up and redone
  -  Don Slutz

*  VPMU - 'perf' support in Xen (good)
  -  Boris Ostrovsky

*  ioreq-server, aka secondary emulators (good)
   Chase down XSM maintainer
  -  Paul Durrant

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
  -  Dario Faggioli

*  Cache QoS Monitoring - hypercalls (fair)
   Just hypercalls - no toolstack changes.
  -  Dongxiao Xu and Shantong Kang

*  HVM guest NUMA (none)
  -  Matt Wilson

*  1TB slow destruction (ok)
  -  Bob Liu

*  extending mem_access support to PV domain (ok)
  -  Aravindh Puthiyaparambil (aravindp)

*  Stability fix (good)
  -  Jan Beulich

== QEMU ==

*  Rebase of QEMU 2.0 and SeaBIOS (fair)
  -  Ian Jackson

*  Using qemu-upstream in a stubdomain (fair)
  -  Ian Jackson

*  Intel IGD PCI GPU passthrough (ok)
   Maintainers of QEMU don't like it. Want a different way.
  -  Chen, Tiejun

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  Bigger PCI hole in QEMU (ok)
   Needs to be rebased
  -  Don Slutz

== lib{xc,xl} and toolstack ==

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  Rearrange and cleanup installation destination directories (/var -> 
var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl work - JSON to keep track of guest configs (ok)
  -  Wei Liu

*  pvscsi should be targeted for 4.5, a prototype exists (fair)
  -  Olaf Hering

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get 
two hung domains
  -  Ian Jackson

*  Remus in Xen (ok)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  COLO in Xen (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

== Linux ==

*  Linux block multiqueue (fair)
  -  Arianna Avanzini

*  Netback grant table manipulations (ok)
  -  Zoltan Kiss

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  Linux pvops of Xen EFI hypercall support (good)
  -  Daniel Kiper

*  Netback multiqueue (good)
  -  Wei Liu

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to Xen 4.6 ==

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

*  vNUMA in Xen (ok)
  -  Elena Ufimtseva

*  cpuid leveling (none)
   
http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

== Up for grabs ==

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with 
PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default 
in xl.conf.  For example, if we could have a default "backend" parameter for 
vifs, then it would be easy to switch back and forth between a backend in a 
driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of HPET (done)
  -  Don Slutz

*  vAPIC in PVHVM guests (done)
  -  Boris Ostrovsky

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except 
one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel