[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [openxt-dev] VirtIO-Argo initial development proposal


  • To: Julien Grall <julien.grall.oss@xxxxxxxxx>
  • From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Date: Thu, 31 Dec 2020 12:38:52 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ErATkNMeYpagVSj1neMVSCRFlx5oN0uzbtN5CjR2NEY=; b=cz6pNqolLogKuIG89k8mYjlQnrAF/EwZu5xC3COyUlWl+8N0dNRIV5ckDn3MayRxs/L0tJltDBrmFeXSt5mfoH49OcWtU4P/zjNK3QbiLSgVe37cJgfoDvDWVHrhgQk24/eUNp8DRlfTTt/aHj6MpTogv3s/27JzsljLuRP/tbb406D51lcMKCbWWZPJfiovJdNCN29fup1dzRU6I6mhs6F26GaxrB3LrSQBLniDWVjKenqrvEd0HfpkMCmi44wi9BqK1U5tRsdKTjfgN986mRlkAMf91vt4cI/ri/bi4dolgsahKbimqyNX4lCZL3vkYiQwquOQft8+6l1/lNjwDQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cnYr/kZEz39dyb9POLb0L/NbdjunkQIIaIi8GrFLYc2Yo4geY9rPqopzNnaG7DR7z2jWW2nVCqoXdT5kor/FJQt8fu+7u3cA7MtxhnvP9bU/bW7k5Np+7qr9IpOsKIDHMhAIj9IzfZ+ljqn0GP2s7YSLvPnzBdigpN8fPy/GUDLE8JaUassBtj80iQCP8YmFvxICUiPPWmt0/fhz4a8Qkkk3JmGhDZY9oeXHHURFOGV6Cs64+QCE/oGccPWSuqVBS404YQAA0yl/pjO0h1AGKJ6T26i3/7qZKpbWDiTzEM06w2+L8TjzQ8ox0LNm94lf82oorPrc7q+VIxU7fS3HJg==
  • Authentication-results: esa1.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
  • Cc: Rich Persaud <persaur@xxxxxxxxx>, Jean-Philippe Ouellet <jpo@xxxxxx>, Christopher Clark <christopher.w.clark@xxxxxxxxx>, openxt <openxt@xxxxxxxxxxxxxxxx>, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Oleksandr Tyshchenko <olekstysh@xxxxxxxxx>, Julien Grall <jgrall@xxxxxxxxxx>, James McKenzie <james@xxxxxxxxxxx>, Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>, Paul Durrant <pdurrant@xxxxxxxxxxxx>
  • Delivery-date: Thu, 31 Dec 2020 11:39:25 +0000
  • Ironport-sdr: oOnwT6W/RrcwnMG6MKptGOw0WkJqDyddswjM1Gw1tqxIELwS3ipwlmddzUjqP7g7GfNmv/9bN6 IzOdtbyhTAWbdB5FHlWahZOwwkbWNGAA5toDyYwy7yl4YS1T4rLWPT/bq8/XEb5Bf0UBUdegTR Q/srjy8C5I0akqD+HZafvBEUxSIyJBKDQ9BYxNVqhnfdFh7gK650Zvug1m0q2hIhro01ki20h/ GYkFNAUAGVs3EtnmqecVXHa1P2y2pxCdEcq71388vd9zkMDul8HitBQFgzPlumDZRd2xL9XKRP wSY=
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Thu, Dec 31, 2020 at 11:02:40AM +0000, Julien Grall wrote:
> On Thu, 31 Dec 2020 at 08:46, Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> >
> > On Wed, Dec 30, 2020 at 11:30:03AM +0000, Julien Grall wrote:
> > > Hi Roger,
> > >
> > > On 29/12/2020 09:17, Roger Pau Monné wrote:
> > > > On Wed, Dec 23, 2020 at 04:32:01PM -0500, Rich Persaud wrote:
> > > > > On Dec 17, 2020, at 07:13, Jean-Philippe Ouellet <jpo@xxxxxx> wrote:
> > > > > > On Wed, Dec 16, 2020 at 2:37 PM Christopher Clark
> > > > > > <christopher.w.clark@xxxxxxxxx> wrote:
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I have written a page for the OpenXT wiki describing a proposal 
> > > > > > > for
> > > > > > > initial development towards the VirtIO-Argo transport driver, and 
> > > > > > > the
> > > > > > > related system components to support it, destined for OpenXT and
> > > > > > > upstream projects:
> > > > > > >
> > > > > > > https://openxt.atlassian.net/wiki/spaces/~cclark/pages/1696169985/VirtIO-Argo+Development+Phase+1
> > > >
> > > > Thanks for the detailed document, I've taken a look and there's indeed
> > > > a lot of work to do listed there :). I have some suggestion and
> > > > questions.
> > > >
> > > > Overall I think it would be easier for VirtIO to take a new transport
> > > > if it's not tied to a specific hypervisor. The way Argo is implemented
> > > > right now is using hypercalls, which is a mechanism specific to Xen.
> > > The concept of hypervisor call is not Xen specific. Any other hypervisor 
> > > can
> > > easily implement them. At least this is the case on Arm because we have an
> > > instruction 'hvc' that acts the same way as a syscall but for the
> > > hypervisor.
> > >
> > > What we would need to do is reserving a range for Argos. It should be
> > > possible to do it on Arm thanks to the SMCCC (see [1]).
> > >
> > > I am not sure whether you have something similar on x86.
> >
> > On x86 Intel has vmcall and AMD vmmcall, but those are only available
> > to HVM guests.
> 
> Right, as it would for other architecture if one decided to implement
> PV (or binary translated) guests.
> While it may be possible to use a different way to communicate on x86
> (see more below), I am not sure this would be the case for other
> architectures.
> The closest thing to MSR on Arm would be the System Registers. But I
> am not aware of a range of IDs that could be used by the software.

I don't really know that much about Arm to make any helpful statement
here. My point was to keep in mind that it might be interesting to try
to define an hypervisor agnostic mediated data exchange interface, so
that whatever is implemented in the VirtIO transport layer could also
be used by other hypervisors without having to modify the transport
layer itself. If that's better done using a vmcall interface that's
fine, as long as we provide a sane interface that other hypervisors
can (easily) implement.

> >
> > > > IMO it might be easier to start by having an Argo interface using
> > > > MSRs, that all hypervisors can implement, and then base the VirtIO
> > > > implementation on top of that interface.
> > > My concern is the interface would need to be arch-specific. Would you mind
> > > to explain what's the problem to implement something based on vmcall?
> >
> > More of a recommendation than a concern, but I think it would be more
> > attractive for upstream if we could provide an interface to Argo (or
> > hypervisor mediated data exchange) that's no too tied into Xen
> > specifics.
> 
> I agree with this statement. We also need an interface that is ideally
> not to arch-specific otherwise it will be more complicated to get
> adopted on other arch.
> For instance, at the moment, I don't really see what else can be used
> on Arm (other that MMIO and PCI) if you want to care about PV (or
> binary translated) guests.

My recommendation was mostly to make Argo easier to propose as an
hypervisor agnostic mediated data exchange, which in turn could make
adding a VirtIO transport layer based on it easier. If that's not the
case let's just forget about this.

> > My suggestion for using MSRs was because I think every x86 hypervisor
> > must have the logic to trap and handle some of those, and also every
> > OS has the helpers to read/write MSRs, and those instructions are not
> > vendor specific.
> 
> In order to use MSRs, you would need to reserve a range of IDs. Do x86
> have a range of ID that can be used for software purposes (i.e. the
> current and future processors will not use it)?

There's a range of MSRs (0x40000000-0x400000FF) that are guaranteed to
always be invalid on bare metal by Intel, so VMware, HyperV and
others have started using this range to add virtualization specific
MSRs. You can grep for HV_X64_MSR_* defines on Xen for some of the
HyperV ones.

Roger.



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.