Re: Design session notes: GPU acceleration in Xen

[Demi Marie Obenour <demi@xxxxxxxxxxxxxxxxxxxxxx>]

On Mon, Jun 17, 2024 at 05:39:23PM +0200, Jan Beulich wrote:
> On 17.06.2024 17:17, Demi Marie Obenour wrote:
> > On Mon, Jun 17, 2024 at 11:07:54AM +0200, Jan Beulich wrote:
> >> On 14.06.2024 18:44, Demi Marie Obenour wrote:
> >>> On Fri, Jun 14, 2024 at 10:12:40AM +0200, Jan Beulich wrote:
> >>>> On 14.06.2024 09:21, Roger Pau Monné wrote:
> >>>>> I'm not sure it's possible to ensure that when using system RAM such
> >>>>> memory comes from the guest rather than the host, as it would likely
> >>>>> require some very intrusive hooks into the kernel logic, and
> >>>>> negotiation with the guest to allocate the requested amount of
> >>>>> memory and hand it over to dom0.  If the maximum size of the buffer is
> >>>>> known in advance maybe dom0 can negotiate with the guest to allocate
> >>>>> such a region and grant it access to dom0 at driver attachment time.
> >>>>
> >>>> Besides the thought of transiently converting RAM to kind-of-MMIO, this
> >>>> makes me think of another possible option: Could Dom0 transfer ownership
> >>>> of the RAM that wants mapping in the guest (remotely resembling
> >>>> grant-transfer)? Would require the guest to have ballooned down enough
> >>>> first, of course. (In both cases it would certainly need working out how
> >>>> the conversion / transfer back could be made work safely and reasonably
> >>>> cleanly.)
> >>>
> >>> The kernel driver needs to be able to reclaim the memory at any time.
> >>> My understanding is that this is used to migrate memory between VRAM and
> >>> system RAM.  It might also be used for other purposes.
> >>
> >> Except: How would the kernel driver reclaim the memory when it's mapped
> >> by a DomU?
> > 
> > The Xen driver in dom0 will register for MMU notifier callbacks.  When
> > the kernel driver reclaims the memory, the Xen driver will be notified,
> > and it will issue a hypercall that tells Xen to remove the memory from
> > the DomU's address space.  Subsequent accesses to the pages will trigger
> > a stage 2 translation fault that is handled by an IOREQ server.
> 
> And such an ioreq server, which I assume isn't going to run in the Dom0
> kernel, will then also need keeping up-to-date on holes in the (virtual)
> BAR. Oh well ...

My initial plan was that it _would_ run in the dom0 kernel, because this
results in a cleaner userspace API.  Ultimately I think it is best to go
with whichever approach keeps the kernel code simpler, but I'm not sure.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature