Re: [Xen-devel] Re: blktap: Sync with XCP, dropping zero-copy.

[Daniel Stodden <daniel.stodden@xxxxxxxxxx>]

On Mon, 2010-11-15 at 15:07 -0500, Ian Campbell wrote:
> On Mon, 2010-11-15 at 19:34 +0000, Jeremy Fitzhardinge wrote: 
> > On 11/15/2010 11:19 AM, Ian Campbell wrote:
> > > On Mon, 2010-11-15 at 18:27 +0000, Jeremy Fitzhardinge wrote:
> > >> Then we'd have a set of frames whose lifetimes are being determined by
> > >> some other subsystem.  We can either maintain a list of them and poll
> > >> waiting for them to become free, or just release them and let them be
> > >> managed by the normal kernel lifetime rules (which requires that the
> > >> memory attached to them be completely normal, of course). 
> > > GNTTABOP_unmap_and_replace is probably the answer to this? This is what
> > > netback uses (via gnttab_copy_grant_page) when it wants to copy and skb
> > > which has remained in flight for too long.
> > >
> > > Maybe this doesn't work since the memcpy and replace are not atomic and
> > > we don't have a lock to use, since we don't know which subsystem holds
> > > the reference. However it should work if we don't really care about the
> > > contents of the replacement too much, which I don't think we do in this
> > > case since the data could just as easily get clobbered by the userspace
> > > process which thinks the write has completed fully, in which case we
> > > just replace the grant mapping without bothering with the copy.
> > 
> > Well, if atomicity is important (and it is a relatively rare event), I
> > dare say Xen could de-schedule all the VCPUs a lot more efficiently than
> > we could run stop_machine() from within the kernel...
> 
> Absolutely. 
> 
> GNTTABOP_unmap_and_replace is itself atomic according to the comment in
> the header, it's not clear what it is atomic WRT, presumably just other
> gnttab or mmu ops. In any case it doesn't handle synchronising the
> content of the old and new pages so if we care about that we would need
> a new GNTAPOP_copy_unmap_and_replace or something like that.

Ah, nice. First time I hear we have a dedicated call for that. So far I
was assuming dom0 would have to unmap/remap and doing so atomically
across cores would be left as an exercise to dom0.

One would need a copy for protocols where it isn't clear that the dup
gets dropped on the receiving side. So I guess the main hazard would be
writes on sth like NFS over UDP.

In that case, I think blktap has a pretty safe spot to copy data between
I/O completion from userland and replacing the page, without extra
atomicity.

Daniel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel