[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH V4 00/13] x86/Hyper-V: Add Hyper-V Isolation VM support

On 9/2/2021 3:59 PM, Christoph Hellwig wrote:
On Tue, Aug 31, 2021 at 05:16:19PM +0000, Michael Kelley wrote:
As a quick overview, I think there are four places where the
shared_gpa_boundary must be applied to adjust the guest physical
address that is used.  Each requires mapping a corresponding
virtual address range.  Here are the four places:

1)  The so-called "monitor pages" that are a core communication
mechanism between the guest and Hyper-V.  These are two single
pages, and the mapping is handled by calling memremap() for
each of the two pages.  See Patch 7 of Tianyu's series.

Ah, interesting.

3)  The network driver send and receive buffers.  vmap_phys_range()
should work here.

Actually it won't.  The problem with these buffers is that they are
physically non-contiguous allocations.  We really have two sensible

  1) use vmap_pfn as in the current series.  But in that case I think
     we should get rid of the other mapping created by vmalloc.  I
     though a bit about finding a way to apply the offset in vmalloc
     itself, but I think it would be too invasive to the normal fast
     path.  So the other sub-option would be to allocate the pages
     manually (maybe even using high order allocations to reduce TLB
     pressure) and then remap them

Agree. In such case, the map for memory below shared_gpa_boundary is not necessary. allocate_pages() is limited by MAX_ORDER and needs to be called repeatedly to get enough memory.

  2) do away with the contiguous kernel mapping entirely.  This means
     the simple memcpy calls become loops over kmap_local_pfn.  As
     I just found out for the send side that would be pretty easy,
     but the receive side would be more work.  We'd also need to check
     the performance implications.

kmap_local_pfn() requires pfn with backing struct page and this doesn't work pfn above shared_gpa_boundary.

4) The swiotlb memory used for bounce buffers.  vmap_phys_range()
should work here as well.

Or memremap if it works for 1.

Now use vmap_pfn() and the hv map function is reused in the netvsc driver.

Case #2 above does unusual mapping.  The ring buffer consists of a ring
buffer header page, followed by one or more pages that are the actual
ring buffer.  The pages making up the actual ring buffer are mapped
twice in succession.  For example, if the ring buffer has 4 pages
(one header page and three ring buffer pages), the contiguous
virtual mapping must cover these seven pages:  0, 1, 2, 3, 1, 2, 3.
The duplicate contiguous mapping allows the code that is reading
or writing the actual ring buffer to not be concerned about wrap-around
because writing off the end of the ring buffer is automatically
wrapped-around by the mapping.  The amount of data read or
written in one batch never exceeds the size of the ring buffer, and
after a batch is read or written, the read or write indices are adjusted
to put them back into the range of the first mapping of the actual
ring buffer pages.  So there's method to the madness, and the
technique works pretty well.  But this kind of mapping is not
amenable to using vmap_phys_range().

Hmm.  Can you point me to where this is mapped?  Especially for the
classic non-isolated case where no vmap/vmalloc mapping is involved
at all?

This is done via vmap() in the hv_ringbuffer_init()

182/* Initialize the ring buffer. */
183int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
184 struct page *pages, u32 page_cnt, u32 max_pkt_size)
186        int i;
187        struct page **pages_wraparound;
189        BUILD_BUG_ON((sizeof(struct hv_ring_buffer) != PAGE_SIZE));
191        /*
192 * First page holds struct hv_ring_buffer, do wraparound mapping for
193         * the rest.
194         */
195 pages_wraparound = kcalloc(page_cnt * 2 - 1, sizeof(struct page *),
196                                   GFP_KERNEL);
197        if (!pages_wraparound)
198                return -ENOMEM;
/* prepare to wrap page array */
200        pages_wraparound[0] = pages;
201        for (i = 0; i < 2 * (page_cnt - 1); i++)
202                pages_wraparound[i + 1] = &pages[i % (page_cnt - 1) + 1];
/* map */
204        ring_info->ring_buffer = (struct hv_ring_buffer *)
205 vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP, PAGE_KERNEL);
207        kfree(pages_wraparound);
210        if (!ring_info->ring_buffer)
211                return -ENOMEM;
213        ring_info->ring_buffer->read_index =
214                ring_info->ring_buffer->write_index = 0;



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.