Re: [PATCH v5 12/15] VT-d: replace all-contiguous page tables by superpage mappings

[Jan Beulich <jbeulich@xxxxxxxx>]

On 02.06.2022 11:35, Roger Pau Monné wrote:
> On Fri, May 27, 2022 at 01:19:55PM +0200, Jan Beulich wrote:
>> When a page table ends up with all contiguous entries (including all
>> identical attributes), it can be replaced by a superpage entry at the
>> next higher level. The page table itself can then be scheduled for
>> freeing.
>>
>> The adjustment to LEVEL_MASK is merely to avoid leaving a latent trap
>> for whenever we (and obviously hardware) start supporting 512G mappings.
>>
>> Note that cache sync-ing is likely more strict than necessary. This is
>> both to be on the safe side as well as to maintain the pattern of all
>> updates of (potentially) live tables being accompanied by a flush (if so
>> needed).
>>
>> Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
>> Reviewed-by: Kevin Tian <kevin.tian@xxxxxxxxx>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>

Thanks.

>> ---
>> Unlike the freeing of all-empty page tables, this causes quite a bit of
>> back and forth for PV domains, due to their mapping/unmapping of pages
>> when they get converted to/from being page tables. It may therefore be
>> worth considering to delay re-coalescing a little, to avoid doing so
>> when the superpage would otherwise get split again pretty soon. But I
>> think this would better be the subject of a separate change anyway.
>>
>> Of course this could also be helped by more "aware" kernel side
>> behavior: They could avoid immediately mapping freed page tables
>> writable again, in anticipation of re-using that same page for another
>> page table elsewhere.
> 
> Could we provide an option to select whether to use super-pages for
> IOMMU, so that PV domains could keep the previous behavior?

Hmm, I did (a while ago) consider adding a command line option, largely
to have something in case of problems, but here you're asking about a
per-domain setting. Possible, sure, but I have to admit I'm always
somewhat hesitant when it comes to changes requiring to touch the tool
stack in non-trivial ways (required besides a separate Dom0 control).

It's also not clear what granularity we'd want to allow control at:
Just yes/no, or also an upper bound on the page sizes permitted, or
even a map of (dis)allowed page sizes?

Finally, what would the behavior be for HVM guests using shared page
tables? Should the IOMMU option there also suppress CPU-side large
pages? Or should the IOMMU option, when not fulfillable with shared
page tables, lead to use of separate page tables (and an error if
shared page tables were explicitly requested)?

Jan