Re: [PATCH v2 01/17] xen/riscv: implement sbi_remote_hfence_gvma()

[Oleksii Kurochko <oleksii.kurochko@xxxxxxxxx>]

On 6/18/25 5:15 PM, Jan Beulich wrote:

> On 10.06.2025 15:05, Oleksii Kurochko wrote:
> > Instruct the remote harts to execute one or more HFENCE.GVMA instructions,
> > covering the range of guest physical addresses between start_addr and
> > start_addr + size for all the guests.
> Here and in the code comment: Why "for all the guests"? Under what conditions
> would you require such a broad (guest) TLB flush?
>     
Hmm, it seems like KVM always do such a broad (guest) TLB flush during detection
of VMIDLEN:
	void __init kvm_riscv_gstage_vmid_detect(void)
	{
		unsigned long old;
	
		/* Figure-out number of VMID bits in HW */
		old = csr_read(CSR_HGATP);
		csr_write(CSR_HGATP, old | HGATP_VMID);
		vmid_bits = csr_read(CSR_HGATP);
		vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
		vmid_bits = fls_long(vmid_bits);
		csr_write(CSR_HGATP, old);
	
		/* We polluted local TLB so flush all guest TLB */
		kvm_riscv_local_hfence_gvma_all();
	
		/* We don't use VMID bits if they are not sufficient */
		if ((1UL << vmid_bits) < num_possible_cpus())
			vmid_bits = 0;
	}

It is not clear actually why so broad and why not hfence_gvma_vmid(vmid_bits).

And I am not really 100% sure that any hfence_gvma() is needed here as I don't see
what could pollutes local guest TLB between csr_write() calls.

RISC-V spec. says that:
	Note that writing hgatp does not imply any ordering constraints between page-table updates and
	subsequent G-stage address translations. If the new virtual machine’s guest physical page tables have
	been modified, or if a VMID is reused, it may be necessary to execute an HFENCE.GVMA instruction
	(see Section 18.3.2) before or after writing hgatp.

But we don't modify VM's guest physical page table. We could potentially reuse VMID between csr_write()
calls, but it is returning back and we don't switch to a guest with this "new" VMID, so it isn't really used.

Do you have any thoughts about that?

~ Oleksii