Re: changing Dom0 data during smc call inside of xen during cache coloring

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Thu, 28 Sep 2023, Oleg Nikitenko wrote:
> Hello Julien,
> 
> I am still fighting with xen Cache Coloring with aes.
> When I sent a request to hardware aes after xen with CC started I got the 
> mistake in CSU.
> When I dumped structure contents on both sides I got the different data.
> Xilinx related contact wrote to me.
> 
> When cache coloring is enabled, Dom0 is not 1:1 mapped (guest physical 
> addresses in Dom0 != physical addresses). If the Xilinx drivers in
> Linux (xcsudma.c) issue EEMI calls with a guest physical address (for 
> instance the address of a memory buffer allocated by Linux), then
> this address is no longer a physical address and would need to be translated. 
> EEMI calls always get forwarded to Xen first, then Xen issues
> a corresponding EEMI call to the firmware (see 
> xen/arch/arm/platforms/xilinx-eemi.c:xilinx_eemi). But Xen is probably 
> passing the EEMI
> calls parameters unmodified. Then PMU tries to read the address but since 
> this is not a physical address, it fails. Basically we need to
> add code to Xen xen/arch/arm/platforms/xilinx-eemi.c:xilinx_eemi to translate 
> any guest physical addresses passed as EEMI calls arguments
> into physical addresses before making the EEMI call to firmware.
> 
> This is an example patch, which is translating the parameter on register x2 
> for the EEMI call 0xC200002F. I haven't checked the EEMI
> protocol for this call but this just an example to show you how to translate 
> parameters.
> 
> diff --git a/xen/arch/arm/platforms/xilinx-eemi.c 
> b/xen/arch/arm/platforms/xilinx-eemi.c index 500c86dc69..bff1b71196 100644 ---
> a/xen/arch/arm/platforms/xilinx-eemi.c +++ 
> b/xen/arch/arm/platforms/xilinx-eemi.c @@ -409,6 +409,30 @@ bool 
> xilinx_eemi(struct
> cpu_user_regs *regs, const uint32_t fid, } goto forward_to_fw;
> 
>  *  case 0xC200002F:
>  *  {
>  *  uint64_t example_possible_address_param = get_user_reg(regs, 2);
>  *  uint64_t translated_address = mfn_to_maddr(gfn_to_mfn(current->domain,
>  *  gaddr_to_gfn(example_possible_address_param)));
>  *  translated_address += example_possible_address_param & ~PAGE_MASK; +
>  *  arm_smccc_1_1_smc(get_user_reg(regs, 0),
>  *  get_user_reg(regs, 1),
>  *  translated_address,
>  *  get_user_reg(regs, 3),
>  *  get_user_reg(regs, 4),
>  *  get_user_reg(regs, 5),
>  *  get_user_reg(regs, 6),
>  *  get_user_reg(regs, 7),
>  *  &res); +
>  *  set_user_reg(regs, 0, res.a0);
>  *  set_user_reg(regs, 1, res.a1);
>  *  set_user_reg(regs, 2, res.a2);
>  *  set_user_reg(regs, 3, res.a3);
>  *  return true;
>  *  }
> 
> + default: if ( is_hardware_domain(current->domain) ) goto forward_to_fw;
> 
> The aes request structure contains physical addresses of the source and 
> destination.
> These addresses are obtained via two calls dma_alloc_coherent.
> The address of this structure is kept at x2 register.

OK I see the question is about the Xilinx Xen tree. In the logs below we
have:

> (XEN) d0v1 Forwarding AES operation: 3254779951 r2 0 -> 11432000              
>                                           log from xen
 
So it looks Linux passed 0 as address in x2, which cannot be right?



> I applied the suggested scheme in xen for xilinx_eemi(...) function.
> 
> case 0xC200002F:
> {
> uint64_t paramaddr = get_user_reg(regs, 2);

It would seem that this is not read correctly? It should not be zero.


> uint64_t phyaddr = mfn_to_maddr(gfn_to_mfn(current->domain, 
> gaddr_to_gfn(paramaddr)));
> phyaddr += (paramaddr & ~PAGE_MASK);
> gprintk(XENLOG_DEBUG, "Forwarding AES operation: %u r2 %lx -> %lx\n", fid, 
> paramaddr, phyaddr);
> set_user_reg(regs, 2, phyaddr);
> }
> goto forward_to_fw;
> 
> As a result I got the same issue as earlier.
> 
> [   17.350086]
> zynq_aes_gcm                                                                  
>                                                       user
> log
> 
> [   17.350202] @ dma_alloc firmware:zynqmp-firmware:zynqmp-aes @              
>                                          kernel log from Dom0
> [   17.353015] @@@ firmware:zynqmp-firmware:zynqmp-aes 0 @@@
> [   17.358515] zynqmp_aes [0] ffffffc00910d000 2806000 
> firmware:zynqmp-firmware:zynqmp-aes
> [   17.366546] @ dma_alloc firmware:zynqmp-firmware:zynqmp-aes @
> [   17.372347] @@@ firmware:zynqmp-firmware:zynqmp-aes 0 @@@
> [   17.377775] zynqmp_aes [1] ffffffc009115000 42a14000 keytype 1
> [   17.383660] zynqmp_aes [2] dump request align 1 ++
> [   17.388501] 00 60 80 02 00 00 00 00
> [   17.392032] 50 60 80 02 00 00 00 00
> [   17.395583] 00 00 00 00 00 00 00 00
> [   17.399117] 00 60 80 02 00 00 00 00
> [   17.402664] 40 00 00 00 00 00 00 00
> [   17.406226] 00 00 00 00 00 00 00 00
> [   17.409755] 01 00 00 00 00 00 00 00
> [   17.413311] zynqmp_aes [3] dump request --
> 
> (XEN) d0v1 Forwarding AES operation: 3254779951 r2 0 -> 11432000              
>                                           log from xen

Here


> @ 000042A14000
> @                                                                             
>                                                          csu
> log from aes
> 04 E4 00 6F 05 E4 00 6F
> 06 E4 00 6F 07 E4 00 6F
> 10 E4 00 6F 11 E4 00 6F
> 12 E4 00 6F 13 E4 00 6F
> 14 E4 00 6F 15 E4 00 6F
> 16 E4 00 6F 17 E4 00 6F
> 18 E4 00 6F 19 E4 00 6F
> 
> ERROR:   pm_aes_engine ### args 6 ret 0 addr 0 42a14000 ###                   
>                                              ATF log
> 
> So the address of the structure was not changed.
> This is the question. 
> How can I map this address to xen and change physical addresses there ?