Re: Fwd: changing Dom0 data during smc call inside of xen during cache coloring

[Stefano Stabellini <stefano.stabellini@xxxxxxx>]

Hi Oleg,

You are getting this output:

> (XEN) d0v0 Forwarding AES operation: 3254779951 136bb00000000000 -> 
> fffffffffffff000

Which means that the guest physical address is 0x136bb00000000000 and
the translated physical address is 0xfffffffffffff000. It generates an
error so you are asking if 0xfffffffffffff000 is incorrect because the
translation is incorrect.

This is possible. However, I think it is more likely that
0x136bb000_00000000 is incorrect. This an extremely high address. Could
it be wrong?

Can you check against your device tree that 0x136bb000_00000000
corresponds to memory? Or something valid?

Cheers,

Stefano


On Mon, 2 Oct 2023, Oleg Nikitenko wrote:
> Hello Stefano,
> Hello Julien,
> 
> > OK I see the question is about the Xilinx Xen tree. In the logs below we 
> > have:
> 
> I see. After the correction r2 to r1 CSU got aborted immediately.
> 
> (XEN) d0v0 Forwarding AES operation: 3254779951 136bb00000000000 -> 
> fffffffffffff000
> Received exception
> MSR: 0x200, EAR: 0x2F, EDR: 0x0, ESR: 0x861
> 
> I printed the value which I got after the expression which I provided in the 
> 1-st email executed.
> 
> So I may conclude that Xilinx's real physical address calculation scheme on 
> the xen CC is incorrect.
> 
> Could you suggest a correct one ?
> 
> Regards,
> Oleg Nikitenko
> 
> пт, 29 сент. 2023 г. в 00:50, Stefano Stabellini <sstabellini@xxxxxxxxxx>:
>       On Thu, 28 Sep 2023, Oleg Nikitenko wrote:
>       > Hello Julien,
>       >
>       > I am still fighting with xen Cache Coloring with aes.
>       > When I sent a request to hardware aes after xen with CC started I got 
> the mistake in CSU.
>       > When I dumped structure contents on both sides I got the different 
> data.
>       > Xilinx related contact wrote to me.
>       >
>       > When cache coloring is enabled, Dom0 is not 1:1 mapped (guest 
> physical addresses in Dom0 != physical addresses). If the
>       Xilinx drivers in
>       > Linux (xcsudma.c) issue EEMI calls with a guest physical address (for 
> instance the address of a memory buffer allocated by
>       Linux), then
>       > this address is no longer a physical address and would need to be 
> translated. EEMI calls always get forwarded to Xen first,
>       then Xen issues
>       > a corresponding EEMI call to the firmware (see 
> xen/arch/arm/platforms/xilinx-eemi.c:xilinx_eemi). But Xen is probably passing
>       the EEMI
>       > calls parameters unmodified. Then PMU tries to read the address but 
> since this is not a physical address, it fails. Basically
>       we need to
>       > add code to Xen xen/arch/arm/platforms/xilinx-eemi.c:xilinx_eemi to 
> translate any guest physical addresses passed as EEMI
>       calls arguments
>       > into physical addresses before making the EEMI call to firmware.
>       >
>       > This is an example patch, which is translating the parameter on 
> register x2 for the EEMI call 0xC200002F. I haven't checked
>       the EEMI
>       > protocol for this call but this just an example to show you how to 
> translate parameters.
>       >
>       > diff --git a/xen/arch/arm/platforms/xilinx-eemi.c 
> b/xen/arch/arm/platforms/xilinx-eemi.c index 500c86dc69..bff1b71196 100644
>       ---
>       > a/xen/arch/arm/platforms/xilinx-eemi.c +++ 
> b/xen/arch/arm/platforms/xilinx-eemi.c @@ -409,6 +409,30 @@ bool
>       xilinx_eemi(struct
>       > cpu_user_regs *regs, const uint32_t fid, } goto forward_to_fw;
>       >
>       >  *  case 0xC200002F:
>       >  *  {
>       >  *  uint64_t example_possible_address_param = get_user_reg(regs, 2);
>       >  *  uint64_t translated_address = 
> mfn_to_maddr(gfn_to_mfn(current->domain,
>       >  *  gaddr_to_gfn(example_possible_address_param)));
>       >  *  translated_address += example_possible_address_param & 
> ~PAGE_MASK; +
>       >  *  arm_smccc_1_1_smc(get_user_reg(regs, 0),
>       >  *  get_user_reg(regs, 1),
>       >  *  translated_address,
>       >  *  get_user_reg(regs, 3),
>       >  *  get_user_reg(regs, 4),
>       >  *  get_user_reg(regs, 5),
>       >  *  get_user_reg(regs, 6),
>       >  *  get_user_reg(regs, 7),
>       >  *  &res); +
>       >  *  set_user_reg(regs, 0, res.a0);
>       >  *  set_user_reg(regs, 1, res.a1);
>       >  *  set_user_reg(regs, 2, res.a2);
>       >  *  set_user_reg(regs, 3, res.a3);
>       >  *  return true;
>       >  *  }
>       >
>       > + default: if ( is_hardware_domain(current->domain) ) goto 
> forward_to_fw;
>       >
>       > The aes request structure contains physical addresses of the source 
> and destination.
>       > These addresses are obtained via two calls dma_alloc_coherent.
>       > The address of this structure is kept at x2 register.
> 
>       OK I see the question is about the Xilinx Xen tree. In the logs below we
>       have:
> 
>       > (XEN) d0v1 Forwarding AES operation: 3254779951 r2 0 -> 11432000      
>                                                   log
>       from xen
> 
>       So it looks Linux passed 0 as address in x2, which cannot be right?
> 
> 
> 
>       > I applied the suggested scheme in xen for xilinx_eemi(...) function.
>       >
>       > case 0xC200002F:
>       > {
>       > uint64_t paramaddr = get_user_reg(regs, 2);
> 
>       It would seem that this is not read correctly? It should not be zero.
> 
> 
>       > uint64_t phyaddr = mfn_to_maddr(gfn_to_mfn(current->domain, 
> gaddr_to_gfn(paramaddr)));
>       > phyaddr += (paramaddr & ~PAGE_MASK);
>       > gprintk(XENLOG_DEBUG, "Forwarding AES operation: %u r2 %lx -> %lx\n", 
> fid, paramaddr, phyaddr);
>       > set_user_reg(regs, 2, phyaddr);
>       > }
>       > goto forward_to_fw;
>       >
>       > As a result I got the same issue as earlier.
>       >
>       > [   17.350086]
>       >
>       zynq_aes_gcm                                                            
>                                                            
>       user
>       > log
>       >
>       > [   17.350202] @ dma_alloc firmware:zynqmp-firmware:zynqmp-aes @      
>                                                  kernel
>       log from Dom0
>       > [   17.353015] @@@ firmware:zynqmp-firmware:zynqmp-aes 0 @@@
>       > [   17.358515] zynqmp_aes [0] ffffffc00910d000 2806000 
> firmware:zynqmp-firmware:zynqmp-aes
>       > [   17.366546] @ dma_alloc firmware:zynqmp-firmware:zynqmp-aes @
>       > [   17.372347] @@@ firmware:zynqmp-firmware:zynqmp-aes 0 @@@
>       > [   17.377775] zynqmp_aes [1] ffffffc009115000 42a14000 keytype 1
>       > [   17.383660] zynqmp_aes [2] dump request align 1 ++
>       > [   17.388501] 00 60 80 02 00 00 00 00
>       > [   17.392032] 50 60 80 02 00 00 00 00
>       > [   17.395583] 00 00 00 00 00 00 00 00
>       > [   17.399117] 00 60 80 02 00 00 00 00
>       > [   17.402664] 40 00 00 00 00 00 00 00
>       > [   17.406226] 00 00 00 00 00 00 00 00
>       > [   17.409755] 01 00 00 00 00 00 00 00
>       > [   17.413311] zynqmp_aes [3] dump request --
>       >
>       > (XEN) d0v1 Forwarding AES operation: 3254779951 r2 0 -> 11432000      
>                                                   log
>       from xen
> 
>       Here
> 
> 
>       > @ 000042A14000
>       >
>       @                                                                       
>                                                               
>       csu
>       > log from aes
>       > 04 E4 00 6F 05 E4 00 6F
>       > 06 E4 00 6F 07 E4 00 6F
>       > 10 E4 00 6F 11 E4 00 6F
>       > 12 E4 00 6F 13 E4 00 6F
>       > 14 E4 00 6F 15 E4 00 6F
>       > 16 E4 00 6F 17 E4 00 6F
>       > 18 E4 00 6F 19 E4 00 6F
>       >
>       > ERROR:   pm_aes_engine ### args 6 ret 0 addr 0 42a14000 ###           
>                                                     
>       ATF log
>       >
>       > So the address of the structure was not changed.
>       > This is the question. 
>       > How can I map this address to xen and change physical addresses there 
> ?
> 
> 
>