Re: [Bug] Bring up Dom0 on Arm board

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Tue, 31 Jan 2023, 蔡力刚 wrote:
> Hi,
> 
> > I think we have two problems here.
> > 
> > One problem, which is a known problem, is that sometimes the kernel can
> > make firmware calls (as SMC calls) to initialize a device driver. This
> > is what the "forward_smc" suggestion was meant to solve.
> > 
> > "forward_smc" is an effective workaround, but the proper solution would
> > be to write a platform driver like zynqmp_eemi which check which SMC
> > calls need to be forwarded and forward only those.
> > 
> > The second problem is that the kernel is making firmware calls using HVC
> > as transport instead of SMC. This is uncommon. That is the reason why
> > "forward_smc" didn't work. "forward_smc" only forward SMC calls, not HVC
> > calls.
> > 
> > But if you had a platform driver like zynqmp_eemi, in theory the
> > platform_smc() call in vsmccc_handle_call should have worked correctly
> > for both SMC and HVC coming from Linux.
> > 
> > Just to see if we understood the problem correctly, the appended patch
> > alone (no need for other changes) should work, if you pass
> > forward_firmware=true to the Xen command line.
> > 
> > 
> > diff --git a/xen/arch/arm/vsmc.c b/xen/arch/arm/vsmc.c
> > index 7335276f3f..1d11634bff 100644
> > --- a/xen/arch/arm/vsmc.c
> > +++ b/xen/arch/arm/vsmc.c
> > @@ -8,6 +8,7 @@
> >  
> >  
> >  #include <xen/lib.h>
> > +#include <xen/param.h>
> >  #include <xen/types.h>
> >  #include <public/arch-arm/smccc.h>
> >  #include <asm/cpuerrata.h>
> > @@ -26,6 +27,9 @@
> >  /* Number of functions currently supported by Standard Service Service 
> >Calls. */
> >  #define SSSC_SMCCC_FUNCTION_COUNT (3 + VPSCI_NR_FUNCS)
> >  
> > +static bool __read_mostly forward_fw = false;
> > +boolean_param("forward_firmware", forward_fw);
> > +
> >  static bool fill_uid(struct cpu_user_regs *regs, xen_uuid_t uuid)
> >  {
> >      int n;
> > @@ -224,6 +228,27 @@ static bool vsmccc_handle_call(struct cpu_user_regs 
> >*regs)
> >      const union hsr hsr = { .bits = regs->hsr };
> >      uint32_t funcid = get_user_reg(regs, 0);
> >  
> > +    if ( forward_fw )
> > +    {
> > +        struct arm_smccc_res res;
> > +
> > +        arm_smccc_1_1_smc(get_user_reg(regs, 0),
> > +                          get_user_reg(regs, 1),
> > +                          get_user_reg(regs, 2),
> > +                          get_user_reg(regs, 3),
> > +                          get_user_reg(regs, 4),
> > +                          get_user_reg(regs, 5),
> > +                          get_user_reg(regs, 6),
> > +                          get_user_reg(regs, 7),
> > +                          &res);
> > +
> > +        set_user_reg(regs, 0, res.a0);
> > +        set_user_reg(regs, 1, res.a1);
> > +        set_user_reg(regs, 2, res.a2);
> > +        set_user_reg(regs, 3, res.a3);
> > +        return true;
> > +    }
> > +
> 
> I tried the change above, but still has a fault in kernel, log as below:
> The change is to pass all HVC and SMC calls forward to firmware,
> but what I have tried successfully is passing all HVC calls and not handling 
> all SMC calls.

I think you should try to find out which ones are the HVCs/SMCs that
need to be forwarded and which ones that need to be handled as usual.

Once you know that, you could write a platform driver like
xen/arch/arm/platforms/xilinx-zynqmp.c that handles things
appropriately.


> I have not idea how to find out 0x1000a0 where comes from, do you have any 
> suggestions?

I would look at device tree to see within which range 0x1000a0 falls.


> (XEN) d0v0 hsr.ec 0x17
> (XEN) d0v0 do_trap_smc trap....
> (XEN) traps.c:1987:d0v0 HSR=0x92000007 pc=0xffffffc01069b830 
> gva=0xffffffc0126e30a0 gpa=0x000000001000a0
> [    6.027388] Unhandled fault at 0xffffffc0126e30a0
> [    6.027415] Mem abort info:
> [    6.027490]   ESR = 0x96000000
> [    6.027514]   EC = 0x25: DABT (current EL), IL = 32 bits
> [    6.027537]   SET = 0, FnV = 0
> [    6.027554]   EA = 0, S1PTW = 0
> [    6.027571] Data abort info:
> [    6.027589]   ISV = 0, ISS = 0x00000000
> [    6.027607]   CM = 0, WnR = 0
> [    6.027627] swapper pgtable: 4k pages, 39-bit VAs, pgdp=0000000019484000
> [    6.027652] [ffffffc0126e30a0] pgd=000000013ffff003, p4d=000000013ffff003, 
> pud=000000013ffff003, pmd=0000000101d86003, pte=006800000010
> 0717
> [    6.027714] Internal error: ttbr address size fault: 96000000 [#1] SMP
> [    6.027740] Modules linked in:
> [    6.027766] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.10.66 #8
> [    6.027790] Hardware name: Rockchip RK3588 EVB7 LP4 V10 Board (DT)
> [    6.027819] pstate: 60c00005 (nZCv daif +PAN +UAO -TCO BTYPE=--)
> [    6.027850] pc : rockchip_gem_get_ddr_info+0x28/0x40
> [    6.027884] lr : rockchip_gem_get_ddr_info+0x1c/0x40
> [    6.027905] sp : ffffffc011e7bd60
> [    6.027923] x29: ffffffc011e7bd60 x28: 0000000000000000 
> [    6.027952] x27: ffffffc011490438 x26: ffffffc0114f1070 
> [    6.027981] x25: 0000000000000006 x24: ffffffc0115bf598 
> [    6.028010] x23: 0000000000000000 x22: ffffffc011d56a40 
> [    6.028039] x21: ffffffc011e34000 x20: 0000000000000000 
> [    6.028067] x19: ffffffc011e34c30 x18: 0000000000000000 
> [    6.028095] x17: 000000000000000e x16: 0000000000000007 
> [    6.028124] x15: 000000000000000a x14: 0000000000000662 
> [    6.028152] x13: ffffffffffffffff x12: ffffffffffffffff 
> [    6.028181] x11: 0000000000000000 x10: ffffff8102730a1c 
> [    6.028209] x9 : ffffffc010b26fe0 x8 : ffffffc011e7bd38 
> [    6.028237] x7 : 0000000000000000 x6 : 0000000000000000 
> [    6.028266] x5 : 0000000000000000 x4 : 0000000000000000 
> [    6.028297] x3 : 0000000000000000 x2 : ffffffc011c99980 
> [    6.028326] x1 : ffffffc011c99000 x0 : ffffffc0126e3000 
> [    6.028355] Call trace:
> [    6.028377]  rockchip_gem_get_ddr_info+0x28/0x40
> [    6.028404]  rockchip_drm_init+0x110/0x114
> [    6.028427]  do_one_initcall+0xa0/0x1e8
> [    6.028450]  kernel_init_freeable+0x2a4/0x2ac
> [    6.028475]  kernel_init+0x20/0x11c
> [    6.028496]  ret_from_fork+0x10/0x30
> [    6.028516] 
> [    6.028516] PC: 0xffffffc01069b730:
> [    6.028536] b730  aa0003f3 f9400400 f9402c14 f9412660 97eb302d f9412a60 
> 97f7be94 f9412a60
> [    6.028598] b750  97ec26a5 f9401280 d2800003 f9407662 f940c661 97f81021 
> a94153f3 a8c27bfd
> [    6.028657] b770  d50323bf d65f03c0 aa1e03e9 d503201f d503233f a9bd7bfd 
> d2804c02 910003fd
> [    6.028716] b790  f90013f5 aa0003f5 d0006b20 a90153f3 2a0103f4 f945f800 
> 5281b801 97ec22a5
> [    6.028775] b7b0  b4000280 aa0003f3 51000682 32002c42 aa0003e1 11000442 
> aa1503e0 97ff3263
> [    6.028835] b7d0  f9400a60 52819a41 72a00201 f9401000 f9401800 b9003001 
> aa1303e0 a94153f3
> [    6.028894] b7f0  f94013f5 a8c37bfd d50323bf d65f03c0 92800173 17fffff9 
> d503245f aa1e03e9
> [    6.028952] b810  d503201f d503233f a9bf7bfd 910003fd 941230e8 b40000c0 
> d000afe1 91260022
> [    6.029011] b830  29540003 b9098020 b9000443 a8c17bfd d50323bf d65f03c0 
> d503245f aa1e03e9
> [    6.029070] b850  d503201f d503233f a9be7bfd aa0103e2 910003fd a90153f3 
> aa0103f4 aa0003f3
> [    6.029128] b870  f9407401 97ff34bd 35000080 aa1403e1 aa1303e0 97fffda7 
> a94153f3 a8c27bfd
> [    6.029187] b890  d50323bf d65f03c0 d503245f aa1e03e9 d503201f d503233f 
> a9be7bfd 910003fd
> [    6.029255] b8b0  f9000bf3 aa0103f3 97ff34ec 350000a0 f9405660 f9004e7f 
> aa1303e1 97fffd95
> [    6.029314] b8d0  f9400bf3 a8c27bfd d50323bf d65f03c0 d503245f aa1e03e9 
> d503201f d503233f
> [    6.029372] b8f0  a9bc7bfd 910003fd a90153f3 2a0303f4 a9025bf5 12001c55 
> a90363f7 97ffff9b
> 
>