Re: [PATCH v2] xen/pci: Refactor PCI MSI interrupts related code

[Rahul Singh <Rahul.Singh@xxxxxxx>]

Hi Jan,

> On 21 Apr 2021, at 10:33 am, Jan Beulich <jbeulich@xxxxxxxx> wrote:
> 
> On 21.04.2021 11:15, Rahul Singh wrote:
>> Hi Roger,
>> 
>>> On 21 Apr 2021, at 9:16 am, Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
>>> 
>>> On Wed, Apr 21, 2021 at 08:07:08AM +0000, Rahul Singh wrote:
>>>> Hi Jan,
>>>> 
>>>>> On 20 Apr 2021, at 4:36 pm, Jan Beulich <jbeulich@xxxxxxxx> wrote:
>>>>> 
>>>>> On 20.04.2021 15:45, Rahul Singh wrote:
>>>>>>> On 19 Apr 2021, at 1:33 pm, Jan Beulich <jbeulich@xxxxxxxx> wrote:
>>>>>>> On 19.04.2021 13:54, Julien Grall wrote:
>>>>>>>> For the time being, I think move this code in x86 is a lot better than 
>>>>>>>> #ifdef or keep the code in common code.
>>>>>>> 
>>>>>>> Well, I would perhaps agree if it ended up being #ifdef CONFIG_X86.
>>>>>>> I would perhaps not agree if there was a new CONFIG_* which other
>>>>>>> (future) arch-es could select if desired.
>>>>>> 
>>>>>> I agree with Julien moving the code to x86 file as currently it is 
>>>>>> referenced only in x86 code
>>>>>> and as of now we are not sure how other architecture will implement the 
>>>>>> Interrupt remapping
>>>>>> (via IOMMU or any other means).  
>>>>>> 
>>>>>> Let me know if you are ok with moving the code to x86.
>>>>> 
>>>>> I can't answer this with "yes" or "no" without knowing what the 
>>>>> alternative
>>>>> would be. As said, if the alternative is CONFIG_X86 #ifdef-ary, then yes.
>>>>> If a separate CONFIG_* gets introduced (and selected by X86), then a
>>>>> separate file (getting built only when that new setting is y) would seem
>>>>> better to me.
>>>> 
>>>> I just made a quick patch. Please let me know if below patch is ok. I move 
>>>> the definition to  "passthrough/x86/iommu.c” file.
>>>> 
>>>> diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
>>>> index 705137f8be..199ce08612 100644
>>>> --- a/xen/drivers/passthrough/pci.c
>>>> +++ b/xen/drivers/passthrough/pci.c
>>>> @@ -1303,13 +1303,6 @@ static int __init setup_dump_pcidevs(void)
>>>> }
>>>> __initcall(setup_dump_pcidevs);
>>>> 
>>>> -int iommu_update_ire_from_msi(
>>>> -    struct msi_desc *msi_desc, struct msi_msg *msg)
>>>> -{
>>>> -    return iommu_intremap
>>>> -           ? iommu_call(&iommu_ops, update_ire_from_msi, msi_desc, msg) : 
>>>> 0;
>>>> -}
>>>> -
>>>> static int iommu_add_device(struct pci_dev *pdev)
>>>> {
>>>>    const struct domain_iommu *hd;
>>>> diff --git a/xen/drivers/passthrough/x86/iommu.c 
>>>> b/xen/drivers/passthrough/x86/iommu.c
>>>> index b90bb31bfe..cf51dec564 100644
>>>> --- a/xen/drivers/passthrough/x86/iommu.c
>>>> +++ b/xen/drivers/passthrough/x86/iommu.c
>>>> @@ -340,6 +340,13 @@ bool arch_iommu_use_permitted(const struct domain *d)
>>>>            likely(!p2m_get_hostp2m(d)->global_logdirty));
>>>> }
>>>> 
>>>> +int iommu_update_ire_from_msi(
>>>> +    struct msi_desc *msi_desc, struct msi_msg *msg)
>>>> +{
>>>> +    return iommu_intremap
>>>> +           ? iommu_call(&iommu_ops, update_ire_from_msi, msi_desc, msg) : 
>>>> 0;
>>>> +}
>>>> +
>>>> /*
>>>> * Local variables:
>>>> * mode: C
>>>> diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
>>>> index ea0cd0f1a2..bd42d87b72 100644
>>>> --- a/xen/include/xen/iommu.h
>>>> +++ b/xen/include/xen/iommu.h
>>>> @@ -243,7 +243,6 @@ struct iommu_ops {
>>>>                           u8 devfn, device_t *dev);
>>>> #ifdef CONFIG_HAS_PCI
>>>>    int (*get_device_group_id)(u16 seg, u8 bus, u8 devfn);
>>>> -    int (*update_ire_from_msi)(struct msi_desc *msi_desc, struct msi_msg 
>>>> *msg);
>>>> #endif /* HAS_PCI */
>>>> 
>>>>    void (*teardown)(struct domain *d);
>>>> @@ -272,6 +271,7 @@ struct iommu_ops {
>>>>    int (*adjust_irq_affinities)(void);
>>>>    void (*sync_cache)(const void *addr, unsigned int size);
>>>>    void (*clear_root_pgtable)(struct domain *d);
>>>> +    int (*update_ire_from_msi)(struct msi_desc *msi_desc, struct msi_msg 
>>>> *msg);
>>> 
>>> You also need to move the function prototype
>>> (iommu_update_ire_from_msi) from iommu.h into asm-x86/iommu.h, or
>>> maybe you could just define the function itself as static inline in
>>> asm-x86/iommu.h?
>> 
>> 
>> Ok. I will move function prototype (iommu_update_ire_from_msi) from 
>> iommu.h into asm-x86/iommu.h.
>> 
>> I first tried to make the function as static inline in "asm-x86/iommu.h" but 
>> after 
>> that I observe the compilation error for debug build because "asm/iommu.h” 
>> is included in "xen/iommu.h” before iommu_call() is defined in "xen/iommu.h”.
>> That's why I decided to move it to the "passthrough/x86/iommu.c” file.
> 
> Which in turn would be an argument for keeping it in xen/iommu.h and
> wrap it in an #ifdef.
> 

Ok. I will move the definition of iommu_update_ire_from_msi() to 
"xen/iommu.h” and will wrap it under CONFIG_X86

If everyone is ok with the above approach then I will send the next
version of the patch series for review.

Regards,
Rahul

> Jan