Re: [PATCH v12 01/15] vpci: use per-domain PCI lock to protect vpci structure

[Stewart Hildebrand <stewart.hildebrand@xxxxxxx>]

On 1/15/24 03:53, Roger Pau Monné wrote:
> On Fri, Jan 12, 2024 at 12:54:56PM -0500, Stewart Hildebrand wrote:
>> On 1/12/24 08:48, Roger Pau Monné wrote:
>>> On Tue, Jan 09, 2024 at 04:51:16PM -0500, Stewart Hildebrand wrote:
>>>> @@ -202,8 +204,20 @@ static int __init apply_map(struct domain *d, const 
>>>> struct pci_dev *pdev,
>>>>      struct map_data data = { .d = d, .map = true };
>>>>      int rc;
>>>>  
>>>> +    ASSERT(rw_is_write_locked(&d->pci_lock));
>>>> +
>>>>      while ( (rc = rangeset_consume_ranges(mem, map_range, &data)) == 
>>>> -ERESTART )
>>>> +    {
>>>> +        /*
>>>> +         * It's safe to drop and reacquire the lock in this context
>>>> +         * without risking pdev disappearing because devices cannot be
>>>> +         * removed until the initial domain has been started.
>>>> +         */
>>>> +        write_unlock(&d->pci_lock);
>>>>          process_pending_softirqs();
>>>> +        write_lock(&d->pci_lock);
>>>
>>> Hm, I should have noticed before, but we already call
>>> process_pending_softirqs() with the pdev->vpci->lock held here, so it
>>> would make sense to drop it also.
>>
>> I don't quite understand this, maybe I'm missing something. I don't see 
>> where we acquire pdev->vpci->lock before calling process_pending_softirqs()?
>>
>> Also, I tried adding
>>
>>     ASSERT(!spin_is_locked(&pdev->vpci->lock));
>>
>> both here in apply_map() and in vpci_process_pending(), and they haven't 
>> triggered in either dom0 or domU test cases, tested on both arm and x86.
> 
> I think I was confused.  Are you sure that pdev->vpci->lock is taken
> in the apply_map() call?

I'm sure that it's NOT taken in apply_map(). See the ! in the test ASSERT above.

> I was mistakenly assuming that
> vpci_add_handlers() called the init function with the vpci->lock
> taken, but that doesn't seem to be case with the current code.  That
> leads to apply_map() also being called without the vpci->lock taken.

Right.

> 
> I was wrongly assuming that apply_map() was called with the vpci->lock
> lock taken, and that would need dropping around the
> process_pending_softirqs() call.
> 
> Thanks, Roger.