Re: [PATCH 8/9] vpci/header: Reset the command register when adding devices

[Oleksandr Andrushchenko <Oleksandr_Andrushchenko@xxxxxxxx>]

On 09.09.21 12:21, Jan Beulich wrote:
> On 09.09.2021 10:50, Oleksandr Andrushchenko wrote:
>> On 09.09.21 11:43, Jan Beulich wrote:
>>> On 09.09.2021 10:39, Oleksandr Andrushchenko wrote:
>>>> On 07.09.21 13:06, Jan Beulich wrote:
>>>>> On 07.09.2021 11:52, Oleksandr Andrushchenko wrote:
>>>>>> On 07.09.21 12:19, Jan Beulich wrote:
>>>>>>> On 07.09.2021 11:07, Oleksandr Andrushchenko wrote:
>>>>>>>> On 07.09.21 11:49, Jan Beulich wrote:
>>>>>>>>> On 07.09.2021 10:18, Oleksandr Andrushchenko wrote:
>>>>>>>>>> So, if we have a hidden PCI device which can be assigned to a guest 
>>>>>>>>>> and it is literally untouched
>>>>>>>>>> (not enabled in Dom0) then I think there will be no such reference 
>>>>>>>>>> as "host assigned values" as
>>>>>>>>>> most probably the command register will remain in its after reset 
>>>>>>>>>> state.
>>>>>>>>> What meaning of "hidden" do you imply here? Devices passed to
>>>>>>>>> pci_{hide,ro}_device() may not be assigned to guests ...
>>>>>>>> You are completely right here.
>>>>>>>>> For any other meaning of "hidden", even if the device is completely
>>>>>>>>> ignored by Dom0,
>>>>>>>> Dom0less is such a case when a device is assigned to the guest
>>>>>>>> without Dom0 at all?
>>>>>>> In this case it is entirely unclear to me what entity it is to have
>>>>>>> a global view on the PCI subsystem.
>>>>>>>
>>>>>>>>>       certain of the properties still cannot be allowed
>>>>>>>>> to be DomU-controlled.
>>>>>>>> The list is not that big, could you please name a few you think cannot
>>>>>>>> be controlled by a guest? I can think of PCI_COMMAND_SPECIAL(?),
>>>>>>>> PCI_COMMAND_INVALIDATE(?), PCI_COMMAND_PARITY, PCI_COMMAND_WAIT,
>>>>>>>> PCI_COMMAND_SERR, PCI_COMMAND_INTX_DISABLE which we may want to
>>>>>>>> be aligned with the "host reference" values, e.g. we only allow those 
>>>>>>>> bits
>>>>>>>> to be set as they are in Dom0.
>>>>>>> Well, you've compile a list already, and I did say so before as well:
>>>>>>> Everything except I/O and memory decoding as well as bus mastering
>>>>>>> needs at least closely looking at. INTX_DISABLE, for example, is
>>>>>>> something I don't think a guest should be able to directly control.
>>>>>>> It may still be the case that the host permits it control, but then
>>>>>>> only indirectly, allowing the host to appropriately adjust its
>>>>>>> internals.
>>>>>>>
>>>>>>> Note that even for I/O and memory decoding as well as bus mastering
>>>>>>> it may be necessary to limit guest control: In case the host wants
>>>>>>> to disable any of these (perhaps transiently) despite the guest
>>>>>>> wanting them enabled.
>>>>>> Ok, so it is now clear that we need a yet another patch to add a proper
>>>>>> command register emulation. What is your preference: drop the current
>>>>>> patch, implement command register emulation and add a "reset patch"
>>>>>> after that or we can have the patch as is now, but I'll only reset 
>>>>>> IO/mem and bus
>>>>>> master bits, e.g. read the real value, mask the wanted bits and write 
>>>>>> back?
>>>>> Either order is fine with me as long as the result will be claimed to
>>>>> be complete until proper emulation is in place.
>>>> I tried to see what others do in order to emulate PCI_COMMAND register
>>>> and it seems that at most they care about the only INTX bit (besides
>>>> IO/memory enable and bus muster which are write through). Please see
>>>> [1] and [2]. Probably I miss something, but it could be because in order
>>>> to properly emulate the COMMAND register we need to know about the
>>>> whole PCI topology, e.g. if any setting in device's command register
>>>> is aligned with the upstream port etc. This makes me think that because
>>>> of this complexity others just ignore that. Neither I think this can be
>>>> easily done in our case. So I would suggest we just add the following
>>>> simple logic to only emulate PCI_COMMAND_INTX_DISABLE: allow guest to
>>>> disable the interrupts, but don't allow to enable if host has disabled
>>>> them. This is also could be tricky a bit for the devices which are not
>>>> enabled and thus not configured in Dom0, e.g. we do not know for sure
>>>> if the value in the PCI_COMMAND register (in particular
>>>> PCI_COMMAND_INTX_DISABLE bit) can be used as the reference host value or
>>>> not. It can be that the value there is just the one after reset or so.
>>>> The rest of the command register bits will go directly to the command
>>>> register untouched.
>>>> So, at the end of the day the question is if PCI_COMMAND_INTX_DISABLE
>>>> is enough and how to get its reference host value.
>>> Well, in order for the whole thing to be security supported it needs to
>>> be explained for every bit why it is safe to allow the guest to drive it.
>> So, do we want at least PCI_COMMAND_INTX_DISABLE bit aligned
>> between the host and guest? If so, what do you you think about
>> the reference value for it (please see above).
> Please may I ask that you come up with a proposal? I don't think I've
> said you need to emulate this or any of the other bits. All I've asked
> for is that for every bit you allow the guest to control directly, you
> justify why that's safe and secure. If no justification can be given,
> emulation is going to be necessary. How to solve that is first and
> foremost part of your undertaking.

The thing here is that we can't truly justify if we can let the guest

control those bits or not as it all may depend on the topology that some

specific setup might have. Not that we technically can't, but not in a

practical and easy way IMO. Taking that into account we come to a

conclusion that we need to emulate those then. But, again we understand

that full emulation, if properly implemented, is going to  be a big piece of 
code

which needs to take into account the physical PCI topology etc.

So, this is my understanding why others do not implement that (QEMU, ARCN)

and let the guest control all bits, but INTxDISABLE (Disclaimer: if I 
understood their

code correctly)

So, my proposal here is to only emulate PCI_COMMAND_INTX_DISABLE and let

the other bits be controlled by the guest (please also see the note below).

I do understand this is not correct, but I can't tell how to deal with this 
other way.

>
> For the bit in question, where the goal appears to be to have hardware
> hold the OR of guest and host values, an approach similar to that used
> for some of the MSI / MSI-X bits might be chosen: Maintain guest and
> host bits in software, and update hardware (at least) when the
> effective resulting value changes. A complicating fact here is, though,
> that unlike for the MSI / MSI-X bits here Dom0 (pciback or its PCI
> susbstem) may also have a view on what the setting ought to be.

The bigger question here is what can we take as the reference for INTx

bit, e.g. if Dom0 didn't enable/configured the device being passed through

than its COMMAND register may still be in after reset state and IMO there is

no guarantee it has the values we can say are "as host wants them"

>
> Jan
>
Thank you,

Oleksandr