Re: [Xen-devel] [PATCH] KEXEC: disconnect all PCI devices from the PCI bus on crash

["Jan Beulich" <JBeulich@xxxxxxxxxx>]

>>> On 07.07.11 at 12:02, Andrew Cooper <andrew.cooper3@xxxxxxxxxx> wrote:

> 
> On 07/07/11 10:53, Ian Campbell wrote:
>> On Thu, 2011-07-07 at 10:42 +0100, Jan Beulich wrote:
>>>>>> On 07.07.11 at 11:12, Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx> wrote:
>>>> On Thu, 2011-07-07 at 10:10 +0100, Jan Beulich wrote:
>>>>>>>> On 07.07.11 at 10:53, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
>>>>>> On Wed, 2011-07-06 at 19:42 +0100, Konrad Rzeszutek Wilk wrote:
>>>>>>> On Wed, Jul 06, 2011 at 01:39:12PM +0100, Andrew Cooper wrote:
>>>>>>>> +/* Disconnect a PCI device from the PCI bus.  From the PCI spec:
>>>>>>>> + *     "When a 0 is written to [the COMMAND] register, the device is
>>>>>>>> + *     logically disconnected from the PCI bus for all accesses except
>>>>>>>> + *     configuration accesses. All devices are required to support
>>>>>>>> + *     this base level of functionality."
>>>>>>>> + */
>>>>>>>> +void disconnect_pci_device(struct pci_dev *pdev)
>>>>>>>> +{
>>>>>>>> +    pci_conf_write16(pdev->bus, PCI_SLOT(pdev->devfn),
>>>>>>>> +                     PCI_FUNC(pdev->devfn), PCI_COMMAND, 0);
>>>>>>> So if you have a PCI serial card (or Intel AMT) and you are using that 
>>>>>>> for
>>>>>>> serial output on the hypervisor line, this will turn it off. There 
>>>>>>> should
>>>>>>> be some whitelist capability to not do it for PCI serial devices that 
>>>>>>> are
>>>>>>> owned (used) by the hypervisor.
>>>>>> That would be useful for debugging the kexec process itself but in the
>>>>>> general case there won't be any further output from the hypervisor and
>>>>>> if the kexec'd kernel wants to use the device it is going to have to set
>>>>>> it up again anyways.
>>>>> No, not generally. Just look at Linux' early-printk code: The device
>>>>> is assumed to be enabled (by the BIOS), as the PCI subsystem can't
>>>>> possibly be initialized at this point already.
>>>> That's arguably a debugging facility as well though.
>>>>
>>>>> This also means that white-listing just devices Xen uses may not be
>>>>> enough: If Xen doesn't use a serial console (or the secondary kernel
>>>>> wants to use some other device Xen doesn't care about - VGA or
>>>>> other kind of console devices come to mind), it must not find it fully
>>>>> disconnected from the bus. Consequently I would think that while
>>>>> interrupt and DMA activity should be forced off, decoding I/O and
>>>>> memory addresses by the devices shouldn't be.
>>>> The problem is that this can't be done without device specific
>>>> knowledge, which the hypervisor generally doesn't have and we can't get
>>>> the device's owning domain to do anything because we are crashing.
>>> Why would there be any device specific knowledge needed? It's
>>> all done through the command word, just that writing zero isn't
>>> really appropriate.
>> So presumably if you disable bus mastering you've effectively disabled
>> DMA but how do you disable interrupts via the command word?
>>
>> Ian.
>>
> Bit 10 of the control word is "disable assertion of INTx# pins"  (so set
> bit to 1 to disable interrupts).  That should cover legacy interrupts. 
> For MSI and above, disabling DMA should prevent the bus writing to the
> magic local APIC addresses.

No, I don't thing the bus mastering bit disables MSI accesses (and
even if officially it did, I wouldn't trust it). But disabling MSI itself
isn't device specific either, so should be an option here (with the
caveat that there are a few buggy devices that can't get back
into MSI mode if it was once disabled, so perhaps masking MSI
when possible should be preferred over disabling it).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel