[Xen-devel] RE: Intel IOMMU: Assigning IGD device: 00:02.1 on some machines, causes Xen to freeze for >2mins because of RMRR mappings

["Han, Weidong" <weidong.han@xxxxxxxxx>]

Tom,

As I know, the 00:02.1 is a little bit legacy, it was used in Win2000 to 
enabled second display functionality. Recent intel platforms doesn't include 
00:02.1. Can you have a try with guest win2000?

Regards,
Weidong

-----Original Message-----
From: Han, Weidong 
Sent: Thursday, December 03, 2009 10:22 AM
To: 'Tom Rotenberg'; Keir Fraser
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: Intel IOMMU: Assigning IGD device: 00:02.1 on some machines, 
causes Xen to freeze for >2mins because of RMRR mappings

I think the issue is not related to preempted hypercall, because 00:02.0 has 
the same RMRR with 00:02.1, and its pass-through succeeds. One correction, the 
RMRR (0x7dc00000 to 0x80000000) is not 10,000 pages, is ~1000 pages.

Preempted hypercall implementation is a bit of complex. If you want to have a 
try, you can refer to existed preempted hypercalls (e.g. 
XENMEM_increase_reservation).

Regards,
Weidong


-----Original Message-----
From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx] 
Sent: Wednesday, December 02, 2009 6:58 PM
To: Han, Weidong; Keir Fraser
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: Intel IOMMU: Assigning IGD device: 00:02.1 on some machines, 
causes Xen to freeze for >2mins because of RMRR mappings

And, the other question: how can i make this hypercall preemptive? as
it may still take too much time for an hypercall (taking into account,
it needs to map ~10.000 pages...)

On Mon, Nov 30, 2009 at 4:54 PM, Han, Weidong <weidong.han@xxxxxxxxx> wrote:
> You also need to include below code in the loop:
>    iommu_flush_cache_entry(pte);
>    unmap_vtd_domain_page(page);
>
> At the end, need to flush iotlb for all pages:
>        if ( iommu_flush_iotlb_psi(iommu, domain_iommu_domid(d),
>                                   (paddr_t)gfn << PAGE_SHIFT_4K, pages, 
>  ----> "pages" means page number
>                                   !pte_present, flush_dev_iotlb) )
>            iommu_flush_write_buffer(iommu);
>
> Regards,
> Weidong
>
> -----Original Message-----
> From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx]
> Sent: Monday, November 30, 2009 7:17 PM
> To: Han, Weidong
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: Intel IOMMU: Assigning IGD device: 00:02.1 on some machines, 
> causes Xen to freeze for >2mins because of RMRR mappings
>
> Can u please guid me a little bit, on how to write such function which
> will work on a batch of pages?
> Should i just enclose the:
> "
> ...
>    pg_maddr = addr_to_dma_page_maddr(d, (paddr_t)gfn << PAGE_SHIFT_4K, 1);
>    if ( pg_maddr == 0 )
>    {
>        spin_unlock(&hd->mapping_lock);
>        return -ENOMEM;
>    }
>    page = (struct dma_pte *)map_vtd_domain_page(pg_maddr);
>    pte = page + (gfn & LEVEL_MASK);
>    pte_present = dma_pte_present(*pte);
>    dma_set_pte_addr(*pte, (paddr_t)mfn << PAGE_SHIFT_4K);
>    dma_set_pte_prot(*pte, DMA_PTE_READ | DMA_PTE_WRITE);
>
>    /* Set the SNP on leaf page table if Snoop Control available */
>    if ( iommu_snoop )
>        dma_set_pte_snp(*pte);
> ...
> "
>
> in a loop?
>
> On Mon, Nov 30, 2009 at 4:30 AM, Han, Weidong <weidong.han@xxxxxxxxx> wrote:
>> Tom,
>>
>> Yes, you can add an API to map batches of pages instead of mapping each page 
>> in a separate call.
>>
>> RMRR regions of USB controller is ignored because they won't be used in 
>> guest. The RMRR 0x7dc00000 to 0x80000000 is shared by 00:02.0 and 00:02.1, 
>> it will be used by IGD in guest. But you can have a try to ignore it for 
>> 00:02.1.
>>
>>
>> Regards,
>> Weidong
>>
>> -----Original Message-----
>> From: Tom Rotenberg [mailto:tom.rotenberg@xxxxxxxxx]
>> Sent: Monday, November 30, 2009 1:56 AM
>> To: xen-devel@xxxxxxxxxxxxxxxxxxx
>> Cc: Han, Weidong
>> Subject: Intel IOMMU: Assigning IGD device: 00:02.1 on some machines, causes 
>> Xen to freeze for >2mins because of RMRR mappings
>>
>> Hi All,
>>
>> I am using a Dell E6400 machine, with IGD (Intel graphics device),
>> which i'm trying to pass-through into a domU with Windows XP. When i
>> assign only the 00:02.0 device, it seems to work ok, but when i try to
>> assign the 00:02.1 device, the whole machine seems to freeze form more
>> then 2 mins, causing a lot of messgaes in dom0, such as: SATA doesn't
>> respond, etc.
>>
>> After some analysis i made, i discovered the followings:
>> * When calling the (Intel) iommu_assign_device, it checks if the
>> device has any details in the RMRR table, and if so it calls the
>> 'iommu_prepare_rmrr_dev()' function.
>> * The device 00:02.1 has an RMRR information, which details the whole
>> area between:  0x7dc00000 to 0x80000000  (it contains ~9000 pages)
>> * For each such page, Xen calls the 'intel_iommu_map_page()' function.
>> This causes the hypercall to take a lot of time to process, and this
>> makes dom0 to stop getting responds from devices, such as SATA, etc,,
>> which may lead to a whole system  crash.
>>
>> My question are:
>> 1. Can this hypercall be in some way preemptive, so it won't stuck the
>> whole system (i noticed that there are other hypercalls, which are
>> preemptive) ?
>> 2. Why can't the mapping of the pages, be done in batches of pages,
>> instead of mapping each page in a separate call to
>> 'intel_iommu_map_page()' ?
>> 3. i noticed that for USB devices, the code there ignores the USB
>> RMRR... can this be done also for the 00:02.1 device?
>>
>> Tom
>>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel