Re: [Xen-devel] Re: Linux Stubdom Problem

[Jiageng Yu <yujiageng734@xxxxxxxxx>]

2011/9/15 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
> On Wed, 14 Sep 2011, Jiageng Yu wrote:
>> Hi Stefano,
>>
>> Â Â ÂI just have a prototype of vram mapping and test it now. The
>> implementation of linux-stubdom kernel part is as follows.
>> xen_remap_domain_mfn_range2 function maps foreign dom's physical
>> address into linux kernel space. It is similar to
>> xen_remap_domain_mfn_range. But xen_remap_domain_mfn_range is used to
>> map foreign pages into linux user space.
>>
>> Â Â But the page info seems wrong after executing 
>> xen_remap_domain_mfn_range2.
>>
>> Â Â struct page *page=pfn_to_page(vmalloc_to_pfn(info->fb));
>>
>> Â Â The page->_count = 0xc2c2c2c2. It is very strange.
>>
>> Â Â Did I do the right thing?
>>
>
> use page_address instead of pfn_to_page to find the struct page
>
>
>> Â Â Greeting.
>>
>> Jiageng Yu.
>>
>>
>> diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
>> index 204e3ba..72a7808 100644
>> --- a/arch/x86/xen/mmu.c
>> +++ b/arch/x86/xen/mmu.c
>> @@ -2693,6 +2693,73 @@ out:
>> Â}
>> ÂEXPORT_SYMBOL_GPL(xen_remap_domain_mfn_range);
>>
>> +int xen_remap_domain_mfn_range2(unsigned long addr,unsigned long gpfn,
>> + Â Â Â Â Â Â Â Âint nr, unsigned domid)
>> +{
>> + Â Âstruct remap_data rmd;
>> + Â Âstruct mmu_update mmu_update[REMAP_BATCH_SIZE];
>> + Â Âint level,i,batch,nr_page = nr;
>> + Â Âunsigned long range;
>> + Â Âint err = 0;
>> + Â Âunsigned long vaddr,base_addr = addr;
>> + Â Âpte_t pte,*ptep;
>> +
>> + Â Ârmd.mfn = gpfn;
>> + Â Ârmd.prot = __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED |
>> _PAGE_IOMAP);
>> +
>> + Â Âwhile(nr_page) {
>> + Â Â Â Âbatch = min(REMAP_BATCH_SIZE, nr);
>> + Â Â Â Ârange = (unsigned long)batch << PAGE_SHIFT;
>> +
>> + Â Â Â Ârmd.mmu_update = mmu_update;
>> +
>> + Â Â Â Âfor(i=0; i < batch; i++){
>> + Â Â Â Â Â Âpte = pte_mkspecial(pfn_pte(rmd.mfn++, rmd.prot));
>> + Â Â Â Â Â Âvaddr = base_addr + i*PAGE_SIZE;
>> + Â Â Â Â Â Âptep = lookup_address(vaddr, &level);
>
> you need to check if ptep is valid here and the level is PG_LEVEL_4K
>
>> + Â Â Â Â Â Ârmd.mmu_update->ptr = arbitrary_virt_to_machine(ptep).maddr |
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂMMU_NORMAL_PT_UPDATE;
>
> you can use pte_mfn(*ptep) instead of arbitrary_virt_to_machine
>
>
>> + Â Â Â Â Â Ârmd.mmu_update->val = pte_val_ma(pte);
>> + Â Â Â Â Â Ârmd.mmu_update++;
>> + Â Â Â Â}
>> +
>> + Â Â Â Âerr = -EFAULT;
>> + Â Â Â Âif(HYPERVISOR_mmu_update(mmu_update, batch, NULL, domid) < 0)
>> + Â Â Â Â Â Âgoto out;
>> +
>> + Â Â Â Ânr_page -= batch;
>> + Â Â Â Âbase_addr += range;
>> + Â Â}
>> +
>> + Â Âerr = 0;
>> +
>> + Â Âbase_addr = addr;
>> + Â Âfor(i=0; i < nr; i++){
>> + Â Â Â Âvaddr = base_addr + i*PAGE_SIZE;
>> + Â Â Â Âset_phys_to_machine(vmalloc_to_pfn(vaddr),
>> + Â Â Â Â Â Â Â Âarbitrary_virt_to_machine(vaddr).maddr >> PAGE_SHIFT);
>> + Â Â}
>
> The second argument (mfn) to set_phys_to_machine is wrong:
> arbitrary_virt_to_machine ends up calling virt_to_machine if
> virt_addr_valid. You need to manually call pte_mfn:
>
> /* the ptep content has been updated by Xen so we can lookup the foreign
> Â* mfn from the pte now */
> pte = lookup_address(vaddr, &level);
> BUG_ON(pte == NULL);
> offset = vaddr & ~PAGE_MASK;
> mfn = XMADDR(((phys_addr_t)pte_mfn(*pte) << PAGE_SHIFT) + offset);
>
>
>> +
>> +out:
>> + Â Â flush_tlb_all();
>> + Â Â return err;
>> +}
>> +EXPORT_SYMBOL_GPL(xen_remap_domain_mfn_range2);
>
> the name should be changed to xen_remap_foreign_gpfn_range
>
>
>> Â#ifdef CONFIG_XEN_PVHVM
>> Âstatic void xen_hvm_exit_mmap(struct mm_struct *mm)
>> Â{
>> diff --git a/drivers/video/xen-fbfront.c b/drivers/video/xen-fbfront.c
>> index dc72563..82da2ee 100644
>> --- a/drivers/video/xen-fbfront.c
>> +++ b/drivers/video/xen-fbfront.c
>> @@ -25,8 +25,12 @@
>> Â#include <linux/module.h>
>> Â#include <linux/vmalloc.h>
>> Â#include <linux/mm.h>
>> +#include <linux/sched.h>
>> +#include <asm/pgtable.h>
>> +#include <asm/page.h>
>>
>> Â#include <asm/xen/hypervisor.h>
>> +#include <asm/xen/page.h>
>>
>> Â#include <xen/xen.h>
>> Â#include <xen/events.h>
>> @@ -34,6 +38,7 @@
>> Â#include <xen/interface/io/fbif.h>
>> Â#include <xen/interface/io/protocols.h>
>> Â#include <xen/xenbus.h>
>> +#include <xen/xen-ops.h>
>>
>> Âstruct xenfb_info {
>>    unsigned char      *fb;
>> @@ -62,6 +67,12 @@ module_param_array(video, int, NULL, 0);
>> ÂMODULE_PARM_DESC(video,
>> Â Â Â "Video memory size in MB, width, height in pixels (default 
>> 2,800,600)");
>>
>> +static unsigned long foreign_vaddr = 0;
>> +module_param(foreign_vaddr, ulong, S_IRUGO);
>> +
>> +static unsigned long foreign_domid = 0;
>> +module_param(foreign_domid, ulong, S_IRUGO);
>> +
>> Âstatic void xenfb_make_preferred_console(void);
>> Âstatic int xenfb_remove(struct xenbus_device *);
>> Âstatic void xenfb_init_shared_page(struct xenfb_info *, struct fb_info *);
>> @@ -398,7 +408,17 @@ static int __devinit xenfb_probe(struct xenbus_device 
>> *dev,
>> Â Â Â if (info->fb == NULL)
>> Â Â Â Â Â Â Â goto error_nomem;
>> Â Â Â memset(info->fb, 0, fb_size);
>> -
>> + Â Âif((foreign_vaddr != 0) && (foreign_domid != 0)){
>> + Â Â Â Âret = xen_remap_domain_mfn_range2((unsigned long)(info->fb),
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Âforeign_vaddr >> PAGE_SHIFT,
>> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Âfb_size >> PAGE_SHIFT, foreign_domid);
>
> you should rename foreign_vaddr to foreign_gpfn and pass the gpfn value
> that is the ram_addr (page shifted) passed to xen_ram_alloc in qemu.
>
>> + Â Â Â Âif(ret < 0){
>> + Â Â Â Â Â Âprintk("Can not remap vram of hvm guest.\n");
>> + Â Â Â Â Â Âgoto error;
>> + Â Â Â Â}
>> + Â Â}
>> Â Â Â info->nr_pages = (fb_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
>>
>> Â Â Â info->mfns = vmalloc(sizeof(unsigned long) * info->nr_pages);
>> diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h
>> index 4349e89..1554531 100644
>> --- a/include/xen/xen-ops.h
>> +++ b/include/xen/xen-ops.h
>> @@ -20,6 +20,10 @@ int xen_remap_domain_mfn_range(struct vm_area_struct *vma,
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Âunsigned long mfn, int nr,
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Âpgprot_t prot, unsigned domid);
>>
>> +int xen_remap_domain_mfn_range2(unsigned long addr,unsigned long mfn,
>> + Â Â Â Â Â Â Â Âint nr, unsigned domid);
>> Âextern unsigned long *xen_contiguous_bitmap;
>> Âint xen_create_contiguous_region(unsigned long vstart, unsigned int order,
>> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â unsigned int address_bits);
>>
>


Hi Stefano,

        I have some progress in linux based stubdom project. As shown in the
attached video, I have started the emulated vga device in the linux
based stubdom.

        In a short conclusion, for the linux based stubdom, there are two
major problems about vga device emulation. The first is the vram
mapping, which we discussed a lot previously and handled it. Another
is the vga BIOS mapping (address 0xc0000-0xc8fff of hvm guest).

        I found the vga BIOS mapping problem in remap_area_mfn_pte_fn()
function. The pte_mkspecial() will return invalid value when I try to
map 0xc0000-0xc8fff into linux based stubdom.

pte_mkspecial()
        ->pte_set_flags()
                ->native_pte_val()
                ->native_make_pte()

        According to my test, the root cause of vga BIOS mapping problem is
native_xxx functions. We could avoid the problem by invoking functions
defined in paravirt.h instead. The patch is as follows. But I think it
is not a good way to handle the problem. Maybe you can give me some
suggestions.

        I also found the hard disk didnât work well. I will investigate it 
these days.


--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -2639,12 +2640,16 @@ static int remap_area_mfn_pte_fn(pte_t *ptep,
pgtable_t token,
                                 unsigned long addr, void *data)
 {
        struct remap_data *rmd = data;
-       pte_t pte = pte_mkspecial(pfn_pte(rmd->mfn++, rmd->prot));
+    if((rmd->mfn & 0xfffffff0) == 0xc0){
+           pte_t pte = pfn_pte(rmd->mfn++, rmd->prot);
+           rmd->mmu_update->val = pte_val(pte);
+    }else{
+           pte_t pte = pte_mkspecial(pfn_pte(rmd->mfn++, rmd->prot));
+           rmd->mmu_update->val = pte_val_ma(pte);
+    }

        rmd->mmu_update->ptr = arbitrary_virt_to_machine(ptep).maddr;
-       rmd->mmu_update->val = pte_val_ma(pte);
        rmd->mmu_update++;
-
        return 0;
 }

Attachment: out.ogv
Description: video/ogg

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel