[Xen-devel] Re: Linux Stubdom Problem

[Jiageng Yu <yujiageng734@xxxxxxxxx>]

2011/8/13 Jiageng Yu <yujiageng734@xxxxxxxxx>:
> 2011/7/29 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>:
>> On Fri, 29 Jul 2011, Jiageng Yu wrote:
>>> 2011/7/29 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
>>> On Fri, 29 Jul 2011, Jiageng Yu wrote:
>>> > 2011/7/29 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
>>> > On Fri, 29 Jul 2011, Jiageng Yu wrote:
>>> > > I have noticed the mistake. In fact, we shall stop the 
>>> > > map_foreign_pages of xen_console and xenfb devices in
>>> qemu.
>>> > Because
>>> > > the front drivers in stubdom has already map the memories. This is my 
>>> > > new patch. But it is not stable, I am
>>> > testing it.
>>> > >
>>> > > We use xc_handle to map foreign pages in xenfb and xen_console devices. 
>>> > > If qemu running on stubdom, the
>>> xc_handle
>>> > is
>>> > > invalid.
>>> > >
>>> > >
>>> > > diff --git a/hw/xen_backend.c b/hw/xen_backend.c
>>> > > index cfb53c8..11c53fe 100644
>>> > > --- a/hw/xen_backend.c
>>> > > +++ b/hw/xen_backend.c
>>> > > @@ -47,6 +47,7 @@ XenXC xen_xc = XC_HANDLER_INITIAL_VALUE;
>>> > > ÂXenGnttab xen_xcg = XC_HANDLER_INITIAL_VALUE;
>>> > > Âstruct xs_handle *xenstore = NULL;
>>> > > Âconst char *xen_protocol;
>>> > > +XenXC xc_handle = XC_HANDLER_INITIAL_VALUE;
>>> > >
>>> > > Â/* private */
>>> > > Âstatic QTAILQ_HEAD(XenDeviceHead, XenDevice) xendevs = 
>>> > > QTAILQ_HEAD_INITIALIZER(xendevs);
>>> > > @@ -655,6 +656,7 @@ static void xen_be_evtchn_event(void *opaque)
>>> > >
>>> > > Âint xen_be_init(void)
>>> > > Â{
>>> > > +#ifndef CONFIG_STUBDOM
>>> > > ÂÂÂÂ xenstore = xs_daemon_open();
>>> > > ÂÂÂÂ if (!xenstore) {
>>> > > ÂÂÂÂÂÂÂÂ xen_be_printf(NULL, 0, "can't connect to xenstored\n");
>>> > > @@ -665,10 +667,16 @@ int xen_be_init(void)
>>> > > ÂÂÂÂÂÂÂÂ goto err;
>>> > > ÂÂÂÂ }
>>> > >
>>> > > +ÂÂÂ if (xc_handle == XC_HANDLER_INITIAL_VALUE) {
>>> > > +ÂÂÂÂÂÂÂ goto err;
>>> > > +ÂÂÂ }
>>> > > +#endif
>>> > > +
>>> > > ÂÂÂÂ if (xen_xc == XC_HANDLER_INITIAL_VALUE) {
>>> > > ÂÂÂÂÂÂÂÂ /* Check if xen_init() have been called */
>>> > > ÂÂÂÂÂÂÂÂ goto err;
>>> > > ÂÂÂÂ }
>>> > > +
>>> > > ÂÂÂÂ return 0;
>>> > >
>>> > > Âerr:
>>> > > diff --git a/hw/xen_backend.h b/hw/xen_backend.h
>>> > > index 9d36df3..bc5a157 100644
>>> > > --- a/hw/xen_backend.h
>>> > > +++ b/hw/xen_backend.h
>>> > > @@ -59,6 +59,9 @@ extern XenXC xen_xc;
>>> > > Âextern struct xs_handle *xenstore;
>>> > > Âextern const char *xen_protocol;
>>> > >
>>> > > +/* invalid in linux stubdom */
>>> > > +extern XenXC xc_handle;
>>> > > +
>>> > > Â/* xenstore helper functions */
>>> > > Âint xenstore_write_str(const char *base, const char *node, const char 
>>> > > *val);
>>> > > Âint xenstore_write_int(const char *base, const char *node, int ival);
>>> > > diff --git a/hw/xen_console.c b/hw/xen_console.c
>>> > > index c6c8163..66b6dd7 100644
>>> > > --- a/hw/xen_console.c
>>> > > +++ b/hw/xen_console.c
>>> > > @@ -213,7 +213,7 @@ static int con_connect(struct XenDevice *xendev)
>>> > > ÂÂÂÂ if (xenstore_read_int(con->console, "limit", &limit) == 0)
>>> > > ÂÂcon->buffer.max_capacity = limit;
>>> > >
>>> > > -ÂÂÂ con->sring = xc_map_foreign_range(xen_xc, con->xendev.dom,
>>> > > +ÂÂÂcon->sring = xc_map_foreign_range(xc_handle, con->xendev.dom,
>>> > > ÂÂÂÂÂÂÂÂÂÂ XC_PAGE_SIZE,
>>> > > ÂÂÂÂÂÂÂÂÂÂ PROT_READ|PROT_WRITE,
>>> > > ÂÂÂÂÂÂÂÂÂÂ con->ring_ref);
>>> > > diff --git a/hw/xenfb.c b/hw/xenfb.c
>>> > > index 039076a..278fa60 100644
>>> > > --- a/hw/xenfb.c
>>> > > +++ b/hw/xenfb.c
>>> > > @@ -104,7 +104,7 @@ static int common_bind(struct common *c)
>>> > > ÂÂÂÂ if (xenstore_read_fe_int(&c->xendev, "event-channel", 
>>> > > &c->xendev.remote_port) == -1)
>>> > > ÂÂreturn -1;
>>> > >
>>> > > -ÂÂÂ c->page = xc_map_foreign_range(xen_xc, c->xendev.dom,
>>> > > +ÂÂÂc->page = xc_map_foreign_range(xc_handle, c->xendev.dom,
>>> > > ÂÂÂÂÂÂÂ XC_PAGE_SIZE,
>>> > > ÂÂÂÂÂÂÂ PROT_READ | PROT_WRITE, mfn);
>>> > > ÂÂÂÂ if (c->page == NULL)
>>> > > @@ -482,14 +482,14 @@ static int xenfb_map_fb(struct XenFB *xenfb)
>>> > > ÂÂÂÂ fbmfns = qemu_mallocz(sizeof(unsigned long) * xenfb->fbpages);
>>> > >
>>> > > ÂÂÂÂ xenfb_copy_mfns(mode, n_fbdirs, pgmfns, pd);
>>> > > -ÂÂÂ map = xc_map_foreign_pages(xen_xc, xenfb->c.xendev.dom,
>>> > > +ÂÂÂmap = xc_map_foreign_pages(xc_handle, xenfb->c.xendev.dom,
>>> > > ÂÂÂÂÂÂÂÂÂÂ PROT_READ, pgmfns, n_fbdirs);
>>> > > ÂÂÂÂ if (map == NULL)
>>> > > ÂÂgoto out;
>>> > > ÂÂÂÂ xenfb_copy_mfns(mode, xenfb->fbpages, fbmfns, map);
>>> > > ÂÂÂÂ munmap(map, n_fbdirs * XC_PAGE_SIZE);
>>> > >
>>> > > -ÂÂÂ xenfb->pixels = xc_map_foreign_pages(xen_xc, xenfb->c.xendev.dom,
>>> > > +ÂÂÂxenfb->pixels = xc_map_foreign_pages(xc_handle, xenfb->c.xendev.dom,
>>> > > ÂÂÂÂÂÂ PROT_READ | PROT_WRITE, fbmfns, xenfb->fbpages);
>>> > > ÂÂÂÂ if (xenfb->pixels == NULL)
>>> > > ÂÂgoto out;
>>> > > diff --git a/xen-all.c b/xen-all.c
>>> > > index b73fc43..04dfb51 100644
>>> > > --- a/xen-all.c
>>> > > +++ b/xen-all.c
>>> > > @@ -527,12 +534,22 @@ int xen_init(void)
>>> > > ÂÂÂÂÂÂÂÂ return -1;
>>> > > ÂÂÂÂ }
>>> > >
>>> > > +#ifdef CONFIG_STUBDOM
>>> > > +ÂÂÂ return 0;
>>> > > +#endif
>>> > > +
>>> > > +ÂÂÂ xc_handle = xen_xc_interface_open(0, 0, 0);
>>> > > +ÂÂÂ if (xc_handle == XC_HANDLER_INITIAL_VALUE) {
>>> > > +ÂÂÂÂÂÂÂ xen_be_printf(NULL, 0, "can't open xen interface\n");
>>> > > +ÂÂÂÂÂÂÂ return -1;
>>> > > +ÂÂÂ }
>>> > > +
>>> > > ÂÂÂÂ return 0;
>>> > > Â}
>>> > >
>>> > >
>>> >
>>> > I think that the backends shouldn't be initialized at all when running
>>> > in the stubdom, so I would do something like this instead:
>>> >
>>> > diff --git a/xen-all.c b/xen-all.c
>>> > index 167bed6..e3f630b 100644
>>> > --- a/xen-all.c
>>> > +++ b/xen-all.c
>>> > @@ -922,6 +922,7 @@ int xen_hvm_init(void)
>>> > Â Â cpu_register_phys_memory_client(&state->client);
>>> > Â Â state->log_for_dirtybit = NULL;
>>> >
>>> > +#ifndef CONFIG_STUBDOM
>>> > Â Â /* Initialize backend core & drivers */
>>> > Â Â if (xen_be_init() != 0) {
>>> > Â Â Â Â fprintf(stderr, "%s: xen backend core setup failed\n", 
>>> > __FUNCTION__);
>>> > @@ -930,7 +931,7 @@ int xen_hvm_init(void)
>>> > Â Â xen_be_register("console", &xen_console_ops);
>>> > Â Â xen_be_register("vkbd", &xen_kbdmouse_ops);
>>> > Â Â xen_be_register("qdisk", &xen_blkdev_ops);
>>> > -
>>> > +#endif
>>> > Â Â return 0;
>>> > Â}
>>> >
>>> >
>>> >
>>> >
>>> > Yes. this implementation of stubdomÂis more closer to the old qemu's 
>>> > implementation. Which branch this patch works
>>> on? The
>>> > qemu-dm-v15 would not so lucky.
>>>
>>> That branch is old now, checkout this one:
>>>
>>> git://xenbits.xen.org/people/sstabellini/qemu-dm.git xen-stable-0.15
>>>
>>>
>>>
>>> Do you mean I migrate my workÂto this qemu branch?
>>
>> Yes, if it doesn't cost too much time for you to rebase your patches.
>
>
> Hi Stefano,
>
> Â Â These days I updated my patches to support Âxen-stable-0.15 qemu
> and latest xen-unstable. The fbdev is still not work. However, I have
> found three vram mapping areas in linux based stubdom. I think maybe
> our problem is caused by the mismatch of these areas. I list them with
> the order of initialization.
>
> 1. Âvga_common_init(s,8M)
> Â Â This vram area is mapped by xc_map_foreign_bulk()
> Â Â Â Â Â Â Â ->linux_privcmd_map_foreign_bulk()
> Â Â Â Â Â Â Â Â Â Â Â Â->mmap(NULL, (unsigned long)num <<
> XC_PAGE_SHIFT, prot, MAP_SHARED,fd, 0);
> Â Â The fd is descriptor of privcmd device. The
> entry->vaddr_base=0xb6bdc000. Therefore, the mapping area is
> 0xb6bdc000 ~ 0xb73dc000, 8M.
>
> Â Â The block->offset allocated by qemu_get_ram_ptr() is ram_size.
> The ram_size is the declared memory of HVM guest. In my case, the hvm
> guest takes 384M memory. So block->offset=0x18000000. The guest
> physical memory of vram is 0x18000000 ~ 0x18800000, 8M.
>
> 2. fbdev_init()
> Â Â This vram area is mapped by fbdev_init()
>
> ->mmap(NULL,fb_fix.smem_len+fb_mem_offset,PROT_READ|PROT_WRITE,MAP_SHARED,fb,0);
> Â Â The fd is descriptor of fb0 device. The mapping area is
> 0xb680a000 ~ 0xb6a0a000, 2M.
>
> 3. xen_add_to_physmap()
> Â Â ÂThis function is mapping the vram to 0xf0000000 ~ 0xf080000, 8M.
>
> Â ÂReferring to minios-based stubdom, I think the stubdom will work
> well if fbdev maps vram according to 0x18000000 ~ 0x18800000. I look
> forward to your comments and suggestions.
>
> Â ÂThanks,
>
>
> Jiageng Yu.
>

Hi Stefano,

    In the linux-pv xenfbfront driver, the vram is allocated by:
    xenfb_probe()
        ->info->fb = vmalloc(fb_size);

    In the linux-stubdom, the memory areas: (info->fb,
info->fb+fb_size) in linux-pv kernel, (s->vram_offset,
s->vram_offset+VGA_RAM_SIZE) in vga_common_init function,
(s->vram_ptr, s->vram_ptr+VGA_RAM_SIZE) in stubdom. These memory areas
should be mapped into the same machine memory region.

    But the (info->fb, info->fb+fb_size) in linux-pv kernel is
allocated independently. I have two optional plans to slove the
problem.

1. Modify linux-pv kernel.
    This plan is more closer to the minios stubdom. First, I delay the
initial process of xenfbfront driver until qemu allocates s->vram_ptr.
Then, I set info->fb = s->vram_ptr.

2. Modify libxc.
    The s->vram_ptr is generated by mmap() function in
linux_privcmd_map_foreign_bulk(). Maybe I could replace the return
value of mmap() with the info->fb, which is obtained from xenstore.

    Can these plans solve the problem? Or there is the better one?

    Regards!

Jiageng Yu.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel