Re: [Xen-devel] Front-end back-end connection

[Daniel Castro <evil.dani@xxxxxxxxx>]

On Wed, Oct 12, 2011 at 10:56 PM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
> On Wed, 2011-10-12 at 13:02 +0100, Daniel Castro wrote:
>> On Wed, Oct 12, 2011 at 7:12 PM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> 
>> wrote:
>> > On Wed, 2011-10-12 at 08:13 +0100, Daniel Castro wrote:
>> >> Hello All,
>> >>
>> >> I am in the process of conecting my front-end to my back end.
>> >> The process is like this:
>> >> 1. Set up xenstore conection
>> >> 2. initiate front rings
>> >> 3. Initiate gran table
>> >> 4. Take my rings mfn address and set it up as a entry (more on this)
>> >> 5. create a unbound port for front-back ring communication (more on this )
>> >> 5.1 I start by changing state to XenbusStateInitialising
>> >> 5.2 ring-ref entry (step 4)
>> >> 5.3 port entry (step 5)
>> >> 5.4 backend state is XenbusStateInitWait
>> >> 5.4 change state to XenbusStateInitialised
>> >> 5.5 back end state is XenbusStateClosing meaning there is an error or
>> >> something is missing.
>> >> 6. on sucess end
>> >>
>> >> More on step 4: I got my grant page table like this:
>> >>       struct gnttab_setup_table gst;
>> >>       grant_entries = (struct grant_entry_v1 *) memalign_high(4096, 4096);
>> >> //asume malloc
>> >>       memset(grant_entries,0,4096);
>> >>       gst.dom = DOMID_SELF; //&me
>> >>       gst.nr_frames = 1; //a single page
>> >>       //gst.frame_list = grant_entries; (I have no idea how to handle 
>> >> this :P )
>> >>       res = hypercall_grant_table_op(GNTTABOP_map_grant_ref, &gst, 1);
>> >> I think this works, but maybe I am wrong.
>> >
>> > I'm afraid you are.
>> >
>> > For one thing simply not initialising one of the fields in the argument
>> > structure is unlikely to be correct.
>> >
>> > Secondly the argument to GNTTABOP_map_grant_ref is a pointer to "struct
>> > gnttab_map_grant_ref" not "struct gnttab_setup_table", likewise "struct
>> > gnttab_setup_table" goes with GNTTABOP_setup_table. I think this should
>> > be pretty clear from the way the GNTTABOP_* and struct definitions are
>> > laid out in xen/include/public/grant_table.h and the naming convention
>> > what goes with what. There are also comments in that header describing
>> > each operation.
>> >
>> > If you are trying to setup the grant table itself then
>> > GNTTABOP_setup_table is what you want. GNTTABOP_map_grant_ref is used
>> > for mapping a grant reference which you have been given by another
>> > domain.
>> >
>> > drivers/xen/grant-table.c:gnttab_map should provide a rough idea how
>> > this needs to be done. Because this is an HVM domain you need to do a
>> > XENMEM_add_to_physmap of XENMAPSPACE_grant_table before you do the
>> > GNTTABOP_setup_table.
>>
>> Fixed. Yet, how can I confirm that the grant table was correctly
>> mapped? The hypercall returned 0 and the status in the struct is also
>> 0.
>> After the mapping I am printing this:
>> allocated grant_entries at 12 bytes at 0x0f7fc000, gpfn 0xf7fc
>> GNTTABOP_setup_table return 0 status:0
>> allocated shared info 2584 bytes at 0x0f7fa000, gpfn 0xf7fa
>
> Did you also call XENMEM_add_to_physmap somewhere?
Its working now after the changes, and also I noticed that back-end
does not read  key "port", instead read "event-channel" so I left both
in case.

For completeness and future references to the list here is my code:
static struct grant_entry_v1 *grant_entries = NULL;
    struct xen_add_to_physmap xatp;
    if(grant_entries!=NULL)
                return grant_entries;
    xatp.domid = DOMID_SELF;
    xatp.space = XENMAPSPACE_grant_table;
    xatp.idx   = 0;
    grant_entries = (struct grant_entry_v1 *) memalign_high(PAGE_SIZE,
PAGE_SIZE);
    memset(grant_entries, 0, PAGE_SIZE);
    xatp.gpfn  = ((unsigned long)grant_entries >> PAGE_SHIFT);
    dprintf(1, "allocated grant_entries at %d bytes at %p, gpfn
0x%lx\n",sizeof(*grant_entries), grant_entries, xatp.gpfn);
    if (hypercall_memory_op(XENMEM_add_to_physmap, &xatp) != 0)
        panic("MAP grant_entries info page fail");
    return grant_entries;

struct gnttab_setup_table gst;
        grant_entries = (struct grant_entry_v1 *) get_grant_table();
        gst.dom = DOMID_SELF;
        gst.nr_frames = 1;
        //gst.frame_list = (struct grant_entry_v1 *) grant_entries;
        res = hypercall_grant_table_op(GNTTABOP_setup_table, &gst, 1);
        if(res!=0){
                dprintf(1,"Error Mapping Grant Table... Abort...\n");
                panic("Map Grant Table Failed\n");
        }
        dprintf(1,"GNTTABOP_setup_table return %d status:%d\n",res,gst.status);

After this two pieces of code I create the grant entry for the rings
as previously discussed.
Must fill in device details from xenstore, and, now that the back and
front connect I have to handle requests, read and write.

I guess is a matter of copy from buffer to ring, and from ring to buffer...

>
> I'm afraid these messages are pretty meaningless without the
> corresponding code, but you should certainly have already setup the
> table before you start allocating grant_entries in it (or more
> importantly writing to them). These messages suggest that is not the
> case?
>
>> I also found that the port I get from the EVTCHNOP_alloc_unbound is 4,
>> but lsevtchn gets me this:
>>   42: VCPU 0: Interdomain (Connected) - Remote Domain 5, Port 2
>>   43: VCPU 0: Interdomain (Connected) - Remote Domain 5, Port 3
>>   44: VCPU 0: Interdomain (Connected) - Remote Domain 5, Port 1
>> Port 4 is not listed, is it because it is not connected yet?
>
> If this is lsevtchn for dom0 then most likely yes. You should be able to
> see the guest's state with lsevtchn <domid>.
>
> In the backend the evtchn gets bound after the frontend's shared ring is
> mapped so the fact that the evtchn didn't get bound suggests an error
> before that point.
>
>> I get no errors in xl dmesg, and in dmesg I get:
>> vbd vbd-5-768: 2 reading /local/domain/5/device/vbd/768/ring-ref and
>> event-channel
>
> Do you see anything else in xenstore?
>
>> I guess there are the keys I write in the front-end, so they are being read.
>>
>> Thanks for the help, I feel we are getting closer to the other side...
>>
>> Daniel
>>
>> >
>> >>  This is needed on step 5, so..
>> >> More on step 5:
>> >> I consider the grant table an array of type struct grant_entry_v1.
>> >> So I simply do grant_entry_v1[0] for my first grant entry, and so
>> >> forth. For this case I read on the list some time ago that entry 0 did
>> >> not work, so I work with entry 1, like this:
>> >> grant_entries[aval_grant].domid = ext_domid;
>> >> grant_entries[aval_grant].frame = _frame; where frame is: (u32)sring
>> >> >> PAGE_SHIFT //meaning mfn of my rings.
>> >> grant_entries[aval_grant].flags = GTF_permit_access;
>> >
>> > This looks approximately correct _if_ you were actually writing to some
>> > memory which was your grant table but due to the above I think you are
>> > not.
>> >
>> >> These last two steps I described may be wrong... I know this because
>> >> the backend state is not XenbusStateConnected.
>> >
>> > That fact alone can't tell you much other than _something_ went wrong.
>> >
>> > Since the backend will have transitioned to XenbusStateClosing by
>> > calling xenbus_dev_fatal() it will have written some error information
>> > to xenstore and the dom0 console which should hint at what actually went
>> > wrong. I expect you will see the message "mapping ring-ref %lu port %u"
>> > because xen_blkif_map will have failed during map_frontend_page().
>> >
>> >> I think my mistake is on step 5, so can someone please shed some light
>> >> on this small issue.
>> >>
>> >> And thank you very much for taking the time to read.
>> >>
>> >> Daniel
>> >>
>> >
>> >
>> >
>>
>>
>>
>
>
>



-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel