Re: [Xen-devel] Front-end back-end connection

[Daniel Castro <evil.dani@xxxxxxxxx>]

On Thu, Oct 13, 2011 at 8:08 PM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
> On Thu, 2011-10-13 at 11:34 +0100, Daniel Castro wrote:
>> On Wed, Oct 12, 2011 at 10:56 PM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> 
>> wrote:
>> > On Wed, 2011-10-12 at 13:02 +0100, Daniel Castro wrote:
>> >> On Wed, Oct 12, 2011 at 7:12 PM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> 
>> >> wrote:
>> >> > On Wed, 2011-10-12 at 08:13 +0100, Daniel Castro wrote:
>> >> >> Hello All,
>> >> >>
>> >> >> I am in the process of conecting my front-end to my back end.
>> >> >> The process is like this:
>> >> >> 1. Set up xenstore conection
>> >> >> 2. initiate front rings
>> >> >> 3. Initiate gran table
>> >> >> 4. Take my rings mfn address and set it up as a entry (more on this)
>> >> >> 5. create a unbound port for front-back ring communication (more on 
>> >> >> this )
>> >> >> 5.1 I start by changing state to XenbusStateInitialising
>> >> >> 5.2 ring-ref entry (step 4)
>> >> >> 5.3 port entry (step 5)
>> >> >> 5.4 backend state is XenbusStateInitWait
>> >> >> 5.4 change state to XenbusStateInitialised
>> >> >> 5.5 back end state is XenbusStateClosing meaning there is an error or
>> >> >> something is missing.
>> >> >> 6. on sucess end
>> >> >>
>> >> >> More on step 4: I got my grant page table like this:
>> >> >>       struct gnttab_setup_table gst;
>> >> >>       grant_entries = (struct grant_entry_v1 *) memalign_high(4096, 
>> >> >> 4096);
>> >> >> //asume malloc
>> >> >>       memset(grant_entries,0,4096);
>> >> >>       gst.dom = DOMID_SELF; //&me
>> >> >>       gst.nr_frames = 1; //a single page
>> >> >>       //gst.frame_list = grant_entries; (I have no idea how to handle 
>> >> >> this :P )
>> >> >>       res = hypercall_grant_table_op(GNTTABOP_map_grant_ref, &gst, 1);
>> >> >> I think this works, but maybe I am wrong.
>> >> >
>> >> > I'm afraid you are.
>> >> >
>> >> > For one thing simply not initialising one of the fields in the argument
>> >> > structure is unlikely to be correct.
>> >> >
>> >> > Secondly the argument to GNTTABOP_map_grant_ref is a pointer to "struct
>> >> > gnttab_map_grant_ref" not "struct gnttab_setup_table", likewise "struct
>> >> > gnttab_setup_table" goes with GNTTABOP_setup_table. I think this should
>> >> > be pretty clear from the way the GNTTABOP_* and struct definitions are
>> >> > laid out in xen/include/public/grant_table.h and the naming convention
>> >> > what goes with what. There are also comments in that header describing
>> >> > each operation.
>> >> >
>> >> > If you are trying to setup the grant table itself then
>> >> > GNTTABOP_setup_table is what you want. GNTTABOP_map_grant_ref is used
>> >> > for mapping a grant reference which you have been given by another
>> >> > domain.
>> >> >
>> >> > drivers/xen/grant-table.c:gnttab_map should provide a rough idea how
>> >> > this needs to be done. Because this is an HVM domain you need to do a
>> >> > XENMEM_add_to_physmap of XENMAPSPACE_grant_table before you do the
>> >> > GNTTABOP_setup_table.
>> >>
>> >> Fixed. Yet, how can I confirm that the grant table was correctly
>> >> mapped? The hypercall returned 0 and the status in the struct is also
>> >> 0.
>> >> After the mapping I am printing this:
>> >> allocated grant_entries at 12 bytes at 0x0f7fc000, gpfn 0xf7fc
>> >> GNTTABOP_setup_table return 0 status:0
>> >> allocated shared info 2584 bytes at 0x0f7fa000, gpfn 0xf7fa
>> >
>> > Did you also call XENMEM_add_to_physmap somewhere?
>> Its working now after the changes, and also I noticed that back-end
>> does not read  key "port", instead read "event-channel" so I left both
>> in case.
>
> You only need to fill in the one which the backend reads. That is
> "event-channel".
>
>> For completeness and future references to the list here is my code:
>> static struct grant_entry_v1 *grant_entries = NULL;
>>     struct xen_add_to_physmap xatp;
>>     if(grant_entries!=NULL)
>>               return grant_entries;
>>     xatp.domid = DOMID_SELF;
>>     xatp.space = XENMAPSPACE_grant_table;
>>     xatp.idx   = 0;
>>     grant_entries = (struct grant_entry_v1 *) memalign_high(PAGE_SIZE,
>> PAGE_SIZE);
>>     memset(grant_entries, 0, PAGE_SIZE);
>>     xatp.gpfn  = ((unsigned long)grant_entries >> PAGE_SHIFT);
>>     dprintf(1, "allocated grant_entries at %d bytes at %p, gpfn
>> 0x%lx\n",sizeof(*grant_entries), grant_entries, xatp.gpfn);
>>     if (hypercall_memory_op(XENMEM_add_to_physmap, &xatp) != 0)
>>       panic("MAP grant_entries info page fail");
>>     return grant_entries;
>>
>> struct gnttab_setup_table gst;
>>       grant_entries = (struct grant_entry_v1 *) get_grant_table();
>>       gst.dom = DOMID_SELF;
>>       gst.nr_frames = 1;
>>       //gst.frame_list = (struct grant_entry_v1 *) grant_entries;
>
> This isn't needed? That would surprise me...
Its part of the " OUT parameters. " in the struct, so I did not fill it out...
>
>>       res = hypercall_grant_table_op(GNTTABOP_setup_table, &gst, 1);
>>       if(res!=0){
>>               dprintf(1,"Error Mapping Grant Table... Abort...\n");
>>               panic("Map Grant Table Failed\n");
>>       }
>>       dprintf(1,"GNTTABOP_setup_table return %d status:%d\n",res,gst.status);
>>
>> After this two pieces of code I create the grant entry for the rings
>> as previously discussed.
>> Must fill in device details from xenstore, and, now that the back and
>> front connect I have to handle requests, read and write.
>>
>> I guess is a matter of copy from buffer to ring, and from ring to buffer...
>
> The block protocol involves granting the buffer and putting the ref on
> the ring, not putting data inline in the ring.
>
> However for simplicity I would suggest granting a static buffer to the
> backend and copying from the SeaBIOS buffer to that.
So would it be better to create another static buffer get a gref for
it and copy from disk_op_s->buf_fl when reading and copy to when
writing?

BTW what are segments in SeaBIOS and whats the use for it? in Xen is
only 32 Bit flat right? so I do not have to use any of the segment
macros, right?

Thanks for the help :)

>
>>
>> >
>> > I'm afraid these messages are pretty meaningless without the
>> > corresponding code, but you should certainly have already setup the
>> > table before you start allocating grant_entries in it (or more
>> > importantly writing to them). These messages suggest that is not the
>> > case?
>> >
>> >> I also found that the port I get from the EVTCHNOP_alloc_unbound is 4,
>> >> but lsevtchn gets me this:
>> >>   42: VCPU 0: Interdomain (Connected) - Remote Domain 5, Port 2
>> >>   43: VCPU 0: Interdomain (Connected) - Remote Domain 5, Port 3
>> >>   44: VCPU 0: Interdomain (Connected) - Remote Domain 5, Port 1
>> >> Port 4 is not listed, is it because it is not connected yet?
>> >
>> > If this is lsevtchn for dom0 then most likely yes. You should be able to
>> > see the guest's state with lsevtchn <domid>.
>> >
>> > In the backend the evtchn gets bound after the frontend's shared ring is
>> > mapped so the fact that the evtchn didn't get bound suggests an error
>> > before that point.
>> >
>> >> I get no errors in xl dmesg, and in dmesg I get:
>> >> vbd vbd-5-768: 2 reading /local/domain/5/device/vbd/768/ring-ref and
>> >> event-channel
>> >
>> > Do you see anything else in xenstore?
>> >
>> >> I guess there are the keys I write in the front-end, so they are being 
>> >> read.
>> >>
>> >> Thanks for the help, I feel we are getting closer to the other side...
>> >>
>> >> Daniel
>> >>
>> >> >
>> >> >>  This is needed on step 5, so..
>> >> >> More on step 5:
>> >> >> I consider the grant table an array of type struct grant_entry_v1.
>> >> >> So I simply do grant_entry_v1[0] for my first grant entry, and so
>> >> >> forth. For this case I read on the list some time ago that entry 0 did
>> >> >> not work, so I work with entry 1, like this:
>> >> >> grant_entries[aval_grant].domid = ext_domid;
>> >> >> grant_entries[aval_grant].frame = _frame; where frame is: (u32)sring
>> >> >> >> PAGE_SHIFT //meaning mfn of my rings.
>> >> >> grant_entries[aval_grant].flags = GTF_permit_access;
>> >> >
>> >> > This looks approximately correct _if_ you were actually writing to some
>> >> > memory which was your grant table but due to the above I think you are
>> >> > not.
>> >> >
>> >> >> These last two steps I described may be wrong... I know this because
>> >> >> the backend state is not XenbusStateConnected.
>> >> >
>> >> > That fact alone can't tell you much other than _something_ went wrong.
>> >> >
>> >> > Since the backend will have transitioned to XenbusStateClosing by
>> >> > calling xenbus_dev_fatal() it will have written some error information
>> >> > to xenstore and the dom0 console which should hint at what actually went
>> >> > wrong. I expect you will see the message "mapping ring-ref %lu port %u"
>> >> > because xen_blkif_map will have failed during map_frontend_page().
>> >> >
>> >> >> I think my mistake is on step 5, so can someone please shed some light
>> >> >> on this small issue.
>> >> >>
>> >> >> And thank you very much for taking the time to read.
>> >> >>
>> >> >> Daniel
>> >> >>
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >
>> >
>> >
>>
>>
>>
>
>
>



-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel