[Xen-devel] Question: How to map granted pages into domU properly?

[abc <abc@xxxxxxxxxxxxxx>]

Hi,

I'm writing a diploma-thesis based on paravirtualized XEN v3.2.1 and 
openSuSE 11.0 (32bit PAE).
The idea of my thesis is to use XEN as a research platform for on-die 
message passing interface hardware. Therefore several domUs should 
communicate via a configurable kernel-module over shared-memory.
Currently I encountered a problem in granting kernel-pages from dom0 to 
domUs.
The pages can be mapped successfully. But when the kernel-module in 
domU, tries to read the content of this page, a kernel oops occurs.
Can you explain me how to grant pages and how to map them into domUs 
properly?



These are my sources (shortened):


Dom0 -----------------------------------------------

static inline unsigned long vaddr2pfn(void *vaddr) {

 struct page* Page = vmalloc_to_page(vaddr);
 unsigned long PFN = page_to_pfn(Page);

 return PFN;
}

 unsigned int ForeignDomainID = 12; // <-- Hard coded
 unsigned long Page = get_zeroed_page(0);
 unsigned long PFN = vaddr2pfn( (void *) Page);
 unsigned long MFN = virt_to_phys( (void *) PFN);
     
 grant_ref = gnttab_grant_foreign_access( ForeignDomainID, MFN, 0 );
----------------------------------------------------

grant_ref then is magically being transported to domU..

DomU (domID=12) ------------------------------------

 struct vm_struct *Area = alloc_vm_area(PAGE_SIZE);
 struct gnttab_map_grant_ref operation;
 gnttab_set_map_op(&operation, (unsigned long) Area->addr, 
GNTMAP_host_map | GNTMAP_application_map, grant_ref, 0);
 HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &operation, 1);
 // print content of operation to syslog..

 Unsigned long* Dword = (Unsigned long*) Area->addr;
 unsigned long Content = *Dword  // <-- BUG: unable to handle kernel 
paging request at d928a000
----------------------------------------------------

Dom0 - /var/log/messages ---------------------------

Sep  4 16:43:19 mpibench1 kernel: sm_communicator: instanciating in dom0
Sep  4 16:43:19 mpibench1 kernel: sm_communicator: granting memory to 
guest #12
Sep  4 16:43:19 mpibench1 kernel: sm_communicator: granted page 
@0xe0738000 (mfn=0x40020738) to domain #12
Sep  4 16:43:19 mpibench1 kernel: sm_communicator: grant_refs[1]==21 (0x15)

----------------------------------------------------

DomU - /var/log/messages ---------------------------

Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: instanciating 
in domU
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: mapping page 
#21 from domain #0
Sep  4 16:43:37 mpibench_guest2 kernel: klogd 1.4.1, ---------- state 
change ----------
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: new 
area={next=0xd7671760, addr=0xd928a000, size=0x2000, flags=0x1, 
pages=0x0, nr_pages=0, phys_addr=0x0}
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: 
HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &Operation, 1);
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: struct 
gnttab_map_grant_ref Operation = {:
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   /* IN 
parameters. */
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint64_t 
host_addr=0x00000000d928a000;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint32_t 
flags=2=GNTMAP_host_map;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   grant_ref_t 
ref=21;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   domid_t  dom=0;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   /* OUT 
parameters. */
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   int16_t  
status=0;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   
grant_handle_t handle=0;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint64_t 
dev_bus_addr=0x0000040020738000;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:}
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: successfully 
mapped page @0xd928a000 (handle=0, phys=0x0000040020738000)
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: GrantedPage 
@0xd928a000
Sep  4 16:43:37 mpibench_guest2 kernel: BUG: unable to handle kernel 
paging request at d928a000
Sep  4 16:43:37 mpibench_guest2 kernel: IP: [<d9460896>] 
:sm_communicator:m_init+0x1e1/0x210
Sep  4 16:43:37 mpibench_guest2 kernel: *pdpt = 0000000002794001 *pde = 
0000000002861067 *pte = 0000040020738863
Sep  4 16:43:37 mpibench_guest2 kernel: Oops: 0009 [#1] SMP
Sep  4 16:43:37 mpibench_guest2 kernel: last sysfs file: 
/sys/devices/xen/vif-0/net/eth0/type
Sep  4 16:43:37 mpibench_guest2 kernel: Modules linked in: 
sm_communicator(+) nfs lockd nfs_acl sunrpc iptable_filter ip_tables 
ip6table_filter ip6_tables x_tables ipv6 af_packet 8250 serial_core loop 
dm_mod joydev reiserfs xenblk xennet
Sep  4 16:43:37 mpibench_guest2 kernel:
Sep  4 16:43:38 mpibench_guest2 kernel: Pid: 2383, comm: insmod Tainted: 
G        N (2.6.25.5-1.1-xen #1)
Sep  4 16:43:38 mpibench_guest2 kernel: EIP: 0061:[<d9460896>] EFLAGS: 
00010246 CPU: 0
Sep  4 16:43:38 mpibench_guest2 kernel: EIP is at m_init+0x1e1/0x210 
[sm_communicator]
Sep  4 16:43:38 mpibench_guest2 kernel: EAX: 0000002c EBX: 00000000 ECX: 
f5636000 EDX: f5636000
Sep  4 16:43:38 mpibench_guest2 kernel: ESI: d928a000 EDI: d9463a20 EBP: 
d6d49e88 ESP: d6d49e64
Sep  4 16:43:38 mpibench_guest2 kernel:  DS: 007b ES: 007b FS: 00d8 GS: 
0033 SS: 0069
Sep  4 16:43:38 mpibench_guest2 kernel: Process insmod (pid: 2383, 
ti=d6d48000 task=d750c160 task.ti=d6d48000)
Sep  4 16:43:38 mpibench_guest2 kernel: Stack: d9461590 d928a000 
d7532900 00000001 00000025 d6d49e88 d7532900 00000001
Sep  4 16:43:38 mpibench_guest2 kernel:        00000025 d6d49fb0 
c013be4d 00000024 000003e8 00000064 d6e27c00 0000e793
Sep  4 16:43:38 mpibench_guest2 kernel:        0000001f d6d49f88 
d7957718 00000000 d9462800 00000000 00000000 00000000
Sep  4 16:43:38 mpibench_guest2 kernel: Call Trace:
Sep  4 16:43:38 mpibench_guest2 kernel:  [<c013be4d>] 
sys_init_module+0x1989/0x1ab6
Sep  4 16:43:38 mpibench_guest2 kernel:  [<c0103f5a>] syscall_call+0x7/0xb
Sep  4 16:43:38 mpibench_guest2 kernel:  [<f57fe416>] 0xf57fe416
Sep  4 16:43:38 mpibench_guest2 kernel:  =======================
Sep  4 16:43:38 mpibench_guest2 kernel: Code: ff eb 4a 8b 04 9f ba 00 00 
00 00 e8 7e fc ff ff 89 c6 a1 98 39 46 d9 85 c0 78 32 89 74 24 04 c7 04 
24 90 15 46 d9 e8 66 a2 e7 e6 <8b> 06 89 44 24 04 c7 04 24 b8 15 46 d9 
e8 54 a2 e7 e6 83 c3 01
Sep  4 16:43:38 mpibench_guest2 kernel: EIP: [<d9460896>] 
m_init+0x1e1/0x210 [sm_communicator] SS:ESP 0069:d6d49e64
Sep  4 16:43:38 mpibench_guest2 kernel: ---[ end trace fc6d563289120224 ]---

------------------------------------------



Any help on this would be appreciated. I'm working on this for a while 
now and being currently stucked at this point.

Regards,

Gregor Rebel
Candidate Diplom Informatiker


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel