Re: [Xen-devel] Problem Reading from XenStore in DomU

[Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx>]

On Mon, May 16, 2016 at 12:03 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> wrote:
> On Mon, May 16, 2016 at 11:55 AM, Doug Goldstein <cardoe@xxxxxxxxxx> wrote:
>> On 5/15/16 8:41 PM, Dagaen Golomb wrote:
>>>> On 5/15/16 8:28 PM, Dagaen Golomb wrote:
>>>>>> On 5/15/16 11:40 AM, Dagaen Golomb wrote:
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I'm having an interesting issue. I am working on a project that
>>>>>>> requires me to share memory between dom0 and domUs. I have this
>>>>>>> successfully working using the grant table and the XenStore to
>>>>>>> communicate grefs.
>>>>>>>
>>>>>>> My issue is this. I have one domU running Ubuntu 12.04 with a default
>>>>>>> 3.8.x kernel that has no issue reading or writing from the XenStore.
>>>>>>> My work also requires some kernel modifications, and we have made
>>>>>>> these changes in the 4.1.0 kernel. In particular, we've only added a
>>>>>>> simple hypercall. This modified kernel is what dom0 is running, on top
>>>>>>> of Xen 4.7 rc1.
>>>>>>
>>>>>> Without reading the rest of the thread but seeing the kernel versions.
>>>>>> Can you check how you're communicating to xenstore? Is it via
>>>>>> /dev/xen/xenbus or /proc/xen/xenbus? Anything after 3.14 will give you
>>>>>> deadlocks if you try to use /proc/xen/xenbus. Xen 4.6 and newer should
>>>>>> prefer /dev/xen/xenbus. Same thing can happen with privcmd but making
>>>>>> that default didn't land until Xen 4.7. Since you're on the right
>>>>>> versions I expect you're using /dev/xen/xenbus but you never know.
>>>>>
>>>>> How do I know which is being used? /dev/xen/xenbus is there and so is
>>>>> process/xen/xenbus. Could this be a problem with header version
>>>>> mismatches or something similar? I'm using the xen/xenstore.h header
>>>>> file for all of my xenstore interactions. I'm running Xen 4.7 so it
>>>>> should be in /dev/, and the old kernel is before 3.14 but the new one
>>>>> is after, but I would presume the standard headers are updated to
>>>>> account for this. Is there an easy way to check for this? Also, would
>>>>> the same issue cause writes to fails? Because writes from the same
>>>>> domain work fine, and appear to other domains using xenstore-ls.
>>>>>
>>>>> Regards,
>>>>> Dagaen Golomb
>>>>>
>>>>
>>>> Use strace on the process and see what gets opened.
>>>
>>> Ah, of course. It seems both the working and non-working domains are
>>> using /proc/...
>>>
>>> Regards,
>>> Dagaen Golomb
>>>
>>
>>
>> How are you starting them? Can you confirm its attempting /dev/ first?
>> Xen 4.7 should prefer /dev/.
>
> The previous versions used /proc/ as well according to strace, but
> worked. According to others, this may be an issue with newer kernels
> (3.14+) using /proc/.
>
> I experimented with our 4.1.0 kernel. At first it still wasn't working
> because I was using sudo as the program needs superuser access... each
> time this cleared out any environment.
> Using sudo su, then setting the environment to use /dev/, and then
> running the program successfully reads! I'll consider this the
> solution. Thanks everyone.
>
> Now to make this environment setting apply to all users/shells,
> directly from boot. I guess /etc/bashrc will do the trick?

Also, this wasn't fixed when I recompiled on this kernel, so if the
newer headers point to /dev/ instead of /proc/, it must be using the
old headers for the 3.2/3.13 kernels even though its running 4.1.0.
I thought installing the kernel did headers as well, does it not? Or
do I need to place custom kernel headers myself?

Regards,
Dagaen Golomb

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel