Re: [Xen-devel] Problem Reading from XenStore in DomU

[Doug Goldstein <cardoe@xxxxxxxxxx>]

On 5/16/16 3:18 PM, Dagaen Golomb wrote:
> On Mon, May 16, 2016 at 3:56 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> wrote:
>> On Mon, May 16, 2016 at 3:52 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> 
>> wrote:
>>> On Mon, May 16, 2016 at 3:12 PM, Doug Goldstein <cardoe@xxxxxxxxxx> wrote:
>>>> On 5/16/16 11:20 AM, Dagaen Golomb wrote:
>>>>> On Mon, May 16, 2016 at 12:11 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> 
>>>>> wrote:
>>>>>> On Mon, May 16, 2016 at 12:03 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> 
>>>>>> wrote:
>>>>>>> On Mon, May 16, 2016 at 11:55 AM, Doug Goldstein <cardoe@xxxxxxxxxx> 
>>>>>>> wrote:
>>>>>>>> On 5/15/16 8:41 PM, Dagaen Golomb wrote:
>>>>>>>>>> On 5/15/16 8:28 PM, Dagaen Golomb wrote:
>>>>>>>>>>>> On 5/15/16 11:40 AM, Dagaen Golomb wrote:
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm having an interesting issue. I am working on a project that
>>>>>>>>>>>>> requires me to share memory between dom0 and domUs. I have this
>>>>>>>>>>>>> successfully working using the grant table and the XenStore to
>>>>>>>>>>>>> communicate grefs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My issue is this. I have one domU running Ubuntu 12.04 with a 
>>>>>>>>>>>>> default
>>>>>>>>>>>>> 3.8.x kernel that has no issue reading or writing from the 
>>>>>>>>>>>>> XenStore.
>>>>>>>>>>>>> My work also requires some kernel modifications, and we have made
>>>>>>>>>>>>> these changes in the 4.1.0 kernel. In particular, we've only 
>>>>>>>>>>>>> added a
>>>>>>>>>>>>> simple hypercall. This modified kernel is what dom0 is running, 
>>>>>>>>>>>>> on top
>>>>>>>>>>>>> of Xen 4.7 rc1.
>>>>>>>>>>>>
>>>>>>>>>>>> Without reading the rest of the thread but seeing the kernel 
>>>>>>>>>>>> versions.
>>>>>>>>>>>> Can you check how you're communicating to xenstore? Is it via
>>>>>>>>>>>> /dev/xen/xenbus or /proc/xen/xenbus? Anything after 3.14 will give 
>>>>>>>>>>>> you
>>>>>>>>>>>> deadlocks if you try to use /proc/xen/xenbus. Xen 4.6 and newer 
>>>>>>>>>>>> should
>>>>>>>>>>>> prefer /dev/xen/xenbus. Same thing can happen with privcmd but 
>>>>>>>>>>>> making
>>>>>>>>>>>> that default didn't land until Xen 4.7. Since you're on the right
>>>>>>>>>>>> versions I expect you're using /dev/xen/xenbus but you never know.
>>>>>>>>>>>
>>>>>>>>>>> How do I know which is being used? /dev/xen/xenbus is there and so 
>>>>>>>>>>> is
>>>>>>>>>>> process/xen/xenbus. Could this be a problem with header version
>>>>>>>>>>> mismatches or something similar? I'm using the xen/xenstore.h header
>>>>>>>>>>> file for all of my xenstore interactions. I'm running Xen 4.7 so it
>>>>>>>>>>> should be in /dev/, and the old kernel is before 3.14 but the new 
>>>>>>>>>>> one
>>>>>>>>>>> is after, but I would presume the standard headers are updated to
>>>>>>>>>>> account for this. Is there an easy way to check for this? Also, 
>>>>>>>>>>> would
>>>>>>>>>>> the same issue cause writes to fails? Because writes from the same
>>>>>>>>>>> domain work fine, and appear to other domains using xenstore-ls.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Dagaen Golomb
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Use strace on the process and see what gets opened.
>>>>>>>>>
>>>>>>>>> Ah, of course. It seems both the working and non-working domains are
>>>>>>>>> using /proc/...
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Dagaen Golomb
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> How are you starting them? Can you confirm its attempting /dev/ first?
>>>>>>>> Xen 4.7 should prefer /dev/.
>>>>>
>>>>> For all kernels in my domU, without setting any environment variables
>>>>> they use /proc/.
>>>>> For 4.1.0 this did not work, but works with /dev/ when using
>>>>> environment variable.
>>>>> Is this supposed to try /dev/ before /proc/? Because this doesn't
>>>>> appear the case according to strace.
>>>>> I think this is a bug.
>>>>>
>>>>> Regards,
>>>>> Dagaen Golomb
>>>>>
>>>>
>>>> Is your domU using Xen 4.7 compiled utilities? or are you using distro
>>>> provided xenstore libraries?
>>>
>>> I am using the newest binaries available from the distro repo (Ubuntu
>>> 12.04), so version 4.4.
>>> I'm going to guess you might say this is part of the issue. 4.4 and
>>> 4.7 aren't really *that* far off, though.
>>> Legacy interfaces should not be breaking, so I still think this is a
>>> bug that needs to be addressed somewhere, maybe by determining what
>>> exactly the issue is with new kernels and /proc/ tree XenStore access,
>>> if thisis what older tools will be using.
>>
>> Should I try to use newer utilities? What would be the easiest method
>> of acquiring and installing them?
>> Should building and installing the *-tools directives in the xen
>> source do the trick? I'd like to see if this fixes the issue.
> 
> I installed a package for xenstore-utils 4.6, and installed over the
> 4.4 in my guest. Issue persists.
> 
> Regards,
> Dagaen Golomb
> 

Changing Xen bits won't help until you use Xen 4.7. The underlying issue
is the Linux kernel interface is broken from 3.14 and on.

-- 
Doug Goldstein

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel