Re: [Xen-devel] Problem Reading from XenStore in DomU

[Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx>]

On May 16, 2016 6:25 PM, "Doug Goldstein" <cardoe@xxxxxxxxxx> wrote:
>
> On 5/16/16 3:18 PM, Dagaen Golomb wrote:
> > On Mon, May 16, 2016 at 3:56 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> wrote:
> >> On Mon, May 16, 2016 at 3:52 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> wrote:
> >>> On Mon, May 16, 2016 at 3:12 PM, Doug Goldstein <cardoe@xxxxxxxxxx> wrote:
> >>>> On 5/16/16 11:20 AM, Dagaen Golomb wrote:
> >>>>> On Mon, May 16, 2016 at 12:11 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> wrote:
> >>>>>> On Mon, May 16, 2016 at 12:03 PM, Dagaen Golomb <dgolomb@xxxxxxxxxxxxxx> wrote:
> >>>>>>> On Mon, May 16, 2016 at 11:55 AM, Doug Goldstein <cardoe@xxxxxxxxxx> wrote:
> >>>>>>>> On 5/15/16 8:41 PM, Dagaen Golomb wrote:
> >>>>>>>>>> On 5/15/16 8:28 PM, Dagaen Golomb wrote:
> >>>>>>>>>>>> On 5/15/16 11:40 AM, Dagaen Golomb wrote:
> >>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'm having an interesting issue. I am working on a project that
> >>>>>>>>>>>>> requires me to share memory between dom0 and domUs. I have this
> >>>>>>>>>>>>> successfully working using the grant table and the XenStore to
> >>>>>>>>>>>>> communicate grefs.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> My issue is this. I have one domU running Ubuntu 12.04 with a default
> >>>>>>>>>>>>> 3.8.x kernel that has no issue reading or writing from the XenStore.
> >>>>>>>>>>>>> My work also requires some kernel modifications, and we have made
> >>>>>>>>>>>>> these changes in the 4.1.0 kernel. In particular, we've only added a
> >>>>>>>>>>>>> simple hypercall. This modified kernel is what dom0 is running, on top
> >>>>>>>>>>>>> of Xen 4.7 rc1.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Without reading the rest of the thread but seeing the kernel versions.
> >>>>>>>>>>>> Can you check how you're communicating to xenstore? Is it via
> >>>>>>>>>>>> /dev/xen/xenbus or /proc/xen/xenbus? Anything after 3.14 will give you
> >>>>>>>>>>>> deadlocks if you try to use /proc/xen/xenbus. Xen 4.6 and newer should
> >>>>>>>>>>>> prefer /dev/xen/xenbus. Same thing can happen with privcmd but making
> >>>>>>>>>>>> that default didn't land until Xen 4.7. Since you're on the right
> >>>>>>>>>>>> versions I expect you're using /dev/xen/xenbus but you never know.
> >>>>>>>>>>>
> >>>>>>>>>>> How do I know which is being used? /dev/xen/xenbus is there and so is
> >>>>>>>>>>> process/xen/xenbus. Could this be a problem with header version
> >>>>>>>>>>> mismatches or something similar? I'm using the xen/xenstore.h header
> >>>>>>>>>>> file for all of my xenstore interactions. I'm running Xen 4.7 so it
> >>>>>>>>>>> should be in /dev/, and the old kernel is before 3.14 but the new one
> >>>>>>>>>>> is after, but I would presume the standard headers are updated to
> >>>>>>>>>>> account for this. Is there an easy way to check for this? Also, would
> >>>>>>>>>>> the same issue cause writes to fails? Because writes from the same
> >>>>>>>>>>> domain work fine, and appear to other domains using xenstore-ls.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>> Dagaen Golomb
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Use strace on the process and see what gets opened.
> >>>>>>>>>
> >>>>>>>>> Ah, of course. It seems both the working and non-working domains are
> >>>>>>>>> using /proc/...
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Dagaen Golomb
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> How are you starting them? Can you confirm its attempting /dev/ first?
> >>>>>>>> Xen 4.7 should prefer /dev/.
> >>>>>
> >>>>> For all kernels in my domU, without setting any environment variables
> >>>>> they use /proc/.
> >>>>> For 4.1.0 this did not work, but works with /dev/ when using
> >>>>> environment variable.
> >>>>> Is this supposed to try /dev/ before /proc/? Because this doesn't
> >>>>> appear the case according to strace.
> >>>>> I think this is a bug.
> >>>>>
> >>>>> Regards,
> >>>>> Dagaen Golomb
> >>>>>
> >>>>
> >>>> Is your domU using Xen 4.7 compiled utilities? or are you using distro
> >>>> provided xenstore libraries?
> >>>
> >>> I am using the newest binaries available from the distro repo (Ubuntu
> >>> 12.04), so version 4.4.
> >>> I'm going to guess you might say this is part of the issue. 4.4 and
> >>> 4.7 aren't really *that* far off, though.
> >>> Legacy interfaces should not be breaking, so I still think this is a
> >>> bug that needs to be addressed somewhere, maybe by determining what
> >>> exactly the issue is with new kernels and /proc/ tree XenStore access,
> >>> if thisis what older tools will be using.
> >>
> >> Should I try to use newer utilities? What would be the easiest method
> >> of acquiring and installing them?
> >> Should building and installing the *-tools directives in the xen
> >> source do the trick? I'd like to see if this fixes the issue.
> >
> > I installed a package for xenstore-utils 4.6, and installed over the
> > 4.4 in my guest. Issue persists.
> >
> > Regards,
> > Dagaen Golomb
> >
>
> Changing Xen bits won't help until you use Xen 4.7. The underlying issue
> is the Linux kernel interface is broken from 3.14 and on.

OK, thanks. So this is actually an issue in the Linux codebase and not the Xen one? Good to know.

For now its been resolved with the environment variable and this will work for out purposes. Thanks for the help!

Regards,
Dagaen Golomb

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel