Re: [Xen-devel] xm commands run very slow ....

[Himanshu Raj <rhim@xxxxxxxxxxxxx>]

I finally figured it out.

The tdb database used by xenstored got out of control - it had become ~300MB,
and that's why xenstored was taking a lot of time accessing it. After cleaning
it up, things are back to normal.

Thanks and Best Regards,
Himanshu
On Wed, Jan 25, 2006 at 07:37:04PM -0500, Himanshu Raj wrote:
> top shows heavy activity by xenstored for a long time (like 30-40% CPU 
> utilization).
> Also there is a huge gap between xenstore write and devcontroller to kick in.
> Excerpt from xend.log follows:
> 
> [2006-01-25 19:22:48 xend.XendDomainInfo] DEBUG (XendDomainInfo:688) Storing 
> dom
> ain details: {'console/ring-ref': '124478', 'console/port': '2', 'name': 
> 'vm1', 
> 'console/limit': '1048576', 'vm': '/vm/c1b17d02-ed25-f67e-2bef-0bbed31d765f', 
> 'd
> omid': '1', 'cpu/0/availability': 'online', 'memory/target': '65536', 
> 'store/rin
> g-ref': '124479', 'store/port': '1'}
> [2006-01-25 19:23:00 xend] DEBUG (DevController:132) Waiting for devices vif.
> [2006-01-25 19:23:00 xend] DEBUG (DevController:132) Waiting for devices usb.
> [2006-01-25 19:23:01 xend] DEBUG (DevController:132) Waiting for devices vbd.
> 
> Does it ring any bells?
> 
> Best regards,
> Himanshu
> 
> On Wed, Jan 25, 2006 at 05:06:36PM -0500, Himanshu Raj wrote:
> > Thanks Ewan.
> > 
> > I also saw one more discrepency which might be the root of this problem.
> > When I do xend start on first machine, 8 processes start up (i.e. ps aux | 
> > grep
> > xend shows actually 8 proceeses).
> > On my other machine, I see only 2 processes. Do you think it is the problem?
> > 
> > -Himanshu
> > 
> > > > I know it is kind of vague problem, but I am banking of somebody seeing 
> > > > the 
> > > > exact thing.
> > > > 
> > > > I have two machines with everything identical, except on one machine I 
> > > > compile
> > > > xen and on other I just use the binaries compiled on the previous 
> > > > machine.
> > > > 
> > > > Now on first machine, where I compile everything, xm invocation is 
> > > > veeeeeeeeeery
> > > > slow. Although it doesn't seem to be "system" problem. Look at a 
> > > >         time xm list 
> > > > invocation:
> > > >         real 32s
> > > >         user .2s
> > > >         sys  .02s
> > > > 
> > > > On another machine the real time is very close to user+sys.
> > > > 
> > > > I just can't comprehend why this is going on. Any clues will be deeply 
> > > > appreciated.
> > > 
> > > xm is a client to Xend, talking over HTTP -- xm is simply blocking for 30
> > > seconds waiting for the server -- you need to figure out why the server is
> > > spinning or stalling.
> > > 
> > > /var/log/xend.log may help.  top may help also.
> > > 
> > > Ewan.
> > 
> > -- 
> > -------------------------------------------------------------------------
> > Himanshu Raj
> > PhD Student, GaTech (www.cc.gatech.edu/~rhim)
> > I prefer to receive attachments in an open, non-proprietary format.
> > -------------------------------------------------------------------------
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> 
> -- 
> -------------------------------------------------------------------------
> Himanshu Raj
> PhD Student, GaTech (www.cc.gatech.edu/~rhim)
> I prefer to receive attachments in an open, non-proprietary format.
> -------------------------------------------------------------------------
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

-- 
-------------------------------------------------------------------------
Himanshu Raj
PhD Student, GaTech (www.cc.gatech.edu/~rhim)
I prefer to receive attachments in an open, non-proprietary format.
-------------------------------------------------------------------------

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel