[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-API] XCP iSCSI boot fails at "Starting the XCP networking daemon"


  • To: 'Damon Smith' <apple@xxxxxxxxxxxxxx>
  • From: Rob Hoes <Rob.Hoes@xxxxxxxxxx>
  • Date: Thu, 22 Nov 2012 14:38:40 +0000
  • Accept-language: en-US
  • Acceptlanguage: en-US
  • Cc: "xen-api@xxxxxxxxxxxxx" <xen-api@xxxxxxxxxxxxx>
  • Delivery-date: Thu, 22 Nov 2012 14:39:32 +0000
  • List-id: User and development list for XCP and XAPI <xen-api.lists.xen.org>
  • Thread-index: Ac3BhoSlJhSx6yUETOywA75H/IN6QJNwpQumk2I+37A=
  • Thread-topic: XCP iSCSI boot fails at "Starting the XCP networking daemon"

Hi Damon,

All the source code for the network daemon is here: 
https://github.com/xen-org/xen-api/tree/master/ocaml/network. It may be a bit 
hard to go through the code, but you can see exactly what it is doing in the 
/var/log/xensource.log file (grep for xcp-networkd). Can you figure out what it 
is doing when the connection breaks?

There aren't really any configuration options for networkd. The way it works is 
roughly as follows. Xapi tells networkd which network configuration it wants 
based on the PIF (and related) records in its database, and networkd then makes 
that happen. Each time the user does a "pif-plug" or "pif-reconfigure-ip" etc, 
xapi instructs networkd to update the config.

Except when the host starts. Networkd starts before xapi, because on a pool 
slave, xapi needs to have a network connection with the pool master. So on 
startup, networkd obtains the network configuration from a file called 
/var/xapi/networkd.db, which basically contains the state from before the 
reboot. Except on first boot, when there is not networkd.db state yet. In this 
case, it looks for the file /etc/firstboot.d/data/management.conf, which is 
created by the host installer, to setup the initial network config.

Hope this helps.

Cheers,
Rob

> -----Original Message-----
> From: Damon Smith [mailto:apple@xxxxxxxxxxxxxx]
> Sent: 14 November 2012 7:14 PM
> To: Rob Hoes
> Cc: xen-api@xxxxxxxxxxxxx
> Subject: Re: XCP iSCSI boot fails at "Starting the XCP networking daemon"
> 
> Rob,
> Thanks for your help! I was able to get XCP booting and "running"
> successfully after disabling xcp-networkd, but it has created so many issues
> that I would prefer to be able to figure out how to boot WITH networkd.
> 
> If I boot with networkd disabled, all the "firstboot" scripts fail to execute,
> leading to a badly broken environment. Presently SSH works but the
> webserver doesn't, meaning I can't use XenCenter to manage the server.
> Rather than trying to hack all these things and fix each problem manually, I'd
> much prefer to simply fix networkd and have everything "just work".
> 
> Is there any additional documentation available for xcp-networkd? Config
> files, command line parameters, etc? Or at the very least I suppose I could
> peruse the source and see if I can figure out why it is breaking...
> 
> Any thoughts before I bury myself in sourcecode that I probably won't
> understand?
> 
> Thanks,
> Damon
> 
> ----- Original Message -----
> From: "Rob Hoes" <Rob.Hoes@xxxxxxxxxx>
> To: "Damon Smith" <apple@xxxxxxxxxxxxxx>, "Adrian Costin"
> <adrian.costin@xxxxxxxxx>
> Cc: xen-api@xxxxxxxxxxxxx
> Sent: Tuesday, November 13, 2012 4:06:16 AM
> Subject: RE: XCP iSCSI boot fails at "Starting the XCP networking daemon"
> 
> The xcp-networkd daemon is new in XCP 1.6. Where the old system used
> the CentOS way of configuring the network using /etc/sysconfig/network-
> scripts, the network daemon uses more general Linux networking
> commands to get things done (this makes it suitable to run on Debian-based
> systems as well).
> 
> I am not sure how all this interacts with booting from iSCSI, though. If it
> helps, you could try switching off the network daemon using "chkconfig xcp-
> networkd off" (if I recall correctly), and then configure the management
> interface yourself.
> 
> Cheers,
> Rob
> 
> > -----Original Message-----
> > From: xen-api-bounces@xxxxxxxxxxxxx [mailto:xen-api-
> > bounces@xxxxxxxxxxxxx] On Behalf Of Damon Smith
> > Sent: 13 November 2012 3:27 AM
> > To: Adrian Costin
> > Cc: xen-api@xxxxxxxxxxxxx
> > Subject: Re: [Xen-API] XCP iSCSI boot fails at "Starting the XCP
> > networking daemon"
> >
> > Adrian,
> >
> > Thanks for your quick response. Unfortunately I wasn't able to get
> > this working.
> >
> > I had to *create* the following files (they were not already present):
> >
> > /etc/sysconfig/network-scripts/ifcfg-eth0:
> > DEVICE=eth0
> > BOOTPROTO=dhcp
> > ONBOOT=yes
> > XENMANAGED=no
> >
> > /etc/sysconfig/network-scripts/ifcfg-xenbr0:
> > DEVICE=xenbr0
> > BOOTPROTO=dhcp
> > ONBOOT=yes
> > XENMANAGED=no
> >
> >
> > At boot I recive notifications about bringing up eth0 and configuring
> > it, but I also see failures that "xenbr0 does not exist".
> >
> > I'll admit I know very little about CentOS, especially networking, so
> > do correct me if I completely misinterpreted what you said.
> >
> > Any other ideas? Does the xenbr0 interface have a different MAC
> > address that might cause it to obtain a different IP address from
> > DHCP? Anything else?
> >
> > Thanks,
> > Damon
> >
> > ----- Original Message -----
> > From: "Adrian Costin" <adrian.costin@xxxxxxxxx>
> > To: xen-api@xxxxxxxxxxxxx
> > Cc: apple@xxxxxxxxxxxxxx
> > Sent: Saturday, November 10, 2012 1:22:01 AM
> > Subject: Re: [Xen-API] XCP iSCSI boot fails at "Starting the XCP
> > networking daemon"
> >
> > Hi,
> >
> > I've managed to get XCP 1.5 to boot off iSCSI which also has this problem.
> > What I've done is edit ifcfg-xenbr0 (or whichever bridge you boot off)
> > and set XENMANAGED=no and assigned the IP directly into the ifcfg file
> > (in your case I guess it can be set to DHCP).
> >
> > It also has a problem when rebooting that it shuts down the network
> > interfaces before it finishes the reboot and then it just stays there
> > unable to continue the reboot.
> >
> > Best regards,
> > Adrian Costin
> >
> >
> > On Sat, Nov 10, 2012 at 7:41 AM, Damon Smith < apple@xxxxxxxxxxxxxx >
> > wrote:
> >
> >
> > Hello,
> > I am working on getting an XCP host to boot via iSCSI. I have had
> > pretty good success getting it installed and configured thus far,
> > needing only a few hacks to get it to a bootable state, but appear to have
> run into an issue.
> >
> > Upon booting, the system loads fine until it reaches "Starting the XCP
> > networking daemon". This command will eventually time-out and a few
> > seconds later I will start getting errors about journal writing. It
> > seems to me like the "Starting the XCP networking daemon" interrupts
> > my connection to my iSCSI target. Presumably xcp-networkd brings the
> > interfaces down in order to reconfigure them, but after bringing them
> > down is unable to finish as it can no longer access the filesystem.
> >
> > I've tried various options in /etc/xcp/network.conf: both "bridge" and
> > "openvswitch" fail in the same manner.
> >
> > Are there any additional configuration options I can use to resolve this?
> > Would modifying my init files help? Can I just disable xcp-networkd
> > completely and manage the networking configuration myself?
> >
> > I feel that once this issue is resolved I will have a working XCP
> > system booting from iSCSI, which would be very useful.
> >
> > I am running the latest version of XCP and am using DHCP to configure
> > my network. The iSCSI target is running FreeNAS.
> >
> > Any thoughts?
> >
> > Damon
> >
> > _______________________________________________
> > Xen-api mailing list
> > Xen-api@xxxxxxxxxxxxx
> > http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
> >
> >
> > Hi,
> >
> > I've managed to get XCP 1.5 to boot off iSCSI which also has this problem.
> > What I've done is edit ifcfg-xenbr0 (or whichever bridge you boot off)
> > and set XENMANAGED=no and assigned the IP directly into the ifcfg file
> > (in your case I guess it can be set to DHCP).
> >
> > It also has a problem when rebooting that it shuts down the network
> > interfaces before it finishes the reboot and then it just stays there
> > unable to continue the reboot.
> >
> > Best regards,
> > Adrian Costin
> >
> >
> > On Sat, Nov 10, 2012 at 7:41 AM, Damon Smith <apple@xxxxxxxxxxxxxx>
> > wrote:
> >
> > > Hello,
> > > I am working on getting an XCP host to boot via iSCSI. I have had
> > > pretty good success getting it installed and configured thus far,
> > > needing only a few hacks to get it to a bootable state, but appear
> > > to have run into an issue.
> > >
> > > Upon booting, the system loads fine until it reaches "Starting the
> > > XCP networking daemon". This command will eventually time-out and a
> > > few
> > seconds
> > > later I will start getting errors about journal writing. It seems to
> > > me like the "Starting the XCP networking daemon" interrupts my
> > > connection
> > to
> > > my iSCSI target. Presumably xcp-networkd brings the interfaces down
> > > in order to reconfigure them, but after bringing them down is unable
> > > to
> > finish
> > > as it can no longer access the filesystem.
> > >
> > > I've tried various options in /etc/xcp/network.conf: both "bridge"
> > > and "openvswitch" fail in the same manner.
> > >
> > > Are there any additional configuration options I can use to resolve this?
> > > Would modifying my init files help? Can I just disable xcp-networkd
> > > completely and manage the networking configuration myself?
> > >
> > > I feel that once this issue is resolved I will have a working XCP
> > > system booting from iSCSI, which would be very useful.
> > >
> > > I am running the latest version of XCP and am using DHCP to
> > > configure my network. The iSCSI target is running FreeNAS.
> > >
> > > Any thoughts?
> > >
> > > Damon
> > >
> > > _______________________________________________
> > > Xen-api mailing list
> > > Xen-api@xxxxxxxxxxxxx
> > > http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
> > >
> >
> > _______________________________________________
> > Xen-api mailing list
> > Xen-api@xxxxxxxxxxxxx
> > http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.