Re: [Xen-users] XCP - Maximum CPU cores

[Dustin Marquess <dmarquess@xxxxxxxxx>]

Okay, I did some digging.

>From what I can tell, the Xen kernel itself in XCP is compiled for 256
physical CPUs.  However, the maximum virtual CPUs to a VM (including
to the dom0) is 32.

I believe this means that the hypervisor itself will use all available
CPUs and schedule VMs across them, however the dom0 itself (or any VM
for that matter) will only see 32.  From my history with using NetBSD
as a dom0 (which doesn't support SMP under Xen), this is how this
works.  So I'm not sure this is really a problem?  I don't see why the
dom0 would need to see all of the CPUs anyways.

-Dustin


On Thu, Apr 14, 2011 at 1:14 PM, Outback Dingo <outbackdingo@xxxxxxxxx> wrote:
> talk was that XCP is build for 32 cores, for 64 a recompile for 64 CPUs is
> needed, ive got
> a 96 core and 2 128 core boxes, which Ive read requires xen 4.1 since now
> that supports
>> 256 CPUs
>
> On Thu, Apr 14, 2011 at 4:00 PM, Dustin Marquess <dmarquess@xxxxxxxxx>
> wrote:
>>
>> Strangely according to the xen.spec that gets installed using the xen
>> SRPM from the sources ISO, it looks like it was setup with a
>> max_phys_cpus setting of 256.
>>
>> Of course it also refers to an empty patch, so who really knows.
>>
>> -Dustin
>>
>> On Thu, Apr 14, 2011 at 12:55 PM, Outback Dingo <outbackdingo@xxxxxxxxx>
>> wrote:
>> > I built and installed XEN 4.1 on the XCP platform, and rebooted, it
>> > loaded
>> > but none of the SRs appeared available
>> > so VMs wouldnt start, i then rebooted back into "stock" with the 3.4.2,
>> > and
>> > the xapi wouldnt even run, so it appears
>> > I tired to rebuild the API to no avail hoping maybe then the SR issues
>> > would
>> > be forgiven
>> > at this point im at XCP 1.0 with the 4.1 in a booted but unusable state
>> > also kinda curious what the next "devel" drop will be ready and if itll
>> > be
>> > at 4.1 by then, ive got a few CPU heavy boxes
>> > which are really not useable to capacity at the moment, so crossing
>> > fingers
>> > for someone with knowledge to pitch in with
>> > pointers
>> >
>> > On Thu, Apr 14, 2011 at 3:48 PM, Dustin Marquess <dmarquess@xxxxxxxxx>
>> > wrote:
>> >>
>> >> I'm having the same issue myself. However I'm trying to stay as close
>> >> to 'stock' XCP 1.0 as possible, since I need stability.
>> >>
>> >> I have the 2 source ISOs & even the xen-api.hg Mercurial repo cloned.
>> >> However, I'm not sure which piece of XCP I need to recompile.  Is it
>> >> just the hypervisor kernel itself (eg, /boot/xen-3.4.2.gz), or the
>> >> dom0 kernel, or the entire xapi?
>> >>
>> >> The build instructions @
>> >> http://wiki.xen.org/xenwiki/XCP_Building_Instructions just show how to
>> >> rebuild xapi itself, not the entire XCP platform, sadly.
>> >>
>> >> -Dustin
>> >>
>> >> On Tue, Mar 8, 2011 at 11:51 AM, Todd Deshane <todd.deshane@xxxxxxx>
>> >> wrote:
>> >> > On Tue, Mar 8, 2011 at 2:46 PM, AP Xen <apxeng@xxxxxxxxx> wrote:
>> >> >> Technically, it is possible as XCP is open source. You would have to
>> >> >> download the source ISO and rebuild Xen with nr_phys_cpus=128.
>> >> >> Trying
>> >> >> to
>> >> >> replace Xen in XCP with the Xen 4.x might be more difficult.
>> >> >>
>> >> >
>> >> > True it is open source and should be possible. You're going to want
>> >> > to
>> >> > make sure you use at least the 4.1 branch as it has better libxl
>> >> > integration.
>> >> >
>> >> > Do report any problems that you find to xen-devel. I think the
>> >> > intention is that this should work, I'm just not sure how much
>> >> > community testing we have gotten to this point. Do let us know
>> >> > if/when
>> >> > you test it and how you progress.
>> >> >
>> >> > Thanks,
>> >> > Todd
>> >> >
>> >> >
>> >> >>
>> >> >>
>> >> >> From: Outback Dingo [mailto:outbackdingo@xxxxxxxxx]
>> >> >> Sent: Tuesday, March 08, 2011 11:08 AM
>> >> >> To: Todd Deshane
>> >> >> Cc: AP Xen; Tapas Mishra; xen-users; Boris Quiroz
>> >> >>
>> >> >> Subject: Re: [Xen-users] XCP - Maximum CPU cores
>> >> >>
>> >> >>
>> >> >>
>> >> >> well, i need 128 cores now on XCP :( so im hardware heavy and cant
>> >> >> utilize
>> >> >> 3/4 my cores on 2 systems, so how can one compensates for this
>> >> >>
>> >> >> is it possible to use xen 4.x on XCP 1.0 ?
>> >> >>
>> >> >> On Tue, Mar 8, 2011 at 1:42 PM, Todd Deshane <todd.deshane@xxxxxxx>
>> >> >> wrote:
>> >> >>
>> >> >> On Tue, Mar 8, 2011 at 6:32 PM, AP Xen <apxeng@xxxxxxxxx> wrote:
>> >> >>> It is called max_phys_cpus=<nr>
>> >> >>>
>> >> >>> http://xenbits.xensource.com/xen-unstable.hg/rev/d7cbcf5d7cd6
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> BTW, the default is 128 for Xen 4.x
>> >> >>>
>> >> >>
>> >> >> Just FYI, the plan will be to move to Xen 4.1+ starting with XCP 1.1
>> >> >>
>> >> >> We do also plan to track xen-unstable with an XCP unstable version
>> >> >> as
>> >> >> well. More to come on this.
>> >> >>
>> >> >> Thanks,
>> >> >> Todd
>> >> >>
>> >> >>>
>> >> >>>
>> >> >>> Thanks,
>> >> >>>
>> >> >>> AP
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> From: Tapas Mishra [mailto:mightydreams@xxxxxxxxx]
>> >> >>> Sent: Tuesday, March 08, 2011 10:03 AM
>> >> >>> To: xen-users
>> >> >>> Cc: AP Xen; Boris Quiroz; Outback Dingo
>> >> >>>
>> >> >>> Subject: Re: [Xen-users] XCP - Maximum CPU cores
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> Hi please elaborate as how the compilation method of hypervisor can
>> >> >>> enhance
>> >> >>> the number of CPU's to be pinned.
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On Tue, Mar 8, 2011 at 11:24 PM, AP Xen <apxeng@xxxxxxxxx> wrote:
>> >> >>>
>> >> >>> The default is 32, however you can up the limit to 64 with a
>> >> >>> compile
>> >> >>> time
>> >> >>> option. So, It depends on how the hypervisor was compiled.
>> >> >>>
>> >> >>> Thanks,
>> >> >>> AP
>> >> >>>
>> >> >>>> -----Original Message-----
>> >> >>>> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-users-
>> >> >>>> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Boris Quiroz
>> >> >>>> Sent: Tuesday, March 08, 2011 6:04 AM
>> >> >>>> To: Outback Dingo
>> >> >>>> Cc: xen-users
>> >> >>>> Subject: Re: [Xen-users] XCP - Maximum CPU cores
>> >> >>>>
>> >> >>>> 2011/3/8 Outback Dingo <outbackdingo@xxxxxxxxx>:
>> >> >>>> > Whats the maximum cpu cores XCP will see ?
>> >> >>>> > _______________________________________________
>> >> >>>> > Xen-users mailing list
>> >> >>>> > Xen-users@xxxxxxxxxxxxxxxxxxx
>> >> >>>> > http://lists.xensource.com/xen-users
>> >> >>>> >
>> >> >>>>
>> >> >>>> Somewhere I read they were 32 cores maximum. I'm checking..
>> >> >>>>
>> >> >>>> Cheers.
>> >> >>>>
>> >> >>>> --
>> >> >>>> @cereal_bars
>> >> >>>>
>> >> >>>> _______________________________________________
>> >> >>>> Xen-users mailing list
>> >> >>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> >> >>>> http://lists.xensource.com/xen-users
>> >> >>>
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> Xen-users mailing list
>> >> >>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> >> >>> http://lists.xensource.com/xen-users
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> http://mightydreams.blogspot.com
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> Xen-users mailing list
>> >> >>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> >> >>> http://lists.xensource.com/xen-users
>> >> >>>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Todd Deshane
>> >> >> http://www.linkedin.com/in/deshantm
>> >> >> http://www.xen.org/products/cloudxen.html
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Todd Deshane
>> >> > http://www.linkedin.com/in/deshantm
>> >> > http://www.xen.org/products/cloudxen.html
>> >> >
>> >> > _______________________________________________
>> >> > Xen-users mailing list
>> >> > Xen-users@xxxxxxxxxxxxxxxxxxx
>> >> > http://lists.xensource.com/xen-users
>> >> >
>> >>
>> >> _______________________________________________
>> >> Xen-users mailing list
>> >> Xen-users@xxxxxxxxxxxxxxxxxxx
>> >> http://lists.xensource.com/xen-users
>> >
>> >
>
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users