Re: [Xen-users] XCP - Maximum CPU cores

[Dustin Marquess <dmarquess@xxxxxxxxx>]

Actually, sorry, I'm wrong.  The 32 limit applies to physical *AND*
virtual CPUs. :)

-Dustin

On Thu, Apr 14, 2011 at 3:39 PM, Dustin Marquess <dmarquess@xxxxxxxxx> wrote:
> Okay, I did some digging.
>
> From what I can tell, the Xen kernel itself in XCP is compiled for 256
> physical CPUs.  However, the maximum virtual CPUs to a VM (including
> to the dom0) is 32.
>
> I believe this means that the hypervisor itself will use all available
> CPUs and schedule VMs across them, however the dom0 itself (or any VM
> for that matter) will only see 32.  From my history with using NetBSD
> as a dom0 (which doesn't support SMP under Xen), this is how this
> works.  So I'm not sure this is really a problem?  I don't see why the
> dom0 would need to see all of the CPUs anyways.
>
> -Dustin
>
>
> On Thu, Apr 14, 2011 at 1:14 PM, Outback Dingo <outbackdingo@xxxxxxxxx> wrote:
>> talk was that XCP is build for 32 cores, for 64 a recompile for 64 CPUs is
>> needed, ive got
>> a 96 core and 2 128 core boxes, which Ive read requires xen 4.1 since now
>> that supports
>>> 256 CPUs
>>
>> On Thu, Apr 14, 2011 at 4:00 PM, Dustin Marquess <dmarquess@xxxxxxxxx>
>> wrote:
>>>
>>> Strangely according to the xen.spec that gets installed using the xen
>>> SRPM from the sources ISO, it looks like it was setup with a
>>> max_phys_cpus setting of 256.
>>>
>>> Of course it also refers to an empty patch, so who really knows.
>>>
>>> -Dustin
>>>
>>> On Thu, Apr 14, 2011 at 12:55 PM, Outback Dingo <outbackdingo@xxxxxxxxx>
>>> wrote:
>>> > I built and installed XEN 4.1 on the XCP platform, and rebooted, it
>>> > loaded
>>> > but none of the SRs appeared available
>>> > so VMs wouldnt start, i then rebooted back into "stock" with the 3.4.2,
>>> > and
>>> > the xapi wouldnt even run, so it appears
>>> > I tired to rebuild the API to no avail hoping maybe then the SR issues
>>> > would
>>> > be forgiven
>>> > at this point im at XCP 1.0 with the 4.1 in a booted but unusable state
>>> > also kinda curious what the next "devel" drop will be ready and if itll
>>> > be
>>> > at 4.1 by then, ive got a few CPU heavy boxes
>>> > which are really not useable to capacity at the moment, so crossing
>>> > fingers
>>> > for someone with knowledge to pitch in with
>>> > pointers
>>> >
>>> > On Thu, Apr 14, 2011 at 3:48 PM, Dustin Marquess <dmarquess@xxxxxxxxx>
>>> > wrote:
>>> >>
>>> >> I'm having the same issue myself. However I'm trying to stay as close
>>> >> to 'stock' XCP 1.0 as possible, since I need stability.
>>> >>
>>> >> I have the 2 source ISOs & even the xen-api.hg Mercurial repo cloned.
>>> >> However, I'm not sure which piece of XCP I need to recompile.  Is it
>>> >> just the hypervisor kernel itself (eg, /boot/xen-3.4.2.gz), or the
>>> >> dom0 kernel, or the entire xapi?
>>> >>
>>> >> The build instructions @
>>> >> http://wiki.xen.org/xenwiki/XCP_Building_Instructions just show how to
>>> >> rebuild xapi itself, not the entire XCP platform, sadly.
>>> >>
>>> >> -Dustin
>>> >>
>>> >> On Tue, Mar 8, 2011 at 11:51 AM, Todd Deshane <todd.deshane@xxxxxxx>
>>> >> wrote:
>>> >> > On Tue, Mar 8, 2011 at 2:46 PM, AP Xen <apxeng@xxxxxxxxx> wrote:
>>> >> >> Technically, it is possible as XCP is open source. You would have to
>>> >> >> download the source ISO and rebuild Xen with nr_phys_cpus=128.
>>> >> >> Trying
>>> >> >> to
>>> >> >> replace Xen in XCP with the Xen 4.x might be more difficult.
>>> >> >>
>>> >> >
>>> >> > True it is open source and should be possible. You're going to want
>>> >> > to
>>> >> > make sure you use at least the 4.1 branch as it has better libxl
>>> >> > integration.
>>> >> >
>>> >> > Do report any problems that you find to xen-devel. I think the
>>> >> > intention is that this should work, I'm just not sure how much
>>> >> > community testing we have gotten to this point. Do let us know
>>> >> > if/when
>>> >> > you test it and how you progress.
>>> >> >
>>> >> > Thanks,
>>> >> > Todd
>>> >> >
>>> >> >
>>> >> >>
>>> >> >>
>>> >> >> From: Outback Dingo [mailto:outbackdingo@xxxxxxxxx]
>>> >> >> Sent: Tuesday, March 08, 2011 11:08 AM
>>> >> >> To: Todd Deshane
>>> >> >> Cc: AP Xen; Tapas Mishra; xen-users; Boris Quiroz
>>> >> >>
>>> >> >> Subject: Re: [Xen-users] XCP - Maximum CPU cores
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> well, i need 128 cores now on XCP :( so im hardware heavy and cant
>>> >> >> utilize
>>> >> >> 3/4 my cores on 2 systems, so how can one compensates for this
>>> >> >>
>>> >> >> is it possible to use xen 4.x on XCP 1.0 ?
>>> >> >>
>>> >> >> On Tue, Mar 8, 2011 at 1:42 PM, Todd Deshane <todd.deshane@xxxxxxx>
>>> >> >> wrote:
>>> >> >>
>>> >> >> On Tue, Mar 8, 2011 at 6:32 PM, AP Xen <apxeng@xxxxxxxxx> wrote:
>>> >> >>> It is called max_phys_cpus=<nr>
>>> >> >>>
>>> >> >>> http://xenbits.xensource.com/xen-unstable.hg/rev/d7cbcf5d7cd6
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> BTW, the default is 128 for Xen 4.x
>>> >> >>>
>>> >> >>
>>> >> >> Just FYI, the plan will be to move to Xen 4.1+ starting with XCP 1.1
>>> >> >>
>>> >> >> We do also plan to track xen-unstable with an XCP unstable version
>>> >> >> as
>>> >> >> well. More to come on this.
>>> >> >>
>>> >> >> Thanks,
>>> >> >> Todd
>>> >> >>
>>> >> >>>
>>> >> >>>
>>> >> >>> Thanks,
>>> >> >>>
>>> >> >>> AP
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> From: Tapas Mishra [mailto:mightydreams@xxxxxxxxx]
>>> >> >>> Sent: Tuesday, March 08, 2011 10:03 AM
>>> >> >>> To: xen-users
>>> >> >>> Cc: AP Xen; Boris Quiroz; Outback Dingo
>>> >> >>>
>>> >> >>> Subject: Re: [Xen-users] XCP - Maximum CPU cores
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> Hi please elaborate as how the compilation method of hypervisor can
>>> >> >>> enhance
>>> >> >>> the number of CPU's to be pinned.
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> On Tue, Mar 8, 2011 at 11:24 PM, AP Xen <apxeng@xxxxxxxxx> wrote:
>>> >> >>>
>>> >> >>> The default is 32, however you can up the limit to 64 with a
>>> >> >>> compile
>>> >> >>> time
>>> >> >>> option. So, It depends on how the hypervisor was compiled.
>>> >> >>>
>>> >> >>> Thanks,
>>> >> >>> AP
>>> >> >>>
>>> >> >>>> -----Original Message-----
>>> >> >>>> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-users-
>>> >> >>>> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Boris Quiroz
>>> >> >>>> Sent: Tuesday, March 08, 2011 6:04 AM
>>> >> >>>> To: Outback Dingo
>>> >> >>>> Cc: xen-users
>>> >> >>>> Subject: Re: [Xen-users] XCP - Maximum CPU cores
>>> >> >>>>
>>> >> >>>> 2011/3/8 Outback Dingo <outbackdingo@xxxxxxxxx>:
>>> >> >>>> > Whats the maximum cpu cores XCP will see ?
>>> >> >>>> > _______________________________________________
>>> >> >>>> > Xen-users mailing list
>>> >> >>>> > Xen-users@xxxxxxxxxxxxxxxxxxx
>>> >> >>>> > http://lists.xensource.com/xen-users
>>> >> >>>> >
>>> >> >>>>
>>> >> >>>> Somewhere I read they were 32 cores maximum. I'm checking..
>>> >> >>>>
>>> >> >>>> Cheers.
>>> >> >>>>
>>> >> >>>> --
>>> >> >>>> @cereal_bars
>>> >> >>>>
>>> >> >>>> _______________________________________________
>>> >> >>>> Xen-users mailing list
>>> >> >>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>> >> >>>> http://lists.xensource.com/xen-users
>>> >> >>>
>>> >> >>>
>>> >> >>> _______________________________________________
>>> >> >>> Xen-users mailing list
>>> >> >>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>> >> >>> http://lists.xensource.com/xen-users
>>> >> >>>
>>> >> >>>
>>> >> >>> --
>>> >> >>> http://mightydreams.blogspot.com
>>> >> >>>
>>> >> >>> _______________________________________________
>>> >> >>> Xen-users mailing list
>>> >> >>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>> >> >>> http://lists.xensource.com/xen-users
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Todd Deshane
>>> >> >> http://www.linkedin.com/in/deshantm
>>> >> >> http://www.xen.org/products/cloudxen.html
>>> >> >>
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Todd Deshane
>>> >> > http://www.linkedin.com/in/deshantm
>>> >> > http://www.xen.org/products/cloudxen.html
>>> >> >
>>> >> > _______________________________________________
>>> >> > Xen-users mailing list
>>> >> > Xen-users@xxxxxxxxxxxxxxxxxxx
>>> >> > http://lists.xensource.com/xen-users
>>> >> >
>>> >>
>>> >> _______________________________________________
>>> >> Xen-users mailing list
>>> >> Xen-users@xxxxxxxxxxxxxxxxxxx
>>> >> http://lists.xensource.com/xen-users
>>> >
>>> >
>>
>>
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users