Re: [Xen-devel] Xen Hypervisor start fail when system memory more than 64GB, and free xen memory over 32 GB

[benian <bestwish.happiness@xxxxxxxxx>]

Sorry, the attachment disappear in the last mail @@"

Â

hi Pasi,

the attachments are the boot log i just dumped

my testing environment Âis  72GB ram + Xen-4.0.1-rc4(rc6)+2.6.32-xendom0-kernelÂ

where i modified ÂCONFIG_XEN_MAX_DOMAIN_MEMORY=32 to =128 in xendom0-kernel

using xen-4.0.1-rc4, it can boot smoothly if the grub option dom0_mem=32G is removed

or modify to >41GÂ

Besides, xen-4.0.1-rc6 keep fail no matter what the grub setting is.

regards,

Ben

2010/8/16 Pasi KÃrkkÃinen <pasik@xxxxxx>

On Mon, Aug 16, 2010 at 06:41:03PM +0800, benian wrote:
> Â Âhi Pasi,

>  ÂI 've tried it before, but 4.0.1-rc6 andà 4.0.1-rc5 à also boot fail, but
>  Âdifferent from the previous problem à error occur after XEN startup
> Â ÂÃÅÂclose to finish and all XEN message is cleared , Ã these two version
>  Âhang on during à XEN is still starting , and the final message isà "(XEN)
> Â ÂDom0 Ã has maximum 16 vcpus " Ã
> Â ÂOn the other hand, Ã 4.0.1-rc4 and 4.0.0 boot smoothlyÃ
>  ÂMoreover, i found thatà à 4.0.1-rc4 and 4.0.0à à takes much time on this
>  Âmessage à "(XEN) brought up 16cpu" à , whereà 4.0.1-rc6 andà 4.0.1-rc5
>  Âjustà fleet.Ã
>

Can you please post the full/complete bootlogs with Xen 4.0.1-rc6
so we can figure it out.. Thanks!

-- Pasi


> Â ÂRegards,
> Â ÂBen
> Â Â2010/8/16 Pasi KÃârkkÃâinen <[1]pasik@xxxxxx>

>
> Â Â ÂOn Mon, Aug 16, 2010 at 12:55:55PM +0300, Pasi KÃârkkÃâinen wrote:
> Â Â Â> On Mon, Aug 16, 2010 at 05:46:24PM +0800, benian wrote:

> Â Â Â> > Ã ÂÃ Hi Pasi
>   Â> > à Âà I finally boot up successfully by simply modified à the kernel
> Â Â Âconfig
> Â Â Â> > Ã ÂÃ Âitem CONFIG_XEN_MAX_DOMAIN_MEMORY=32 to

> Â Â ÂCONFIG_XEN_MAX_DOMAIN_MEMORY=128

> Â Â Â> > Ã ÂÃ (just choose a random value >72 ),
> Â Â Â> > Ã ÂÃ then rebuild the kernel-2.6.32 , and everything is ok.
> Â Â Â> > Ã ÂÃ By the way, i 've done many test and i finally found that
> Â Â Â> > Ã ÂÃ if unallocated memory(total - dom0 ) >32GB, boot up would be
> Â Â Âfailed in my
> Â Â Â> > Ã ÂÃ environment.
> Â Â Â> > Ã ÂÃ for example, after I modified the kernel config, hypervisor
> Â Â Âboot normally
> Â Â Â> > Ã ÂÃ and dom0 get 72GB.
> Â Â Â> > Ã ÂÃ If i add dom0_mem= 32G(unallocated 40G) to grub option,
> Â Â Âhypervisor boot
> Â Â Â> > Ã ÂÃ fail again,
> Â Â Â> > Ã ÂÃ and if dom0_mem= 41G(unallocated 31G), hypervisor boot normally
> Â Â Âagain.
> Â Â Â> > Ã ÂÃ So , might it be a problem of xen-4.0 that unable to handle
> Â Â Âunallocated
> Â Â Â> > Ã ÂÃ memory >32GB?

> Â Â Â> >
> Â Â Â>
> Â Â Â> Sounds like a bug yeah.. How does it fail in the dom0_mem=32G case?
> Â Â ÂWhat's the error?
> Â Â Â>
> Â Â Â> Can you please paste the full bootlog of the failing boot from a
> Â Â Âserial console?

> Â Â Â> [2]http://wiki.xensource.com/xenwiki/XenSerialConsole
> Â Â Â>
>

> Â Â ÂOh, one more thing, could you also try with Xen 4.0.1-rc6?
>
> Â Â ÂThanks!
> Â Â Â-- Pasi
>
> Â Â Â>
> Â Â Â>

> Â Â Â> > Ã ÂÃ Regards,
> Â Â Â> > Ã ÂÃ Ben
> Â Â Â> > Ã ÂÃ 2010/8/12 Pasi KÃârkkÃâinen <[1][3]pasik@xxxxxx>
> Â Â Â> >
> Â Â Â> > Ã ÂÃ ÂÃ On Thu, Aug 12, 2010 at 03:08:25PM +0800, benian wrote:
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ My environment is xen-4.0.0 + pvops2.6.32 kernel +
> Â Â Âfedora12
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ I can run xen hypervisor smoothly when system memory
> Â Â Âbelow 64G
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ But if I plug 4GB more memory to my server, XEN
> Â Â Âhypervisor start
> Â Â Â> > Ã ÂÃ ÂÃ fail then
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ crash and restart automatically.
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ The final screen is some message about memory address
> Â Â Âwhich will
> Â Â Â> > Ã ÂÃ ÂÃ not be
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ displayed in normal process
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ I read from XEN 4.0.0 features that Xen support 1 TB
> Â Â Âof RAM per
> Â Â Â> > Ã ÂÃ ÂÃ host, so I
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ am not sure what *s wrong with this situation.
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ > Ã ÂÃ Any idea is appreciated and welcome
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> > Ã ÂÃ ÂÃ >
> Â Â Â> >
> Â Â Â> > Ã ÂÃ ÂÃ How much memory does your dom0 have? ie. what's your
> Â Â Âdom0_mem= option
> Â Â Â> > Ã ÂÃ ÂÃ for xen.gz in grub.conf.
> Â Â Â> > Ã ÂÃ ÂÃ Have you tried different values? ie. 2GB, 4GB, 8GB?
> Â Â Â> >
> Â Â Â> > Ã ÂÃ ÂÃ Also setup a serial console so you can figure out if it's
> Â Â Âthe Xen
> Â Â Â> > Ã ÂÃ ÂÃ hypervisor or dom0 kernel crashing:
> Â Â Â> > Ã ÂÃ ÂÃ [2][4]http://wiki.xensource.com/xenwiki/XenSerialConsole
> Â Â Â> >
> Â Â Â> > Ã ÂÃ ÂÃ and more info about troubleshooting pvops dom0 kernels (also
> Â Â Âan example
> Â Â Â> > Ã ÂÃ ÂÃ about serial console with pvops dom0):
> Â Â Â> > Ã ÂÃ ÂÃ [3][5]http://wiki.xensource.com/xenwiki/XenParavirtOps
> Â Â Â> >
> Â Â Â> > Ã ÂÃ ÂÃ Those should help.
> Â Â Â> > Ã ÂÃ ÂÃ -- Pasi
> Â Â Â> >
> Â Â Â> > References
> Â Â Â> >
> Â Â Â> > Ã ÂÃ Visible links
> Â Â Â> > Ã ÂÃ 1. mailto:[6]pasik@xxxxxx
> Â Â Â> > Ã ÂÃ 2. [7]http://wiki.xensource.com/xenwiki/XenSerialConsole
> Â Â Â> > Ã ÂÃ 3. [8]http://wiki.xensource.com/xenwiki/XenParavirtOps
> Â Â Â>
> Â Â Â> _______________________________________________
> Â Â Â> Xen-devel mailing list
> Â Â Â> [9]Xen-devel@xxxxxxxxxxxxxxxxxxx
> Â Â Â> [10]http://lists.xensource.com/xen-devel

>
> References
>
> Â ÂVisible links
> Â Â1. mailto:pasik@xxxxxx
> Â Â2. http://wiki.xensource.com/xenwiki/XenSerialConsole

> Â Â3. mailto:pasik@xxxxxx
> Â Â4. http://wiki.xensource.com/xenwiki/XenSerialConsole
> Â Â5. http://wiki.xensource.com/xenwiki/XenParavirtOps
> Â Â6. mailto:pasik@xxxxxx
> Â Â7. http://wiki.xensource.com/xenwiki/XenSerialConsole
> Â Â8. http://wiki.xensource.com/xenwiki/XenParavirtOps
> Â Â9. mailto:Xen-devel@xxxxxxxxxxxxxxxxxxx
> Â 10. http://lists.xensource.com/xen-devel

Attachment: ERROR-4.0.1-rc6.txt
Description: Text document

Attachment: ERROR-32G.txt
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel