Re: [Xen-devel] multi-core VMM

["G.R." <firemeteor@xxxxxxxxxxxxxxxxxxxxx>]

Hmm, I know that there is a 'SMT scheduler support' in the linux kernel config.
But I never tried to check out how it works.
Having application developer to control is not an effective way.
And I never heard of such API...
Maybe I just need to check out the linux solution...
I still tend to believe there are HW support for application behavior
statistics.

On Fri, Jan 11, 2013 at 12:14 AM, François-Frédéric Ozog <ff@xxxxxxxx> wrote:
> Before SandyBridge, it was not a good idea to have two mem bounded threads
> run on a single core: there was only ONE memory read port. On SandyBridge,
> IvvyBridge and Haswell, there are more memory ports and L3 cache is split
> between cores. Each core has a "CBo" (cache box) indirectly attached to
> 2.5MB of L3 cache. A consistent hash function across cores is calculated on
> cacheline address to obtain the target CBo. So there are no "locality"
> benefit from this chunking.
>
> I can't see a way for schedulers to decide the policy, that would be an
> application developer controlled thing.
> So I would rather leverage HT information to actually make sure some threads
> are NOT on the same core rather than trying to group them.
> This also means that VCPU to CPU mapping needs to be controlled tightly...
>
> François-Frédéric
>
> -----Message d'origine-----
> De : xen-devel-bounces@xxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxx] De la part de G.R.
> Envoyé : jeudi 10 janvier 2013 16:25
> À : ZHANG Zhi
> Cc : xen-devel@xxxxxxxxxxxxx
> Objet : Re: [Xen-devel] multi-core VMM
>
> I'm not quite sure. Exposing HT info to guest is only part of the story (but
> absolutely required)
>
> HT is a tech to share compute resources provided by a single physical core
> so as to achieve better utilization.
> OS should schedule two complementary instead of competing threads to one
> physical core to achieve better performance.
> Good example includes INT thread + FP thread, calc thread + mem bounded
> thread etc.
> If two threads contents for same resources in a physical core, the overall
> throughput for one physical core may even drop as compare to the case of
> only one thread.
> I'm not sure how OS scheduler obtain such info, but maybe there are
> instruction statistics for this purpose.
>
> So it appears to me that only when the OS have full control over the core,
> can it schedule best for HT.
> (but this really depends on what kind of statistics is available. The
> statement above is what I can guess best.)
>
> In virtualized env, the above assumption can not be easily achieved.
> People may be able to manipulate the CPU affinity through cpuset etc.
> On the other hand, without human invention, I'm not sure if the xen
> scheduler is doing anything special to handle HT.
>
> On Thu, Jan 10, 2013 at 9:00 PM, ZHANG Zhi <zhizhang@xxxxxxxxxx> wrote:
>>         Yeah, I think you're right.
>>         Maybe it also depends on the virtualization techniques. If it's
> hardware assisted virtualization, guest OS may detect the HT info, because
> it can also scan the BIOS memory, right?
>>
>>
>> re: Re: [Xen-devel] multi-core VMM
>>
>> Well, I can't agree.
>> HW provide features and SW use it. Otherwise, HW feature is just a vain.
>>
>> PS: please keep the list CCed so that people are aware of the discussion.
> Probably they have something to share on this topic.
>>
>> On Wed, Jan 9, 2013 at 1:37 PM, ZHANG Zhi <zhizhang@xxxxxxxxxx> wrote:
>>> I think it's the hardware's business, namely the Intel or AMD, not the
> hyperviosr's business. Xen does not have to consider such an issue.
>>>
>>> Re:
>>> I think that's possible, you can check the cpupool related command in the
> xl manpage.
>>>
>>> But I have a similar question -- In CPU like i7, 8 logical cores are not
> fully equivalent.
>>> They are hyper-threading over CMP. The two HT cores from a single
> physical core are contending for pipeline resource. In linux kernel, there
> is specific scheduler optimization for this case.
>>> My question is that how XEN hypervisor handle this? Will such HT info be
> available to the VM?
>>> I guess there may be some different depend on whether you bind VCPU to
> host core or not.
>>>
>>> Also, is there any different for dom0 && domU in this aspect?
>>>
>>> Thanks,
>>> Timothy
>>>
>>> On Wed, Dec 19, 2012 at 2:20 PM, ZHANG Zhi <zhizhang@xxxxxxxxxx> wrote:
>>>> Hi, list,
>>>>
>>>> A VMM provides a VMCS for each VM. How does the VMM assign system
>>>> resources for each VM? For example, in a multi-core environment, how
>>>> can I enable the VMM to run, say, on a two-core intel's processor
>>>> while I am able to force a VM to execute only on a specific core
>>>> upon initializing the Guest OS ? that is to say, how I can assure
>>>> the VM to believe that there is only one physical core running on this
> machine?
>>>>
>>>> What should I do? Is it possible to assign a core when VMM starts a
>>>> new VM after the VMM is booted ?
>>>>
>>>>    Thanks a lot!
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@xxxxxxxxxxxxx
>>>> http://lists.xen.org/xen-devel
>>>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel