Re: [Xen-devel] Branch Trace Storage for guests and VPMUinitialization

[<Kevin.Mayer@xxxxxxxx>]

> -----UrsprÃngliche Nachricht-----
> Von: Boris Ostrovsky [mailto:boris.ostrovsky@xxxxxxxxxx]
> Gesendet: Dienstag, 24. Februar 2015 18:13
> An: Mayer, Kevin; xen-devel@xxxxxxxxxxxxx
> Betreff: Re: [Xen-devel] Branch Trace Storage for guests and VPMU
> initialization
> 
> On 02/24/2015 10:27 AM, Kevin.Mayer@xxxxxxxx wrote:
> >
> > Hi guys
> >
> > I`m trying to set up the BTS so that I can log the branches taken in
> > the guest using Xen 4.4.1 with a WinXP SP3 guest on a Core i7 Sandy
> > Bridge.
> >
> > I added the vpmu=bts boot parameter to my grub2 configuration and
> > extended the libxl,libxc,domctl,â with an own command so that I can
> > trigger the activation of the BTS whenever I want.
> >
> 
> 
> I am not sure why you are doing all these changes to Xen code. BTS is
> supposed to be managed from the guest. For example, a Fedora HVM guest
> will produce this:
> 
> [root@dhcp-burlington7-2nd-B-east-10-152-55-140 ~]# perf record -e
> branches:u -c 1 -d sleep 1 [ perf record: Woken up 3838 times to write data ] 
> [
> perf record: Captured and wrote 0.704 MB perf.data (~30756 samples) ]
> [root@dhcp-burlington7-2nd-B-east-10-152-55-140 ~]# perf script -f
> ip,addr,sym,dso,symoff --show-kernel-path
>   ffffffff8167c347 native_irq_return_iret+0x0 (/proc/kcore) =>
> 328c001590 [unknown] (/proc/kcore)
>   ffffffff8167c347 native_irq_return_iret+0x0 (/proc/kcore) =>
> 328c001590 [unknown] ([unknown])
>         328c001593 [unknown] ([unknown]) =>       328c004b70 [unknown]
> ([unknown])
> ...
> 

I want to be able to log the taken branches (of the guest) without the need to 
modify the guest at all.
This means I have to do all the logic in the hypervisor, or am I wrong?

> > In this command I do the following:
> >
> > I set up the memory region for the BTS Buffer and the DS Buffer
> > Management Area using xzalloc_bytes
> >
> 
> 
> I don't think you should be allocating BTS buffers in the hypervisor, they are
> in guest's memory.

I agree. As I said I think this is where my main problem is at the moment. 
Is there any way I can allocate memory in the hypervisor in a way the guest can 
access it?
Of course the guest must not be able to use this memory in its normal 
operations but just for BTS.
Is this even possible? I am rather confused at the moment. :-D

> > Then I write the pointer to the BTS Buffer into the DS Buffer
> > Management Area at +0x0 and +0x8 (BTS Buffer Base and BTS Index)
> >
> > When I use vmx_msr_write_intercept to store the value in
> > MSR_IA32_DS_AREA the host reboots (my idea is he tries to access a
> > vpmu-struct that isnÂt there in the current vcpu and panics).
> >
> 
> Can you post hypervisor log? (hard to say how helpful it will be without
> seeing your code changes though)
> 

Right after enabling the BTS I get a triple fault.
hvm.c:1357:d2 Triple fault on VCPU0 - invoking HVM shutdown action 1. 

> > When I use a modified version of vmx_msr_write_intercept I donât get
> > any crashes as long as I donât enable BTS and TR in the
> > GUEST_IA32_DEBUGCTL (BTR works). When I enable the BTS (and TR) the
> > guest crashes. I suppose he gets killed by the hypervisor for
> > accessing forbidden memory.
> >
> 
> Possibly because DS area point to hypervisor memory.
> 
> 
> Having said all this, I am not sure how well BTS works. You did notice
> this in the hypervisor log:
> 
> (XEN) ******************************************************
> (XEN) ** WARNING: Emulation of BTS Feature is switched on **
> (XEN) ** Using this processor feature in a virtualized **
> (XEN) ** environment is not 100% safe. **
> (XEN) ** Setting the DS buffer address with wrong values **
> (XEN) ** may lead to hypervisor hangs or crashes. **
> (XEN) ** It is NOT recommended for production use! **
> (XEN) ******************************************************
> 

Yes, I saw that. It doesnât state that BTS is not working at all, just that it 
is not that safe to use.
As I understand it as long as I set the DS buffer address correctly I should be 
fine, right?
Since I donât want to use for production that is fine with me. At least for now.


Kevin
> 
> -boris
> 
> 
> > The modified version of vmx_msr_write_intercept takes a vcpu-struct as
> > a parameter and uses this instead of the current vcpu.
> >
> > Instead of
> >
> > staticint vmx_msr_write_intercept(unsigned int msr, uint64_t
> msr_content)
> >
> > {
> >
> >     struct vcpu *v = current;
> >
> > I just have
> >
> > staticint own_vmx_msr_write_intercept(unsigned int msr, uint64_t
> > msr_content, struct vcpu *v)
> >
> > I get this vcpu by d->vcpu[0] as I have limited my guest domain to one
> > vcpu atm.
> >
> > Of course I also use similarly modified version of the called
> > functions(vpmu_do_wrmsr,â).
> >
> > IÂm pretty sure that my problem is with a wrong scope/usage of the
> > vcpus/memory, but I have no idea how to fix this.
> >
> > I can see a potential problem with the memory allocation (in the host)
> > into which the cpu in guest-mode is supposed to write.
> >
> > Or maybe I got the principle of a vcpu/vpmu all wrong.
> >
> > Since I couldnât find any project that uses the BTS for the guest, I
> > am wondering if anyone has ever done this and if it is possible at all.
> >
> > Any input is welcome as I am pretty much stuck atmâ
> >
> > Cheers
> >
> > Kevin
> >
> >
> > ____________
> > Virus checked by G Data MailSecurity
> > Version: AVA 25.404 dated 24.02.2015
> > Virus news: www.antiviruslab.com <http://www.antiviruslab.com>
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxx
> > http://lists.xen.org/xen-devel

____________
Virus checked by G Data MailSecurity
Version: AVA 25.418 dated 25.02.2015
Virus news: www.antiviruslab.com
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel