[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Show HVM guest stack

Abhinav Srivastava wrote:
> Hi Steve,
> Thanks for the reply. I will look into the crash utility to see how are they 
> doing.
> However, I have one question. I am doing stack walk whenever a HVM guest 
> makes VMExit due to a page fault. The way I am doing this: I am making some 
> kernel memory pages write protected from the hypervisor using shadow page 
> tables. And, whenever the kernel code writes to those pages, the guest 
> faults, VMExit happens, and control comes to the hypervisor's page fault 
> handler, where my code checks whether this is due my protection or not. If 
> yes, then I do the stack walk.

OK.  This may make your job simpler.  What type of kernel addresses are you 
write protecting?  Be careful when making any kernel pages read-only that are 
normally written in the page fault path (kernel stacks, some kernel data, 
per_cpu data).  These may cause nested exceptions that may confuse your stack 
tracer (as well as crash the guest if you don't properly hide them).  If you 
add faults to any exception entry/exit code, you will also see confusing stack 
trace conditions.

I guess the bottom line is, if you are only adding write faults to paths that 
can already write fault, any standard kernel stack trace should work (ala 
crash).  If you are adding new write fault points to the guest kernel (faults 
that would crash the guest if not intercepted), you may see stack conditions 
that a normal kernel stack tracer would not see.

> So, I am not doing stack walk from user-mode context. But, I am doing stack 
> walk whenever kernel page fault is happening? Will it make my case less 
> difficult or still I have to do all the things that you mentioned in your 
> email.

A user process can write to a kernel address (causing a protection fault).  
Normal user programs don't do this.  As long as you don't need to worry about 
malicious programs doing this, then you should be OK.


> Thanks for your help.
> -Abhinav
> --- On Tue, 7/10/08, Steve Ofsthun <sofsthun@xxxxxxxxxxxxxxx> wrote:
>> From: Steve Ofsthun <sofsthun@xxxxxxxxxxxxxxx>
>> Subject: Re: [Xen-devel] Show HVM guest stack
>> To: abhinavs_iitkgp@xxxxxxxxxxx
>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
>> Date: Tuesday, 7 October, 2008, 4:31 AM
>> Abhinav Srivastava wrote:
>>> Hi there,
>>> I am using Xen-3.2-testing (non-PAE and x86-32) on
>> Intel's Core 2
>>> Quad processor. My HVM domain is Fedora i386 with
>> Linux kernel 2.6.16 
>>> (with 8K of stack and CONFIG_FRAME_POINTER = y).
>>> In my project, I am trying to perform stack walk on
>> the HVM guest's
>>> kernel stack from the Xen hypervisor. In order to do
>> that, whenever
>>> a VMEXIT happens, I get the ebp value from
>> cpu_user_regs and try
>>> to use this address as a starting point for the walk.
>> I add 4 bytes
>>> in this address (for return address ) and try to get
>> the value at
>>> this location using Xen's function
>> hvm_copy_from_guest_virt(), which
>>> reads 4 bytes of return address. I, then, do this
>> process recursively
>>> by reading the value at the location pointed by the
>> ebp to get the 
>>> previous frame and so on.
>>> During the recursive process, sometimes when I try to
>> get the
>>> return address or next frame address,
>> hvm_copy_from_guest_virt() function
>>> crashes as it is not able to read that memory.
>> There are several reasons why you can run into trouble. 
>> The simplest reason is that the guest may be running in user
>> mode.  Since you can't predict that state of the stack
>> in user mode, you should first check for guest kernel mode
>> before you try to trace the stack.  The second issue is that
>> for hvm guests, VMEXITs may occur at arbitrary points in the
>> guest kernel execution.  This means that you may find the
>> guest kernel in the middle of handling an exception or
>> interrupt.  In these situations, the stack layout will be
>> different than you expect.  To avoid this, you may want to
>> ignore stacks when interrupts are disabled in the guest
>> kernel.  Note here that disabling interrupts in a guest
>> kernel doesn't prevent a VMEXIT.
>>> I saw a function named "show_guest_stack" in
>> xen/arch/x86/traps.c,
>>> which does the stack walk incase of PV domain. And,
>> returns if
>>> it finds HVM domain. Is there any function available
>> for doing
>>> stack walk in HVM domain?
>> You may want to look at the crash utility
>> (http://people.redhat.com/anderson/).  It knows how to dump
>> kernel stacks from live systems or dump files.  It is also
>> smart enough to properly recognize exception frames and
>> properly trace past them.
>>> If not, could you please let me know whether my stack
>> walk 
>>> procedure is correct or not? Or, do I need to do
>> something different 
>>> incase of an HVM domain? I tried using different
>> methods of putting 
>>> exit condition in my stack walk code such as
>> frame-pointer = 0, 
>>> return address = 0, Stack walk depth = 24, decreasing
>> frame pointer and
>>> ebp should be less than guest's current
>> thread_info + 8912.
>> Your starting and ending conditions are too simplistic to
>> work in the case of an arbitrary VMEXIT from an hvm domain. 
>> You need to handle every variation possible with a linux
>> kernel based stack tracer, and in addition, deal with
>> partially constructed/deconstructed exception frames.
>> Steve
>>> As of now, I am clueless as how to do this. Any help
>> would be highly appreciated.
>>> Thanks,
>>> Abhinav
>>>       Add more friends to your messenger and enjoy! Go
>> to http://messenger.yahoo.com/invite/
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-devel
>       Add more friends to your messenger and enjoy! Go to 
> http://messenger.yahoo.com/invite/

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.