[Xen-devel] Proposal for Xen support of performance monitoring and debug hardware

[William Cohen <wcohen@xxxxxxxxxx>]

I have been working on a proposal to add Xen support for performance 
monitoring and debugging hardware. The goal of this would be enable 
OProfile, perfmon, and perfctr to work on Xen. The proposal is still 
pretty preliminary, but I would like comments on the current version.
-Will

                        A Proposal for Xen
          to Support Processor Performance and Debugging Hardware

                               version 0.1
                               April 22, 2005

                               Will Cohen


This is a proposed interface for Xen support of processor performance
and debugging hardware. Xen virtualizes the processor and allow
multiple host OS instances (domains) to run concurrently on a single
machine. However, the approach that Xen uses prevents all virtual
domains with the exception of the Domain-0 from accessing devices that
require ring 0 access on the x86 and x86-64 processors.

This proposal suggests an interface to allow use of the processors
performance monitoring and debugging hardware.  The interface would
allow the VMM to track which machine specific registers (MSRs) are in
use by particular domain, allow safe access to the registers, and
vector the interrupts to the appropriate handlers in the guest
domains.


The MSRs will be grouped together into classes described by the enum below:

enum xen_msr_type{
     INVALID_MSR=0,
     PERF_MSR,
     DEBUG_MSR
};

The classes in the enum will be used to determine whether the
particular register access is valid. A lookup table in the vmm or
domain-0 OS can be used to check validity.  The xen_msr_access below
is used to determine whether the registers are accessible to a domain
or whether they are used globally.

enum xen_msr_access {
     UNAVAILABLE=0,
     LOCAL,
     GLOBAL
};

The following data structures are used in the interface:

struct xen_msr_value {
        long msr;
        long value; /* needs to be a 64 bit value */
}

struct xen_msr_entries {
       int entries;
       struct xen_msr_value data[0]; /* number of entries */
}

The struct xen_msr_value hold the address of the msr and the value for
that msr. The struct xen_msr_entries is used to bundle msr operations
together to minimize the number of calls to set up and read out the
msr.

Controlling access to the register sets is provided by the following function:

extern int xen_msr_allocate(enum xen_msr_type class, domid_t domain,
       enum xen_msr_access scope);

The xen_msr_allocate function is used by Domain 0 to control access to
the machine specific registers used for debug and performance
monitoring. The first argument indicates the type of msr registers
being allocated. The second argument indicates which domain this
affects. The last argument, scope, indicates whether the msrs are
local (and virtualized) or globally control by a domain. The function
returns 0 if the operations was successful. It should be possible to
use this function to set domain 0 to globally control the performance
monitoring hardware and do system-wide sampling. When a domain has
global control of the registers no other domains can have control of
the registers.


Setting the interrupt handler is implemented with the following function:

extern int xen_msr_irq_handler(enum xen_msr_type class,
       int (*handler_function)(struct pt_regs * regs, int cpu));

The msr_irq_handler sets up the interrupt handler for a particular
class event, e.g. PERF_MSR or DEBUG_MSR, and the handler_function. If
the handler_function is NULL this indicates there is no
handler. Unlike OProfile which uses non-maskable interrupts (NMIs),
this will be restricted to Interrupt Requests (IRQs). The
handler_function returns 1 if it was able to handle the interrupt and
0 if it was unable to. The xen_msr_irq_handler function returns 0 for
success and -EACCESS for failure.

***FIXME*** How to handle performance monitoring interrupts from other domains?

There are three functions to handle the operation of the msr registers:

extern int xen_msr_setdefault(enum xen_msr_type class, struct xen_msr_entries 
*values);

extern int xen_msr_write(enum xen_msr_type class, int rearm_irq, struct 
xen_msr_entries *values);

extern int xen_msr_read(enum msr_type class, struct xen_msr_entries *values);

Each of the functions checks to make sure that each entry in the
xen_msr_entries is valid for the class. Thus, a user cannot use this
to access msr registers outside the set. Each of the functions also
returns the number of entries in values that it was able to operate
on. If there were no errors the function's return value is equal to
values->entries. If there were errors, the return value would indicate
which element what the problem.

The xen_msr_setdefault function is mainly to provide an easy way for
the VMM to know that certain msrs are no longer in uses and should be
set to the default values. The function goes through
values->data[].msr and setsthem to the default value.  The value part
of the struct xen_msr_value are set to the default value to reflect
the current state of the registers. Thus, the VMM should be able to
track which msrs need to be preserved for a particular domain.

The xen_msr_write function is used to set the msrs to the values
described by the variable values. The rearm_irq variable is needed to
signal that the interrupt mechanism needs to be rearmed. This is
required for the Pentium 4 and Pentium M processors.

The xen_msr_read function fills in the value field of the
xen_msr_value structs of values. based on the xen_msr_value struct's
msr values.

It should be possible to implement these interfaces so that the same
interface is used for both the kernel that touches the physical MSR
registers and the guest kernels that need to make calls through the
VMM. Thus, minimizing the differences between Xen and non-Xen kernels.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel